Posts

矩阵求导

MATH

矩阵求导的基本法则

标量对标量

...

标量对向量

ff

xp×1=(x1,x2,…,xp)Tx_{p\times1}=(x_1,x_2,\dots,x_p)^T

∂f∂xp×1=(∂f∂x1,∂f∂x2,…,∂f∂xp)T\frac{\partial f}{\partial x}_{p\times1}=(\frac{\partial f}{\partial x_1},\frac{\partial f}{\partial x_2},\dots,\frac{\partial f}{\partial x_p})^T

向量对标量

fm×1=(f1,f2,…,fm)Tf_{m\times1}=(f_1,f_2,\dots,f_m)^T

xx

∂f∂x1×m=(∂f1∂x,∂f2∂x,…,∂fm∂x)\frac{\partial f}{\partial x}_{1\times m}=(\frac{\partial f_1}{\partial x},\frac{\partial f_2}{\partial x},\dots,\frac{\partial f_m}{\partial x})

向量对向量

fm×1=(f1,f2,…,fm)Tf_{m\times1}=(f_1,f_2,\dots,f_m)^T

xp×1=(x1,x2,…,xp)Tx_{p\times1}=(x_1,x_2,\dots,x_p)^T

∂f∂xp×m=[∂f1∂x1∂f2∂x1⋯∂fm∂x1∂f1∂x2∂f2∂x2⋯∂fm∂x1⋮⋮⋱⋮∂f1∂xp∂f2∂xp⋯∂fm∂xp]\frac{\partial f}{\partial x}_{p\times m}= \begin{bmatrix} \frac{\partial f_1}{\partial x_1} & \frac{\partial f_2}{\partial x_1} & \cdots & \frac{\partial f_m}{\partial x_1} \\ \frac{\partial f_1}{\partial x_2} & \frac{\partial f_2}{\partial x_2} & \cdots & \frac{\partial f_m}{\partial x_1} \\ \vdots & \vdots & \ddots & \vdots\\ \frac{\partial f_1}{\partial x_p} & \frac{\partial f_2}{\partial x_p} & \cdots & \frac{\partial f_m}{\partial x_p}\\ \end{bmatrix}

标量对矩阵

ff

xp×q=[x11x12⋯x1qx21x22⋯x2q⋮⋮⋱⋮xp1xp2⋯xpq]x_{p\times q}=\begin{bmatrix}x_{11} & x_{12} & \cdots & x_{1q}\\x_{21} & x_{22} & \cdots & x_{2q}\\\vdots & \vdots &\ddots & \vdots\\x_{p1} & x_{p2} & \cdots & x_{pq}\end{bmatrix}

∂f∂xp×q=[∂f∂x11∂f∂x12⋯∂f∂x1q∂f∂x21∂f∂x22⋯∂f∂x2q⋮⋮⋱⋮∂f∂xp1∂f∂xp2⋯∂f∂xpq]\frac{\partial f}{\partial x}_{p\times q}=\begin{bmatrix} \frac{\partial f}{\partial x_{11}} & \frac{\partial f}{\partial x_{12}} & \cdots & \frac{\partial f}{\partial x_{1q}}\\ \frac{\partial f}{\partial x_{21}} & \frac{\partial f}{\partial x_{22}} & \cdots & \frac{\partial f}{\partial x_{2q}}\\ \vdots & \vdots &\ddots & \vdots\\ \frac{\partial f}{\partial x_{p1}} & \frac{\partial f}{\partial x_{p2}} & \cdots & \frac{\partial f}{\partial x_{pq}}\\ \end{bmatrix}

矩阵对标量

fm×n=[f11f12⋯f1nf21f22⋯f2n⋮⋮⋱⋮fm1fm2⋯fmn]f_{m\times n}=\begin{bmatrix}f_{11} & f_{12} & \cdots & f_{1n} \\ f_{21} & f_{22} & \cdots & f_{2n}\\\vdots&\vdots&\ddots&\vdots\\f_{m1}& f_{m2} &\cdots & f_{mn}\end{bmatrix}

xx

∂f∂xn×m=[∂f11∂x∂f21∂x⋯∂fm1∂x∂f12∂x∂f22∂x⋯∂fm2∂x⋮⋮⋱⋮∂f1n∂x∂f2n∂x⋯∂fmn∂x]\frac{\partial f}{\partial x}_{n\times m}=\begin{bmatrix} \frac{\partial f_{11}}{\partial x} & \frac{\partial f_{21}}{\partial x} & \cdots & \frac{\partial f_{m1}}{\partial x}\\ \frac{\partial f_{12}}{\partial x} & \frac{\partial f_{22}}{\partial x} & \cdots & \frac{\partial f_{m2}}{\partial x}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial f_{1n}}{\partial x} & \frac{\partial f_{2n}}{\partial x} & \cdots & \frac{\partial f_{mn}}{\partial x}\\ \end{bmatrix}

更高维度的

...

Reference