Is there such thing as a multivariate chain rule?

Question

I learned in Calculus that if y=f(x), then

d/dx g(y) = g'(f(x))f'(x)

but what if y is a vector? (x and g(x) are both scalars.) If g(y) is a function of an array y, and each element of y is a (possibly different) function of x, then how do I express the derivative of g(f(x)) with respect to x? Is there a generalized version of the chain rule for vectors?

I'm trying to understand neural nets and this keeps coming up.

Answer 1

Yes, there is a general version of the chain rule. However, to really understand it, a more complicated definition of the derivative is needed as well as a bunch of terminology, so I will try to stick to the basics. The general chain rule is Dg(f(x)) = Dg(f(x)) # Df(x) where # means "composed with" and if h is a function Dh is its frechet derivative.

In your case x and g(y) are scalar valued and y is a vector and f(x) is a vector. We can write f(x) = (f1(x),f2(x),...) where fi is a scalar function of x. From here, f'(x) = (f1'(x), f2'(x), f3'(x),...). Additionally, we can write g(y) = g(y1,y2,...). From here g'(y) = (dg(y)/dy1, dg(y)/dy2,...) where dg(y)/dyi is the partial derivative of g with repect to yi. This can be calculated by finding the derivative of g if yi is the only variable changing, that is, holding yj constant for j not equal to i - so this is the same as a scalar derivative.

Finally, we get to the answer to your question, which is that
d/dx g(f(x)) = g'(f(x)) dot f'(x)
= dg(f(x))/dy1 * f1'(x) + dg(f(x))/dy2 * f2'(x) + ...
where dot means the dot product and * is scalar multiplication .

Answer 2

It depends: if the "x" you are using is a parameter like angle whose change would mix the components of your vector, then you have to account for the mixing of components and the possible functional changes in separate steps.

Answer 3

See if the two of the two hyperlinks under enable you to out. in case you nonetheless get caught e mail me and that i will try, yet this format is a soreness interior the butt devoid of being waiting to of course write fractions and trademarks.

Answer 4

Yep!
Damn Right!