The gradients will be summed automatically when the same variable is
used more than once.
>>> x = tensor.vector()
>>> c1 = x.sum()
>>> c2 = (2 * x).sum()
>>> c = c1 + c2
>>> gx = theano.grad(c, x)
>>> gx.eval({x: [0, 1, 2]})
array([ 3., 3., 3.])
If you wish to prevent that, then you can use .copy() to create the
"identity" operation in the graph, so that you can operate on different
variables in different parts of the graph. For instance:
>>> x = tensor.vector()
>>> x1 = x.copy()
>>> c1 = x1.sum()
>>> x2 = x.copy()
>>> c2 = (2 * x2).sum()
>>> c = c1 + c2
>>> gx, gx1, gx2 = theano.grad(c, [x, x1, x2])
>>> gx.eval({x: [0, 1, 2]})
array([ 3., 3., 3.])
>>> gx1.eval({x: [0, 1, 2]})
array([ 1., 1., 1.])
>>> gx2.eval({x: [0, 1, 2]})
array([ 2., 2., 2.])
On Fri, Oct 07, 2016, John Moore wrote:
> Hi All,
>
> In one of my neural net models, I have shared weights at several different
> parts of the network. For example, let's say weights are completely shared
> at layer 1 and layer 3.
> Will the gradient update from theano.grad sum the gradients of the shared
> weights? Or will it simply take the gradient update from layer 1?
>
> Any insight appreciated.
>
> Thanks,
> John
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
--
Pascal
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
