Maybe the following can help you.
http://deeplearning.net/software/theano/tutorial/faq_tutorial.html#how-to-update-a-subset-of-weights
Also, if you take a subtensor of the gradient itself, some optimizations
can apply that would avoid the computation of the full gradient.
For instance, with your example, the "subtensor" and "* 2" operations
are swapped:
>>> grad0 = full_grad[0]
>>> g0 = theano.function([X, Y], grad0)
>>> theano.printing.debugprint(g0)
Elemwise{mul,no_inplace} [id A] '' 1
|TensorConstant{(1,) of 2.0} [id B]
|Subtensor{int64} [id C] '' 0
|<TensorType(float64, matrix)> [id D]
|Constant{0} [id E]
On 2017-09-27 05:25 PM, Daniel Hernandez wrote:
Hi,
I was wondering if someone here had an answer to this unsolved question
over in stack overflow:
https://stackoverflow.com/questions/37545325/theano-gradient-of-subtensor
Basically, how do you compute gradients w.r.t. a subtensor?
The question arises in the context of large tensors, say Y and X, where
it is known that each entry in Y depends only on a small subset of the
entries of X. Taking T.grad(Y, X) is computationally expensive since it
will compute every possible gradient so one would like to be able to
compute, e.g. T.grad(Y, X[i]) . Here is some basic code illustrating the
problem.
X = T.matrix()
Y = T.sum(X**2)
full_grad = T.grad(Y, X) # This works
X0 = X[0]
test = T.grad(Y, X0) # This pukes a Disconnected Input error
Silencing the Disconnected Input can be done in grad, but of course,
that doesn't solve anything, evaluating the gradients only results in a
bunch of 0s. So, is there a way of taking these gradients with respect
to a subtensor?
--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
For more options, visit https://groups.google.com/d/optout.
--
Pascal Lamblin
--
---
You received this message because you are subscribed to the Google Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
