Re: [theano-users] Theano variable sent to for slicing instead of constant while using theano.grad

[Pascal Lamblin]

OK, thanks.

self.truncate_gradient should not be a Python float, it should be an 
integer.
This is probably why the dtype of grad_steps is float64, instead of 
int64 (or another integer dtype).


Do you have any idea why self.truncate_gradient would not be "-1" (the 
default value)? Did you set it?


On 2018-03-06 02:07 PM, Siddhartha Saxena wrote:
The dtype of  grad_steps and s_ is float64 while self.truncate_gradient 
is a python float.

Sorry I didn't answer it properly previously.

Thanks

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano variable sent to for slicing instead of constant while using theano.grad

[Siddhartha Saxena]

Thanks a lot Pascal, I have solved the problem now, the issue was that  
 self.truncate_gradient was a float instead of being an int.


-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano variable sent to for slicing instead of constant while using theano.grad

[Siddhartha Saxena]

The dtype of  grad_steps and s_ is float64 while self.truncate_gradient is 
a python float.
Sorry I didn't answer it properly previously.

Thanks 

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano variable sent to for slicing instead of constant while using theano.grad

[Pascal Lamblin]

OK, but what is the `dtype` (data type) of those variables?

On 2018-03-06 01:48 PM, Siddhartha Saxena wrote:
grad_steps itself is of  with 
value "Elemwise{minimum,no_inplace}.0". So here a tensor that is s_ ( of 
type Subtensor{::int64}.0}) is being sliced by a variable. Again how it 
is reaching there is what i am unable to understand.


Thanks

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano variable sent to for slicing instead of constant while using theano.grad

[Siddhartha Saxena]

grad_steps itself is of  with 
value "Elemwise{minimum,no_inplace}.0". So here a tensor that is s_ ( of 
type Subtensor{::int64}.0}) is being sliced by a variable. Again how it is 
reaching there is what i am unable to understand. 

Thanks

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano variable sent to for slicing instead of constant while using theano.grad

[Pascal Lamblin]

Can you open that trace in pdb, and let us know what the dtype of 
`grad_steps` here is?

What about the dtype of `self.truncate_gradient`?
And the type of `s_`?

On 2018-03-03 03:12 AM, Siddhartha Saxena wrote:

Hi

I am training a custom LSTM model on theano with LSTM layers as in 
(https://github.com/asheshjain399/NeuralModels/tree/master/neuralmodels/layers/LSTM.py) 
and 
(https://github.com/asheshjain399/NeuralModels/blob/master/neuralmodels/layers/multilayerLSTM.py). 
Now the model that I have created makes a valid theano graph but at the 
step the find the updates (RMSprop) of the model with a cost (mean 
square) function via the theano.grad function, I keep getting the 
following error. The source of the error is that a theano op 
(Elemwise{minimum,no_inplace}.0) is being sent for slicing to the 
subtensor.py file instead of a theano constant. The issue is that I am 
not sure how it ends up there. The model works if I remove the LSTM 
layer, but this LSTM layer works with some other model perfectly.


I'll be extremely thankful if someone can help me through this error 
which is bugging me for days now.


Best
Siddhartha

-
   File "trainGCNN_NoGraph.py", line 251, in trainGCNN
     gcnn = 
GCNNmodelRegression(preGraphNets,nodeList,nodeFeatureLength,temporalNodeFeatureLength,new_idx,featureRange)

   File "trainGCNN_NoGraph.py", line 202, in GCNNmodelRegression
     gcnn = 
GCNN(graphLayers,finalLayer,preGraphNets,nodeNames,temporalNodeRNN,nodeRNNs,topLayer,euclidean_loss,nodeLabels,learning_rate,adjacency,new_idx,featureRange,clipnorm=args.clipnorm,update_type=gradient_method,weight_decay=args.weight_decay)
   File "/home/siddhartha/Graph_CNNs/neuralmodels/models/GCNN.py", line 
169, in __init__
     [self.updates,self.grads] = 
self.update_type.get_updates(self.params_all,self.cost)
   File "/home/siddhartha/Graph_CNNs/neuralmodels/updates.py", line 108, 
in get_updates

     grads_unclipped = T.grad(cost, params)
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 555, in grad

     grad_dict, wrt, cost_name)
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 1317, in _populate_grad_dict

     rval = [access_grad_cache(elem) for elem in wrt]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 1272, in access_grad_cache

     term = access_term_cache(node)[idx]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 967, in access_term_cache

     output_grads = [access_grad_cache(var) for var in node.outputs]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 1272, in access_grad_cache

     term = access_term_cache(node)[idx]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 967, in access_term_cache

     output_grads = [access_grad_cache(var) for var in node.outputs]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 1272, in access_grad_cache

     term = access_term_cache(node)[idx]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 967, in access_term_cache

     output_grads = [access_grad_cache(var) for var in node.outputs]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 1272, in access_grad_cache

     term = access_term_cache(node)[idx]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 967, in access_term_cache

     output_grads = [access_grad_cache(var) for var in node.outputs]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 1272, in access_grad_cache

     term = access_term_cache(node)[idx]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 967, in access_term_cache

     output_grads = [access_grad_cache(var) for var in node.outputs]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 1272, in access_grad_cache

     term = access_term_cache(node)[idx]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 967, in access_term_cache

     output_grads = [access_grad_cache(var) for var in node.outputs]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 1272, in access_grad_cache

     term = access_term_cache(node)[idx]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 967, in access_term_cache

     output_grads = [access_grad_cache(var) for var in node.outputs]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 1272, in access_grad_cache

     term = access_term_cache(node)[idx]
   File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", 
line 1108, in access_term_cache

     new_output_grads)
   File 
"/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan_op.py", 
line 2250, in L_op

     outer_inp_seqs = [s_[:grad_steps] for s_ in outer_inp_seqs]
   File 

[theano-users] Theano variable sent to for slicing instead of constant while using theano.grad

[Siddhartha Saxena]

Hi 

I am training a custom LSTM model on theano with LSTM layers as in 
(https://github.com/asheshjain399/NeuralModels/tree/master/neuralmodels/layers/LSTM.py)
 
and 
(https://github.com/asheshjain399/NeuralModels/blob/master/neuralmodels/layers/multilayerLSTM.py).
 
Now the model that I have created makes a valid theano graph but at the 
step the find the updates (RMSprop) of the model with a cost (mean square) 
function via the theano.grad function, I keep getting the following error. 
The source of the error is that a theano op 
(Elemwise{minimum,no_inplace}.0) is being sent for slicing to the 
subtensor.py file instead of a theano constant. The issue is that I am not 
sure how it ends up there. The model works if I remove the LSTM layer, but 
this LSTM layer works with some other model perfectly.

I'll be extremely thankful if someone can help me through this error which 
is bugging me for days now.

Best 
Siddhartha  

-
  File "trainGCNN_NoGraph.py", line 251, in trainGCNN
gcnn = 
GCNNmodelRegression(preGraphNets,nodeList,nodeFeatureLength,temporalNodeFeatureLength,new_idx,featureRange)
  File "trainGCNN_NoGraph.py", line 202, in GCNNmodelRegression
gcnn = 
GCNN(graphLayers,finalLayer,preGraphNets,nodeNames,temporalNodeRNN,nodeRNNs,topLayer,euclidean_loss,nodeLabels,learning_rate,adjacency,new_idx,featureRange,clipnorm=args.clipnorm,update_type=gradient_method,weight_decay=args.weight_decay)
  File "/home/siddhartha/Graph_CNNs/neuralmodels/models/GCNN.py", line 169, 
in __init__
[self.updates,self.grads] = 
self.update_type.get_updates(self.params_all,self.cost)
  File "/home/siddhartha/Graph_CNNs/neuralmodels/updates.py", line 108, in 
get_updates
grads_unclipped = T.grad(cost, params)
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
555, in grad
grad_dict, wrt, cost_name)
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
1317, in _populate_grad_dict
rval = [access_grad_cache(elem) for elem in wrt]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
1272, in access_grad_cache
term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
1272, in access_grad_cache
term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
1272, in access_grad_cache
term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
1272, in access_grad_cache
term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
1272, in access_grad_cache
term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
1272, in access_grad_cache
term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
1272, in access_grad_cache
term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
1272, in access_grad_cache
term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line 
1108, in access_term_cache
new_output_grads)
  File 
"/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan_op.py", 
line 2250, in L_op
outer_inp_seqs = [s_[:grad_steps] for s_ in outer_inp_seqs]
  File "/usr/local/lib/python2.7/dist-packages/theano/tensor/var.py", line 
519, in __getitem__
theano.tensor.subtensor.Subtensor.convert(arg)
  File "/usr/local/lib/python2.7/dist-packages/theano/tensor/subtensor.py", 
line 378, in convert
slice_b = Subtensor.convert(b, False)
  File