I'm new to theano, and my research group is using it as the backend for
Keras. We are using some Sparse matrices for our weights, and I wanted to
use RMSprop as our optimizer, so I had to write my own to work with these
Sparse matrices. However, I am running into errors that I don't understand.
For example, here is the end of the Traceback:
File "/home/aalavi/single_cell_reducer/sparse_optimizers.py", line 83, in
get_updates
new_p = p - lr * g / (sparse.sqrt(new_a) + self.epsilon)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
, line 225, in __add__
return add(left, right)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
, line 2174, in add
return add_s_d(x, y)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/gof/op.py"
, line 615, in __call__
node = self.make_node(*inputs, **kwargs)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
, line 2039, in make_node
assert y.type.ndim == 2
AssertionError
To put into context, here is the part of the built-in RMSprop optimizer
from Keras, which I am trying to get to work with Sparse:
for p, g, a in zip(params, grads, accumulators):
# update accumulator
new_a = self.rho * a + (1. - self.rho) * K.square(g)
self.updates.append(K.update(a, new_a))
new_p = p - lr * g / (K.sqrt(new_a) + self.epsilon)
# apply constraints
if p in constraints:
c = constraints[p]
new_p = c(new_p)
self.updates.append(K.update(p, new_p))
return self.updates
I originally had an error with the line:
new_a = self.rho * a + (1. - self.rho) * K.square(g)
and the error was:
File "/home/aalavi/single_cell_reducer/sparse_optimizers.py", line 73, in
get_updates
new_a = self.rho * a + (1. - self.rho) * K.square(g)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/keras/backend/theano_backend.py"
, line 472, in square
return T.sqr(x)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/gof/op.py"
, line 615, in __call__
node = self.make_node(*inputs, **kwargs)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/tensor/elemwise.py"
, line 576, in make_node
inputs = list(map(as_tensor_variable, inputs))
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/tensor/basic.py"
, line 171, in as_tensor_variable
"Variable type field must be a TensorType.", x, x.type)
theano.tensor.var.AsTensorError: ('Variable type field must be a
TensorType.', SparseVariable{csr,float32}, Sparse[float32, csr])
I fixed this by using theano.sparse.sqr(g) in the calculation for new_a,
but now I can't get paste the error in calculating new_p, even after trying
theano.sparse.sqrt(new_a) as above.
I'd appreciate any help on this
Is this similar to below?
Discussion about comparing sparse to
scalar: https://groups.google.com/d/topic/theano-users/sbKdzoWOCDI/discussion
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
