The problem is probably the + in
sparse.sqrt(new_a) + self.epsilon
Here you add a scalar to a sparse variable. WHich will make it non sparse!
We force the user to make explicit conversion from sparse to dense to
prevent unexpected memory grow. You can manually force the conversion of
the sparse variable like this:
sparse.dense_from_sparse(sparse.sqrt(new_a)) + self.epsilon
On Thu, Sep 14, 2017 at 9:35 PM Amir Alavi <s.amir.al...@gmail.com> wrote:
> I'm new to theano, and my research group is using it as the backend for
> Keras. We are using some Sparse matrices for our weights, and I wanted to
> use RMSprop as our optimizer, so I had to write my own to work with these
> Sparse matrices. However, I am running into errors that I don't understand.
> For example, here is the end of the Traceback:
>
> File "/home/aalavi/single_cell_reducer/sparse_optimizers.py", line 83,
> in get_updates
> new_p = p - lr * g / (sparse.sqrt(new_a) + self.epsilon)
> File
> "/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
> , line 225, in __add__
> return add(left, right)
> File
> "/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
> , line 2174, in add
> return add_s_d(x, y)
> File
> "/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/gof/op.py"
> , line 615, in __call__
> node = self.make_node(*inputs, **kwargs)
> File
> "/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
> , line 2039, in make_node
> assert y.type.ndim == 2
> AssertionError
>
>
> To put into context, here is the part of the built-in RMSprop optimizer
> from Keras, which I am trying to get to work with Sparse:
>
> for p, g, a in zip(params, grads, accumulators):
> # update accumulator
> new_a = self.rho * a + (1. - self.rho) * K.square(g)
> self.updates.append(K.update(a, new_a))
> new_p = p - lr * g / (K.sqrt(new_a) + self.epsilon)
>
>
> # apply constraints
> if p in constraints:
> c = constraints[p]
> new_p = c(new_p)
> self.updates.append(K.update(p, new_p))
> return self.updates
>
> I originally had an error with the line:
> new_a = self.rho * a + (1. - self.rho) * K.square(g)
>
> and the error was:
> File "/home/aalavi/single_cell_reducer/sparse_optimizers.py", line 73,
> in get_updates
> new_a = self.rho * a + (1. - self.rho) * K.square(g)
> File
> "/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/keras/backend/theano_backend.py"
> , line 472, in square
> return T.sqr(x)
> File
> "/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/gof/op.py"
> , line 615, in __call__
> node = self.make_node(*inputs, **kwargs)
> File
> "/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/tensor/elemwise.py"
> , line 576, in make_node
> inputs = list(map(as_tensor_variable, inputs))
> File
> "/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/tensor/basic.py"
> , line 171, in as_tensor_variable
> "Variable type field must be a TensorType.", x, x.type)
> theano.tensor.var.AsTensorError: ('Variable type field must be a
> TensorType.', SparseVariable{csr,float32}, Sparse[float32, csr])
> I fixed this by using theano.sparse.sqr(g) in the calculation for new_a,
> but now I can't get paste the error in calculating new_p, even after trying
> theano.sparse.sqrt(new_a) as above.
>
> I'd appreciate any help on this
>
> Is this similar to below?
> Discussion about comparing sparse to scalar:
> https://groups.google.com/d/topic/theano-users/sbKdzoWOCDI/discussion
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
