On Tue, Sep 06, 2016, Kasirat Kasfi wrote:
> Hi I am trying to build a simple autoencoder model in Theano but I also
> want to optimize using Scipy Optimize Minimize. I found a code here
> :
> http://dlacombejr.github.io/programming/2015/09/13/sparse-filtering-implemenation-in-theano.html
>
> which I modified a bit. The problem is since scipy minimum takes in an
> array, but I need to optimize more that one weight parameters. what is the
> solution to this problem if I want to keep using the same structure as the
> code in the link?
>
> I found a possible way of getting around this problem, that is to pass in
> all the weights as a single array flattened and appended, and then
> assigning parts of the array to various variables, according to this
> https://github.com/Theano/Theano/issues/3131
Yes, that would be the way.
>
> import theano
> from theano import tensor as t
> import numpy as np
>
> class SparseFilter(object):
>
> def __init__(self, theta, dims, x):
> # assign inputs to sparse filter
> self.theta = theta
> self.x = x
> self.dims = dims
>
> self.w = theano.shared(np.random.randn(self.dims[1], self.dims[0]),
> name='w')
> self.w = t.reshape(self.theta[0:(self.dims[0]*self.dims[1])],
> self.dims)
>
> self.b = theano.shared(np.random.randn(self.dims[1]), name='b')
> self.b =
> t.reshape(self.theta[(self.dims[0]*self.dims[1]):self.theta.get_value().shape[0]],
>
> (self.dims[0],1))
>
> # the feed-forward function is not fully written
> def feed_forward(self):
> f = t.dot(self.w, self.x.T) + self.b
> return f
>
> def get_cost_grads(self):
> cost = t.sum(t.abs_(self.feed_forward()))
> gradw = t.grad(cost=cost, wrt=self.w).flatten()
> gradb = t.grad(cost=cost, wrt=self.b).flatten()
>
> return cost, gradw, gradb
>
>
> def training_functions(data, model, dims):
> cost, grad = model.get_cost_grads()
> fn = theano.function(inputs=[], outputs=[cost, grad],
> givens={model.x: data}, allow_input_downcast=True)
>
> def train_fn(theta_value):
> # reshape the theta value for Theano and convert to float32
> model.w = t.reshape(theta_value[0:(dims[0]*dims[1])], dims)
> model.b =
> t.reshape(theta_value[(dims[0]*dims[1]):theta_value.shape[0]], (dims[0],1))
>
> c, gw, gb = fn()
> # convert values to float64 for SciPy
> c = np.asarray(c, dtype=np.float64)
> gw = np.asarray(gw, dtype=np.float64)
> gb = np.asarray(gb, dtype=np.float64)
> grad = np.append(gw, gb)
>
> return c, grad
>
> return train_fn
>
>
> theta = theano.shared(np.random.randn((input_dim*hdim) + hdim),
> name='theta')
>
> np.random.seed(0)
> theta.set_value(np.random.randn((input_dim*hdim) + hdim).astype('float32'))
>
> model = SparseFilter(theta, dims, x)
> train_fn = training_functions(data, model, dims)
>
> from scipy.optimize import minimize
>
> weights = minimize(train_fn, model.theta.eval(),
> method='L-BFGS-B', jac=True,
> options={'maxiter': 100, 'disp': True})
>
>
> There is no iteration and the oparation stops.
> There is a message in the output
>
> message: 'ABNORMAL_TERMINATION_IN_LNSRCH'
>
>
> Any suggestion? Thank you.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
--
Pascal
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
