Hi I am trying to build a simple autoencoder model in Theano but I also want to optimize using Scipy Optimize Minimize. I found a code here : http://dlacombejr.github.io/programming/2015/09/13/sparse-filtering-implemenation-in-theano.html which I modified a bit. The problem is since scipy minimum takes in an array, but I need to optimize more that one weight parameters. what is the solution to this problem if I want to keep using the same structure as the code in the link?
I found a possible way of getting around this problem, that is to pass in all the weights as a single array flattened and appended, and then assigning parts of the array to various variables, according to this https://github.com/Theano/Theano/issues/3131 import theano from theano import tensor as t import numpy as np class SparseFilter(object): def __init__(self, theta, dims, x): # assign inputs to sparse filter self.theta = theta self.x = x self.dims = dims self.w = theano.shared(np.random.randn(self.dims[1], self.dims[0]), name='w') self.w = t.reshape(self.theta[0:(self.dims[0]*self.dims[1])], self.dims) self.b = theano.shared(np.random.randn(self.dims[1]), name='b') self.b = t.reshape(self.theta[(self.dims[0]*self.dims[1]):self.theta.get_value().shape[0]], (self.dims[0],1)) # the feed-forward function is not fully written def feed_forward(self): f = t.dot(self.w, self.x.T) + self.b return f def get_cost_grads(self): cost = t.sum(t.abs_(self.feed_forward())) gradw = t.grad(cost=cost, wrt=self.w).flatten() gradb = t.grad(cost=cost, wrt=self.b).flatten() return cost, gradw, gradb def training_functions(data, model, dims): cost, grad = model.get_cost_grads() fn = theano.function(inputs=[], outputs=[cost, grad], givens={model.x: data}, allow_input_downcast=True) def train_fn(theta_value): # reshape the theta value for Theano and convert to float32 model.w = t.reshape(theta_value[0:(dims[0]*dims[1])], dims) model.b = t.reshape(theta_value[(dims[0]*dims[1]):theta_value.shape[0]], (dims[0],1)) c, gw, gb = fn() # convert values to float64 for SciPy c = np.asarray(c, dtype=np.float64) gw = np.asarray(gw, dtype=np.float64) gb = np.asarray(gb, dtype=np.float64) grad = np.append(gw, gb) return c, grad return train_fn theta = theano.shared(np.random.randn((input_dim*hdim) + hdim), name='theta') np.random.seed(0) theta.set_value(np.random.randn((input_dim*hdim) + hdim).astype('float32')) model = SparseFilter(theta, dims, x) train_fn = training_functions(data, model, dims) from scipy.optimize import minimize weights = minimize(train_fn, model.theta.eval(), method='L-BFGS-B', jac=True, options={'maxiter': 100, 'disp': True}) There is no iteration and the oparation stops. There is a message in the output message: 'ABNORMAL_TERMINATION_IN_LNSRCH' Any suggestion? Thank you. -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
