Hi @Daniel Renshaw： I plan to use the REINFORCE learning and adopt "known_grads" w.r.t. some params, which involves the argmax operation. However, I find that theano can just calculate the gradient automaticly. I wonder how can I apply my REINFORCE rule ??? Or if the automaticly-calculated gradient are correct in my case??? Thanks a lot!!!
On Wednesday, July 1, 2015 at 2:18:45 AM UTC+8, Daniel Renshaw wrote: > > argmax is differentiable: > > import theano > import theano.tensor as tt > > x = tt.matrix() > y = tt.vector() > > d = x.argmax(axis=1) - y > c = d.sum() > gs = theano.grad(c, [x, y]) > > > On 30 June 2015 at 19:38, Jesse Livezey <[email protected] <javascript:> > > wrote: > >> >> >> On Monday, June 29, 2015 at 3:25:30 AM UTC-7, Abhishek Shivkumar wrote: >>> >>> Hi, >>> >>> I am using a single output node (tanh) that outputs between -1 / +1. >>> >> >>> My y_expected array is one-hot vector for binary classification and >>> consists of 500 rows (minibatch size) of [0,1] or [1,0] in every row. >>> >> If your actual y values are [0, 1], it would be more natural to use >> sigmoid rather than tanh. >> >>> >>> In any minibatch, I want to weight the individual errors based on what >>> is the expected class and then add them together before calculating the >>> final minimum squared error loss. >>> >>> I have been learning Theano and wrote the following code but it throws >>> lots of errors I think because of syntax and not a thorough understanding >>> of how Theano operations work. I would appreciate if anyone could correct >>> it. The idea is to multiply the error on every sample in the minibatch by >>> the weight of the corresponding expected class. >>> >>> --------------------------------------------- >>> >>> def negative_log_likelihood(self, y): #y is the expected output in >>> one-hot coded vectors >>> >>> >>> y_actual = T.argmax(y, axis=1) # calculate the expected output as a >>> single column vector here >>> >> The argmax operation is not differentiable, and so it won't work in a >> cost function. >> >>> >>> ones_matrix = theano.shared(numpy.ones((500, 1), >>> dtype=theano.config.floatX), borrow=True) #create a ones matrix of size >>> (500, 1) >>> >>> >>> class_counts = T.sum(y, axis=0) # y_train = array of 1-hot encoded >>> vectors >>> >>> >>> weights = (1. / class_counts) * class_counts.mean() #Create a row >>> vector that has the correspending weights for each of the classes (binary >>> classification). >>> >>> ones = T.matrix('o') >>> >>> cl = T.matrix('c') >>> >>> f = theano.function([ones , cl], T.dot(ones, cl)) >>> >> You don't need to make a function here, you can just let weighs_matrix = >> T.dot(ones, cl) >> >>> weights_matrix = f(ones_matrix, weights) #The idea is to multiply the >>> (550,1) matrix of 1s with (1, 2) matrix of weights to repeat the weights >>> matrix (1, 2) 500 times as rows >>> >>> weights_actual = weights_matrix * y >>> >>> >>> >>> weights_actual_sum = T.sum(weights_actual, axis=0) >>> >>> return T.mean(((y_actual - self.y_pred)**2) * weights_actual_sum >>> >> You should be comparing y_actual and self.y_pred in their one-hot >> representation, not as class indices (output of argmax). >> >>> ---------------------------------------------- >>> >>> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "theano-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
