You can also add names to your intermediate variables. theano.grad() will use them to create names for the grads nodes. This will help you understand what is going on. Maybe the debugprint parameter stop_on_name=True could also help make that graph more readable.
On Thu, Jun 29, 2017 at 9:22 AM Frédéric Bastien <[email protected]> wrote: > The + of the + T.dot(u, v). > > The debugprint command I gave you will help separate the forward > computation from the grad computation. > > The grad of a dot is a another dot. So what would explain a 0 outputs > would be too many or only zeros in the inputs. Can you very the values of m > and n? Make sure there is no zeros in them. > > On Thu, Jun 29, 2017 at 9:05 AM Mohamed Akrout <[email protected]> > wrote: > >> Yes I printed the gradient function of m but it is extremely big. I find >> it unreadable (file attached). I don't know how this tree will help me find >> the problem. There are nodes who are Alloc and second but I don't know how >> to change and/or control them. >> >> When you say: "Only the extra addition will be done at each iterations", >> about which extra addition are you talking? >> >> Thank you Fred. >> >> Med >> >> Regarding your notice, if m and n are non sequence, Theano will not updat >> >> >> On Thursday, June 29, 2017 at 8:34:32 AM UTC-4, nouiz wrote: >> >>> I don't know, but you can use theano.printing.debugprint([cost, >>> grads...]) >>> >>> To see the gradient function. Maybe it will help you understand what is >>> going on. >>> >>> Don't forget m and n are non sequence. This mean the dot will be lifted >>> out of the loop by Theano. Only the extra addition will be done at each >>> iterations. >>> >>> Fred >>> >>> Le mer. 28 juin 2017 19:12, Mohamed Akrout <[email protected]> a >>> écrit : >>> >> Hi all, >>>> >>>> I am running a neuroscience with an recurrent neural network model with >>>> Theano: >>>> >>>> >>>> >>>> def rnn(u_t, x_tm1, r_tm1, Wrec): >>>> x_t = ( (1 - alpha)*x_tm1 + alpha*(T.dot(r_tm1, Wrec ) + brec >>>> + u_t[:,Nin:]) ) >>>> r_t = f_hidden(x_t) >>>> >>>> >>>> then I define the scan function to iterate at each time step iteration >>>> >>>> [x, r], _ = theano.scan(fn=rnn, >>>> outputs_info=[x0_, f_hidden(x0_)], >>>> sequences=u, >>>> non_sequences=[Wrec]) >>>> >>>> Wrec and brec are learnt by stochastic gradient descent: g = >>>> T.grad(cost , [Wrec, brec]) >>>> >>>> where cost is the cost function: T.sum(f_loss(z, target[:,:,:Nout])) >>>> with z = f_output(T.dot(r, Wout_.T) + bout ) >>>> >>>> Until now, everything works good. >>>> >>>> >>>> >>>> Now I want to add two new vectors, let's call them u and v so that the >>>> initial rnn function becomes: >>>> >>>> >>>> def rnn(u_t, x_tm1, r_tm1, Wrec, *u, v*): >>>> x_t = ( (1 - alpha)*x_tm1 + alpha*(T.dot(r_tm1, Wrec + *T.dot(u, >>>> v)* ) + brec + u_t[:,Nin:]) ) >>>> r_t = f_hidden(x_t) >>>> >>>> [x, r], _ = theano.scan(fn=rnn, >>>> outputs_info=[x0_, f_hidden(x0_)], >>>> sequences=u, >>>> non_sequences=[Wrec,* m, n*]) >>>> >>>> m and n are the variables corresponding to u and v in the main function. >>>> >>>> and suddenly, the gradient T.grad(cost, m) and T.grad(cost, n) are zeros >>>> >>>> I am blocked since 2 weeks now on this problem. I verified that the >>>> values are not integer by using dtype=theano.config.floatX every where in >>>> the definition of the variables. >>>> >>>> As you can see the link between the cost and m (or n) is: the cost >>>> function depends on z, and z depends on r and r is one of the outputs of >>>> the rnn function that uses m and n in the equation. >>>> >>>> Do you have any ideas why this does not work ? >>>> >>>> Any idea is welcome. I hope I can unblock this problem soon. >>>> Thank you! >>>> >>>> -- >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "theano-users" group. >>>> >>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>> >>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "theano-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
