I just looked into Lasagne's implementation of Adam, and it became clear how they keep the 'm' and 'v' quantities away from the main code of a model. Unfortunately, it does not seem to work well with Probabilistic Matrix Factorization, with an explanation that I posted in their group: https://groups.google.com/forum/#!topic/lasagne-users/gjVZ7wvURwk
Maybe someone here knows how to have an elegant solution to this problem? best regards, -Francesco On Thursday, 8 September 2016 18:27:04 UTC+2, fstab wrote: > > On Thursday, 8 September 2016 18:05:58 UTC+2, nouiz wrote: >> >> Why do you want to subclass SharedVariable to implement Adam? >> > > Because I want to store 'm' and 'v' quantities alongside parameters and > latent variables and the most obvious way to do similar tricks in the past > was to subclass a certain class and add new instance variables. > I posted some more information here: > http://stackoverflow.com/questions/39394349/theano-shared-variable-is-it-possible-to-add-custom-fields-instance-variables-c > > >> Many people have implement Adam without subclassing SharedVariable. >> > > I know that is possible by keeping a cumbersome accounting of 'm' and 'v' > quantities for each parameter and latent variable, but maybe you are > referring to more smart and code-readable way to do this? In that case I > would be interested in reading such implementations. Would you mind telling > me which projects those are? > > >> Normally, why we create a new subclass of SharedVariable is when we >> create a new Variable subclass. This is when we want to represent a new >> object in the graph, like a sparse object, an object on the gpu, python >> list, python dict (not done, but would request a new Variable subclass). >> > > so in this case I would need to create a VariableAdam class and then a > SharedVariableAdam subclass. Would that be easy for me to implement? > > >> Frédéric >> > > Thank you very much for replying > Francesco > > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
