On Fri, Nov 04, 2011 at 06:12:48PM +0100, Andreas Müller wrote: > > > In my case I don't use RPROP (I don't know what it is and I just use a > > simple backprop) and I use Leon Bottou's trick to perform a burn-in on > > the first 10k samples with a grid search of learning rate parameters > > and then select the most effective learning rate and multiply it by 2 > > (it brings robustness). In my experiment it did work pretty well. > > > I only learned about this trick recently and haven't really used it yet. > We tried it on convolutional nets and it didn't work well :-/ > Maybe I'll give it another shot. > > RPROP maintains a dynamic learn rate for each parameter.
Sounds a bit like "delta-bar-delta". > It only looks at the sign of the gradient. Surely it can't work online then, can it? David ------------------------------------------------------------------------------ RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
