2011/11/4 Andreas Müller <[email protected]>: > My question is: has anyone started with a mlp implementation yet?
I was just working on one :) I have the predict function for an arbitrary number of hidden layers (classifier case) and some snippets of the RPROP algorithm. I've been using weight vectors that come out of a Matlab implementation for now. There used to be an MLP implementation in older versions (around 0.2, I believe) but it was abandoned. > My feature list would be: > - online, minibatch and batch learning I only need batch learning and classification for now... shall we keep it simple? > - vanilla gradient descent and rprop > - l2 weight decay optional > - tanh nonlinearities Logistic activation functions seem fashionable; that's what Bishop and other textbooks use. I'm not sure if there's a big difference, but it seems to me that gradient computations might be slightly more efficient (guesswork, I admit). We can always add a steepness parameter later. I've been reading the RPROP papers and it looks like IRPROP- is the algorithm to go for; it's simple and not significantly worse than RPROP+. We could look at the RPROP implementation in Wapiti (and maybe even copy bits of it, it's MIT-licensed). -- Lars Buitinck Scientific programmer, ILPS University of Amsterdam ------------------------------------------------------------------------------ RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
