The TheanoSGDClassifier that you mentioned is something I wrote, you can
find it here:
https://github.com/jaberg/scikit-learn/blob/ogrisel_image-patches/examples/applications/plot_image_classification_convolutional_features.py#L296
It is a pretty simple algorithm:
- normalize the input features to have 0 mean, unit variance
- update a linear classifier by minibatches with an annealed learning rate.
- stop when validation set performance plateaus.
The caller is responsible for picking a learning rate, there's no heuristic
for that.
N. Pinto is working on a port of this algorithm that doesn't rely on Theano,
and which does use multicore. I'm not sure what his plans are for that code.
- James
On Tue, Oct 4, 2011 at 12:12 PM, David Warde-Farley <
[email protected]> wrote:
> On 2011-10-04, at 3:37 AM, Peter Prettenhofer wrote:
>
> > I haven't looked at Theano's SGD yet - do they calibrate the learning
> > rate on held-out data or do they use an heuristic?
>
> Just to clear up, Theano doesn't contain any learning rate logic at all.
> It's just a tool to let you define your cost function, get the gradients
> automatically, and quickly generate/compile fast C code for evaluating each.
> Any learning rate tweaking would be Nicolas Pinto's own doing.
>
> David
>
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2dcopy1
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
