Ok, sounds good. I only brought it up because it made unit testing a
little bit trickier but I found a work around. I'll see if I can do some
tests to try results with different intercept_decay values.
On 7/30/14, 12:01 PM, Peter Prettenhofer wrote:
The way I implemented it, the learning rate for the intercept should
be 0.01 times the learning rate of the other features.
The value of .01 is something that I set empirically, I adopted it
from Leon Buttou's sgd project and experimented with different values.
I found that lower intercept learning rates help a bit but the
concrete value is not too important - so I decided to use a fixed value.
I think the decay value might in fact be a function of the number of
non-zero values per feature. If you have a dataset with sparse and
dense features then intercept decay should be turned off --
alternatively, you can also scale the dense features to decrease their
magnitude.
2014-07-30 11:42 GMT+02:00 Danny Sullivan <[email protected]
<mailto:[email protected]>>:
I found that for sparse data, the scikit implementation of sgd
uses an intercept_decay variable set to .01
(SPARSE_INTERCEPT_DECAY) to avoid intercept oscillation. Shouldn't
this be determined by the learning_rate instead? I'm asking
because it adds a layer of tuning that the user doesn't have
control over.
Danny
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
[email protected]
<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
Peter Prettenhofer
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
