On Tue, Feb 19, 2013 at 7:52 PM, Mathieu Blondel <math...@mblondel.org> wrote: > On Wed, Feb 20, 2013 at 7:36 AM, James Bergstra > <james.bergs...@gmail.com> wrote: > And who would have thought that the >> Perceptron would have 8 hyper-parameters?? > > I think the Perceptron is not a good candidate. I'd rather choose > SGDClassifier (you can thus add the loss function to the parameter > space). Perceptron in scikit-learn has many parameters because it > inherits from the SGDClassifier machinery. However, if you use the > default options, you get the standard Perceptron (which doesn't have > any hyperparameter). Since it is indeed confusing, we could remove the > parameters (people who want to tune those parameters can use > SGDClassifier(loss="perceptron") anyway) or at the very least update > the docstring to reflect that the default options lead to the standard > Perceptron.
Interesting, I didn't dig under the hood of the Perceptron class. If the Perceptron is essentially just a simplified interface to the underlying SGDClassifier machinery, then yes - the hyper-parameter tuning code should instead target the more general underlying API. Thanks. > Is it possible to gain insights from the hyperparameter search? Like > what parameter (or combination of parameters) contributes the most to > the accuracy? When you try some hyper-parameter assignments and measure some model fitness (e.g. validation set classification accuracy) then you accumulate a new (input, output) data set. Insight is about finding some kind of statistical patterns in the regression input -> output. So for sure you can get insight, by doing... machine learning :) The algorithm for hyperparameter search that I'm using in hyperopt is doing that. It's a regression algorithm that slowly adapts to the hyper-parameter -> performance relationship to make hyperparameter search faster. Have a look for "Sequential Model Based Optimization" to learn more about this, or "Bayesian Optimization" Hyperopt comes with some visualization tools for trying to understand high-dimensional hyperparameter spaces. It can be interesting to visualize correlations between individual hyperparameters and fitness, or pairs, but beyond that there isn't usually enough data to estimate a correlation accurately (to say nothing of how many possible triples there are to fit on the screen). - James ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
