We are searching for the model that minimises a loss (the norm of the vector of differences between predictions and true targets) with a penalty/regularization term (the norm of the vector of weights). l1 and l2 are types of vector norm: l1 refers to the sum of the absolute values of a vector; l2 refers to the square-root of the sum-of-squares of a vector.
The penalty term accounts for our belief that not all features are equally predictive of the target. In particular, l1 regularization (also known as the Lasso) has a tendency to select models where some features have zero weight (effectively, a form of feature selection). The extent of the penalty is determined by a parameter C in this case (and by its reciprocal, alpha, in other scikit-learn estimators), and you can see some examples of this induced model sparsity at http://scikit-learn.org/stable/auto_examples/linear_model/plot_logistic_l1_l2_sparsity.html and http://scikit-learn.org/stable/auto_examples/linear_model/plot_lasso_and_elasticnet.html . This is knowledge you should be able to obtain from almost any machine learning course or textbook, and you should almost certainly be asking it on a wider forum than scikit-learn's mailing list, such as stats.stackexchange.com. On 15 August 2014 04:15, Pagliari, Roberto <[email protected]> wrote: > Can someone, please, expand on this comment from the documentation ( > http://scikit-learn.org/stable/modules/svm.html#tips-on-practical-use): > > > > · Using L1 penalization as provided by LinearSVC(loss='l2', > penalty='l1', dual=False) yields a sparse solution, i.e. only a subset of > feature weights is different from zero and contribute to the decision > function. Increasing C yields a more complex model (more feature are > selected). The C value that yields a “null” model (all weights equal to > zero) can be calculated using *l1_min_c* > <http://scikit-learn.org/stable/modules/generated/sklearn.svm.l1_min_c.html#sklearn.svm.l1_min_c> > . > > > > Why is the number of features reduced? > > > > Thank you, > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Scikit-learn-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > >
------------------------------------------------------------------------------
_______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
