On Tue, Dec 06, 2011 at 09:04:22AM +0100, Alexandre Gramfort wrote: > > This actually gets at something I've been meaning to fiddle with and report > > but haven't had time: I'm not sure I completely trust the coordinate > > descent implementation in scikit-learn, because it seems to give me bogus > > answers a lot (i.e., the optimality conditions necessary for it to be an > > actual solution are not even approximately satisfied). Are you guys using > > something weird for the termination condition? > > can you give us a sample X and y that shows the pb? > > it should ultimately use the duality gap to stop the iterations but > there might be a corner case …
In [34]: rng = np.random.RandomState(0) In [35]: dictionary = rng.normal(size=(100, 500)) / 1000; dictionary /= np.sqrt((dictionary ** 2).sum(axis=0)) In [36]: signal = rng.normal(size=100) / 1000 In [37]: from sklearn.linear_model import Lasso In [38]: lasso = Lasso(alpha=0.0001, max_iter=1e6, fit_intercept=False, tol=1e-8) In [39]: lasso.fit(dictionary, signal) Out[39]: Lasso(alpha=0.0001, copy_X=True, fit_intercept=False, max_iter=1000000.0, normalize=False, precompute='auto', tol=1e-08) In [40]: max(abs(lasso.coef_)) Out[40]: 0.0 In [41]: from pylearn2.optimization.feature_sign import feature_sign_search In [42]: coef = feature_sign_search(dictionary, signal, 0.0001) In [43]: max(abs(coef)) Out[43]: 0.0027295761244725018 And I'm pretty sure the latter result is the right one, since In [45]: def gradient(coefs): ....: gram = np.dot(dictionary.T, dictionary) ....: corr = np.dot(dictionary.T, signal) ....: return - 2 * corr + 2 * np.dot(gram, coefs) + 0.0001 * np.sign(coefs) ....: and In [51]: max(abs(gradient(coef)[coef == 0])) Out[51]: 9.9849794040467979e-05 In [52]: max(abs(gradient(lasso.coef_))) Out[52]: 0.0065556004140218012 meaning that whatever that maximum gradient coefficient is should have been activated, since the gradient from the linear portion should override the sparsity penalty. ------------------------------------------------------------------------------ Cloud Services Checklist: Pricing and Packaging Optimization This white paper is intended to serve as a reference, checklist and point of discussion for anyone considering optimizing the pricing and packaging model of a cloud services business. Read Now! http://www.accelacomm.com/jaw/sfnl/114/51491232/ _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
