here is where to look for the bug: https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/svm/base.py#L543
please send us a pull request if you find the problem. Alex On Thu, May 24, 2012 at 5:20 PM, Ian Goodfellow <[email protected]> wrote: > OK, here's a final version of my script demonstrating that there's a > bug somewhere in the computation of coef_. If I compute coef_ myself > from dual_coef_ and support_vectors_ I am able to match the predict > function with the dot product method, and the new coef_ has a dot > product of about -1.5 with the coef_ provided by the SVC class, > meaning they are not very similar at all. > > > > > > > import numpy as np > from sklearn.svm import SVC > import time > > rng = np.random.RandomState([1,2,3]) > > m = 1000 > n = 1000 > > X = rng.randn(m,n) > w = rng.randn(n) > b = rng.randn(1) > y = (np.dot(X,w) + b ) > 0 > > t1 = time.time() > svm = SVC(kernel = 'linear', C = 1.0).fit(X,y) > t2 = time.time() > print 'train time ',t2 - t1 > > X2 = X > > t1 = time.time() > y1 = svm.predict(X2) > t2 = time.time() > print 'predict time ',t2 - t1 > print '# support vectors:',svm.n_support_ > print 'predict time per support vector:',(t2-t1)/float(svm.n_support_.sum()) > > t1 = time.time() > y2 = ( np.dot(X2, svm.coef_.T) - svm.intercept_ ) > 0 > t2 = time.time() > print 'dot product time',t2 -t1 > > print 'class 1 prevalence ',y.mean() > print 'predict accuracy ',(y1 == y).mean() > print 'dot product accuracy ',(y2 == y).mean() > print 'predict and dot agreement rate',(y1 == y2).mean() > > coefs = svm.dual_coef_ > assert len(coefs.shape) == 2 > assert coefs.shape[0] == 1 > coefs = coefs[0,:] > w = np.dot(svm.support_vectors_.T, coefs) > f = np.dot(X,w) + b > y3 = (f < 0) > print 'agreement rate with my method: ',(y3 == y1).mean() > > print 'dot prod between sklearn coef_ and my coef_: ',np.dot(w,svm.coef_[0,:]) > > > > > > > > On Thu, May 24, 2012 at 11:17 AM, Ian Goodfellow > <[email protected]> wrote: >> On Thu, May 24, 2012 at 10:55 AM, Alexandre Gramfort >> <[email protected]> wrote: >>> is this test buggy: >> >> Yes. The test passes for me, but if I replace X and Y from the binary >> section of that test with the X and y from my script, the test fails. >> >>> >>> https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/svm/tests/test_svm.py#L264 >>> >>> ? >>> >>> could it be a numerical error? >> >> I don't know why the test passes for the default data. >> >> >>> >>> Alex >>> >>> On Thu, May 24, 2012 at 4:50 PM, Ian Goodfellow >>> <[email protected]> wrote: >>>> On Thu, May 24, 2012 at 10:47 AM, Olivier Grisel >>>> <[email protected]> wrote: >>>>> 2012/5/24 Ian Goodfellow <[email protected]>: >>>>>> Well that's the thing, coef_ and intercept_ seem wrong, given the >>>>>> results of my script below. My implementation of predict based on >>>>>> coef_ only agrees with predict 50% of the time. >>>>>> Does anyone know if coef_ and intercept_ are just getting set wrong? >>>>>> Or does predict implement a different decision function than >>>>>> (X*coef_+intercept_)>0? >>>>> >>>>> If X is dense then this should be, for binary classification with a >>>>> linear kernel: >>>>> >>>>> np.dot(X, clf.coef_) - clf.intercept > 0 (if I remember correctly the >>>>> sign of the intercept) >>>> >>>> I've tried both signs of the intercept and neither one agrees with the >>>> predict function. >>>> >>>>> >>>>> Off course this formula is specific to the dense linear case. It won't >>>>> work for the SVC models with kernels. >>>> >>>> Yes, of course. You'll notice that my debugging script uses kernel = >>>> 'linear'... >>>> >>>>> >>>>> -- >>>>> Olivier >>>>> http://twitter.com/ogrisel - http://github.com/ogrisel >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Live Security Virtual Conference >>>>> Exclusive live event will cover all the ways today's security and >>>>> threat landscape has changed and how IT managers can respond. Discussions >>>>> will include endpoint security, mobile security and the latest in malware >>>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >>>>> _______________________________________________ >>>>> Scikit-learn-general mailing list >>>>> [email protected] >>>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>>> >>>> ------------------------------------------------------------------------------ >>>> Live Security Virtual Conference >>>> Exclusive live event will cover all the ways today's security and >>>> threat landscape has changed and how IT managers can respond. Discussions >>>> will include endpoint security, mobile security and the latest in malware >>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >>>> _______________________________________________ >>>> Scikit-learn-general mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>> >>> ------------------------------------------------------------------------------ >>> Live Security Virtual Conference >>> Exclusive live event will cover all the ways today's security and >>> threat landscape has changed and how IT managers can respond. Discussions >>> will include endpoint security, mobile security and the latest in malware >>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >>> _______________________________________________ >>> Scikit-learn-general mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Scikit-learn-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
