2012/7/9 Emanuele Olivetti <[email protected]>: > Hi Peter, > > Thanks for your answer and the interesting thread you linked > (I should have catch it before sending my email ;-)). > > I agree that Platt scaling would be of help. As far as I understand > in the multiclass case [0] it should be like: > > s = clf.decision_function(x) > p(s) = exp(A*s+B) / exp(A*s+B).sum() > > where s is, say, the distance from the separating plane for the > example x, while A and B should be estimated by minimizing the > loss of interest on a new set of examples {(x_1,y_1),...,(x_n,y_n)}.
To be honest, I haven't worked with Platt scaling yet - Paolo has discussed it on the mailing list a while ago [1] - maybe he has some code to share. Furthermore, Mathieu has a PR on calibration plots which might be handy as well [2] - AFAIK it should allow you to assess the probability calibration. [1] http://sourceforge.net/mailarchive/message.php?msg_id=27518584 [2] https://github.com/scikit-learn/scikit-learn/pull/882 > > Best, > > Emanuele > > [0]: http://en.wikipedia.org/wiki/Multinomial_logit > > > On 07/09/2012 10:53 AM, Peter Prettenhofer wrote: >> Hi Emanuele, >> >> 2012/7/9 Emanuele Olivetti <[email protected]>: >>> Hi, >>> >>> I'm not expert in Stochastic Gradient Descent but I'm wondering why >>> predict_proba() is not available in the multiclass case. Precisely >>> I get this (sklearn v0.11): >>> ---- >>> NotImplementedError: predict_(log_)proba only supported for binary >>> classification >>> ---- >>> I see that the multiclass case is supported through a "one versus all" >>> scheme. >>> Is this one the motivation for the lack of multiclass probabilities? >> Precisely, I'm not an expert on probability calibration thus I choose >> to not support predict_proba in the OVA case. We have started working >> on a proper multinomial logistic regression model for SGDClassifier >> but progress has stalled a bit. Having said that: liblinear's Logistic >> Regression model simply normalizes the individual OVA probabilities. >> >>> Are there solutions available? >> Take a look at this thread on the mailing list >> http://sourceforge.net/mailarchive/message.php?msg_id=29413195 >> >> The simplest "solution" is to monkey patch SGDClassifier such that it >> normalizes the OVA probabilities as liblinear does. >> >> Maybe a better solution is to use Platt scaling (do we have that in scikit?) >> >> best, >> Peter >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Scikit-learn-general mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> >> > > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Scikit-learn-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general -- Peter Prettenhofer ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
