Hello Andreas, Thanks for the reply. > Why do you want to use probability estimates in liblinear? We need probability estimates because our use case requires a way of gauging how 'confident' the match is, when the classifier chooses a category for an input.
> Do you know what they are doing? I would expect they just do a soft-max. I don't. :) But according to the LIBLINEAR FAQ: "If you really would like to have probability outputs for SVM in LIBLINEAR, you can consider using the simple probability model of logistic regression. Simply modify the following subroutine in linear.cpp." So I assume that it's using the same method logistic regression is using to determine probability estimates. Dr Lin also wrote, in an email: "I think you can use [the simple probability model of logistic regression] in general. Because svm isn't a prob model, some more formal ways have been developed for its prob outputs (we mentioned that in faq)" FWIW, the probability outputs seemed reliable enough, and correlated pretty well with correct/incorrect predictions in our test data. > This could be very easily implemented in Python - or by changing the check. I > think the reason we haven't done this yet is that this functionality doesn't > really have a mathematical interpretation, in particular since liblinear only > implements one-vs-rest multi-class classification. I think it's worth doing, because as I mentioned, we seemed to be getting meaningful results. We also compared probability outputs from LinearSVC() and OneVsRestClassifier(LinearSVC()); in the former, we would get N probabilities that the input belonged to each class, and in the latter we would get N "IS" and "IS NOT" pairs, showing for each class the probability that the input was closer to that class or to the rest of the classes. Again, these probability estimates did not seem like meaningless noise! > (Do you have two or more classes?). We have approximately 1200 classes. As I mentioned we tried feeding them all into a single LinearSVC instance and also wrapping it in a OneVsRestClassifier, both achieving meaningful probability outputs. > Do you have strong reasons not to use logistic regression? Correct me if I've misunderstood, but regression is meant for fitting to a continuous variable or something, not classifying inputs to discrete classes, right? We're classifying emails into ~1200 distinct classes, so Logistic Regression is meaningless for us (in fact, when we tried it, it achieved a hilarious 48% cross-validated k=3 accuracy. LinearSVC achieves 95% accuracy.) Thanks again. Afik Cohen Andreas Mueller <amueller@...> writes: > > Hi Afik. > Thanks for your mail. > Why do you want to use probability estimates in liblinear? > Do you know what they are doing? I would expect they just do a soft-max. > This is not really "probability output", it is just a way to normalize > the decision > function. > This could be very easily implemented in Python - or by changing the check. > I think the reason we haven't done this yet is that this functionality > doesn't > really have a mathematical interpretation, in particular since liblinear > only > implements one-vs-rest multi-class classification. > (Do you have two or more classes?). > > Do you have strong reasons not to use logistic regression? > > Cheers, > Andy ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
