2012/10/30 Afik Cohen <[email protected]>:
>> Do you know what they are doing? I would expect they just do a soft-max.
> I don't. :) But according to the LIBLINEAR FAQ: "If you really would like to
> have probability outputs for SVM in LIBLINEAR, you can consider using the
> simple
> probability model of logistic regression. Simply modify the following
> subroutine
> in linear.cpp." So I assume that it's using the same method logistic
> regression
> is using to determine probability estimates.
Judging from your code, it is, which is why I suggested copying over
the new predict_proba. It does exactly the same thing as Liblinear's
code for logistic regression probability predictions.
> I think it's worth doing, because as I mentioned, we seemed to be getting
> meaningful results. We also compared probability outputs from LinearSVC() and
> OneVsRestClassifier(LinearSVC()); in the former, we would get N probabilities
> that the input belonged to each class, and in the latter we would get N "IS"
> and
> "IS NOT" pairs, showing for each class the probability that the input was
> closer to that class or to the rest of the classes. Again, these probability
> estimates did not seem like meaningless noise!
Those "probabilities" are guaranteed to be >.5 for positive
predictions and <=.5 for negative ones in binary classification. They
will always sum to one for each sample in multiclass classification.
That's still the case with the hack I suggested. However, you could
also have used the output from decision_function; that produces a more
appropriate confidence score for linear SVMs. It's not a number
between zero and one, though, but either positive or negative and
farther away from zero as the confidence increases.
I don't get the remark about OneVsRestClassifier. What do you mean by
"is" and "is not" pairs? What does your target vector y look like?
>> Do you have strong reasons not to use logistic regression?
> Correct me if I've misunderstood, but regression is meant for fitting to a
> continuous variable or something, not classifying inputs to discrete classes,
> right? We're classifying emails into ~1200 distinct classes, so Logistic
> Regression is meaningless for us (in fact, when we tried it, it achieved a
> hilarious 48% cross-validated k=3 accuracy. LinearSVC achieves 95% accuracy.)
Actually, logistic regression *is* a classification model, it just has
a very unfortunate name ("In the terminology of statistics, this model
is known as logistic regression, although it should be emphasized that
this is a model for classification rather than regression" -- C.M.
Bishop, Pattern Recognition and Machine Learning, p. 205).
48% accuracy is extreme compared to 95%, though. How were you applying
logistic regression?
--
Lars Buitinck
Scientific programmer, ILPS
University of Amsterdam
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
