Re: [Scikit-learn-general] congratulations to Peter and to scikit-learn!

2012-07-09 Thread federico vaggi
Peter - did you get any updates from Kaggle? If not, is there anything that we as a community can do to sway them? On Sat, Jul 7, 2012 at 7:46 PM, Emanuele Olivetti wrote: > On 07/05/2012 04:37 PM, Olivier Grisel wrote: > > 2012/7/5 Emanuele Olivetti : > >> On 07/05/2012 08:49 AM, Olivier Grisel

Re: [Scikit-learn-general] SGDClassifier: multiclass predict_proba() ?

2012-07-09 Thread Peter Prettenhofer
2012/7/9 Alexandre Gramfort : > hi all, > >> In addition to Platt scaling, there is isotonic regression. > > I started to play with the calibration based on isotonic regression > this morning: > > https://github.com/agramfort/scikit-learn/commit/5b1017df6ca69c7733f9e5c0c26926ce6c1c8775 > > it's bas

Re: [Scikit-learn-general] SGDClassifier: multiclass predict_proba() ?

2012-07-09 Thread Alexandre Gramfort
hi all, > In addition to Platt scaling, there is isotonic regression. I started to play with the calibration based on isotonic regression this morning: https://github.com/agramfort/scikit-learn/commit/5b1017df6ca69c7733f9e5c0c26926ce6c1c8775 it's based on mathieu's calibration_plot PR a sklear

Re: [Scikit-learn-general] SGDClassifier: multiclass predict_proba() ?

2012-07-09 Thread Mathieu Blondel
In addition to Platt scaling, there is isotonic regression. Two references: - Transforming Classifier Scores into Accurate Multiclass Probability Estimates (KDD 2002) - Predicting accurate probabilities with a ranking loss (ICML 2012) Mathieu On Mon, Jul 9, 2012 at 10:53 PM, Mathieu Blondel wrote

Re: [Scikit-learn-general] SGDClassifier: multiclass predict_proba() ?

2012-07-09 Thread Mathieu Blondel
> To be honest, I haven't worked with Platt scaling yet - Paolo has > discussed it on the mailing list a while ago [1] - maybe he has some > code to share. > Paolo's code is here: https://github.com/paolo-losi/scikit-learn/tree/calibration Mathieu -

Re: [Scikit-learn-general] SGDClassifier: multiclass predict_proba() ?

2012-07-09 Thread Emanuele Olivetti
On 07/09/2012 01:56 PM, Peter Prettenhofer wrote: > 2012/7/9 Emanuele Olivetti : >> Hi Peter, >> >> Thanks for your answer and the interesting thread you linked >> (I should have catch it before sending my email ;-)). >> >> I agree that Platt scaling would be of help. As far as I understand >> in t

Re: [Scikit-learn-general] Incorporation of extra training examples

2012-07-09 Thread Peter Prettenhofer
2012/7/9 Philipp Singer : > Am 09.07.2012 13:47, schrieb Peter Prettenhofer: >> Hi, > > Hey! >> >> some quick thoughts: >> >> - if you use a multinomial Naive Bayes classifier (aka a language >> model) you can fit a background model on the large dataset and use >> that to smooth the model fitted on

Re: [Scikit-learn-general] Incorporation of extra training examples

2012-07-09 Thread Philipp Singer
Am 09.07.2012 13:47, schrieb Peter Prettenhofer: > Hi, Hey! > > some quick thoughts: > > - if you use a multinomial Naive Bayes classifier (aka a language > model) you can fit a background model on the large dataset and use > that to smooth the model fitted on the smaller dataset. That's a nice i

Re: [Scikit-learn-general] Incorporation of extra training examples

2012-07-09 Thread Philipp Singer
Am 09.07.2012 13:59, schrieb Vlad Niculae: > Another (hackish) idea to try would be to keep the labels of the extra > data bit give it a sample_weight low enough not to override your good > training data. That's actually a great and simple idea. Would I do that similar to that example: http://sc

Re: [Scikit-learn-general] Incorporation of extra training examples

2012-07-09 Thread Vlad Niculae
Another (hackish) idea to try would be to keep the labels of the extra data bit give it a sample_weight low enough not to override your good training data. On 09.07.2012, at 12:43, Philipp Singer wrote: > Hey! > > I am currently doing text classification. I have the following setup: > > 78 class

Re: [Scikit-learn-general] SGDClassifier: multiclass predict_proba() ?

2012-07-09 Thread Peter Prettenhofer
2012/7/9 Emanuele Olivetti : > Hi Peter, > > Thanks for your answer and the interesting thread you linked > (I should have catch it before sending my email ;-)). > > I agree that Platt scaling would be of help. As far as I understand > in the multiclass case [0] it should be like: > > s = clf.decis

Re: [Scikit-learn-general] Incorporation of extra training examples

2012-07-09 Thread Peter Prettenhofer
Hi, some quick thoughts: - if you use a multinomial Naive Bayes classifier (aka a language model) you can fit a background model on the large dataset and use that to smooth the model fitted on the smaller dataset. - you should look at the domain adaptation / multi-task learning literature - this

Re: [Scikit-learn-general] Incorporation of extra training examples

2012-07-09 Thread Gael Varoquaux
Hi, You can try setting this as a semi-supervised learning problem and using label propagation: http://scikit-learn.org/stable/modules/label_propagation.html#label-propagation HTH, G -- Live Security Virtual Conference

Re: [Scikit-learn-general] SGDClassifier: multiclass predict_proba() ?

2012-07-09 Thread Emanuele Olivetti
Hi Peter, Thanks for your answer and the interesting thread you linked (I should have catch it before sending my email ;-)). I agree that Platt scaling would be of help. As far as I understand in the multiclass case [0] it should be like: s = clf.decision_function(x) p(s) = exp(A*s+B) / exp(A*s+

[Scikit-learn-general] Incorporation of extra training examples

2012-07-09 Thread Philipp Singer
Hey! I am currently doing text classification. I have the following setup: 78 classes max 1500 train examples per class overall around 90.000 train examples same amount of test examples I am pretty happy with the classification results (~52% f1 score) which is fine for my task. But now I have

Re: [Scikit-learn-general] SGDClassifier: multiclass predict_proba() ?

2012-07-09 Thread Peter Prettenhofer
Hi Emanuele, 2012/7/9 Emanuele Olivetti : > Hi, > > I'm not expert in Stochastic Gradient Descent but I'm wondering why > predict_proba() is not available in the multiclass case. Precisely > I get this (sklearn v0.11): > > NotImplementedError: predict_(log_)proba only supported for binary >

[Scikit-learn-general] SGDClassifier: multiclass predict_proba() ?

2012-07-09 Thread Emanuele Olivetti
Hi, I'm not expert in Stochastic Gradient Descent but I'm wondering why predict_proba() is not available in the multiclass case. Precisely I get this (sklearn v0.11): NotImplementedError: predict_(log_)proba only supported for binary classification I see that the multiclass case is supp