2012/7/18 Philipp Singer <[email protected]>: > Am 09.07.2012 14:44, schrieb Peter Prettenhofer: >> 2012/7/9 Philipp Singer <[email protected]>: >>> Am 09.07.2012 13:47, schrieb Peter Prettenhofer: >>>> Hi, >>> >>> Hey! >>>> >>>> some quick thoughts: >>>> >>>> - if you use a multinomial Naive Bayes classifier (aka a language >>>> model) you can fit a background model on the large dataset and use >>>> that to smooth the model fitted on the smaller dataset. >>> >>> That's a nice idea. Is there a simple way to try this out fast in >>> scikit-learn? >> >> not really, you would need to write you own estimator that simply >> takes the two MultinomialNaiveBayes models as arguments and does the >> smoothing on predict(_proba); shouldn't be too much code though. If >> you do a simple linear interpolation the following should suffice:: >> >> def predict_proba(X): >> return self.lambda * self.foreground.predict_proba(X) + (1.0 - >> self.lambda) * self.background.predict_proba(X) >> >> >> You could estimate lambda via EM but I'd rather tune using CV. >> > > Hey! > > I finally found some time to implement this. But I have stumbled upon > this problem: > > In this case I would fit one MultinomialNB for the foreground model and > one for the background model. But how would I do the feature extraction > (I have text documents) in this case? Would I fit (e.g., tfidf) on the > whole corpus (foreground + background) and then transform both datasets > on the fitted infos and the test dataset as well?
Sounds reasonable. -- Olivier http://twitter.com/ogrisel - http://github.com/ogrisel ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
