2012/7/18 Philipp Singer <[email protected]>: > Am 18.07.2012 15:32, schrieb Peter Prettenhofer: >>>> In this case I would fit one MultinomialNB for the foreground model and >>>> one for the background model. But how would I do the feature extraction >>>> (I have text documents) in this case? Would I fit (e.g., tfidf) on the >>>> whole corpus (foreground + background) and then transform both datasets >>>> on the fitted infos and the test dataset as well? >> >> Personally, I'd start without using IDF; Otherwise, wrap both >> estimators using a Pipeline and add a TfidfTransformer (see [1]). >> >> best, >> Peter >> >> [1] >> http://scikit-learn.org/stable/auto_examples/grid_search_text_feature_extraction.html >> >> > > Yes, I am currently trying around with tf only, but the vocabulary is > still dependen on the corpus.
I would fit the vectorizor on both datasets (such that the vocabulary covers the union) and then fit the IDF transformers on each dataset individually. Disclaimer: I hardly use sklearn's text utilities > > Philipp > > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Scikit-learn-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general -- Peter Prettenhofer ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
