Thanks for the responses. I'm very interested in being able to use base
learners other than decision trees (actually that's what I'm most interested
in, since coming across http://elf-project.sourceforge.net/).
On Oct 16, 2012, at 11:30 AM, Olivier Grisel wrote:
> 2012/10/16 Peter Prettenhofer <[email protected]>:
>> Hi Siddhant,
>>
>> currently, we only provide two ensemble techniques: Random/Extra
>> Forest and Gradient Boosted Regression Trees - both use decision trees
>> as weak learners. Gradient Boosting itself can be used with different
>> base learners but AFAIK it hardly is*.
>>
>> Unfortunately, scikit-learn lacks classical ensemble technique such as
>> Bagging or AdaBoost (there is a PR for AdaBoost though [1]); what kind
>> of techniques are you looking for (stacking, bagging, adaboost, ..)?
>
> Would be good to implement linear blending (either with fixed weights
> or using CV score as weights) and stacking of heterogeneous models.
>
> For instance stacking an extra tree on top of [2 logistic regression
> probabilities trained for different feature extractor (e.g. char
> ngrams and words bigrams) + top 10 RadomizedPCA transformed features +
> top 10 NMF transformed features].
>
> --
> Olivier
> http://twitter.com/ogrisel - http://github.com/ogrisel
>
> ------------------------------------------------------------------------------
> Don't let slow site performance ruin your business. Deploy New Relic APM
> Deploy New Relic app performance management and know exactly
> what is happening inside your Ruby, Python, PHP, Java, and .NET app
> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
> http://p.sf.net/sfu/newrelic-dev2dev
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
