Sounds good to me, however, I am not sure how "high" this is on the current
scikit-priority list.
In terms of the inheritance hierarchy I could think of something like
MetaAggregator (collects predictions of estimators)
VotingClassifier (majority vote for classification)
VotingRegressor (averaging for regression)
StackingRegressor (feed to meta regressor)
StackingClassifier (feed to meta classifier)
> On Nov 11, 2015, at 6:41 PM, Scott Turner <[email protected]> wrote:
>
> On Wed, Nov 11, 2015 at 6:18 PM,
> <[email protected]
> <mailto:[email protected]>> wrote:
> I am only a little concerned if averaging the results of different regressors
> could potentially help with the predictive performance.
>
> I believe it has been proven that averaging uncorrelated regressors improves
> performance. (Whether it is easy to find uncorrelated regressors in practice
> is another question.) The Wikipedia article has some references, although
> they seem to focus primarily on ensemble averaging in neural networks:
>
> https://en.wikipedia.org/wiki/Ensemble_Averaging
> <https://en.wikipedia.org/wiki/Ensemble_Averaging>
>
> I think Wolpert 1992 is the seminal paper.
>
> But I think the implementation should be general-purpose stacking, permitting
> a user-specified meta regressor for combining the base regressors (with or
> without the base features). Averaging (or taking the median) is just a
> simplified special case. And I think stacking is well-established, e.g.,
> http://link.springer.com/article/10.1007%2FBF00117832
> <http://link.springer.com/article/10.1007%2FBF00117832> although perhaps less
> used than boosting, etc. these days.
>
> If this does get implemented as a generalization of Voting, it would be good
> to back-fit stacking to the VotingClassifier as well.
>
> -- Scott
> ------------------------------------------------------------------------------
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
