In my opinion the reference is distorting a concept that has a consolidated definition in the community. I am also familiar with the definition of WL as "an estimator slightly better than guessing", mostly decision stumps ( https://en.m.wikipedia.org/wiki/Decision_stump), which is not an component of RFs.
On Sun, Aug 16, 2020, 16:22 Nicolas Hug <nio...@gmail.com> wrote: > As previously mentioned, a "weak learner" is just a learner that barely > performs better than random. It's more common in the context of boosting, > but I think weak learning predates boosting, and the original RF paper by > Breiman does make reference to "weak learners": > > It's interesting that Forest-RI could produce error rates not far above > the Bayeserror rate. The individual classifiers are weak. For F=1, the > average tree errorrate is 80%; for F=10, it is 65%; and for F=25, it is > 60%. Forests seem to have theability to work with very weak classifiers > as long as their correlation is low > > Nicolas > > > On 8/16/20 2:29 PM, Guillaume Lemaître wrote: > > One needs to define what is the definition of weak learner. > > In boosting, if I recall well the literature, weak learner refers to > learner which unfit performing slightly better than a random learner. In > this regard, a tree with shallow depth will be a weak learner and is used > in adaboost or gradient boosting. > > However, in random forest the tree used are trees that overfit (deep tree) > so they are not weak for the same reason. However, one will never be able > to do what a forest will do with a single tree. In this regard, a single > tree is weaker than the forest. However, I never read the term for "weak > learner" in the context of the random forest. > > Sent from my phone - sorry to be brief and potential misspell. > *From:* fernando.wittm...@gmail.com > *Sent:* 16 August 2020 20:06 > *To:* scikit-learn@python.org > *Reply to:* scikit-learn@python.org > *Subject:* [scikit-learn] Opinion on reference mentioning that RF uses > weak learners > > Hello guys, > > The the following reference states that Random Forests uses weak learners: > - > https://blog.citizennet.com/blog/2012/11/10/random-forests-ensembles-and-performance-metrics#:~:text=The%20random%20forest%20starts%20with,corresponds%20to%20our%20weak%20learner.&text=Thus%2C%20in%20ensemble%20terms%2C%20the,forest%20is%20a%20strong%20learner > > The random forest starts with a standard machine learning technique called >> a “decision tree” which, in ensemble terms, corresponds to our weak learner. > > ... > > Thus, in ensemble terms, the trees are weak learners and the random >> forest is a strong learner. > > > I completely disagree with that statement. But I would like the opinion of > the community to double check if I am not missing something. > > > _______________________________________________ > scikit-learn mailing > listscikit-learn@python.orghttps://mail.python.org/mailman/listinfo/scikit-learn > > _______________________________________________ > scikit-learn mailing list > scikit-learn@python.org > https://mail.python.org/mailman/listinfo/scikit-learn >
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
