thanks for this brian. very informative document.
cheers,
satra
On Wed, Nov 16, 2011 at 7:35 AM, Brian Holt <[email protected]> wrote:
> For those who might be interested, there was a very interesting tutorial
> on decision trees[1] presented by Antonio Criminisi and Jamie Shotton (the
> guys at MSR behind the human pose estimation algorithm for the Kinect) at
> ICCV last week.
>
> Their approach differs from the implementation that exists in scikit-learn
> in a few ways.
> Firstly, they introduce randomness by selecting at each node a subset of
> the total dimensions, and only attempt to find the most optimal split on
> this subset. They go even further by restricting the number of possible
> thresholds to a fixed number, further increasing the speed of training and
> making it more random. Because of this, their ensembles do not need to be
> trained on varying datasets (like sampling with replacement). It is also
> possible to use this framework to train trees on datasets that would be
> enormous if held in memory, but the features of which are now only computed
> when required.
>
> Secondly, they make use of splits perpendicular to dimensions, linear
> combination splits and even non-linear splits (although I have yet to
> determine exactly how this works).
>
> Thirdly, they make use of various predictor functions at leaf nodes. One
> simple difference is that scikit-learn uses a constant value (the mean) to
> approximate a region of the space for regression, whereas a piecewise
> linear model might (and often does) perform better.
>
> Finally, they show that the same unified framework can be used for
> classification, regression, density estimation (using mixtures of
> gaussians), manifold learning and active learning.
>
> This is by no means a complete summary of the work (I'm still reading
> through it), but I would be interested in implementing some of these ideas
> in scikit-learn.
>
> [1]
> http://research.microsoft.com/pubs/155552/decisionForests_MSR_TR_2011_114.pdf
>
>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
