Thank you all, The Orange will work just fine: http://docs.orange.biolab.si/3/modules/evaluation.cd.html
Andreas, I'm not sure if this is the kind of thing that needed to be in sklearn, maybe scipy (stats) and matplotlib (graph). Thanks, On Thu, Oct 29, 2015 at 1:55 PM, Arnaud Joly <a.j...@ulg.ac.be> wrote: > scipy allows to perform the friedman test. > Orange has the tool to drawn the critical distance diagram. > > And you can easily compute the critical distance using stats model: > > from statsmodels.stats.libqsturng import qsturng > q_alpha = qsturng(1 - alpha, n_methods, np.inf) / np.sqrt(2) > cd = q_alpha * np.sqrt(n_methods * (n_methods + 1) / (6 * n_datasets)) > > > Best regards, > Arnaud > > > On 29 Oct 2015, at 17:48, Andreas Mueller <t3k...@gmail.com> wrote: > > Sorry, don't know of a package. But it might be interesting for sklearn? > > So that's a Nemenyi test? > https://en.wikipedia.org/wiki/Nemenyi_test > > I never heard of that but it sounds interesting. > > It seems a bit hard to interpret, though. > Also: does the diagram punt if the initial multiple comparison null > hypothesis can not be rejected? > > Only looking at ranks also seems to discard a lot of information.... > > Here is the reference (I think): > http://www.jmlr.org/papers/v7/demsar06a.html > > Seems pretty well-cited. > > > > On 10/29/2015 09:28 AM, Dayvid Victor wrote: > > Hi, > > Do you guys know any tool to generate CDdiagram - in order to evaluate the > difference of performance of sklearn classifiers? > > <Mail Attachment.png> > http://theoval.cmp.uea.ac.uk/matlab/critdiff/cd1.png > > There is a R package called performanceEstimation which has > a CDdiagram implementation, but it uses an specific R object > [it is not as simple as it should be to connect using rpy2]. > > > Thanks, > -- > *Dayvid Victor R. de Oliveira* > PhD Candidate in Computer Science at Federal University of Pernambuco > (UFPE) > MSc in Computer Science at Federal University of Pernambuco (UFPE) > BSc in Computer Engineering - Federal University of Pernambuco (UFPE) > > > ------------------------------------------------------------------------------ > > > > _______________________________________________ > Scikit-learn-general mailing > listScikit-learn-general@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/scikit-learn-general > > > > ------------------------------------------------------------------------------ > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > > -- *Dayvid Victor R. de Oliveira* PhD Candidate in Computer Science at Federal University of Pernambuco (UFPE) MSc in Computer Science at Federal University of Pernambuco (UFPE) BSc in Computer Engineering - Federal University of Pernambuco (UFPE)
------------------------------------------------------------------------------
_______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
