scipy allows to perform the friedman test. Orange has the tool to drawn the critical distance diagram.
And you can easily compute the critical distance using stats model: from statsmodels.stats.libqsturng import qsturng q_alpha = qsturng(1 - alpha, n_methods, np.inf) / np.sqrt(2) cd = q_alpha * np.sqrt(n_methods * (n_methods + 1) / (6 * n_datasets)) Best regards, Arnaud > On 29 Oct 2015, at 17:48, Andreas Mueller <t3k...@gmail.com> wrote: > > Sorry, don't know of a package. But it might be interesting for sklearn? > > So that's a Nemenyi test? > https://en.wikipedia.org/wiki/Nemenyi_test > <https://en.wikipedia.org/wiki/Nemenyi_test> > > I never heard of that but it sounds interesting. > > It seems a bit hard to interpret, though. > Also: does the diagram punt if the initial multiple comparison null > hypothesis can not be rejected? > > Only looking at ranks also seems to discard a lot of information.... > > Here is the reference (I think): > http://www.jmlr.org/papers/v7/demsar06a.html > <http://www.jmlr.org/papers/v7/demsar06a.html> > > Seems pretty well-cited. > > > > On 10/29/2015 09:28 AM, Dayvid Victor wrote: >> Hi, >> >> Do you guys know any tool to generate CDdiagram - in order to evaluate the >> difference of performance of sklearn classifiers? >> >> <Mail Attachment.png> >> http://theoval.cmp.uea.ac.uk/matlab/critdiff/cd1.png >> <http://theoval.cmp.uea.ac.uk/matlab/critdiff/cd1.png> >> >> There is a R package called performanceEstimation which has >> a CDdiagram implementation, but it uses an specific R object >> [it is not as simple as it should be to connect using rpy2]. >> >> >> Thanks, >> -- >> Dayvid Victor R. de Oliveira >> PhD Candidate in Computer Science at Federal University of Pernambuco (UFPE) >> MSc in Computer Science at Federal University of Pernambuco (UFPE) >> BSc in Computer Engineering - Federal University of Pernambuco (UFPE) >> >> >> ------------------------------------------------------------------------------ >> >> >> _______________________________________________ >> Scikit-learn-general mailing list >> Scikit-learn-general@lists.sourceforge.net >> <mailto:Scikit-learn-general@lists.sourceforge.net> >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> <https://lists.sourceforge.net/lists/listinfo/scikit-learn-general> > > ------------------------------------------------------------------------------ > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > <mailto:Scikit-learn-general@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > <https://lists.sourceforge.net/lists/listinfo/scikit-learn-general>
------------------------------------------------------------------------------
_______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
