Just specific to Nemenyi and Dunns tests, I didn't check the other parts of this discussion.
They were discussed here https://github.com/statsmodels/statsmodels/issues/852 (starting after a few comments) with code available in gists but not yet in a PR for statsmodels Josef On Sat, Oct 31, 2015 at 2:42 PM, Dayvid Victor <victor.d...@gmail.com> wrote: > Thank you all, > > The Orange will work just fine: > http://docs.orange.biolab.si/3/modules/evaluation.cd.html > > Andreas, I'm not sure if this is the kind of thing that needed > to be in sklearn, maybe scipy (stats) and matplotlib (graph). > > Thanks, > > On Thu, Oct 29, 2015 at 1:55 PM, Arnaud Joly <a.j...@ulg.ac.be> wrote: > >> scipy allows to perform the friedman test. >> Orange has the tool to drawn the critical distance diagram. >> >> And you can easily compute the critical distance using stats model: >> >> from statsmodels.stats.libqsturng import qsturng >> q_alpha = qsturng(1 - alpha, n_methods, np.inf) / np.sqrt(2) >> cd = q_alpha * np.sqrt(n_methods * (n_methods + 1) / (6 * n_datasets)) >> >> >> Best regards, >> Arnaud >> >> >> On 29 Oct 2015, at 17:48, Andreas Mueller <t3k...@gmail.com> wrote: >> >> Sorry, don't know of a package. But it might be interesting for sklearn? >> >> So that's a Nemenyi test? >> https://en.wikipedia.org/wiki/Nemenyi_test >> >> I never heard of that but it sounds interesting. >> >> It seems a bit hard to interpret, though. >> Also: does the diagram punt if the initial multiple comparison null >> hypothesis can not be rejected? >> >> Only looking at ranks also seems to discard a lot of information.... >> >> Here is the reference (I think): >> http://www.jmlr.org/papers/v7/demsar06a.html >> >> Seems pretty well-cited. >> >> >> >> On 10/29/2015 09:28 AM, Dayvid Victor wrote: >> >> Hi, >> >> Do you guys know any tool to generate CDdiagram - in order to evaluate >> the difference of performance of sklearn classifiers? >> >> <Mail Attachment.png> >> http://theoval.cmp.uea.ac.uk/matlab/critdiff/cd1.png >> >> There is a R package called performanceEstimation which has >> a CDdiagram implementation, but it uses an specific R object >> [it is not as simple as it should be to connect using rpy2]. >> >> >> Thanks, >> -- >> *Dayvid Victor R. de Oliveira* >> PhD Candidate in Computer Science at Federal University of Pernambuco >> (UFPE) >> MSc in Computer Science at Federal University of Pernambuco (UFPE) >> BSc in Computer Engineering - Federal University of Pernambuco (UFPE) >> >> >> ------------------------------------------------------------------------------ >> >> >> >> _______________________________________________ >> Scikit-learn-general mailing >> listScikit-learn-general@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> >> >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Scikit-learn-general mailing list >> Scikit-learn-general@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> >> >> >> >> ------------------------------------------------------------------------------ >> >> _______________________________________________ >> Scikit-learn-general mailing list >> Scikit-learn-general@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> >> > > > -- > *Dayvid Victor R. de Oliveira* > PhD Candidate in Computer Science at Federal University of Pernambuco > (UFPE) > MSc in Computer Science at Federal University of Pernambuco (UFPE) > BSc in Computer Engineering - Federal University of Pernambuco (UFPE) > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > >
------------------------------------------------------------------------------
_______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
