thanks Olivier, it fails on both Linux and Mac OS X here the output of sudo gdb python <pid> from a Linux machine http://pastebin.com/4sU5RCtg
and here of (gdb) bt http://pastebin.com/bt8ULLwn I am sorry, I have no clue on how to debug this properly... Best, Matthias On 2/27/12 8:34 AM, [email protected] wrote: > 2012/2/23 Matthias Ekman <[email protected]>: >> > Hi, >> > >> > I only recently started using sklearn and it's an impressive and well >> > documented library. Thanks! >> > >> > I run into some strange behavior while using the function >> > 'permutation_test_score'. >> > >> > When using permutation_test_score with n_permutations = 50, everything >> > looks alright >> > >> > In [4]: cv_scores, permutation_scores, pval = >> > permutation_test_score(clf, X, Y, zero_one_score, cv=cv, >> > n_permutations=50, n_jobs=4,verbose=1, random_state=0) >> > [Parallel(n_jobs=4)]: Done 1 out of 50 | elapsed: 0.0s remaining: >> > 1.5s >> > [Parallel(n_jobs=4)]: Done 50 out of 50 | elapsed: 0.2s finished >> > >> > However, when using the exact same data, but with n_permutations = 200 >> > I don't get a result and this runs forever. >> > >> > In [6]: cv_scores, permutation_scores, pval = >> > permutation_test_score(clf, X, Y, zero_one_score, cv=cv, >> > n_permutations=200, n_jobs=4,verbose=1, random_state=0) >> > [Parallel(n_jobs=4)]: Done 1 out of 54 | elapsed: 0.0s >> > remaining: 2.0s #<-- stops here >> > >> > My code is here:https://gist.github.com/1884451 and the data to >> > reproduce the problem is here: >> > http://dl.dropbox.com/u/38470419/wired_data.dat # sample x feature matrix >> > http://dl.dropbox.com/u/38470419/Y.txt # binary labels >> > >> > I am using sklearn .10 and joblib 0.6.1. >> > >> > I am not sure if that can be caused by some irregularities in my data. >> > I would be grateful for every pointer. > Strange, it might be related to a problem Vlad is investigating on Mac > OS X Lion: > > https://github.com/scikit-learn/scikit-learn/issues/636 > > Which platform / OS are you using? > >> > As a related question, as far as I can see permutation_test_score does >> > not assure permuted labels, right? Couldn't >> > >> > pvalue = (np.sum(permutation_scores>= score) + 1.0) / (n_permutations + 1) >> > >> > be in some cases too conservative? I would count +1_only_ when the >> > true labels are_not_ included in the permutation set. > No idea, maybe @agramfort or @GaelVaroquaux have an opinion? > > -- Olivier http://twitter.com/ogrisel - http://github.com/ogrisel ------------------------------------------------------------------------------ Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
