Hi, Just to let you know, it is basically useless to grid-search over the n_estimators parameter in your forests. The higher, the better. However, you might try to tune min_sample_split (from 1 to n_features). It is one of the few parameters that will actually lead to any improvement in terms of accuracy.
Best, Gilles On 16 January 2013 19:54, Ronnie Ghose <[email protected]> wrote: > well items is actually neverused so that is irrelevant. > > gridEngines is: > > GRID_SEARCH={ > > #'Poly SVM':{'eng':svm.SVR(), > # 'params':{'kernel':('poly','rbf'),'degree':[2,4,6,8,10], > # 'C':[.1,1,5,10,50,100,500,1000]}}, > > 'Poly SVM':{'eng':svm.SVR(cache_size=svm_cs,max_iter=svm_maxiter), > > 'params':{'kernel':('poly','rbf'),'degree':[2,5,8,10,15,20,30,40], > 'C':[.1,1,5,10,20,40,80,160,320]}}, > > > 'RandomForestRegressor':{'eng':ens.RandomForestRegressor(), > > 'params':{'n_estimators':[10,20,30,40,50,100,150,200,350,500,750,1000,2000,3000]}}, > 'ExtraForestRegressor':{'eng':ens.ExtraTreesRegressor(), > > 'params':{'n_estimators':[10,20,30,40,50,100,200,500,750,1000,2000,3000],}}, > 'LogisticRegression':{'eng':lm.LogisticRegression(), > 'params':{ > 'C':[.01,.1,10,20,30,40,50,100,200], > }}, > > 'DecisionTrees':{'eng':tr.DecisionTreeRegressor(), > 'params':{ > 'min_samples_split':[2,3,4], > 'min_samples_leaf':[1,2,3,4], > 'min_density':[.05,.1,.2,.3,.4,.5], > } > }, > # "Stoch. Grad. Regressor" :{'eng':lm.SGDRegressor(), > # 'params':{ > # 'alpha':[.0001,.001], > # }}, > } > > > it runs GridSearchCV with the engine and the params passed. > > that's all that is 'mine' really, so I'm not sure how it is having what > seems to be race conditions with itself? > > > On Wed, Jan 16, 2013 at 1:49 PM, Andrew Winterman <[email protected]> > wrote: >> >> could you include definitions of some of those global variables that are >> used in the function? >> >> On Wed, Jan 16, 2013 at 9:02 AM, Ronnie Ghose <[email protected]> >> wrote: >>> >>> Anyway I can get around the following? >>> >>> I seem to get this Pool Closed error frequently: >>> >>> >>> sghose@adapt-ghose:~$ python -m ADAPT.engine.engine >>> num samples: 417 >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> [Parallel] Pool seems closed >>> Traceback (most recent call last): >>> File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main >>> "__main__", fname, loader, pkg_name) >>> File "/usr/lib/python2.7/runpy.py", line 72, in _run_code >>> exec code in run_globals >>> File "/home/sghose/ADAPT/engine/engine.py", line 86, in <module> >>> main() >>> File "/home/sghose/ADAPT/engine/engine.py", line 81, in main >>> dns() >>> File "/home/sghose/ADAPT/engine/engine.py", line 78, in dns >>> print m.average_score(10) >>> File "ADAPT/genclf/model.py", line 275, in average_score >>> _ = self.asynch_cv(gridsearch=True) >>> File "ADAPT/genclf/model.py", line 246, in asynch_cv >>> _ = mapfunc(items) >>> File "ADAPT/genclf/model.py", line 228, in mapfunc >>> self._gridENGINES[k]=eng.fit(X_train,Y_train) >>> File "/usr/local/lib/python2.7/dist-packages/sklearn/grid_search.py", >>> line 372, in fit >>> for clf_params in grid for train, test in cv) >>> File >>> "/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/parallel.py", >>> line 513, in __call__ >>> for function, args, kwargs in iterable: >>> ValueError: generator already executing >>> >>> >>> >>> that line is >>> >>> >>> >>> try: >>> svm_maxiter >>> except NameError: >>> _defined_svm_maxiter=False >>> #ks - keys >>> @catchwarnings(_defined_svm_maxiter) >>> def mapfunc(items): >>> if not gridsearch: >>> a = parmap(fit_and_score,items) >>> sleep(1) >>> elif gridsearch: >>> # gridsearch parallelizes itself ~ as in the >>> # library already does that >>> for k,eng in self._gridENGINES.items(): >>> vprint('Currently CVing' + " "+ str(k)) >>> self._gridENGINES[k]=eng.fit(X_train,Y_train) >>> else: >>> raise RuntimeError,"I have no clue how you are here. It's >>> a\ >>> binary comparison" >>> >>> >>> >>> >>> >>> self._gridENGINES is as expected a dictionary of names and classifiers. >>> I'm running this in serial (gridsearch == 1), so why is joblib having this >>> issue? >>> >>> Thank you, >>> Shomiron Ghose >>> >>> >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery >>> and much more. Keep your Java skills current with LearnJavaNow - >>> 200+ hours of step-by-step video tutorials by Java experts. >>> SALE $49.99 this month only -- learn more at: >>> http://p.sf.net/sfu/learnmore_122612 >>> _______________________________________________ >>> Scikit-learn-general mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>> >> >> >> >> -- >> Andrew Winterman >> 714 362 6823 >> >> >> ------------------------------------------------------------------------------ >> Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery >> and much more. Keep your Java skills current with LearnJavaNow - >> 200+ hours of step-by-step video tutorials by Java experts. >> SALE $49.99 this month only -- learn more at: >> http://p.sf.net/sfu/learnmore_122612 >> _______________________________________________ >> Scikit-learn-general mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> > > > ------------------------------------------------------------------------------ > Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery > and much more. Keep your Java skills current with LearnJavaNow - > 200+ hours of step-by-step video tutorials by Java experts. > SALE $49.99 this month only -- learn more at: > http://p.sf.net/sfu/learnmore_122612 > _______________________________________________ > Scikit-learn-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > ------------------------------------------------------------------------------ Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery and much more. Keep your Java skills current with LearnJavaNow - 200+ hours of step-by-step video tutorials by Java experts. SALE $49.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122612 _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
