well items is actually neverused so that is irrelevant.
gridEngines is:
GRID_SEARCH={
#'Poly SVM':{'eng':svm.SVR(),
# 'params':{'kernel':('poly','rbf'),'degree':[2,4,6,8,10],
# 'C':[.1,1,5,10,50,100,500,1000]}},
'Poly SVM':{'eng':svm.SVR(cache_size=svm_cs,max_iter=svm_maxiter),
'params':{'kernel':('poly','rbf'),'degree':[2,5,8,10,15,20,30,40],
'C':[.1,1,5,10,20,40,80,160,320]}},
'RandomForestRegressor':{'eng':ens.RandomForestRegressor(),
'params':{'n_estimators':[10,20,30,40,50,100,150,200,350,500,750,1000,2000,3000]}},
'ExtraForestRegressor':{'eng':ens.ExtraTreesRegressor(),
'params':{'n_estimators':[10,20,30,40,50,100,200,500,750,1000,2000,3000],}},
'LogisticRegression':{'eng':lm.LogisticRegression(),
'params':{
'C':[.01,.1,10,20,30,40,50,100,200],
}},
'DecisionTrees':{'eng':tr.DecisionTreeRegressor(),
'params':{
'min_samples_split':[2,3,4],
'min_samples_leaf':[1,2,3,4],
'min_density':[.05,.1,.2,.3,.4,.5],
}
},
# "Stoch. Grad. Regressor" :{'eng':lm.SGDRegressor(),
# 'params':{
# 'alpha':[.0001,.001],
# }},
}
it runs GridSearchCV with the engine and the params passed.
that's all that is 'mine' really, so I'm not sure how it is having what
seems to be race conditions with itself?
On Wed, Jan 16, 2013 at 1:49 PM, Andrew Winterman
<[email protected]>wrote:
> could you include definitions of some of those global variables that are
> used in the function?
>
> On Wed, Jan 16, 2013 at 9:02 AM, Ronnie Ghose <[email protected]>wrote:
>
>> Anyway I can get around the following?
>>
>> I seem to get this Pool Closed error frequently:
>>
>>
>> sghose@adapt-ghose:~$ python -m ADAPT.engine.engine
>> num samples: 417
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> [Parallel] Pool seems closed
>> Traceback (most recent call last):
>> File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
>> "__main__", fname, loader, pkg_name)
>> File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
>> exec code in run_globals
>> File "/home/sghose/ADAPT/engine/engine.py", line 86, in <module>
>> main()
>> File "/home/sghose/ADAPT/engine/engine.py", line 81, in main
>> dns()
>> File "/home/sghose/ADAPT/engine/engine.py", line 78, in dns
>> print m.average_score(10)
>> File "ADAPT/genclf/model.py", line 275, in average_score
>> _ = self.asynch_cv(gridsearch=True)
>> File "ADAPT/genclf/model.py", line 246, in asynch_cv
>> _ = mapfunc(items)
>> File "ADAPT/genclf/model.py", line 228, in mapfunc
>> self._gridENGINES[k]=eng.fit(X_train,Y_train)
>> File "/usr/local/lib/python2.7/dist-packages/sklearn/grid_search.py",
>> line 372, in fit
>> for clf_params in grid for train, test in cv)
>> File
>> "/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/parallel.py",
>> line 513, in __call__
>> for function, args, kwargs in iterable:
>> ValueError: generator already executing
>>
>>
>>
>> that line is
>>
>>
>>
>> try:
>> svm_maxiter
>> except NameError:
>> _defined_svm_maxiter=False
>> #ks - keys
>> @catchwarnings(_defined_svm_maxiter)
>> def mapfunc(items):
>> if not gridsearch:
>> a = parmap(fit_and_score,items)
>> sleep(1)
>> elif gridsearch:
>> # gridsearch parallelizes itself ~ as in the
>> # library already does that
>> for k,eng in self._gridENGINES.items():
>> vprint('Currently CVing' + " "+ str(k))
>> self._gridENGINES[k]=eng.fit(X_train,Y_train)
>> else:
>> raise RuntimeError,"I have no clue how you are here. It's
>> a\
>> binary comparison"
>>
>>
>>
>>
>>
>> self._gridENGINES is as expected a dictionary of names and classifiers.
>> I'm running this in serial (gridsearch == 1), so why is joblib having this
>> issue?
>>
>> Thank you,
>> Shomiron Ghose
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery
>> and much more. Keep your Java skills current with LearnJavaNow -
>> 200+ hours of step-by-step video tutorials by Java experts.
>> SALE $49.99 this month only -- learn more at:
>> http://p.sf.net/sfu/learnmore_122612
>> _______________________________________________
>> Scikit-learn-general mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>>
>
>
> --
> Andrew Winterman
> 714 362 6823
>
>
> ------------------------------------------------------------------------------
> Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery
> and much more. Keep your Java skills current with LearnJavaNow -
> 200+ hours of step-by-step video tutorials by Java experts.
> SALE $49.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122612
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery
and much more. Keep your Java skills current with LearnJavaNow -
200+ hours of step-by-step video tutorials by Java experts.
SALE $49.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122612
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
