Loky is the default backend in joblib master. Once joblib will be released it will be vendor in scikit-learn. So there a good hope to get it in the next release of scikit-learn and even available in master quite soon. ogrisel should be able to confirm. 

FWIW, a nice blog post of Tom Augspurger regarding nested parallelism‎ done during the last joblib sprint. 

‎http://tomaugspurger.github.io/distributed-joblib.html

Guillaume Lemaitre 
INRIA Saclay Ile-de-France / Equipe PARIETAL
guillaume.lemai...@inria.fr - https://glemaitre.github.io/
From: Beaugnon Anael
Sent: Tuesday, 20 March 2018 17:32
To: scikit-learn@python.org
Reply To: Scikit-learn mailing list
Subject: [scikit-learn] scikit-learn with celery (joblib issue)

Dear all,

When I run scikit-learn (version 0.19) into celery tasks, I cannot use scikit-learn multi-threading. The joblib library raises the following warning :

UserWarning: Multiprocessing-backend parallel loops cannot be nested, setting n_jobs=1

I attended a presentation about the Loky project at PyParis in June 2017. It was presented as a solution to this issue.

Is there any plan in the near future to solve this issue directly in scikit-learn ?

Cheers,
Anaël Beaugnon
_______________________________________________
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn

Reply via email to