For Test/Main, I have the user's ~/.bash_profile set $PYTHON_EGG_CACHE on a
per-node basis. This could also be done per-node and per-pty to ensure
uniqueness per job.
On Sep 18, 2012, at 11:24 AM, James Taylor wrote:
> Interesting. If I'm reading this correctly the problem is happening
> inside pkg_resources? (galaxy.eggs unzips eggs, but I think it does so
> on install [fetch_eggs] time not run time which would avoid this). If
> so this would seem to be a locking bug in pkg_resources. Dannon, we
> could put a guard around the imports in extract_dataset_part.py as an
> (overly aggressive and hacky) fix.
> -- jt
> On Tue, Sep 18, 2012 at 10:37 AM, Jorrit Boekel
> <jorrit.boe...@scilifelab.se> wrote:
>> - which lead to unzipping .so libraries from python eggs into the nodes'
>> - this runs into lib/pkg_resources.py and its _bypass_ensure_directory
>> method that creates the temporary dir for the egg unzip
>> - since there are 8 processes on the node, sometimes this method tries to
>> mkdir a directory that was just made by the previous process after the
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at: