No clue what is causing this. Maybe some limit in the implementation of flufl.lock?
The reason we use file lock is to prevent other jobs, which potentially runs the same code and hence generate the same forms, to interfere with the present run. There is no need to lock for inter process write in a single run as it is only the first process that writes. A naive question? Do we really need file lock for reading an already compiled module? This is by far the most usage of file lock we have. See: Line 113 in cache.py and line 549 in build.py. Such lock will be grabbed for each process and race conditions in creating and releasing these locks might be a problem. But isn't it just when we write that we need a file lock? If so we should be able to remove these locks and only keep the one (Line: 177 in build.py) which is active during copying a compiled module from /tmp/somewhere to instant cache. This is also just done for one process in a run and should be much nicer for the file locking mechanism to handle. Johan On Fri, Mar 28, 2014 at 1:41 PM, Chris Richardson <[email protected]>wrote: > > I am hitting a problem with locking in instant when the processor count > gets high. I > am OK with 96 cores, but with 384, it is giving me an error every time > (see below). > As you can see, I have installed 'flufl'. > Any ideas? > > Chris > > > File "/work/fenics-dev/lib/python2.7/site-packages/ffc/jitcompiler.py", > line 75, in jit > return jit_element(ufl_object, parameters) > File "/work/fenics-dev/lib/python2.7/site-packages/ffc/jitcompiler.py", > line 186, in jit_element > compiled_form, module, form_data, prefix = jit_form(form, parameters) > File "/work/fenics-dev/lib/python2.7/site-packages/ffc/jitcompiler.py", > line 113, in jit_form > module = instant.import_module(module_name, cache_dir=cache_dir) > File "/work/fenics-dev/lib/python2.7/site-packages/instant/cache.py", > line 156, in import_module > return check_disk_cache(modulename, cache_dir, moduleids) > File "/work/fenics-dev/lib/python2.7/site-packages/instant/cache.py", > line 126, in check_disk_cache > release_lock(lock) > File "/work/fenics-dev/lib/python2.7/site-packages/instant/locking.py", > line 63, in release_lock > lock.unlock() > File "/work/fenics-dev/lib/python2.7/site-packages/flufl.lock-2. > 2.1-py2.7.egg/flufl/lock/_lockfile.py", line 293, in unlock > raise NotLockedError('Already unlocked') > flufl.lock._lockfile.NotLockedError: Already unlocked > > _______________________________________________ > fenics mailing list > [email protected] > http://fenicsproject.org/mailman/listinfo/fenics >
_______________________________________________ fenics mailing list [email protected] http://fenicsproject.org/mailman/listinfo/fenics
