I agree. I just turned locking off and it now works fine. So there is really no need for it, once the modules are in the cache already.
Chris
On 28 Mar 2014 14:34, Johan Hake <[email protected]> wrote:
No clue what is causing this. Maybe some limit in the implementation of flufl.lock?The reason we use file lock is to prevent other jobs, which potentially runs the same code and hence generate the same forms, to interfere with the present run. There is no need to lock for inter process write in a single run as it is only the first process that writes.A naive question? Do we really need file lock for reading an already compiled module? This is by far the most usage of file lock we have. See: Line 113 in cache.py and line 549 in build.py. Such lock will be grabbed for each process and race conditions in creating and releasing these locks might be a problem. But isn't it just when we write that we need a file lock? If so we should be able to remove these locks and only keep the one (Line: 177 in build.py) which is active during copying a compiled module from /tmp/somewhere to instant cache. This is also just done for one process in a run and should be much nicer for the file locking mechanism to handle.JohanOn Fri, Mar 28, 2014 at 1:41 PM, Chris Richardson <[email protected]> wrote:
I am hitting a problem with locking in instant when the processor count gets high. I
am OK with 96 cores, but with 384, it is giving me an error every time (see below).
As you can see, I have installed 'flufl'.
Any ideas?
Chris
File "/work/fenics-dev/lib/python2.7/site-packages/ffc/jitcompiler.py", line 75, in jit
return jit_element(ufl_object, parameters)
File "/work/fenics-dev/lib/python2.7/site-packages/ffc/jitcompiler.py", line 186, in jit_element
compiled_form, module, form_data, prefix = jit_form(form, parameters)
File "/work/fenics-dev/lib/python2.7/site-packages/ffc/jitcompiler.py", line 113, in jit_form
module = instant.import_module(module_name, cache_dir=cache_dir)
File "/work/fenics-dev/lib/python2.7/site-packages/instant/cache.py", line 156, in import_module
return check_disk_cache(modulename, cache_dir, moduleids)
File "/work/fenics-dev/lib/python2.7/site-packages/instant/cache.py", line 126, in check_disk_cache
release_lock(lock)
File "/work/fenics-dev/lib/python2.7/site-packages/instant/locking.py", line 63, in release_lock
lock.unlock()
File "/work/fenics-dev/lib/python2.7/site-packages/flufl.lock-2.2.1-py2.7.egg/flufl/lock/_lockfile.py", line 293, in unlock
raise NotLockedError('Already unlocked')
flufl.lock._lockfile.NotLockedError: Already unlocked
_______________________________________________
fenics mailing list
[email protected]
http://fenicsproject.org/mailman/listinfo/fenics
_______________________________________________ fenics mailing list [email protected] http://fenicsproject.org/mailman/listinfo/fenics
