Yes, we really just need to lock the cache and other related files like _sanity when they are written. After the cache is fully populated, none of those should be written to.
The tricky thing with that locking stuff is the corner cases, like a lockfile getting left if your machine powers off in the middle of an operation. So you need to handle all that. git for example will now and then tell me "previous lockfile on the rebase directory" which is a similar situation... You really just want to know if another instance of your app is actually running now, and allow only one. Things like DBUS support that on linux, but i am not aware of a cross-platform way. - Alon On Fri, Jan 31, 2014 at 8:27 AM, Jukka Jylänki <[email protected]> wrote: > I've run into this very often on the bots, when switching between versions > and trying something in ssh shell while the bots are running. I think that > in the current codebase, it should be safe to build multiple if you first > make sure that .emscripten_cache is fully built and up-to-date? After that, > all emscripten compiler instances will only read that directory, and never > write into it. (Is that correct?) > > Another thing btw I was thinking some weeks ago would be to make the cache > have versioned subdirectories according to emcc compiler versions. This > would help the users if they're using multiple compiler versions, or > switching between them e.g. when bisecting. But then of course it would > require manually cleaning up the cache when upgrading to a newer compiler > if one doesn't want to use old versions.. :/ > > I guess the usual solution is to use a dummy touched file as a lock to > signal that a read or write operation is going on in the directory, and > clean it up when the process finishes. I am not sure either what the exact > cross-platform procedure is on all OSes to guarantee that the scheme will > be atomic, but it surely is doable. > > > 2014-01-30 Alon Zakai <[email protected]>: > > We need to do proper locking between processes when generating libcxx and >> other dependencies. I'm not sure how to do that properly in a >> cross-platform manner though, which is why I didn't do it yet. If someone >> could help out there that would be much appreciated. >> >> - Alon >> >> >> >> On Mon, Jan 27, 2014 at 1:57 AM, wolfviking0 <[email protected]>wrote: >> >>> Hi, >>> >>> I have a python script who build my extern library in same time, for >>> increase a little more the build time. >>> >>> Unfortunately, seems with fastcomp is not possible, when I do that I >>> have an error like : >>> >>> Traceback (most recent call last): >>> File "/Users/aliot/Desktop/webcl/webcl-translator/emscripten/emcc", >>> line 1767, in <module> >>> libfile = shared.Cache.get(name, create) >>> File >>> "/Users/aliot/Desktop/webcl/webcl-translator/emscripten/tools/cache.py", >>> line 36, in get >>> shutil.copyfile(creator(), cachename) >>> File >>> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", >>> line 83, in copyfile >>> with open(dst, 'wb') as fdst: >>> IOError: [Errno 2] No such file or directory: >>> '/Users/aliot/.emscripten_cache/libcxx.bc' >>> >>> I know Emscripten is already using all the core during some pass, but >>> could be useful to call two emscripten at the same time. >>> >>> Tony >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
