[email protected] (Taylan Ulrich "Bayırlı/Kammer") skribis:
> [email protected] (Ludovic Courtès) writes: > >> Mark H Weaver <[email protected]> skribis: >> >>> I haven't yet investigated, but my strong suspicion is that this is due >>> to the fact that Guile's module system is not thread safe. More >>> specifically, when a new module is created, it mutates the global >>> directory of modules in a way that is not thread safe. >>> >>> New modules are created by 'compile-file', both for the module being >>> compiled and for any imported modules that haven't been previously >>> loaded. Unfortunately, this means that this approach of compiling files >>> in multiple threads within a single guile process is not safe. There >>> are likely to be random crashes and corruptions. >> >> Right. This is one of the concerns I raised before I forgot again. ;-) >> >> https://lists.gnu.org/archive/html/guix-devel/2015-11/msg00359.html >> >> Taylan writes: >> >>> The latest version of the patch loads all needed modules first, in the >>> main thread, and then compiles the files corresponding to the modules in >>> parallel. Does that still lead to mutation in the directory of modules >>> in the parallelized segment of the code? >> >> It seems to be safe, but we’re treading in a risky zone. >> >> We’re pushing ‘compile-file’ and related code to its limits. It would >> be great to address these issues in Guile itself. > > If we don't call compile-file in parallel but still do everything in one > process, we will still be cutting down the time significantly. > > For instance, currently the load phase and compile phase take about one > minute each on my machine with 4 cores. Making it non-parallel would > mean the compile time takes 4x longer, i.e. 4 min., which brings us to 5 > minutes total. The original was around 20. On devices with fewer > cores, the parallel compile is even less important. > > So in the worst case we can just turn that par-for-each into a for-each > and still get benefits. Right. Mark, WDYT? I would say: go for it, and let’s switch back to ‘for-each’ if/when we have evidence of things going wrong. I think that if things go wrong, that’ll be a hard failure (segfault or some random run-time error) and not a silent hard-to-detect corruption. Ludo’.
