Andy Wingo <[email protected]> skribis: > On Tue 19 Dec 2017 10:05, [email protected] (Ludovic Courtès) writes: > >> I’ve been looking for ways to make evaluations (computing the derivation >> of every package for every supported architecture, like Hydra and >> Cuirass do) faster. >> >> I measured evaluations with: >> >> rm -rf /tmp/cache hydra-jobs.scm >> mkdir /tmp/cache >> XDG_CACHE_HOME=/tmp/cache make hydra-jobs.scm >> >> Currently build-aux/hydra/gnu-system.scm, which performs the evaluation >> work, turns on auto-compile such that every Guix module gets compiled. >> >> I measured with 2.2.3 the impact of turning off auto-compilation for >> everything but the core modules (meaning that package modules get >> interpreted instead) and surprisingly, this is slower than compiling >> everything: >> >> * 2.2.3, auto-compile everything minus (system base compile) & co. >> >> 1362.79user 3.35system 20:59.40elapsed 108%CPU (0avgtext+0avgdata >> 1201444maxresident)k >> 0inputs+203560outputs (0major+285018minor)pagefaults 0swaps >> >> * 2.2.3, auto-compile (guix packages) only >> >> 2462.05user 3.69system 39:26.05elapsed 104%CPU (0avgtext+0avgdata >> 2121532maxresident)k >> 128inputs+36568outputs (0major+242281minor)pagefaults 0swaps >> >> * 2.2.3, auto-compile ((guix packages) (guix build-system gnu) (guix >> download)) >> >> 2364.22user 3.21system 37:44.45elapsed 104%CPU (0avgtext+0avgdata >> 2061496maxresident)k >> 256inputs+41800outputs (0major+236514minor)pagefaults 0swaps >> >> I guess the extra source properties that are retained when evaluating >> account for part the space and time overhead, but I’m not sure this >> explains everything. >> >> Andy, what do you think of this? > > I have literally no idea :) This isn't really enough information for me > to understand things. I guess my biggest question would be, why are you > using the auto-compile infrastructure? Why wouldn't you compile the > things you want to compile and otherwise turn off auto-compilation?
Evaluations run code from (guix …) as well as everything in (gnu packages …). At every evaluation, we have to start afresh: we cannot just reuse .go files from a previous evaluation, for instance because the ABI might have changed. So, to be on the safe side, we re-compile (or re-evaluate) all these files. Auto-compilation is the simplest way to do that. I thought we could improve performance by compiling only the hot code and interpreting (gnu packages …), but the measurements above suggest it’s not the case. I hope that clarifies the situation. :-) As an aside, I also feel (though I haven’t tried) that if our packages were inert sexps, reading them and turning them into <package> records would be more efficient than it is to compile or interpret our package modules. This is understandable in a way (the sexps would be a dumb language, so no wonder it can be implemented efficiently), but still a bit puzzling. Ludo’.
