Hi, Andreas Enge <andr...@enge.fr> skribis:
> This is a wishlist bug, but it is important for architectures where we > are currently short on build power, and where this issue can stall builds > and waste an arbitrary amount of build power. > > Cuirass should sort builds and only offload derivations for which all > inputs are available. Cuirass has two build backends: talking to the build daemon, and using the ZeroMQ-based “remote worker” protocol. In practice we use the latter, which fixes scalability issues with the former. The worker protocol implements work stealing: workers periodically send messages to “remote server” asking for work. Said server replies with a derivation for one of the systems the worker supports; that derivation is chosen among the “builds” whose dependencies have all been successfully built. And here’s the trick: the server is doing the right thing, but it has a partial view. Namely, the server sees “builds” rather than “derivations”. “Builds” are the things explicitly declared in the jobset. If you have a declared build for GCC, but no build for MPC and MPFR, then the server will consider that GCC has zero non-built dependencies, even though MPFR and MPC may still need to be built. Having ‘remote-worker’ operate on derivations rather than builds would address this impedance mismatch, though there are complications (there are bits of the database schema that amalgamate build/derivation). Food for thought! Ludo’.