On Tue, Sep 15, 2026 at 09:49:54AM -0700, Kees Cook wrote: > On Tue, Sep 15, 2026 at 11:44:01AM +0100, Lorenzo Stoakes (ARM) wrote: > > On Mon, Sep 14, 2026 at 11:00:27AM -0700, Kees Cook wrote: > > > On Mon, Sep 14, 2026 at 10:22:13AM +0100, Lorenzo Stoakes (ARM) wrote: > > > > Module finalisation on allmodconfig builds consists of a large number of > > > > very short-lived jobs, and the make job dispatcher cannot possibly > > > > dispatch > > > > jobs fast enough. > > > > > > This "cannot possibly" sounds like a weird LLM language intensifier. I'd > > > > Again that's my writing actually, probably best to stop assuming LLM now ;) > > Apologies; whatever the opposite of the Turing Test is, I have failed. > :P The AIs are winning: humans are suspicious of each other now. :) > > > > rather a concrete description of the problem, not this kind of > > > (redundant?) vagueness. > > > > In the very next paragraph I say: > > > > For allmodconfig x86-64 this can be on the order of ~22,000 jobs of > > a few milliseconds in duration each. > > > > You did also say that the commit messages were over-long, so there's a > > trade-off here :) > > Right, sorry, I meant I understand the need to collapse the jobs > together, I was just trying to understand where the "128" came from. > > > But in more detail, allmodconfig tree (~11k modules on x86-64), best of 2, > > make modules with *.ko *.mod.o deleted: > > > > modules per chunk instances wall > > ----------------- --------- ------ > > 1 11171 10.22s > > 2 5586 6.72s > > 4 2793 4.97s > > 8 1397 4.26s > > 16 699 4.05s > > 32 350 4.05s > > 64 175 4.07s > > 128 88 4.05s > > 256 44 4.03s > > 512 22 4.18s > > > > which I think makes things clearer. > > > > So realistically 32 - 256 is the right sort of range. You also have to take > > into account the fact that you might be building fewer modules. > > This answers it, thank you! > > > The argument for 128 is that this is the mid-point of where the graph > > flattens off for a larger number of modules. > > > > For a smaller number, you're going to have a single dispatch or less and > > the delta won't be that much anyway. > > Understood. I guess I'm trying to better understand the chosen balance > point. Does machine CPU count matter at all for this? i.e. "16" is the > first one to hit the 4 second line, can that be "better" than 128 for > smaller system (or fewer modules)? I'm trying to consider CIs and such > that are bounded to 8 or fewer threads, etc.
I would say no, because the parallel aspect is coming from the jobserver in each chunk, not from the chunk count, so the CPUs are busy whatever the chunk size. And the size is balancing the start-up against the cost that grows with the number of targets in each, so there's an equilibrium that's reached and past that you don't really gain anything, and that doesn't change with CPU count. > > > So it's very much a sensibly derived empirical value. > > > > I'll update the commit message to give this rationale there. > > Thanks! NP, Already done with all the other changes queued up for v3 :) I will give time for more feedback before respinning though! > > -Kees > > -- > Kees Cook -- Cheers, Lorenzo
