I use BUILDHISTORY_COMMIT_forcevariable = "1" PARALLEL_MAKE = "-j 11" BB_NUMBER_THREADS = "11" INHERIT += "rm_work" XZ_DEFAULTS = "--threads=8"
On Tue, Apr 13, 2021 at 6:15 PM Randy MacLeod <randy.macl...@windriver.com> wrote: > > On 2021-04-11 12:19 p.m., Alexander Kanavin wrote: > > make already has -l option for limiting new instances if load average is > > too high, so it's only natural to add a RAM limiter too. > > > > -l [N], --load-average[=N], --max-load[=N] > > Don't start multiple jobs unless load is > > below N. > > > > In any case, patches welcome :) > > During today's Yocto technical call (1), > we talked about approaches to limiting the system load and avoiding > swap and/or OOM events. Here's what (little!) i recall from the > discussion, 9 busy hours later. > > In the short run, instead of independently maintaining changes to > configurations to limit parallelism or xz memory usage, etc, we > could develop an optional common include file where such limits > are shared across the community. > > In the longer run, changes to how bitbake schedules work may be needed. > > Richard says that there was a make/build server idea and maybe even a > patch from a while ago. It may be in one of his poky-contrib branches. > I took a few minutes to look but nothing popped up. A set of keywords to > search for might help me find it. > > Someone mentioned that while ninja has not been open to accepting any > patches that would complicate and potentially slow down builds, there > is a fork of ninja calls 'samurai' that does seem to be open to some > improvements that we could benefit from. > > It was also suggested that there were existing defects in the YP BZ (2) > but I didn't find any earlier and it's too late in my day to start > looking now! If no one replies with a relevant BZ ID, I'll create one. > > I'm sure I missed some things that were mentioned but Trevor Woerner > sometimes takes notes so I'll check on them once / if they are sent out. > > ../Randy > > > 1) https://www.yoctoproject.org/public-virtual-meetings/ > > 2) https://bugzilla.yoctoproject.org/ > > > > > Alex > > > > On Sun, 11 Apr 2021 at 18:08, Gmane Admin <gley-yo...@m.gmane-mx.org > > <mailto:gley-yo...@m.gmane-mx.org>> wrote: > > > > Op 11-04-2021 om 17:55 schreef Alexander Kanavin: > > > On Sun, 11 Apr 2021 at 17:49, Gmane Admin > > <gley-yo...@m.gmane-mx.org <mailto:gley-yo...@m.gmane-mx.org> > > > <mailto:gley-yo...@m.gmane-mx.org > > <mailto:gley-yo...@m.gmane-mx.org>>> wrote: > > > > > > Yes, and make project doesn't care, because make is called > > with -j > > > 16 so > > > that is what it does. > > > > > > So here's my pitch: bitbake can stop processes spawned by > > make, because > > > it knows that it started make on 4 recipies, each with -j 16. The > > > individual makes don't know about each other. > > > > > > > > > And neither they should. They can simply abstain from spawning new > > > compilers if used RAM is, say, at 90% total. Then bitbake does > > not have > > > to get involved in babysitting those makes. > > > > > > Alex > > Bitbake does a lot of babysitting anyway :-) And is pretty good at > > it too. > > > > To me, fixing make et al. is more work and less effective then adding a > > feature to bitbake. The only way to know how much memory the compiler > > will use for each spawned compiler is to let it run. And then it's > > too late. > > > > This memory issue is all over our eco system and nobody cares (kernel, > > make etc.) The only thing moving is systemd's oom killer will arrive > > and > > start killing processes. So that will just stop our builds from > > completing. > > > > Yeah, I prefer a babysitter over a child murderer :-) > > > > Ferry > > > > > > > > > > > > > > > > > > > -- > # Randy MacLeod > # Wind River Linux > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#53122): https://lists.yoctoproject.org/g/yocto/message/53122 Mute This Topic: https://lists.yoctoproject.org/mt/82015730/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
