On Wednesday, 15 February 2023 11:31:49 GMT Peter Böhm wrote: > Am Mittwoch, 15. Februar 2023, 10:56:22 CET schrieb Peter Humphrey: > > Hello list, > > > > Not long ago I read that we should allow 2GB RAM for every emerge job - > > that is, we should divide our RAM size by 2 to get the maximum number of > > simultaneous jobs. I'm trying to get that right, but I'm not there yet. > > > > I have these entries in make.conf: > > EMERGE_DEFAULT_OPTS="--jobs=16 --load-average=32 --autounmask=n --quiet- > > unmerge-warn --ke>
The above determine how may ebuilds will be emerged in parallel. If you are rebuilding your whole system with hundreds of packages stacking up to be emerged, then having as high as 16 packages being emerged in parallel could be advantageous. > > MAKEOPTS="-j16" This determines how many MAKE jobs will run in parallel in any one emerge. Large packages like chromium will benefit from maximising the number of jobs here, as long as you have enough RAM. Given you have 24 threads and your RAM is 64GB, you should be able to ratchet this up to -j24, but not if you specify a high EMERGE_DEFAULT_OPTS at the same time, or if the compiler is eating up more than 2G per process. > > Today, though, I saw load averages going up to 72. Can anyone suggest > > better values to suit my 24 threads and 64GB RAM? Since you have specified up to 16 parallel emerges and each one could run up to 16 MAKE jobs, you can understand why you would soon find loads escalating and your machine becoming unresponsive. You should consider what is more important for you, emerging as many packages in parallel as possible, or emerging any one large package as fast as possible. Two extreme examples would be setting EMERGE_DEFAULT_OPTS at "--jobs 24", with MAKEOPTS at "-j1", or conversely setting EMERGE_DEFAULT_OPTS at "--jobs 1", with MAKEOPTS at "-j24". On my old and slow laptop with only 4 threads and 16G of RAM my priority is to finish large packages faster. I leave EMERGE_DEFAULT_OPTS unset, while specifying MAKEOPTS="-j5 -l4.8". This uses a job number I determined by trial and error, building ffmpeg repeatedly by progressively increasing the number for -j from 1 to 12. The two faster times were achieved with -j5 and -j10, which aligns with the old myth of using CPU+1. Regarding RAM being used being ~2G per MAKE job, this fluctuates with successive compiler versions. I have seen up to 3.4G of RAM per process, while emerging chromium. For such huge packages which cause excessive swapping, unresponsiveness and thrashing of disk, I limit the MAKEOPTS to 3 in package.env. > Maybe you are interested in this wiki article: > > https://wiki.gentoo.org/wiki/User:Pietinger/Tutorials/Optimize_compile_times > > Regards, > Peter I'd start by reading the suggestions in this article first, which is a good introduction to the concepts involved: https://wiki.gentoo.org/wiki/MAKEOPTS
signature.asc
Description: This is a digitally signed message part.
