I think you write a script to do all the WATCH-STOP-CONTINUE stuff.
e.g.
memwatch bitbake core-image-minimal
Options could be added.
e.g.
memwatch --interval 5 --logfile test.log bitbake core-image-minimal
This script first becomes a session leader, then forks to start the
'bitbake' command as its child process.
Then, every several seconds (say 10s by default), it checks the current
memory usage, and according to its policy, determines whether to
stop/continue some process or not.
For stopping the process, you can first get all its child process by
simply using the 'ps' command.
e.g.
$ ps o vsize,comm,pid -s 28303 | sort -n -r
317284 emacs 12883
28912 ps 36302
26248 sort 36303
21432 man 24797
17992 bash 28303
9852 pager 24807
VSZ COMMAND PID
Then skip the bitbake processes, stop the first one that uses the
largest memory, record its PID.
For continuing processes, just start it according to the records. Maybe
using FILO by default?
Best Regards,
Chen Qi
On 04/11/2021 11:23 PM, Gmane Admin wrote:
My build machine has 8 cores + HT so bitbake enthusiastically assumes
16. However I have (only?) 16GB of RAM (+24GB swap space).
16GB of RAM has always been more then enough with 4 core + HT, but now
building certain recipes (nodejs, but rust will probably do it as
well) eats up more memory then there actually is RAM causing excessive
swapping.
In fact the swapping is so excessive that just this single recipe
determines largely the total image build time. However restricting
bitbake (or actually parallel make) to use only 4 cores + HT sounds
also like a waste.
I know this has been looked at in the past, but I think it needs
fundamental resolving (other then, hé, why not add just another stick
of memory).
What I do manually when I run into swapping so bad my desktop becomes
unresponsive is ssh from another machine and then stop (not kill) the
processes that use the most memory.
These then get swapped out, but not back in, allowing the remaining
tasks to complete without swapping. Then when sufficient memory
becomes free again I continue the stopped processes.
Isn't this something that could be added to bitbake to automate using
memory efficiently?
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#53075): https://lists.yoctoproject.org/g/yocto/message/53075
Mute This Topic: https://lists.yoctoproject.org/mt/82015730/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
