Hi Janne,
Op 03-01-2023 om 16:03 schreef Janne Kiiskila:
Hei,
Is not the easiest solution to build the actual full build in steps, rather
than build the whole thing?
So, instead of doing a
Sure, that is easy enough. Setting PARALLEL_MAKE is easy enough too.
I was hoping to find a fundamental solution.
full bitbake <image>
Do
bitbake nodejs
bitbake binutils
bitbake rust (if you happen to need it)
Yeah, building rust is a resource hog too. I didn't build that for a
long time.
and only after that do
bitbake <image>
That way you can control the memory load and build it piecemeal to avoid
out-of-memory situations.
For cheapness of memory - it's not always that straight forward,
a) some cloud build machines might go insanely up in price, if you add memory
(it usually then adds also cores) OR
b) some laptops etc. might simply not have any memory slots available, so you
can't expand even if you want.
Yes, and I note that many people trying to build my image
(meta-intel-edison) do so from a virtual machine as they only have a
windows host.
And then there are even more limitations.
Best Regards,
Janne Kiiskilä
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of
Ferry Toth via lists.yoctoproject.org
Sent: tiistai 3. tammikuuta 2023 16.41
To: Quentin Schulz <[email protected]>; Alexander Kanavin
<[email protected]>
Cc: [email protected]
Subject: Re: [yocto] bitbake controlling memory use
Hi Quentin,
Op 03-01-2023 om 15:36 schreef Quentin Schulz:
Hi Ferry,
On 1/3/23 15:29, Ferry Toth wrote:
Hi Alex,
Op 03-01-2023 om 15:18 schreef Alexander Kanavin:
I have to note that even the most expensive 16 Gb RAM module is less
than 100 Euro, and can be obtained for half that much. Surely you
value your time more than that?
Of course. And if I didn't I could reduce the number of `PARALLEL_MAKE`.
But I have also seen bitbake attempting to build nodejs,
nodejs-native and binutils in parallel.
I think by default there are 4 tasks, with 16 threads, each could
require 1GB RAM. Would you say, in such a case Yocto requires 64GB RAM?
And with increasing #cores it gets worse.
Not that it is too expensive to throw hardware at it, but it seems to
be a fundamental problem that I would like to resolve. In my spare
time of course.
Just to add that bitbake now supports pressure thresholds (since
Kirkstone release I believe):
https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user
-manual-ref-variables.html#term-BB_PRESSURE_MAX_MEMORY
If your recipes put enough pressure on RAM before one or two of
nodejs, nodejs-native and binutils gets scheduled, it would prevent
that. However I believe if the timing is just right (unfortunate) and
there's not enough pressure when all three recipes do_compile start,
they would all start and you would have the same issue.
Exactly
Cheers,
Quentin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#58903): https://lists.yoctoproject.org/g/yocto/message/58903
Mute This Topic: https://lists.yoctoproject.org/mt/82015730/21656
Group Owner: [email protected]
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
