On Thu, Mar 19, 2020 at 11:04:26AM +0000, Richard Purdie wrote:
> On Thu, 2020-03-19 at 08:05 +0000, Mikko Rapeli wrote:
> > Once this is done, IO still happens when anything calls sync() and
> > fsync() and worst offenders are package management tools. In yocto
> > builds, package manager actions to flush to disk are always useless
> > since rootfs images are going to be compressed and original ones
> > wiped by rm_work anyway.
> > I've tried to hook eatmydata library into the build which makes
> > sync() and fsync() calls no-ops but I've still failed to fix all the
> > tools and processes called during build from python code. For shell
> > based tasks this does it:
> >
> > $ export LD_LIBRARY_PATH=/usr/lib/libeatmydata
> > $ export LD_PRELOAD=libeatmydata.so
> > $ grep -rn LD_PRELOAD conf/local.conf
> > conf/local.conf:305:BB_HASHBASE_WHITELIST_append = " LD_PRELOAD"
> > conf/local.conf:306:BB_HASHCONFIG_WHITELIST_append = " LD_PRELOAD"
>
> Doesn't pseudo intercept and stop these sync calls already? Its
> supposed to so if its not, we should fix that.
I will double check, but I'm sure I see IO going to disk when plenty of RAM
is still available in page cache.
> > The effect is clearly visible during build time using Performance Co-
> > Pilot (pcp) or similar tools to monitor CPU, memory, IO and network
> > IO. The usage of RAM as page cache grows until limits are hit and
> > only then writes to disk start, except for the python image
> > classes... Hints to fix this are welcome!
> >
> > To my knowledge of monitoring our builds, there is a lot of
> > optimization
> > potential to better build times. CPU are under utilized during
> > bitbake recipe parsing
>
> Recipe parsing should hit 100% CPU, its one of the few places we can do
> that.
I'm not fully aware what bitbake does before starting task execution.
With sumo, there is an initial spike in CPU use and then a long single
thread wait where log shows "Initialising tasks..." and Cooker process
is using a single core. For me this takes at least one minutes for
every build. Same is visible with zeus too.
Example graph from pmchart:
https://mcfrisk.kapsi.fi/temp/bitbake_start_to_task_execution.png
> > , fetch, configure, package and rootfs tasks.
>
> Sadly these tasks are much harder.
Yep.
> > Memory is not fully utilized either since IO through sync()/fsync()
> > happens everywhere
>
> non-pseudo tasks?
I'll try to check this case once more.
-Mikko
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#48853): https://lists.yoctoproject.org/g/yocto/message/48853
Mute This Topic: https://lists.yoctoproject.org/mt/72047879/21656
Group Owner: [email protected]
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
