We've discovered that perl seems to be a pretty bad pathological case for
setscene.
Easy way to see what I mean:
bitbake perl
bitbake -c clean perl
time bitbake perl
The system isn't doing anything but running the setscene elements to bring perl
from the sstate-cache into the sysroot and related directories.. but it takes a
very long time to do this:
real 3m7.430s
user 0m36.316s
sys 2m46.742s
That is on my 8 core w/ 12 GB of ram system.. (standard SATA disks)
Digging into this, I found that the majority of the time is spent in
meta/classes/sstate.bbclass: sstate_installpkg
If you look for the three os.system("sed ... calls, almost all of the 3 minutes
is there. Perl has so many text files with embedded paths that a significant
amount of time is spent preparing these files and fixing paths.
I did a test and changed the three sed cases into a single sed operation:
- os.system("sed -i -e s:FIXMESTAGINGDIRTARGET:%s:g %s" %
(staging_target, sstateinst + file))
- os.system("sed -i -e s:FIXMESTAGINGDIRHOST:%s:g %s" %
(staging_host, sstateinst + file))
- os.system("sed -i -e s:FIXMESTAGINGDIR:%s:g %s" % (staging,
sstateinst + file))
+ os.system("sed -i -e s:FIXMESTAGINGDIRTARGET:%s:g -e
s:FIXMESTAGINGDIRHOST:%s:g -e s:FIXMESTAGINGDIR:%s:g %s " % (staging_target,
staging_host, staging, sstateinst + file))
The result:
real 1m20.979s
user 0m19.400s
sys 1m4.115s
This is a very significant performance improvement. I'm not sure if it's
possible to speed this up even more, but it would be nice if we could get things
under a minute.
Does anyone see a way we could do this quicker or in a better way to boost
performance? (One possibility is figure out how to thread the file processing
and use available processors...)
--Mark
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
