On Thu, May 10, 2012 at 05:20:31PM -0500, Mark Hatle wrote:
> We've discovered that perl seems to be a pretty bad pathological case for
> setscene.
>
> Easy way to see what I mean:
>
> bitbake perl
> bitbake -c clean perl
> time bitbake perl
>
> The system isn't doing anything but running the setscene elements to bring
> perl
> from the sstate-cache into the sysroot and related directories.. but it takes
> a
> very long time to do this:
>
> real 3m7.430s
> user 0m36.316s
> sys 2m46.742s
>
> That is on my 8 core w/ 12 GB of ram system.. (standard SATA disks)
>
> Digging into this, I found that the majority of the time is spent in
> meta/classes/sstate.bbclass: sstate_installpkg
>
> If you look for the three os.system("sed ... calls, almost all of the 3
> minutes
> is there. Perl has so many text files with embedded paths that a significant
> amount of time is spent preparing these files and fixing paths.
>
> I did a test and changed the three sed cases into a single sed operation:
>
> - os.system("sed -i -e s:FIXMESTAGINGDIRTARGET:%s:g %s" %
> (staging_target, sstateinst + file))
> - os.system("sed -i -e s:FIXMESTAGINGDIRHOST:%s:g %s" %
> (staging_host, sstateinst + file))
> - os.system("sed -i -e s:FIXMESTAGINGDIR:%s:g %s" % (staging,
> sstateinst + file))
> + os.system("sed -i -e s:FIXMESTAGINGDIRTARGET:%s:g -e
> s:FIXMESTAGINGDIRHOST:%s:g -e s:FIXMESTAGINGDIR:%s:g %s " % (staging_target,
> staging_host, staging, sstateinst + file))
>
> The result:
>
> real 1m20.979s
> user 0m19.400s
> sys 1m4.115s
>
> This is a very significant performance improvement. I'm not sure if it's
> possible to speed this up even more, but it would be nice if we could get
> things
> under a minute.
>
> Does anyone see a way we could do this quicker or in a better way to boost
> performance? (One possibility is figure out how to thread the file
> processing
> and use available processors...)I guess it does process for all files in fixmepath file, so try to run sed on all of them (or chunks) together with xargs like in: http://git.openembedded.org/openembedded-core/commit/meta/classes/sstate.bbclass?id=94c52d68fc2ce258bcc5b0978ac73413480a1a93 Cheers, -- Martin 'JaMa' Jansa jabber: [email protected]
signature.asc
Description: Digital signature
_______________________________________________ Openembedded-core mailing list [email protected] http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
