Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
On Thu, Nov 03, 2011 at 11:08:37PM +0100, Martin Jansa wrote: On Thu, Nov 03, 2011 at 05:33:28PM +0100, Martin Jansa wrote: Hi, to recap what we have talked about on ELCE; Currently it doesn't seem to change sstate checksums after switching MACHINE, but still it rebuilds everything at least once after switch. Today I've tried to wipe tmp and rebuild from sstate-cache and I've noticed interesting thing: All recipes are starting with do_populate_lic_setscene not do_populate_sysroot_setscene like ie koen's build does NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Started NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Started NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Succeeded NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Succeeded Does do_populate_lic_setscene wipe all stamps after do_populate_lic so it causes the rebuild instead of population of sysroot from sstate? IIRC I had similar issue when I was starting to use oe-core and it was in the end caused by inheriting package_ipk (once directly from distro config and once through sane-feed-ipk.inc) and it was fixed 6 months ago by: http://git.shr-project.org/git/?p=meta-shr.git;a=commit;h=00623b3c6dd01c4a227008f7cfdb0801cb06f49a And IIRC we've talked about some ways to make sstate less error-prone for such things. Now I have similar symptoms but only after switching MACHINE. Any hint? With few more hours of debuging this.. I can say that sometimes it works, but sometimes it's too strict to be usefull anyway. This is also related to older thread about sstate http://lists.linuxtogo.org/pipermail/openembedded-core/2011-May/003308.html I'm going to try BB_SIGNATURE_HANDLER = basichash as it's supposed to become default anyway IIRC. Regards, Whole logs are here: http://build.shr-project.org/tests/jama/sstate.test here I'll try to shorten as much as possible. OE @shr ~/shr-core $ export MACHINE=qemuarm OE qemuarm@shr ~/shr-core $ bitbake -k core-image-core . NOTE: package core-image-core-1.0-r0: task do_rm_work_all: Succeeded NOTE: Tasks Summary: Attempted 5306 tasks of which 5250 didn't need to be rerun and 0 failed. OE qemuarm@shr ~/shr-core $ bitbake -k expat Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors. NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks NOTE: Running task 2428 of 2428 (ID: 11, /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb, do_rm_work_all) NOTE: package expat-2.0.1-r1: task do_rm_work_all: Started NOTE: package expat-2.0.1-r1: task do_rm_work_all: Succeeded NOTE: Tasks Summary: Attempted 2428 tasks of which 2427 didn't need to be rerun and 0 failed. All done OE qemuarmB@shr ~/shr-core $ cp openembedded-core/meta/conf/machine/qemuarm.conf openembedded-core/meta/conf/machine/qemuarmcopy.conf OE qemuarmB@shr ~/shr-core $ export MACHINE=qemuarmcopy OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat Parsing of 1436 .bb files complete (0 cached, 1436 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors. NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks .. libproxy fail (around task 2232) fix OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat libproxy built NOTE: Tasks Summary: Attempted 2428 tasks of which 2232 didn't need to be rerun and 0 failed. So after 3 hours we have qemuarm and qemuarmcopy (identical machine) with expat built! hurray But seems like nothing was reused from qemuarm build. once again for check OE qemuarmcopy@shr ~/shr-core $ export MACHINE=qemuarm OE qemuarm@shr ~/shr-core $ bitbake -k expat Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors. ... NOTE: Tasks Summary: Attempted 2428 tasks of which 2428 didn't need to be rerun and 0 failed. OE qemuarm@shr ~/shr-core $ export MACHINE=qemuarmcopy OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors. ... NOTE: Tasks Summary: Attempted 2428 tasks of which 2428 didn't need to be rerun and 0 failed. great OE qemuarmcopy@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* manifest-spitz-*expat* | sed 's/.*bitbake//g' 460 Oct 30 00:54 manifest-qemuarm-expat.deploy-ipk 1.2K Oct 30 00:54 manifest-qemuarm-expat.package 138 Oct 30 00:34 manifest-qemuarm-expat.populate-lic 732 Oct 30 00:54 manifest-qemuarm-expat.populate-sysroot 460 Nov 3 19:50 manifest-qemuarmcopy-expat.deploy-ipk 1.2K Nov 3
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
On Mon, Oct 10, 2011 at 03:17:51PM +0100, Richard Purdie wrote: On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote: On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote: Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven: On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote: Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven: sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning: 1) wipe out tmp, sstate-cache, pseudodone 2) MACHINE=beagleboard bitbake u-boot 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf 4) MACHINE=brokensstate bitbake u-boot Same here.. :/ (building 3 armv7 machines.. always from scratch). To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above? DISTRO=shr MACHINE=nokia900 bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, finished ok without wiping anything (same tmp, same sstate-cache, same pseudodone) MACHINE=palmpre bitbake shr-image - builds from scratch, again!, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, again!, finished ok MACHINE=nokia900 bitbake shr-image - builds from scratch, again!, finished ok palmpre and palmpre2 are the same $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^# require conf/machine/include/palmpre.inc $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^# require conf/machine/include/palmpre.inc nokia900 is also armv7a-vfp-neon (that's why all packages except MACHINE_ARCH were always reused without rebuilding in OE-classic). I have tried to debug sigdata files to see why and also tried to add MACHINEOVERRIDES[vardepsexclude] = MACHINE as RP suggested, but sofar haven't found solution at least to reuse already built sstate packages for armv7a-vfp-neon. I've posted a patch which at least locally solves the issue with the cp MACHINEA MACHINEB. If there are more issues I'd be interested, see my separate email with some debugging tips. Hi, to recap what we have talked about on ELCE; Currently it doesn't seem to change sstate checksums after switching MACHINE, but still it rebuilds everything at least once after switch. Today I've tried to wipe tmp and rebuild from sstate-cache and I've noticed interesting thing: All recipes are starting with do_populate_lic_setscene not do_populate_sysroot_setscene like ie koen's build does NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Started NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Started NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Succeeded NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Succeeded Does do_populate_lic_setscene wipe all stamps after do_populate_lic so it causes the rebuild instead of population of sysroot from sstate? IIRC I had similar issue when I was starting to use oe-core and it was in the end caused by inheriting package_ipk (once directly from distro config and once through sane-feed-ipk.inc) and it was fixed 6 months ago by: http://git.shr-project.org/git/?p=meta-shr.git;a=commit;h=00623b3c6dd01c4a227008f7cfdb0801cb06f49a And IIRC we've talked about some ways to make sstate less error-prone for such things. Now I have similar symptoms but only after switching MACHINE. Any hint? -- Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com signature.asc Description: Digital signature ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
On Thu, Nov 03, 2011 at 05:33:28PM +0100, Martin Jansa wrote: Hi, to recap what we have talked about on ELCE; Currently it doesn't seem to change sstate checksums after switching MACHINE, but still it rebuilds everything at least once after switch. Today I've tried to wipe tmp and rebuild from sstate-cache and I've noticed interesting thing: All recipes are starting with do_populate_lic_setscene not do_populate_sysroot_setscene like ie koen's build does NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Started NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Started NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Succeeded NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Succeeded Does do_populate_lic_setscene wipe all stamps after do_populate_lic so it causes the rebuild instead of population of sysroot from sstate? IIRC I had similar issue when I was starting to use oe-core and it was in the end caused by inheriting package_ipk (once directly from distro config and once through sane-feed-ipk.inc) and it was fixed 6 months ago by: http://git.shr-project.org/git/?p=meta-shr.git;a=commit;h=00623b3c6dd01c4a227008f7cfdb0801cb06f49a And IIRC we've talked about some ways to make sstate less error-prone for such things. Now I have similar symptoms but only after switching MACHINE. Any hint? With few more hours of debuging this.. I can say that sometimes it works, but sometimes it's too strict to be usefull anyway. Whole logs are here: http://build.shr-project.org/tests/jama/sstate.test here I'll try to shorten as much as possible. OE @shr ~/shr-core $ export MACHINE=qemuarm OE qemuarm@shr ~/shr-core $ bitbake -k core-image-core . NOTE: package core-image-core-1.0-r0: task do_rm_work_all: Succeeded NOTE: Tasks Summary: Attempted 5306 tasks of which 5250 didn't need to be rerun and 0 failed. OE qemuarm@shr ~/shr-core $ bitbake -k expat Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors. NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks NOTE: Running task 2428 of 2428 (ID: 11, /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb, do_rm_work_all) NOTE: package expat-2.0.1-r1: task do_rm_work_all: Started NOTE: package expat-2.0.1-r1: task do_rm_work_all: Succeeded NOTE: Tasks Summary: Attempted 2428 tasks of which 2427 didn't need to be rerun and 0 failed. All done OE qemuarmB@shr ~/shr-core $ cp openembedded-core/meta/conf/machine/qemuarm.conf openembedded-core/meta/conf/machine/qemuarmcopy.conf OE qemuarmB@shr ~/shr-core $ export MACHINE=qemuarmcopy OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat Parsing of 1436 .bb files complete (0 cached, 1436 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors. NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks .. libproxy fail (around task 2232) fix OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat libproxy built NOTE: Tasks Summary: Attempted 2428 tasks of which 2232 didn't need to be rerun and 0 failed. So after 3 hours we have qemuarm and qemuarmcopy (identical machine) with expat built! hurray But seems like nothing was reused from qemuarm build. once again for check OE qemuarmcopy@shr ~/shr-core $ export MACHINE=qemuarm OE qemuarm@shr ~/shr-core $ bitbake -k expat Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors. ... NOTE: Tasks Summary: Attempted 2428 tasks of which 2428 didn't need to be rerun and 0 failed. OE qemuarm@shr ~/shr-core $ export MACHINE=qemuarmcopy OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors. ... NOTE: Tasks Summary: Attempted 2428 tasks of which 2428 didn't need to be rerun and 0 failed. great OE qemuarmcopy@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* manifest-spitz-*expat* | sed 's/.*bitbake//g' 460 Oct 30 00:54 manifest-qemuarm-expat.deploy-ipk 1.2K Oct 30 00:54 manifest-qemuarm-expat.package 138 Oct 30 00:34 manifest-qemuarm-expat.populate-lic 732 Oct 30 00:54 manifest-qemuarm-expat.populate-sysroot 460 Nov 3 19:50 manifest-qemuarmcopy-expat.deploy-ipk 1.2K Nov 3 19:50 manifest-qemuarmcopy-expat.package 138 Nov 3 19:17 manifest-qemuarmcopy-expat.populate-lic 788 Nov 3 19:50 manifest-qemuarmcopy-expat.populate-sysroot 460 Oct 25 15:35 manifest-spitz-expat.deploy-ipk 1.2K Oct 25 15:35 manifest-spitz-expat.package 138 Oct 25 15:35 manifest-spitz-expat.populate-lic 704 Oct 25 15:35 manifest-spitz-expat.populate-sysroot OE qemuarmcopy@shr ~/shr-core/sstate-cache $ ls -la1
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote: Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven: On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote: Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven: sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning: 1) wipe out tmp, sstate-cache, pseudodone 2) MACHINE=beagleboard bitbake u-boot 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf 4) MACHINE=brokensstate bitbake u-boot Same here.. :/ (building 3 armv7 machines.. always from scratch). To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above? DISTRO=shr MACHINE=nokia900 bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, finished ok without wiping anything (same tmp, same sstate-cache, same pseudodone) MACHINE=palmpre bitbake shr-image - builds from scratch, again!, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, again!, finished ok MACHINE=nokia900 bitbake shr-image - builds from scratch, again!, finished ok palmpre and palmpre2 are the same $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^# require conf/machine/include/palmpre.inc $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^# require conf/machine/include/palmpre.inc nokia900 is also armv7a-vfp-neon (that's why all packages except MACHINE_ARCH were always reused without rebuilding in OE-classic). I have tried to debug sigdata files to see why and also tried to add MACHINEOVERRIDES[vardepsexclude] = MACHINE as RP suggested, but sofar haven't found solution at least to reuse already built sstate packages for armv7a-vfp-neon. Regards, -- Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com signature.asc Description: Digital signature ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote: On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote: Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven: On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote: Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven: sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning: 1) wipe out tmp, sstate-cache, pseudodone 2) MACHINE=beagleboard bitbake u-boot 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf 4) MACHINE=brokensstate bitbake u-boot Same here.. :/ (building 3 armv7 machines.. always from scratch). To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above? DISTRO=shr MACHINE=nokia900 bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, finished ok without wiping anything (same tmp, same sstate-cache, same pseudodone) MACHINE=palmpre bitbake shr-image - builds from scratch, again!, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, again!, finished ok MACHINE=nokia900 bitbake shr-image - builds from scratch, again!, finished ok palmpre and palmpre2 are the same $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^# require conf/machine/include/palmpre.inc $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^# require conf/machine/include/palmpre.inc nokia900 is also armv7a-vfp-neon (that's why all packages except MACHINE_ARCH were always reused without rebuilding in OE-classic). I have tried to debug sigdata files to see why and also tried to add MACHINEOVERRIDES[vardepsexclude] = MACHINE as RP suggested, but sofar haven't found solution at least to reuse already built sstate packages for armv7a-vfp-neon. Also as a tip for debugging this, you can do what is in the following commit (patch also inline below): http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip9id=1261cc69b152f33b934c2448020dfa3c95f312c6 which will mean bitbake -n generates stamp files for a given bitbake run. You can then quickly do: MACHINE=A bitbake -n foo MACHINE=B bitbake -n foo for example and then look for cases where two stamp sigdata files were generated. If you see them you can bitbake-diffsigs A B and I'll be very interested in the results of that and bitbake-diffsigs A... Cheers, Richard bitbake: Allow easier dry run stamp debugging Signed-off-by: Richard Purdie richard.pur...@linuxfoundation.org --- diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py index 8937f08..efa9ffe 100644 --- a/bitbake/lib/bb/build.py +++ b/bitbake/lib/bb/build.py @@ -255,7 +255,7 @@ def _task_data(fn, task, d): data.expandKeys(localdata) return localdata -def _exec_task(fn, task, d, quieterr): +def _exec_task(fn, task, d, quieterr, dryrun=False): Execute a BB 'task' Execution of a task involves a bit more setup than executing a function, @@ -317,10 +317,13 @@ def _exec_task(fn, task, d, quieterr): event.fire(TaskStarted(task, localdata), localdata) try: for func in (prefuncs or '').split(): -exec_func(func, localdata) -exec_func(task, localdata) +if not dryrun: +exec_func(func, localdata) +if not dryrun: +exec_func(task, localdata) for func in (postfuncs or '').split(): -exec_func(func, localdata) +if not dryrun: +exec_func(func, localdata) except FuncFailed as exc: if not quieterr: logger.error(str(exc)) @@ -355,13 +358,13 @@ def _exec_task(fn, task, d, quieterr): return 0 -def exec_task(fn, task, d): +def exec_task(fn, task, d, dryrun=False): try: quieterr = False if d.getVarFlag(task, quieterrors) is not None: quieterr = True -return _exec_task(fn, task, d, quieterr) +return _exec_task(fn, task, d, quieterr, dryrun) except Exception: from traceback import format_exc if not quieterr: diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py index aca06b5..e53da56 100644 --- a/bitbake/lib/bb/runqueue.py +++ b/bitbake/lib/bb/runqueue.py @@ -1156,8 +1156,7 @@ class RunQueueExecute: logger.critical(str(exc)) os._exit(1) try: -if not
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote: On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote: Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven: On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote: Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven: sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning: 1) wipe out tmp, sstate-cache, pseudodone 2) MACHINE=beagleboard bitbake u-boot 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf 4) MACHINE=brokensstate bitbake u-boot Same here.. :/ (building 3 armv7 machines.. always from scratch). To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above? DISTRO=shr MACHINE=nokia900 bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, finished ok without wiping anything (same tmp, same sstate-cache, same pseudodone) MACHINE=palmpre bitbake shr-image - builds from scratch, again!, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, again!, finished ok MACHINE=nokia900 bitbake shr-image - builds from scratch, again!, finished ok palmpre and palmpre2 are the same $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^# require conf/machine/include/palmpre.inc $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^# require conf/machine/include/palmpre.inc nokia900 is also armv7a-vfp-neon (that's why all packages except MACHINE_ARCH were always reused without rebuilding in OE-classic). I have tried to debug sigdata files to see why and also tried to add MACHINEOVERRIDES[vardepsexclude] = MACHINE as RP suggested, but sofar haven't found solution at least to reuse already built sstate packages for armv7a-vfp-neon. I've posted a patch which at least locally solves the issue with the cp MACHINEA MACHINEB. If there are more issues I'd be interested, see my separate email with some debugging tips. Cheers, Richard ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
Op 10 okt. 2011, om 16:16 heeft Richard Purdie het volgende geschreven: On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote: On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote: Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven: On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote: Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven: sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning: 1) wipe out tmp, sstate-cache, pseudodone 2) MACHINE=beagleboard bitbake u-boot 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf 4) MACHINE=brokensstate bitbake u-boot Same here.. :/ (building 3 armv7 machines.. always from scratch). To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above? DISTRO=shr MACHINE=nokia900 bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, finished ok without wiping anything (same tmp, same sstate-cache, same pseudodone) MACHINE=palmpre bitbake shr-image - builds from scratch, again!, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, again!, finished ok MACHINE=nokia900 bitbake shr-image - builds from scratch, again!, finished ok palmpre and palmpre2 are the same $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^# require conf/machine/include/palmpre.inc $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^# require conf/machine/include/palmpre.inc nokia900 is also armv7a-vfp-neon (that's why all packages except MACHINE_ARCH were always reused without rebuilding in OE-classic). I have tried to debug sigdata files to see why and also tried to add MACHINEOVERRIDES[vardepsexclude] = MACHINE as RP suggested, but sofar haven't found solution at least to reuse already built sstate packages for armv7a-vfp-neon. Also as a tip for debugging this, you can do what is in the following commit (patch also inline below): http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip9id=1261cc69b152f33b934c2448020dfa3c95f312c6 which will mean bitbake -n generates stamp files for a given bitbake run. You can then quickly do: MACHINE=A bitbake -n foo MACHINE=B bitbake -n foo for example and then look for cases where two stamp sigdata files were generated. If you see them you can bitbake-diffsigs A B and I'll be very interested in the results of that and bitbake-diffsigs A... Cheers, Richard bitbake: Allow easier dry run stamp debugging Signed-off-by: Richard Purdie richard.pur...@linuxfoundation.org --- diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py This seems to be against poky instead of bitbake and doesn't apply against bitbake master: koen@dominion:/OE/tentacle/sources/bitbake$ patch -p2 dry patching file lib/bb/build.py Hunk #1 succeeded at 257 (offset 2 lines). Hunk #2 succeeded at 319 (offset 2 lines). Hunk #3 succeeded at 360 (offset 2 lines). patching file lib/bb/runqueue.py Hunk #1 FAILED at 1156. 1 out of 1 hunk FAILED -- saving rejects to file lib/bb/runqueue.py.rej ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
On Mon, Oct 10, 2011 at 04:37:51PM +0200, Koen Kooi wrote: Op 10 okt. 2011, om 16:16 heeft Richard Purdie het volgende geschreven: On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote: On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote: Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven: On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote: Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven: sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning: 1) wipe out tmp, sstate-cache, pseudodone 2) MACHINE=beagleboard bitbake u-boot 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf 4) MACHINE=brokensstate bitbake u-boot Same here.. :/ (building 3 armv7 machines.. always from scratch). To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above? DISTRO=shr MACHINE=nokia900 bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, finished ok without wiping anything (same tmp, same sstate-cache, same pseudodone) MACHINE=palmpre bitbake shr-image - builds from scratch, again!, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, again!, finished ok MACHINE=nokia900 bitbake shr-image - builds from scratch, again!, finished ok palmpre and palmpre2 are the same $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^# require conf/machine/include/palmpre.inc $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^# require conf/machine/include/palmpre.inc nokia900 is also armv7a-vfp-neon (that's why all packages except MACHINE_ARCH were always reused without rebuilding in OE-classic). I have tried to debug sigdata files to see why and also tried to add MACHINEOVERRIDES[vardepsexclude] = MACHINE as RP suggested, but sofar haven't found solution at least to reuse already built sstate packages for armv7a-vfp-neon. Also as a tip for debugging this, you can do what is in the following commit (patch also inline below): http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip9id=1261cc69b152f33b934c2448020dfa3c95f312c6 which will mean bitbake -n generates stamp files for a given bitbake run. You can then quickly do: MACHINE=A bitbake -n foo MACHINE=B bitbake -n foo for example and then look for cases where two stamp sigdata files were generated. If you see them you can bitbake-diffsigs A B and I'll be very interested in the results of that and bitbake-diffsigs A... Cheers, Richard bitbake: Allow easier dry run stamp debugging Signed-off-by: Richard Purdie richard.pur...@linuxfoundation.org --- diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py This seems to be against poky instead of bitbake and doesn't apply against bitbake master: koen@dominion:/OE/tentacle/sources/bitbake$ patch -p2 dry patching file lib/bb/build.py Hunk #1 succeeded at 257 (offset 2 lines). Hunk #2 succeeded at 319 (offset 2 lines). Hunk #3 succeeded at 360 (offset 2 lines). patching file lib/bb/runqueue.py Hunk #1 FAILED at 1156. 1 out of 1 hunk FAILED -- saving rejects to file lib/bb/runqueue.py.rej Same problem here.. seems like diff is bigger then it was before: both current master OE @ ~/projects $ diff -rq poky/bitbake/ bitbake/ | grep -v pyc | grep -v pyo | grep -v .git Only in bitbake/: MANIFEST.in Only in bitbake/: TODO Only in poky/bitbake/bin: bitbake-runtask Only in bitbake/: classes Only in bitbake/: conf Files poky/bitbake/lib/bb/__init__.py and bitbake/lib/bb/__init__.py differ Files poky/bitbake/lib/bb/build.py and bitbake/lib/bb/build.py differ Files poky/bitbake/lib/bb/cooker.py and bitbake/lib/bb/cooker.py differ Files poky/bitbake/lib/bb/parse/parse_py/BBHandler.py and bitbake/lib/bb/parse/parse_py/BBHandler.py differ Files poky/bitbake/lib/bb/pysh/pyshtables.py and bitbake/lib/bb/pysh/pyshtables.py differ Files poky/bitbake/lib/bb/runqueue.py and bitbake/lib/bb/runqueue.py differ Only in poky/bitbake/lib/bb: shell.py Only in bitbake/: setup.py -- Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com signature.asc Description: Digital signature ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
On Mon, 2011-10-10 at 16:41 +0200, Martin Jansa wrote: On Mon, Oct 10, 2011 at 04:37:51PM +0200, Koen Kooi wrote: Op 10 okt. 2011, om 16:16 heeft Richard Purdie het volgende geschreven: On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote: On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote: Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven: On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote: Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven: sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning: 1) wipe out tmp, sstate-cache, pseudodone 2) MACHINE=beagleboard bitbake u-boot 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf 4) MACHINE=brokensstate bitbake u-boot Same here.. :/ (building 3 armv7 machines.. always from scratch). To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above? DISTRO=shr MACHINE=nokia900 bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre bitbake shr-image - builds from scratch, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, finished ok without wiping anything (same tmp, same sstate-cache, same pseudodone) MACHINE=palmpre bitbake shr-image - builds from scratch, again!, finished ok MACHINE=palmpre2 bitbake shr-image - builds from scratch, again!, finished ok MACHINE=nokia900 bitbake shr-image - builds from scratch, again!, finished ok palmpre and palmpre2 are the same $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^# require conf/machine/include/palmpre.inc $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^# require conf/machine/include/palmpre.inc nokia900 is also armv7a-vfp-neon (that's why all packages except MACHINE_ARCH were always reused without rebuilding in OE-classic). I have tried to debug sigdata files to see why and also tried to add MACHINEOVERRIDES[vardepsexclude] = MACHINE as RP suggested, but sofar haven't found solution at least to reuse already built sstate packages for armv7a-vfp-neon. Also as a tip for debugging this, you can do what is in the following commit (patch also inline below): http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip9id=1261cc69b152f33b934c2448020dfa3c95f312c6 which will mean bitbake -n generates stamp files for a given bitbake run. You can then quickly do: MACHINE=A bitbake -n foo MACHINE=B bitbake -n foo for example and then look for cases where two stamp sigdata files were generated. If you see them you can bitbake-diffsigs A B and I'll be very interested in the results of that and bitbake-diffsigs A... Cheers, Richard bitbake: Allow easier dry run stamp debugging Signed-off-by: Richard Purdie richard.pur...@linuxfoundation.org --- diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py This seems to be against poky instead of bitbake and doesn't apply against bitbake master: koen@dominion:/OE/tentacle/sources/bitbake$ patch -p2 dry patching file lib/bb/build.py Hunk #1 succeeded at 257 (offset 2 lines). Hunk #2 succeeded at 319 (offset 2 lines). Hunk #3 succeeded at 360 (offset 2 lines). patching file lib/bb/runqueue.py Hunk #1 FAILED at 1156. 1 out of 1 hunk FAILED -- saving rejects to file lib/bb/runqueue.py.rej Same problem here.. seems like diff is bigger then it was before: both current master OE @ ~/projects $ diff -rq poky/bitbake/ bitbake/ | grep -v pyc | grep -v pyo | grep -v .git Only in bitbake/: MANIFEST.in Only in bitbake/: TODO Only in bitbake/: classes Only in bitbake/: conf Only in bitbake/: setup.py These are deliberate and don't make sense in poky. Only in poky/bitbake/bin: bitbake-runtask Files poky/bitbake/lib/bb/cooker.py and bitbake/lib/bb/cooker.py differ These are related and I need to get this fixed, there is an open bug about this. Files poky/bitbake/lib/bb/__init__.py and bitbake/lib/bb/__init__.py differ Not showing up here for me... Files poky/bitbake/lib/bb/parse/parse_py/BBHandler.py and bitbake/lib/bb/parse/parse_py/BBHandler.py differ This is whitespace only. Files poky/bitbake/lib/bb/pysh/pyshtables.py and bitbake/lib/bb/pysh/pyshtables.py differ This is a generated file and in neither repository. Only in poky/bitbake/lib/bb: shell.py This is a
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven: sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning: 1) wipe out tmp, sstate-cache, pseudodone 2) MACHINE=beagleboard bitbake u-boot 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf 4) MACHINE=brokensstate bitbake u-boot And behold: NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Started NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Started NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Started NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Started NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Started NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Succeeded NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Succeeded NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Succeeded NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Started NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Started NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Started NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Started NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Succeeded NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Started NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Started NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Succeeded NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Started NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Started NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Started NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Succeeded NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Succeeded NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Started NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Started NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Succeeded NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Succeeded NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Started NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Succeeded NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: Executing RunQueue Tasks NOTE: Running task 536 of 920 (ID: 918, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/gcc/gcc-cross-initial_4.5.bb, do_populate_lic) NOTE: Running task 549 of 920 (ID: 452, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/binutils/binutils-cross_2.20.1.bb, do_fetch) NOTE: Running task 553 of 920 (ID: 504, /OE/tentacle/sources/openembedded-core/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_2.6.39.bb, do_fetch) NOTE: Running task 558 of 920 (ID: 538,
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote: Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven: sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning: 1) wipe out tmp, sstate-cache, pseudodone 2) MACHINE=beagleboard bitbake u-boot 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf 4) MACHINE=brokensstate bitbake u-boot Same here.. :/ (building 3 armv7 machines.. always from scratch). And behold: NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Started NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Started NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Started NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Started NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Started NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Succeeded NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Succeeded NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Succeeded NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Started NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Started NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Started NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Started NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Succeeded NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Started NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Started NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Succeeded NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Started NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Started NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Started NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Succeeded NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Succeeded NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Started NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Started NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Succeeded NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Succeeded NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Started NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Succeeded NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: Executing RunQueue Tasks NOTE: Running task 536 of 920 (ID: 918, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/gcc/gcc-cross-initial_4.5.bb, do_populate_lic) NOTE: Running task 549 of 920 (ID: 452, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/binutils/binutils-cross_2.20.1.bb, do_fetch) NOTE:
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven: On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote: Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven: sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning: 1) wipe out tmp, sstate-cache, pseudodone 2) MACHINE=beagleboard bitbake u-boot 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf 4) MACHINE=brokensstate bitbake u-boot Same here.. :/ (building 3 armv7 machines.. always from scratch). To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above? And behold: NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Started NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Started NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Started NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Started NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Started NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Succeeded NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Succeeded NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Succeeded NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Started NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Started NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Started NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Started NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Succeeded NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Started NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Started NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Succeeded NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Started NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Started NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Started NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Succeeded NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Succeeded NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Started NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Started NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Succeeded NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Succeeded NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Succeeded NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Started NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Succeeded NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Succeeded NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded NOTE: Executing RunQueue Tasks NOTE: Running task 536 of 920 (ID: 918, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/gcc/gcc-cross-initial_4.5.bb, do_populate_lic) NOTE:
[OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Based on a patch from Phil Blundell ph...@gnu.org with some tweaks from me. Signed-off-by: Richard Purdie richard.pur...@linuxfoundation.org --- diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass index eee04ab..6abf55b 100644 --- a/meta/classes/sstate.bbclass +++ b/meta/classes/sstate.bbclass @@ -259,10 +259,15 @@ def sstate_clean(ss, d): bb.utils.unlockfile(lock) stfile = d.getVar(STAMP, True) + .do_ + ss['task'] +extrainf = d.getVarFlag(do_ + ss['task'], 'stamp-extra-info') oe.path.remove(stfile) oe.path.remove(stfile + _setscene) -oe.path.remove(stfile + .*) -oe.path.remove(stfile + _setscene + .*) +if extrainf: +oe.path.remove(stfile + .* + extrainf) +oe.path.remove(stfile + _setscene + .* + extrainf) +else: +oe.path.remove(stfile + .*) +oe.path.remove(stfile + _setscene + .*) CLEANFUNCS += sstate_cleanall ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
let's get this in before my buildserver melts a hole in the floor :) Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven: sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Based on a patch from Phil Blundell ph...@gnu.org with some tweaks from me. Signed-off-by: Richard Purdie richard.pur...@linuxfoundation.org --- diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass index eee04ab..6abf55b 100644 --- a/meta/classes/sstate.bbclass +++ b/meta/classes/sstate.bbclass @@ -259,10 +259,15 @@ def sstate_clean(ss, d): bb.utils.unlockfile(lock) stfile = d.getVar(STAMP, True) + .do_ + ss['task'] +extrainf = d.getVarFlag(do_ + ss['task'], 'stamp-extra-info') oe.path.remove(stfile) oe.path.remove(stfile + _setscene) -oe.path.remove(stfile + .*) -oe.path.remove(stfile + _setscene + .*) +if extrainf: +oe.path.remove(stfile + .* + extrainf) +oe.path.remove(stfile + _setscene + .* + extrainf) +else: +oe.path.remove(stfile + .*) +oe.path.remove(stfile + _setscene + .*) CLEANFUNCS += sstate_cleanall ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
