Re: [yocto] Shared state doesn't live up to its name?
On 2016-12-01 18:35, Paul Eggleton wrote: On Thu, 01 Dec 2016 10:27:50 Gary Thomas wrote: I have a build machine where I build for lots of targets. On all of these targets (save a primary), I set the SSTATE_MIRROR to point to the sstate-cache of the primary target. I always build for the primary target first, then later the secondary ones. For the most part, the sstate mechanism works pretty well, but I see some anomalies. For example, why on the same host, built back to back, would a secondary build target (which uses the primary as it's SSTATE_MIRROR and that build is complete) need to rebuild from scratch such NATIVE packages as openssl? Note that these are native packages I'm asking about, so in my mind they should always be sharable. Any ideas? Is there something I can look at to investigate this? bitbake-diffsigs is the basic tool to compare signatures when those have changed. Find the siginfo / sigdata file for one of the early tasks that executed and compare them to see what changed. How are you setting SSTATE_MIRRORS in this scenario btw? SSTATE_MIRRORS ?= "\ file://.* file:///local/p0382_2016-01-13/sstate-cache/PATH" I ran a build yesterday that caused me to comment on this pattern. Here are the siginfo files for the secondary target (the latest build): -rw-rw-r-- 1 gthomas gthomas 21240 Dec 1 10:12 sstate-cache/universal/99/sstate:openssl-native:x86_64-linux:1.0.2j:r0:x86_64:3:99515597e2223aa4413f7f2e4acf6532_configure.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 8917 Dec 1 10:17 sstate-cache/universal/a4/sstate:openssl-native:x86_64-linux:1.0.2j:r0:x86_64:3:a49c66a9c786fe100f8e8b6e3bbd1e86_compile.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 14864 Dec 1 10:18 sstate-cache/universal/a7/sstate:openssl-native:x86_64-linux:1.0.2j:r0:x86_64:3:a747322ce06467949097c3da58497c7d_install.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 36490 Dec 1 10:18 sstate-cache/universal/e6/sstate:openssl-native:x86_64-linux:1.0.2j:r0:x86_64:3:e65ab886428d88f8735dba617268892f_populate_sysroot.tgz.siginfo Here are the corresponding ones from my primary target: cb9e0e1440492b85a6a9683b_unpack.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 6278 Nov 28 08:11 sstate-cache/44/sstate:openssl-native::1.0.2j:r0::3:44adeda2ff6ac235331af5dae45e45ea_patch.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 4997 Nov 28 08:11 sstate-cache/e9/sstate:openssl-native::1.0.2j:r0::3:e9ccda2229e69e2138ad0465a709e33a_fetch.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 21336 Nov 28 08:13 sstate-cache/universal/99/sstate:openssl-native:x86_64-linux:1.0.2j:r0:x86_64:3:99515597e2223aa4413f7f2e4acf6532_configure.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 8833 Nov 28 08:14 sstate-cache/universal/a4/sstate:openssl-native:x86_64-linux:1.0.2j:r0:x86_64:3:a49c66a9c786fe100f8e8b6e3bbd1e86_compile.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 14750 Nov 28 08:14 sstate-cache/universal/a7/sstate:openssl-native:x86_64-linux:1.0.2j:r0:x86_64:3:a747322ce06467949097c3da58497c7d_install.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 31029 Dec 1 09:45 sstate-cache/57/sstate:openssl-native::1.0.2j:r0::3:5725888ec8cde61e1417bc0b6ea51c6c_populate_lic.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 36502 Dec 1 09:45 sstate-cache/universal/e6/sstate:openssl-native:x86_64-linux:1.0.2j:r0:x86_64:3:e65ab886428d88f8735dba617268892f_populate_sysroot.tgz.siginfo Clearly, they are identical and the ones from the primary were built long before the ones on the secondary target. I'm still confused why it didn't work as expected. -- Gary Thomas | Consulting for the MLB Associates |Embedded world -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Shared state doesn't live up to its name?
On Thu, 01 Dec 2016 10:27:50 Gary Thomas wrote: > I have a build machine where I build for lots of targets. On > all of these targets (save a primary), I set the SSTATE_MIRROR > to point to the sstate-cache of the primary target. I always build > for the primary target first, then later the secondary ones. > > For the most part, the sstate mechanism works pretty well, but I > see some anomalies. For example, why on the same host, built back > to back, would a secondary build target (which uses the primary > as it's SSTATE_MIRROR and that build is complete) need to rebuild > from scratch such NATIVE packages as openssl? Note that these are > native packages I'm asking about, so in my mind they should always > be sharable. > > Any ideas? Is there something I can look at to investigate this? bitbake-diffsigs is the basic tool to compare signatures when those have changed. Find the siginfo / sigdata file for one of the early tasks that executed and compare them to see what changed. How are you setting SSTATE_MIRRORS in this scenario btw? Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Shared state doesn't live up to its name?
Compare the signatures of these 2 native builds. I'm still using sstate-diff-machines.sh to create archives of sstate signatures for interesting builds, so that I can easily compare them with other builds or the same build performed on different builder to see why something wasn't reused form sstate. On Thu, Dec 1, 2016 at 10:27 AM, Gary Thomas wrote: > I have a build machine where I build for lots of targets. On > all of these targets (save a primary), I set the SSTATE_MIRROR > to point to the sstate-cache of the primary target. I always build > for the primary target first, then later the secondary ones. > > For the most part, the sstate mechanism works pretty well, but I > see some anomalies. For example, why on the same host, built back > to back, would a secondary build target (which uses the primary > as it's SSTATE_MIRROR and that build is complete) need to rebuild > from scratch such NATIVE packages as openssl? Note that these are > native packages I'm asking about, so in my mind they should always > be sharable. > > Any ideas? Is there something I can look at to investigate this? > > Thanks > > -- > > Gary Thomas | Consulting for the > MLB Associates |Embedded world > > -- > ___ > yocto mailing list > yocto@yoctoproject.org > https://lists.yoctoproject.org/listinfo/yocto > -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
