> -----Original Message-----
> From: Richard Purdie <[email protected]>
> Sent: den 14 februari 2022 11:32
> To: Peter Kjellerstedt <[email protected]>; openembedded-
> [email protected]
> Subject: Re: [OE-core] [PATCH] bitbake.conf: Add COMMON_LICENSE_DIR to
> BB_HASHEXCLUDE_COMMON
>
> On Sun, 2022-02-13 at 22:36 +0000, Peter Kjellerstedt wrote:
> > > -----Original Message-----
> > > From: Richard Purdie <[email protected]>
> > > Sent: den 13 februari 2022 22:44
> > > To: Peter Kjellerstedt <[email protected]>;
> > > [email protected]
> > > Subject: Re: [OE-core] [PATCH] bitbake.conf: Add COMMON_LICENSE_DIR to
> > > BB_HASHEXCLUDE_COMMON
> > >
> > > On Sun, 2022-02-13 at 21:34 +0100, Peter Kjellerstedt wrote:
> > > > Differences in COMMON_LICENSE_DIR should not affect the task hashes.
> > > >
> > > > Signed-off-by: Peter Kjellerstedt <[email protected]>
> > > > ---
> > > > meta/conf/bitbake.conf | 2 +-
> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > > > index fba99e8f0c..47c8cb39f9 100644
> > > > --- a/meta/conf/bitbake.conf
> > > > +++ b/meta/conf/bitbake.conf
> > > > @@ -922,7 +922,7 @@ BB_HASHEXCLUDE_COMMON ?= "TMPDIR FILE PATH PWD
> > > > BB_TASKHASH BBPATH BBSERVER DL_DI
> > > > BB_WORKERCONTEXT BB_LIMITEDDEPS BB_UNIHASH
> extend_recipe_sysroot
> > > > DEPLOY_DIR \
> > > > SSTATE_HASHEQUIV_METHOD SSTATE_HASHEQUIV_REPORT_TASKDATA \
> > > > SSTATE_HASHEQUIV_OWNER CCACHE_TOP_DIR BB_HASHSERVE
> > > > GIT_CEILING_DIRECTORIES \
> > > > - OMP_NUM_THREADS BB_CURRENTTASK"
> > > > + OMP_NUM_THREADS BB_CURRENTTASK COMMON_LICENSE_DIR"
> > > > BB_HASHBASE_WHITELIST ?= "${BB_HASHEXCLUDE_COMMON}
> PSEUDO_IGNORE_PATHS
> > > > BUILDHISTORY_DIR \
> > > > SSTATE_DIR SOURCE_DATE_EPOCH"
> > > > BB_HASHCONFIG_WHITELIST ?= "${BB_HASHEXCLUDE_COMMON} DATE TIME
> > > > SSH_AGENT_PID \
> > >
> > > I think this has been discussed before and I'm very uneasy at the
> idea. Some
> > > users would expect that if they add "their" version of a license in a
> layer
> > > with
> > > higher priority, they'd expect the hashes to change.
> >
> > If the value was a relative path I could buy that, but not for an
> > absolute path.
>
> True, however my concern is more that if you're changing this you are changing
> the configuration and you'd expect the hashes to change as a result. Adding it
> to the exclusion list hides that.
On the other hand, most (all?) other variables that take absolute paths are
also present in BB_HASHEXCLUDE_COMMON so that differences to where the build
directory happens to be do not affect the sstate.
> > > Where is this causing an issue?
> >
> > Due to the huge number of licenses in meta/files/common-licenses after
> > all SPDX licenses were added, we cannot use that directory anymore as
> > it triples the recipe parsing time for us (since we define
> > INCOMPATIBLE LICENSE as AVAILABLE_LICENSES minus COMPATIBLE_LICENSES,
> > and the license code doesn't really handle having many hundreds of
> > licenses in INCOMPATIBLE LICENSE). Thus as a workaround I have had to
> > create a common-licenses directory in one of our layers which only has
> > symbolic links to the licenses in meta that we need. Then we set:
> >
> > COMMON_LICENSE_DIR := "${LAYERDIR}/files/common-licenses"
> >
> > in that layer's layer.conf, but since ${LAYERDIR} varies from build host
> > to build host, our whole global sstate was now only useable on the build
> > servers and not for local developer builds.
>
> So shouldn't you just add COMMON_LICENSE_DIR to BB_HASHEXCLUDE_COMMON in
> your config?
Well, based on your reluctance to add it to bitbake.conf, I have now instead
added COMMON_LICENSE_DIR[vardepvalue] = "" together with where we redefine
COMMON_LICENSE_DIR.
However, the reason I suggested adding it to bitbake.conf is that if you
change this variable, it is easy to miss that it can cause your sstate to
be different for each build host. I know I did, and we have had it modified
for half a year. It wasn't until yesterday, when I made a local build that
I expected to fully build from our global sstate and it instead rebuilt a
lot of recipes, that I found out there was something wrong. It took me a
couple of hours to dig into the siginfo files to finally end up at the
absolute path in COMMON_LICENSE_DIR as the source of the differing sstate.
> Cheers,
>
> Richard
//Peter
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161703):
https://lists.openembedded.org/g/openembedded-core/message/161703
Mute This Topic: https://lists.openembedded.org/mt/89121479/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
