On Tue, 2020-03-03 at 16:05 +0000, André Draszik wrote: > xz suffers from a reproducibility problem when not using multi- > threaded mode: > a) archives are created differently in single- vs multi-threaded > modes > b) xz will scale down the compression level so as to be able to > work within any memory limit given to it when being launched > in single-threaded mode. > > Thus, for reproducible xz archives we need to launch xz with > at least two threads. > > Add a little sanity test, and error out otherwise, so as to > guarantee no difference due this fact. > > Assumptions: > * The thread limit should be set via XZ_THREADS, not via > modifying XZ_DEFAULTS or XZ_OPTS, or any other way > * The thread limit should not be set to xz's magic value > zero (0), as that will lead to single-threaded mode on > single-core systems > > This patch here doesn't prevent people from shooting themselves > into the foot by changing XZ_DEFAULTS to change the number > of threads directly, but it's can serve as a hint at least.
I don't know if this patch is useful, feel free to drop it. In an ideal world, it'd parse the output of xz --verbose --verbose, to catch all possible ways people might be adjusting the thread limit, but that's non-trivial. Cheers, Andre' > > Signed-off-by: André Draszik <[email protected]> > --- > meta/classes/reproducible_build.bbclass | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/meta/classes/reproducible_build.bbclass > b/meta/classes/reproducible_build.bbclass > index 750eb950f2..e07bef87d8 100644 > --- a/meta/classes/reproducible_build.bbclass > +++ b/meta/classes/reproducible_build.bbclass > @@ -35,6 +35,7 @@ > # SOURCE_DATE_EPOCH is set for all tasks that might use it (do_configure, > do_compile, do_package, ...) > > BUILD_REPRODUCIBLE_BINARIES ??= '1' > +BUILD_REPRODUCIBLE_XZ_ARCHIVES ??= '1' > inherit ${@oe.utils.ifelse(d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1', > 'reproducible_build_simple', '')} > > SDE_DIR ="${WORKDIR}/source-date-epoch" > @@ -198,4 +199,8 @@ BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH" > python () { > if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1': > d.appendVarFlag("do_unpack", "postfuncs", " > do_create_source_date_epoch_stamp") > + > + if d.getVar('BUILD_REPRODUCIBLE_XZ_ARCHIVES') == '1': > + if int(d.getVar('XZ_THREADS')) < 2: > + bb.fatal("Can not build reproducible XZ archives without > threading") > } -- _______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-core
