On Tue, Feb 22, 2022 at 3:42 AM Jimmy Assarsson <[email protected]> wrote: > > On 2022-02-21 05:13, Bruce Ashfield wrote: > > on 19/02/2022 Jimmy Assarsson wrote: > >> On 2022-02-19 20:42, Bruce Ashfield wrote: > >>> On Sat, Feb 19, 2022 at 5:28 AM Jimmy Assarsson > >>> <[email protected]> wrote: > >>>> > >>>> When patching a kernel git repository, in a recipe inheriting > >>>> kernel-yocto.bbclass, > >>>> the resulting commit hash will become different every time the source is > >>>> unpacked and patched. > >>>> > >>>> This is a problem that causes non-reproducible builds, when the commit > >>>> hash is built > >>>> into the kernel (CONFIG_LOCALVERSION_AUTO=y). > >>>> > >>>> > >>>> Currently it is not a problem in linux-yocto, since an empty .scmversion > >>>> is > >>>> created in kernel_do_configure [1]. This is preventing the kernel build > >>>> from > >>>> generating its own .scmversion. > >>>> > >>>> If removing this commit, setting CONFIG_LOCALVERSION_AUTO=y and applying > >>>> any > >>>> patch in the linux-yocto recipe, this will result in a non-reproducible > >>>> build. > >> > >> ... > >> > >>>> diff --git a/tools/kgit-s2q b/tools/kgit-s2q > >>>> index 706783e..b46a138 100755 > >>>> --- a/tools/kgit-s2q > >>>> +++ b/tools/kgit-s2q > >>>> @@ -558,7 +558,7 @@ do > >>>> echo "($APPLIED/$COUNT) `basename $i`" > >>>> fi > >>>> > >>>> - git am --keep-non-patch $DIR/$i > /dev/null 2>&1 > >>>> + git am --keep-non-patch --committer-date-is-author-date $DIR/$i > >>>> > /dev/null 2>&1 > >>>> if [ $? != 0 ];then > >>>> git am --abort > /dev/null 2>&1 > >>>> echo "[INFO]: check of $DIR/$i with \"git am\" did > >>>> not pass, trying reduced context." > >>>> > >>>> > >>>> > >>>> I'm not sure if this is a proper solution to fix the problem or what > >>>> side effects it may introduce? > >>> > >>> There's nothing fundamentally wrong with that solution, but there are > >>> modes for the kernel builds where we absolutely do not want the > >>> reproducible build (and older dates) to be in effect (see my comments > >>> on the OE core mailing list when the ability to disable reproducible > >>> builds was almost removed). > >> > >> Thanks, now I've looked it up. > >> > >>> We could add an option to the patch queue management that was enabled > >>> when reproducible builds are in play, so git am could operate like > >>> that. > >>> > >>> Alternatively, you could bbappend the other kernel's do_configure and > >>> do the same .scmversion trick to make sure that everything is > >>> consistent. > >> > >> I don't follow, can you be more specific? > >> We encountered this problem when building linux-imx (from meta-freescale). > >> Preferably the .scmversion workaround in [1] should be dropped, so that > >> there is no need to reinvent/mimic the output of > >> linux/scripts/setlocalversion, in a bitbake recipe. > > > > I see no compelling reason to drop the .scmversion in the main > > kernel.bbclass > > (at least not in the short term). It has been there for a significant amount > > of time, and removing it now would very likely cause some ripple effects > > througout the ecosystem (it is there for a few different issues). While not > > elegant, it is an available and workable solution to the problem it is > > solving. > > I agree. And obviously this is not a big issue for others. > There have been at least two failed attempts on removing this [4][5][6]. > > > I was suggesting that anyone can bbappend any kernel recipe to do the > > similar > > .scmversion set (assuming they aren't using the core kernel bbclass > > configure()). > > > > But from your next statement, that isn't going to work as the sole solution, > > since yes, that will only restore where we keep the git rev out of the > > localversion. > > > >> > >> To be clear, we want the result of CONFIG_LOCALVERSION_AUTO=y, but > >> with a reproducible output. > > > > And that's actually part of the reason the .scmversion is set, the > > setlocalversion script was appending values in some configurations and > > combinations of patching, etc, such that issues were popping up with version > > matching. > > > > Part of getting that to work, definitely could be to apply patches with the > > author date, but that has to be triggered on reproducible builds being > > enabled (which is of course the default) and also preserving the use of the > > .scmversion for the baseline / basic kernel configuration routines. > > > > .. so one config to say "don't set scmversion" and another to say "apply > > patches with author date" (this one toggling based on reproducible builds). > > And then both scenarios can be supported. > > Currently I'm only interested in fixing the "apply patches with > author date", to get reproducible builds. > > So we will end up with a new config in kernel-yocto.bbclass, since this > is the only location where kgit-s2q is used. > And we will get a new option in kgit-s2q, that will result in > "git am --committer-date-is-author-date" being used. > > Any name suggestions for the config and option? > KERNEL_PATCH_WITH_AUTHOR_DATE > KERNEL_PATCH_REPRODUCIBLE_HASH > KERNEL_PATCH_REPRODUCIBLE_COMMIT
It is better to abstract it behind the reproducible feature description, since there are other kernel reproducibility components that exist, or may appear in the future. If you want, I can have a go at the patch, which will allow me to run my local tests and put it in my next consolidated pull request. Bruce > --patch-with-author-date > --apply-patch-with-author-date > --committer-date-is-author-date > > > [4] https://patchwork.openembedded.org/patch/138333 > [5] https://patchwork.openembedded.org/patch/171479 > [6] https://patchwork.openembedded.org/patch/171513 > > > Best regards, > jimmy > > > > Bruce > > > >> > >> Best regards, > >> jimmy > >> > >> > >>> Bruce > >>> > >>>> > >>>> [1] > >>>> https://git.yoctoproject.org/poky/tree/meta/classes/kernel.bbclass?h=honister#n589 > >>>> [2] > >>>> https://git.yoctoproject.org/meta-freescale/tree/classes/fsl-kernel-localversion.bbclass?h=honister > >>>> [3] > >>>> https://github.com/OE4T/meta-tegra/blob/honister/recipes-kernel/linux/linux-tegra_4.9.bb -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#10979): https://lists.yoctoproject.org/g/linux-yocto/message/10979 Mute This Topic: https://lists.yoctoproject.org/mt/89251386/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/linux-yocto/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
