On Sat, Feb 26, 2022 at 2:17 AM Jimmy Assarsson <[email protected]> wrote: > > On 2022-02-24 16:41, Bruce Ashfield wrote: > > On Tue, Feb 22, 2022 at 3:42 AM Jimmy Assarsson > > <[email protected]> wrote: > >> On 2022-02-21 05:13, Bruce Ashfield wrote: > >>> on 19/02/2022 Jimmy Assarsson wrote: > >>>> On 2022-02-19 20:42, Bruce Ashfield wrote: > >>>>> On Sat, Feb 19, 2022 at 5:28 AM Jimmy Assarsson > >>>>> <[email protected]> wrote: > >>>>>> > >>>>>> When patching a kernel git repository, in a recipe inheriting > >>>>>> kernel-yocto.bbclass, > >>>>>> the resulting commit hash will become different every time the source > >>>>>> is unpacked and patched. > >>>>>> > >>>>>> This is a problem that causes non-reproducible builds, when the commit > >>>>>> hash is built > >>>>>> into the kernel (CONFIG_LOCALVERSION_AUTO=y). > >>>>>> > >>>>>> > >>>>>> Currently it is not a problem in linux-yocto, since an empty > >>>>>> .scmversion is > >>>>>> created in kernel_do_configure [1]. This is preventing the kernel > >>>>>> build from > >>>>>> generating its own .scmversion. > >>>>>> > >>>>>> If removing this commit, setting CONFIG_LOCALVERSION_AUTO=y and > >>>>>> applying any > >>>>>> patch in the linux-yocto recipe, this will result in a > >>>>>> non-reproducible build. > >>>> > >>>> ... > >>>> > >>>>>> diff --git a/tools/kgit-s2q b/tools/kgit-s2q > >>>>>> index 706783e..b46a138 100755 > >>>>>> --- a/tools/kgit-s2q > >>>>>> +++ b/tools/kgit-s2q > >>>>>> @@ -558,7 +558,7 @@ do > >>>>>> echo "($APPLIED/$COUNT) `basename $i`" > >>>>>> fi > >>>>>> > >>>>>> - git am --keep-non-patch $DIR/$i > /dev/null 2>&1 > >>>>>> + git am --keep-non-patch --committer-date-is-author-date > >>>>>> $DIR/$i > /dev/null 2>&1 > >>>>>> if [ $? != 0 ];then > >>>>>> git am --abort > /dev/null 2>&1 > >>>>>> echo "[INFO]: check of $DIR/$i with \"git am\" did > >>>>>> not pass, trying reduced context." > >>>>>> > >>>>>> > >>>>>> > >>>>>> I'm not sure if this is a proper solution to fix the problem or what > >>>>>> side effects it may introduce? > >>>>> > >>>>> There's nothing fundamentally wrong with that solution, but there are > >>>>> modes for the kernel builds where we absolutely do not want the > >>>>> reproducible build (and older dates) to be in effect (see my comments > >>>>> on the OE core mailing list when the ability to disable reproducible > >>>>> builds was almost removed). > >>>> > >>>> Thanks, now I've looked it up. > >>>> > >>>>> We could add an option to the patch queue management that was enabled > >>>>> when reproducible builds are in play, so git am could operate like > >>>>> that. > >>>>> > >>>>> Alternatively, you could bbappend the other kernel's do_configure and > >>>>> do the same .scmversion trick to make sure that everything is > >>>>> consistent. > >>>> > >>>> I don't follow, can you be more specific? > >>>> We encountered this problem when building linux-imx (from > >>>> meta-freescale). > >>>> Preferably the .scmversion workaround in [1] should be dropped, so that > >>>> there is no need to reinvent/mimic the output of > >>>> linux/scripts/setlocalversion, in a bitbake recipe. > >>> > >>> I see no compelling reason to drop the .scmversion in the main > >>> kernel.bbclass > >>> (at least not in the short term). It has been there for a significant > >>> amount > >>> of time, and removing it now would very likely cause some ripple effects > >>> througout the ecosystem (it is there for a few different issues). While > >>> not > >>> elegant, it is an available and workable solution to the problem it is > >>> solving. > >> > >> I agree. And obviously this is not a big issue for others. > >> There have been at least two failed attempts on removing this [4][5][6]. > >> > >>> I was suggesting that anyone can bbappend any kernel recipe to do the > >>> similar > >>> .scmversion set (assuming they aren't using the core kernel bbclass > >>> configure()). > >>> > >>> But from your next statement, that isn't going to work as the sole > >>> solution, > >>> since yes, that will only restore where we keep the git rev out of the > >>> localversion. > >>> > >>>> > >>>> To be clear, we want the result of CONFIG_LOCALVERSION_AUTO=y, but > >>>> with a reproducible output. > >>> > >>> And that's actually part of the reason the .scmversion is set, the > >>> setlocalversion script was appending values in some configurations and > >>> combinations of patching, etc, such that issues were popping up with > >>> version > >>> matching. > >>> > >>> Part of getting that to work, definitely could be to apply patches with > >>> the > >>> author date, but that has to be triggered on reproducible builds being > >>> enabled (which is of course the default) and also preserving the use of > >>> the > >>> .scmversion for the baseline / basic kernel configuration routines. > >>> > >>> .. so one config to say "don't set scmversion" and another to say "apply > >>> patches with author date" (this one toggling based on reproducible > >>> builds). > >>> And then both scenarios can be supported. > >> > >> Currently I'm only interested in fixing the "apply patches with > >> author date", to get reproducible builds. > >> > >> So we will end up with a new config in kernel-yocto.bbclass, since this > >> is the only location where kgit-s2q is used. > >> And we will get a new option in kgit-s2q, that will result in > >> "git am --committer-date-is-author-date" being used. > >> > >> Any name suggestions for the config and option? > >> KERNEL_PATCH_WITH_AUTHOR_DATE > >> KERNEL_PATCH_REPRODUCIBLE_HASH > >> KERNEL_PATCH_REPRODUCIBLE_COMMIT > > > > It is better to abstract it behind the reproducible feature > > description, since there are other kernel reproducibility components > > that exist, or may appear in the future. > > Sure. > > > If you want, I can have a go at the patch, which will allow me to run > > my local tests and put it in my next consolidated pull request. > > That would be great :) > Thanks!
It took me a bit longer than expected, but the lightly tested v1 is here: https://git.yoctoproject.org/poky-contrib/commit/?h=zedd/kernel&id=cfbc3bf788ba1fc7d5f4de1a95c72a66243562e6 And by lightly tested, I mean I can still patch the kernel and things didn't explode in flames. i haven't checked if the commit dates, etc, do what you were looking for. I'm heading out on vacation for the next week, so I won't send this for merging into OE core yet, but feel free to try it out, send feedback and I'll do a v2 when I return (after March 19th). Bruce > > Best regards, > jimmy > > > Bruce > > > >> --patch-with-author-date > >> --apply-patch-with-author-date > >> --committer-date-is-author-date > >> > >> > >> [4] https://patchwork.openembedded.org/patch/138333 > >> [5] https://patchwork.openembedded.org/patch/171479 > >> [6] https://patchwork.openembedded.org/patch/171513 > >> > >> > >> Best regards, > >> jimmy > >> > >> > >>> Bruce > >>> > >>>> > >>>> Best regards, > >>>> jimmy > >>>> > >>>> > >>>>> Bruce > >>>>> > >>>>>> > >>>>>> [1] > >>>>>> https://git.yoctoproject.org/poky/tree/meta/classes/kernel.bbclass?h=honister#n589 > >>>>>> [2] > >>>>>> https://git.yoctoproject.org/meta-freescale/tree/classes/fsl-kernel-localversion.bbclass?h=honister > >>>>>> [3] > >>>>>> https://github.com/OE4T/meta-tegra/blob/honister/recipes-kernel/linux/linux-tegra_4.9.bb -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11031): https://lists.yoctoproject.org/g/linux-yocto/message/11031 Mute This Topic: https://lists.yoctoproject.org/mt/89251386/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/linux-yocto/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
