On Fri, Feb 23, 2024 at 3:05 AM Munehisa Kamata <kama...@amazon.com> wrote:
>
> The pkg-config workaround has been applied for kernel image building, but
> not for module building. So pkg-config variables are different between
> do_compile and do_compile_kernelmodules tasks. It may unnecessary trigger
> rebuilding of a few host tools at the later task.
>
> Especially when CONFIG_DEBUG_INFO_BTF is enabled in the kernel, it may even
> trigger rebuilding vmlinux at do_compile_kernelmodules due to the rebuilt
> host tools such as certs/extract-cert or objtool (on x86). This eventually
> creates an inconsistent set of kernel binaries.
>
> Here is the repro steps:
>
> - Check out nanbield on x86
> - The unexpected rebuild happens on kirkstone or possibly earlier
>
> - Ensure that pahole is available (e.g. via meta-oe)
>
> - Set KERNEL_DEBUG to "True" to properly set up PAHOLE
> e.g.
> $ export KERNEL_DEBUG="True"
> $ export BB_ENV_PASSTHROUGH_ADDITIONS="${BB_ENV_PASSTHROUGH_ADDITIONS}
> KERNEL_DEBUG"
>
> - Enable CONFIG_DEBUG_INFO_BTF=y
> e.g.
> $ bitbake -c menuconfig virtual/kernel
> -> Kernel hacking
> -> Compile-time checks and compiler options
> -> Generate BTF typeinfo
>
> - Build the kernel
> e.g.
> $ bitbake virtual/kernel
>
> The BTF information in the resulting bzImage and kernel modules are
> inconsistent, because the module's BTF information is generated using the
> "second" vmlinux that doesn't have the identical BTF to the "first" vmlinux.
> These modules can't be loaded at runtime due to the BTF mismatch.
>
> This also leads to a build-id mismatch between the installed bzImage and
> vmlinux since the bzImage is created from the first vmlinux, but the
> installed vmlinux is the second one.
>
> $ eu-readelf -n
> tmp/work/qemux86_64-poky-linux/linux-yocto/6.5.13+git/image/boot/{bzImage*,vmlinux*}
> | grep "Build ID"
> Build ID: 4a0d62ee7fef0244950f0f604253729875bea493
> Build ID: fb99b3d91399dbe42bf67ddee59e0f5a0c7f74d9
>
> To avoid the unexpected rebuilding that results in such inconsistency, set
> the same pkg-config variables when building kernel and modules. For kernel
> 5.19 and above, simply set the HOSTPKG_CONFIG in the make command line.
That is indeed not a simple workflow!
In the past, we've always had the existing packageconfig results picked up and
used by later stages of the kernel build which prevented things like this from
happening.
Have you figured out exactly which packageconfig is triggering the rebuild, and
had a look to see if something change such that the existing results
aren't used ?
All that being said, rather than repeating the exports, it would be
better to have
them in a common place, since eventually everything but the HOSTPKG_CONFIG
definition will be dropped.
Have you tried a more class-global export of the values ? We have a
few of those,
but admittedly, we have way more exports that are duplicated between do_compile
and do_compile_kernelmodules(), so the duplicated exports aren't a big issue.
Bruce
>
> Signed-off-by: Munehisa Kamata <kama...@amazon.com>
> ---
>
> v1 -> v2: Rewrote the commit message with the repro steps
>
> meta/classes-recipe/kernel.bbclass | 12 +++++++++---
> 1 file changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/meta/classes-recipe/kernel.bbclass
> b/meta/classes-recipe/kernel.bbclass
> index a76aaee5ba..db4461e551 100644
> --- a/meta/classes-recipe/kernel.bbclass
> +++ b/meta/classes-recipe/kernel.bbclass
> @@ -239,6 +239,8 @@ KERNEL_EXTRA_ARGS ?= ""
> EXTRA_OEMAKE += ' CC="${KERNEL_CC}" LD="${KERNEL_LD}"
> OBJCOPY="${KERNEL_OBJCOPY}" STRIP="${KERNEL_STRIP}"'
> EXTRA_OEMAKE += ' HOSTCC="${BUILD_CC}" HOSTCFLAGS="${BUILD_CFLAGS}"
> HOSTLDFLAGS="${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}"'
> EXTRA_OEMAKE += ' HOSTCXX="${BUILD_CXX}" HOSTCXXFLAGS="${BUILD_CXXFLAGS}"'
> +# Only for newer kernels (5.19+), native pkg-config variables are set for
> older kernels when building kernel and modules
> +EXTRA_OEMAKE += ' HOSTPKG_CONFIG="pkg-config-native"'
>
> KERNEL_ALT_IMAGETYPE ??= ""
>
> @@ -356,9 +358,6 @@ kernel_do_compile() {
> export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR"
> export PKG_CONFIG_SYSROOT_DIR=""
>
> - # for newer kernels (5.19+) there's a dedicated variable
> - export HOSTPKG_CONFIG="pkg-config-native"
> -
> if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then
> # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH
> may not
> # be set....
> @@ -408,6 +407,13 @@ addtask transform_kernel after do_compile before
> do_install
>
> do_compile_kernelmodules() {
> unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
> +
> + # setup native pkg-config variables (kconfig scripts call pkg-config
> directly, cannot generically be overriden to pkg-config-native)
> + export
> PKG_CONFIG_DIR="${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig"
> + export
> PKG_CONFIG_PATH="$PKG_CONFIG_DIR:${STAGING_DATADIR_NATIVE}/pkgconfig"
> + export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR"
> + export PKG_CONFIG_SYSROOT_DIR=""
> +
> if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then
> # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH
> may not
> # be set....
> --
> 2.34.1
>
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196091):
https://lists.openembedded.org/g/openembedded-core/message/196091
Mute This Topic: https://lists.openembedded.org/mt/104524682/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
