Re: [OE-core] [PATCH] bitbake.conf: Add -fcanon-prefix-map to DEBUG_PREFIX_MAP

[Jacob Kroon]

On 4/28/23 05:20, Khem Raj wrote:

This should help canonicalize the relative paths and symlinks
during cross compile, -fcanon-prefix-map is newly added in gcc-13+ [1]

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108464#c8
Signed-off-by: Khem Raj 
---
  meta/conf/bitbake.conf | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index d94ffe1df9..453bef37a9 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -644,7 +644,8 @@ EXTRA_OEMAKE:prepend:task-install = "${PARALLEL_MAKEINST} "
  # Optimization flags.
  ##
  # Beware: applied last to first
-DEBUG_PREFIX_MAP ?= 
"-fmacro-prefix-map=${S}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \
+DEBUG_PREFIX_MAP ?= "-fcanon-prefix-map \
+ -fmacro-prefix-map=${S}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \
   -fdebug-prefix-map=${S}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \
   -fmacro-prefix-map=${B}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \
   -fdebug-prefix-map=${B}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \



Maybe we can take the opportunity to also cleanup DEBUG_PREFIX_MAP by 
replacing debug/macro/canon with a single -ffile-prefix-map ?


From the gcc 13.1.1 manpage:


-ffile-prefix-map=old=new
   When compiling files residing in directory old, record any 
references to them in the result of the
   compilation as if the files resided in directory new instead.  
Specifying this option is equivalent to
   specifying all the individual -f*-prefix-map options.  This can be 
used to make reproducible builds
   that are location independent.  Directories referenced by directives 
are not affected by these options.
   See also -fmacro-prefix-map, -fdebug-prefix-map, 
-fprofile-prefix-map and -fcanon-prefix-map.


Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181789): 
https://lists.openembedded.org/g/openembedded-core/message/181789
Mute This Topic: https://lists.openembedded.org/mt/98551742/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] elfutils: disable deprecation errors in all builds, not just native

[Jacob Kroon]

On 1/6/23 17:16, Ross Burton wrote:

The curl-related deprecation errors affect all builds not just native,
so set CFLAGS instead of BUILD_CFLAGS.

Signed-off-by: Ross Burton 
---
  meta/recipes-devtools/elfutils/elfutils_0.188.bb | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/elfutils/elfutils_0.188.bb 
b/meta/recipes-devtools/elfutils/elfutils_0.188.bb
index c94e36071cd..084908a38c0 100644
--- a/meta/recipes-devtools/elfutils/elfutils_0.188.bb
+++ b/meta/recipes-devtools/elfutils/elfutils_0.188.bb
@@ -34,7 +34,7 @@ EXTRA_OECONF = "--program-prefix=eu-"
  
  BUILD_CFLAGS += "-Wno-error=stringop-overflow"

  # compatibility with curl 7.87; can be removed when elfutils upstream fixes 
the deprecation fails
-BUILD_CFLAGS += "-Wno-error=deprecated-declarations"
+CFLAGS:append = " -Wno-error=deprecated-declarations"
  


Why use ":append" and not "+=" ? I thought the general idea is that "+=" 
is preferred, since it is easier to remove the snippet in a .bbappend, 
so core should avoid :append when possible ?



  DEPENDS_BZIP2 = "bzip2-replacement-native"
  DEPENDS_BZIP2:class-target = "bzip2"






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#175733): 
https://lists.openembedded.org/g/openembedded-core/message/175733
Mute This Topic: https://lists.openembedded.org/mt/96096249/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [AUH] acpid: upgrading to 2.0.34 SUCCEEDED

[Jacob Kroon]

On 10/16/22 14:05, Alexander Kanavin wrote:

On Sun, 16 Oct 2022 at 09:56, Jacob Kroon  wrote:

I do think it is reasonable to ask for not being mail-bombed with
machine generated emails.

I am asking for either to:
* put them on a separate mailing list
* merge them into one email

If keeping core updated still is "Alex's problem" then the AUH emails
are not having the intended effect *anyway*.


They do help, just not to the level I would like. They also provide an
easy way for someone who is not yet a contributor to become one,
particularly when AUH did succeed and produced a working patch that
doesn't need manual fixing. They also make the maintenance problem
known and visible. Sweeping it all under the carpet as suggested would
make the situation worse, not better.



Don't put words in my mouth. I gave two alternatives, not to sweep them 
under the carpet. A third option would be to publish the AUH results on 
some webpage and post the link.



If you do not want to mix those emails with those written by humans, I
trust your email client has a filter system. I still cling to the
belief that you would want to help the core, and only personal
circumstances prevent you, and therefore this filter will be set to
put those mails into a separate folder instead of deleting them
ouright.


Fine, I'll setup filtering to forward all AUH emails to the trashbin then.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#171899): 
https://lists.openembedded.org/g/openembedded-core/message/171899
Mute This Topic: https://lists.openembedded.org/mt/94354924/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [AUH] acpid: upgrading to 2.0.34 SUCCEEDED

[Jacob Kroon]

Hi Alex,

On 10/16/22 09:24, Alexander Kanavin wrote:
We are asking for help here with version updates. Specifically that you 
take at least some of these reports and act on them. If you don’t want 
to do this, fine, but demanding that those reports are not produced 
because deleting them inconveniences you is both unreasonable and 
antagonizing. Keeping core updated should not be ‘Alex problem’ but it 
de facto is.




I do think it is reasonable to ask for not being mail-bombed with 
machine generated emails.


I am asking for either to:
* put them on a separate mailing list
* merge them into one email

If keeping core updated still is "Alex's problem" then the AUH emails 
are not having the intended effect *anyway*.



Alex

On Sun 16. Oct 2022 at 8.50, Jacob Kroon <mailto:jacob.kr...@gmail.com>> wrote:


Another AUH mail bomb, another time where I am pressing "delete" 148
times in my email client. I don't review machine generated emails.
I would not cry if AUH mails were sent to a designated mailing list,
one
that I am not subscribed to.

Jacob



Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#171891): 
https://lists.openembedded.org/g/openembedded-core/message/171891
Mute This Topic: https://lists.openembedded.org/mt/94354924/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [AUH] acpid: upgrading to 2.0.34 SUCCEEDED

[Jacob Kroon]

Another AUH mail bomb, another time where I am pressing "delete" 148 
times in my email client. I don't review machine generated emails.
I would not cry if AUH mails were sent to a designated mailing list, one 
that I am not subscribed to.


Jacob

On 10/16/22 00:03, Auto Upgrade Helper wrote:

Hello,

this email is a notification from the Auto Upgrade Helper
that the automatic attempt to upgrade the recipe *acpid* to *2.0.34* has 
Succeeded.

Next steps:
 - apply the patch: git am 0001-acpid-upgrade-2.0.33-2.0.34.patch
 - check the changes to upstream patches and summarize them in the commit 
message,
 - compile an image that contains the package
 - perform some basic sanity tests
 - amend the patch and sign it off: git commit -s --reset-author --amend
 - send it to the appropriate mailing list

Alternatively, if you believe the recipe should not be upgraded at this time,
you can fill RECIPE_NO_UPDATE_REASON in respective recipe file so that
automatic upgrades would no longer be attempted.

Please review the attached files for further information and build/update 
failures.
Any problem please file a bug at 
https://bugzilla.yoctoproject.org/enter_bug.cgi?product=Automated%20Update%20Handler

Regards,
The Upgrade Helper






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#171889): 
https://lists.openembedded.org/g/openembedded-core/message/171889
Mute This Topic: https://lists.openembedded.org/mt/94354924/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] libxml2: don't override XML_CATALOG_FILES in xmllint wrapper if already set

[Jacob Kroon]

On 9/9/22 19:00, Ross Burton wrote:

  - create_wrapper ${D}${bindir}/xmllint 
XML_CATALOG_FILES=${sysconfdir}/xml/catalog
+   create_wrapper ${D}${bindir}/xmllint 
'XML_CATALOG_FILES=${XML_CATALOG_FILES:-${sysconfdir}/xml/catalog}'


That ...:-${sysconfdir}/... looks odd. Is that hyphen intentional ?


Yes:

  ${parameter:-word}Use Default Values.  If parameter is unset or null, 
the expansion of word is substituted; otherwise, the value of
parameter is substituted.

Ross



Aha, very nice, sorry for the noise.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170503): 
https://lists.openembedded.org/g/openembedded-core/message/170503
Mute This Topic: https://lists.openembedded.org/mt/93577134/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] libxml2: don't override XML_CATALOG_FILES in xmllint wrapper if already set

[Jacob Kroon]

On 9/9/22 18:36, Ross Burton wrote:

The KDE build uses custom catalogs by setting XML_CATALOG_FILES, so this
wrapper should not override that value if it has already been set.

Signed-off-by: Ross Burton 
---
  meta/recipes-core/libxml/libxml2_2.9.14.bb | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/libxml/libxml2_2.9.14.bb 
b/meta/recipes-core/libxml/libxml2_2.9.14.bb
index 2b2289e38a6..165c92d4114 100644
--- a/meta/recipes-core/libxml/libxml2_2.9.14.bb
+++ b/meta/recipes-core/libxml/libxml2_2.9.14.bb
@@ -121,7 +121,7 @@ do_install:append:class-native () {
# Docs are not needed in the native case
rm ${D}${datadir}/gtk-doc -rf
  
-	create_wrapper ${D}${bindir}/xmllint XML_CATALOG_FILES=${sysconfdir}/xml/catalog

+   create_wrapper ${D}${bindir}/xmllint 
'XML_CATALOG_FILES=${XML_CATALOG_FILES:-${sysconfdir}/xml/catalog}'


That ...:-${sysconfdir}/... looks odd. Is that hyphen intentional ?


  }
  
  BBCLASSEXTEND = "native nativesdk"






Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170500): 
https://lists.openembedded.org/g/openembedded-core/message/170500
Mute This Topic: https://lists.openembedded.org/mt/93577134/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] Error with TEMPLATECONF value

[Jacob Kroon]

On 9/1/22 13:49, Jacob Kroon via lists.openembedded.org wrote:

Hi,

After updating to master branches I'm getting:

Error: TEMPLATECONF value (which is 
/home/jkroon/Projects/codab-linux/openembedded-core/meta/conf) must 
point to meta-some-layer/conf/templates/template-name


What is the proper solution to this ?



After wiping build/conf/* and rerunning bitbake the problem seems to 
have resolved itself.


Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170161): 
https://lists.openembedded.org/g/openembedded-core/message/170161
Mute This Topic: https://lists.openembedded.org/mt/93392457/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] Error with TEMPLATECONF value

[Jacob Kroon]

Hi,

After updating to master branches I'm getting:


Error: TEMPLATECONF value (which is 
/home/jkroon/Projects/codab-linux/openembedded-core/meta/conf) must point to 
meta-some-layer/conf/templates/template-name


What is the proper solution to this ?

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170160): 
https://lists.openembedded.org/g/openembedded-core/message/170160
Mute This Topic: https://lists.openembedded.org/mt/93392457/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] python3-cython: Remove debug lines

[Jacob Kroon]

Fixes: ccbbed323b5a96bbdaec4411fdd26cb9dca583e8
   ("python3-cython: Update code to match debug path changes")
Signed-off-by: Jacob Kroon 
---
 meta/recipes-devtools/python/python3-cython_0.29.32.bb | 2 --
 1 file changed, 2 deletions(-)

diff --git a/meta/recipes-devtools/python/python3-cython_0.29.32.bb 
b/meta/recipes-devtools/python/python3-cython_0.29.32.bb
index 28a1386310..8fed1cf94d 100644
--- a/meta/recipes-devtools/python/python3-cython_0.29.32.bb
+++ b/meta/recipes-devtools/python/python3-cython_0.29.32.bb
@@ -29,9 +29,7 @@ cython_fix_sources () {

${PKGD}/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}/Cython/Runtime/refnanny.c \

${PKGD}/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}/Cython/Tempita/_tempita.c \

${PKGD}${libdir}/${PYTHON_DIR}/site-packages/Cython*/SOURCES.txt; do
-   echo $f >> /tmp/rp5
if [ -e $f ]; then
-   echo sed -i -e 
's#${WORKDIR}/Cython-${PV}#/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}#g'  >> 
/tmp/rp5
sed -i -e 
's#${WORKDIR}/Cython-${PV}#/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}#g' $f
fi
done

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169510): 
https://lists.openembedded.org/g/openembedded-core/message/169510
Mute This Topic: https://lists.openembedded.org/mt/93096122/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][PATCH] kernel.bbclass: Add shared_workdir_prepare task

[Jacob Kroon]

On 7/20/22 13:30, Jose Quaresma wrote:

The task do_compile_kernelmodules runs after the shared_workdir and
is installing some files in STAGING_KERNEL_BUILDDIR, this can races
in other recipes that depends on "virtual/kernel:do_shared_workdir"
as the STAGING_KERNEL_BUILDDIR is not fully populated when the
shared_workdir task ends.

To address this issue a new task is added in place of the previows one
so the shared_workdir will run after the do_compile_kernelmodules and
the new shared_workdir_prepare will replce of the old shared_workdir.

Signed-off-by: Jose Quaresma 
---
  meta/classes/kernel.bbclass | 11 +--
  1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 5d2f17c3be..5558769c92 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -504,7 +504,8 @@ do_kernel_version_sanity_check() {
exit 0
  }
  
-addtask shared_workdir after do_compile before do_compile_kernelmodules

+addtask shared_workdir_prepare after do_compile before do_compile_kernelmodules
+addtask shared_workdir after do_compile_kernelmodules
  addtask shared_workdir_setscene
  
  do_shared_workdir_setscene () {

@@ -520,10 +521,16 @@ emit_depmod_pkgdata() {
  
  PACKAGEFUNCS += "emit_depmod_pkgdata"
  
-do_shared_workdir[cleandirs] += " ${STAGING_KERNEL_BUILDDIR}"

  do_shared_workdir () {
cd ${B}
  
+	kerneldir=${STAGING_KERNEL_BUILDDIR}

+}


Does the above do anything actually useful ? I thought neither the 
current workdir or a variable set in a shell function would be preserved 
for the next task ?



+
+do_shared_workdir_prepare[cleandirs] += " ${STAGING_KERNEL_BUILDDIR}"
+do_shared_workdir_prepare () {
+   cd ${B}
+
kerneldir=${STAGING_KERNEL_BUILDDIR}
install -d $kerneldir
  


Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#168794): 
https://lists.openembedded.org/g/openembedded-core/message/168794
Mute This Topic: https://lists.openembedded.org/mt/92502346/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v3 1/2] poky-altcfg: enable usrmerge

[Jacob Kroon]

On 7/13/22 19:53, Richard Purdie wrote:

On Wed, 2022-07-13 at 17:55 +0100, Luca Bocassi wrote:

From: Luca Boccassi 

Ensure it gets tested by the CI

Signed-off-by: Luca Boccassi 
---
  meta-poky/conf/distro/poky-altcfg.conf | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/meta-poky/conf/distro/poky-altcfg.conf 
b/meta-poky/conf/distro/poky-altcfg.conf
index f03306e798..57a76e8bb9 100644
--- a/meta-poky/conf/distro/poky-altcfg.conf
+++ b/meta-poky/conf/distro/poky-altcfg.conf
@@ -9,6 +9,8 @@ DISTROOVERRIDES = "poky:poky-altcfg"
  #DISTROOVERRIDES = "poky:linuxstdbase"
  
  INIT_MANAGER:poky-altcfg = "systemd"

+# systemd will soon require usrmerge
+DISTRO_FEATURES:poky-altcfg += "usrmerge"
  # systemd isn't suitable with musl
  INIT_MANAGER:poky-altcfg:libc-musl = "sysvinit"
  


This blew up builds since it clears everything else out of
DISTRO_FEATURES which isn't what was intended. I've tweaked the patch
and re-queued.



Maybe I'm missing something, but what is the benefit in introducing and 
using the "poky-altcfg" override in this file ? If DISTRO_FEATURES was 
just appended with


DISTRO_FEATURES += "usrmerge"

it would all be fine, since it is after the

require conf/distro/poky.conf

line, right ? And if someone wants to base a distro in 
"poky-altcfg.conf", they would include "poky-altcfg.conf" first in the 
same way, then do customizations.


Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#168030): 
https://lists.openembedded.org/g/openembedded-core/message/168030
Mute This Topic: https://lists.openembedded.org/mt/92361983/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v6 6/7] utils: Add cmdline_shebang_wrapper util.

[Jacob Kroon]

On 6/19/22 21:20, Paulo Neves wrote:

Useful to work around shebang relocation issues, where
shebangs are too long or have arguments in them, thus preventing them
from using the /usr/bin/env shebang.
---
  .../wrapper/cmdline-shebang-wrapper-test.bb   | 21 
  .../recipes-test/wrapper/files/test.awk   |  2 ++
  meta/classes/utils.bbclass| 34 +++
  meta/lib/oeqa/selftest/cases/wrapper.py   | 11 ++
  4 files changed, 68 insertions(+)
  create mode 100644 
meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
  create mode 100644 meta-selftest/recipes-test/wrapper/files/test.awk
  create mode 100644 meta/lib/oeqa/selftest/cases/wrapper.py

diff --git a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb 
b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
new file mode 100644
index 00..c4126a41fc
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
@@ -0,0 +1,21 @@
+SUMMARY = "Check that create_cmdline_shebang works"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = 
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+INHIBIT_DEFAULT_DEPS = "1"
+
+SRC_URI += "file://test.awk"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+install -d ${D}${bindir}
+install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/test
+sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test
+create_cmdline_shebang_wrapper ${D}${bindir}/test
+if [ $(${D}${bindir}/test) != "Don't Panic!" ]; then
+bbfatal "Wrapper is broken"
+else
+bbnote "Wrapper is good"
+fi
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta-selftest/recipes-test/wrapper/files/test.awk 
b/meta-selftest/recipes-test/wrapper/files/test.awk
new file mode 100644
index 00..91429197b1
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/files/test.awk
@@ -0,0 +1,2 @@
+#! @AWK_BIN@ -f
+BEGIN { print "Don't Panic!" }
diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index b4eb3d38ab..b58c22771f 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -184,6 +184,40 @@ END
chmod +x $cmd
  }
  
+create_cmdline_shebang_wrapper () {

+   # Create a wrapper script where commandline options are needed
+   #
+   # These are useful to work around shebang relocation issues, where 
shebangs are too
+   # long or have arguments in them, thus preventing them from using the 
/usr/bin/env
+   # shebang
+   #
+   # Usage: create_cmdline_wrapper FILENAME 
+
+   cmd=$1
+   shift
+
+   echo "Generating wrapper script for $cmd"
+
+   # Strip #! and get remaining interpreter + arg
+   argument="$(sed -ne 's/^#! *//p;q' $cmd)"
+   # strip the shebang from the real script as we do not want it to be 
usable anyway
+   tail -n +2 $cmd > $cmd.real
+   cmdname=$(basename $cmd)
+   dirname=$(dirname $cmd)
+   cmdoptions=$@
+   if [ "${base_prefix}" != "" ]; then
+   relpath=`python3 -c "import os; 
print(os.path.relpath('${D}${base_prefix}', '$dirname'))"`
+   cmdoptions=`echo $@ | sed -e 
"s:${base_prefix}:\\$realdir/$relpath:g"`
+   fi
+   cat <$cmd
+#!/usr/bin/env bash
+realpath=\`readlink -fn \$0\`
+realdir=\`dirname \$realpath\`
+exec -a \$realdir/$cmdname $argument \$realdir/$cmdname.real $cmdoptions "\$@"
+END
+   chmod +x $cmd
+}
+


Maybe this has already been raised before, but the wrapper above does a 
couple of addtitional forks, which could be avoided if we use a python 
wrapper script instead.


Jacob


  create_wrapper () {
# Create a wrapper script where extra environment variables are needed
#
diff --git a/meta/lib/oeqa/selftest/cases/wrapper.py 
b/meta/lib/oeqa/selftest/cases/wrapper.py
new file mode 100644
index 00..6de63310c0
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/wrapper.py
@@ -0,0 +1,11 @@
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake
+
+class WrapperTests(OESelftestTestCase):
+def test_shebang_wrapper(self):
+"""
+Summary:   Build a recipe which will fail if the 
cmdline_shebang_wrapper function is defective.
+Expected:  Exit status to be 0.
+Author:Paulo Neves 
+"""
+res = bitbake("cmdline-shebang-wrapper-test -c install", 
ignore_status=False)






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167638): 
https://lists.openembedded.org/g/openembedded-core/message/167638
Mute This Topic: https://lists.openembedded.org/mt/91863579/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] gperf: Add a patch to work around reproducibility issues

[Jacob Kroon]

On 7/4/22 15:25, Richard Purdie wrote:

Add a patch to avoid writing the full pathname to gperf into source
files which leads to reproducibility issues.

This fixes issues with systemd reproducibility in particular.

Signed-off-by: Richard Purdie 
---
  .../gperf/gperf/reproducibility.patch | 26 +++
  meta/recipes-extended/gperf/gperf_3.1.bb  |  2 ++
  2 files changed, 28 insertions(+)
  create mode 100644 meta/recipes-extended/gperf/gperf/reproducibility.patch

diff --git a/meta/recipes-extended/gperf/gperf/reproducibility.patch 
b/meta/recipes-extended/gperf/gperf/reproducibility.patch
new file mode 100644
index 000..9f80828dbd7
--- /dev/null
+++ b/meta/recipes-extended/gperf/gperf/reproducibility.patch
@@ -0,0 +1,26 @@
+By default gperf puts a header into generated files with the full path to
+the tool along with the commandline used. This patch removes the path to
+the binary, allowing reproducible source files (which can be included in
+debug source packages).
+
+Upstream-Status: Pending
+Signed-off-by: Richard Purdie 
+
+Index: gperf-3.1/src/options.cc
+===
+--- gperf-3.1.orig/src/options.cc
 gperf-3.1/src/options.cc
+@@ -280,6 +280,13 @@ Options::print_options () const
+ {
+   const char *arg = _argument_vector[i];
+
++  if (i == 0) {
++  const char *shortarg = strrchr(arg, '/');
++  if (shortarg) {
++  arg = shortarg + 1;
++  }
++  }
++
+   /* Escape arg if it contains shell metacharacters.  */
+   if (*arg == '-')
+ {
diff --git a/meta/recipes-extended/gperf/gperf_3.1.bb 
b/meta/recipes-extended/gperf/gperf_3.1.bb
index 82750fca05c..3564ac0805b 100644
--- a/meta/recipes-extended/gperf/gperf_3.1.bb
+++ b/meta/recipes-extended/gperf/gperf_3.1.bb
@@ -9,6 +9,8 @@ SRC_URI  = "${GNU_MIRROR}/${BPN}/${BP}.tar.gz"
  SRC_URI[md5sum] = "9e251c0a618ad0824b51117d5d9db87e"
  SRC_URI[sha256sum] = 
"588546b945bba4b70b6a3a616e80b4ab466e3f33024a352fc2198112cdbb3ae2"
  
+SRC_URI:append = " file://reproducibility.patch"

+


Can I ask if :append was really necessary here, couldn't we just have 
used += ? I'm a little saddened with all these :append/:prepend/:remove 
creeping in to OE-Core.


Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167637): 
https://lists.openembedded.org/g/openembedded-core/message/167637
Mute This Topic: https://lists.openembedded.org/mt/92164384/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v1] glibc-locale: work around host-user-contaminated QA failure

[Jacob Kroon]

On 6/16/22 07:54, Muhammad Hamza wrote:

Work around long standing periodic host-user-contaminated QA failure by
explicitly correcting the ownership.

See `glibc-locale: Rewrite do_install using install utility instead of cp`
on the oe-core mailing list for discussion. This should be dropped when
a real fix is implemented.

Explicitly disable host-user-contaminated to further work around the
pseudo bug. With pseudo acting up, even if the ownership is correct, it
may well think it is not, so just sidestep the issue until upstream
fixes the root cause.

Signed-off-by: Christopher Larson 

Signed-off-by: Muhammad Hamza 
---
  meta/recipes-core/glibc/glibc-locale.inc | 17 +
  1 file changed, 17 insertions(+)

diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
b/meta/recipes-core/glibc/glibc-locale.inc
index b8de7d3192..34c178268b 100644
--- a/meta/recipes-core/glibc/glibc-locale.inc
+++ b/meta/recipes-core/glibc/glibc-locale.inc
@@ -69,6 +69,23 @@ FILES:localedef = "${bindir}/localedef"
  
  LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
  
+# Work around long standing periodic host-user-contaminated QA failure by

+# explicitly correcting the ownership.
+#
+# See `glibc-locale: Rewrite do_install using install utility instead of cp`
+# on the oe-core mailing list for discussion. This should be dropped when
+# a real fix is implemented.
+
+do_prep_locale_tree_append () {
+   chown -R root:root $treedir
+}
+
+# Explicitly disable host-user-contaminated to further work around the
+# pseudo bug. With pseudo acting up, even if the ownership is correct,
+# it may well think it is not, so just sidestep the issue until upstream
+# fixes the root cause.
+ERROR_QA_remove = "host-user-contaminated"
+
  copy_locale_files() {
local dir=$1 mode=$2
  



Given the append/remove override syntax used here, I guess this patch is 
aimed at an older Yocto release, and shouldn't be necessary for master ?


Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167001): 
https://lists.openembedded.org/g/openembedded-core/message/167001
Mute This Topic: https://lists.openembedded.org/mt/91793626/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [swat] [OE-core] perl makefile race - any make experts who can help?

[Jacob Kroon]

On Sat, 4 Jun 2022, 19:40 Richard Purdie, <
richard.pur...@linuxfoundation.org> wrote:

> On Sat, 2022-06-04 at 17:12 +0200, Jacob Kroon wrote:
> > On 6/4/22 16:55, Khem Raj wrote:
> > >
> > >
> > > On Sat, Jun 4, 2022 at 6:23 AM Richard Purdie
> > >  > > <mailto:richard.pur...@linuxfoundation.org>> wrote:
> > >
> > > On Sat, 2022-06-04 at 13:36 +0100, Richard Purdie via
> > > lists.yoctoproject.org <http://lists.yoctoproject.org> wrote:
> > >  > On Sat, 2022-06-04 at 13:51 +0200, Alexander Kanavin wrote:
> > >  > > Here's something I didn't think of before. Has this occurred
> > > anywhere
> > >  > > else except Ubuntu 18.04?
> > >  >
> > >  > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14096
> > > <https://bugzilla.yoctoproject.org/show_bug.cgi?id=14096>
> > >  >
> > >  > I'm struggling to get the data out from the old builds, one
> mentions
> > >  > ubuntu1604, there is an ubuntu1804 on both x86 and arm hosts.
> > >  >
> > >  > It is possible this is an ubuntu specific make issue or a make
> bug.
> > >
> > > Ubuntu 18.04 uses make 4.1 which is old (Oct 2014).
> > >
> > > I noticed these patches from 2016:
> > >
> > >
> https://git.savannah.gnu.org/cgit/make.git/commit/?id=9bb994e8319c2b153cd3d6d61e2c2882895e7c3a
> > > <
> https://git.savannah.gnu.org/cgit/make.git/commit/?id=9bb994e8319c2b153cd3d6d61e2c2882895e7c3a
> >
> > >
> https://git.savannah.gnu.org/cgit/make.git/commit/?id=4762480ae9cb8df4878286411f178d32db14eff0
> > > <
> https://git.savannah.gnu.org/cgit/make.git/commit/?id=4762480ae9cb8df4878286411f178d32db14eff0
> >
> > >
> > > I think we may want to mandate a modern make for both this class of
> > > issues and also perhaps for better loadavg support to keep load
> under
> > > control on the autobuilders.
> > >
> > > I'm torn, on the one hand we need to test the distros people use,
> on
> > > the other we do need to remove sources of intermittent issues. I
> think
> > > this bug must be some issue with make itself.
> > >
> > > Adding a make-native dependency to perl would "hurt" people on
> modern
> > > distros...
> > >
> > >
> > > Make perhaps does not have many complex dependency needs so it might
> not
> > > be as bad
> > >
> >
> > My master build is already building make-native due to a dependency from
> > glibc, since 2018:
> >
> >
> https://git.openembedded.org/openembedded-core/commit/?id=0cd89e4af625941f8ab8c033f72f900a2979b304
> >
> > Don't know if that dependency is still valid though.
>
> It is a fair point. We may as well add it to perl/perl-native. Centos7
> still has make 3.82 but I think we now already require buildtools
> tarball there so we could probably drop the glibc dependency on make-
> native now.
>

Would it be a bad idea to add make-native to DEPENDS depending on whether
the host version of make is new enough or not ? Would it break sstate cache
reuse in some way ?

>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166573): 
https://lists.openembedded.org/g/openembedded-core/message/166573
Mute This Topic: https://lists.openembedded.org/mt/91540379/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [swat] [OE-core] perl makefile race - any make experts who can help?

[Jacob Kroon]

On 6/4/22 16:55, Khem Raj wrote:



On Sat, Jun 4, 2022 at 6:23 AM Richard Purdie 
> wrote:


On Sat, 2022-06-04 at 13:36 +0100, Richard Purdie via
lists.yoctoproject.org  wrote:
 > On Sat, 2022-06-04 at 13:51 +0200, Alexander Kanavin wrote:
 > > Here's something I didn't think of before. Has this occurred
anywhere
 > > else except Ubuntu 18.04?
 >
 > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14096

 >
 > I'm struggling to get the data out from the old builds, one mentions
 > ubuntu1604, there is an ubuntu1804 on both x86 and arm hosts.
 >
 > It is possible this is an ubuntu specific make issue or a make bug.

Ubuntu 18.04 uses make 4.1 which is old (Oct 2014).

I noticed these patches from 2016:


https://git.savannah.gnu.org/cgit/make.git/commit/?id=9bb994e8319c2b153cd3d6d61e2c2882895e7c3a



https://git.savannah.gnu.org/cgit/make.git/commit/?id=4762480ae9cb8df4878286411f178d32db14eff0



I think we may want to mandate a modern make for both this class of
issues and also perhaps for better loadavg support to keep load under
control on the autobuilders.

I'm torn, on the one hand we need to test the distros people use, on
the other we do need to remove sources of intermittent issues. I think
this bug must be some issue with make itself.

Adding a make-native dependency to perl would "hurt" people on modern
distros...


Make perhaps does not have many complex dependency needs so it might not 
be as bad




My master build is already building make-native due to a dependency from 
glibc, since 2018:


https://git.openembedded.org/openembedded-core/commit/?id=0cd89e4af625941f8ab8c033f72f900a2979b304

Don't know if that dependency is still valid though.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166570): 
https://lists.openembedded.org/g/openembedded-core/message/166570
Mute This Topic: https://lists.openembedded.org/mt/91540379/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][PATCH] webkitgtk: limit the number of compilation threads

[Jacob Kroon]

On 5/20/22 14:30, Thomas Perrot wrote:

Hello Jacob,

On Fri, 2022-05-20 at 14:19 +0200, Jacob Kroon wrote:

On 5/20/22 14:16, Thomas Perrot via lists.openembedded.org wrote:

Otherwise the compilation may lead to out-of-memories.

Signed-off-by: Thomas Perrot 
---
   meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++-
   1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb
b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb
index 65757c36a733..e7cfe49e2b87 100644
--- a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb
+++ b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb
@@ -80,6 +80,8 @@ setup_python_link() {
 fi
   }
   
+PARALLEL_MAKE = ""

+
   EXTRA_OECMAKE = " \
 -DPORT=GTK \
 ${@bb.utils.contains('GI_DATA_ENABLED', 'True', '-
DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \
@@ -164,4 +166,3 @@ src_package_preprocess () {
   ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h
   
   }

-



Why punish build machines with lots of RAM ? Maybe it could be set
depending on available RAM/number of cores ?



Why not.



Because it slows down the build for them, which is not good.


I tested on a build server with more than ~12GB ram available and 16
cores, then I tried to reduce the number of threads and when the value
is greater than 1 the build fails.



12GB RAM does not sound like much for a 16 core system.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165931): 
https://lists.openembedded.org/g/openembedded-core/message/165931
Mute This Topic: https://lists.openembedded.org/mt/91229473/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][PATCH] webkitgtk: limit the number of compilation threads

[Jacob Kroon]

On 5/20/22 14:16, Thomas Perrot via lists.openembedded.org wrote:

Otherwise the compilation may lead to out-of-memories.

Signed-off-by: Thomas Perrot 
---
  meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb 
b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb
index 65757c36a733..e7cfe49e2b87 100644
--- a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb
+++ b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb
@@ -80,6 +80,8 @@ setup_python_link() {
fi
  }
  
+PARALLEL_MAKE = ""

+
  EXTRA_OECMAKE = " \
-DPORT=GTK \
${@bb.utils.contains('GI_DATA_ENABLED', 'True', 
'-DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \
@@ -164,4 +166,3 @@ src_package_preprocess () {
  ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h
  
  }

-



Why punish build machines with lots of RAM ? Maybe it could be set 
depending on available RAM/number of cores ?


Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165928): 
https://lists.openembedded.org/g/openembedded-core/message/165928
Mute This Topic: https://lists.openembedded.org/mt/91229473/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] selftest/imagefeatures/overlayfs: Always append to DISTRO_FEATURES

[Jacob Kroon]

On 5/16/22 15:01, Quentin Schulz wrote:

Hi Jacob,

On 5/16/22 14:52, Jacob Kroon wrote:

On 5/16/22 14:38, richard.pur...@linuxfoundation.org wrote:

On Mon, 2022-05-16 at 13:43 +0200, Jacob Kroon wrote:

Hi Richard,

On 5/16/22 13:28, Richard Purdie wrote:
Using += unintentionally removes all over entries from 
DISTRO_FEATURES and


"over", you mean "other" ? If so, how is that possible ?


I mean other, I'll fix the typo. It happens due to:

meta-poky/conf/distro/poky.conf:DISTRO_FEATURES ?= 
"${DISTRO_FEATURES_DEFAULT} ${POKY_DEFAULT_DISTRO_FEATURES}"


which interacts badly with "+=" :(

I'm not happy about the situation with variable interactions but still
not entirely sure the best way to proceed.



That is surprising to me. I tried this snippet in a Makefile:

foo ?= bar1



foo += bar2



Same for Bitbake.

However:
foo += bar2

foo ?= bar1

will return " bar2", while make returns "bar2" (note the missing leading 
space).


The issue is where the ?= is located. If it is in a recipe, 
configuration files need to use :append otherwise they will override ?= 
(as configuration files are parsed before recipes).




Right, I see. So the "+="-operator is behaving as expected, but in this 
case the order of the statements forces usage of the append-override :-/


Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165682): 
https://lists.openembedded.org/g/openembedded-core/message/165682
Mute This Topic: https://lists.openembedded.org/mt/91137386/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] selftest/imagefeatures/overlayfs: Always append to DISTRO_FEATURES

[Jacob Kroon]

On 5/16/22 14:38, richard.pur...@linuxfoundation.org wrote:

On Mon, 2022-05-16 at 13:43 +0200, Jacob Kroon wrote:

Hi Richard,

On 5/16/22 13:28, Richard Purdie wrote:

Using += unintentionally removes all over entries from DISTRO_FEATURES and


"over", you mean "other" ? If so, how is that possible ?


I mean other, I'll fix the typo. It happens due to:

meta-poky/conf/distro/poky.conf:DISTRO_FEATURES ?= "${DISTRO_FEATURES_DEFAULT} 
${POKY_DEFAULT_DISTRO_FEATURES}"

which interacts badly with "+=" :(

I'm not happy about the situation with variable interactions but still
not entirely sure the best way to proceed.



That is surprising to me. I tried this snippet in a Makefile:

foo ?= bar1 





foo += bar2 





$(info $(foo))

which produces:

bar1 bar2

which is what I'd expect (maybe because I'm more used to make than bitbake).

Aligning bitbake's and make's handling of variables would make a lot of 
sense to me...


Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165679): 
https://lists.openembedded.org/g/openembedded-core/message/165679
Mute This Topic: https://lists.openembedded.org/mt/91137386/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] selftest/imagefeatures/overlayfs: Always append to DISTRO_FEATURES

[Jacob Kroon]

Hi Richard,

On 5/16/22 13:28, Richard Purdie wrote:

Using += unintentionally removes all over entries from DISTRO_FEATURES and


"over", you mean "other" ? If so, how is that possible ?


this reduces sstate reusage on the autobuilder. Fix this to speed up builds.


/Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165677): 
https://lists.openembedded.org/g/openembedded-core/message/165677
Mute This Topic: https://lists.openembedded.org/mt/91137386/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] About the sstate cache directory hierarchy

[Jacob Kroon]

On 1/28/22 08:06, Chen Qi wrote:

Hi All,

I'm sending out this email because I'm wondering if we can change the 
sstate cache directory to use ${PN} and taskname as subditories. Hope to 
hear your opinions.


Below is the long story.

Recently I noticed that running `bitbake xxx -c cleansstate' usually 
takes more than 10 minutes.


After some investigation, I can see that most of the time is spent on 
file searching. This is because we have:


SSTATE_PATHSPEC   = 
"${SSTATE_DIR}/${SSTATE_EXTRAPATHWILDCARD}${PN}/${SSTATE_PATH_CURRTASK}/${SSTATE_PKGSPEC}*_${SSTATE_PATH_CURRTASK}.tar.zst*" 



And our sstate cache directory's hierarchy uses hash[:2]/hash[2:4]/ as 
sub-directories.


This essentially means that all sub-directories are searched. This would 
take a long time, especially when run for the first time. I made some 
changes to  output the time and the logs are as below.


$ bitbake glibc -c cleansstate
WARNING: glibc-2.34-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/*/*/sstate:glibc:core2-64-poky-linux:2.34:r0:core2-64:7:*_deploy_source_date_epoch.tar.zst* 


WARNING: glibc-2.34-r0 do_cleansstate: Took 611.8865714073181 seconds
WARNING: glibc-2.34-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/*/*/sstate:glibc:core2-64-poky-linux:2.34:r0:core2-64:7:*_package.tar.zst* 


WARNING: glibc-2.34-r0 do_cleansstate: Took 1.3219327926635742 seconds
WARNING: glibc-2.34-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/*/*/sstate:glibc:core2-64-poky-linux:2.34:r0:core2-64:7:*_package_qa.tar.zst* 


WARNING: glibc-2.34-r0 do_cleansstate: Took 1.470815658569336 seconds
WARNING: glibc-2.34-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/*/*/sstate:glibc:core2-64-poky-linux:2.34:r0:core2-64:7:*_package_write_rpm.tar.zst* 


WARNING: glibc-2.34-r0 do_cleansstate: Took 1.251939058303833 seconds
WARNING: glibc-2.34-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/*/*/sstate:glibc:core2-64-poky-linux:2.34:r0:core2-64:7:*_packagedata.tar.zst* 


WARNING: glibc-2.34-r0 do_cleansstate: Took 1.2369801998138428 seconds
WARNING: glibc-2.34-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/*/*/sstate:glibc::2.34:r0::7:*_populate_lic.tar.zst* 


WARNING: glibc-2.34-r0 do_cleansstate: Took 1.1668426990509033 seconds
WARNING: glibc-2.34-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/*/*/sstate:glibc:core2-64-poky-linux:2.34:r0:core2-64:7:*_populate_sysroot.tar.zst* 


WARNING: glibc-2.34-r0 do_cleansstate: Took 1.385568380355835 seconds
WARNING: glibc-2.34-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/*/*/sstate:glibc:core2-64-poky-linux:2.34:r0:core2-64:7:*_stash_locale.tar.zst* 


WARNING: glibc-2.34-r0 do_cleansstate: Took 1.4884181022644043 seconds

I figured that unlike git, we do have knowledge on our sstate objects. 
It does not seem necessary to use hash value as sub directory. So I 
changed the sstate directory hierarchy to use ${PN}/taskname/ as sub 
directories, and here's the result.


$ bitbake libgcc -c cleansstate

WARNING: libgcc-11.2.0-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/libgcc/deploy_source_date_epoch/sstate:libgcc:core2-64-poky-linux:11.2.0:r0:core2-64:7:*_deploy_source_date_epoch.tar.zst* 


WARNING: libgcc-11.2.0-r0 do_cleansstate: Took 0.020630598068237305 seconds
WARNING: libgcc-11.2.0-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/libgcc/package/sstate:libgcc:core2-64-poky-linux:11.2.0:r0:core2-64:7:*_package.tar.zst* 

WARNING: libgcc-11.2.0-r0 do_cleansstate: Took 0.0011608600616455078 
seconds
WARNING: libgcc-11.2.0-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/libgcc/package_qa/sstate:libgcc:core2-64-poky-linux:11.2.0:r0:core2-64:7:*_package_qa.tar.zst* 

WARNING: libgcc-11.2.0-r0 do_cleansstate: Took 0.0007557868957519531 
seconds
WARNING: libgcc-11.2.0-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/libgcc/package_write_rpm/sstate:libgcc:core2-64-poky-linux:11.2.0:r0:core2-64:7:*_package_write_rpm.tar.zst* 

WARNING: libgcc-11.2.0-r0 do_cleansstate: Took 0.0013995170593261719 
seconds
WARNING: libgcc-11.2.0-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/libgcc/packagedata/sstate:libgcc:core2-64-poky-linux:11.2.0:r0:core2-64:7:*_packagedata.tar.zst* 

WARNING: libgcc-11.2.0-r0 do_cleansstate: Took 0.0007488727569580078 
seconds
WARNING: libgcc-11.2.0-r0 do_cleansstate: Removing 
/ala-lpggp72/qichen/LAT/builds/share/sstate-cache/libgcc/populate_lic/sstate:libgcc::11.2.0:r0::7:*_populate_lic.tar.zst* 

WARNING: libgcc-11.2.0-r0 do_cleansstate: Took 0.0005896091461181641 
seconds
WARNING: libgcc-11.2.0-r0 do_cleansstate: Removing 

[OE-core] [PATCH] Revert "image.bbclass: allow overriding dependency on virtual/kernel:do_deploy"

[Jacob Kroon]

As pointed out in
https://lists.openembedded.org/g/openembedded-core/message/165058
https://lists.openembedded.org/g/openembedded-core/message/165216
this patch sets KERNELDEPLOYDEPEND but then uses KERNELDEPMODDEPEND.

Revert the changes since no one seems interested enough to fix it.
If someone wants this then make the variable name readable by
adding underscores where appropriate, for example by calling it
KERNEL_DEPLOY_DEPEND.

This reverts commit dcf9dfa4e6305786cd713aa28deda94a50bd6635.

Signed-off-by: Jacob Kroon 
---
 meta/classes/image.bbclass | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 47776db2b0..7f1f6f80a4 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -132,12 +132,7 @@ def rootfs_variables(d):
 
 do_rootfs[vardeps] += "${@rootfs_variables(d)}"
 
-# This is needed to have kernel image in DEPLOY_DIR.
-# This follow many common usecases and user expectations.
-# But if you are building an image which doesn't need the kernel image at all,
-# you can unset this variable manually.
-KERNELDEPLOYDEPEND ?= "virtual/kernel:do_deploy"
-do_build[depends] += "${KERNELDEPMODDEPEND}"
+do_build[depends] += "virtual/kernel:do_deploy"
 
 
 python () {

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165390): 
https://lists.openembedded.org/g/openembedded-core/message/165390
Mute This Topic: https://lists.openembedded.org/mt/90986084/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] image.bbclass: allow overriding dependency on virtual/kernel:do_deploy

[Jacob Kroon]

Hi Dmitry,

Do you want to send a patch for fixing this ? Otherwise I'll send a
patch reverting the changes.

Jacob

On 4/29/22 22:53, Jacob Kroon wrote:
> On 4/27/22 09:37, Dmitry Baryshkov wrote:
>> Since the commit fe26b2379ecd ("image.bbclass: Depend on
>> virtual/kernel:do_deploy"), the image.bbclass made building images
>> depend on virtual/kernel. For some images, including small initramfs,
>> this is not the case. Allow overriding this dependency in case
>> developers knows what they are doing.
>>
>> Signed-off-by: Dmitry Baryshkov 
>> ---
>>  meta/classes/image.bbclass | 7 ++-
>>  1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
>> index 7f1f6f80a464..47776db2b0e6 100644
>> --- a/meta/classes/image.bbclass
>> +++ b/meta/classes/image.bbclass
>> @@ -132,7 +132,12 @@ def rootfs_variables(d):
>>  
>>  do_rootfs[vardeps] += "${@rootfs_variables(d)}"
>>  
>> -do_build[depends] += "virtual/kernel:do_deploy"
>> +# This is needed to have kernel image in DEPLOY_DIR.
>> +# This follow many common usecases and user expectations.
>> +# But if you are building an image which doesn't need the kernel image at 
>> all,
>> +# you can unset this variable manually.
>> +KERNELDEPLOYDEPEND ?= "virtual/kernel:do_deploy"
>> +do_build[depends] += "${KERNELDEPMODDEPEND}"
>>  
> 
> I saw this got merged to master.
> 
> The patch doesn't make sense.
> 
> It sets
> 
> KERNELDEPLOYDEPEND
> 
> then uses
> 
> KERNELDEPMODDEPEND
> 
> ?
> 
> And please make it readable by adding some underscores, like
> 
> KERNEL_DEPLOY_DEPEND
> 
> Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165216): 
https://lists.openembedded.org/g/openembedded-core/message/165216
Mute This Topic: https://lists.openembedded.org/mt/90726077/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] image.bbclass: allow overriding dependency on virtual/kernel:do_deploy

[Jacob Kroon]

On 4/27/22 09:37, Dmitry Baryshkov wrote:
> Since the commit fe26b2379ecd ("image.bbclass: Depend on
> virtual/kernel:do_deploy"), the image.bbclass made building images
> depend on virtual/kernel. For some images, including small initramfs,
> this is not the case. Allow overriding this dependency in case
> developers knows what they are doing.
> 
> Signed-off-by: Dmitry Baryshkov 
> ---
>  meta/classes/image.bbclass | 7 ++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 7f1f6f80a464..47776db2b0e6 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -132,7 +132,12 @@ def rootfs_variables(d):
>  
>  do_rootfs[vardeps] += "${@rootfs_variables(d)}"
>  
> -do_build[depends] += "virtual/kernel:do_deploy"
> +# This is needed to have kernel image in DEPLOY_DIR.
> +# This follow many common usecases and user expectations.
> +# But if you are building an image which doesn't need the kernel image at 
> all,
> +# you can unset this variable manually.
> +KERNELDEPLOYDEPEND ?= "virtual/kernel:do_deploy"
> +do_build[depends] += "${KERNELDEPMODDEPEND}"
>  

I saw this got merged to master.

The patch doesn't make sense.

It sets

KERNELDEPLOYDEPEND

then uses

KERNELDEPMODDEPEND

?

And please make it readable by adding some underscores, like

KERNEL_DEPLOY_DEPEND

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165058): 
https://lists.openembedded.org/g/openembedded-core/message/165058
Mute This Topic: https://lists.openembedded.org/mt/90726077/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] staging: Ensure we filter out ourselves

[Jacob Kroon]

On 4/27/22 17:25, Joshua Watt wrote:
> Reviewed-by: Joshua Watt 
> 
> On 4/27/22 10:23, Richard Purdie wrote:
>> Adding a dependency on ourselves in this function doesn't make sense,
>> the hash
>> may change after hash equivalence is applied. Other code using
>> BB_TASKDEPDATA does
>> handle the self reference correctly (which is there for a reason),
>> update this
>> code to do likewise.
>>
>> Signed-off-by: Richard Purdie 
>> ---
>>   meta/classes/staging.bbclass | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
>> index ab827766bef..9fc8f4f2839 100644
>> --- a/meta/classes/staging.bbclass
>> +++ b/meta/classes/staging.bbclass
>> @@ -651,7 +651,7 @@ python target_add_sysroot_deps () {
>>   taskdepdata = d.getVar("BB_TASKDEPDATA", False)
>>   deps = {}
>>   for dep in taskdepdata.values():
>> -    if dep[1] == "do_populate_sysroot" and not
>> dep[0].endswith(("-native", "-initial")) and "-cross-" not in dep[0]:
>> +    if dep[1] == "do_populate_sysroot" and not
>> dep[0].endswith(("-native", "-initial")) and "-cross-" not in dep[0]
>> and dep[0] != pn:
>>   deps[dep[0]] = dep[6]
>>     d.setVar("HASHEQUIV_EXTRA_SIGDATA", "\n".join("%s: %s" % (k,
>> deps[k]) for k in sorted(deps.keys(
>>

Thanks Richard/Joshua. With the patch applied everything behaves nicely
here again.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164930): 
https://lists.openembedded.org/g/openembedded-core/message/164930
Mute This Topic: https://lists.openembedded.org/mt/90733144/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2] zlib: Add patch to fix building icedtea7-native from meta-java

[Jacob Kroon]

Signed-off-by: Jacob Kroon 
---
Changes in v2:
* Add SoB tag in patch

 ...t-inputs-provided-to-the-CRC-functio.patch | 54 +++
 meta/recipes-core/zlib/zlib_1.2.12.bb |  1 +
 2 files changed, 55 insertions(+)
 create mode 100644 
meta/recipes-core/zlib/zlib/0001-Correct-incorrect-inputs-provided-to-the-CRC-functio.patch

diff --git 
a/meta/recipes-core/zlib/zlib/0001-Correct-incorrect-inputs-provided-to-the-CRC-functio.patch
 
b/meta/recipes-core/zlib/zlib/0001-Correct-incorrect-inputs-provided-to-the-CRC-functio.patch
new file mode 100644
index 00..ad5e59de04
--- /dev/null
+++ 
b/meta/recipes-core/zlib/zlib/0001-Correct-incorrect-inputs-provided-to-the-CRC-functio.patch
@@ -0,0 +1,54 @@
+From ec3df00224d4b396e2ac6586ab5d25f673caa4c2 Mon Sep 17 00:00:00 2001
+From: Mark Adler 
+Date: Wed, 30 Mar 2022 11:14:53 -0700
+Subject: [PATCH] Correct incorrect inputs provided to the CRC functions.
+
+The previous releases of zlib were not sensitive to incorrect CRC
+inputs with bits set above the low 32. This commit restores that
+behavior, so that applications with such bugs will continue to
+operate as before.
+
+Upstream-Status: Backport 
[https://github.com/madler/zlib/commit/ec3df00224d4b396e2ac6586ab5d25f673caa4c2]
+Signed-off-by: Jacob Kroon 
+---
+ crc32.c | 8 
+ 1 file changed, 4 insertions(+), 4 deletions(-)
+
+diff --git a/crc32.c b/crc32.c
+index a1bdce5..451887b 100644
+--- a/crc32.c
 b/crc32.c
+@@ -630,7 +630,7 @@ unsigned long ZEXPORT crc32_z(crc, buf, len)
+ #endif /* DYNAMIC_CRC_TABLE */
+ 
+ /* Pre-condition the CRC */
+-crc ^= 0x;
++crc = (~crc) & 0x;
+ 
+ /* Compute the CRC up to a word boundary. */
+ while (len && ((z_size_t)buf & 7) != 0) {
+@@ -749,7 +749,7 @@ unsigned long ZEXPORT crc32_z(crc, buf, len)
+ #endif /* DYNAMIC_CRC_TABLE */
+ 
+ /* Pre-condition the CRC */
+-crc ^= 0x;
++crc = (~crc) & 0x;
+ 
+ #ifdef W
+ 
+@@ -1077,7 +1077,7 @@ uLong ZEXPORT crc32_combine64(crc1, crc2, len2)
+ #ifdef DYNAMIC_CRC_TABLE
+ once(, make_crc_table);
+ #endif /* DYNAMIC_CRC_TABLE */
+-return multmodp(x2nmodp(len2, 3), crc1) ^ crc2;
++return multmodp(x2nmodp(len2, 3), crc1) ^ (crc2 & 0x);
+ }
+ 
+ /* = 
*/
+@@ -1112,5 +1112,5 @@ uLong crc32_combine_op(crc1, crc2, op)
+ uLong crc2;
+ uLong op;
+ {
+-return multmodp(op, crc1) ^ crc2;
++return multmodp(op, crc1) ^ (crc2 & 0x);
+ }
diff --git a/meta/recipes-core/zlib/zlib_1.2.12.bb 
b/meta/recipes-core/zlib/zlib_1.2.12.bb
index 95e873584b..e921703137 100644
--- a/meta/recipes-core/zlib/zlib_1.2.12.bb
+++ b/meta/recipes-core/zlib/zlib_1.2.12.bb
@@ -11,6 +11,7 @@ SRC_URI = "https://zlib.net/${BP}.tar.xz \
file://ldflags-tests.patch \
file://0001-configure-Pass-LDFLAGS-to-link-tests.patch \
file://run-ptest \
+   
file://0001-Correct-incorrect-inputs-provided-to-the-CRC-functio.patch \
"
 UPSTREAM_CHECK_URI = "http://zlib.net/;
 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164882): 
https://lists.openembedded.org/g/openembedded-core/message/164882
Mute This Topic: https://lists.openembedded.org/mt/90717214/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] zlib: Add patch to fix building icedtea7-native from meta-java

[Jacob Kroon]

Signed-off-by: Jacob Kroon 
---
 ...t-inputs-provided-to-the-CRC-functio.patch | 53 +++
 meta/recipes-core/zlib/zlib_1.2.12.bb |  1 +
 2 files changed, 54 insertions(+)
 create mode 100644 
meta/recipes-core/zlib/zlib/0001-Correct-incorrect-inputs-provided-to-the-CRC-functio.patch

diff --git 
a/meta/recipes-core/zlib/zlib/0001-Correct-incorrect-inputs-provided-to-the-CRC-functio.patch
 
b/meta/recipes-core/zlib/zlib/0001-Correct-incorrect-inputs-provided-to-the-CRC-functio.patch
new file mode 100644
index 00..6bd36d56d2
--- /dev/null
+++ 
b/meta/recipes-core/zlib/zlib/0001-Correct-incorrect-inputs-provided-to-the-CRC-functio.patch
@@ -0,0 +1,53 @@
+From ec3df00224d4b396e2ac6586ab5d25f673caa4c2 Mon Sep 17 00:00:00 2001
+From: Mark Adler 
+Date: Wed, 30 Mar 2022 11:14:53 -0700
+Subject: [PATCH] Correct incorrect inputs provided to the CRC functions.
+
+The previous releases of zlib were not sensitive to incorrect CRC
+inputs with bits set above the low 32. This commit restores that
+behavior, so that applications with such bugs will continue to
+operate as before.
+
+Upstream-Status: Backport 
[https://github.com/madler/zlib/commit/ec3df00224d4b396e2ac6586ab5d25f673caa4c2]
+---
+ crc32.c | 8 
+ 1 file changed, 4 insertions(+), 4 deletions(-)
+
+diff --git a/crc32.c b/crc32.c
+index a1bdce5..451887b 100644
+--- a/crc32.c
 b/crc32.c
+@@ -630,7 +630,7 @@ unsigned long ZEXPORT crc32_z(crc, buf, len)
+ #endif /* DYNAMIC_CRC_TABLE */
+ 
+ /* Pre-condition the CRC */
+-crc ^= 0x;
++crc = (~crc) & 0x;
+ 
+ /* Compute the CRC up to a word boundary. */
+ while (len && ((z_size_t)buf & 7) != 0) {
+@@ -749,7 +749,7 @@ unsigned long ZEXPORT crc32_z(crc, buf, len)
+ #endif /* DYNAMIC_CRC_TABLE */
+ 
+ /* Pre-condition the CRC */
+-crc ^= 0x;
++crc = (~crc) & 0x;
+ 
+ #ifdef W
+ 
+@@ -1077,7 +1077,7 @@ uLong ZEXPORT crc32_combine64(crc1, crc2, len2)
+ #ifdef DYNAMIC_CRC_TABLE
+ once(, make_crc_table);
+ #endif /* DYNAMIC_CRC_TABLE */
+-return multmodp(x2nmodp(len2, 3), crc1) ^ crc2;
++return multmodp(x2nmodp(len2, 3), crc1) ^ (crc2 & 0x);
+ }
+ 
+ /* = 
*/
+@@ -1112,5 +1112,5 @@ uLong crc32_combine_op(crc1, crc2, op)
+ uLong crc2;
+ uLong op;
+ {
+-return multmodp(op, crc1) ^ crc2;
++return multmodp(op, crc1) ^ (crc2 & 0x);
+ }
diff --git a/meta/recipes-core/zlib/zlib_1.2.12.bb 
b/meta/recipes-core/zlib/zlib_1.2.12.bb
index 95e873584b..e921703137 100644
--- a/meta/recipes-core/zlib/zlib_1.2.12.bb
+++ b/meta/recipes-core/zlib/zlib_1.2.12.bb
@@ -11,6 +11,7 @@ SRC_URI = "https://zlib.net/${BP}.tar.xz \
file://ldflags-tests.patch \
file://0001-configure-Pass-LDFLAGS-to-link-tests.patch \
file://run-ptest \
+   
file://0001-Correct-incorrect-inputs-provided-to-the-CRC-functio.patch \
"
 UPSTREAM_CHECK_URI = "http://zlib.net/;
 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164875): 
https://lists.openembedded.org/g/openembedded-core/message/164875
Mute This Topic: https://lists.openembedded.org/mt/90707645/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] zlib: upgrade to 1.2.12

[Jacob Kroon]

On 3/29/22 15:45, Ross Burton wrote:
> First upstream release since 2017!
> - Fix a deflate bug when using the Z_FIXED strategy that can result in 
> out-of-bound accesses.
> - Fix a deflate bug when the window is full in deflate_stored().
> - Speed up CRC-32 computations by a factor of 1.5 to 3.
> - Use the hardware CRC-32 instruction on ARMv8 processors.
> - Speed up crc32_combine() with powers of x tables.
> - Add crc32_combine_gen() and crc32_combine_op() for fast combines.
> 
> Drop CVE-2018-25032 as this is in the .12 release.
> 
> Rebase 0001-configure-Pass-LDFLAGS-to-link-tests.patch to apply cleanly.
> 
> Backport cc.patch to fix compilation with our CC.
> 
> Signed-off-by: Ross Burton 

For whatever reason, this breaks building icedtea7-native from meta-java
on my Fedora 35 system.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164758): 
https://lists.openembedded.org/g/openembedded-core/message/164758
Mute This Topic: https://lists.openembedded.org/mt/90108253/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] default-distrovars.inc: Switch connectivity check to a yoctoproject.org page

[Jacob Kroon]

fwiw, +1 from me too

On 2/11/22 20:27, Ross Burton wrote:
> +1
> 
> On Fri, 11 Feb 2022 at 17:46, Richard Purdie
>  wrote:
>>
>> example.com is proving unreliable at present so switch to our own 
>> connectivity
>> page instead. That page is very simple avoiding app overhead on our web 
>> server
>> which was an original reason for switching to example.com.
>>
>> Signed-off-by: Richard Purdie 
>> ---
>>  meta/classes/sanity.bbclass | 2 +-
>>  meta/conf/distro/include/default-distrovars.inc | 2 +-
>>  2 files changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
>> index f288b4c84c9..773902e619d 100644
>> --- a/meta/classes/sanity.bbclass
>> +++ b/meta/classes/sanity.bbclass
>> @@ -353,7 +353,7 @@ def check_connectivity(d):
>>  msg += "Please ensure your host's network is configured 
>> correctly.\n"
>>  msg += "If your ISP or network is blocking the above 
>> URL,\n"
>>  msg += "try with another domain name, for example by 
>> setting:\n"
>> -msg += "CONNECTIVITY_CHECK_URIS = 
>> \"https://www.yoctoproject.org/\";
>> +msg += "CONNECTIVITY_CHECK_URIS = 
>> \"https://www.example.com/\";
>>  msg += "You could also set BB_NO_NETWORK = \"1\" to 
>> disable network\n"
>>  msg += "access if all required sources are on local 
>> disk.\n"
>>  retval = msg
>> diff --git a/meta/conf/distro/include/default-distrovars.inc 
>> b/meta/conf/distro/include/default-distrovars.inc
>> index fb0f1097da9..3bba651a776 100644
>> --- a/meta/conf/distro/include/default-distrovars.inc
>> +++ b/meta/conf/distro/include/default-distrovars.inc
>> @@ -54,4 +54,4 @@ KERNEL_IMAGETYPES ??= "${KERNEL_IMAGETYPE}"
>>  # fetch from the network (and warn you if not). To disable the test set
>>  # the variable to be empty.
>>  # Git example url: 
>> git://git.yoctoproject.org/yocto-firewall-test;protocol=git;rev=master;branch=master
>> -CONNECTIVITY_CHECK_URIS ?= "https://www.example.com/;
>> +CONNECTIVITY_CHECK_URIS ?= "https://yoctoproject.org/connectivity.html;
>> --
>> 2.32.0
>>
>>
>>
>>
>>
>>
>> 
>>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161663): 
https://lists.openembedded.org/g/openembedded-core/message/161663
Mute This Topic: https://lists.openembedded.org/mt/89077055/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] sanity.class: Add possibility to configure networking check

[Jacob Kroon]

On 2/11/22 12:39, Ross Burton wrote:
> On Fri, 11 Feb 2022 at 11:23, Jacob Kroon  wrote:
>> I vote for removing the check altogether.
>>
>> As I understand it, the check was added in order to prevent
>> hard-to-interpret error messages when network failed. I can live with that.
> 
> I think the check should remain, and example.com has served well for
> some time now (since 2016 in
> 423a2645377bff2cd7292e1dc9796eeb3595e937).
> 
> We moved to example.com to speed up the check: the old tests were for
> two yoctoproject.org URLs and it downloads the entire page, so that
> took time.
> 
> If we can get a minimal page added to yoctoproject.org that doesn't
> involve hitting WordPress, then we can own the URL and have fast
> responses.
> 

I suppose if the intention is that the Yocto server will never ever be
down, the check is ok. But if it is ever going to need to go down
because of maintenance you will break bitbake builds for everyone in the
world, right ? Then I don't think users will be happy, even though the
downtime was planned and publicly announced on some website.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161646): 
https://lists.openembedded.org/g/openembedded-core/message/161646
Mute This Topic: https://lists.openembedded.org/mt/89066994/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] sanity.class: Add possibility to configure networking check

[Jacob Kroon]

On 2/11/22 12:18, Ross Burton wrote:
> On Fri, 11 Feb 2022 at 08:44, Pavel Zhukov  wrote:
>> +for uri in test_uris:
>> +try:
>> +if check_all:
>> +fetcher = bb.fetch2.Fetch(test_uris, data)
>> +else:
>> +fetcher = bb.fetch2.Fetch([uri], data)
>> +fetcher.checkstatus()
>> +return retval
> 
> This seems like it's overcomplicating things to me.  example.com was
> having uptime issues, and we can change it to www.yoctoproject.org so
> that we know in advance of expected downtime, and have direct contact
> with the people responsible for keeping it up. So let's just change
> the URL to www.yoctoproject.org instead of adding more variables and
> non-trivial logic.
> 

I vote for removing the check altogether.

As I understand it, the check was added in order to prevent
hard-to-interpret error messages when network failed. I can live with that.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161642): 
https://lists.openembedded.org/g/openembedded-core/message/161642
Mute This Topic: https://lists.openembedded.org/mt/89066994/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [poky][master][PATCHv2] buildhistory.bbclass: Enable exporting more recipe and package data

[Jacob Kroon]

On 2/11/22 09:00, Mikko Rapeli wrote:
> Hi,
> 
> On Fri, Feb 11, 2022 at 04:49:21PM +1300, Paul Eggleton wrote:
>> FWIW I'll just chime in here as the original author[1] and say I agree with 
>> Richard. If folks are needing an alternative SBOM generation mechanism to 
>> SPDX, or have other use cases for extracting build information, then I'd 
>> prefer we take a step back and design such a thing properly. The original 
>> idea 
>> was (1) to expose changes in the output that might not readily apparent 
>> otherwise, and (2) expose selected "input" values that would (or could) 
>> materially influence the output, so you hopefully have a ready explanation 
>> for 
>> why an image / package increased in size or dependencies got added etc. 
> 
> First of all, thanks for buildhistory, Paul!
> 
>> I will accept that over time buildhistory has added things here and there 
>> that 
>> further the idea of using it for SBOM, folks no doubt have been using these 
>> for such purposes, and I'm not 100% opposed to making small additions where 
>> they make sense. However, even with this proposed extension, buildhistory is 
>> still not capable of recording sufficient information that (I believe) is 
>> expected in a modern SBOM - for example, the list of patches applied / CVEs 
>> resolved as a result - and I don't think it would be wise to extend it to do 
>> so. We certainly don't want to give folks the idea that buildhistory is good 
>> enough to resolve their SBOM requirements when we haven't designed it to do 
>> so, particularly as for many folks there are legal and regulatory concerns 
>> involved.
> 
> Maybe I'm "abusing" buildhistory then. The way it enables to have a git repo
> view with full history of releases on metadata available in SW builds is
> really invaluable. We seem to be abusing it to export various bitbake recipe
> variables from product configuration so that we can later on do some QA checks
> on them and record the details in nice git history view.
> 
> There could be other ways to implement the same use cases. We could have real
> QA checks in the bitbake builds which make sure certain recipe variables
> are correctly filled. We could export them into image manifest files or even 
> SDPX
> to record the details per release. Then we could manually diff the files 
> between
> releases to check for delta. Or a custom data format, maybe in json.
> 
> Yocto CVE checker uses a custom text format, a bit similar to buildhistory as 
> output.
> There is the json patch from Microsoft in review.
> 
> Currently I don't see SPDX output replacing buildhistory and CVE checker 
> output completely.
> 
> Then there is the current discussion about ABI dumps and how to store and 
> export them
> and buildhistory was at least initially mentioned there.
> 
> Gah, now I thought about saving SPDX, CVE checker and ABI dump output in 
> buildhistory too.
> Would actually be nice. Sorry, yet another use case where easy access build 
> metadata
> to compare over time and releases would be really handy. This never ends.
> You've created a monster :)
> 

I want to add file checksums to that wishlist.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161636): 
https://lists.openembedded.org/g/openembedded-core/message/161636
Mute This Topic: https://lists.openembedded.org/mt/89018266/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] sstate: Preserve permissions when extracting tar archive

[Jacob Kroon]

On 1/17/22 09:40, Richard Purdie wrote:
> On Sun, 2022-01-16 at 23:28 -0800, Jacob Kroon wrote:
>> Although this is now fixed in master, I was wondering why this doesn't seem 
>> to
>> be a problem in dunfell.
>> In dunfell, sstate packages are unpacked with "tar -xvzf", no -p flag, but
>> still
>> my buildhistory doesn't show any noise when doing:
>>
>> bitbake -c cleansstate shadow-native && bitbake shadow-native && bitbake -c
>> clean shadow-native && bitbake shadow-native
>>
>> After dunfell was the switch to tar+zstd, but I don't understand why that
>> would
>> change the extracted permissions.
>>
>> If anyone has got a clue, i'd be interested to know why.
> 
> I think dunfell doesn't have the umask changes I linked to earlier in the
> discussion? That means it will behave differently but there are probably other
> permissions issues e.g. a host with your current umask compared to a host 
> with a
> different one, particularly around the group bits.
> 

Yes, that is true. I'll just leave it at that, thanks for clarifying.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160637): 
https://lists.openembedded.org/g/openembedded-core/message/160637
Mute This Topic: https://lists.openembedded.org/mt/88416502/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] sstate: Preserve permissions when extracting tar archive

[Jacob Kroon]

Although this is now fixed in master, I was wondering why this doesn't seem to 
be a problem in dunfell.
In dunfell, sstate packages are unpacked with "tar -xvzf", no -p flag, but 
still my buildhistory doesn't show any noise when doing:

bitbake -c cleansstate shadow-native && bitbake shadow-native && bitbake -c 
clean shadow-native && bitbake shadow-native

After dunfell was the switch to tar+zstd, but I don't understand why that would 
change the extracted permissions.

If anyone has got a clue, i'd be interested to know why.

Jacob
(testing the message reply function from the archived mailing list)

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160633): 
https://lists.openembedded.org/g/openembedded-core/message/160633
Mute This Topic: https://lists.openembedded.org/mt/88416502/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][RFC] classes/native: Propagate dependencies to outhash

[Jacob Kroon]

On 1/14/22 18:12, Joshua Watt wrote:
> Native task outputs are directly run on the target (host) system after

"target" or "host" ? the latter i suppose

> being built. Even if the output of a native recipe doesn't change, a
> change in one of its dependencies may cause a change in the output it
> generates (e.g. rpm output depends on the output of its dependent zstd
> library).
> 
> This can cause poor interactions with hash equivalence, since this
> recipes output-changing dependency is "hidden" and downstream task only
> see that this recipe has the same outhash and therefore is equivalent.
> This can result in different output in different cases.
> 
> To resolve this, unhide the output-changing dependency by adding it's
> unihash to this tasks outhash calculation. Unfortunately, don't know
> specifically know which dependencies are output-changing, so we have to
> add all of them.
> 

"don't know specifically know which.."

> [YOCTO #14685]
> 
> Signed-off-by: Joshua Watt 
> ---
>  meta/classes/native.bbclass | 31 +++
>  meta/lib/oe/sstatesig.py| 10 +++---
>  2 files changed, 38 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/classes/native.bbclass b/meta/classes/native.bbclass
> index 76a599bc15..fc7422c5d7 100644
> --- a/meta/classes/native.bbclass
> +++ b/meta/classes/native.bbclass
> @@ -195,3 +195,34 @@ USE_NLS = "no"
>  
>  RECIPERDEPTASK = "do_populate_sysroot"
>  do_populate_sysroot[rdeptask] = "${RECIPERDEPTASK}"
> +
> +#
> +# Native task outputs are directly run on the target (host) system after 
> being

see above

> +# built. Even if the output of this recipe doesn't change, a change in one of
> +# its dependencies may cause a change in the output it generates (e.g. rpm
> +# output depends on the output of its dependent zstd library).
> +#
> +# This can cause poor interactions with hash equivalence, since this recipes
> +# output-changing dependency is "hidden" and downstream task only see that 
> this
> +# recipe has the same outhash and therefore is equivalent. This can result in
> +# different output in different cases.
> +#
> +# To resolve this, unhide the output-changing dependency by adding its 
> unihash
> +# to this tasks outhash calculation. Unfortunately, don't know specifically
> +# know which dependencies are output-changing, so we have to add all of them.
> +#

see above

> +python native_add_do_populate_sysroot_deps () {
> +current_task = "do_" + d.getVar("BB_CURRENTTASK")
> +if current_task != "do_populate_sysroot":
> +return
> +
> +taskdepdata = d.getVar("BB_TASKDEPDATA", False)
> +pn = d.getVar("PN")
> +deps = {
> +dep[0]:dep[6] for dep in taskdepdata.values() if
> +dep[1] == current_task and dep[0] != pn
> +}
> +
> +d.setVar("HASHEQUIV_EXTRA_SIGDATA", "\n".join("%s: %s" % (k, deps[k]) 
> for k in sorted(deps.keys(
> +}
> +SSTATECREATEFUNCS += "native_add_do_populate_sysroot_deps"
> diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
> index 038404e377..abcd96231e 100644
> --- a/meta/lib/oe/sstatesig.py
> +++ b/meta/lib/oe/sstatesig.py
> @@ -491,7 +491,8 @@ def OEOuthashBasic(path, sigfile, task, d):
>  if task == "package":
>  include_timestamps = True
>  include_root = False
> -extra_content = d.getVar('HASHEQUIV_HASH_VERSION')
> +hash_version = d.getVar('HASHEQUIV_HASH_VERSION')
> +extra_sigdata = d.getVar("HASHEQUIV_EXTRA_SIGDATA")
>  
>  filemaps = {}
>  for m in (d.getVar('SSTATE_HASHEQUIV_FILEMAP') or '').split():
> @@ -506,8 +507,11 @@ def OEOuthashBasic(path, sigfile, task, d):
>  basepath = os.path.normpath(path)
>  
>  update_hash("OEOuthashBasic\n")
> -if extra_content:
> -update_hash(extra_content + "\n")
> +if hash_version:
> +update_hash(hash_version + "\n")
> +
> +if extra_sigdata:
> +update_hash(extra_sigdata + "\n")
>  
>  # It is only currently useful to get equivalent hashes for things 
> that
>  # can be restored from sstate. Since the sstate object is named using
> 
> 
> 
> 
> 

Sounds to me like something we should do.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160573): 
https://lists.openembedded.org/g/openembedded-core/message/160573
Mute This Topic: https://lists.openembedded.org/mt/88425608/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] sstate: Preserve permissions when extracting tar archive

[Jacob Kroon]

This is done by default when tar is run by the superuser, but for native
recipes the corresponding task is not run as root under pseudo, so pass
the flag explicitly.

Suggested-by: Richard Purdie 
Signed-off-by: Jacob Kroon 
---
 meta/classes/sstate.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 645377fdd8..8ee32dba2d 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -901,7 +901,7 @@ sstate_unpack_package () {
ZSTD="pzstd -p ${ZSTD_THREADS}"
fi
 
-   tar -I "$ZSTD" -xvf ${SSTATE_PKG}
+   tar -I "$ZSTD" -xvpf ${SSTATE_PKG}
# update .siginfo atime on local/NFS mirror if it is a symbolic link
[ ! -h ${SSTATE_PKG}.siginfo ] || touch -a ${SSTATE_PKG}.siginfo 
2>/dev/null || true
# update each symbolic link instead of any referenced file

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160559): 
https://lists.openembedded.org/g/openembedded-core/message/160559
Mute This Topic: https://lists.openembedded.org/mt/88416502/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] Zero umask when unpacking sstate packages

[Jacob Kroon]

On 1/13/22 23:20, Richard Purdie wrote:
> On Thu, 2022-01-13 at 23:11 +0100, Jacob Kroon wrote:
>> On 1/13/22 23:07, Richard Purdie wrote:
>>> On Thu, 2022-01-13 at 16:44 +0100, Jacob Kroon wrote:
>>>> Hi,
>>>>
>>>> I often see this diff churn in my buildistory for shadow-native (and
>>>> similar issues with icedtea7-native from meta-java):
>>>>
>>>> -drwxr-xr-x -  -  40 ./var/spool/mail
>>>> +drwxrwxr-x -  -  40 ./var/spool/mail
>>>>
>>>> One can reproduce it with:
>>>>
>>>> # bitbake -c cleansstate shadow-native && \
>>>>   bitbake shadow-native && \
>>>>   bitbake -c clean shadow-native && \
>>>>   bitbake shadow-native
>>>>
>>>> I see that the sstate package contains the correct permissions for
>>>> 'mail', 775. So it would seem to me that it is the unpacking from sstate
>>>> that strips the group write permission.
>>>>
>>>> Testing with the attached patch and the problem goes away.
>>>>
>>>> Is something like this the correct solution ?
>>>
>>> Is this with master?
>>>
>>> I'm wondering if your code has:
>>>
>>> https://git.yoctoproject.org/poky/commit/?id=c4ecf7c1122380cdbc74fe692aa91756dc5bdf6b
>>>
>>> applied?
>>>
>>
>> Yes, this is with master branches of bitbake/openembedded-core as of
>> right now.
>>
>> So yes, I do have that patch applied.
> 
> I guess that whilst the default task umask is 022, some chmod operations may 
> be
> run to change permissions that don't match the umask. These are correctly
> captured by the sstate archive but not preserved by tar at extraction.
> 
> I wonder if the correct fix is to all -p to the tar unpack command in
> sstate.bbclass?
> 
> In many cases it would execute under pseudo so there wouldn't be this issue 
> but
> a native populate_sysroot wouldn't be under pseudo do we'd only see that 
> here...
> 

Thanks Richard, passing -p in sstate_unpack_package() also fixes the
issue. If it is run as root under pseudo for target packages, then the
-p is implicit according to the tar docs, which would explain why the
issue isn't seen there.

I'll send a new patch.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160558): 
https://lists.openembedded.org/g/openembedded-core/message/160558
Mute This Topic: https://lists.openembedded.org/mt/88399226/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] Zero umask when unpacking sstate packages

[Jacob Kroon]

On 1/13/22 23:07, Richard Purdie wrote:
> On Thu, 2022-01-13 at 16:44 +0100, Jacob Kroon wrote:
>> Hi,
>>
>> I often see this diff churn in my buildistory for shadow-native (and
>> similar issues with icedtea7-native from meta-java):
>>
>> -drwxr-xr-x -  -  40 ./var/spool/mail
>> +drwxrwxr-x -  -  40 ./var/spool/mail
>>
>> One can reproduce it with:
>>
>> # bitbake -c cleansstate shadow-native && \
>>   bitbake shadow-native && \
>>   bitbake -c clean shadow-native && \
>>   bitbake shadow-native
>>
>> I see that the sstate package contains the correct permissions for
>> 'mail', 775. So it would seem to me that it is the unpacking from sstate
>> that strips the group write permission.
>>
>> Testing with the attached patch and the problem goes away.
>>
>> Is something like this the correct solution ?
> 
> Is this with master?
> 
> I'm wondering if your code has:
> 
> https://git.yoctoproject.org/poky/commit/?id=c4ecf7c1122380cdbc74fe692aa91756dc5bdf6b
> 
> applied?
> 

Yes, this is with master branches of bitbake/openembedded-core as of
right now.

So yes, I do have that patch applied.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160536): 
https://lists.openembedded.org/g/openembedded-core/message/160536
Mute This Topic: https://lists.openembedded.org/mt/88399226/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] Zero umask when unpacking sstate packages

[Jacob Kroon]

Hi,

I often see this diff churn in my buildistory for shadow-native (and
similar issues with icedtea7-native from meta-java):

-drwxr-xr-x -  -  40 ./var/spool/mail
+drwxrwxr-x -  -  40 ./var/spool/mail

One can reproduce it with:

# bitbake -c cleansstate shadow-native && \
  bitbake shadow-native && \
  bitbake -c clean shadow-native && \
  bitbake shadow-native

I see that the sstate package contains the correct permissions for
'mail', 775. So it would seem to me that it is the unpacking from sstate
that strips the group write permission.

Testing with the attached patch and the problem goes away.

Is something like this the correct solution ?

Jacob
From dcebf2977cb6bdc5198ab13272cf358c9ecd6b32 Mon Sep 17 00:00:00 2001
From: Jacob Kroon 
Date: Thu, 13 Jan 2022 16:31:47 +0100
Subject: [PATCH] sstate.bbclass: Zero umask when unpacking from sstate cache

Signed-off-by: Jacob Kroon 
---
 meta/classes/sstate.bbclass | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 645377fdd8..0c32387f43 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -793,7 +793,9 @@ pstaging_fetch[vardepsexclude] += "SRCPV"
 
 def sstate_setscene(d):
 shared_state = sstate_state_fromvars(d)
+omask = os.umask(0)
 accelerate = sstate_installpkg(shared_state, d)
+os.umask(omask)
 if not accelerate:
 bb.fatal("No suitable staging package found")
 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160525): 
https://lists.openembedded.org/g/openembedded-core/message/160525
Mute This Topic: https://lists.openembedded.org/mt/88399226/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] do_prepare_recipe_sysroot takes >5min @100% CPU

[Jacob Kroon]

Den 2022-01-04 kl. 18:47, skrev Andreas Müller:

Hi,

with relatively actual states of layers/bitbake I see huge delays in
my builds on tasks do_prepare_recipe_sysroot and sometimes or do_patch
- even if recipe does not have patches.

The time the tasks take is 5-10min and each of them load one CPU with
100%. Is this something others have noticed or even better: any hint
how to get over?



Some years ago I ran into

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13843

It is probably not related, but maybe you can use gdb/py-bt in a similar 
way and get a Python backtrace so as to at least get a hint of why it is 
taking so long.


Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160176): 
https://lists.openembedded.org/g/openembedded-core/message/160176
Mute This Topic: https://lists.openembedded.org/mt/88195633/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] libsdl2: Move to CMake build

[Jacob Kroon]

On 1/4/22 00:44, Andreas Müller wrote:
> Signed-off-by: Andreas Müller 
> ---
>  .../libsdl2/libsdl2_2.0.18.bb | 47 +++
>  1 file changed, 18 insertions(+), 29 deletions(-)
> 
> diff --git a/meta/recipes-graphics/libsdl2/libsdl2_2.0.18.bb 
> b/meta/recipes-graphics/libsdl2/libsdl2_2.0.18.bb
> index 5e645b443c..dbc270d858 100644
> --- a/meta/recipes-graphics/libsdl2/libsdl2_2.0.18.bb
> +++ b/meta/recipes-graphics/libsdl2/libsdl2_2.0.18.bb
> @@ -24,20 +24,19 @@ S = "${WORKDIR}/SDL2-${PV}"
>  
>  SRC_URI[sha256sum] = 
> "94d40cd73dbfa10bb6eadfbc28f355992bb2d6ef6761ad9d4074eff95ee5711c"
>  
> -inherit autotools lib_package binconfig-disabled pkgconfig
> +inherit cmake lib_package binconfig-disabled pkgconfig
>  
>  BINCONFIG = "${bindir}/sdl2-config"
>  
>  CVE_PRODUCT = "simple_directmedia_layer sdl"
>  
> -EXTRA_OECONF = "--disable-oss --disable-esd --disable-arts \
> ---disable-diskaudio --disable-nas --disable-esd-shared 
> --disable-esdtest \
> ---disable-video-dummy \
> ---disable-video-rpi \
> ---enable-pthreads \
> ---disable-rpath \
> ---disable-sndio \
> ---disable-fcitx --disable-ibus \
> +EXTRA_OECMAKE = "-DSDL_OSS==OFF -DSDL_ESD=OFF -DSDL_ARTS=OFF \

^^^
s/==/= ?

> + -DSDL_DISKAUDIO=OFF -DSDL_NAS=OFF -DSDL_ESD_SHARED=OFF \
> + -DSDL_DUMMYVIDEO=OFF \
> + -DSDL_RPI=OFF \
> + -DSDL_PTHREADS=ON \
> + -DSDL_RPATH=OFF \
> + -DSDL_SNDIO=OFF \
>  "
>  
>  # opengl packageconfig factored out to make it easy for distros
> @@ -52,27 +51,17 @@ PACKAGECONFIG ??= " \
>  ${@bb.utils.contains('DISTRO_FEATURES', 'wayland', 'wayland gles2', '', 
> d)} \
>  ${@bb.utils.contains("TUNE_FEATURES", "neon","arm-neon","",d)} \
>  "
> -PACKAGECONFIG[alsa]   = "--enable-alsa 
> --disable-alsatest,--disable-alsa,alsa-lib,"
> -PACKAGECONFIG[arm-neon]   = "--enable-arm-neon,--disable-arm-neon"
> -PACKAGECONFIG[directfb]   = 
> "--enable-video-directfb,--disable-video-directfb,directfb,directfb"
> -PACKAGECONFIG[gles2]  = 
> "--enable-video-opengles,--disable-video-opengles,virtual/libgles2"
> -PACKAGECONFIG[jack]   = "--enable-jack,--disable-jack,jack"
> -PACKAGECONFIG[kmsdrm] = 
> "--enable-video-kmsdrm,--disable-video-kmsdrm,libdrm virtual/libgbm"
> -PACKAGECONFIG[opengl] = 
> "--enable-video-opengl,--disable-video-opengl,virtual/libgl"
> -PACKAGECONFIG[pulseaudio] = 
> "--enable-pulseaudio,--disable-pulseaudio,pulseaudio"
> -PACKAGECONFIG[wayland]= 
> "--enable-video-wayland,--disable-video-wayland,wayland-native wayland 
> wayland-protocols libxkbcommon"
> -PACKAGECONFIG[x11]= 
> "--enable-video-x11,--disable-video-x11,virtual/libx11 libxext libxrandr 
> libxrender"
> +PACKAGECONFIG[alsa]   = "-DSDL_ALSA=ON,-DSDL_ALSA=OFF,alsa-lib,"
> +PACKAGECONFIG[arm-neon]   = "-DSDL_ARMNEON=ON,-DSDL_ARMNEON=OFF"
> +PACKAGECONFIG[directfb]   = 
> "-DSDL_DIRECTFB=ON,-DSDL_DIRECTFB=OFF,directfb,directfb"
> +PACKAGECONFIG[gles2]  = 
> "-DSDL_OPENGLES=ON,-DSDL_OPENGLES=OFF,virtual/libgles2"
> +PACKAGECONFIG[jack]   = "-DSDL_JACK=ON,-DSDL_JACK=OFF,jack"
> +PACKAGECONFIG[kmsdrm] = "-DSDL_KMSDRM=ON,-DSDL_KMSDRM=OFF,libdrm 
> virtual/libgbm"
> +PACKAGECONFIG[opengl] = "-DSDL_OPENGL=ON,-DSDL_OPENGL=OFF,virtual/libgl"
> +PACKAGECONFIG[pulseaudio] = 
> "-DSDL_PULSEAUDIO=ON,-DSDL_PULSEAUDIO=OFF,pulseaudio"
> +PACKAGECONFIG[wayland]= 
> "-DSDL_WAYLAND=ON,-DSDL_WAYLAND=OFF,wayland-native wayland wayland-protocols 
> libxkbcommon"
> +PACKAGECONFIG[x11]= "-DSDL_X11=ON,-DSDL_X11=OFF,virtual/libx11 
> libxext libxrandr libxrender"
>  
> -EXTRA_AUTORECONF += "--include=acinclude --exclude=autoheader"
>  CFLAGS:append:class-native = " -DNO_SHARED_MEMORY"
>  
> -do_configure:prepend() {
> -# Remove old libtool macros.
> -MACROS="libtool.m4 lt~obsolete.m4 ltoptions.m4 ltsugar.m4 
> ltversion.m4"
> -for i in ${MACROS}; do
> -   rm -f ${S}/acinclude/$i
> -done
> -export SYSROOT=$PKG_CONFIG_SYSROOT_DIR
> -}
> -
>  BBCLASSEXTEND = "native nativesdk"
> 
> 
> 
> 
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160156): 
https://lists.openembedded.org/g/openembedded-core/message/160156
Mute This Topic: https://lists.openembedded.org/mt/88180295/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][dunfell 02/14] openssh: Fix CVE-2021-41617

[Jacob Kroon]

On Thu, 30 Dec 2021, 21:17 Steve Sakoman,  wrote:

On Thu, Dec 30, 2021 at 9:04 AM Jacob Kroon  wrote:
>
> On 12/30/21 19:54, Jacob Kroon via lists.openembedded.org wrote:
> > On 12/22/21 15:12, Steve Sakoman wrote:
> >> From: sana kazi 
> >>
> >> Add patch to fix CVE-2021-41617
> >> Link: https://bugzilla.suse.com/attachment.cgi?id=854015
> >>
> >> Signed-off-by: Sana Kazi 
> >> Signed-off-by: Sana Kazi 
> >> Signed-off-by: Steve Sakoman 
> >> ---
> >>  .../openssh/openssh/CVE-2021-41617.patch  | 52 +++
> >>  .../openssh/openssh_8.2p1.bb  |  1 +
> >>  2 files changed, 53 insertions(+)
> >>  create mode 100644
meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch
> >>
> >> diff --git
a/meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch
b/meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch
> >> new file mode 100644
> >> index 00..bda896f581
> >> --- /dev/null
> >> +++ b/meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch
> >> @@ -0,0 +1,52 @@
> >> +From a6414400ec94a17871081f7df24f910a6ee01b8b Mon Sep 17 00:00:00 2001
> >> +From: Ali Abdallah 
> >> +Date: Wed, 24 Nov 2021 13:33:39 +0100
> >> +Subject: [PATCH] CVE-2021-41617 fix
> >> +
> >> +backport of the following two upstream commits
> >> +
> >> +f3cbe43e28fe71427d41cfe3a17125b972710455
> >> +bf944e3794eff5413f2df1ef37cddf96918c6bde
> >> +
> >> +CVE-2021-41617 failed to correctly initialise supplemental groups
> >> +when executing an AuthorizedKeysCommand or
AuthorizedPrincipalsCommand,
> >> +where a AuthorizedKeysCommandUser or AuthorizedPrincipalsCommandUser
> >> +directive has been set to run the command as a different user. Instead
> >> +these commands would inherit the groups that sshd(8) was started with.
> >> +---
> >> + auth.c | 8 
> >> + 1 file changed, 8 insertions(+)
> >> +
> >> +CVE: CVE-2021-41617
> >> +Upstream-Status: Backport [
https://bugzilla.suse.com/attachment.cgi?id=854015]
> >> +Comment: No change in any hunk
> >> +Signed-off-by: Sana Kazi 
> >> +
> >> +diff --git a/auth.c b/auth.c
> >> +index 163038f..a47b267 100644
> >> +--- a/auth.c
> >>  b/auth.c
> >> +@@ -52,6 +52,7 @@
> >> + #include 
> >> + #include 
> >> + #include 
> >> ++#include 
> >> +
> >> + #include "xmalloc.h"
> >> + #include "match.h"
> >> +@@ -851,6 +852,13 @@ subprocess(const char *tag, struct passwd *pw,
const char *command,
> >> +}
> >> +closefrom(STDERR_FILENO + 1);
> >> +
> >> ++   if (geteuid() == 0 &&
> >> ++   initgroups(pw->pw_name, pw->pw_gid) == -1) {
> >> ++   error("%s: initgroups(%s, %u): %s", tag,
> >> ++   pw->pw_name, (u_int)pw->pw_gid,
strerror(errno));
> >> ++   _exit(1);
> >> ++   }
> >> ++
> >> +/* Don't use permanently_set_uid() here to avoid fatal()
*/
> >> +if (setresgid(pw->pw_gid, pw->pw_gid, pw->pw_gid) == -1) {
> >> +error("%s: setresgid %u: %s", tag,
(u_int)pw->pw_gid,
> >> +--
> >> +2.26.2
> >> diff --git a/meta/recipes-connectivity/openssh/openssh_8.2p1.bb
 b/meta/recipes-connectivity/openssh/openssh_8.2p1.bb
> >> index b60d1a6bd4..e903ec487d 100644
> >> --- a/meta/recipes-connectivity/openssh/openssh_8.2p1.bb
> >> +++ b/meta/recipes-connectivity/openssh/openssh_8.2p1.bb
> >> @@ -26,6 +26,7 @@ SRC_URI = "
http://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-${PV}.tar
> >> file://add-test-support-for-busybox.patch \
> >> file://CVE-2020-14145.patch \
> >> file://CVE-2021-28041.patch \
> >> +   file://CVE-2021-41617.patch \
> >> "
> >>  SRC_URI[md5sum] = "3076e6413e8dbe56d33848c1054ac091"
> >>  SRC_URI[sha256sum] =
"43925151e6cf6cee1450190c0e9af4dc36b41c12737619edff8bcebdff64e671"
> >>
> >>
> >>
> >>
> >>
> >
> > I would have expected this patch to leave a mark in my buildhistory, but
> > nothing related to openssh(d) shows up.
> >
> > Size of /usr/sbin/sshd stays the same, which at least to me is a lit

Re: [OE-core][dunfell 02/14] openssh: Fix CVE-2021-41617

[Jacob Kroon]

On 12/30/21 19:54, Jacob Kroon via lists.openembedded.org wrote:
> On 12/22/21 15:12, Steve Sakoman wrote:
>> From: sana kazi 
>>
>> Add patch to fix CVE-2021-41617
>> Link: https://bugzilla.suse.com/attachment.cgi?id=854015
>>
>> Signed-off-by: Sana Kazi 
>> Signed-off-by: Sana Kazi 
>> Signed-off-by: Steve Sakoman 
>> ---
>>  .../openssh/openssh/CVE-2021-41617.patch  | 52 +++
>>  .../openssh/openssh_8.2p1.bb  |  1 +
>>  2 files changed, 53 insertions(+)
>>  create mode 100644 
>> meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch
>>
>> diff --git a/meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch 
>> b/meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch
>> new file mode 100644
>> index 00..bda896f581
>> --- /dev/null
>> +++ b/meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch
>> @@ -0,0 +1,52 @@
>> +From a6414400ec94a17871081f7df24f910a6ee01b8b Mon Sep 17 00:00:00 2001
>> +From: Ali Abdallah 
>> +Date: Wed, 24 Nov 2021 13:33:39 +0100
>> +Subject: [PATCH] CVE-2021-41617 fix
>> +
>> +backport of the following two upstream commits
>> +
>> +f3cbe43e28fe71427d41cfe3a17125b972710455
>> +bf944e3794eff5413f2df1ef37cddf96918c6bde
>> +
>> +CVE-2021-41617 failed to correctly initialise supplemental groups
>> +when executing an AuthorizedKeysCommand or AuthorizedPrincipalsCommand,
>> +where a AuthorizedKeysCommandUser or AuthorizedPrincipalsCommandUser
>> +directive has been set to run the command as a different user. Instead
>> +these commands would inherit the groups that sshd(8) was started with.
>> +---
>> + auth.c | 8 
>> + 1 file changed, 8 insertions(+)
>> +
>> +CVE: CVE-2021-41617
>> +Upstream-Status: Backport 
>> [https://bugzilla.suse.com/attachment.cgi?id=854015]
>> +Comment: No change in any hunk
>> +Signed-off-by: Sana Kazi 
>> +
>> +diff --git a/auth.c b/auth.c
>> +index 163038f..a47b267 100644
>> +--- a/auth.c
>>  b/auth.c
>> +@@ -52,6 +52,7 @@
>> + #include 
>> + #include 
>> + #include 
>> ++#include 
>> + 
>> + #include "xmalloc.h"
>> + #include "match.h"
>> +@@ -851,6 +852,13 @@ subprocess(const char *tag, struct passwd *pw, const 
>> char *command,
>> +}
>> +closefrom(STDERR_FILENO + 1);
>> + 
>> ++   if (geteuid() == 0 &&
>> ++   initgroups(pw->pw_name, pw->pw_gid) == -1) {
>> ++   error("%s: initgroups(%s, %u): %s", tag,
>> ++   pw->pw_name, (u_int)pw->pw_gid, strerror(errno));
>> ++   _exit(1);
>> ++   }
>> ++
>> +/* Don't use permanently_set_uid() here to avoid fatal() */
>> +if (setresgid(pw->pw_gid, pw->pw_gid, pw->pw_gid) == -1) {
>> +error("%s: setresgid %u: %s", tag, (u_int)pw->pw_gid,
>> +-- 
>> +2.26.2
>> diff --git a/meta/recipes-connectivity/openssh/openssh_8.2p1.bb 
>> b/meta/recipes-connectivity/openssh/openssh_8.2p1.bb
>> index b60d1a6bd4..e903ec487d 100644
>> --- a/meta/recipes-connectivity/openssh/openssh_8.2p1.bb
>> +++ b/meta/recipes-connectivity/openssh/openssh_8.2p1.bb
>> @@ -26,6 +26,7 @@ SRC_URI = 
>> "http://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-${PV}.tar
>> file://add-test-support-for-busybox.patch \
>> file://CVE-2020-14145.patch \
>> file://CVE-2021-28041.patch \
>> +   file://CVE-2021-41617.patch \
>> "
>>  SRC_URI[md5sum] = "3076e6413e8dbe56d33848c1054ac091"
>>  SRC_URI[sha256sum] = 
>> "43925151e6cf6cee1450190c0e9af4dc36b41c12737619edff8bcebdff64e671"
>>
>>
>>
>>
>>
> 
> I would have expected this patch to leave a mark in my buildhistory, but
> nothing related to openssh(d) shows up.
> 
> Size of /usr/sbin/sshd stays the same, which at least to me is a little
> odd.. but I can see that the sha256sum output of sshd changes.
> 
> (It would be nice to have sha256sum hashes of files in buildhistory)
> 
> Am I the only one who thinks this is a little strange ?
> 
> /Jacob
> 

Let me rephrase, I do see changes related to debug information and the
debug package, but no change in the resulting '/usr/sbin/sshd' size that
goes in the final image.

/Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160064): 
https://lists.openembedded.org/g/openembedded-core/message/160064
Mute This Topic: https://lists.openembedded.org/mt/87898179/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][dunfell 02/14] openssh: Fix CVE-2021-41617

[Jacob Kroon]

On 12/22/21 15:12, Steve Sakoman wrote:
> From: sana kazi 
> 
> Add patch to fix CVE-2021-41617
> Link: https://bugzilla.suse.com/attachment.cgi?id=854015
> 
> Signed-off-by: Sana Kazi 
> Signed-off-by: Sana Kazi 
> Signed-off-by: Steve Sakoman 
> ---
>  .../openssh/openssh/CVE-2021-41617.patch  | 52 +++
>  .../openssh/openssh_8.2p1.bb  |  1 +
>  2 files changed, 53 insertions(+)
>  create mode 100644 
> meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch
> 
> diff --git a/meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch 
> b/meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch
> new file mode 100644
> index 00..bda896f581
> --- /dev/null
> +++ b/meta/recipes-connectivity/openssh/openssh/CVE-2021-41617.patch
> @@ -0,0 +1,52 @@
> +From a6414400ec94a17871081f7df24f910a6ee01b8b Mon Sep 17 00:00:00 2001
> +From: Ali Abdallah 
> +Date: Wed, 24 Nov 2021 13:33:39 +0100
> +Subject: [PATCH] CVE-2021-41617 fix
> +
> +backport of the following two upstream commits
> +
> +f3cbe43e28fe71427d41cfe3a17125b972710455
> +bf944e3794eff5413f2df1ef37cddf96918c6bde
> +
> +CVE-2021-41617 failed to correctly initialise supplemental groups
> +when executing an AuthorizedKeysCommand or AuthorizedPrincipalsCommand,
> +where a AuthorizedKeysCommandUser or AuthorizedPrincipalsCommandUser
> +directive has been set to run the command as a different user. Instead
> +these commands would inherit the groups that sshd(8) was started with.
> +---
> + auth.c | 8 
> + 1 file changed, 8 insertions(+)
> +
> +CVE: CVE-2021-41617
> +Upstream-Status: Backport 
> [https://bugzilla.suse.com/attachment.cgi?id=854015]
> +Comment: No change in any hunk
> +Signed-off-by: Sana Kazi 
> +
> +diff --git a/auth.c b/auth.c
> +index 163038f..a47b267 100644
> +--- a/auth.c
>  b/auth.c
> +@@ -52,6 +52,7 @@
> + #include 
> + #include 
> + #include 
> ++#include 
> + 
> + #include "xmalloc.h"
> + #include "match.h"
> +@@ -851,6 +852,13 @@ subprocess(const char *tag, struct passwd *pw, const 
> char *command,
> + }
> + closefrom(STDERR_FILENO + 1);
> + 
> ++if (geteuid() == 0 &&
> ++initgroups(pw->pw_name, pw->pw_gid) == -1) {
> ++error("%s: initgroups(%s, %u): %s", tag,
> ++pw->pw_name, (u_int)pw->pw_gid, strerror(errno));
> ++_exit(1);
> ++}
> ++
> + /* Don't use permanently_set_uid() here to avoid fatal() */
> + if (setresgid(pw->pw_gid, pw->pw_gid, pw->pw_gid) == -1) {
> + error("%s: setresgid %u: %s", tag, (u_int)pw->pw_gid,
> +-- 
> +2.26.2
> diff --git a/meta/recipes-connectivity/openssh/openssh_8.2p1.bb 
> b/meta/recipes-connectivity/openssh/openssh_8.2p1.bb
> index b60d1a6bd4..e903ec487d 100644
> --- a/meta/recipes-connectivity/openssh/openssh_8.2p1.bb
> +++ b/meta/recipes-connectivity/openssh/openssh_8.2p1.bb
> @@ -26,6 +26,7 @@ SRC_URI = 
> "http://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-${PV}.tar
> file://add-test-support-for-busybox.patch \
> file://CVE-2020-14145.patch \
> file://CVE-2021-28041.patch \
> +   file://CVE-2021-41617.patch \
> "
>  SRC_URI[md5sum] = "3076e6413e8dbe56d33848c1054ac091"
>  SRC_URI[sha256sum] = 
> "43925151e6cf6cee1450190c0e9af4dc36b41c12737619edff8bcebdff64e671"
> 
> 
> 
> 
> 

I would have expected this patch to leave a mark in my buildhistory, but
nothing related to openssh(d) shows up.

Size of /usr/sbin/sshd stays the same, which at least to me is a little
odd.. but I can see that the sha256sum output of sshd changes.

(It would be nice to have sha256sum hashes of files in buildhistory)

Am I the only one who thinks this is a little strange ?

/Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160063): 
https://lists.openembedded.org/g/openembedded-core/message/160063
Mute This Topic: https://lists.openembedded.org/mt/87898179/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] no-static-libs.inc: Fixes

[Jacob Kroon]

 * pciutils/libcap/libpcap all seem to build fine even with the flag set
 * Disable static libraries in libjpeg-turbo-native

Signed-off-by: Jacob Kroon 
---
 meta/conf/distro/include/no-static-libs.inc | 9 +
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/meta/conf/distro/include/no-static-libs.inc 
b/meta/conf/distro/include/no-static-libs.inc
index 7c6cf35934..ee67383460 100644
--- a/meta/conf/distro/include/no-static-libs.inc
+++ b/meta/conf/distro/include/no-static-libs.inc
@@ -5,14 +5,6 @@ DISABLE_STATIC:pn-qemu = ""
 DISABLE_STATIC:pn-qemu-native = ""
 DISABLE_STATIC:pn-nativesdk-qemu = ""
 DISABLE_STATIC:pn-qemu-system-native = ""
-# pciutils fails build
-DISABLE_STATIC:pn-pciutils = ""
-# libcap aborts on unrecognised option
-DISABLE_STATIC:pn-libcap = ""
-DISABLE_STATIC:pn-libcap-native = ""
-DISABLE_STATIC:pn-nativesdk-libcap = ""
-# libpcap aborts on unrecognised option
-DISABLE_STATIC:pn-libpcap = ""
 # needed by gdb
 DISABLE_STATIC:pn-readline = ""
 # openjade/sgml-common have build issues without static libs
@@ -31,6 +23,7 @@ EXTRA_OECONF:append = "${DISABLE_STATIC}"
 
 EXTRA_OECMAKE:append:pn-libical = " -DSHARED_ONLY=True"
 EXTRA_OECMAKE:append:pn-libjpeg-turbo = " -DENABLE_STATIC=False"
+EXTRA_OECMAKE:append:pn-libjpeg-turbo-native = " -DENABLE_STATIC=False"
 
 EXCONFIG_ARGS:append:pn-ncurses = " --without-normal"
 EXCONFIG_ARGS:append:pn-ncurses-native = " --without-normal"

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159989): 
https://lists.openembedded.org/g/openembedded-core/message/159989
Mute This Topic: https://lists.openembedded.org/mt/87920043/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH v4 2/2] Improve native reproducibility in recipes

[Jacob Kroon]

Avoid encoding build-specific paths in the resulting binaries.

Signed-off-by: Jacob Kroon 
---
 ...sysroot-and-debug-prefix-map-from-co.patch | 24 ---
 .../openssl/openssl_3.0.0.bb  | 12 ++
 meta/recipes-core/ncurses/ncurses.inc |  9 +--
 .../util-linux/util-linux_2.37.2.bb   |  2 +-
 .../libtool/libtool-native_2.4.6.bb   |  1 +
 meta/recipes-devtools/perl/perl_5.34.0.bb |  3 +++
 .../pkgconfig/pkgconfig_git.bb|  1 +
 .../python/python3/determinism.patch  | 17 +
 .../recipes-devtools/python/python3_3.10.1.bb |  8 +++
 9 files changed, 61 insertions(+), 16 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python3/determinism.patch

diff --git 
a/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 
b/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
index 60890c666d..b725f11ff5 100644
--- 
a/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
+++ 
b/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
@@ -29,22 +29,27 @@ Update to fix buildpaths qa issue for '-ffile-prefix-map'.
 
 Signed-off-by: Khem Raj 
 
+Removed buildpath from being passed in '-isystem' flag
+
+Signed-off-by: Jacob Kroon 
+
 ---
  Configurations/unix-Makefile.tmpl | 12 +++-
  crypto/build.info |  2 +-
  2 files changed, 12 insertions(+), 2 deletions(-)
 
-diff --git a/Configurations/unix-Makefile.tmpl 
b/Configurations/unix-Makefile.tmpl
-index f88a70f..528cdef 100644
 a/Configurations/unix-Makefile.tmpl
-+++ b/Configurations/unix-Makefile.tmpl
-@@ -471,13 +471,23 @@ BIN_LDFLAGS={- join(' ', $target{bin_lflags} || (),
+Index: openssl-3.0.0/Configurations/unix-Makefile.tmpl
+===
+--- openssl-3.0.0.orig/Configurations/unix-Makefile.tmpl
 openssl-3.0.0/Configurations/unix-Makefile.tmpl
+@@ -471,13 +471,25 @@ BIN_LDFLAGS={- join(' ', $target{bin_lfl
   '$(CNF_LDFLAGS)', '$(LDFLAGS)') -}
  BIN_EX_LIBS=$(CNF_EX_LIBS) $(EX_LIBS)
  
 -# CPPFLAGS_Q is used for one thing only: to build up buildinf.h
 +# *_Q variables are used for one thing only: to build up buildinf.h
  CPPFLAGS_Q={- $cppflags1 =~ s|([\\"])|\\$1|g;
++  $cppflags1 =~ s|-isystem[^ ]+||g;
$cppflags2 =~ s|([\\"])|\\$1|g;
$lib_cppflags =~ s|([\\"])|\\$1|g;
join(' ', $lib_cppflags || (), $cppflags2 || (),
@@ -54,6 +59,7 @@ index f88a70f..528cdef 100644
 +  s|-fdebug-prefix-map=[^ ]+|-fdebug-prefix-map=|g;
 +  s|-fmacro-prefix-map=[^ ]+|-fmacro-prefix-map=|g;
 +  s|-ffile-prefix-map=[^ ]+|-ffile-prefix-map=|g;
++  s|-isystem[^ ]+||g;
 +}
 +join(' ', @{$config{CFLAGS}}) -}
 +
@@ -63,10 +69,10 @@ index f88a70f..528cdef 100644
  PERLASM_SCHEME= {- $target{perlasm_scheme} -}
  
  # For x86 assembler: Set PROCESSOR to 386 if you want to support
-diff --git a/crypto/build.info b/crypto/build.info
-index efca6cc..eda433e 100644
 a/crypto/build.info
-+++ b/crypto/build.info
+Index: openssl-3.0.0/crypto/build.info
+===
+--- openssl-3.0.0.orig/crypto/build.info
 openssl-3.0.0/crypto/build.info
 @@ -109,7 +109,7 @@ DEFINE[../libcrypto]=$UPLINKDEF
  
  DEPEND[info.o]=buildinf.h
diff --git a/meta/recipes-connectivity/openssl/openssl_3.0.0.bb 
b/meta/recipes-connectivity/openssl/openssl_3.0.0.bb
index da73ed6bc3..caf12a9802 100644
--- a/meta/recipes-connectivity/openssl/openssl_3.0.0.bb
+++ b/meta/recipes-connectivity/openssl/openssl_3.0.0.bb
@@ -47,10 +47,6 @@ EXTRA_OECONF:append:libc-musl:powerpc64 = " no-asm"
 EXTRA_OECONF:class-native = "--with-rand-seed=os,devrandom"
 EXTRA_OECONF:class-nativesdk = "--with-rand-seed=os,devrandom"
 
-# Relying on hardcoded built-in paths causes openssl-native to not be 
relocateable from sstate.
-CFLAGS:append:class-native = " -DOPENSSLDIR=/not/builtin 
-DENGINESDIR=/not/builtin"
-CFLAGS:append:class-nativesdk = " -DOPENSSLDIR=/not/builtin 
-DENGINESDIR=/not/builtin"
-
 # This allows disabling deprecated or undesirable crypto algorithms.
 # The default is to trust upstream choices.
 DEPRECATED_CRYPTO_FLAGS ?= ""
@@ -135,6 +131,14 @@ do_configure () {
perl ${B}/configdata.pm --dump
 }
 
+do_compile:class-native () {
+   oe_runmake OPENSSLDIR=/non/existent ENGINESDIR=/non/existent 
MODULESDIR=/non/existent
+}
+
+do_compile:class-nativesdk () {
+   oe_runmake OPENSSLDIR=/non/existent ENGINESDIR=/non/existent 
MODULESDIR=/non/existent
+}
+
 do_install () {
oe_runmake DESTDIR="${D}" MANDIR="${m

[OE-core] [RFC PATCH v4 1/2] bitbake.conf: Pad rpath in native binaries

[Jacob Kroon]

Try to make sure that the RUNTIME dynamic entry size is the same for all
binaries produced with the native compiler. This is necessary in order to
produce identical binaries when using differently sized buildpaths.

Remove the build ID since it depends on the rpath at link-time, otherwise
binaries built in two different build paths will differ.

Signed-off-by: Jacob Kroon 
---
 meta/classes/chrpath.bbclass | 3 +++
 meta/classes/sstate.bbclass  | 1 +
 meta/conf/bitbake.conf   | 3 +++
 3 files changed, 7 insertions(+)

diff --git a/meta/classes/chrpath.bbclass b/meta/classes/chrpath.bbclass
index 26b984c4db..f3fb3eb323 100644
--- a/meta/classes/chrpath.bbclass
+++ b/meta/classes/chrpath.bbclass
@@ -24,6 +24,9 @@ def process_file_linux(cmd, fpath, rootdir, baseprefix, 
tmpdir, d, break_hardlin
 new_rpaths = []
 modified = False
 for rpath in rpaths:
+if rpath.startswith('/native-rpath-padding-'):
+modified = True
+continue
 # If rpath is already dynamic copy it to new_rpath and continue
 if rpath.find("$ORIGIN") != -1:
 new_rpaths.append(rpath)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 0326d27c74..92c816a692 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -101,6 +101,7 @@ SSTATEPREINSTFUNCS = ""
 SSTATEPOSTUNPACKFUNCS = "sstate_hardcode_path_unpack"
 SSTATEPOSTINSTFUNCS = ""
 EXTRA_STAGING_FIXMES ?= "HOSTTOOLS_DIR"
+EXTRA_STAGING_FIXMES:append:class-native = " NATIVE_RPATH_PADDING"
 
 # Check whether sstate exists for tasks that support sstate and are in the
 # locked signatures file.
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index fba99e8f0c..330aab3c91 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -580,6 +580,7 @@ BUILDSDK_CXXFLAGS = "${BUILDSDK_CFLAGS}"
 export CXXFLAGS = "${TARGET_CXXFLAGS}"
 TARGET_CXXFLAGS = "${TARGET_CFLAGS}"
 
+NATIVE_RPATH_PADDING = "/native-rpath-padding-${@'x' * (512 - 
len(d.expand('${STAGING_LIBDIR_NATIVE}:${STAGING_BASE_LIBDIR_NATIVE}:/native-rpath-padding-')))}"
 export BUILD_LDFLAGS = "-L${STAGING_LIBDIR_NATIVE} \
 -L${STAGING_BASE_LIBDIR_NATIVE} \
 -Wl,--enable-new-dtags \
@@ -587,6 +588,8 @@ export BUILD_LDFLAGS = "-L${STAGING_LIBDIR_NATIVE} \
 -Wl,-rpath-link,${STAGING_BASE_LIBDIR_NATIVE} \
 -Wl,-rpath,${STAGING_LIBDIR_NATIVE} \
 -Wl,-rpath,${STAGING_BASE_LIBDIR_NATIVE} \
+-Wl,-rpath,${NATIVE_RPATH_PADDING} \
+-Wl,--build-id=none \
 -Wl,-O1"
 
 BUILDSDK_LDFLAGS = "-Wl,-O1"

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159578): 
https://lists.openembedded.org/g/openembedded-core/message/159578
Mute This Topic: https://lists.openembedded.org/mt/87675011/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH v4 0/2] Improve native/cross reproducibility

[Jacob Kroon]

This patch series is not intended for merge. I only send it out to
highlight where the problems are and to get some discussion going on how/if
we want to improve the situation.

This is a patch series that tries to improve the reproducibility of the
native/cross binaries when building in different directories. Whether this has
any real benefits I'm still not sure of, but I thought I'd share this final
version of the patch series in case someone comes up with an idea. This has
been tested on a Fedora 35 system which uses gcc 11.2.1 at the time of writing.

The RUNTIME hack is questionable, maybe there is a better way to enforce
a fixed RUNTIME entry size during linking. The build system does not do any
additional rpath manipulation for target binaries, so in those cases this is
not a problem. I did ask for some guidance on the gcc-help mailing list [1] on
how to reserve a specific RPATH size during linking, but it would seem that is
not currently possible.

If in the end we do come up with a solution, then it should be tested on the
autobuilders, since otherwise this is going to degrade overtime. It would then
be important that the build paths are of significantly different lengths.
TMPDIR=/tmp/sysrootA/ and TMPDIR=/tmp/sysrootB/ will *not* uncover all
rpath problems.

Another thing that would be useful is if buildhistory could start monitoring
the depsig.do_populate_sysroot files, since that would highlight changes in
the actual file contents, which is currently not covered by buildhistory.

The end result of this patch series is that I can build python3-native in
two different build paths, and the resuling sysroot-components/x86_64/
directories are identical, except for the 'fixmepath.cmd' files, which are not
included in the hash equiv. comparisons. Even so, there remains a lot of
native builds that are going to need to be fixed in similar ways as the
ones in this patch series.

Changes in v2:
 * Fixed building icedtea7-native/openjdk-8-native by prepending a '/' to the
   rpath padding
 * Squashed all the recipe-fixes together in one commit for now

Changes in v3:
 * Rebase on top of master
 * Instead of adding a padding entry to rpath, set phony rpaths in bitbake.conf
   and add patches for unbreaking python3-native and kernel
 * With the changes above we don't need to remove the build ID, so add it back

Changes in v4:
 * Rebase on top of master
 * Cleanup fixes for openssl/ncurses/perlcross/perl
 * Go back to padding rpath and revert the python3-native/kernel fixes, since
   this feels like the least intrusive change, and from what I understand
   cmake also does a similar type of rpath padding in some situations

Previous versions of the patch series:
v1: https://lists.openembedded.org/g/openembedded-core/message/158867
v2: https://lists.openembedded.org/g/openembedded-core/message/158998
v3: https://lists.openembedded.org/g/openembedded-core/message/159181

[1] https://gcc.gnu.org/pipermail/gcc-help/2021-November/140920.html

Jacob Kroon (2):
  bitbake.conf: Pad rpath in native binaries
  Improve native reproducibility in recipes

 meta/classes/chrpath.bbclass  |  3 +++
 meta/classes/sstate.bbclass   |  1 +
 meta/conf/bitbake.conf|  3 +++
 ...sysroot-and-debug-prefix-map-from-co.patch | 24 ---
 .../openssl/openssl_3.0.0.bb  | 12 ++
 meta/recipes-core/ncurses/ncurses.inc |  9 +--
 .../util-linux/util-linux_2.37.2.bb   |  2 +-
 .../libtool/libtool-native_2.4.6.bb   |  1 +
 meta/recipes-devtools/perl/perl_5.34.0.bb |  3 +++
 .../pkgconfig/pkgconfig_git.bb|  1 +
 .../python/python3/determinism.patch  | 17 +
 .../recipes-devtools/python/python3_3.10.1.bb |  8 +++
 12 files changed, 68 insertions(+), 16 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python3/determinism.patch


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159577): 
https://lists.openembedded.org/g/openembedded-core/message/159577
Mute This Topic: https://lists.openembedded.org/mt/87675009/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH v3 3/4] bitbake.conf: Use dummy rpaths in native binaries

[Jacob Kroon]

Try to make sure that the RUNTIME dynamic entry size is the same for all
binaries produced with the native compiler. This is necessary in order to
produce identical binaries when using differently sized buildpaths.

This is a first step for producing identical native binaries when using
different build paths. 'zstd-native' is a working example.

Signed-off-by: Jacob Kroon 
---
 meta/classes/chrpath.bbclass | 5 -
 meta/conf/bitbake.conf   | 4 ++--
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/meta/classes/chrpath.bbclass b/meta/classes/chrpath.bbclass
index 26b984c4db..a0f2cf0598 100644
--- a/meta/classes/chrpath.bbclass
+++ b/meta/classes/chrpath.bbclass
@@ -2,7 +2,7 @@ CHRPATH_BIN ?= "chrpath"
 PREPROCESS_RELOCATE_DIRS ?= ""
 
 def process_file_linux(cmd, fpath, rootdir, baseprefix, tmpdir, d, 
break_hardlinks = False):
-import subprocess, oe.qa
+import subprocess, oe.qa, re
 
 with oe.qa.ELFFile(fpath) as elf:
 try:
@@ -20,6 +20,9 @@ def process_file_linux(cmd, fpath, rootdir, baseprefix, 
tmpdir, d, break_hardlin
 # Throw away everything other than the rpath list
 curr_rpath = out.partition("RPATH=")[2]
 #bb.note("Current rpath for %s is %s" % (fpath, curr_rpath.strip()))
+# Inject native libdir and baselibdir
+curr_rpath = re.sub(r'/non/existent/libdir-native-marker-x+', 
d.expand('${STAGING_LIBDIR_NATIVE}'), curr_rpath)
+curr_rpath = re.sub(r'/non/existent/base-libdir-native-marker-x+', 
d.expand('${STAGING_BASE_LIBDIR_NATIVE}'), curr_rpath)
 rpaths = curr_rpath.strip().split(":")
 new_rpaths = []
 modified = False
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index fba99e8f0c..1799127c2a 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -585,8 +585,8 @@ export BUILD_LDFLAGS = "-L${STAGING_LIBDIR_NATIVE} \
 -Wl,--enable-new-dtags \
 -Wl,-rpath-link,${STAGING_LIBDIR_NATIVE} \
 -Wl,-rpath-link,${STAGING_BASE_LIBDIR_NATIVE} \
--Wl,-rpath,${STAGING_LIBDIR_NATIVE} \
--Wl,-rpath,${STAGING_BASE_LIBDIR_NATIVE} \
+
-Wl,-rpath,${@'/non/existent/libdir-native-marker-'.ljust(256, 'x')} \
+
-Wl,-rpath,${@'/non/existent/base-libdir-native-marker-'.ljust(256, 'x')} \
 -Wl,-O1"
 
 BUILDSDK_LDFLAGS = "-Wl,-O1"

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159184): 
https://lists.openembedded.org/g/openembedded-core/message/159184
Mute This Topic: https://lists.openembedded.org/mt/87507058/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH v3 4/4] Improve native reproducibility in recipes

[Jacob Kroon]

Avoid encoding build-specific paths in the resulting binaries.

Signed-off-by: Jacob Kroon 
---
 ...sysroot-and-debug-prefix-map-from-co.patch | 78 ---
 .../openssl/openssl/strip-buildinfo.patch | 13 
 .../openssl/openssl_3.0.0.bb  | 10 +--
 meta/recipes-core/ncurses/ncurses.inc |  4 +
 .../util-linux/util-linux_2.37.2.bb   |  2 +-
 .../libtool/libtool-native_2.4.6.bb   |  1 +
 ...ism.patch => perl-cross-determinism.patch} |  0
 .../perl-cross/perlcross_1.3.6.bb |  4 +-
 meta/recipes-devtools/perl/perl_5.34.0.bb |  3 +
 .../pkgconfig/pkgconfig_git.bb|  1 +
 .../python/python3/determinism.patch  | 15 
 .../recipes-devtools/python/python3_3.10.0.bb |  8 ++
 12 files changed, 53 insertions(+), 86 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 create mode 100644 
meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch
 rename meta/recipes-devtools/perl-cross/files/{determinism.patch => 
perl-cross-determinism.patch} (100%)
 create mode 100644 meta/recipes-devtools/python/python3/determinism.patch

diff --git 
a/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 
b/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
deleted file mode 100644
index 60890c666d..00
--- 
a/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
+++ /dev/null
@@ -1,78 +0,0 @@
-From 5985253f2c9025d7c127443a3a9938946f80c2a1 Mon Sep 17 00:00:00 2001
-From: =?UTF-8?q?Martin=20Hundeb=C3=B8ll?= 
-Date: Tue, 6 Nov 2018 14:50:47 +0100
-Subject: [PATCH] buildinfo: strip sysroot and debug-prefix-map from compiler
- info
-MIME-Version: 1.0
-Content-Type: text/plain; charset=UTF-8
-Content-Transfer-Encoding: 8bit
-
-The openssl build system generates buildinf.h containing the full
-compiler command line used to compile objects. This breaks
-reproducibility, as the compile command is baked into libcrypto, where
-it is used when running `openssl version -f`.
-
-Add stripped build variables for the compiler and cflags lines, and use
-those when generating buildinfo.h.
-
-This is based on a similar patch for older openssl versions:
-https://patchwork.openembedded.org/patch/147229/
-
-Upstream-Status: Inappropriate [OE specific]
-Signed-off-by: Martin Hundebøll 
-
-Update to fix buildpaths qa issue for '-fmacro-prefix-map'.
-
-Signed-off-by: Kai Kang 
-
-Update to fix buildpaths qa issue for '-ffile-prefix-map'.
-
-Signed-off-by: Khem Raj 
-

- Configurations/unix-Makefile.tmpl | 12 +++-
- crypto/build.info |  2 +-
- 2 files changed, 12 insertions(+), 2 deletions(-)
-
-diff --git a/Configurations/unix-Makefile.tmpl 
b/Configurations/unix-Makefile.tmpl
-index f88a70f..528cdef 100644
 a/Configurations/unix-Makefile.tmpl
-+++ b/Configurations/unix-Makefile.tmpl
-@@ -471,13 +471,23 @@ BIN_LDFLAGS={- join(' ', $target{bin_lflags} || (),
-  '$(CNF_LDFLAGS)', '$(LDFLAGS)') -}
- BIN_EX_LIBS=$(CNF_EX_LIBS) $(EX_LIBS)
- 
--# CPPFLAGS_Q is used for one thing only: to build up buildinf.h
-+# *_Q variables are used for one thing only: to build up buildinf.h
- CPPFLAGS_Q={- $cppflags1 =~ s|([\\"])|\\$1|g;
-   $cppflags2 =~ s|([\\"])|\\$1|g;
-   $lib_cppflags =~ s|([\\"])|\\$1|g;
-   join(' ', $lib_cppflags || (), $cppflags2 || (),
- $cppflags1 || ()) -}
- 
-+CFLAGS_Q={- for (@{$config{CFLAGS}}) {
-+  s|-fdebug-prefix-map=[^ ]+|-fdebug-prefix-map=|g;
-+  s|-fmacro-prefix-map=[^ ]+|-fmacro-prefix-map=|g;
-+  s|-ffile-prefix-map=[^ ]+|-ffile-prefix-map=|g;
-+}
-+join(' ', @{$config{CFLAGS}}) -}
-+
-+CC_Q={- $config{CC} =~ s|--sysroot=[^ ]+|--sysroot=recipe-sysroot|g;
-+join(' ', $config{CC}) -}
-+
- PERLASM_SCHEME= {- $target{perlasm_scheme} -}
- 
- # For x86 assembler: Set PROCESSOR to 386 if you want to support
-diff --git a/crypto/build.info b/crypto/build.info
-index efca6cc..eda433e 100644
 a/crypto/build.info
-+++ b/crypto/build.info
-@@ -109,7 +109,7 @@ DEFINE[../libcrypto]=$UPLINKDEF
- 
- DEPEND[info.o]=buildinf.h
- DEPEND[cversion.o]=buildinf.h
--GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(LIB_CFLAGS) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
-+GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC_Q) $(CFLAGS_Q) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
- 
- GENERATE[uplink-x86.s]=../ms/uplink-x86.pl
- GENERATE[uplink-x86_64.s]=../ms/uplink-x86_64.pl
diff --git a/meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch 
b/meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch
new file mode 100644
index 00..0a4a60273d
--- /dev/null
+++ b/me

[OE-core] [RFC PATCH v3 2/4] python3-native: Skip testing import of modules

[Jacob Kroon]

Building python3-native involves running the resulting native binary
as part of a test to import the built modules. If this fails the modules
will be renamed with an appended "failed" suffix. One of the upcoming patches
removes ${STAGING_LIBDIR_NATIVE} and ${STAGING_BASE_LIBDIR_NATIVE} from the
rpath, which means this step will fail. Patch python3 not to perform the
import test, so that the modules won't end up with invalid names.

Signed-off-by: Jacob Kroon 
---
 .../python/python3/no-import-test.patch| 14 ++
 meta/recipes-devtools/python/python3_3.10.0.bb |  1 +
 2 files changed, 15 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3/no-import-test.patch

diff --git a/meta/recipes-devtools/python/python3/no-import-test.patch 
b/meta/recipes-devtools/python/python3/no-import-test.patch
new file mode 100644
index 00..d944e55cc5
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/no-import-test.patch
@@ -0,0 +1,14 @@
+Index: Python-3.10.0/setup.py
+===
+--- Python-3.10.0.orig/setup.py
 Python-3.10.0/setup.py
+@@ -644,6 +644,9 @@ class PyBuildExt(build_ext):
+ if CROSS_COMPILING:
+ return
+ 
++# Skip import test
++return
++
+ loader = importlib.machinery.ExtensionFileLoader(ext.name, 
ext_filename)
+ spec = importlib.util.spec_from_file_location(ext.name, ext_filename,
+   loader=loader)
diff --git a/meta/recipes-devtools/python/python3_3.10.0.bb 
b/meta/recipes-devtools/python/python3_3.10.0.bb
index e3300b6495..c9f21b5e16 100644
--- a/meta/recipes-devtools/python/python3_3.10.0.bb
+++ b/meta/recipes-devtools/python/python3_3.10.0.bb
@@ -40,6 +40,7 @@ SRC_URI:append:class-native = " \

file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
file://12-distutils-prefix-is-inside-staging-area.patch \
file://0001-Don-t-search-system-for-headers-libraries.patch \
+   file://no-import-test.patch \
"
 SRC_URI[sha256sum] = 
"5a99f8e7a6a11a7b98b4e75e0d1303d3832cada5534068f69c7b6222a7b1b002"
 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159183): 
https://lists.openembedded.org/g/openembedded-core/message/159183
Mute This Topic: https://lists.openembedded.org/mt/87507056/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH v3 1/4] kernel: Pass rpaths in BUILD_LDFLAGS

[Jacob Kroon]

Building the kernel involves building native tools that are run during
the kernel build itself. One of the upcoming patches removes
${STAGING_LIBDIR_NATIVE} and ${STAGING_BASE_LIBDIR_NATIVE} from the rpath,
so pass them here so that the kernel tools can continue to run.

Signed-off-by: Jacob Kroon 
---
 meta/classes/kernel.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 2d219cb5e5..aae3fc887e 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -228,6 +228,7 @@ KERNEL_EXTRA_ARGS ?= ""
 
 EXTRA_OEMAKE = " HOSTCC="${BUILD_CC}" HOSTCFLAGS="${BUILD_CFLAGS}" 
HOSTLDFLAGS="${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}""
 EXTRA_OEMAKE += " HOSTCXX="${BUILD_CXX}" HOSTCXXFLAGS="${BUILD_CXXFLAGS}""
+BUILD_LDFLAGS += "-Wl,-rpath,${STAGING_LIBDIR_NATIVE} 
-Wl,-rpath,${STAGING_BASE_LIBDIR_NATIVE}"
 
 KERNEL_ALT_IMAGETYPE ??= ""
 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159182): 
https://lists.openembedded.org/g/openembedded-core/message/159182
Mute This Topic: https://lists.openembedded.org/mt/87507054/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH v3 0/4] Improve native/cross reproducibility

[Jacob Kroon]

This patch series is not intended for merge. I only send it out to
highlight where the problems are and to get some discussion going on
how/if we want to improve the sitation.

This is a patch series that tries to improve the reproducibility of the
native/cross binaries when building in different directories. This has
been tested on a Fedora 35 system which uses gcc 11.2.1 at the time of
writing.

The RUNTIME hack is questionable, maybe there is a better way to enforce
a fixed RUNTIME entry size during linking. The build system does not do
any additional rpath manipulation for target binaries, so in those cases
this is not a problem.

If in the end we do come up with a solution, then it should be tested on
the autobuilders, since otherwise this is going to degrade overtime. It
would then be important that the build paths are of significantly different
lengths. TMPDIR=/tmp/sysrootA/ and TMPDIR=/tmp/sysrootB/ will *not* uncover all
rpath problems.

Another thing that would be useful is if buildhistory could start
monitoring the depsig.do_populate_sysroot files, since that would highlight
changes in the actual file contents, which is currently not covered by
buildhistory.

The end result of this patch series is that I can build python3-native
in two different build paths, and the resuling sysroot-components/x86_64/
directories are identical, except for the 'fixmepath.cmd' files, which
are not included in the hash equiv. comparisons. Even so, there remains a lot of
native builds that are going to need to be fixed in similar ways as the
ones in this patch series.

Changes in v2:
 * Fixed building icedtea7-native/openjdk-8-native by prepending a '/' to the
   rpath padding
 * Squashed all the recipe-fixes together in one commit for now

Changes in v3:
 * Rebase on top of master
 * Instead of adding a padding entry to rpath, set phony rpaths in bitbake.conf
   and add patches for unbreaking python3-native and kernel
 * With the changes above we don't need to remove the build ID, so add it back

Jacob Kroon (4):
  kernel: Pass rpaths in BUILD_LDFLAGS
  python3-native: Skip testing import of modules
  bitbake.conf: Use dummy rpaths in native binaries
  Improve native reproducibility in recipes

 meta/classes/chrpath.bbclass  |  5 +-
 meta/classes/kernel.bbclass   |  1 +
 meta/conf/bitbake.conf|  4 +-
 ...sysroot-and-debug-prefix-map-from-co.patch | 78 ---
 .../openssl/openssl/strip-buildinfo.patch | 13 
 .../openssl/openssl_3.0.0.bb  | 10 +--
 meta/recipes-core/ncurses/ncurses.inc |  4 +
 .../util-linux/util-linux_2.37.2.bb   |  2 +-
 .../libtool/libtool-native_2.4.6.bb   |  1 +
 ...ism.patch => perl-cross-determinism.patch} |  0
 .../perl-cross/perlcross_1.3.6.bb |  4 +-
 meta/recipes-devtools/perl/perl_5.34.0.bb |  3 +
 .../pkgconfig/pkgconfig_git.bb|  1 +
 .../python/python3/determinism.patch  | 15 
 .../python/python3/no-import-test.patch   | 14 
 .../recipes-devtools/python/python3_3.10.0.bb |  9 +++
 16 files changed, 75 insertions(+), 89 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 create mode 100644 
meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch
 rename meta/recipes-devtools/perl-cross/files/{determinism.patch => 
perl-cross-determinism.patch} (100%)
 create mode 100644 meta/recipes-devtools/python/python3/determinism.patch
 create mode 100644 meta/recipes-devtools/python/python3/no-import-test.patch


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159181): 
https://lists.openembedded.org/g/openembedded-core/message/159181
Mute This Topic: https://lists.openembedded.org/mt/87507052/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [RFC PATCH v2 1/2] bitbake.conf: Pad rpath and remove build ID in native binaries

[Jacob Kroon]

On 12/2/21 12:09, Richard Purdie wrote:
> On Thu, 2021-12-02 at 12:03 +0100, Jacob Kroon wrote:
>> On 12/2/21 11:51, Richard Purdie wrote:
>>> On Thu, 2021-12-02 at 11:19 +0100, Jacob Kroon wrote:
>>>> On 12/2/21 00:11, Richard Purdie wrote:
>>>>> On Tue, 2021-11-30 at 23:37 +0100, Jacob Kroon wrote:
>>>>>> Try to make sure that the RUNTIME dynamic entry size is the same for all
>>>>>> binaries produced with the native compiler. This is necessary in order to
>>>>>> produce identical binaries when using differently sized buildpaths. I've
>>>>>> tried using only patchelf, and keeping the linker flags as they are, but
>>>>>> I am unable to produce identical binaries. Has anyone else managed to do
>>>>>> this with patchelf ? If not, maybe we can write a new tool that can 
>>>>>> handle it ?
>>>>>>
>>>>>> The build-id also needs to be removed since it is calculated based on
>>>>>> the data present at link time. This includes STAGING_LIBDIR_NATIVE
>>>>>> and STAGING_BASE_LIBDIR_NATIVE. Both will differ and they need to be 
>>>>>> temporarily
>>>>>> preserved since some recipes will execute the binaries during 
>>>>>> do_install()
>>>>>> (for example python3-native). Later on these are removed in 
>>>>>> chrpath.bbclass.
>>>>>>
>>>>>> This hack is the first step for producing identical native binaries when 
>>>>>> using
>>>>>> different build paths. 'zstd-native' is a working example.
>>>>>>
>>>>>> Signed-off-by: Jacob Kroon 
>>>>>> ---
>>>>>>  meta/classes/chrpath.bbclass | 3 +++
>>>>>>  meta/conf/bitbake.conf   | 5 -
>>>>>>  2 files changed, 7 insertions(+), 1 deletion(-)
>>>>>
>>>>> I'm a little torn on this. Our other option would be to hardcoded a 
>>>>> specific
>>>>> dummy path and then edit it later to the correct value. That may be 
>>>>> neater than
>>>>> adding the padding. It will change the end binaries but hopefully only 
>>>>> after
>>>>> they're installed so should give the same net end result more neatly?
>>>>>
>>>>
>>>> Hmm not sure I follow. This patch adds a new dummy rpath entry,
>>>> "/rpath-padding-xxx...", then we remove it in chrpath. I don't know what
>>>> other value we would like to put there. If I understand you correctly,
>>>> we could perhaps pad one of the ones we already pass
>>>>
>>>> -Wl,-rpath,${STAGING_LIBDIR_NATIVE}
>>>> -Wl,-rpath,${STAGING_BASE_LIBDIR_NATIVE}
>>>>
>>>> with spaces, like:
>>>>
>>>> -Wl,-rpath,${STAGING_LIBDIR_NATIVE}
>>>> -Wl,-rpath,"${STAGING_BASE_LIBDIR_NATIVE}${RPATH_PADDING}"
>>>
>>>
>>> I'm wondering if:
>>>
>>> -Wl,-rpath,/not/exist/our-native-libdir-marker
>>> -Wl,-rpath,/not/exist/our-native-base-libdir-marker
>>>
>>> would work.
>>>
>>
>> Right, I'll give it a try.
>>

Unfortunatley this breaks building python3-native. Although it compiles,
during the build the python build scripts tries to import the created
modules, and if this fails (which it does) it renames the modules:

> *** WARNING: renaming "_curses" since importing it failed: libncurses.so.5: 
> cannot open shared object file: No such file or directory
> *** WARNING: renaming "_curses_panel" since importing it failed: 
> libpanel.so.5: cannot open shared object file: No such file or directory
> *** WARNING: renaming "_ssl" since importing it failed: libssl.so.3: cannot 
> open shared object file: No such file or directory
> *** WARNING: renaming "_hashlib" since importing it failed: libssl.so.3: 
> cannot open shared object file: No such file or directory
> *** WARNING: renaming "nis" since importing it failed: libnsl.so.3: cannot 
> open shared object file: No such file or directory
> *** WARNING: renaming "_ctypes" since importing it failed: libffi.so.8: 
> cannot open shared object file: No such file or directory

I suppose it tries to import using the built python which has those
phony rpaths, and can't find the per-recipe-sysroot
lbncurses.so.5/libpanel.so.5/etc and fails.

The new modules will be called:

> sysroots-components/x86_64/python3-native/usr/lib/pyth

Re: [OE-core] [RFC PATCH v2 1/2] bitbake.conf: Pad rpath and remove build ID in native binaries

[Jacob Kroon]

On 12/2/21 11:51, Richard Purdie wrote:
> On Thu, 2021-12-02 at 11:19 +0100, Jacob Kroon wrote:
>> On 12/2/21 00:11, Richard Purdie wrote:
>>> On Tue, 2021-11-30 at 23:37 +0100, Jacob Kroon wrote:
>>>> Try to make sure that the RUNTIME dynamic entry size is the same for all
>>>> binaries produced with the native compiler. This is necessary in order to
>>>> produce identical binaries when using differently sized buildpaths. I've
>>>> tried using only patchelf, and keeping the linker flags as they are, but
>>>> I am unable to produce identical binaries. Has anyone else managed to do
>>>> this with patchelf ? If not, maybe we can write a new tool that can handle 
>>>> it ?
>>>>
>>>> The build-id also needs to be removed since it is calculated based on
>>>> the data present at link time. This includes STAGING_LIBDIR_NATIVE
>>>> and STAGING_BASE_LIBDIR_NATIVE. Both will differ and they need to be 
>>>> temporarily
>>>> preserved since some recipes will execute the binaries during do_install()
>>>> (for example python3-native). Later on these are removed in 
>>>> chrpath.bbclass.
>>>>
>>>> This hack is the first step for producing identical native binaries when 
>>>> using
>>>> different build paths. 'zstd-native' is a working example.
>>>>
>>>> Signed-off-by: Jacob Kroon 
>>>> ---
>>>>  meta/classes/chrpath.bbclass | 3 +++
>>>>  meta/conf/bitbake.conf   | 5 -
>>>>  2 files changed, 7 insertions(+), 1 deletion(-)
>>>
>>> I'm a little torn on this. Our other option would be to hardcoded a specific
>>> dummy path and then edit it later to the correct value. That may be neater 
>>> than
>>> adding the padding. It will change the end binaries but hopefully only after
>>> they're installed so should give the same net end result more neatly?
>>>
>>
>> Hmm not sure I follow. This patch adds a new dummy rpath entry,
>> "/rpath-padding-xxx...", then we remove it in chrpath. I don't know what
>> other value we would like to put there. If I understand you correctly,
>> we could perhaps pad one of the ones we already pass
>>
>> -Wl,-rpath,${STAGING_LIBDIR_NATIVE}
>> -Wl,-rpath,${STAGING_BASE_LIBDIR_NATIVE}
>>
>> with spaces, like:
>>
>> -Wl,-rpath,${STAGING_LIBDIR_NATIVE}
>> -Wl,-rpath,"${STAGING_BASE_LIBDIR_NATIVE}${RPATH_PADDING}"
> 
> 
> I'm wondering if:
> 
> -Wl,-rpath,/not/exist/our-native-libdir-marker
> -Wl,-rpath,/not/exist/our-native-base-libdir-marker
> 
> would work.
> 

Right, I'll give it a try.

>> If that works that would be less intrusive I think.
>>
>>> If we separate out the build-id patch we could hopefully get that piece 
>>> merged
>>> as that shouldn't be controversial? 
>>>
>>
>> Yes, I can split it out into a separate patch.
>>
>> But now that I've looked at this for a while, I've asked myself what
>> good does all this do ? The only optimization I can think of is that if
>> we rebuild a native recipes, and the sysroot component turns out the
>> same, then we don't need to create a new sstate cache entry. So we save
>> disk space, but disk space is cheap. We still need to build it. What I
>> would like is to have a common sstate dir for multiple build
>> directories. So if I build libtool-native in one build path, then at my
>> other build path it would just pick it up from sstate cache when I build
>> there. In the end, is that something that would be possible ?
> 
> We originally started here with gcc-cross so lets consider that and multiple
> build directories where a patch changes gcc-cross in a way that is irrelavent 
> to
> the output.
> 
> The "win" is that regardless of whether I build in location A or B, I get the
> same gcc-cross binary. Hash-equiv will then not rebuild the target binaries.
> Yes, I pay the price of a gcc-cross rebuild but hashequiv saves the targets
> rebuilding.
> 
> Currently it would only happen if you always build gcc-cross in a specific 
> build
> path.
> 

I know the build path will change if I upgrade to a new version of gcc,
but then the output is most definitely gonna change as well.

> Like everything, it is a question of looking at the changes and deciding 
> whether
> they are worth any maintenance burden/code complication or additional overhead
> they generate. I don't know the answer here yet but I do appreciate the 
> research
> in helping get us data to make dec

Re: [OE-core] [RFC PATCH v2 1/2] bitbake.conf: Pad rpath and remove build ID in native binaries

[Jacob Kroon]

On 12/2/21 00:11, Richard Purdie wrote:
> On Tue, 2021-11-30 at 23:37 +0100, Jacob Kroon wrote:
>> Try to make sure that the RUNTIME dynamic entry size is the same for all
>> binaries produced with the native compiler. This is necessary in order to
>> produce identical binaries when using differently sized buildpaths. I've
>> tried using only patchelf, and keeping the linker flags as they are, but
>> I am unable to produce identical binaries. Has anyone else managed to do
>> this with patchelf ? If not, maybe we can write a new tool that can handle 
>> it ?
>>
>> The build-id also needs to be removed since it is calculated based on
>> the data present at link time. This includes STAGING_LIBDIR_NATIVE
>> and STAGING_BASE_LIBDIR_NATIVE. Both will differ and they need to be 
>> temporarily
>> preserved since some recipes will execute the binaries during do_install()
>> (for example python3-native). Later on these are removed in chrpath.bbclass.
>>
>> This hack is the first step for producing identical native binaries when 
>> using
>> different build paths. 'zstd-native' is a working example.
>>
>> Signed-off-by: Jacob Kroon 
>> ---
>>  meta/classes/chrpath.bbclass | 3 +++
>>  meta/conf/bitbake.conf   | 5 -
>>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> I'm a little torn on this. Our other option would be to hardcoded a specific
> dummy path and then edit it later to the correct value. That may be neater 
> than
> adding the padding. It will change the end binaries but hopefully only after
> they're installed so should give the same net end result more neatly?
> 

Hmm not sure I follow. This patch adds a new dummy rpath entry,
"/rpath-padding-xxx...", then we remove it in chrpath. I don't know what
other value we would like to put there. If I understand you correctly,
we could perhaps pad one of the ones we already pass

-Wl,-rpath,${STAGING_LIBDIR_NATIVE}
-Wl,-rpath,${STAGING_BASE_LIBDIR_NATIVE}

with spaces, like:

-Wl,-rpath,${STAGING_LIBDIR_NATIVE}
-Wl,-rpath,"${STAGING_BASE_LIBDIR_NATIVE}${RPATH_PADDING}"

If that works that would be less intrusive I think.

> If we separate out the build-id patch we could hopefully get that piece merged
> as that shouldn't be controversial? 
> 

Yes, I can split it out into a separate patch.

But now that I've looked at this for a while, I've asked myself what
good does all this do ? The only optimization I can think of is that if
we rebuild a native recipes, and the sysroot component turns out the
same, then we don't need to create a new sstate cache entry. So we save
disk space, but disk space is cheap. We still need to build it. What I
would like is to have a common sstate dir for multiple build
directories. So if I build libtool-native in one build path, then at my
other build path it would just pick it up from sstate cache when I build
there. In the end, is that something that would be possible ?

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159082): 
https://lists.openembedded.org/g/openembedded-core/message/159082
Mute This Topic: https://lists.openembedded.org/mt/87415016/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] native/cross: make ar wrapper support to read options from file

[Jacob Kroon]

On 12/2/21 09:37, Hongxu Jia wrote:
> If ar input params starts with @, it means to read options from file
> $ ar -h
> ...
>   @  - read options from 
> ...
> 
> It failed to call ar wrapper to read options from file:
> $ path_to/oe-core/scripts/native-intercept/ar 
> @bazel-out/k8-opt/bin/external/llvm-project/llvm/libSupport.a-2.params
> |path_to/oe-core/scripts/native-intercept/ar: invalid option -- '@'
> | Usage: path_to/oe-core/scripts/native-intercept/ar [emulation options]
> [-]{dmpqrstx}[abcDfilMNoPsSTuvV] [--plugin ] [member-name] [count] 
> archive-file file...
> 
> If input params start with @, append option -D to argv list
> 
> Signed-off-by: Hongxu Jia 
> ---
>  scripts/native-intercept/ar | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/scripts/native-intercept/ar b/scripts/native-intercept/ar
> index dcc623e3ed..32f45171d6 100755
> --- a/scripts/native-intercept/ar
> +++ b/scripts/native-intercept/ar
> @@ -19,6 +19,8 @@ argv = sys.argv
>  if argv[1].startswith('--'):
>  # No modifier given
>  None
> +elif argv[1].startswith('@'):
> +argv.append('-D')
>  else:
>  # remove the optional '-'
>  if argv[1][0] == '-':
> 

I am a little surprised this works, since "-D" would be placed after any
archive/member parameters. But if does, ok.

Otherwise maybe we should just give up if there is a '@' and fallback to
not modifying the args, unless we want to process the file itself in the
wrapper.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159077): 
https://lists.openembedded.org/g/openembedded-core/message/159077
Mute This Topic: https://lists.openembedded.org/mt/87448903/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH v2 2/2] Improve native reproducibility in recipes

[Jacob Kroon]

Avoid encoding build-specific paths in the resulting binaries.

Signed-off-by: Jacob Kroon 
---
 ...sysroot-and-debug-prefix-map-from-co.patch | 78 ---
 .../openssl/openssl/strip-buildinfo.patch | 13 
 .../openssl/openssl_3.0.0.bb  | 10 +--
 meta/recipes-core/ncurses/ncurses.inc |  4 +
 .../util-linux/util-linux_2.37.2.bb   |  2 +-
 .../libtool/libtool-native_2.4.6.bb   |  1 +
 ...ism.patch => perl-cross-determinism.patch} |  0
 .../perl-cross/perlcross_1.3.6.bb |  4 +-
 meta/recipes-devtools/perl/perl_5.34.0.bb |  5 ++
 .../pkgconfig/pkgconfig_git.bb|  1 +
 .../python/python3/determinism.patch  | 15 
 .../recipes-devtools/python/python3_3.10.0.bb |  8 ++
 12 files changed, 55 insertions(+), 86 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 create mode 100644 
meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch
 rename meta/recipes-devtools/perl-cross/files/{determinism.patch => 
perl-cross-determinism.patch} (100%)
 create mode 100644 meta/recipes-devtools/python/python3/determinism.patch

diff --git 
a/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 
b/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
deleted file mode 100644
index 60890c666d..00
--- 
a/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
+++ /dev/null
@@ -1,78 +0,0 @@
-From 5985253f2c9025d7c127443a3a9938946f80c2a1 Mon Sep 17 00:00:00 2001
-From: =?UTF-8?q?Martin=20Hundeb=C3=B8ll?= 
-Date: Tue, 6 Nov 2018 14:50:47 +0100
-Subject: [PATCH] buildinfo: strip sysroot and debug-prefix-map from compiler
- info
-MIME-Version: 1.0
-Content-Type: text/plain; charset=UTF-8
-Content-Transfer-Encoding: 8bit
-
-The openssl build system generates buildinf.h containing the full
-compiler command line used to compile objects. This breaks
-reproducibility, as the compile command is baked into libcrypto, where
-it is used when running `openssl version -f`.
-
-Add stripped build variables for the compiler and cflags lines, and use
-those when generating buildinfo.h.
-
-This is based on a similar patch for older openssl versions:
-https://patchwork.openembedded.org/patch/147229/
-
-Upstream-Status: Inappropriate [OE specific]
-Signed-off-by: Martin Hundebøll 
-
-Update to fix buildpaths qa issue for '-fmacro-prefix-map'.
-
-Signed-off-by: Kai Kang 
-
-Update to fix buildpaths qa issue for '-ffile-prefix-map'.
-
-Signed-off-by: Khem Raj 
-

- Configurations/unix-Makefile.tmpl | 12 +++-
- crypto/build.info |  2 +-
- 2 files changed, 12 insertions(+), 2 deletions(-)
-
-diff --git a/Configurations/unix-Makefile.tmpl 
b/Configurations/unix-Makefile.tmpl
-index f88a70f..528cdef 100644
 a/Configurations/unix-Makefile.tmpl
-+++ b/Configurations/unix-Makefile.tmpl
-@@ -471,13 +471,23 @@ BIN_LDFLAGS={- join(' ', $target{bin_lflags} || (),
-  '$(CNF_LDFLAGS)', '$(LDFLAGS)') -}
- BIN_EX_LIBS=$(CNF_EX_LIBS) $(EX_LIBS)
- 
--# CPPFLAGS_Q is used for one thing only: to build up buildinf.h
-+# *_Q variables are used for one thing only: to build up buildinf.h
- CPPFLAGS_Q={- $cppflags1 =~ s|([\\"])|\\$1|g;
-   $cppflags2 =~ s|([\\"])|\\$1|g;
-   $lib_cppflags =~ s|([\\"])|\\$1|g;
-   join(' ', $lib_cppflags || (), $cppflags2 || (),
- $cppflags1 || ()) -}
- 
-+CFLAGS_Q={- for (@{$config{CFLAGS}}) {
-+  s|-fdebug-prefix-map=[^ ]+|-fdebug-prefix-map=|g;
-+  s|-fmacro-prefix-map=[^ ]+|-fmacro-prefix-map=|g;
-+  s|-ffile-prefix-map=[^ ]+|-ffile-prefix-map=|g;
-+}
-+join(' ', @{$config{CFLAGS}}) -}
-+
-+CC_Q={- $config{CC} =~ s|--sysroot=[^ ]+|--sysroot=recipe-sysroot|g;
-+join(' ', $config{CC}) -}
-+
- PERLASM_SCHEME= {- $target{perlasm_scheme} -}
- 
- # For x86 assembler: Set PROCESSOR to 386 if you want to support
-diff --git a/crypto/build.info b/crypto/build.info
-index efca6cc..eda433e 100644
 a/crypto/build.info
-+++ b/crypto/build.info
-@@ -109,7 +109,7 @@ DEFINE[../libcrypto]=$UPLINKDEF
- 
- DEPEND[info.o]=buildinf.h
- DEPEND[cversion.o]=buildinf.h
--GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(LIB_CFLAGS) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
-+GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC_Q) $(CFLAGS_Q) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
- 
- GENERATE[uplink-x86.s]=../ms/uplink-x86.pl
- GENERATE[uplink-x86_64.s]=../ms/uplink-x86_64.pl
diff --git a/meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch 
b/meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch
new file mode 100644
index 00..0a4a60273d
--- /d

[OE-core] [RFC PATCH v2 1/2] bitbake.conf: Pad rpath and remove build ID in native binaries

[Jacob Kroon]

Try to make sure that the RUNTIME dynamic entry size is the same for all
binaries produced with the native compiler. This is necessary in order to
produce identical binaries when using differently sized buildpaths. I've
tried using only patchelf, and keeping the linker flags as they are, but
I am unable to produce identical binaries. Has anyone else managed to do
this with patchelf ? If not, maybe we can write a new tool that can handle it ?

The build-id also needs to be removed since it is calculated based on
the data present at link time. This includes STAGING_LIBDIR_NATIVE
and STAGING_BASE_LIBDIR_NATIVE. Both will differ and they need to be temporarily
preserved since some recipes will execute the binaries during do_install()
(for example python3-native). Later on these are removed in chrpath.bbclass.

This hack is the first step for producing identical native binaries when using
different build paths. 'zstd-native' is a working example.

Signed-off-by: Jacob Kroon 
---
 meta/classes/chrpath.bbclass | 3 +++
 meta/conf/bitbake.conf   | 5 -
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/meta/classes/chrpath.bbclass b/meta/classes/chrpath.bbclass
index 26b984c4db..56a68768e0 100644
--- a/meta/classes/chrpath.bbclass
+++ b/meta/classes/chrpath.bbclass
@@ -24,6 +24,9 @@ def process_file_linux(cmd, fpath, rootdir, baseprefix, 
tmpdir, d, break_hardlin
 new_rpaths = []
 modified = False
 for rpath in rpaths:
+if rpath.startswith('/rpath-padding-'):
+modified = True
+continue
 # If rpath is already dynamic copy it to new_rpath and continue
 if rpath.find("$ORIGIN") != -1:
 new_rpaths.append(rpath)
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index fba99e8f0c..61124e10b2 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -580,6 +580,7 @@ BUILDSDK_CXXFLAGS = "${BUILDSDK_CFLAGS}"
 export CXXFLAGS = "${TARGET_CXXFLAGS}"
 TARGET_CXXFLAGS = "${TARGET_CFLAGS}"
 
+RPATH_PADDING ?= "/rpath-padding-${@'x' * (512 - 
len(d.expand('${STAGING_LIBDIR_NATIVE}:${STAGING_BASE_LIBDIR_NATIVE}:/rpath-padding-')))}"
 export BUILD_LDFLAGS = "-L${STAGING_LIBDIR_NATIVE} \
 -L${STAGING_BASE_LIBDIR_NATIVE} \
 -Wl,--enable-new-dtags \
@@ -587,7 +588,9 @@ export BUILD_LDFLAGS = "-L${STAGING_LIBDIR_NATIVE} \
 -Wl,-rpath-link,${STAGING_BASE_LIBDIR_NATIVE} \
 -Wl,-rpath,${STAGING_LIBDIR_NATIVE} \
 -Wl,-rpath,${STAGING_BASE_LIBDIR_NATIVE} \
--Wl,-O1"
+-Wl,-rpath,${RPATH_PADDING} \
+-Wl,-O1 \
+-Wl,--build-id=none"
 
 BUILDSDK_LDFLAGS = "-Wl,-O1"
 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158999): 
https://lists.openembedded.org/g/openembedded-core/message/158999
Mute This Topic: https://lists.openembedded.org/mt/87415016/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH v2 0/2] Improve native/cross reproducibility

[Jacob Kroon]

This patch series is not intended for merge. I only send it out to
highlight where the problems are and to get some discussion going on
how/if we want to improve the sitation.

This is a patch series that tries to improve the reproducibility of the
native/cross binaries when building in different directories. This has
been tested on a Fedora 35 system which uses gcc 11.2.1 at the time of
writing.

The RUNTIME hack is questionable, maybe there is a better way to enforce
a fixed RUNTIME entry size during linking.

If in the end we do come up with a solution, then it should be tested on
the autobuilders, since otherwise this is going to degrade overtime. It
would then be important that the build paths are of significantly different
lengths. TMPDIR=/tmp/sysrootA/ and TMPDIR=/tmp/sysrootB/ will *not* uncover all
rpath problems.

Another thing that would be useful is if buildhistory could start
monitoring the depsig.do_populate_sysroot files, since that would highlight
changes in the actual file contents, which is currently not covered by
buildhistory.

The end result of this patch series is that I can build python3-native
in two different build paths, and the resuling sysroot-components/x86_64/
directories are identical, except for the 'fixmepath.cmd' files, which
are not included in the hash equiv. comparisons. Even so, there remains a lot of
native builds that are going to need to be fixed in similar ways as the
ones in this patch series.

Changes in v2:
 * Fixed building icedtea7-native/openjdk-8-native by prepending a '/'
   to the rpath padding
 * Squashed all the recipe-fixes together in one commit for now

Jacob

Jacob Kroon (2):
  bitbake.conf: Pad rpath and remove build ID in native binaries
  Improve native reproducibility in recipes

 meta/classes/chrpath.bbclass  |  3 +
 meta/conf/bitbake.conf|  5 +-
 ...sysroot-and-debug-prefix-map-from-co.patch | 78 ---
 .../openssl/openssl/strip-buildinfo.patch | 13 
 .../openssl/openssl_3.0.0.bb  | 10 +--
 meta/recipes-core/ncurses/ncurses.inc |  4 +
 .../util-linux/util-linux_2.37.2.bb   |  2 +-
 .../libtool/libtool-native_2.4.6.bb   |  1 +
 ...ism.patch => perl-cross-determinism.patch} |  0
 .../perl-cross/perlcross_1.3.6.bb |  4 +-
 meta/recipes-devtools/perl/perl_5.34.0.bb |  5 ++
 .../pkgconfig/pkgconfig_git.bb|  1 +
 .../python/python3/determinism.patch  | 15 
 .../recipes-devtools/python/python3_3.10.0.bb |  8 ++
 14 files changed, 62 insertions(+), 87 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 create mode 100644 
meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch
 rename meta/recipes-devtools/perl-cross/files/{determinism.patch => 
perl-cross-determinism.patch} (100%)
 create mode 100644 meta/recipes-devtools/python/python3/determinism.patch


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158998): 
https://lists.openembedded.org/g/openembedded-core/message/158998
Mute This Topic: https://lists.openembedded.org/mt/87415010/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [RFC PATCH 4/9] perl/perlcross: Improve native reproducability

[Jacob Kroon]

On 11/29/21 10:07, Alexander Kanavin wrote:
> Can you split the determinism patch fix into a separate patch please?
> 

I don't know what exactly is the right fix here. Should both
"determinism.patch":es be applied when building perl-native ?

Jacob

> Alex
> 
> On Sun, 28 Nov 2021 at 10:46, Jacob Kroon  <mailto:jacob.kr...@gmail.com>> wrote:
> 
> In order to make perlcross-native independent of build path we need
> to follow
> the symlinks when copying the patches, otherwise they will point to
> whereever
> oe-core is checked out for that particular build.
> 
> Doing this reveals an issue in perl-native, where it copies the patches
> from perlcross-native's sysroot, but both perlcross and perl have a
> patch called "determinism.patch", so one of them gets overridden. Rename
> the patch in perlcross so that this doesn't happen.
> 
> Signed-off-by: Jacob Kroon  <mailto:jacob.kr...@gmail.com>>
> ---
>  .../{determinism.patch => perl-cross-determinism.patch}      | 0
>  meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb
> <http://perlcross_1.3.6.bb>          | 4 ++--
>  meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> 
>                   | 5 +
>  3 files changed, 7 insertions(+), 2 deletions(-)
>  rename meta/recipes-devtools/perl-cross/files/{determinism.patch =>
> perl-cross-determinism.patch} (100%)
> 
> diff --git
> a/meta/recipes-devtools/perl-cross/files/determinism.patch
> b/meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patch
> similarity index 100%
> rename from meta/recipes-devtools/perl-cross/files/determinism.patch
> rename to
> meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patch
> diff --git a/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb
> <http://perlcross_1.3.6.bb>
> b/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb
> <http://perlcross_1.3.6.bb>
> index 2759ef8a53..dab7f4558f 100644
> --- a/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb
> <http://perlcross_1.3.6.bb>
> +++ b/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb
> <http://perlcross_1.3.6.bb>
> @@ -15,7 +15,7 @@ SRC_URI =
> "https://github.com/arsv/perl-cross/releases/download/${PV}/perl-cross
> 
> <https://github.com/arsv/perl-cross/releases/download/$%7BPV%7D/perl-cross>
>            
> file://0001-configure_tool.sh-do-not-quote-the-argument-to-comma.patch \
>            
> file://0001-perl-cross-add-LDFLAGS-when-linking-libperl.patch \
>            
> file://0001-configure_path.sh-do-not-hardcode-prefix-lib-as-libr.patch \
> -           file://determinism.patch \
> +           file://perl-cross-determinism.patch \
>            
> file://0001-cnf-configure_func_sel.sh-disable-thread_safe_nl_lan.patch \
>            
> file://0001-Makefile-check-the-file-if-patched-or-not.patch \
>             "
> @@ -33,7 +33,7 @@ do_compile () {
> 
>  do_install:class-native() {
>      mkdir -p ${D}/${datadir}/perl-cross/
> -    cp -rf ${S}/* ${D}/${datadir}/perl-cross/
> +    cp -rfL ${S}/* ${D}/${datadir}/perl-cross/
>  }
> 
>  BBCLASSEXTEND = "native"
> diff --git a/meta/recipes-devtools/perl/perl_5.34.0.bb
> <http://perl_5.34.0.bb> b/meta/recipes-devtools/perl/perl_5.34.0.bb
> <http://perl_5.34.0.bb>
> index 16d45ccff3..0b74d5f072 100644
> --- a/meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb>
> +++ b/meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb>
> @@ -97,6 +97,9 @@ do_configure:class-native() {
>      -Dvendorprefix=${prefix} \
>      -Ui_xlocale \
>      ${PACKAGECONFIG_CONFARGS}
> +
> +    # See the comment above
> +    sed -i -e "s,${STAGING_DIR_NATIVE},/non/existent,g" config.h
>  }
> 
>  do_configure:append() {
> @@ -395,3 +398,5 @@ SSTATE_HASHEQUIV_FILEMAP = " \
>      populate_sysroot:*/lib*/perl5/config.sh:${TMPDIR} \
>      populate_sysroot:*/lib*/perl5/config.sh:${COREBASE} \
>      "
> +
> +EXTRA_STAGING_FIXMES:append:class-native = " RPATH_PADDING"
> 
> 
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158955): 
https://lists.openembedded.org/g/openembedded-core/message/158955
Mute This Topic: https://lists.openembedded.org/mt/87352797/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [RFC PATCH 0/9] Improve native/cross reproducibility

[Jacob Kroon]

On 11/29/21 10:16, Alexander Kanavin wrote:
> Thanks Jacob. When looking at this patchset I kept asking myself, why is
> this or that change necessary for -native but not for -target. I think
> it would help if you include that information in the commits,
> particularly, in 1/9 as it is the most invasive change of all. Maybe we
> can then figure out a better way.
> 
> Alex
> 

Yeah, I think its because there is no extra manipulation of rpath done
for the target binaries at all, since they wont be running in a location
other than what they were configured for. RP, Khem, please correct me if
I'm wrong. The native binaries on the other hand will need to run from
each recipe specific sysroot.

I can update the commit message in patch 1 if that makes sense.

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158954): 
https://lists.openembedded.org/g/openembedded-core/message/158954
Mute This Topic: https://lists.openembedded.org/mt/87352786/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH 9/9] bitbake.conf: Avoid rpath hack for Java recipes

[Jacob Kroon]

These shouldn't go in here, they are just here to hightlight that the padding
trick will not work in all recipes.

Signed-off-by: Jacob Kroon 
---
 meta/conf/bitbake.conf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 9621023354..1cbe4648f8 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -592,6 +592,8 @@ export BUILD_LDFLAGS = "-L${STAGING_LIBDIR_NATIVE} \
 ${RPATH_PADDING_FLAG} \
 -Wl,-O1 \
 -Wl,--build-id=none"
+RPATH_PADDING_FLAG:pn-icedtea7-native = ""
+RPATH_PADDING_FLAG:pn-openjdk-8-native = ""
 
 BUILDSDK_LDFLAGS = "-Wl,-O1"
 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158876): 
https://lists.openembedded.org/g/openembedded-core/message/158876
Mute This Topic: https://lists.openembedded.org/mt/87352806/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH 7/9] util-linux: Improve native reproducibility

[Jacob Kroon]

Avoid encoding build-specific paths in the resulting binaries.

Signed-off-by: Jacob Kroon 
---
 meta/recipes-core/util-linux/util-linux_2.37.2.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/util-linux/util-linux_2.37.2.bb 
b/meta/recipes-core/util-linux/util-linux_2.37.2.bb
index d609c30067..09f83eb4dd 100644
--- a/meta/recipes-core/util-linux/util-linux_2.37.2.bb
+++ b/meta/recipes-core/util-linux/util-linux_2.37.2.bb
@@ -83,7 +83,7 @@ EXTRA_OECONF = "\
 "
 
 EXTRA_OECONF:append:class-target = " --enable-setpriv"
-EXTRA_OECONF:append:class-native = " --without-cap-ng --disable-setpriv"
+EXTRA_OECONF:append:class-native = " --without-cap-ng --disable-setpriv 
--runstatedir=/non/existent SYSCONFSTATICDIR=/non/existent"
 EXTRA_OECONF:append:class-nativesdk = " --without-cap-ng --disable-setpriv"
 EXTRA_OECONF:append = " --disable-hwclock-gplv3"
 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158874): 
https://lists.openembedded.org/g/openembedded-core/message/158874
Mute This Topic: https://lists.openembedded.org/mt/87352802/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH 8/9] python3: Improve native reproducibility

[Jacob Kroon]

Lots of hacks to get python reproduce in different build paths:
 * Avoid encoding build-specific paths in the resulting binaries
 * Build without debug information
 * Remove __pycache__/ since the cached files seem to depend on the run-paths

Signed-off-by: Jacob Kroon 
---
 .../python/python3/determinism.patch  | 15 +++
 meta/recipes-devtools/python/python3_3.10.0.bb|  8 
 2 files changed, 23 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3/determinism.patch

diff --git a/meta/recipes-devtools/python/python3/determinism.patch 
b/meta/recipes-devtools/python/python3/determinism.patch
new file mode 100644
index 00..eca7755d4e
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/determinism.patch
@@ -0,0 +1,15 @@
+Index: Python-3.10.0/Makefile.pre.in
+===
+--- Python-3.10.0.orig/Makefile.pre.in
 Python-3.10.0/Makefile.pre.in
+@@ -791,8 +791,8 @@ Modules/getbuildinfo.o: $(PARSER_OBJS) \
+ 
+ Modules/getpath.o: $(srcdir)/Modules/getpath.c Makefile
+   $(CC) -c $(PY_CORE_CFLAGS) -DPYTHONPATH='"$(PYTHONPATH)"' \
+-  -DPREFIX='"$(prefix)"' \
+-  -DEXEC_PREFIX='"$(exec_prefix)"' \
++  -DPREFIX='"/non/existent"' \
++  -DEXEC_PREFIX='"/non/existent"' \
+   -DVERSION='"$(VERSION)"' \
+   -DVPATH='"$(VPATH)"' \
+   -o $@ $(srcdir)/Modules/getpath.c
diff --git a/meta/recipes-devtools/python/python3_3.10.0.bb 
b/meta/recipes-devtools/python/python3_3.10.0.bb
index e3300b6495..ba2e9f7dcb 100644
--- a/meta/recipes-devtools/python/python3_3.10.0.bb
+++ b/meta/recipes-devtools/python/python3_3.10.0.bb
@@ -40,6 +40,7 @@ SRC_URI:append:class-native = " \

file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
file://12-distutils-prefix-is-inside-staging-area.patch \
file://0001-Don-t-search-system-for-headers-libraries.patch \
+   file://determinism.patch \
"
 SRC_URI[sha256sum] = 
"5a99f8e7a6a11a7b98b4e75e0d1303d3832cada5534068f69c7b6222a7b1b002"
 
@@ -79,6 +80,8 @@ DEPENDS:append:class-nativesdk = " python3-native"
 # force to use the mutex+cond implementation 
(https://bugs.python.org/issue41710)
 CFLAGS += "-DHAVE_BROKEN_POSIX_SEMAPHORES"
 
+CFLAGS:append:class-native = " -ffile-prefix-map=${WORKDIR}=/usr/src"
+
 EXTRA_OECONF = " --without-ensurepip --enable-shared 
--with-platlibdir=${baselib}"
 EXTRA_OECONF:append:class-native = " --bindir=${bindir}/${PN}"
 
@@ -94,6 +97,7 @@ CACHED_CONFIGUREVARS = " \
 ac_cv_file__dev_ptc=no \
 ac_cv_working_tzset=yes \
 "
+CACHED_CONFIGUREVARS:append:class-native = " ac_cv_prog_cc_g=no"
 
 # PGO currently causes builds to not be reproducible so disable by default, 
see YOCTO #13407
 PACKAGECONFIG:class-target ??= "readline gdbm 
${@bb.utils.filter('DISTRO_FEATURES', 'lto', d)}"
@@ -180,6 +184,8 @@ do_install:append() {
 # More info: http://benno.id.au/blog/2013/01/15/python-determinism
 rm 
${D}${libdir}/python${PYTHON_MAJMIN}/test/__pycache__/test_range.cpython*
 rm 
${D}${libdir}/python${PYTHON_MAJMIN}/test/__pycache__/test_xml_etree.cpython*
+
+find ${D}${libdir}/python${PYTHON_MAJMIN} -name __pycache__ | xargs 
-n1 rm -r
 }
 
 do_install:append:class-nativesdk () {
@@ -398,3 +404,5 @@ SYSROOT_PREPROCESS_FUNCS += " py3_sysroot_cleanup"
 py3_sysroot_cleanup () {
rm -rf ${SYSROOT_DESTDIR}${libdir}/python${PYTHON_MAJMIN}/test
 }
+
+EXTRA_STAGING_FIXMES:append:class-native = " RPATH_PADDING WORKDIR"

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158875): 
https://lists.openembedded.org/g/openembedded-core/message/158875
Mute This Topic: https://lists.openembedded.org/mt/87352805/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH 5/9] pkgconfig: Improve native reproducibility

[Jacob Kroon]

Avoid encoding build-specific paths in the resulting binaries.

Signed-off-by: Jacob Kroon 
---
 meta/recipes-devtools/pkgconfig/pkgconfig_git.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb 
b/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb
index c220bafd90..a7b2cae624 100644
--- a/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb
+++ b/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb
@@ -28,6 +28,7 @@ inherit autotools
 # so just continue that behaviour.
 #
 EXTRA_OECONF += "--disable-indirect-deps"
+EXTRA_OECONF:append:class-native = " --libdir=/non/existent 
--with-pc-path=/non/existent"
 
 PACKAGECONFIG ??= "glib"
 PACKAGECONFIG:class-native = ""

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158872): 
https://lists.openembedded.org/g/openembedded-core/message/158872
Mute This Topic: https://lists.openembedded.org/mt/87352798/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH 6/9] ncurses: Improve native reproducibility

[Jacob Kroon]

Avoid encoding build-specific paths in the resulting binaries.

Signed-off-by: Jacob Kroon 
---
 meta/recipes-core/ncurses/ncurses.inc | 4 
 1 file changed, 4 insertions(+)

diff --git a/meta/recipes-core/ncurses/ncurses.inc 
b/meta/recipes-core/ncurses/ncurses.inc
index a0ecd8a80b..3c15498dd4 100644
--- a/meta/recipes-core/ncurses/ncurses.inc
+++ b/meta/recipes-core/ncurses/ncurses.inc
@@ -91,10 +91,14 @@ ncurses_configure() {
--with-manpage-format=normal \
--without-manpage-renames \
--disable-stripping \
+   ${EXTRA_CLASS_FLAGS} \
"$@" || return 1
cd ..
 }
 
+EXTRA_CLASS_FLAGS = ""
+EXTRA_CLASS_FLAGS:class-native = "--datadir=/non/existent 
--with-terminfo-dirs=/non/existent"
+
 # Override the function from the autotools class; ncurses requires a
 # patched autoconf213 to generate the configure script. This autoconf
 # is not available so that the shipped script will be used.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158873): 
https://lists.openembedded.org/g/openembedded-core/message/158873
Mute This Topic: https://lists.openembedded.org/mt/87352801/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH 4/9] perl/perlcross: Improve native reproducability

[Jacob Kroon]

In order to make perlcross-native independent of build path we need to follow
the symlinks when copying the patches, otherwise they will point to whereever
oe-core is checked out for that particular build.

Doing this reveals an issue in perl-native, where it copies the patches
from perlcross-native's sysroot, but both perlcross and perl have a
patch called "determinism.patch", so one of them gets overridden. Rename
the patch in perlcross so that this doesn't happen.

Signed-off-by: Jacob Kroon 
---
 .../{determinism.patch => perl-cross-determinism.patch}  | 0
 meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb  | 4 ++--
 meta/recipes-devtools/perl/perl_5.34.0.bb| 5 +
 3 files changed, 7 insertions(+), 2 deletions(-)
 rename meta/recipes-devtools/perl-cross/files/{determinism.patch => 
perl-cross-determinism.patch} (100%)

diff --git a/meta/recipes-devtools/perl-cross/files/determinism.patch 
b/meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patch
similarity index 100%
rename from meta/recipes-devtools/perl-cross/files/determinism.patch
rename to meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patch
diff --git a/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb 
b/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb
index 2759ef8a53..dab7f4558f 100644
--- a/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb
+++ b/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb
@@ -15,7 +15,7 @@ SRC_URI = 
"https://github.com/arsv/perl-cross/releases/download/${PV}/perl-cross

file://0001-configure_tool.sh-do-not-quote-the-argument-to-comma.patch \
file://0001-perl-cross-add-LDFLAGS-when-linking-libperl.patch \

file://0001-configure_path.sh-do-not-hardcode-prefix-lib-as-libr.patch \
-   file://determinism.patch \
+   file://perl-cross-determinism.patch \

file://0001-cnf-configure_func_sel.sh-disable-thread_safe_nl_lan.patch \
file://0001-Makefile-check-the-file-if-patched-or-not.patch \
"
@@ -33,7 +33,7 @@ do_compile () {
 
 do_install:class-native() {
 mkdir -p ${D}/${datadir}/perl-cross/
-cp -rf ${S}/* ${D}/${datadir}/perl-cross/
+cp -rfL ${S}/* ${D}/${datadir}/perl-cross/
 }
 
 BBCLASSEXTEND = "native"
diff --git a/meta/recipes-devtools/perl/perl_5.34.0.bb 
b/meta/recipes-devtools/perl/perl_5.34.0.bb
index 16d45ccff3..0b74d5f072 100644
--- a/meta/recipes-devtools/perl/perl_5.34.0.bb
+++ b/meta/recipes-devtools/perl/perl_5.34.0.bb
@@ -97,6 +97,9 @@ do_configure:class-native() {
 -Dvendorprefix=${prefix} \
 -Ui_xlocale \
 ${PACKAGECONFIG_CONFARGS}
+
+# See the comment above
+sed -i -e "s,${STAGING_DIR_NATIVE},/non/existent,g" config.h
 }
 
 do_configure:append() {
@@ -395,3 +398,5 @@ SSTATE_HASHEQUIV_FILEMAP = " \
 populate_sysroot:*/lib*/perl5/config.sh:${TMPDIR} \
 populate_sysroot:*/lib*/perl5/config.sh:${COREBASE} \
 "
+
+EXTRA_STAGING_FIXMES:append:class-native = " RPATH_PADDING"

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158871): 
https://lists.openembedded.org/g/openembedded-core/message/158871
Mute This Topic: https://lists.openembedded.org/mt/87352797/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH 3/9] openssl: Improve native reproducibility

[Jacob Kroon]

The proposed changes here should probably be fixed to have
no impact on target.

Signed-off-by: Jacob Kroon 
---
 ...sysroot-and-debug-prefix-map-from-co.patch | 78 ---
 .../openssl/openssl/strip-buildinfo.patch | 13 
 .../openssl/openssl_3.0.0.bb  | 10 +--
 3 files changed, 18 insertions(+), 83 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 create mode 100644 
meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch

diff --git 
a/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 
b/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
deleted file mode 100644
index 60890c666d..00
--- 
a/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
+++ /dev/null
@@ -1,78 +0,0 @@
-From 5985253f2c9025d7c127443a3a9938946f80c2a1 Mon Sep 17 00:00:00 2001
-From: =?UTF-8?q?Martin=20Hundeb=C3=B8ll?= 
-Date: Tue, 6 Nov 2018 14:50:47 +0100
-Subject: [PATCH] buildinfo: strip sysroot and debug-prefix-map from compiler
- info
-MIME-Version: 1.0
-Content-Type: text/plain; charset=UTF-8
-Content-Transfer-Encoding: 8bit
-
-The openssl build system generates buildinf.h containing the full
-compiler command line used to compile objects. This breaks
-reproducibility, as the compile command is baked into libcrypto, where
-it is used when running `openssl version -f`.
-
-Add stripped build variables for the compiler and cflags lines, and use
-those when generating buildinfo.h.
-
-This is based on a similar patch for older openssl versions:
-https://patchwork.openembedded.org/patch/147229/
-
-Upstream-Status: Inappropriate [OE specific]
-Signed-off-by: Martin Hundebøll 
-
-Update to fix buildpaths qa issue for '-fmacro-prefix-map'.
-
-Signed-off-by: Kai Kang 
-
-Update to fix buildpaths qa issue for '-ffile-prefix-map'.
-
-Signed-off-by: Khem Raj 
-

- Configurations/unix-Makefile.tmpl | 12 +++-
- crypto/build.info |  2 +-
- 2 files changed, 12 insertions(+), 2 deletions(-)
-
-diff --git a/Configurations/unix-Makefile.tmpl 
b/Configurations/unix-Makefile.tmpl
-index f88a70f..528cdef 100644
 a/Configurations/unix-Makefile.tmpl
-+++ b/Configurations/unix-Makefile.tmpl
-@@ -471,13 +471,23 @@ BIN_LDFLAGS={- join(' ', $target{bin_lflags} || (),
-  '$(CNF_LDFLAGS)', '$(LDFLAGS)') -}
- BIN_EX_LIBS=$(CNF_EX_LIBS) $(EX_LIBS)
- 
--# CPPFLAGS_Q is used for one thing only: to build up buildinf.h
-+# *_Q variables are used for one thing only: to build up buildinf.h
- CPPFLAGS_Q={- $cppflags1 =~ s|([\\"])|\\$1|g;
-   $cppflags2 =~ s|([\\"])|\\$1|g;
-   $lib_cppflags =~ s|([\\"])|\\$1|g;
-   join(' ', $lib_cppflags || (), $cppflags2 || (),
- $cppflags1 || ()) -}
- 
-+CFLAGS_Q={- for (@{$config{CFLAGS}}) {
-+  s|-fdebug-prefix-map=[^ ]+|-fdebug-prefix-map=|g;
-+  s|-fmacro-prefix-map=[^ ]+|-fmacro-prefix-map=|g;
-+  s|-ffile-prefix-map=[^ ]+|-ffile-prefix-map=|g;
-+}
-+join(' ', @{$config{CFLAGS}}) -}
-+
-+CC_Q={- $config{CC} =~ s|--sysroot=[^ ]+|--sysroot=recipe-sysroot|g;
-+join(' ', $config{CC}) -}
-+
- PERLASM_SCHEME= {- $target{perlasm_scheme} -}
- 
- # For x86 assembler: Set PROCESSOR to 386 if you want to support
-diff --git a/crypto/build.info b/crypto/build.info
-index efca6cc..eda433e 100644
 a/crypto/build.info
-+++ b/crypto/build.info
-@@ -109,7 +109,7 @@ DEFINE[../libcrypto]=$UPLINKDEF
- 
- DEPEND[info.o]=buildinf.h
- DEPEND[cversion.o]=buildinf.h
--GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(LIB_CFLAGS) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
-+GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC_Q) $(CFLAGS_Q) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
- 
- GENERATE[uplink-x86.s]=../ms/uplink-x86.pl
- GENERATE[uplink-x86_64.s]=../ms/uplink-x86_64.pl
diff --git a/meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch 
b/meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch
new file mode 100644
index 00..0a4a60273d
--- /dev/null
+++ b/meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch
@@ -0,0 +1,13 @@
+Index: openssl-3.0.0/crypto/build.info
+===
+--- openssl-3.0.0.orig/crypto/build.info
 openssl-3.0.0/crypto/build.info
+@@ -109,7 +109,7 @@ DEFINE[../libcrypto]=$UPLINKDEF
+ 
+ DEPEND[info.o]=buildinf.h
+ DEPEND[cversion.o]=buildinf.h
+-GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(LIB_CFLAGS) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
++GENERATE[buildinf.h]=../util/mkbuildinf.pl "empty"
+ 
+ GENERATE[uplink-x86.s]=../ms/uplink-x86.pl
+ GENERATE[uplink-x86_64.s]=../ms/uplink-x86_

[OE-core] [RFC PATCH 2/9] libtool: Improve native reproducibility

[Jacob Kroon]

Avoid encoding build-specific paths in the resulting binaries.

Signed-off-by: Jacob Kroon 
---
 meta/recipes-devtools/libtool/libtool-native_2.4.6.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-devtools/libtool/libtool-native_2.4.6.bb 
b/meta/recipes-devtools/libtool/libtool-native_2.4.6.bb
index 3b20ce3e69..ea19b86d4a 100644
--- a/meta/recipes-devtools/libtool/libtool-native_2.4.6.bb
+++ b/meta/recipes-devtools/libtool/libtool-native_2.4.6.bb
@@ -7,6 +7,7 @@ SRC_URI += "file://prefix.patch"
 inherit native
 
 EXTRA_OECONF = " --with-libtool-sysroot=${STAGING_DIR_NATIVE}"
+CACHED_CONFIGUREVARS += "lt_cv_sys_dlsearch_path=/non/existent"
 
 do_configure:prepend () {
# Remove any existing libtool m4 since old stale versions would break

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158869): 
https://lists.openembedded.org/g/openembedded-core/message/158869
Mute This Topic: https://lists.openembedded.org/mt/87352795/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH 1/9] bitbake.conf: Pad rpath and remove build ID in native binaries

[Jacob Kroon]

Try to make sure that the RUNTIME dynamic entry size is the same for all
binaries produced with the native compiler. This is necessary in order to
produce identical binaries when using differently sized buildpaths. I've
tried using only patchelf, and keeping the linker flags as they are, but
I am unable to produce identical binaries. Has anyone else managed to do
this with patchelf ? If not, maybe we can write a new tool that can handle it ?

The build-id also needs to be removed since it is calculated based on
the data present at link time. This includes STAGING_LIBDIR_NATIVE
and STAGING_BASE_LIBDIR_NATIVE. Both will differ and they need to be temporarily
preserved since some recipes will execute the binaries during do_install()
(for example python3-native). Later on these are removed in chrpath.bbclass.

This hack is the first step for producing identical native binaries when using
different build paths. 'zstd-native' is a working example.

Signed-off-by: Jacob Kroon 
---
 meta/classes/chrpath.bbclass | 3 +++
 meta/conf/bitbake.conf   | 6 +-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/meta/classes/chrpath.bbclass b/meta/classes/chrpath.bbclass
index 26b984c4db..3c6956304a 100644
--- a/meta/classes/chrpath.bbclass
+++ b/meta/classes/chrpath.bbclass
@@ -24,6 +24,9 @@ def process_file_linux(cmd, fpath, rootdir, baseprefix, 
tmpdir, d, break_hardlin
 new_rpaths = []
 modified = False
 for rpath in rpaths:
+if rpath.startswith('rpath-padding-'):
+modified = True
+continue
 # If rpath is already dynamic copy it to new_rpath and continue
 if rpath.find("$ORIGIN") != -1:
 new_rpaths.append(rpath)
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index fba99e8f0c..9621023354 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -580,6 +580,8 @@ BUILDSDK_CXXFLAGS = "${BUILDSDK_CFLAGS}"
 export CXXFLAGS = "${TARGET_CXXFLAGS}"
 TARGET_CXXFLAGS = "${TARGET_CFLAGS}"
 
+RPATH_PADDING ?= "rpath-padding-${@'x' * (512 - 
len(d.expand('${STAGING_LIBDIR_NATIVE}:${STAGING_BASE_LIBDIR_NATIVE}:rpath-padding-')))}"
+RPATH_PADDING_FLAG ?= "-Wl,-rpath,${RPATH_PADDING}"
 export BUILD_LDFLAGS = "-L${STAGING_LIBDIR_NATIVE} \
 -L${STAGING_BASE_LIBDIR_NATIVE} \
 -Wl,--enable-new-dtags \
@@ -587,7 +589,9 @@ export BUILD_LDFLAGS = "-L${STAGING_LIBDIR_NATIVE} \
 -Wl,-rpath-link,${STAGING_BASE_LIBDIR_NATIVE} \
 -Wl,-rpath,${STAGING_LIBDIR_NATIVE} \
 -Wl,-rpath,${STAGING_BASE_LIBDIR_NATIVE} \
--Wl,-O1"
+${RPATH_PADDING_FLAG} \
+-Wl,-O1 \
+-Wl,--build-id=none"
 
 BUILDSDK_LDFLAGS = "-Wl,-O1"
 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158868): 
https://lists.openembedded.org/g/openembedded-core/message/158868
Mute This Topic: https://lists.openembedded.org/mt/87352793/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH 0/9] Improve native/cross reproducibility

[Jacob Kroon]

This patch series is not intended for merge. I only send it out to
highlight where the problems are and to get some discussion going on
how/if we want to improve the sitation.

This is a patch series that tries to improve the reproducibility of the
native/cross binaries when building in different directories. This has
been tested on a Fedora 35 system which uses gcc 11.2.1 at the time of
writing.

The RUNTIME hack is questionable, maybe there is a better way to enforce
a fixed RUNTIME entry size during linking. It probably breaks for
recipes that do additional rpath manipulations at link-time.

If in the end we do come up with a solution, then it should be tested on
the autobuilders, since otherwise this is going to degrade overtime. It
would then be important that the build paths are of significantly different
lengths. TMPDIR=/tmp/sysrootA/ and TMPDIR=/tmp/sysrootB/ will *not* uncover all
rpath problems.

The end result of this patch series is that I can build python3-native
in two different build paths, and the resuling sysroot-components/x86_64/
directories are identical, except for the 'fixmepath.cmd' files, which
are not included in the hash equiv calculations. Even so, there remains a lot of
other native builds that are going to need to be fixed in similar ways
as the ones in this patch series.

For my images to build I had to avoid the rpath-hack for icedtea7-native
and openjdk-8-native.

/Jacob

Jacob Kroon (9):
  bitbake.conf: Pad rpath and remove build ID in native binaries
  libtool: Improve native reproducibility
  openssl: Improve native reproducibility
  perl/perlcross: Improve native reproducability
  pkgconfig: Improve native reproducibility
  ncurses: Improve native reproducibility
  util-linux: Improve native reproducibility
  python3: Improve native reproducibility
  bitbake.conf: Avoid rpath hack for Java recipes

 meta/classes/chrpath.bbclass  |  3 +
 meta/conf/bitbake.conf|  8 +-
 ...sysroot-and-debug-prefix-map-from-co.patch | 78 ---
 .../openssl/openssl/strip-buildinfo.patch | 13 
 .../openssl/openssl_3.0.0.bb  | 10 +--
 meta/recipes-core/ncurses/ncurses.inc |  4 +
 .../util-linux/util-linux_2.37.2.bb   |  2 +-
 .../libtool/libtool-native_2.4.6.bb   |  1 +
 ...ism.patch => perl-cross-determinism.patch} |  0
 .../perl-cross/perlcross_1.3.6.bb |  4 +-
 meta/recipes-devtools/perl/perl_5.34.0.bb |  5 ++
 .../pkgconfig/pkgconfig_git.bb|  1 +
 .../python/python3/determinism.patch  | 15 
 .../recipes-devtools/python/python3_3.10.0.bb |  8 ++
 14 files changed, 65 insertions(+), 87 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 create mode 100644 
meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch
 rename meta/recipes-devtools/perl-cross/files/{determinism.patch => 
perl-cross-determinism.patch} (100%)
 create mode 100644 meta/recipes-devtools/python/python3/determinism.patch


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158867): 
https://lists.openembedded.org/g/openembedded-core/message/158867
Mute This Topic: https://lists.openembedded.org/mt/87352786/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [docs] [PATCH] dev-manual: how to purge duplicate sstate cache files

[Jacob Kroon]

On 11/26/21 18:23, Michael Opdenacker wrote:
> Hi Jacob,
> 
> Thank you for the suggestion!
> 
> On 11/26/21 10:20 AM, Jacob Kroon wrote:
>> I also use the --stamps-dir method since that removes additional files.
>> You just gotta remember to do a build of everything you want to keep first.
> 
> 
> I don't understand the value of this option in the simple case when you
> have a single build directory.  Here's the description of this option:
> 
>   --stamps-dir=,...
>     Specify the build directory's stamps directories, the sstate
>     cache file which IS USED by these build diretories will be KEPT,
>     other sstate cache files in cache-dir will be removed. Use ","
>     as the separator. For example:
>     --stamps-dir=build1/tmp/stamps,build2/tmp/stamps
> 
> What am I missing?
> 

--stamps-dir=${TMPDIR}/stamps

in a default build dir I think that would be "build/tmp-glibc/stamps"

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158841): 
https://lists.openembedded.org/g/openembedded-core/message/158841
Mute This Topic: https://lists.openembedded.org/mt/87255323/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [docs] [PATCH] dev-manual: how to purge duplicate sstate cache files

[Jacob Kroon]

Hi Michael,

On 11/26/21 10:13, Vyacheslav Yurkov wrote:
> Hi Michael
> 
> On 23.11.2021 10:14, Michael Opdenacker wrote:
>> Greetings,
>>
>> On 11/19/21 5:06 PM, Michael Opdenacker wrote:
>>> Signed-off-by: Michael Opdenacker 
>>> ---
>>>   documentation/dev-manual/common-tasks.rst | 25 ++-
>>>   1 file changed, 24 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/documentation/dev-manual/common-tasks.rst
>>> b/documentation/dev-manual/common-tasks.rst
>>> index 3eead147a3..37612c84b7 100644
>>> --- a/documentation/dev-manual/common-tasks.rst
>>> +++ b/documentation/dev-manual/common-tasks.rst
>>> @@ -6242,8 +6242,11 @@ Changing the listed common targets is as easy
>>> as editing your version of
>>>   ``conf-notes.txt`` in your custom template configuration directory and
>>>   making sure you have ``TEMPLATECONF`` set to your directory.
>>>   +Conserving Disk Space
>>> +=
>>> +
>>>   Conserving Disk Space During Builds
>>> -===
>>> +---
>>>     To help conserve disk space during builds, you can add the following
>>>   statement to your project's ``local.conf`` configuration file found in
>>> @@ -6257,6 +6260,26 @@ building a recipe once the recipe is built.
>>> For more information on
>>>   :ref:`rm_work ` class in the
>>>   Yocto Project Reference Manual.
>>>   +Purging Duplicate Shared State Cache Files
>>> +---
>>> +
>>> +After multiple build iterations, the Shared State (sstate) cache can
>>> contain
>>> +duplicate cache files for a given package, while only the most
>>> recent one
>>> +is likely to be reusable. The following command purges all but the
>>> +newest sstate cache file for each package::
>>> +
>>> +   sstate-cache-management.sh --remove-duplicated
>>> --cache-dir=build/sstate-cache
>>> +
>>> +This command will ask you to confirm the deletions it identifies.
>>> +
>>> +Note::
>>> +
>>> +   The duplicated sstate cache files of one package must have the same
>>> +   architecture, which means that sstate cache files with multiple
>>> +   architectures are not considered as duplicate.
>>> +
>>> +Run ``sstate-cache-management.sh`` for more details about this script.
>>> +
>>>   Working with Packages
>>>   =
>>>   
>> Did anyone have time to have a look at what I wrote ? There may be
>> better ways to purge the sstate cache files.
>> Thanks in advance
>> Michael.
> 
> The description looks good to me. By other ways you mean other usage
> scenarios of sstate-cache-management.sh script or by means of something
> else?
> 

I also use the --stamps-dir method since that removes additional files.
You just gotta remember to do a build of everything you want to keep first.

/Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158828): 
https://lists.openembedded.org/g/openembedded-core/message/158828
Mute This Topic: https://lists.openembedded.org/mt/87255323/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCHv2] native/cross: Add ar wrapper for determinism

[Jacob Kroon]

Add a wrapper around ar calls for native/cross recipes. This wrapper adds
the -D option so that deterministic archives are built for native/cross
output. This improves the changes of hash equivalence matches and hence
build artefact reuse.

We don't need this in the target case since we compile binutils-cross
with an option making this the default. We need a wrapper since we need
to remove the "u" option and replace it with "D" but also allow things like
"--version" to continue to work too.

Signed-off-by: Jacob Kroon 
---

Changes in v2:
 * Don't modify arguments if 'U' is given, instead print a message to
   stderr that non-deterministic mode is requested

 meta/classes/cross.bbclass  |  2 ++
 scripts/cross-intercept/ar  |  1 +
 scripts/native-intercept/ar | 32 
 3 files changed, 35 insertions(+)
 create mode 12 scripts/cross-intercept/ar
 create mode 100755 scripts/native-intercept/ar

diff --git a/meta/classes/cross.bbclass b/meta/classes/cross.bbclass
index 3e6a2f60b9..9d951076a7 100644
--- a/meta/classes/cross.bbclass
+++ b/meta/classes/cross.bbclass
@@ -93,3 +93,5 @@ python do_addto_recipe_sysroot () {
 }
 addtask addto_recipe_sysroot after do_populate_sysroot
 do_addto_recipe_sysroot[deptask] = "do_populate_sysroot"
+
+PATH:prepend = "${COREBASE}/scripts/cross-intercept:"
diff --git a/scripts/cross-intercept/ar b/scripts/cross-intercept/ar
new file mode 12
index 00..bc68ffd7a2
--- /dev/null
+++ b/scripts/cross-intercept/ar
@@ -0,0 +1 @@
+../native-intercept/ar
\ No newline at end of file
diff --git a/scripts/native-intercept/ar b/scripts/native-intercept/ar
new file mode 100755
index 00..dcc623e3ed
--- /dev/null
+++ b/scripts/native-intercept/ar
@@ -0,0 +1,32 @@
+#!/usr/bin/env python3
+#
+# Wrapper around 'ar' that defaults to deterministic archives
+
+import os
+import shutil
+import sys
+
+# calculate path to the real 'ar'
+path = os.environ['PATH']
+path = path.replace(os.path.dirname(sys.argv[0]), '')
+real_ar = shutil.which('ar', path=path)
+
+if len(sys.argv) == 1:
+os.execl(real_ar, 'ar')
+
+# modify args to mimic 'ar' configured with --default-deterministic-archives
+argv = sys.argv
+if argv[1].startswith('--'):
+# No modifier given
+None
+else:
+# remove the optional '-'
+if argv[1][0] == '-':
+argv[1] = argv[1][1:]
+if 'U' in argv[1]:
+sys.stderr.write("ar: non-deterministic mode requested\n")
+else:
+argv[1] = argv[1].replace('u', '')
+argv[1] = 'D' + argv[1]
+
+os.execv(real_ar, argv)

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158654): 
https://lists.openembedded.org/g/openembedded-core/message/158654
Mute This Topic: https://lists.openembedded.org/mt/87275969/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] native/cross: Add ar wrapper for determinism

[Jacob Kroon]

On 11/24/21 00:06, Richard Purdie wrote:
> On Tue, 2021-11-23 at 21:22 +0100, Jacob Kroon wrote:
>>
>> On 11/23/21 15:12, Richard Purdie wrote:
>>> From: Jacob Kroon 
>>>
>>> Add a wrapper around ar calls for native/cross recipes. This wrapper adds
>>> the -D option so that deterministic archives are built for native/cross
>>> output. This improves the changes of hash equivalence matches and hence
>>> build artefact reuse.
>>>
>>> We don't need this in the target case since we compile binutils-cross
>>> with an option making this the default. We need a wrapper since we need
>>> to remove the "u" option and replace it with "D" but also allow things like
>>> "--version" to continue to work too.
>>>
>>> Signed-off-by: Jacob Kroon 
>>> Signed-off-by: Richard Purdie 
>>> ---
>>>  meta/classes/cross.bbclass  |  2 ++
>>>  scripts/cross-intercept/ar  |  1 +
>>>  scripts/native-intercept/ar | 29 +
>>>  3 files changed, 32 insertions(+)
>>>  create mode 12 scripts/cross-intercept/ar
>>>  create mode 100755 scripts/native-intercept/ar
>>>
>>> diff --git a/meta/classes/cross.bbclass b/meta/classes/cross.bbclass
>>> index 3e6a2f60b9e..9d951076a78 100644
>>> --- a/meta/classes/cross.bbclass
>>> +++ b/meta/classes/cross.bbclass
>>> @@ -93,3 +93,5 @@ python do_addto_recipe_sysroot () {
>>>  }
>>>  addtask addto_recipe_sysroot after do_populate_sysroot
>>>  do_addto_recipe_sysroot[deptask] = "do_populate_sysroot"
>>> +
>>> +PATH:prepend = "${COREBASE}/scripts/cross-intercept:"
>>> diff --git a/scripts/cross-intercept/ar b/scripts/cross-intercept/ar
>>> new file mode 12
>>> index 000..bc68ffd7a21
>>> --- /dev/null
>>> +++ b/scripts/cross-intercept/ar
>>> @@ -0,0 +1 @@
>>> +../native-intercept/ar
>>> \ No newline at end of file
>>> diff --git a/scripts/native-intercept/ar b/scripts/native-intercept/ar
>>> new file mode 100755
>>> index 000..642b6eae864
>>> --- /dev/null
>>> +++ b/scripts/native-intercept/ar
>>> @@ -0,0 +1,29 @@
>>> +#!/usr/bin/env python3
>>> +#
>>> +# Wrapper around 'ar' that defaults to deterministic archives
>>> +
>>> +import os
>>> +import shutil
>>> +import sys
>>> +
>>> +# calculate path to the real 'ar'
>>> +path = os.environ['PATH']
>>> +path = path.replace(os.path.dirname(sys.argv[0]), '')
>>> +real_ar = shutil.which('ar', path=path)
>>> +
>>> +if len(sys.argv) == 1:
>>> +os.execl(real_ar, 'ar')
>>> +
>>> +# modify args to mimic 'ar' configured with 
>>> --default-deterministic-archives
>>> +argv = sys.argv
>>> +if argv[1].startswith('--'):
>>> +# No modifier given
>>> +None
>>> +else:
>>> +# remove the optional '-'
>>> +if argv[1][0] == '-':
>>> +argv[1] = argv[1][1:]
>>> +argv[1] = argv[1].replace('u', '')
>>> +argv[1] = 'D' + argv[1]
>>> +
>>> +os.execv(real_ar, argv)
>>>
>>
>> What should we do if the user is explicitly passing U, "Do not operate
>> in deterministic mode." ? Don't replace any 'u' and don't pass D ?
> 
> That would seem to make sense to me...
> 

Could we log a warning somewhere when this is done ? I can't write to
stdout here, but stderr perhaps ?

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158646): 
https://lists.openembedded.org/g/openembedded-core/message/158646
Mute This Topic: https://lists.openembedded.org/mt/87259014/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] native/cross: Add ar wrapper for determinism

[Jacob Kroon]

On 11/23/21 15:12, Richard Purdie wrote:
> From: Jacob Kroon 
> 
> Add a wrapper around ar calls for native/cross recipes. This wrapper adds
> the -D option so that deterministic archives are built for native/cross
> output. This improves the changes of hash equivalence matches and hence
> build artefact reuse.
> 
> We don't need this in the target case since we compile binutils-cross
> with an option making this the default. We need a wrapper since we need
> to remove the "u" option and replace it with "D" but also allow things like
> "--version" to continue to work too.
> 
> Signed-off-by: Jacob Kroon 
> Signed-off-by: Richard Purdie 
> ---
>  meta/classes/cross.bbclass  |  2 ++
>  scripts/cross-intercept/ar  |  1 +
>  scripts/native-intercept/ar | 29 +
>  3 files changed, 32 insertions(+)
>  create mode 12 scripts/cross-intercept/ar
>  create mode 100755 scripts/native-intercept/ar
> 
> diff --git a/meta/classes/cross.bbclass b/meta/classes/cross.bbclass
> index 3e6a2f60b9e..9d951076a78 100644
> --- a/meta/classes/cross.bbclass
> +++ b/meta/classes/cross.bbclass
> @@ -93,3 +93,5 @@ python do_addto_recipe_sysroot () {
>  }
>  addtask addto_recipe_sysroot after do_populate_sysroot
>  do_addto_recipe_sysroot[deptask] = "do_populate_sysroot"
> +
> +PATH:prepend = "${COREBASE}/scripts/cross-intercept:"
> diff --git a/scripts/cross-intercept/ar b/scripts/cross-intercept/ar
> new file mode 12
> index 000..bc68ffd7a21
> --- /dev/null
> +++ b/scripts/cross-intercept/ar
> @@ -0,0 +1 @@
> +../native-intercept/ar
> \ No newline at end of file
> diff --git a/scripts/native-intercept/ar b/scripts/native-intercept/ar
> new file mode 100755
> index 000..642b6eae864
> --- /dev/null
> +++ b/scripts/native-intercept/ar
> @@ -0,0 +1,29 @@
> +#!/usr/bin/env python3
> +#
> +# Wrapper around 'ar' that defaults to deterministic archives
> +
> +import os
> +import shutil
> +import sys
> +
> +# calculate path to the real 'ar'
> +path = os.environ['PATH']
> +path = path.replace(os.path.dirname(sys.argv[0]), '')
> +real_ar = shutil.which('ar', path=path)
> +
> +if len(sys.argv) == 1:
> +os.execl(real_ar, 'ar')
> +
> +# modify args to mimic 'ar' configured with --default-deterministic-archives
> +argv = sys.argv
> +if argv[1].startswith('--'):
> +# No modifier given
> +None
> +else:
> +# remove the optional '-'
> +if argv[1][0] == '-':
> +argv[1] = argv[1][1:]
> +argv[1] = argv[1].replace('u', '')
> +argv[1] = 'D' + argv[1]
> +
> +os.execv(real_ar, argv)
> 

What should we do if the user is explicitly passing U, "Do not operate
in deterministic mode." ? Don't replace any 'u' and don't pass D ?

Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158618): 
https://lists.openembedded.org/g/openembedded-core/message/158618
Mute This Topic: https://lists.openembedded.org/mt/87259014/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] how to "prune" dead sstate-cache?

[Jacob Kroon]

On 11/18/21 11:48, Alexander Kanavin wrote:
It would be also nice to confirm that the script is in use by someone, 
and functions properly. There's a ton of stuff under scripts/  that 
isn't getting a lot of attention or testing.




I use it regularly.

/Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158473): 
https://lists.openembedded.org/g/openembedded-core/message/158473
Mute This Topic: https://lists.openembedded.org/mt/87140377/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [hardknott][PATCH 00/14] Review request

[Jacob Kroon]

On 11/16/21 04:24, Anuj Mittal wrote:

Please review these changes for hardknott. No issues seen while testing
on autobuilder.

Thanks,

Anuj

The following changes since commit 0ca080a23c2770a15138f702d4c879bbd90ca360:

   build-appliance-image: Update to hardknott head revision (2021-11-04 
11:58:28 +)

are available in the Git repository at:

   git://push.openembedded.org/openembedded-core-contrib anujm/hardknott

Alexander Kanavin (2):
   linux-firmware: upgrade 20210919 -> 20211027
   cross-canadian: correct the location of pkg-config files

Anuj Mittal (2):
   meta: add explicit branch and protocol to SRC_URI
   llvm: bump HASHEQUIV_HASH_VERSION

Bruce Ashfield (2):
   linux-yocto/5.10: update to v5.10.76
   linux-yocto-rt/5.10: update to -rt54

Chen Qi (1):
   avahi: update CVE id fixed by local-ping.patch

Jose Quaresma (1):
   sstate: another fix for touching files inside pseudo

Manuel Leonhardt (1):
   sstate: Account for reserved characters when shortening sstate
 filenames

Richard Purdie (5):
   linunistring: Add missing gperf-native dependency
   pseudo: Add in ability to flush database with shutdown request
   pseudo: Add fcntl64 wrapper
   scripts/convert-srcuri: Backport SRC_URI conversion script from master
 branch
   meta/scripts: Manual git url branch additions

  .../devtool/devtool-upgrade-test2_git.bb  |  2 +-
  .../devtool-upgrade-test2_git.bb.upgraded |  2 +-
  .../git-submodule-test/git-submodule-test.bb  |  2 +-
  meta/classes/cross-canadian.bbclass   |  2 +-
  meta/classes/devupstream.bbclass  |  2 +-
  meta/classes/sstate.bbclass   | 14 ++--
  .../distro/include/default-distrovars.inc |  2 +-
  meta/lib/oeqa/selftest/cases/devtool.py   |  4 +-
  meta/lib/oeqa/selftest/cases/fetch.py |  2 +-
  meta/lib/oeqa/selftest/cases/recipetool.py|  6 +-
  meta/lib/oeqa/selftest/cases/sstatetests.py   |  2 +-
  meta/recipes-bsp/efibootmgr/efibootmgr_17.bb  |  2 +-
  meta/recipes-bsp/efivar/efivar_37.bb  |  2 +-
  meta/recipes-bsp/opensbi/opensbi_0.9.bb   |  2 +-
  meta/recipes-bsp/u-boot/libubootenv_0.3.1.bb  |  2 +-
  meta/recipes-bsp/u-boot/u-boot-common.inc |  2 +-
  .../avahi/files/local-ping.patch  |  1 +
  .../connman/connman-gnome_0.7.bb  |  2 +-
  .../libnss-mdns/libnss-mdns_0.14.1.bb |  2 +-
  .../libuv/libuv_1.41.0.bb |  2 +-
  .../mobile-broadband-provider-info_git.bb |  2 +-
  meta/recipes-core/dbus-wait/dbus-wait_git.bb  |  2 +-
  meta/recipes-core/fts/fts_1.2.7.bb|  2 +-
  .../glibc/cross-localedef-native_2.33.bb  |  2 +-
  meta/recipes-core/ifupdown/ifupdown_0.8.36.bb |  2 +-
  .../initscripts/init-system-helpers_1.60.bb   |  2 +-
  meta/recipes-core/libxcrypt/libxcrypt.inc |  2 +-
  meta/recipes-core/musl/libucontext_git.bb |  2 +-
  meta/recipes-core/musl/musl-obstack.bb|  2 +-
  meta/recipes-core/musl/musl-utils.bb  |  2 +-
  meta/recipes-core/musl/musl_git.bb|  2 +-
  meta/recipes-core/ncurses/ncurses.inc |  2 +-
  meta/recipes-core/netbase/netbase_6.2.bb  |  2 +-
  meta/recipes-core/psplash/psplash_git.bb  |  2 +-
  meta/recipes-core/systemd/systemd.inc |  2 +-
  .../update-rc.d/update-rc.d_0.8.bb|  2 +-
  .../bootchart2/bootchart2_0.14.9.bb   |  2 +-
  .../btrfs-tools/btrfs-tools_5.10.1.bb |  2 +-
  .../createrepo-c/createrepo-c_0.17.0.bb   |  2 +-
  meta/recipes-devtools/distcc/distcc_3.3.5.bb  |  2 +-
  meta/recipes-devtools/dnf/dnf_4.6.0.bb|  2 +-
  meta/recipes-devtools/e2fsprogs/e2fsprogs.inc |  2 +-
  meta/recipes-devtools/file/file_5.39.bb   |  2 +-
  meta/recipes-devtools/glide/glide_0.13.3.bb   |  2 +-
  .../gnu-config/gnu-config_git.bb  |  2 +-
  .../libcomps/libcomps_0.1.15.bb   |  2 +-
  meta/recipes-devtools/libdnf/libdnf_0.58.0.bb |  2 +-
  .../librepo/librepo_1.13.0.bb |  2 +-
  meta/recipes-devtools/llvm/llvm_git.bb|  6 +-
  meta/recipes-devtools/mtd/mtd-utils_git.bb|  2 +-
  meta/recipes-devtools/ninja/ninja_1.10.2.bb   |  2 +-
  .../patchelf/patchelf_0.12.bb |  2 +-
  meta/recipes-devtools/pseudo/pseudo_git.bb|  2 +-
  meta/recipes-devtools/rpm/rpm_4.16.1.3.bb |  2 +-
  .../squashfs-tools/squashfs-tools_git.bb  |  2 +-
  .../systemd-bootchart_234.bb  |  2 +-
  .../tcf-agent/tcf-agent_git.bb|  2 +-
  meta/recipes-devtools/unfs3/unfs3_git.bb  |  2 +-
  meta/recipes-extended/bzip2/bzip2_1.0.8.bb|  2 +-
  .../go-examples/go-helloworld_0.1.bb  |  2 +-
  .../iputils/iputils_s20200821.bb  |  2 +-
  .../recipes-extended/libaio/libaio_0.3.112.bb |  2 +-
  meta/recipes-extended/libnsl/libnsl2_git.bb   |  2 +-
  .../recipes-extended/libnss-nis/libnss-nis.bb |  2 +-
  .../libsolv/libsolv_0.7.17.bb |  2 +-
  

Re: [OE-core] [hardknott][PATCH] mklibs: remove recipes and class

[Jacob Kroon]

On 11/15/21 01:59, Khem Raj wrote:

On Sun, Nov 14, 2021 at 11:47 AM Jacob Kroon  wrote:


On 11/14/21 20:36, Richard Purdie wrote:

On Sun, 2021-11-14 at 07:50 +0100, Jacob Kroon wrote:

From: Alexander Kanavin 

This is not enabled or tested by default, and has never been
ported to python 3 upstream[1], which means it doesn't work at all
with plain poky. If you need it, please put it in a separate layer
and/or modernize to work with py3.

https://salsa.debian.org/installer-team/mklibs/-/blob/master/src/mklibs

Signed-off-by: Alexander Kanavin 
Signed-off-by: Jacob Kroon 
Signed-off-by: Richard Purdie 
(cherry picked from commit 908df863b419d1cad7317153101fc827e7e3a354)


What issue is it causing in hardknott? Normally we don't backport invasive
changes like that...



The problem is that mklibs-native doesn't build with gcc 11, and
"image-mklibs" was in USER_CLASSES by default in hardknott.

I sent a patch to fix mklibs build here:

https://lists.openembedded.org/g/openembedded-core/message/151982


This fix is correct and it is trivial. We can apply it



but after some discussions we just decided to drop mklibs support
entirely with the proposed patch.


Was there any other reason ?



To have it successfully build (with mklibs actually enabled by putting 
your image in MKLIBS_OPTIMIZED_IMAGES) would require additional fixes, 
including updating mklibs to python3 I believe.


/Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158275): 
https://lists.openembedded.org/g/openembedded-core/message/158275
Mute This Topic: https://lists.openembedded.org/mt/87042685/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [hardknott][PATCH] mklibs-native: drop deprecated cpp17 exceptions

[Jacob Kroon]

From: Andrej Valek 

gcc11 has -std=gnu++17 as default. Remove deprecated C++17 exceptions based
on http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0003r5.html.

Signed-off-by: Andrej Valek 
Signed-off-by: Steve Sakoman 
(cherry picked from commit ef8b7946b4793db653ef7dd716e1d3f919a84725)
---
 ...ecated-exception-specification-cpp17.patch | 431 ++
 .../mklibs/mklibs-native_0.1.44.bb|   1 +
 2 files changed, 432 insertions(+)
 create mode 100644 
meta/recipes-devtools/mklibs/files/remove-deprecated-exception-specification-cpp17.patch

diff --git 
a/meta/recipes-devtools/mklibs/files/remove-deprecated-exception-specification-cpp17.patch
 
b/meta/recipes-devtools/mklibs/files/remove-deprecated-exception-specification-cpp17.patch
new file mode 100644
index 00..f96cc7d302
--- /dev/null
+++ 
b/meta/recipes-devtools/mklibs/files/remove-deprecated-exception-specification-cpp17.patch
@@ -0,0 +1,431 @@
+From 597c7a8333df84a87cc48fb8477b603ffbf372a6 Mon Sep 17 00:00:00 2001
+From: Andrej Valek 
+Date: Mon, 23 Aug 2021 12:45:11 +0200
+Subject: [PATCH] feat(cpp17): remove deprecated exception specifications for
+ C++ 17
+
+Upstream-Status: Submitted 
[https://salsa.debian.org/installer-team/mklibs/-/merge_requests/2]
+
+based on: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0003r5.html
+
+Signed-off-by: Andrej Valek 
+---
+ src/mklibs-readelf/elf.cpp  | 48 -
+ src/mklibs-readelf/elf.hpp  | 18 
+ src/mklibs-readelf/elf_data.hpp | 36 +++
+ 3 files changed, 51 insertions(+), 51 deletions(-)
+
+diff --git a/src/mklibs-readelf/elf.cpp b/src/mklibs-readelf/elf.cpp
+index 0e4c0f3..2e6d0f6 100644
+--- a/src/mklibs-readelf/elf.cpp
 b/src/mklibs-readelf/elf.cpp
+@@ -36,7 +36,7 @@ file::~file () throw ()
+ delete *it;
+ }
+ 
+-file *file::open (const char *filename) throw (std::bad_alloc, 
std::runtime_error)
++file *file::open (const char *filename) throw ()
+ {
+   struct stat buf;
+   int fd;
+@@ -72,7 +72,7 @@ file *file::open (const char *filename) throw 
(std::bad_alloc, std::runtime_erro
+ }
+ 
+ template
+-file *file::open_class(uint8_t *mem, size_t len) throw (std::bad_alloc, 
std::runtime_error)
++file *file::open_class(uint8_t *mem, size_t len) throw ()
+ {
+   switch (mem[EI_DATA])
+   {
+@@ -86,7 +86,7 @@ file *file::open_class(uint8_t *mem, size_t len) throw 
(std::bad_alloc, std::run
+ }
+ 
+ template 
+-file_data<_class, _data>::file_data(uint8_t *mem, size_t len) throw 
(std::bad_alloc, std::runtime_error)
++file_data<_class, _data>::file_data(uint8_t *mem, size_t len) throw ()
+ : file(mem, len)
+ {
+   if (mem[EI_CLASS] != _class::id)
+@@ -190,7 +190,7 @@ section_data<_class, _data>::section_data(Shdr *shdr, 
uint8_t *mem) throw ()
+ }
+ 
+ template 
+-void section_data<_class, _data>::update(const file ) throw 
(std::bad_alloc)
++void section_data<_class, _data>::update(const file ) throw ()
+ {
+   const section_type  =
+ dynamic_cast 
&>(file.get_section(file.get_shstrndx()));
+@@ -204,7 +204,7 @@ section_type::~section_type() throw 
()
+ }
+ 
+ template 
+-section_real<_class, _data, section_type_DYNAMIC>::section_real(Shdr *header, 
uint8_t *mem) throw (std::bad_alloc)
++section_real<_class, _data, section_type_DYNAMIC>::section_real(Shdr *header, 
uint8_t *mem) throw ()
+ : section_data<_class, _data>(header, mem)
+ {
+   if (this->type != SHT_DYNAMIC)
+@@ -221,7 +221,7 @@ section_real<_class, _data, 
section_type_DYNAMIC>::section_real(Shdr *header, ui
+ }
+ 
+ template 
+-void section_real<_class, _data, section_type_DYNAMIC>::update(const file 
) throw (std::bad_alloc)
++void section_real<_class, _data, section_type_DYNAMIC>::update(const file 
) throw ()
+ {
+   section_data<_class, _data>::update(file);
+ 
+@@ -243,7 +243,7 @@ section_type::~section_type() throw ()
+ }
+ 
+ template 
+-section_real<_class, _data, section_type_DYNSYM>::section_real(Shdr *header, 
uint8_t *mem) throw (std::bad_alloc)
++section_real<_class, _data, section_type_DYNSYM>::section_real(Shdr *header, 
uint8_t *mem) throw ()
+ : section_data<_class, _data>(header, mem)
+ {
+   if (this->type != SHT_DYNSYM)
+@@ -260,7 +260,7 @@ section_real<_class, _data, 
section_type_DYNSYM>::section_real(Shdr *header, uin
+ }
+ 
+ template 
+-void section_real<_class, _data, section_type_DYNSYM>::update(const file 
) throw (std::bad_alloc)
++void section_real<_class, _data, section_type_DYNSYM>::update(const file 
) throw ()
+ {
+   section_data<_class, _data>::update (file);
+ 
+@@ -285,7 +285,7 @@ const version_definition 
*section_type::get_version_def
+ }
+ 
+ template 
+-section_real<_class, _data, section_type_GNU_VERDEF>::section_real(Shdr 
*header, uint8_t *mem) throw (std::bad_alloc)
++section_real<_class, _data, section_type_GNU_VERDEF>::section_real(Shdr 
*header, uint8_t *mem) throw ()
+ : section_data<_class, _data>(header, mem)
+ {
+   if (this->type != 

Re: [OE-core] [hardknott][PATCH] mklibs: remove recipes and class

[Jacob Kroon]

On 11/14/21 20:51, Richard Purdie wrote:

On Sun, 2021-11-14 at 20:47 +0100, Jacob Kroon wrote:

On 11/14/21 20:36, Richard Purdie wrote:

On Sun, 2021-11-14 at 07:50 +0100, Jacob Kroon wrote:

From: Alexander Kanavin 

This is not enabled or tested by default, and has never been
ported to python 3 upstream[1], which means it doesn't work at all
with plain poky. If you need it, please put it in a separate layer
and/or modernize to work with py3.

https://salsa.debian.org/installer-team/mklibs/-/blob/master/src/mklibs

Signed-off-by: Alexander Kanavin 
Signed-off-by: Jacob Kroon 
Signed-off-by: Richard Purdie 
(cherry picked from commit 908df863b419d1cad7317153101fc827e7e3a354)


What issue is it causing in hardknott? Normally we don't backport invasive
changes like that...



The problem is that mklibs-native doesn't build with gcc 11, and
"image-mklibs" was in USER_CLASSES by default in hardknott.

I sent a patch to fix mklibs build here:

https://lists.openembedded.org/g/openembedded-core/message/151982

but after some discussions we just decided to drop mklibs support
entirely with the proposed patch.


ok, that is something we should fix. My worry is that the backport as is will
need everyone to update their local.conf since the class will no longer be there
and it will throw errors if people try and include it. This isn't what people
expect from a point release.

Could we just remove it from local.conf in hardknott? That should solve the
issue but reduce risk to anyone who has hardknott already working fine with an
older gcc.



Yeah I'm ok with that, I'm even ok with leaving it as it is :-)

Or perhaps we should forward-port

ef8b7946b4793db653ef7dd716e1d3f919a84725

in dunfell branch to hardknott, which also should take care of it.

/Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158265): 
https://lists.openembedded.org/g/openembedded-core/message/158265
Mute This Topic: https://lists.openembedded.org/mt/87042685/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [hardknott][PATCH] mklibs: remove recipes and class

[Jacob Kroon]

On 11/14/21 20:36, Richard Purdie wrote:

On Sun, 2021-11-14 at 07:50 +0100, Jacob Kroon wrote:

From: Alexander Kanavin 

This is not enabled or tested by default, and has never been
ported to python 3 upstream[1], which means it doesn't work at all
with plain poky. If you need it, please put it in a separate layer
and/or modernize to work with py3.

https://salsa.debian.org/installer-team/mklibs/-/blob/master/src/mklibs

Signed-off-by: Alexander Kanavin 
Signed-off-by: Jacob Kroon 
Signed-off-by: Richard Purdie 
(cherry picked from commit 908df863b419d1cad7317153101fc827e7e3a354)


What issue is it causing in hardknott? Normally we don't backport invasive
changes like that...



The problem is that mklibs-native doesn't build with gcc 11, and 
"image-mklibs" was in USER_CLASSES by default in hardknott.


I sent a patch to fix mklibs build here:

https://lists.openembedded.org/g/openembedded-core/message/151982

but after some discussions we just decided to drop mklibs support 
entirely with the proposed patch.


/Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#158263): 
https://lists.openembedded.org/g/openembedded-core/message/158263
Mute This Topic: https://lists.openembedded.org/mt/87042685/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [hardknott][PATCH] mklibs: remove recipes and class

[Jacob Kroon]

From: Alexander Kanavin 

This is not enabled or tested by default, and has never been
ported to python 3 upstream[1], which means it doesn't work at all
with plain poky. If you need it, please put it in a separate layer
and/or modernize to work with py3.

https://salsa.debian.org/installer-team/mklibs/-/blob/master/src/mklibs

Signed-off-by: Alexander Kanavin 
Signed-off-by: Jacob Kroon 
Signed-off-by: Richard Purdie 
(cherry picked from commit 908df863b419d1cad7317153101fc827e7e3a354)
---
 meta/classes/image-mklibs.bbclass |  56 --
 meta/conf/distro/include/maintainers.inc  |   1 -
 meta/conf/local.conf.sample   |   5 +-
 meta/conf/local.conf.sample.extended  |   9 --
 .../mklibs/files/ac_init_fix.patch|  19 
 ...re-on-symbol-provided-by-application.patch | 103 --
 .../mklibs/files/fix_STT_GNU_IFUNC.patch  |  26 -
 .../mklibs/files/fix_cross_compile.patch  |  81 --
 ...U-unique-symbols-as-provided-symbols.patch |  34 --
 .../mklibs/files/sysrooted-ldso.patch |  18 ---
 .../mklibs/mklibs-native_0.1.44.bb|  22 
 11 files changed, 1 insertion(+), 373 deletions(-)
 delete mode 100644 meta/classes/image-mklibs.bbclass
 delete mode 100644 meta/recipes-devtools/mklibs/files/ac_init_fix.patch
 delete mode 100644 
meta/recipes-devtools/mklibs/files/avoid-failure-on-symbol-provided-by-application.patch
 delete mode 100644 meta/recipes-devtools/mklibs/files/fix_STT_GNU_IFUNC.patch
 delete mode 100644 meta/recipes-devtools/mklibs/files/fix_cross_compile.patch
 delete mode 100644 
meta/recipes-devtools/mklibs/files/show-GNU-unique-symbols-as-provided-symbols.patch
 delete mode 100644 meta/recipes-devtools/mklibs/files/sysrooted-ldso.patch
 delete mode 100644 meta/recipes-devtools/mklibs/mklibs-native_0.1.44.bb

diff --git a/meta/classes/image-mklibs.bbclass 
b/meta/classes/image-mklibs.bbclass
deleted file mode 100644
index 68e11d4365..00
--- a/meta/classes/image-mklibs.bbclass
+++ /dev/null
@@ -1,56 +0,0 @@
-do_rootfs[depends] += "mklibs-native:do_populate_sysroot"
-
-IMAGE_PREPROCESS_COMMAND += "mklibs_optimize_image; "
-
-inherit linuxloader
-
-mklibs_optimize_image_doit() {
-   rm -rf ${WORKDIR}/mklibs
-   mkdir -p ${WORKDIR}/mklibs/dest
-   cd ${IMAGE_ROOTFS}
-   du -bs > ${WORKDIR}/mklibs/du.before.mklibs.txt
-
-   # Build a list of dynamically linked executable ELF files.
-   # Omit libc/libpthread as a special case because it has an interpreter
-   # but is primarily what we intend to strip down.
-   for i in `find . -type f -executable ! -name 'libc-*' ! -name 
'libpthread-*'`; do
-   file $i | grep -q ELF || continue
-   ${HOST_PREFIX}readelf -l $i | grep -q INTERP || continue
-   echo $i
-   done > ${WORKDIR}/mklibs/executables.list
-
-   dynamic_loader=${@get_linuxloader(d)}
-
-   mklibs -v \
-   --ldlib ${dynamic_loader} \
-   --libdir ${baselib} \
-   --sysroot ${PKG_CONFIG_SYSROOT_DIR} \
-   --gcc-options "--sysroot=${PKG_CONFIG_SYSROOT_DIR}" \
-   --root ${IMAGE_ROOTFS} \
-   --target `echo ${TARGET_PREFIX} | sed 's/-$//' ` \
-   -d ${WORKDIR}/mklibs/dest \
-   `cat ${WORKDIR}/mklibs/executables.list`
-
-   cd ${WORKDIR}/mklibs/dest
-   for i in *
-   do
-   cp $i `find ${IMAGE_ROOTFS} -name $i`
-   done
-
-   cd ${IMAGE_ROOTFS}
-   du -bs > ${WORKDIR}/mklibs/du.after.mklibs.txt
-
-   echo rootfs size before mklibs optimization: `cat 
${WORKDIR}/mklibs/du.before.mklibs.txt`
-   echo rootfs size after mklibs optimization: `cat 
${WORKDIR}/mklibs/du.after.mklibs.txt`
-}
-
-mklibs_optimize_image() {
-   for img in ${MKLIBS_OPTIMIZED_IMAGES}
-   do
-   if [ "${img}" = "${PN}" ] || [ "${img}" = "all" ]
-   then
-   mklibs_optimize_image_doit
-   break
-   fi
-   done
-}
diff --git a/meta/conf/distro/include/maintainers.inc 
b/meta/conf/distro/include/maintainers.inc
index 5d453a6fcd..543c6438f5 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -502,7 +502,6 @@ RECIPE_MAINTAINER_pn-mingetty = "Yi Zhao 
"
 RECIPE_MAINTAINER_pn-mini-x-session = "Armin Kuster "
 RECIPE_MAINTAINER_pn-minicom = "Anuj Mittal "
 RECIPE_MAINTAINER_pn-mkfontscale = "Armin Kuster "
-RECIPE_MAINTAINER_pn-mklibs-native = "Robert Yang "
 RECIPE_MAINTAINER_pn-mmc-utils = "Anuj Mittal "
 RECIPE_MAINTAINER_pn-mobile-broadband-provider-info = "Alexander Kanavin 
"
 RECIPE_MAINTAINER_pn-modutils-initscripts = "Yi Zhao "
diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sam

Re: [OE-core] [PATCH 1/4] gcc: Merge three related patches together

[Jacob Kroon]

On 10/26/21 13:11, Richard Purdie wrote:

The SYSTEMLIBS_DIR change was spread over three patches, merge these
together since there is no value in having them separate.

Signed-off-by: Richard Purdie 
---
  meta/recipes-devtools/gcc/gcc-11.2.inc|  2 -
  ...AMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch | 52 ++-
  ...IR-replacement-instead-of-hardcoding.patch | 26 --
  ...22-aarch64-Add-support-for-musl-ldso.patch | 25 -
  4 files changed, 28 insertions(+), 77 deletions(-)
  delete mode 100644 
meta/recipes-devtools/gcc/gcc/0021-Use-SYSTEMLIBS_DIR-replacement-instead-of-hardcoding.patch
  delete mode 100644 
meta/recipes-devtools/gcc/gcc/0022-aarch64-Add-support-for-musl-ldso.patch

diff --git a/meta/recipes-devtools/gcc/gcc-11.2.inc 
b/meta/recipes-devtools/gcc/gcc-11.2.inc
index 9fd30f52a88..6fa344e9612 100644
--- a/meta/recipes-devtools/gcc/gcc-11.2.inc
+++ b/meta/recipes-devtools/gcc/gcc-11.2.inc
@@ -50,8 +50,6 @@ SRC_URI = "\
 file://0018-export-CPP.patch \
 file://0019-Ensure-target-gcc-headers-can-be-included.patch \
 
file://0020-Don-t-search-host-directory-during-relink-if-inst_pr.patch \
-   
file://0021-Use-SYSTEMLIBS_DIR-replacement-instead-of-hardcoding.patch \
-   file://0022-aarch64-Add-support-for-musl-ldso.patch \
 file://0023-libcc1-fix-libcc1-s-install-path-and-rpath.patch \
 file://0024-handle-sysroot-support-for-nativesdk-gcc.patch \
 
file://0025-Search-target-sysroot-gcc-version-specific-dirs-with.patch \
diff --git 
a/meta/recipes-devtools/gcc/gcc/0011-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch
 
b/meta/recipes-devtools/gcc/gcc/0011-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch
index 4726267a80f..0884730786c 100644
--- 
a/meta/recipes-devtools/gcc/gcc/0011-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch
+++ 
b/meta/recipes-devtools/gcc/gcc/0011-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch
@@ -30,8 +30,7 @@ Upstream-Status: Inappropriate [OE configuration]
   gcc/config/sparc/linux64.h   |  4 ++--
   12 files changed, 29 insertions(+), 34 deletions(-)
  
-diff --git a/gcc/config/alpha/linux-elf.h b/gcc/config/alpha/linux-elf.h

-index c1dae8ca2cf..3ce2b76c1a4 100644
+unchanged:
  --- a/gcc/config/alpha/linux-elf.h
  +++ b/gcc/config/alpha/linux-elf.h
  @@ -23,8 +23,8 @@ along with GCC; see the file COPYING3.  If not see
@@ -45,8 +44,7 @@ index c1dae8ca2cf..3ce2b76c1a4 100644
   #if DEFAULT_LIBC == LIBC_UCLIBC
   #define CHOOSE_DYNAMIC_LINKER(G, U) "%{mglibc:" G ";:" U "}"
   #elif DEFAULT_LIBC == LIBC_GLIBC
-diff --git a/gcc/config/arm/linux-eabi.h b/gcc/config/arm/linux-eabi.h
-index 85d0136e76e..6bd95855827 100644
+unchanged:


Doesn't the above change how the patch is applied ?

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#157474): 
https://lists.openembedded.org/g/openembedded-core/message/157474
Mute This Topic: https://lists.openembedded.org/mt/86599747/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 2/2] gawk: reduce strictness of the time test

[Jacob Kroon]

On 10/19/21 13:35, Ross Burton wrote:

The time.awk test does a sleep() and verifies that the actual delay is
close to the requested time. However on a loaded system the range of
acceptable durations is quite tight and will occasionally fail.

Solve this by increasing the range of acceptable delays slightly to
between 50% and 200% of the requested delay.

[ YOCTO #14371 ]



I would have removed the test. These load-sensitive tests are gonna fail 
occasionally, whatever limit you put there.


Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#157112): 
https://lists.openembedded.org/g/openembedded-core/message/157112
Mute This Topic: https://lists.openembedded.org/mt/86437428/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] linux-yocto: Set CONFIG_ATA_PIIX=y by default

[Jacob Kroon]

On 10/19/21 04:40, Bruce Ashfield wrote:

On Mon, Oct 18, 2021 at 2:39 AM Jacob Kroon  wrote:


On 10/17/21 15:32, Bruce Ashfield wrote:

On Sun, Oct 17, 2021 at 4:26 AM Jacob Kroon  wrote:


Hi Bruce and Saul,

On 10/16/21 09:18, Jacob Kroon via lists.openembedded.org wrote:

Hi Bruce,

My Yocto images (which uses the linux-yocto kernel) stopped booting in
QEMU some time ago, and after some debugging it turns out this was
because the upstream Linux kernel removed the legacy IDE driver. Instead
one should use the libata driver. However, I don't think the linux-yocto
kernel has builtin support for the HW that is emulated by QEMU by
default (PIIX), instead it is built as a module, CONFIG_ATA_PIIX=m. If I
set CONFIG_ATA_PIIX=y, my images boot again.

I did a "make ARCH=i386 defconfig" in Torvalds master linux tree, and
the .config has CONFIG_ATA_PIIX=y.

Do you think it would make sense to have the support builtin in
linux-yocto aswell ?



I'm using KMACHINE = "common-pc". CC:ing Saul Wold, since I see that
commit 0d4f5ed5dca41a48423ce738131e52f7863d8ca6 in yocto-kernel-cache did:


diff --git a/bsp/common-pc/common-pc-drivers.cfg 
b/bsp/common-pc/common-pc-drivers.cfg
index 71608433..0b821903 100644
--- a/bsp/common-pc/common-pc-drivers.cfg
+++ b/bsp/common-pc/common-pc-drivers.cfg
@@ -5,7 +5,8 @@ CONFIG_PCI_MSI=y
   CONFIG_ATA=y
   CONFIG_ATA_ACPI=y
   CONFIG_ATA_SFF=y
-CONFIG_ATA_PIIX=y
+CONFIG_ATA_BMDMA=y
+CONFIG_ATA_PIIX=m

   CONFIG_INPUT=y
   CONFIG_INPUT_MOUSEDEV=y


which changed ATA_PIIX from a builtin to a module. Maybe this wasn't
intentional ?


It was definitely intentional.
  > We try and keep the configuration space of the targets as small as
possible. In particular, this is exactly why qemux86* was created, so
we wouldn't have to enable options in either the h/w targets or the
emulated targets that are specific to a use case on one (and vice
versa). That being said, they still actually do share the same machine
definitions in the kernel-cache, since nothing significant has forced
me to use that namespace split.

qemux86 doesn't need PIIX out of the box to boot, and the Kconfig
indicates "set it to N unless ...", and we do know the built-ins we
want, so it is set to 'm' as a middle ground.



When I start qemu-system-x86_64/qemu-system-i386 and type "info qtree"
in the QEMU monitor (both version 5.2.0 and 6.1.0) I see "piix3-ide" for
the IDE controller. Given that for older Yocto kernels the legacy IDE
driver was builtin (CONFIG_BLK_DEV_PIIX=y), but is now removed in the
kernel present on master, I'd figure ATA_PIIX=y would be required
nowadays in order to get the kernels to boot in QEMU (since libata is
replacing the old legacy IDE driver nowadays).


It all depends on how the boot media is wired into the qemu run.
The runqemu boots are using vda and virtio-blk to start up, with
an explicit kernel and image passed to qemu.

ATA_PIIX is loaded as a module after that, and is active (but not
used) .. and of course it is loaded since the piix is detected and
triggers the load.

Obviously we've been booting qemux86* all throughout the
development cycle :D Even after I cleaned up all the old
obsolete IDE options.




That being said, it isn't out of the question to enable it (slippery
slope to giant defconfig like beasts with everything and the kitchen
sink enabled notwithstanding .. not that this is much of a step down
that slope!) .. just let me ask a few more things first.



Yeah, the only reason I'm asking is because I think I'm using default
QEMU emulated hardware, and I'm sure we want a Yocto kernel to boot in
default QEMU hardware.


It's less about the h/w and more about what we want to enable
as boot media, and the boot process. If we were going through
an initramfs, etc, we could load the modules from rootfs and
have everything available as potential boot options.




You say you are using KMACHINE='common-pc', that's good. But what else
is at play ? What OE MACHINE are you building ? What image FStypes,
etc ?



It is a custom machine for

https://www.compulab.com/products/computer-on-modules/cm-itc/

machine/cm-itc.conf:
---
IMAGE_FSTYPES ?= "ext4"

require conf/machine/include/x86/tune-i686.inc
require conf/machine/include/x86/x86-base.inc

PREFERRED_PROVIDER_virtual/bootloader = "grub-efi"
---


I won't be able to do some build testing until Monday, but do you
happen to have the qemu command lines (via runqemu) and machine
definitions for qemux88 and your machine?  I'd like to look at them
and confirm exactly what image, and boot parameters are in play, since
one boots and the other doesn't.



I don't use the "runqemu"-script, I boot the images using Fedora 34 QEMU
installation (QEMU version 5.2.0). Commandline is:


qemu-system-x86_64 -m 1G -enable-kvm -nic user,hostfwd=tcp::10022-:22 -cpu n270 -bios 
/usr/share/edk2/ovmf-ia32/OVMF_CODE.fd -drive format=raw,file=




I obviously wasn't abl

Re: [OE-core] Little sstate reuse when using uninative and yocto sstate cache

[Jacob Kroon]

On 10/18/21 12:15, Richard Purdie wrote:

On Mon, 2021-10-18 at 09:16 +0200, Jacob Kroon wrote:

I'm on latest dunfell and using uninative and have put the yocto servers
in my SSTATE_MIRRORS:

my-distro.conf:
...

SSTATE_MIRRORS ?= "file://.* 
http://sstate.yoctoproject.org/3.1.11/PATH;downloadfilename=PATH;
require conf/distro/include/yocto-uninative.inc
INHERIT += "uninative"

...

but I thought I'd be getting more sstate reuse than I am:


bitbake m4-native

...

Sstate summary: Wanted 12 Found 1 Missed 11 Current 0 (8% match, 0% complete)


Is the above reasonable, or should I be able to pull m4-native directly
from yocto servers sstate ? If so, is there a way to debug what differs
between my local siginfos and the ones on the sstate server ?



In dunfell hash equivalence is enabled and this unfortunately means the sstate
doesn't work very well unless you also have the matching hash equivalence data.

We do now have a way to have a readonly hash equivalence server and we have made
one available from the autobuilder so this data is shared. Unfortunately there
isn't a way to configure dunfell to use one though :(. That patch is a feature
but could be something we consider backporting.

That said, I did a lot of testing/work with master recently as sstate reuse with
hash equivalence was not what we'd expected. We ended up making a lot of
changes, including a bit of a rewrite of the server side of the hash
equivalence. I mention this just to illustrate it isn't a simple problem and
that fixing this in dunfell may be tricky and would need backporting of some
invasive changes. I'm trying to at least get master/honister working right in
the first instance.

Cheers,

Richard




Thanks for the detailed answer. I will focus on getting this to perform 
well with my master build then.


Regards
Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#157054): 
https://lists.openembedded.org/g/openembedded-core/message/157054
Mute This Topic: https://lists.openembedded.org/mt/86408232/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] Little sstate reuse when using uninative and yocto sstate cache

[Jacob Kroon]

Hi,

I'm on latest dunfell and using uninative and have put the yocto servers 
in my SSTATE_MIRRORS:


my-distro.conf:
...

SSTATE_MIRRORS ?= "file://.* 
http://sstate.yoctoproject.org/3.1.11/PATH;downloadfilename=PATH;
require conf/distro/include/yocto-uninative.inc
INHERIT += "uninative"

...

but I thought I'd be getting more sstate reuse than I am:


bitbake m4-native

...

Sstate summary: Wanted 12 Found 1 Missed 11 Current 0 (8% match, 0% complete)


Is the above reasonable, or should I be able to pull m4-native directly 
from yocto servers sstate ? If so, is there a way to debug what differs 
between my local siginfos and the ones on the sstate server ?


Regards Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#157050): 
https://lists.openembedded.org/g/openembedded-core/message/157050
Mute This Topic: https://lists.openembedded.org/mt/86408232/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] linux-yocto: Set CONFIG_ATA_PIIX=y by default

[Jacob Kroon]

On 10/17/21 15:32, Bruce Ashfield wrote:

On Sun, Oct 17, 2021 at 4:26 AM Jacob Kroon  wrote:


Hi Bruce and Saul,

On 10/16/21 09:18, Jacob Kroon via lists.openembedded.org wrote:

Hi Bruce,

My Yocto images (which uses the linux-yocto kernel) stopped booting in
QEMU some time ago, and after some debugging it turns out this was
because the upstream Linux kernel removed the legacy IDE driver. Instead
one should use the libata driver. However, I don't think the linux-yocto
kernel has builtin support for the HW that is emulated by QEMU by
default (PIIX), instead it is built as a module, CONFIG_ATA_PIIX=m. If I
set CONFIG_ATA_PIIX=y, my images boot again.

I did a "make ARCH=i386 defconfig" in Torvalds master linux tree, and
the .config has CONFIG_ATA_PIIX=y.

Do you think it would make sense to have the support builtin in
linux-yocto aswell ?



I'm using KMACHINE = "common-pc". CC:ing Saul Wold, since I see that
commit 0d4f5ed5dca41a48423ce738131e52f7863d8ca6 in yocto-kernel-cache did:


diff --git a/bsp/common-pc/common-pc-drivers.cfg 
b/bsp/common-pc/common-pc-drivers.cfg
index 71608433..0b821903 100644
--- a/bsp/common-pc/common-pc-drivers.cfg
+++ b/bsp/common-pc/common-pc-drivers.cfg
@@ -5,7 +5,8 @@ CONFIG_PCI_MSI=y
  CONFIG_ATA=y
  CONFIG_ATA_ACPI=y
  CONFIG_ATA_SFF=y
-CONFIG_ATA_PIIX=y
+CONFIG_ATA_BMDMA=y
+CONFIG_ATA_PIIX=m

  CONFIG_INPUT=y
  CONFIG_INPUT_MOUSEDEV=y


which changed ATA_PIIX from a builtin to a module. Maybe this wasn't
intentional ?


It was definitely intentional.
 > We try and keep the configuration space of the targets as small as
possible. In particular, this is exactly why qemux86* was created, so
we wouldn't have to enable options in either the h/w targets or the
emulated targets that are specific to a use case on one (and vice
versa). That being said, they still actually do share the same machine
definitions in the kernel-cache, since nothing significant has forced
me to use that namespace split.

qemux86 doesn't need PIIX out of the box to boot, and the Kconfig
indicates "set it to N unless ...", and we do know the built-ins we
want, so it is set to 'm' as a middle ground.



When I start qemu-system-x86_64/qemu-system-i386 and type "info qtree" 
in the QEMU monitor (both version 5.2.0 and 6.1.0) I see "piix3-ide" for 
the IDE controller. Given that for older Yocto kernels the legacy IDE 
driver was builtin (CONFIG_BLK_DEV_PIIX=y), but is now removed in the 
kernel present on master, I'd figure ATA_PIIX=y would be required 
nowadays in order to get the kernels to boot in QEMU (since libata is 
replacing the old legacy IDE driver nowadays).



That being said, it isn't out of the question to enable it (slippery
slope to giant defconfig like beasts with everything and the kitchen
sink enabled notwithstanding .. not that this is much of a step down
that slope!) .. just let me ask a few more things first.



Yeah, the only reason I'm asking is because I think I'm using default 
QEMU emulated hardware, and I'm sure we want a Yocto kernel to boot in 
default QEMU hardware.



You say you are using KMACHINE='common-pc', that's good. But what else
is at play ? What OE MACHINE are you building ? What image FStypes,
etc ?



It is a custom machine for

https://www.compulab.com/products/computer-on-modules/cm-itc/

machine/cm-itc.conf:
---
IMAGE_FSTYPES ?= "ext4"

require conf/machine/include/x86/tune-i686.inc
require conf/machine/include/x86/x86-base.inc

PREFERRED_PROVIDER_virtual/bootloader = "grub-efi"
---


I won't be able to do some build testing until Monday, but do you
happen to have the qemu command lines (via runqemu) and machine
definitions for qemux88 and your machine?  I'd like to look at them
and confirm exactly what image, and boot parameters are in play, since
one boots and the other doesn't.



I don't use the "runqemu"-script, I boot the images using Fedora 34 QEMU 
installation (QEMU version 5.2.0). Commandline is:



qemu-system-x86_64 -m 1G -enable-kvm -nic user,hostfwd=tcp::10022-:22 -cpu n270 -bios 
/usr/share/edk2/ovmf-ia32/OVMF_CODE.fd -drive format=raw,file=


I have the following bbappended in my yocto kernel recipe:
KMACHINE:cm-itc ?= "common-pc"
COMPATIBLE_MACHINE:cm-itc = "cm-itc"

together with some custom kernel configuration.

Regards Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#157049): 
https://lists.openembedded.org/g/openembedded-core/message/157049
Mute This Topic: https://lists.openembedded.org/mt/86367293/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] linux-yocto: Set CONFIG_ATA_PIIX=y by default

[Jacob Kroon]

Hi Bruce and Saul,

On 10/16/21 09:18, Jacob Kroon via lists.openembedded.org wrote:

Hi Bruce,

My Yocto images (which uses the linux-yocto kernel) stopped booting in 
QEMU some time ago, and after some debugging it turns out this was 
because the upstream Linux kernel removed the legacy IDE driver. Instead 
one should use the libata driver. However, I don't think the linux-yocto 
kernel has builtin support for the HW that is emulated by QEMU by 
default (PIIX), instead it is built as a module, CONFIG_ATA_PIIX=m. If I 
set CONFIG_ATA_PIIX=y, my images boot again.


I did a "make ARCH=i386 defconfig" in Torvalds master linux tree, and 
the .config has CONFIG_ATA_PIIX=y.


Do you think it would make sense to have the support builtin in 
linux-yocto aswell ?




I'm using KMACHINE = "common-pc". CC:ing Saul Wold, since I see that 
commit 0d4f5ed5dca41a48423ce738131e52f7863d8ca6 in yocto-kernel-cache did:



diff --git a/bsp/common-pc/common-pc-drivers.cfg 
b/bsp/common-pc/common-pc-drivers.cfg
index 71608433..0b821903 100644
--- a/bsp/common-pc/common-pc-drivers.cfg
+++ b/bsp/common-pc/common-pc-drivers.cfg
@@ -5,7 +5,8 @@ CONFIG_PCI_MSI=y
 CONFIG_ATA=y
 CONFIG_ATA_ACPI=y
 CONFIG_ATA_SFF=y
-CONFIG_ATA_PIIX=y
+CONFIG_ATA_BMDMA=y
+CONFIG_ATA_PIIX=m
 
 CONFIG_INPUT=y

 CONFIG_INPUT_MOUSEDEV=y


which changed ATA_PIIX from a builtin to a module. Maybe this wasn't 
intentional ?


Regards Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#157031): 
https://lists.openembedded.org/g/openembedded-core/message/157031
Mute This Topic: https://lists.openembedded.org/mt/86367293/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] linux-yocto: Set CONFIG_ATA_PIIX=y by default

[Jacob Kroon]

Hi Bruce,

My Yocto images (which uses the linux-yocto kernel) stopped booting in 
QEMU some time ago, and after some debugging it turns out this was 
because the upstream Linux kernel removed the legacy IDE driver. Instead 
one should use the libata driver. However, I don't think the linux-yocto 
kernel has builtin support for the HW that is emulated by QEMU by 
default (PIIX), instead it is built as a module, CONFIG_ATA_PIIX=m. If I 
set CONFIG_ATA_PIIX=y, my images boot again.


I did a "make ARCH=i386 defconfig" in Torvalds master linux tree, and 
the .config has CONFIG_ATA_PIIX=y.


Do you think it would make sense to have the support builtin in 
linux-yocto aswell ?


Regards Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#157011): 
https://lists.openembedded.org/g/openembedded-core/message/157011
Mute This Topic: https://lists.openembedded.org/mt/86367293/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [oe] Fedora 34, shadow-native/icedtea7-native, umask problems ?

[Jacob Kroon]

On 6/2/21 6:01 PM, Jacob Kroon via lists.openembedded.org wrote:

On 6/2/21 8:32 AM, Jacob Kroon via lists.openembedded.org wrote:

Hi,

I'm using Fedora 34 and OE-Core/Bitbake/layers from git master as of 
today, with reproducible builds. Sometimes when rebuilding my image I 
see sudden changes in file permissions (jumping back and forth) in the 
buildhistory output for two native recipes:


shadow-native: (OE-Core)

-drwxr-xr-x -  -  40 ./var/spool/mail
+drwxrwxr-x -  -  40 ./var/spool/mail




I can reproduce it reliably with these commands:

After I do:
# bitbake -c cleansstate shadow-native && bitbake shadow-native
I get:
drwxrwxr-x - - 40 ./var/spool/mail

After I do:
# bitbake -c clean shadow-native && bitbake shadow-native
I get:
drwxr-xr-x - - 40 ./var/spool/mail

shadow.inc does:


do_install_append() {
     ...
     install -m 0775 -d ${D}${localstatedir}/spool/mail


So it looks to me like those permissions are either not recorded in the 
sstate cache, or they are not restored when extracting the cache.




I see this issue in master and hardknot builds, but not in dunfell builds.
/Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#152809): 
https://lists.openembedded.org/g/openembedded-core/message/152809
Mute This Topic: https://lists.openembedded.org/mt/83262243/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 05/10] perl: split perl-cross into its own recipe

[Jacob Kroon]

On 6/4/21 11:14 AM, Alexander Kanavin wrote:

As perl and perl-cross need to be updated (and patches rebased)
in lockstep, devtool upgrade (and therefore AUH) can't cope with it.
Manually updating is still possible, but painful.

Split determinism.patch into perl and perl-cross parts, move the
rest of the perl-cross patches.



Not sure if this is intentional, but when I build master now I get a new 
dependency pulled in that is built called "perlcross-native". Given the 
sound of that name, is that intentional ?


Regards,
Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#152753): 
https://lists.openembedded.org/g/openembedded-core/message/152753
Mute This Topic: https://lists.openembedded.org/mt/83304705/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [oe] Fedora 34, shadow-native/icedtea7-native, umask problems ?

[Jacob Kroon]

On 6/2/21 8:32 AM, Jacob Kroon via lists.openembedded.org wrote:

Hi,

I'm using Fedora 34 and OE-Core/Bitbake/layers from git master as of 
today, with reproducible builds. Sometimes when rebuilding my image I 
see sudden changes in file permissions (jumping back and forth) in the 
buildhistory output for two native recipes:


shadow-native: (OE-Core)

-drwxr-xr-x -  -  40 ./var/spool/mail
+drwxrwxr-x -  -  40 ./var/spool/mail




I can reproduce it reliably with these commands:

After I do:
# bitbake -c cleansstate shadow-native && bitbake shadow-native
I get:
drwxrwxr-x - - 40 ./var/spool/mail

After I do:
# bitbake -c clean shadow-native && bitbake shadow-native
I get:
drwxr-xr-x - - 40 ./var/spool/mail

shadow.inc does:


do_install_append() {
...
install -m 0775 -d ${D}${localstatedir}/spool/mail


So it looks to me like those permissions are either not recorded in the 
sstate cache, or they are not restored when extracting the cache.


/Jacob


icedtea7-native: (meta-java)
-drwxrwxr-x -  - 120 
./usr/lib/jvm/icedtea7-native
-drwxrwxr-x -  - 800 
./usr/lib/jvm/icedtea7-native/bin
--rwxrwxr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/apt
--rwxrwxr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/extcheck
--rwxrwxr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/idlj

...
+drwxr-xr-x -  - 120 
./usr/lib/jvm/icedtea7-native
+drwxr-xr-x -  - 800 
./usr/lib/jvm/icedtea7-native/bin
+-rwxr-xr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/apt
+-rwxr-xr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/extcheck
+-rwxr-xr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/idlj

...

Before I start debugging them, does anyone have an obvious fix or hint 
for any of these ?


Regards,
Jacob





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#152548): 
https://lists.openembedded.org/g/openembedded-core/message/152548
Mute This Topic: https://lists.openembedded.org/mt/83262243/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] Fedora 34, shadow-native/icedtea7-native, umask problems ?

[Jacob Kroon]

Hi,

I'm using Fedora 34 and OE-Core/Bitbake/layers from git master as of 
today, with reproducible builds. Sometimes when rebuilding my image I 
see sudden changes in file permissions (jumping back and forth) in the 
buildhistory output for two native recipes:


shadow-native: (OE-Core)

-drwxr-xr-x -  -  40 ./var/spool/mail
+drwxrwxr-x -  -  40 ./var/spool/mail


icedtea7-native: (meta-java)

-drwxrwxr-x -  - 120 ./usr/lib/jvm/icedtea7-native
-drwxrwxr-x -  - 800 ./usr/lib/jvm/icedtea7-native/bin
--rwxrwxr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/apt
--rwxrwxr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/extcheck
--rwxrwxr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/idlj

...

+drwxr-xr-x -  - 120 ./usr/lib/jvm/icedtea7-native
+drwxr-xr-x -  - 800 ./usr/lib/jvm/icedtea7-native/bin
+-rwxr-xr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/apt
+-rwxr-xr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/extcheck
+-rwxr-xr-x -  -   15016 
./usr/lib/jvm/icedtea7-native/bin/idlj

...

Before I start debugging them, does anyone have an obvious fix or hint 
for any of these ?


Regards,
Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#152533): 
https://lists.openembedded.org/g/openembedded-core/message/152533
Mute This Topic: https://lists.openembedded.org/mt/83252996/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] mklibs-native: Fix build with gcc 11

[Jacob Kroon]

On 5/18/21 12:20 AM, Khem Raj wrote:



On 5/17/21 10:01 AM, Jacob Kroon wrote:
In gcc 11 the default mode for C++ is now -std=gnu++17 instead of 
-std=gnu++14,

in which support for dynamic exception specifications has been removed.

See http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0003r5.html

Signed-off-by: Jacob Kroon 
---
  .../no-dynamic-exception-specifications.patch | 420 ++
  .../mklibs/mklibs-native_0.1.44.bb    |   1 +
  2 files changed, 421 insertions(+)
  create mode 100644 
meta/recipes-devtools/mklibs/files/no-dynamic-exception-specifications.patch 



diff --git 
a/meta/recipes-devtools/mklibs/files/no-dynamic-exception-specifications.patch 
b/meta/recipes-devtools/mklibs/files/no-dynamic-exception-specifications.patch 


new file mode 100644
index 00..5989a67c4f
--- /dev/null
+++ 
b/meta/recipes-devtools/mklibs/files/no-dynamic-exception-specifications.patch 


@@ -0,0 +1,420 @@
+In gcc 11 the default mode for C++ is now -std=gnu++17 instead of 
-std=gnu++14,

+in which support for dynamic exception specifications has been removed.
+
+See http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0003r5.html
+
+Upstream-Status: Pending
+


the patch could be submitted upstream. Please try to do so



I tried to find a mailing list or maintainer I could post the patch to, 
but I'm not able to. Does anyone know who I should contact ?


Given that it sounds more likely that we'll remove mklibs, I'll try to 
forward the patch to the project, but not promising anything.


/Jacob

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#152022): 
https://lists.openembedded.org/g/openembedded-core/message/152022
Mute This Topic: https://lists.openembedded.org/mt/82891868/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2] mklibs: remove recipes and class

[Jacob Kroon]

From: Alexander Kanavin 

This is not enabled or tested by default, and has never been
ported to python 3 upstream[1], which means it doesn't work at all
with plain poky. If you need it, please put it in a separate layer
and/or modernize to work with py3.

https://salsa.debian.org/installer-team/mklibs/-/blob/master/src/mklibs

Signed-off-by: Alexander Kanavin 
Signed-off-by: Jacob Kroon 
---

Changes in v2:
  * Removed references to mklibs in
  meta/conf/local.conf.sample
  meta/conf/local.conf.sample.extended

 meta/classes/image-mklibs.bbclass |  56 --
 meta/conf/distro/include/maintainers.inc  |   1 -
 meta/conf/local.conf.sample   |   5 +-
 meta/conf/local.conf.sample.extended  |   9 --
 .../mklibs/files/ac_init_fix.patch|  19 
 ...re-on-symbol-provided-by-application.patch | 103 --
 .../mklibs/files/fix_STT_GNU_IFUNC.patch  |  26 -
 .../mklibs/files/fix_cross_compile.patch  |  81 --
 ...U-unique-symbols-as-provided-symbols.patch |  34 --
 .../mklibs/files/sysrooted-ldso.patch |  18 ---
 .../mklibs/mklibs-native_0.1.44.bb|  22 
 11 files changed, 1 insertion(+), 373 deletions(-)
 delete mode 100644 meta/classes/image-mklibs.bbclass
 delete mode 100644 meta/recipes-devtools/mklibs/files/ac_init_fix.patch
 delete mode 100644 
meta/recipes-devtools/mklibs/files/avoid-failure-on-symbol-provided-by-application.patch
 delete mode 100644 meta/recipes-devtools/mklibs/files/fix_STT_GNU_IFUNC.patch
 delete mode 100644 meta/recipes-devtools/mklibs/files/fix_cross_compile.patch
 delete mode 100644 
meta/recipes-devtools/mklibs/files/show-GNU-unique-symbols-as-provided-symbols.patch
 delete mode 100644 meta/recipes-devtools/mklibs/files/sysrooted-ldso.patch
 delete mode 100644 meta/recipes-devtools/mklibs/mklibs-native_0.1.44.bb

diff --git a/meta/classes/image-mklibs.bbclass 
b/meta/classes/image-mklibs.bbclass
deleted file mode 100644
index 68e11d4365..00
--- a/meta/classes/image-mklibs.bbclass
+++ /dev/null
@@ -1,56 +0,0 @@
-do_rootfs[depends] += "mklibs-native:do_populate_sysroot"
-
-IMAGE_PREPROCESS_COMMAND += "mklibs_optimize_image; "
-
-inherit linuxloader
-
-mklibs_optimize_image_doit() {
-   rm -rf ${WORKDIR}/mklibs
-   mkdir -p ${WORKDIR}/mklibs/dest
-   cd ${IMAGE_ROOTFS}
-   du -bs > ${WORKDIR}/mklibs/du.before.mklibs.txt
-
-   # Build a list of dynamically linked executable ELF files.
-   # Omit libc/libpthread as a special case because it has an interpreter
-   # but is primarily what we intend to strip down.
-   for i in `find . -type f -executable ! -name 'libc-*' ! -name 
'libpthread-*'`; do
-   file $i | grep -q ELF || continue
-   ${HOST_PREFIX}readelf -l $i | grep -q INTERP || continue
-   echo $i
-   done > ${WORKDIR}/mklibs/executables.list
-
-   dynamic_loader=${@get_linuxloader(d)}
-
-   mklibs -v \
-   --ldlib ${dynamic_loader} \
-   --libdir ${baselib} \
-   --sysroot ${PKG_CONFIG_SYSROOT_DIR} \
-   --gcc-options "--sysroot=${PKG_CONFIG_SYSROOT_DIR}" \
-   --root ${IMAGE_ROOTFS} \
-   --target `echo ${TARGET_PREFIX} | sed 's/-$//' ` \
-   -d ${WORKDIR}/mklibs/dest \
-   `cat ${WORKDIR}/mklibs/executables.list`
-
-   cd ${WORKDIR}/mklibs/dest
-   for i in *
-   do
-   cp $i `find ${IMAGE_ROOTFS} -name $i`
-   done
-
-   cd ${IMAGE_ROOTFS}
-   du -bs > ${WORKDIR}/mklibs/du.after.mklibs.txt
-
-   echo rootfs size before mklibs optimization: `cat 
${WORKDIR}/mklibs/du.before.mklibs.txt`
-   echo rootfs size after mklibs optimization: `cat 
${WORKDIR}/mklibs/du.after.mklibs.txt`
-}
-
-mklibs_optimize_image() {
-   for img in ${MKLIBS_OPTIMIZED_IMAGES}
-   do
-   if [ "${img}" = "${PN}" ] || [ "${img}" = "all" ]
-   then
-   mklibs_optimize_image_doit
-   break
-   fi
-   done
-}
diff --git a/meta/conf/distro/include/maintainers.inc 
b/meta/conf/distro/include/maintainers.inc
index bfebd68acf..1467a10f8a 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -508,7 +508,6 @@ RECIPE_MAINTAINER_pn-mingetty = "Yi Zhao 
"
 RECIPE_MAINTAINER_pn-mini-x-session = "Armin Kuster "
 RECIPE_MAINTAINER_pn-minicom = "Anuj Mittal "
 RECIPE_MAINTAINER_pn-mkfontscale = "Armin Kuster "
-RECIPE_MAINTAINER_pn-mklibs-native = "Robert Yang "
 RECIPE_MAINTAINER_pn-mmc-utils = "Anuj Mittal "
 RECIPE_MAINTAINER_pn-mobile-broadband-provider-info = "Alexander Kanavin 
"
 RECIPE_MAINTAINER_pn-modutils-initscripts = "Yi Zhao "
diff --git a/meta/conf/local.conf