Re: [OE-core] linux-yocto: Enable goldfish RTC for qemuriscv

[Konrad Weihmann]

Hi all,

I disagree with the proposed change, as this is unconditional, even 
though the commit message claims otherwise.
It would be better to have it limited to qemuriscv as the commit message 
proposes.


Fun fact: I can't find this message on the web interface of 
https://lists.openembedded.org/g/openembedded-core


Cheers
Konrad

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195789): 
https://lists.openembedded.org/g/openembedded-core/message/195789
Mute This Topic: https://lists.openembedded.org/mt/104397061/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] python3-yamllint: add missing dependency

[Konrad Weihmann]

yamllint requires pathspec module to be available

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-devtools/python/python3-yamllint_1.33.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3-yamllint_1.33.0.bb 
b/meta/recipes-devtools/python/python3-yamllint_1.33.0.bb
index 4b7bd065f9..7e51fc9395 100644
--- a/meta/recipes-devtools/python/python3-yamllint_1.33.0.bb
+++ b/meta/recipes-devtools/python/python3-yamllint_1.33.0.bb
@@ -10,6 +10,6 @@ PYPI_PACKAGE = "yamllint"
 SRC_URI[sha256sum] = 
"2dceab9ef2d99518a2fcf4ffc964d44250ac4459be1ba3ca315118e4a1a81f7d"
 
 DEPENDS += "${PYTHON_PN}-setuptools-scm-native"
-RDEPENDS:${PN} += "${PYTHON_PN}-pyyaml"
+RDEPENDS:${PN} += "${PYTHON_PN}-pathspec ${PYTHON_PN}-pyyaml"
 
 BBCLASSEXTEND = "native nativesdk"
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#194438): 
https://lists.openembedded.org/g/openembedded-core/message/194438
Mute This Topic: https://lists.openembedded.org/mt/104009503/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] recipetool: limit added checksums

[Konrad Weihmann]

to the same values that are used in bb.fetch.
Currently that would only add sha256sum to the created recipe,
which is the preferred choice over md5

Signed-off-by: Konrad Weihmann 
---
 scripts/lib/recipetool/create.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 293198d1c8..0016be3e7b 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -677,7 +677,10 @@ def create_recipe(args):
 if not srcuri:
 lines_before.append('# No information for SRC_URI yet (only an 
external source tree was specified)')
 lines_before.append('SRC_URI = "%s"' % srcuri)
+checksums_shown_list = ["%ssum" % x for x in bb.fetch2.SHOWN_CHECKSUM_LIST]
 for key, value in sorted(checksums.items()):
+if key not in checksums_shown_list:
+continue
 lines_before.append('SRC_URI[%s] = "%s"' % (key, value))
 if srcuri and supports_srcrev(srcuri):
 lines_before.append('')
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191708): 
https://lists.openembedded.org/g/openembedded-core/message/191708
Mute This Topic: https://lists.openembedded.org/mt/102949370/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] python3-requests broken after urlib3 update

[Konrad Weihmann]

FYI: 
https://git.yoctoproject.org/poky/commit/meta/recipes-devtools/python?h=master-next=4641605c794db6648311b931fb3a36fe9e5dea56 
breaks building python3-requests (as of version 2.28.2 in master).


> recipe-sysroot-native/usr/bin/python3-native/python3', '-m', 'pip', 
'--disable-pip-version-check', 'wheel', '--no-deps', '-w', 
'/tmp/tmptsmcl2c1', '--quiet', 'urllib3<1.27,>=1.21.1' --> exit code 1


Tried with native variants.

An update to python3-requests 2.30.0 should fix the issue.

But I'm not sure how that could have passed CI - so someone with more 
knowledge than me of the autobuilder might want to have a look.


Regards
Konrad

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181048): 
https://lists.openembedded.org/g/openembedded-core/message/181048
Mute This Topic: https://lists.openembedded.org/mt/98787186/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] lib/oeqa: add __init__.py

[Konrad Weihmann]

to allow loading of oeqa modules outside of bitbake.
Calls like from oeqa.runtime.case import OERuntimeTestCase or
from oeqa.runtime.case import OERuntimeTestCase assume that oeqa
is a python module, but a the folder is lacking an __init__.py imports
fail when the file is loaded outside of bitbake with manipulated
sys.path.
Fix that by adding an empty file.
This allow to inspect for instance test case and generate proper
test documentation from docstring or even other use cases.

The file got deleted to fix testimage and testexport features in
d9b3ee8b38f877c1723e48f1aa311c28102786fb, but it should have
been not deleted but just emptied

Signed-off-by: Konrad Weihmann 
---
 meta/lib/oeqa/__init__.py | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 meta/lib/oeqa/__init__.py

diff --git a/meta/lib/oeqa/__init__.py b/meta/lib/oeqa/__init__.py
new file mode 100644
index 00..e69de29bb2
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178832): 
https://lists.openembedded.org/g/openembedded-core/message/178832
Mute This Topic: https://lists.openembedded.org/mt/97734360/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 0/1] create-spdx: default share_src for shared sources

[Konrad WEIHMANN via lists.openembedded.org]

as it's practically impossible to send an email patch from corporate
environment, please find the following patch at the mentioned
remote location.

It would be nice to have that backported to kirkstone as well, once merged, as 
this is
where I encountered the issue first

The following changes since commit 482c493cf681121fb041f98066a42af14145ff9e:

  valgrind: update to 3.20.0 (2022-11-03 07:46:47 +)

are available in the Git repository at:

  https://github.com/witekio-kwe/poky spdx-fix
  https://github.com/witekio-kwe/poky/tree/spdx-fix

Konrad Weihmann (1):
  create-spdx: default share_src for shared sources

 meta/classes/create-spdx.bbclass | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)


Witekio GmbH is registered in Frankfurt. Registered office: Am Wartfeld 7 - 
61169 Friedberg. Registered company number: HRB 95432. VAT number DE287400071
This message contains confidential information and is intended only for the 
individual(s) addressed in the message. If you aren't the named addressee, you 
should not disseminate, distribute, or copy this e-mail.

We continuously commit to comply with the applicable data protection laws and 
ensure fair and transparent processing of your personal data. Please read our 
privacy statement including an information notice and data protection policy 
for detailed information on our website.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172630): 
https://lists.openembedded.org/g/openembedded-core/message/172630
Mute This Topic: https://lists.openembedded.org/mt/94757133/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] python3-idna: migrate to flit-core

[Konrad Weihmann]

setup.py in latest release is broken after move to flit-core
was released by the project.
This broke the version detection in consuming libs like requests.
Remove the not needed egg.info removal as well

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-devtools/python/python3-idna_3.4.bb | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/meta/recipes-devtools/python/python3-idna_3.4.bb 
b/meta/recipes-devtools/python/python3-idna_3.4.bb
index 10c99ee0ea..41a666f3f4 100644
--- a/meta/recipes-devtools/python/python3-idna_3.4.bb
+++ b/meta/recipes-devtools/python/python3-idna_3.4.bb
@@ -5,12 +5,7 @@ LIC_FILES_CHKSUM = 
"file://LICENSE.md;md5=239668a7c6066d9e0c5382e9c8c6c0e1"
 
 SRC_URI[sha256sum] = 
"814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4"
 
-inherit pypi setuptools3
-
-# Remove bundled egg-info
-do_compile:prepend() {
-rm -rf ${S}/idna.egg-info
-}
+inherit pypi python_flit_core
 
 RDEPENDS:${PN}:class-target = "\
 ${PYTHON_PN}-codecs \
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170843): 
https://lists.openembedded.org/g/openembedded-core/message/170843
Mute This Topic: https://lists.openembedded.org/mt/93757846/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCHv2 1/3] oe-setup-builddir: Correct when validation of the templates dir is run

[Konrad Weihmann]

A general remark from my side about this patch is that this limitation 
isn't mentioned in the documentation at all [1].
I would have been in favor of just dropping this piece of code, esp as 
the check wasn't run for years now - but here we are with another 
undocumented and breaking change (at least for me, as the template confs 
in my projects weren't located under templates, until everything fell 
apart in CI over night)


[1] 
https://docs.yoctoproject.org/ref-manual/structure.html?highlight=templateconf#oe-init-build-env

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170479): 
https://lists.openembedded.org/g/openembedded-core/message/170479
Mute This Topic: https://lists.openembedded.org/mt/93502783/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] python3: add wrapper for native

[Konrad Weihmann]

I read a more on this topic and found a less invasive way of disabling 
this feature globally for the native target.

Sadly there's no configure switch

Just send out a new patch, so this original series can be safely ignored

On 22.08.22 22:03, Alexander Kanavin wrote:

This needs to be applied to python3-native/python3, not just to the
nativepython3 link to it. You can use 'create_wrapper' helper that was
written exactly for the purpose (see meta/ for examples).

Also, can the lookup in ~ be simply disabled at python3-native build time?

Alex

On Mon, 22 Aug 2022 at 21:57, Konrad Weihmann  wrote:


which makes sure to set PYTHONNOUSERSITE to prevent
dynamically loading site packages from the host sided
package dirs.
In before when a module was using pluggy and/or
importlib_metadata the host sided path ~./local/... was still
part of the sys.path search tree and had priority over the
recipe-sysroot-native, leading to host/workspace contamination

Signed-off-by: Konrad Weihmann 
---
  meta/recipes-devtools/python/python3/nativepython3 | 2 ++
  meta/recipes-devtools/python/python3_3.10.6.bb | 9 -
  2 files changed, 6 insertions(+), 5 deletions(-)
  create mode 100644 meta/recipes-devtools/python/python3/nativepython3

diff --git a/meta/recipes-devtools/python/python3/nativepython3 
b/meta/recipes-devtools/python/python3/nativepython3
new file mode 100644
index 00..0413e44e32
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/nativepython3
@@ -0,0 +1,2 @@
+#!/bin/sh
+PYTHONNOUSERSITE=1 python3 $@
\ No newline at end of file
diff --git a/meta/recipes-devtools/python/python3_3.10.6.bb 
b/meta/recipes-devtools/python/python3_3.10.6.bb
index f19cd50273..1121c17887 100644
--- a/meta/recipes-devtools/python/python3_3.10.6.bb
+++ b/meta/recipes-devtools/python/python3_3.10.6.bb
@@ -43,6 +43,7 @@ SRC_URI:append:class-native = " \
 
file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
 file://12-distutils-prefix-is-inside-staging-area.patch \
 file://0001-Don-t-search-system-for-headers-libraries.patch \
+   file://nativepython3 \
 "
  SRC_URI[sha256sum] = 
"f795ff87d11d4b0c7c33bc8851b0c28648d8a4583aa2100a98c22b4326b6d3f3"

@@ -153,11 +154,9 @@ do_install:append:class-native() {
  for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python 
${D}${bindir}/${PN}`; do
  sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
  done
-# Add a symlink to the native Python so that scripts can just invoke
-# "nativepython" and get the right one without needing absolute paths
-# (these often end up too long for the #! parser in the kernel as the
-# buffer is 128 bytes long).
-ln -s python3-native/python3 ${D}${bindir}/nativepython3
+# Create a wrapper for python3 in native environment
+# make sure that no host sided modules are inherited
+install -m 755 ${WORKDIR}/nativepython3 ${D}${bindir}/nativepython3

  # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of 
them
  # and the overhead in each recipe-sysroot-native isn't worth it, 
particularly
--
2.34.1





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169695): 
https://lists.openembedded.org/g/openembedded-core/message/169695
Mute This Topic: https://lists.openembedded.org/mt/93189857/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] python3: disable user site-pkg for native target

[Konrad Weihmann]

by setting the ENABLE_USER_SITE flag globally.
This is the recommended way according to
https://peps.python.org/pep-0370/#implementation

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-devtools/python/python3_3.10.6.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-devtools/python/python3_3.10.6.bb 
b/meta/recipes-devtools/python/python3_3.10.6.bb
index f19cd50273..1b28728732 100644
--- a/meta/recipes-devtools/python/python3_3.10.6.bb
+++ b/meta/recipes-devtools/python/python3_3.10.6.bb
@@ -171,6 +171,9 @@ do_install:append:class-native() {
 # Nothing should be looking into ${B} for python3-native
 sed -i -e 's:${B}:/build/path/unavailable/:g' \
 
${D}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}*/Makefile
+
+# disable the lookup in user's site-packages globally
+sed -i 's#ENABLE_USER_SITE = None#ENABLE_USER_SITE = False#' 
${D}${libdir}/python${PYTHON_MAJMIN}/site.py
 }
 
 do_install:append() {
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169694): 
https://lists.openembedded.org/g/openembedded-core/message/169694
Mute This Topic: https://lists.openembedded.org/mt/93199390/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/2] python3: add wrapper for native

[Konrad Weihmann]

which makes sure to set PYTHONNOUSERSITE to prevent
dynamically loading site packages from the host sided
package dirs.
In before when a module was using pluggy and/or
importlib_metadata the host sided path ~./local/... was still
part of the sys.path search tree and had priority over the
recipe-sysroot-native, leading to host/workspace contamination

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-devtools/python/python3/nativepython3 | 2 ++
 meta/recipes-devtools/python/python3_3.10.6.bb | 9 -
 2 files changed, 6 insertions(+), 5 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python3/nativepython3

diff --git a/meta/recipes-devtools/python/python3/nativepython3 
b/meta/recipes-devtools/python/python3/nativepython3
new file mode 100644
index 00..0413e44e32
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/nativepython3
@@ -0,0 +1,2 @@
+#!/bin/sh
+PYTHONNOUSERSITE=1 python3 $@
\ No newline at end of file
diff --git a/meta/recipes-devtools/python/python3_3.10.6.bb 
b/meta/recipes-devtools/python/python3_3.10.6.bb
index f19cd50273..1121c17887 100644
--- a/meta/recipes-devtools/python/python3_3.10.6.bb
+++ b/meta/recipes-devtools/python/python3_3.10.6.bb
@@ -43,6 +43,7 @@ SRC_URI:append:class-native = " \

file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
file://12-distutils-prefix-is-inside-staging-area.patch \
file://0001-Don-t-search-system-for-headers-libraries.patch \
+   file://nativepython3 \
"
 SRC_URI[sha256sum] = 
"f795ff87d11d4b0c7c33bc8851b0c28648d8a4583aa2100a98c22b4326b6d3f3"
 
@@ -153,11 +154,9 @@ do_install:append:class-native() {
 for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python 
${D}${bindir}/${PN}`; do
 sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
 done
-# Add a symlink to the native Python so that scripts can just invoke
-# "nativepython" and get the right one without needing absolute paths
-# (these often end up too long for the #! parser in the kernel as the
-# buffer is 128 bytes long).
-ln -s python3-native/python3 ${D}${bindir}/nativepython3
+# Create a wrapper for python3 in native environment
+# make sure that no host sided modules are inherited
+install -m 755 ${WORKDIR}/nativepython3 ${D}${bindir}/nativepython3
 
 # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of 
them
 # and the overhead in each recipe-sysroot-native isn't worth it, 
particularly
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169681): 
https://lists.openembedded.org/g/openembedded-core/message/169681
Mute This Topic: https://lists.openembedded.org/mt/93189857/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/2] python3_pep517: use plain python

[Konrad Weihmann]

as nativepython3 is now a shell wrapper, prevent endless
recursion loops, by just calling python3 instead of nativepython3.
The interpreter setting will still have precedence over the plain
python binary called

Signed-off-by: Konrad Weihmann 
---
 meta/classes-recipe/python_pep517.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-recipe/python_pep517.bbclass 
b/meta/classes-recipe/python_pep517.bbclass
index 202dde0bc3..9484d61dec 100644
--- a/meta/classes-recipe/python_pep517.bbclass
+++ b/meta/classes-recipe/python_pep517.bbclass
@@ -48,7 +48,7 @@ python_pep517_do_install () {
 bbfatal More than one wheel found in ${PEP517_WHEEL_PATH}, this should 
not happen
 fi
 
-nativepython3 -m installer ${INSTALL_WHEEL_COMPILE_BYTECODE} --interpreter 
"${USRBINPATH}/env ${PEP517_INSTALL_PYTHON}" --destdir=${D} 
${PEP517_WHEEL_PATH}/*.whl
+python3 -m installer ${INSTALL_WHEEL_COMPILE_BYTECODE} --interpreter 
"${USRBINPATH}/env ${PEP517_INSTALL_PYTHON}" --destdir=${D} 
${PEP517_WHEEL_PATH}/*.whl
 }
 
 # A manual do_install that just uses unzip for bootstrapping purposes. Callers 
should DEPEND on unzip-native.
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169682): 
https://lists.openembedded.org/g/openembedded-core/message/169682
Mute This Topic: https://lists.openembedded.org/mt/93189858/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2] linux-firmware: replace mkdir by install

[Konrad Weihmann]

if a setup is using RPM for packaging and there are multiple
recipes that install to ${nonarch_base_libdir}/firmware by using
install -d ${nonarch_base_libdir}/firmware, it will create installation
clashes on image install, as linux-firmware in before this patch
used mkdir -p, which creates different file mode bits (depending
on the current user's settings).

In a particular example
linux-fimware created /lib/firmware with 0600
while other-firmware-package created it with 0644
making the combination not installable by rpm backend

Signed-off-by: Konrad Weihmann 
---
v2: - add ML link for Submission status
- rebase on latest master

 ...01-Makefile-replace-mkdir-by-install.patch | 84 +++
 .../linux-firmware/linux-firmware_20220411.bb |  5 +-
 2 files changed, 88 insertions(+), 1 deletion(-)
 create mode 100644 
meta/recipes-kernel/linux-firmware/files/0001-Makefile-replace-mkdir-by-install.patch

diff --git 
a/meta/recipes-kernel/linux-firmware/files/0001-Makefile-replace-mkdir-by-install.patch
 
b/meta/recipes-kernel/linux-firmware/files/0001-Makefile-replace-mkdir-by-install.patch
new file mode 100644
index 00..b1ac5a16ab
--- /dev/null
+++ 
b/meta/recipes-kernel/linux-firmware/files/0001-Makefile-replace-mkdir-by-install.patch
@@ -0,0 +1,84 @@
+From 71514e74f35f2b51ca24062573d6d913525b30db Mon Sep 17 00:00:00 2001
+From: Konrad Weihmann 
+Date: Mon, 9 May 2022 12:57:57 +0200
+Subject: [PATCH] Makefile: replace mkdir by install
+
+mkdir -p creates paths that are bound to user's settings and therefore
+can lead to different file mode bits of the base paths accross different
+machines.
+Use install instead, as this tool is not prone to such behavior.
+
+Signed-off-by: Konrad Weihmann 
+Upstream-Status: Submitted 
[https://lore.kernel.org/linux-firmware/pr2pr09mb310088ea719e6d7ca5c268f1a8...@pr2pr09mb3100.eurprd09.prod.outlook.com/]
+---
+ Makefile  | 2 +-
+ carl9170fw/toolchain/Makefile | 4 ++--
+ copy-firmware.sh  | 6 +++---
+ 3 files changed, 6 insertions(+), 6 deletions(-)
+
+diff --git a/Makefile b/Makefile
+index e1c362f..83a0ec6 100644
+--- a/Makefile
 b/Makefile
+@@ -9,5 +9,5 @@ check:
+   @./check_whence.py
+ 
+ install:
+-  mkdir -p $(DESTDIR)$(FIRMWAREDIR)
++  install -d $(DESTDIR)$(FIRMWAREDIR)
+   ./copy-firmware.sh $(DESTDIR)$(FIRMWAREDIR)
+diff --git a/carl9170fw/toolchain/Makefile b/carl9170fw/toolchain/Makefile
+index 2b25ffe..aaea8e8 100644
+--- a/carl9170fw/toolchain/Makefile
 b/carl9170fw/toolchain/Makefile
+@@ -46,14 +46,14 @@ src/gcc-$(GCC_VER): src/$(GCC_TAR) src/newlib-$(NEWLIB_VER)
+   ln -s $(BASEDIR)/src/newlib-$(NEWLIB_VER)/libgloss $@
+ 
+ binutils: src/binutils-$(BINUTILS_VER)
+-  mkdir -p build/binutils
++  install -d build/binutils
+   cd build/binutils; \
+   $(BASEDIR)/$ //g' 
| while read f d; d
+ if test -L "$f"; then
+ test -f "$destdir/$f" && continue
+ $verbose "copying link $f"
+-mkdir -p $destdir/$(dirname "$f")
++install -d $destdir/$(dirname "$f")
+ cp -d "$f" $destdir/"$f"
+ 
+ if test "x$d" != "x"; then
+@@ -63,7 +63,7 @@ grep -E '^Link:' WHENCE | sed -e's/^Link: *//g' -e's/-> //g' 
| while read f d; d
+ fi
+ else
+ $verbose "creating link $f -> $d"
+-mkdir -p $destdir/$(dirname "$f")
++install -d $destdir/$(dirname "$f")
+ ln -sf "$d" "$destdir/$f"
+ fi
+ done
+-- 
+2.25.1
+
diff --git a/meta/recipes-kernel/linux-firmware/linux-firmware_20220411.bb 
b/meta/recipes-kernel/linux-firmware/linux-firmware_20220411.bb
index 89e1b8cbaf..19b970c091 100644
--- a/meta/recipes-kernel/linux-firmware/linux-firmware_20220411.bb
+++ b/meta/recipes-kernel/linux-firmware/linux-firmware_20220411.bb
@@ -203,7 +203,10 @@ NO_GENERIC_LICENSE[WHENCE] = "WHENCE"
 
 PE = "1"
 
-SRC_URI = "${KERNELORG_MIRROR}/linux/kernel/firmware/${BPN}-${PV}.tar.xz"
+SRC_URI = "\
+  ${KERNELORG_MIRROR}/linux/kernel/firmware/${BPN}-${PV}.tar.xz \
+  file://0001-Makefile-replace-mkdir-by-install.patch \
+"
 
 SRC_URI[sha256sum] = 
"020b11f6412f4956f5a6f98de7d41867d2b30ea0ce81b1e2d206ec9840363849"
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165396): 
https://lists.openembedded.org/g/openembedded-core/message/165396
Mute This Topic: https://lists.openembedded.org/mt/90986447/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] linux-firmware: replace mkdir by install

[Konrad Weihmann]

if a setup is using RPM for packaging and there are multiple
recipes that install to ${nonarch_base_libdir}/firmware by using
install -d ${nonarch_base_libdir}/firmware, it will create installation
clashes on image install, as linux-firmware in before this patch
used mkdir -p, which creates different file mode bits (depending
on the current user's settings).

In a particular example
linux-fimware created /lib/firmware with 0600
while other-firmware-package created it with 0644
making the combination not installable by rpm backend

Signed-off-by: Konrad Weihmann 
---
 ...01-Makefile-replace-mkdir-by-install.patch | 84 +++
 .../linux-firmware/linux-firmware_20220209.bb |  5 +-
 2 files changed, 88 insertions(+), 1 deletion(-)
 create mode 100644 
meta/recipes-kernel/linux-firmware/files/0001-Makefile-replace-mkdir-by-install.patch

diff --git 
a/meta/recipes-kernel/linux-firmware/files/0001-Makefile-replace-mkdir-by-install.patch
 
b/meta/recipes-kernel/linux-firmware/files/0001-Makefile-replace-mkdir-by-install.patch
new file mode 100644
index 00..16e0fa8986
--- /dev/null
+++ 
b/meta/recipes-kernel/linux-firmware/files/0001-Makefile-replace-mkdir-by-install.patch
@@ -0,0 +1,84 @@
+From 81bbf4495ab59979948043fa13f1c18db249fbdd Mon Sep 17 00:00:00 2001
+From: Konrad Weihmann 
+Date: Mon, 9 May 2022 12:23:28 +0200
+Subject: [PATCH] Makefile: replace mkdir by install
+
+mkdir -p creates paths that are bound to user's settings and therefore
+can lead to different file mode bits of the base paths accross different
+machines.
+Use install instead, as this tool is not prone to such behavior.
+
+Signed-off-by: Konrad Weihmann 
+Upstream-Status: Submitted
+---
+ Makefile  | 2 +-
+ carl9170fw/toolchain/Makefile | 4 ++--
+ copy-firmware.sh  | 6 +++---
+ 3 files changed, 6 insertions(+), 6 deletions(-)
+
+diff --git a/Makefile b/Makefile
+index e1c362f..83a0ec6 100644
+--- a/Makefile
 b/Makefile
+@@ -9,5 +9,5 @@ check:
+   @./check_whence.py
+ 
+ install:
+-  mkdir -p $(DESTDIR)$(FIRMWAREDIR)
++  install -d $(DESTDIR)$(FIRMWAREDIR)
+   ./copy-firmware.sh $(DESTDIR)$(FIRMWAREDIR)
+diff --git a/carl9170fw/toolchain/Makefile b/carl9170fw/toolchain/Makefile
+index 2b25ffe..aaea8e8 100644
+--- a/carl9170fw/toolchain/Makefile
 b/carl9170fw/toolchain/Makefile
+@@ -46,14 +46,14 @@ src/gcc-$(GCC_VER): src/$(GCC_TAR) src/newlib-$(NEWLIB_VER)
+   ln -s $(BASEDIR)/src/newlib-$(NEWLIB_VER)/libgloss $@
+ 
+ binutils: src/binutils-$(BINUTILS_VER)
+-  mkdir -p build/binutils
++  install -d build/binutils
+   cd build/binutils; \
+   $(BASEDIR)/$ //g' 
| while read f d; d
+ if test -L "$f"; then
+ test -f "$destdir/$f" && continue
+ $verbose "copying link $f"
+-mkdir -p $destdir/$(dirname "$f")
++install -d $destdir/$(dirname "$f")
+ cp -d "$f" $destdir/"$f"
+ 
+ if test "x$d" != "x"; then
+@@ -63,7 +63,7 @@ grep -E '^Link:' WHENCE | sed -e's/^Link: *//g' -e's/-> //g' 
| while read f d; d
+ fi
+ else
+ $verbose "creating link $f -> $d"
+-mkdir -p $destdir/$(dirname "$f")
++install -d $destdir/$(dirname "$f")
+ ln -sf "$d" "$destdir/$f"
+ fi
+ done
+-- 
+2.25.1
+
diff --git a/meta/recipes-kernel/linux-firmware/linux-firmware_20220209.bb 
b/meta/recipes-kernel/linux-firmware/linux-firmware_20220209.bb
index fe51892eb4..4758466aa9 100644
--- a/meta/recipes-kernel/linux-firmware/linux-firmware_20220209.bb
+++ b/meta/recipes-kernel/linux-firmware/linux-firmware_20220209.bb
@@ -203,7 +203,10 @@ NO_GENERIC_LICENSE[WHENCE] = "WHENCE"
 
 PE = "1"
 
-SRC_URI = "${KERNELORG_MIRROR}/linux/kernel/firmware/${BPN}-${PV}.tar.xz"
+SRC_URI = "\
+  ${KERNELORG_MIRROR}/linux/kernel/firmware/${BPN}-${PV}.tar.xz \
+  file://0001-Makefile-replace-mkdir-by-install.patch \
+"
 
 SRC_URI[sha256sum] = 
"e2e46fa618414952bbf2f6920cd3abcddbef45bfb7d1352994b4bfc35394d177"
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165392): 
https://lists.openembedded.org/g/openembedded-core/message/165392
Mute This Topic: https://lists.openembedded.org/mt/90986193/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 1/4] git: correct license

[Konrad Weihmann]

by default git pulls in several code fragments not being licensed
under just GPL-2.0-only.

obstack and poll are licensed under GPL-2.0-or-later
reftable being BSD-3-Clause
sha1dc and inet_ntop being MIT
netmalloc being Bosst-1.0 aka BSL-1.0
regex being LGPL-2.1-or-later

Signed-off-by: Konrad Weihmann 
---
v2:
- list GPL-2.0-only explicitly

 meta/recipes-devtools/git/git_2.35.1.bb | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/git/git_2.35.1.bb 
b/meta/recipes-devtools/git/git_2.35.1.bb
index 47c2211864..918ff68bbf 100644
--- a/meta/recipes-devtools/git/git_2.35.1.bb
+++ b/meta/recipes-devtools/git/git_2.35.1.bb
@@ -2,7 +2,7 @@ SUMMARY = "Distributed version control system"
 HOMEPAGE = "http://git-scm.com;
 DESCRIPTION = "Git is a free and open source distributed version control 
system designed to handle everything from small to very large projects with 
speed and efficiency."
 SECTION = "console/utils"
-LICENSE = "GPL-2.0-only"
+LICENSE = "GPL-2.0-only & GPL-2.0-or-later & BSD-3-Clause & MIT & BSL-1.0 & 
LGPL-2.1-or-later"
 DEPENDS = "openssl zlib"
 
 PROVIDES:append:class-native = " git-replacement-native"
@@ -14,7 +14,16 @@ SRC_URI = 
"${KERNELORG_MIRROR}/software/scm/git/git-${PV}.tar.gz;name=tarball \
 
 S = "${WORKDIR}/git-${PV}"
 
-LIC_FILES_CHKSUM = "file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1"
+LIC_FILES_CHKSUM = "\
+   file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1 \
+   file://reftable/LICENSE;md5=1a6424cafc4c9c88c689848e165af33b \
+   file://sha1dc/LICENSE.txt;md5=9bbe4c990a9e98ea4b98ef5d3bcb8a7a \
+   
file://compat/nedmalloc/License.txt;md5=e4224ccaecb14d942c71d31bef20d78c \
+   
file://compat/inet_ntop.c;md5=76593c6f74e8ced5b24520175688d59b;endline=16 \
+   file://compat/obstack.h;md5=08ad25fee5428cd879ceef451ce3a22e;endline=18 
\
+   
file://compat/poll/poll.h;md5=9fc00170a53b8e3e52157c91ac688dd1;endline=19 \
+   
file://compat/regex/regex.h;md5=30cc8af0e6f0f8a25acec6d8783bb763;beginline=4;endline=22
 \
+"
 
 CVE_PRODUCT = "git-scm:git"
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164391): 
https://lists.openembedded.org/g/openembedded-core/message/164391
Mute This Topic: https://lists.openembedded.org/mt/90459114/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 2/4] ncurses: use COPYING file

[Konrad Weihmann]

which has the same info as the in-file header used in before

Signed-off-by: Konrad Weihmann 
---
v2:
- strip control codes at the end of the copying file

 meta/recipes-core/ncurses/ncurses.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/ncurses/ncurses.inc 
b/meta/recipes-core/ncurses/ncurses.inc
index a0ecd8a80b..7a7c7dd227 100644
--- a/meta/recipes-core/ncurses/ncurses.inc
+++ b/meta/recipes-core/ncurses/ncurses.inc
@@ -2,7 +2,7 @@ SUMMARY = "The New Curses library"
 DESCRIPTION = "SVr4 and XSI-Curses compatible curses library and terminfo 
tools including tic, infocmp, captoinfo. Supports color, multiple highlights, 
forms-drawing characters, and automatic recognition of keypad and function-key 
sequences. Extensions include resizable windows and mouse support on both xterm 
and Linux console using the gpm library."
 HOMEPAGE = "http://www.gnu.org/software/ncurses/ncurses.html;
 LICENSE = "MIT"
-LIC_FILES_CHKSUM = 
"file://ncurses/base/version.c;beginline=1;endline=27;md5=5526f2f3a29edc95538b368a4771edda"
+LIC_FILES_CHKSUM = 
"file://COPYING;md5=9529289636145d1bf093c96af067695a;endline=27"
 SECTION = "libs"
 DEPENDS = "ncurses-native"
 DEPENDS:class-native = ""
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164392): 
https://lists.openembedded.org/g/openembedded-core/message/164392
Mute This Topic: https://lists.openembedded.org/mt/90459115/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] cmake: add missing licenses

[Konrad Weihmann]

in target and native variant a different set of vendored libraries
is pulled from the cmake sources.
Add those licenses and there texts

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-devtools/cmake/cmake-native_3.22.3.bb | 8 
 meta/recipes-devtools/cmake/cmake_3.22.3.bb| 7 +++
 2 files changed, 15 insertions(+)

diff --git a/meta/recipes-devtools/cmake/cmake-native_3.22.3.bb 
b/meta/recipes-devtools/cmake/cmake-native_3.22.3.bb
index 8293fe5e56..ee1f7761c4 100644
--- a/meta/recipes-devtools/cmake/cmake-native_3.22.3.bb
+++ b/meta/recipes-devtools/cmake/cmake-native_3.22.3.bb
@@ -9,6 +9,14 @@ SRC_URI += "file://OEToolchainConfig.cmake \
 
file://0005-Disable-use-of-ext2fs-ext2_fs.h-by-cmake-s-internal-.patch \
 "
 
+LICENSE:append = " & BSD-1-Clause & MIT & BSD-2-Clause"
+LIC_FILES_CHKSUM:append = " \
+file://Utilities/cmjsoncpp/LICENSE;md5=fa2a23dd1dc6c139f35105379d76df2b \
+file://Utilities/cmlibarchive/COPYING;md5=d499814247adaee08d88080841cb5665 
\
+file://Utilities/cmexpat/COPYING;md5=9e2ce3b3c4c0f2670883a23bbd7c37a9 \
+file://Utilities/cmlibrhash/COPYING;md5=a8c2a557a5c53b1c12cddbee98c099af \
+file://Utilities/cmlibuv/LICENSE;md5=a68902a430e32200263d182d44924d47 \
+"
 
 B = "${WORKDIR}/build"
 do_configure[cleandirs] = "${B}"
diff --git a/meta/recipes-devtools/cmake/cmake_3.22.3.bb 
b/meta/recipes-devtools/cmake/cmake_3.22.3.bb
index e111dd3c5a..752c37ba7d 100644
--- a/meta/recipes-devtools/cmake/cmake_3.22.3.bb
+++ b/meta/recipes-devtools/cmake/cmake_3.22.3.bb
@@ -12,6 +12,13 @@ SRC_URI:append:class-nativesdk = " \
 file://0001-CMakeDetermineSystem-use-oe-environment-vars-to-load.patch \
 "
 
+LICENSE:append = " & BSD-1-Clause & MIT"
+LIC_FILES_CHKSUM:append = " \
+file://Utilities/cmjsoncpp/LICENSE;md5=fa2a23dd1dc6c139f35105379d76df2b \
+file://Utilities/cmlibrhash/COPYING;md5=a8c2a557a5c53b1c12cddbee98c099af \
+file://Utilities/cmlibuv/LICENSE;md5=a68902a430e32200263d182d44924d47 \
+"
+
 # Strip ${prefix} from ${docdir}, set result into docdir_stripped
 python () {
 prefix=d.getVar("prefix")
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164348): 
https://lists.openembedded.org/g/openembedded-core/message/164348
Mute This Topic: https://lists.openembedded.org/mt/90446195/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/2] ncurses: use COPYING file

[Konrad Weihmann]

which has the same info as the in-file header used in before

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-core/ncurses/ncurses.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/ncurses/ncurses.inc 
b/meta/recipes-core/ncurses/ncurses.inc
index a0ecd8a80b..8d374a1522 100644
--- a/meta/recipes-core/ncurses/ncurses.inc
+++ b/meta/recipes-core/ncurses/ncurses.inc
@@ -2,7 +2,7 @@ SUMMARY = "The New Curses library"
 DESCRIPTION = "SVr4 and XSI-Curses compatible curses library and terminfo 
tools including tic, infocmp, captoinfo. Supports color, multiple highlights, 
forms-drawing characters, and automatic recognition of keypad and function-key 
sequences. Extensions include resizable windows and mouse support on both xterm 
and Linux console using the gpm library."
 HOMEPAGE = "http://www.gnu.org/software/ncurses/ncurses.html;
 LICENSE = "MIT"
-LIC_FILES_CHKSUM = 
"file://ncurses/base/version.c;beginline=1;endline=27;md5=5526f2f3a29edc95538b368a4771edda"
+LIC_FILES_CHKSUM = "file://COPYING;md5=f852913c5d988a5f5a2f1df7ba7ee893"
 SECTION = "libs"
 DEPENDS = "ncurses-native"
 DEPENDS:class-native = ""
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164345): 
https://lists.openembedded.org/g/openembedded-core/message/164345
Mute This Topic: https://lists.openembedded.org/mt/90445642/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/2] python3-pip: correct license

[Konrad Weihmann]

as described in src/pip/_vendor/README.rst pip ships plenty
of vendored copies of other python modules.
Correct the license of the resulting package and
reference all the vendor copy license files correctly

Signed-off-by: Konrad Weihmann 
---
 .../python/python3-pip_22.0.3.bb  | 32 +--
 1 file changed, 30 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/python/python3-pip_22.0.3.bb 
b/meta/recipes-devtools/python/python3-pip_22.0.3.bb
index ee3e578b2c..09a305edf8 100644
--- a/meta/recipes-devtools/python/python3-pip_22.0.3.bb
+++ b/meta/recipes-devtools/python/python3-pip_22.0.3.bb
@@ -1,8 +1,36 @@
 SUMMARY = "The PyPA recommended tool for installing Python packages"
 HOMEPAGE = "https://pypi.org/project/pip;
 SECTION = "devel/python"
-LICENSE = "MIT"
-LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=63ec52baf95163b597008bb46db68030"
+LICENSE = "MIT & Apache-2.0 & MPL-2.0 & LGPL-2.1-only & BSD-3-Clause & PSF-2.0 
& BSD-2-Clause"
+LIC_FILES_CHKSUM = "\
+  file://LICENSE.txt;md5=63ec52baf95163b597008bb46db68030 \
+  
file://src/pip/_vendor/cachecontrol/LICENSE.txt;md5=6572692148079ebbbd800be4b9f36c6d
 \
+  file://src/pip/_vendor/certifi/LICENSE;md5=67da0714c3f9471067b729eca6c9fbe8 \
+  file://src/pip/_vendor/chardet/LICENSE;md5=a6f89e2100d9b6cdffcea4f398e37343 \
+  
file://src/pip/_vendor/colorama/LICENSE.txt;md5=b4936429a56a652b84c5c01280dcaa26
 \
+  
file://src/pip/_vendor/distlib/LICENSE.txt;md5=f6a11430d5cd6e2cd3832ee94f22ddfc 
\
+  file://src/pip/_vendor/distro.LICENSE;md5=d2794c0df5b907fdace235a619d80314 \
+  file://src/pip/_vendor/html5lib/LICENSE;md5=1ba5ada9e6fead1fdc32f43c9f10ba7c 
\
+  file://src/pip/_vendor/idna/LICENSE.md;md5=239668a7c6066d9e0c5382e9c8c6c0e1 \
+  file://src/pip/_vendor/msgpack/COPYING;md5=cd9523181d9d4fbf7ffca52eaa2a5751 \
+  
file://src/pip/_vendor/packaging/LICENSE;md5=faadaedca9251a90b205c9167578ce91 \
+  
file://src/pip/_vendor/packaging/LICENSE.APACHE;md5=2ee41112a44fe7014dce33e26468ba93
 \
+  file://src/pip/_vendor/pep517/LICENSE;md5=aad69c93f605003e3342b174d9b0708c \
+  
file://src/pip/_vendor/pkg_resources/LICENSE;md5=9a33897f1bca1160d7aad3835152e158
 \
+  
file://src/pip/_vendor/platformdirs/LICENSE.txt;md5=282c970bb844954c8535dd6e9733db7f
 \
+  file://src/pip/_vendor/progress/LICENSE;md5=00ab78a4113b09aacf63d762a7bb9644 
\
+  file://src/pip/_vendor/pygments/LICENSE;md5=98419e351433ac106a24e3ad435930bc 
\
+  
file://src/pip/_vendor/pyparsing/LICENSE;md5=657a566233888513e1f07ba13e2f47f1 \
+  file://src/pip/_vendor/requests/LICENSE;md5=34400b68072d710fecd0a2940a0d1658 
\
+  
file://src/pip/_vendor/resolvelib/LICENSE;md5=78e1c0248051c32a38a7f820c30bd7a5 \
+  file://src/pip/_vendor/rich/LICENSE;md5=b5f0b94fbc94f5ad9ae4efcf8a778303 \
+  file://src/pip/_vendor/six.LICENSE;md5=43cfc9e4ac0e377acfb9b76f56b8415d \
+  file://src/pip/_vendor/tenacity/LICENSE;md5=175792518e4ac015ab6696d16c4f607e 
\
+  file://src/pip/_vendor/tomli/LICENSE;md5=f0879d17df0110d1aa8c8c9f46f5 \
+  
file://src/pip/_vendor/typing_extensions.LICENSE;md5=64fc2b30b67d0a8423c250e0386ed72f
 \
+  
file://src/pip/_vendor/urllib3/LICENSE.txt;md5=c2823cb995439c984fd62a973d79815c 
\
+  
file://src/pip/_vendor/webencodings/LICENSE;md5=81fb24cd7823cce23b69f721993dce4d
 \
+"
 
 inherit pypi python_setuptools_build_meta
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164346): 
https://lists.openembedded.org/g/openembedded-core/message/164346
Mute This Topic: https://lists.openembedded.org/mt/90445643/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] git: correct license

[Konrad Weihmann]

On 13.04.22 18:56, Jose Quaresma wrote:

Hi

Konrad Weihmann mailto:kweihm...@outlook.com>> 
escreveu no dia quarta, 13/04/2022 à(s) 17:34:


by default git pulls in several code fragments not being licensed
under just GPL-2.0-only.
In fact obstack and poll are taken from glib, being licensed
under GPL-2.0-or-later - as this includes GPL-2.0-only, use this instead

Furthermore
reftable being BSD-3-Clause
sha1dc and inet_ntop being MIT
netmalloc being Bosst-1.0 aka BSL-1.0
regex being LGPL-2.1-or-later

Signed-off-by: Konrad Weihmann mailto:kweihm...@outlook.com>>
---
  meta/recipes-devtools/git/git_2.35.1.bb <http://git_2.35.1.bb> |
13 +++--
  1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/git/git_2.35.1.bb
<http://git_2.35.1.bb> b/meta/recipes-devtools/git/git_2.35.1.bb
<http://git_2.35.1.bb>
index 47c2211864..f8756b6dec 100644
--- a/meta/recipes-devtools/git/git_2.35.1.bb <http://git_2.35.1.bb>
+++ b/meta/recipes-devtools/git/git_2.35.1.bb <http://git_2.35.1.bb>
@@ -2,7 +2,7 @@ SUMMARY = "Distributed version control system"
  HOMEPAGE = "http://git-scm.com <http://git-scm.com>"
  DESCRIPTION = "Git is a free and open source distributed version
control system designed to handle everything from small to very
large projects with speed and efficiency."
  SECTION = "console/utils"
-LICENSE = "GPL-2.0-only"
+LICENSE = "GPL-2.0-or-later & BSD-3-Clause & MIT & BSL-1.0 &
LGPL-2.1-or-later"
  DEPENDS = "openssl zlib"

  PROVIDES:append:class-native = " git-replacement-native"
@@ -14,7 +14,16 @@ SRC_URI =
"${KERNELORG_MIRROR}/software/scm/git/git-${PV}.tar.gz;name=tarball \

  S = "${WORKDIR}/git-${PV}"

-LIC_FILES_CHKSUM =
"file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1"
+LIC_FILES_CHKSUM = "\
+       file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1 \
+       file://reftable/LICENSE;md5=1a6424cafc4c9c88c689848e165af33b \
+       file://sha1dc/LICENSE.txt;md5=9bbe4c990a9e98ea4b98ef5d3bcb8a7a \
+ 
  file://compat/nedmalloc/License.txt;md5=e4224ccaecb14d942c71d31bef20d78c \
+ 
  file://compat/inet_ntop.c;md5=76593c6f74e8ced5b24520175688d59b;endline=16 \
+ 
  file://compat/obstack.h;md5=08ad25fee5428cd879ceef451ce3a22e;endline=18 \
+ 
  file://compat/poll/poll.h;md5=9fc00170a53b8e3e52157c91ac688dd1;endline=19 \
+ 
  file://compat/regex/regex.h;md5=30cc8af0e6f0f8a25acec6d8783bb763;beginline=4;endline=22 \

+"


I don't know if it is possible but if we can add a compat PACKAGECONFIG 
will be great

so we can disable this part which simplifies a little the license.


That would be indeed nice to have - from my understanding of the 
makefile those are build unconditionally, so it would require a upstream 
work first I guess.


For now my patch is just reflecting what is actually used



Jose


  CVE_PRODUCT = "git-scm:git"

-- 
2.25.1








--
Best regards,

José Quaresma

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164344): 
https://lists.openembedded.org/g/openembedded-core/message/164344
Mute This Topic: https://lists.openembedded.org/mt/90444706/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] git: correct license

[Konrad Weihmann]

by default git pulls in several code fragments not being licensed
under just GPL-2.0-only.
In fact obstack and poll are taken from glib, being licensed
under GPL-2.0-or-later - as this includes GPL-2.0-only, use this instead

Furthermore
reftable being BSD-3-Clause
sha1dc and inet_ntop being MIT
netmalloc being Bosst-1.0 aka BSL-1.0
regex being LGPL-2.1-or-later

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-devtools/git/git_2.35.1.bb | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/git/git_2.35.1.bb 
b/meta/recipes-devtools/git/git_2.35.1.bb
index 47c2211864..f8756b6dec 100644
--- a/meta/recipes-devtools/git/git_2.35.1.bb
+++ b/meta/recipes-devtools/git/git_2.35.1.bb
@@ -2,7 +2,7 @@ SUMMARY = "Distributed version control system"
 HOMEPAGE = "http://git-scm.com;
 DESCRIPTION = "Git is a free and open source distributed version control 
system designed to handle everything from small to very large projects with 
speed and efficiency."
 SECTION = "console/utils"
-LICENSE = "GPL-2.0-only"
+LICENSE = "GPL-2.0-or-later & BSD-3-Clause & MIT & BSL-1.0 & LGPL-2.1-or-later"
 DEPENDS = "openssl zlib"
 
 PROVIDES:append:class-native = " git-replacement-native"
@@ -14,7 +14,16 @@ SRC_URI = 
"${KERNELORG_MIRROR}/software/scm/git/git-${PV}.tar.gz;name=tarball \
 
 S = "${WORKDIR}/git-${PV}"
 
-LIC_FILES_CHKSUM = "file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1"
+LIC_FILES_CHKSUM = "\
+   file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1 \
+   file://reftable/LICENSE;md5=1a6424cafc4c9c88c689848e165af33b \
+   file://sha1dc/LICENSE.txt;md5=9bbe4c990a9e98ea4b98ef5d3bcb8a7a \
+   
file://compat/nedmalloc/License.txt;md5=e4224ccaecb14d942c71d31bef20d78c \
+   
file://compat/inet_ntop.c;md5=76593c6f74e8ced5b24520175688d59b;endline=16 \
+   file://compat/obstack.h;md5=08ad25fee5428cd879ceef451ce3a22e;endline=18 
\
+   
file://compat/poll/poll.h;md5=9fc00170a53b8e3e52157c91ac688dd1;endline=19 \
+   
file://compat/regex/regex.h;md5=30cc8af0e6f0f8a25acec6d8783bb763;beginline=4;endline=22
 \
+"
 
 CVE_PRODUCT = "git-scm:git"
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164341): 
https://lists.openembedded.org/g/openembedded-core/message/164341
Mute This Topic: https://lists.openembedded.org/mt/90444706/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 3/3] gettext: add MIT conditional as license

[Konrad Weihmann]

depending on the actual PACKAGECONFIG some
internal vendor copies of libxml, libcroco and glib will
be used.
In the case of libxml this adds MIT to the license.
Reference the license statements based on the actual choosen
PACKAGECONFIG

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-core/gettext/gettext_0.21.bb | 9 +
 1 file changed, 9 insertions(+)

diff --git a/meta/recipes-core/gettext/gettext_0.21.bb 
b/meta/recipes-core/gettext/gettext_0.21.bb
index 30c1955d86..364e6a52a1 100644
--- a/meta/recipes-core/gettext/gettext_0.21.bb
+++ b/meta/recipes-core/gettext/gettext_0.21.bb
@@ -8,6 +8,15 @@ SECTION = "libs"
 LICENSE = "GPL-3.0-or-later & LGPL-2.1-or-later"
 LIC_FILES_CHKSUM = "file://COPYING;md5=c678957b0c8e964aa6c70fd77641a71e"
 
+# without libxml in PACKAGECONFIG vendor copy of the lib will be used
+LICENSE:append = " ${@bb.utils.contains('PACKAGECONFIG', 'libxml', '', '& 
MIT', d)}"
+LIC_FILES_CHKSUM:append = " ${@bb.utils.contains('PACKAGECONFIG', 'libxml', 
'', 
'file://libtextstyle/lib/libxml/COPYING;md5=2044417e2e5006b65a8b9067b683fcf1', 
d)}"
+# without croco in PACKAGECONFIG vendor copy of the lib will be used
+LIC_FILES_CHKSUM:append = " ${@bb.utils.contains('PACKAGECONFIG', 'croco', '', 
'file://libtextstyle/lib/libcroco/libcroco.h;md5=915a46e7307c2f7f8d2b9c503fc434ed;beginline=10;endline=28',
 d)}"
+# without glib in PACKAGECONFIG vendor copy of the lib will be used
+LIC_FILES_CHKSUM:append = " ${@bb.utils.contains('PACKAGECONFIG', 'glib', '', 
'file://libtextstyle/lib/glib/ghash.c;md5=af89a160226edf0b276b6183888037d0;beginline=10;endline=27',
 d)}"
+
+
 DEPENDS = "gettext-native virtual/libiconv"
 DEPENDS:class-native = "gettext-minimal-native"
 PROVIDES = "virtual/libintl virtual/gettext"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164335): 
https://lists.openembedded.org/g/openembedded-core/message/164335
Mute This Topic: https://lists.openembedded.org/mt/90443246/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/3] libidn2: add Unicode-DFS-2016 license

[Konrad Weihmann]

as COPYING clearly states that unicode data is baked into
the lib.
Add the license and reference the COPYING file for that

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-extended/libidn/libidn2_2.3.2.bb | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-extended/libidn/libidn2_2.3.2.bb 
b/meta/recipes-extended/libidn/libidn2_2.3.2.bb
index 7316c031ce..e1b25b4b8c 100644
--- a/meta/recipes-extended/libidn/libidn2_2.3.2.bb
+++ b/meta/recipes-extended/libidn/libidn2_2.3.2.bb
@@ -2,10 +2,11 @@ SUMMARY = "Internationalized Domain Name support library"
 DESCRIPTION = "Implementation of the Stringprep, Punycode and IDNA 
specifications defined by the IETF Internationalized Domain Names (IDN) working 
group."
 HOMEPAGE = "http://www.gnu.org/software/libidn/;
 SECTION = "libs"
-LICENSE = "(GPL-2.0-or-later | LGPL-3.0-only) & GPL-3.0-or-later"
+LICENSE = "(GPL-2.0-or-later | LGPL-3.0-only) & GPL-3.0-or-later & 
Unicode-DFS-2016"
 LIC_FILES_CHKSUM = "file://COPYING;md5=2d834ea7d480438ada04e5d846152395 \
 
file://COPYING.LESSERv3;md5=e6a600fd5e1d9cbde2d983680233ad02 \
 file://COPYINGv2;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
+
file://COPYING.unicode;md5=684cf5f7e3fded3546679424528261a9 \
 
file://src/idn2.c;endline=16;md5=e4b6d628a84a55f1fd8ae4c76c5f6509 \
 
file://lib/idn2.h.in;endline=27;md5=d0fc8ec628be130a1d5b889107e92477"
 
@@ -25,7 +26,7 @@ do_install:append() {
sed -i -e 's|-L${STAGING_LIBDIR}||' -e 's/  */ /g' 
${D}${libdir}/pkgconfig/libidn2.pc
 }
 
-LICENSE:${PN} = "(GPL-2.0-or-later | LGPL-3.0-only)"
+LICENSE:${PN} = "(GPL-2.0-or-later | LGPL-3.0-only) & Unicode-DFS-2016"
 LICENSE:${PN}-bin = "GPL-3.0-or-later"
 
 BBCLASSEXTEND = "native nativesdk"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164334): 
https://lists.openembedded.org/g/openembedded-core/message/164334
Mute This Topic: https://lists.openembedded.org/mt/90443245/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/3] libsdl2: fix license

[Konrad Weihmann]

by default libsdl2 is build with code from src/video/yuv2rgb, which
is licensed under BSD-2-Clause.
Additional by default hidapi is build, which is licensed under
GPL3 | BSD-2-Clause | HIDAPI license, pick the least restrictive
and best matching BSD-2-Clause.

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-graphics/libsdl2/libsdl2_2.0.20.bb | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-graphics/libsdl2/libsdl2_2.0.20.bb 
b/meta/recipes-graphics/libsdl2/libsdl2_2.0.20.bb
index 4ba22c1fe6..c1c827af79 100644
--- a/meta/recipes-graphics/libsdl2/libsdl2_2.0.20.bb
+++ b/meta/recipes-graphics/libsdl2/libsdl2_2.0.20.bb
@@ -7,8 +7,13 @@ BUGTRACKER = "http://bugzilla.libsdl.org/;
 
 SECTION = "libs"
 
-LICENSE = "Zlib"
-LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=68a088513da90254b2fbe664f42af315"
+LICENSE = "Zlib & BSD-2-Clause"
+LIC_FILES_CHKSUM = "\
+file://LICENSE.txt;md5=68a088513da90254b2fbe664f42af315 \
+file://src/hidapi/LICENSE.txt;md5=7c3949a631240cb6c31c50f3eb696077 \
+file://src/hidapi/LICENSE-bsd.txt;md5=b5fa085ce0926bb50d0621620a82361f \
+file://src/video/yuv2rgb/LICENSE;md5=79f8f3418d91531e05f0fc94ca67e071 \
+"
 
 # arm-neon adds MIT license
 LICENSE:append = " ${@bb.utils.contains('PACKAGECONFIG', 'arm-neon', '& MIT', 
'', d)}"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164333): 
https://lists.openembedded.org/g/openembedded-core/message/164333
Mute This Topic: https://lists.openembedded.org/mt/90443244/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2] libcap: add pam_cap license

[Konrad Weihmann]

If libcap is compiled with pam in PACKAGECONFIG
one additional license text becomes effective,
add that as a conditional

Signed-off-by: Konrad Weihmann 
---
v2:
- use spaces instead of tabs
- simplify by using bb.utils.contains

 meta/recipes-support/libcap/libcap_2.63.bb | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-support/libcap/libcap_2.63.bb 
b/meta/recipes-support/libcap/libcap_2.63.bb
index f3133f9ded..9e341c4bd0 100644
--- a/meta/recipes-support/libcap/libcap_2.63.bb
+++ b/meta/recipes-support/libcap/libcap_2.63.bb
@@ -5,7 +5,11 @@ users, without giving them full root permissions."
 HOMEPAGE = "http://sites.google.com/site/fullycapable/;
 # no specific GPL version required
 LICENSE = "BSD-3-Clause | GPL-2.0-only"
-LIC_FILES_CHKSUM = "file://License;md5=e2370ba375efe9e1a095c26d37e483b8"
+LIC_FILES_CHKSUM_PAM = 
"file://pam_cap/License;md5=0ad4c9c052b9719ee4fce1bfc7c7dee4"
+LIC_FILES_CHKSUM = "\
+file://License;md5=e2370ba375efe9e1a095c26d37e483b8 \
+${@bb.utils.contains('PACKAGECONFIG', 'pam', '${LIC_FILES_CHKSUM_PAM}', 
'', d)} \
+"
 
 DEPENDS = "hostperl-runtime-native gperf-native"
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164304): 
https://lists.openembedded.org/g/openembedded-core/message/164304
Mute This Topic: https://lists.openembedded.org/mt/90435857/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/4] kern-tools-native: add missing license

[Konrad Weihmann]

add the Kconfiglib license, as this was missing in before.
Add MIT identifier to LICENSE

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-kernel/kern-tools/kern-tools-native_git.bb | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb 
b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
index 0dcf1281f0..a6ab9ca56d 100644
--- a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
+++ b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
@@ -3,8 +3,11 @@ DESCRIPTION = "Powerful set of tools or managing Yocto Linux 
kernel sources \
 and configuration data. You can use these tools to make a single configuration 
\
 change, apply multiple patches, or work with your own kernel sources."
 HOMEPAGE = "https://www.yoctoproject.org/;
-LICENSE = "GPL-2.0-only"
-LIC_FILES_CHKSUM = 
"file://tools/kgit;beginline=5;endline=9;md5=9c30e971d435e249624278c3e343e501"
+LICENSE = "GPL-2.0-only & MIT"
+LIC_FILES_CHKSUM = "\
+   
file://tools/kgit;beginline=5;endline=9;md5=9c30e971d435e249624278c3e343e501 \
+   file://Kconfiglib/LICENSE.txt;md5=712177a72a3937909543eda3ad1bfb7c \
+"
 
 DEPENDS = "git-native"
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164255): 
https://lists.openembedded.org/g/openembedded-core/message/164255
Mute This Topic: https://lists.openembedded.org/mt/90413416/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/4] gmp: add missing COPYINGv3

[Konrad Weihmann]

add COPYINGv3 license text to LIC_FILES_CHKSUM

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-support/gmp/gmp_6.2.1.bb | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-support/gmp/gmp_6.2.1.bb 
b/meta/recipes-support/gmp/gmp_6.2.1.bb
index 72fbf4eefe..29055492c2 100644
--- a/meta/recipes-support/gmp/gmp_6.2.1.bb
+++ b/meta/recipes-support/gmp/gmp_6.2.1.bb
@@ -2,9 +2,11 @@ require gmp.inc
 
 LICENSE = "GPL-2.0-or-later | LGPL-3.0-or-later"
 
-LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
-   
file://COPYING.LESSERv3;md5=6a6a8e020838b23406c81b19c1d46df6 \
-   file://COPYINGv2;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
+LIC_FILES_CHKSUM = "\
+file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
+file://COPYING.LESSERv3;md5=6a6a8e020838b23406c81b19c1d46df6 \
+file://COPYINGv2;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
+file://COPYINGv3;md5=11cc2d3ee574f9d6b7ee797bdce4d423 \
 "
 
 REVISION = ""
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164256): 
https://lists.openembedded.org/g/openembedded-core/message/164256
Mute This Topic: https://lists.openembedded.org/mt/90413417/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 3/4] itstool: add missing COPYING.GPL3

[Konrad Weihmann]

to LIC_FILES_CHKSUM.
Format the list for better readability.
Remove useless line continuation from SRC_URI

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-support/itstool/itstool_2.0.7.bb | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-support/itstool/itstool_2.0.7.bb 
b/meta/recipes-support/itstool/itstool_2.0.7.bb
index 5af16995d7..7af231b1f2 100644
--- a/meta/recipes-support/itstool/itstool_2.0.7.bb
+++ b/meta/recipes-support/itstool/itstool_2.0.7.bb
@@ -5,14 +5,16 @@ XML files. It determines what to translate and how to chunk 
it into \
 messages using the W3C Internationalization Tag Set (ITS). "
 HOMEPAGE = "http://itstool.org/;
 LICENSE = "GPL-3.0-only"
-LIC_FILES_CHKSUM = "file://COPYING;md5=59c57b95fd7d0e9e238ebbc7ad47c5a5"
+LIC_FILES_CHKSUM = "\
+file://COPYING;md5=59c57b95fd7d0e9e238ebbc7ad47c5a5 \
+file://COPYING.GPL3;md5=d32239bcb673463ab874e80d47fae504 \
+"
 
 inherit autotools python3native
 
 DEPENDS = "libxml2-native"
 
-SRC_URI = "http://files.itstool.org/${BPN}/${BPN}-${PV}.tar.bz2 \
-   "
+SRC_URI = "http://files.itstool.org/${BPN}/${BPN}-${PV}.tar.bz2;
 SRC_URI:append:class-native = " 
file://0001-Native-Don-t-use-build-time-hardcoded-python-binary-.patch"
 SRC_URI:append:class-nativesdk = " 
file://0001-Native-Don-t-use-build-time-hardcoded-python-binary-.patch"
 SRC_URI:append:class-target = " 
file://0002-Don-t-use-build-time-hardcoded-python-binary-path.patch"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164257): 
https://lists.openembedded.org/g/openembedded-core/message/164257
Mute This Topic: https://lists.openembedded.org/mt/90413418/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 4/4] libcap: add pam_cap license

[Konrad Weihmann]

If libcap is compiled with pam in PACKAGECONFIG
additional one additional license text becomes effective,
add that as a conditional

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-support/libcap/libcap_2.63.bb | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-support/libcap/libcap_2.63.bb 
b/meta/recipes-support/libcap/libcap_2.63.bb
index f3133f9ded..aed1f0a510 100644
--- a/meta/recipes-support/libcap/libcap_2.63.bb
+++ b/meta/recipes-support/libcap/libcap_2.63.bb
@@ -5,7 +5,11 @@ users, without giving them full root permissions."
 HOMEPAGE = "http://sites.google.com/site/fullycapable/;
 # no specific GPL version required
 LICENSE = "BSD-3-Clause | GPL-2.0-only"
-LIC_FILES_CHKSUM = "file://License;md5=e2370ba375efe9e1a095c26d37e483b8"
+LIC_FILES_CHKSUM_PAM = 
"file://pam_cap/License;md5=0ad4c9c052b9719ee4fce1bfc7c7dee4"
+LIC_FILES_CHKSUM = "\
+   file://License;md5=e2370ba375efe9e1a095c26d37e483b8 \
+   ${@oe.utils.ifelse('pam' in (d.getVar('PACKAGECONFIG') or '').split(), 
'${LIC_FILES_CHKSUM_PAM}', '')} \
+"
 
 DEPENDS = "hostperl-runtime-native gperf-native"
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164258): 
https://lists.openembedded.org/g/openembedded-core/message/164258
Mute This Topic: https://lists.openembedded.org/mt/90413419/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] cve-check: get_cve_info should open the database read-only

[Konrad Weihmann]

On 21.03.22 14:43, Ross Burton wrote:

On Thu, 3 Mar 2022 at 11:43, Konrad Weihmann  wrote:

After scrolling through the log that I can get from a github pipeline
(sorry no runner log available) - I noticed

2022-03-03T05:39:23.2334189Z WARNING: cve-update-db-native-1.0-r0
do_fetch: Failed to fetch CVE data ([Errno 99] Cannot assign requested
address)
2022-03-03T05:39:23.2569941Z NOTE: recipe cve-update-db-native-1.0-r0:
task do_fetch: Succeeded


That's... special.


and then do_cve_check run on that recipe

2022-03-03T05:39:56.6239005Z NOTE: Running task 4428 of 5120
(/opt/build/sources/poky/meta/recipes-core/meta/cve-update-db-native.bb:do_cve_check)

before failing with 2022-03-03T05:39:56.8063851Z Exception:
sqlite3.OperationalError: attempt to write a readonly database

this doesn't look right to me - maybe there's something special about my
settings - I will try to recreate that locally


Have you managed to reproduce this failure since?


I tried, but without success - but I did a bit of reading on the 
reported Error 99, which seems to be related to using containers (in the 
pipeline itself a container based on ubuntu 18.04 is used) to run the build.


looking at the code block from cve-update-db-native:do_fetch

try:
response = urllib.request.urlopen(meta_url)
except urllib.error.URLError as e:
cve_f.write('Warning: CVE db update error, Unable to 
fetch CVE data.\n\n')

bb.warn("Failed to fetch CVE data (%s)" % e.reason)
return

I'm not a 100% sure, if the sudden return doesn't leave the 
pre-populated DB file in a limbo state.
I at least would expect a flush + close of the DB file here, or even 
better a recreation of an empty DB + flush and close - but I'm not sure 
what to do in the case the network stack of the OS fails us


But as I mentioned a couple of times, this issue is hard to reproduce 
and just hits me in on those mentioned GH pipelines every now and then.


BTW just one idea for improvement would be to globally exclude 
cve-update-db-native from running do_cve_check... as this doesn't make 
much sense to me - guess setting CVE_CHECK_SKIP_RECIPE = 
"cve-update-db-native" as a sane default in cve-check class should do it 
- any thoughts?




Ross

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#163511): 
https://lists.openembedded.org/g/openembedded-core/message/163511
Mute This Topic: https://lists.openembedded.org/mt/89339702/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] python3-installer: add installer module

[Konrad Weihmann]

On 10.03.22 19:25, Ross Burton wrote:


On Thu, 10 Mar 2022 at 17:36, Konrad Weihmann <mailto:kweihm...@outlook.com>> wrote:


Sorry to say that - but to me (even though it's more work) pip seems to
be the better option - the proposed tool is ~8 months old and not part
of pypa community as it seems - so in comparison to pip this could not
be labeled "battle proven".


It’s not that unheard of, for example the flit_core bootstrap 
documentation says to use it:


https://flit.readthedocs.io/en/latest/bootstrap.html 
<https://flit.readthedocs.io/en/latest/bootstrap.html>


It also does one job and just one job, which is A Very Good Thing.


Still doesn't make it the best choice TBH.
Just diving into the code makes me wonder about a lot, like

- can this tool run on non arm/aarch64/x86/x86-64 hosts 
(https://github.com/pradyunsg/installer/blob/fad2894a572d5497a3dceec58407c276f21e8c11/src/installer/utils.py#L140) 
- in its full feature set? - the assumptions made by this function, 
makes me think, it doesn't
- support of this here https://github.com/pradyunsg/installer/issues/98, 
should be given by a matured tool




Especially as the second patch of the series removes the possibility to
use the tooling proposed by python upstream for installing stuff.


Do you mean Pip here? That’s one option.  Installing a wheel is a 
glorified unzip, pip brings a lot of baggage that we don’t care about.


I should make it clear that this class is not for installing arbitrary 
wheels, it installs a wheel we just built and in the future will build 
the wheel too.


Then that's definitely missing in the commit message, that this should 
be an option and not the default




If one would want to have that kind of tooling the switch from pure
setup.py to toml and friends could have been done already a year ago
(python-build was the originally proposed tool iirc) - so this feels to
me like a step in the wrong direction (esp. the part that this would
rely on a tool **not** supported by upstream)


Adding support for build is next on the list.

Ross



Anyway, I think in before this patch series the switch to wheels was 
good enough to work with it, now it's just introducing another tool - 
that doesn't feel right to me, but I will stop arguing about that

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#163038): 
https://lists.openembedded.org/g/openembedded-core/message/163038
Mute This Topic: https://lists.openembedded.org/mt/89691492/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] python3-installer: add installer module

[Konrad Weihmann]

Sorry to say that - but to me (even though it's more work) pip seems to 
be the better option - the proposed tool is ~8 months old and not part 
of pypa community as it seems - so in comparison to pip this could not 
be labeled "battle proven".


Especially as the second patch of the series removes the possibility to 
use the tooling proposed by python upstream for installing stuff.


If one would want to have that kind of tooling the switch from pure 
setup.py to toml and friends could have been done already a year ago 
(python-build was the originally proposed tool iirc) - so this feels to 
me like a step in the wrong direction (esp. the part that this would 
rely on a tool **not** supported by upstream)


On 10.03.22 18:16, Ross Burton wrote:

Add a recipe for Installer, a minimal library/tool to install Python
Wheels.  Unlike PIP, it explicitly only installs wheels and does nothing
else.

Signed-off-by: Ross Burton 
---
  .../python/python3-installer_0.5.0.bb | 20 +++
  1 file changed, 20 insertions(+)
  create mode 100644 meta/recipes-devtools/python/python3-installer_0.5.0.bb

diff --git a/meta/recipes-devtools/python/python3-installer_0.5.0.bb 
b/meta/recipes-devtools/python/python3-installer_0.5.0.bb
new file mode 100644
index 00..bde9397569
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-installer_0.5.0.bb
@@ -0,0 +1,20 @@
+SUMMARY = "A library for installing Python wheels"
+DESCRIPTION = "a low-level library for installing a Python package from a wheel 
distribution."
+HOMEPAGE = "https://installer.readthedocs.io/;
+BUGTRACKER = "https://github.com/pradyunsg/installer/issues;
+
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=5038641aec7a77451e31da828ebfae00"
+
+SRC_URI[sha256sum] = 
"0cd6bdab3b358cf7e8749370b99aef9e12202751271c5ddb22126599b34dc665"
+
+inherit pypi flit_core
+
+DEPENDS:remove:class-native = "python3-installer-native"
+DEPENDS:append:class-native = " unzip-native"
+
+do_install:class-native () {
+pip_install_wheel_do_bootstrap_install
+}
+
+BBCLASSEXTEND = "native nativesdk"






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#163034): 
https://lists.openembedded.org/g/openembedded-core/message/163034
Mute This Topic: https://lists.openembedded.org/mt/89691492/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/3] pip_install_wheel: add a generic do_install for bootstrapping

[Konrad Weihmann]

On 10.03.22 12:40, Ross Burton wrote:

Several recipes are duplicating the same bootstrap logic for installing
a wheel without using any tools.  Add an implementation to
pip_install_wheel to centralise the code, and remove the duplicated code
from the following recipes:

- python3-flit-core
- python3-pip
- python3-setuptools
- python3-wheel

Signed-off-by: Ross Burton 
---
  meta/classes/pip_install_wheel.bbclass| 6 ++
  meta/recipes-devtools/python/python3-flit-core_3.7.1.bb   | 3 +--
  meta/recipes-devtools/python/python3-pip_22.0.3.bb| 4 +---
  meta/recipes-devtools/python/python3-setuptools_59.5.0.bb | 4 +---
  meta/recipes-devtools/python/python3-wheel_0.37.1.bb  | 4 +---
  5 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/meta/classes/pip_install_wheel.bbclass 
b/meta/classes/pip_install_wheel.bbclass
index 3861dae75a..29cd544aa3 100644
--- a/meta/classes/pip_install_wheel.bbclass
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -40,4 +40,10 @@ pip_install_wheel_do_install () {
  done
  }
  
+# A manual do_install that just uses unzip for bootstrapping purposes. Callers should DEPEND on unzip-native.


Minor nitpick - wouldn't it make sense to add a hard DEPENDS on 
unzip-native right here into the class - it should do no harm when done 
like DEPENDS:append:class-native = " unzip-native"



+pip_install_wheel_do_bootstrap_install () {
+install -d ${D}${PYTHON_SITEPACKAGES_DIR}
+unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl
+}


I think these two lines could be moved into a generic do_install:native 
- and then removing all the override code from the recipes below.


Or is there some special case I'm not seeing here that would require 
having this extra function.



Thanks for the patches - after all these rounds code looks good and 
should provide a seamless transition for most of the layers



+
  EXPORT_FUNCTIONS do_install
diff --git a/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb 
b/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb
index 196e391a8d..1f14345d50 100644
--- a/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb
+++ b/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb
@@ -21,8 +21,7 @@ PYPI_PACKAGE = "flit"
  PEP517_SOURCE_PATH = "${S}/flit_core"
  
  do_install:class-native () {

-install -d ${D}${PYTHON_SITEPACKAGES_DIR}
-unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} 
${PIP_INSTALL_DIST_PATH}/flit_core*.whl
+pip_install_wheel_do_bootstrap_install
  }
  
  PACKAGES =+ "${PN}-tests"

diff --git a/meta/recipes-devtools/python/python3-pip_22.0.3.bb 
b/meta/recipes-devtools/python/python3-pip_22.0.3.bb
index d7000b891a..f7fcf47c06 100644
--- a/meta/recipes-devtools/python/python3-pip_22.0.3.bb
+++ b/meta/recipes-devtools/python/python3-pip_22.0.3.bb
@@ -19,9 +19,7 @@ SRC_URI += "file://reproducible.patch"
  SRC_URI[sha256sum] = 
"f29d589df8c8ab99c060e68ad294c4a9ed896624f6368c5349d70aa581b333d0"
  
  do_install:class-native() {

-# Bootstrap to prevent dependency loop in python3-pip-native
-install -d ${D}${PYTHON_SITEPACKAGES_DIR}
-unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl
+pip_install_wheel_do_bootstrap_install
  
  # pip install would normally generate [console_scripts] in ${bindir}

  install -d ${D}/${bindir}
diff --git a/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb 
b/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb
index 9d2c2fd744..7cd67b85f9 100644
--- a/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb
+++ b/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb
@@ -22,9 +22,7 @@ DEPENDS:remove:class-native = "python3-pip-native 
python3-setuptools-native"
  DEPENDS:append:class-native = " unzip-native"
  
  do_install:class-native() {

-# Bootstrap to prevent dependency loop in python3-pip-native
-install -d ${D}${PYTHON_SITEPACKAGES_DIR}
-unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl
+pip_install_wheel_do_bootstrap_install
  }
  
  RDEPENDS:${PN} = "\

diff --git a/meta/recipes-devtools/python/python3-wheel_0.37.1.bb 
b/meta/recipes-devtools/python/python3-wheel_0.37.1.bb
index 83a7c986a8..cf0d8191b4 100644
--- a/meta/recipes-devtools/python/python3-wheel_0.37.1.bb
+++ b/meta/recipes-devtools/python/python3-wheel_0.37.1.bb
@@ -13,9 +13,7 @@ SRC_URI += " 
file://0001-Backport-pyproject.toml-from-flit-backend-branch.patch"
  DEPENDS:remove:class-native = "python3-pip-native"
  
  do_install:class-native () {

-# We need to bootstrap python3-wheel-native
-install -d ${D}${PYTHON_SITEPACKAGES_DIR}
-unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl
+pip_install_wheel_do_bootstrap_install
  
  # pip install would normally generate [project.scripts] in ${bindir}

  install -d ${D}/${bindir}






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online 

Re: [OE-core] [PATCH] pip_install_wheel: install wheel with a glob

[Konrad Weihmann]

On 09.03.22 16:53, Ross Burton wrote:

On Wed, 9 Mar 2022 at 15:47, Konrad Weihmann  wrote:

Can we, for the very unlikely case that do have more than one wheel in
this dir, bbfatal out here as well - otherwise I'm worried that this
will never get noticed


I'm undecided on this.  In the unlikely case (impossible, by design)
that a number of wheels are built for a single package, wouldn't we
want to install all of them as we're building the package?


Classic assertion case IMO - as you said there shouldn't more than one, 
but in the event of having more than one, I would just terminate the 
build, as likely something is very wrong (like people playing around if 
task appends or custom tasks in between compile and install).


I'm just concerned that this here would create a blind spot, where there 
none needed




Ross

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162968): 
https://lists.openembedded.org/g/openembedded-core/message/162968
Mute This Topic: https://lists.openembedded.org/mt/89664344/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] pip_install_wheel: install wheel with a glob

[Konrad Weihmann]

On 09.03.22 16:02, Ross Burton wrote:

Now that the build systems that use pip_install_wheel are all building
their wheel into a directory that we knew was empty before, we can just
install *.whl and not need to know the precise names.

By design a pyproject.toml will always build a single wheel, so there
shouldn't be any way for this to end up installing more than expected.

This obsoletes PIP_INSTALL_PACKAGE and PYPA_WHEEL, neither of which are
needed anymore.

Signed-off-by: Ross Burton 
---
  meta/classes/pip_install_wheel.bbclass | 14 +++---
  1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/meta/classes/pip_install_wheel.bbclass 
b/meta/classes/pip_install_wheel.bbclass
index 954a6b750f..5d09e795d0 100644
--- a/meta/classes/pip_install_wheel.bbclass
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -1,20 +1,10 @@
  DEPENDS:append = " python3-pip-native"
  
-def guess_pip_install_package_name(d):

-import re
-'''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
-name = d.getVar('PYPI_PACKAGE') or re.sub(r"^python3-", "", 
d.getVar('BPN'))
-return name.replace('-', '_')
-
-PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
-
  # The directory where wheels should be written too. Build classes
  # will ideally [cleandirs] this but we don't do that here in case
  # a recipe wants to install prebuilt wheels.
  PIP_INSTALL_DIST_PATH ?= "${WORKDIR}/dist"
  
-PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-*-*.whl"

-
  PIP_INSTALL_ARGS = "\
  - \
  --ignore-installed \
@@ -29,7 +19,9 @@ PIP_INSTALL_PYTHON = "python3"
  PIP_INSTALL_PYTHON:class-native = "nativepython3"
  
  pip_install_wheel_do_install () {

-nativepython3 -m pip install ${PIP_INSTALL_ARGS} ${PYPA_WHEEL} ||
+test -f ${PIP_INSTALL_DIST_PATH}/*.whl || bbfatal No wheels in 
${PIP_INSTALL_DIST_PATH}


Can we, for the very unlikely case that do have more than one wheel in 
this dir, bbfatal out here as well - otherwise I'm worried that this 
will never get noticed



+
+nativepython3 -m pip install ${PIP_INSTALL_ARGS} 
${PIP_INSTALL_DIST_PATH}/*.whl ||
bbfatal_log "Failed to pip install wheel. Check the logs."
  
  cd ${D}







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162965): 
https://lists.openembedded.org/g/openembedded-core/message/162965
Mute This Topic: https://lists.openembedded.org/mt/89664344/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/4] pip_install_wheel: don't lazy assign PIPINSTALLARGS

[Konrad Weihmann]

On 07.03.22 22:17, Ross Burton wrote:
Personally I can’t see any reason to extend the pip install arguments. 
What option do you think would be used by a recipe?


For instance the --user option is something that I used once in a setup.
Plus --global-option for passing additional options into the 
installation scripting.
I give you that, that these are used rarely - and still could be used, 
if the recipes would be tweaked to append after the inherit statement.


And in my mind it was always like a variable with a weak assignment 
could be changed by the user, the ones with a hard assignment shouldn't 
be touched - but that could be just my mind 




Ross

On Mon, 7 Mar 2022 at 20:06, Konrad Weihmann <mailto:kweihm...@outlook.com>> wrote:



On 07.03.22 20:42, Ross Burton wrote:
 > If we expect users to extend this we should use =, as otherwise a
recipe
 > that does += will replace the default value.
 >
 > Signed-off-by: Ross Burton mailto:ross.bur...@arm.com>>
 > ---
 >   meta/classes/pip_install_wheel.bbclass | 2 +-
 >   1 file changed, 1 insertion(+), 1 deletion(-)
 >
 > diff --git a/meta/classes/pip_install_wheel.bbclass
b/meta/classes/pip_install_wheel.bbclass
 > index c1680a24ed..febcc8e445 100644
 > --- a/meta/classes/pip_install_wheel.bbclass
 > +++ b/meta/classes/pip_install_wheel.bbclass
 > @@ -10,7 +10,7 @@ PIP_INSTALL_PACKAGE ?=
"${@guess_pip_install_package_name(d)}"
 >   PIP_INSTALL_DIST_PATH ?= "${@d.getVar('SETUPTOOLS_SETUP_PATH')
or d.getVar('B')}/dist"
 >   PYPA_WHEEL ??=
"${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-*-*.whl"
 >
 > -PIP_INSTALL_ARGS ?= "\
 > +PIP_INSTALL_ARGS = "\
 >       - \
 >       --ignore-installed \
 >       --no-cache \
 >
 >

Hmm all of the classes use ?= and I think for a reason - this isn't the
fault of the bbclass but of the recipe in the end, as += is known to
cause this depending on where the additions have been added (before the
inherit or after it) - I think the better way would be to use
PIP_INSTALL_ARGS:append in the actual recipe - guess the fallout would
be way less than the one I expect from this change.

The big question is if that should changeable by the user or not - I
personally think it should be

 >
 > 
 >



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162874): 
https://lists.openembedded.org/g/openembedded-core/message/162874
Mute This Topic: https://lists.openembedded.org/mt/89621292/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/4] pip_install_wheel: don't lazy assign PIPINSTALLARGS

[Konrad Weihmann]

On 07.03.22 20:42, Ross Burton wrote:

If we expect users to extend this we should use =, as otherwise a recipe
that does += will replace the default value.

Signed-off-by: Ross Burton 
---
  meta/classes/pip_install_wheel.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/pip_install_wheel.bbclass 
b/meta/classes/pip_install_wheel.bbclass
index c1680a24ed..febcc8e445 100644
--- a/meta/classes/pip_install_wheel.bbclass
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -10,7 +10,7 @@ PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
  PIP_INSTALL_DIST_PATH ?= "${@d.getVar('SETUPTOOLS_SETUP_PATH') or 
d.getVar('B')}/dist"
  PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-*-*.whl"
  
-PIP_INSTALL_ARGS ?= "\

+PIP_INSTALL_ARGS = "\
  - \
  --ignore-installed \
  --no-cache \




Hmm all of the classes use ?= and I think for a reason - this isn't the 
fault of the bbclass but of the recipe in the end, as += is known to 
cause this depending on where the additions have been added (before the 
inherit or after it) - I think the better way would be to use 
PIP_INSTALL_ARGS:append in the actual recipe - guess the fallout would 
be way less than the one I expect from this change.


The big question is if that should changeable by the user or not - I 
personally think it should be







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162864): 
https://lists.openembedded.org/g/openembedded-core/message/162864
Mute This Topic: https://lists.openembedded.org/mt/89621292/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] python3-native: Remove all pyc files

[Konrad Weihmann]

On 07.03.22 10:45, Richard Purdie wrote:

On Mon, 2022-03-07 at 08:36 +0100, Konrad Weihmann wrote:


On 04.03.22 00:46, Richard Purdie wrote:

On Thu, 2022-03-03 at 15:28 -0800, Tim Orling wrote:



On Thu, Mar 3, 2022 at 9:16 AM Richard Purdie
 wrote:

On Thu, 2022-03-03 at 17:14 +, Ross Burton wrote:

On Thu, 3 Mar 2022 at 16:34, Richard Purdie
 wrote:

This removes a further 1600 files from sstate handling and lets python
create the ones it wants at runtime which is likely much better overall
for performance.


Playing devil's advocate: doesn't having them in sstate mean they're
generated once and hardlinked, instead of needing to be generated for
every recipe which runs pythonnative?

Whilst I can't disagree that 1600 files being dropped from sstate is
good, we're just punting the recompile step to every recipe when it
runs python code.

I guess the question here is how long does the Python library take to

recompile.




At runtime it should only generate pyc for modules actually used?


Yes.


Another consideration is that there are many sysroots pulling in python3-
native
which don't run python and they're only there as there are python scripts
being
added which means python has to come too.

I suspect for that reason it could be a net win but it is a tough call.



I tend to be in favor of the reduced number of files moving around. I suppose
we could do some timing comparisons if we really care. It will be a trade-off
of network/io vs compute/io, correct?


Yes. I suspect the compile cost is tiny compared to the io/network cost of
chucking so many small files around when I suspect they're not used that often.

Cheers,

Richard


Now as this is in master and works reasonably well, could we extent the
removal of pyc for native recipes to *all* python packages.
I'm still seeing occasional staging issues on random native python
recipes as described in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13868.
And as the common understanding here seems to be that the additional
compile costs are negligible compared to the staging costs of additional
files, I would propose to add

do_install:append:class-native() {
find ${D} -name *.pyc -delete
}

to setuptools3.bbclass (as this looks like the one piece of code that
almost all python packages have in common)

Any thoughts?


I don't really like working around errors like the above bug where we haven't
understood the underlying issue :/

I suspect those pyc files are nowhere near as large an issue as the ones in the
python recipe itself either, there are likely only small numbers of them.


I can come up with a setup that has ~21k pyc files of different native 
python site packages - and due to the nature of python's import system I 
guess only a very very small chunk of these will be actually needed to 
be recompiled.


I know that bug is hard to reproduce - I can only manage that (but there 
with a quite good percentage) in a GitHub actions pipeline (2vCPUs, 7GB 
RAM, ~12GB HDD) - and I had it once that could reproduce it locally. And 
the pyc files, even though they should have been there, disappeared 
while hardlinking the files from the components dir to the workspace.


So like with the other issue I reported last week, my question is what 
information do you need to look at these things? - a browseable 
workspace is likely out of scope, but certain files and system infos 
could be gathered I guess.




Cheers,

Richard


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162818): 
https://lists.openembedded.org/g/openembedded-core/message/162818
Mute This Topic: https://lists.openembedded.org/mt/89528765/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] python3-native: Remove all pyc files

[Konrad Weihmann]

On 04.03.22 00:46, Richard Purdie wrote:

On Thu, 2022-03-03 at 15:28 -0800, Tim Orling wrote:



On Thu, Mar 3, 2022 at 9:16 AM Richard Purdie
 wrote:

On Thu, 2022-03-03 at 17:14 +, Ross Burton wrote:

On Thu, 3 Mar 2022 at 16:34, Richard Purdie
 wrote:

This removes a further 1600 files from sstate handling and lets python
create the ones it wants at runtime which is likely much better overall
for performance.


Playing devil's advocate: doesn't having them in sstate mean they're
generated once and hardlinked, instead of needing to be generated for
every recipe which runs pythonnative?

Whilst I can't disagree that 1600 files being dropped from sstate is
good, we're just punting the recompile step to every recipe when it
runs python code.

I guess the question here is how long does the Python library take to

recompile.




At runtime it should only generate pyc for modules actually used?


Yes.


Another consideration is that there are many sysroots pulling in python3-
native
which don't run python and they're only there as there are python scripts
being
added which means python has to come too.

I suspect for that reason it could be a net win but it is a tough call.



I tend to be in favor of the reduced number of files moving around. I suppose
we could do some timing comparisons if we really care. It will be a trade-off
of network/io vs compute/io, correct?


Yes. I suspect the compile cost is tiny compared to the io/network cost of
chucking so many small files around when I suspect they're not used that often.

Cheers,

Richard


Now as this is in master and works reasonably well, could we extent the 
removal of pyc for native recipes to *all* python packages.
I'm still seeing occasional staging issues on random native python 
recipes as described in 
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13868.
And as the common understanding here seems to be that the additional 
compile costs are negligible compared to the staging costs of additional 
files, I would propose to add


do_install:append:class-native() {
find ${D} -name *.pyc -delete
}

to setuptools3.bbclass (as this looks like the one piece of code that 
almost all python packages have in common)


Any thoughts?











-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162797): 
https://lists.openembedded.org/g/openembedded-core/message/162797
Mute This Topic: https://lists.openembedded.org/mt/89528765/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] setuptools_build_meta: remove python dependency

[Konrad Weihmann]

when building additional native python tools that should be used
for additional tasks in python3 package (class-target), the dependency
on target class python creates a dependency loop:

python3-foo-native_1.2.3.bb:do_prepare_recipe_sysroot
python3-pep8-native_1.7.1.bb:do_populate_sysroot
python3-wheel_0.37.1.bb:do_populate_sysroot
python3-bar-native_0.1.2.bb:do_populate_sysroot
python3-pip_22.0.3.bb:do_populate_sysroot
python3-pyflakes-native_2.4.0.bb:do_populate_sysroot
python3_3.10.2.bb:do_populate_sysroot
python3-foo-native_1.2.3_1.2.0.bb:do_fetch
python3-setuptools_59.5.0.bb:do_populate_sysroot

resolve this by removing python3 from DEPENDS.
The correct dependencies are already injected by
setuptools-base class

Signed-off-by: Konrad Weihmann 
---
 meta/classes/setuptools_build_meta.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/setuptools_build_meta.bbclass 
b/meta/classes/setuptools_build_meta.bbclass
index b1441e65dd..8f5479e922 100644
--- a/meta/classes/setuptools_build_meta.bbclass
+++ b/meta/classes/setuptools_build_meta.bbclass
@@ -1,6 +1,6 @@
 inherit pip_install_wheel setuptools3-base
 
-DEPENDS += "python3 python3-setuptools-native python3-wheel-native"
+DEPENDS += "python3-setuptools-native python3-wheel-native"
 
 setuptools_build_meta_do_configure () {
 mkdir -p ${S}/dist
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162779): 
https://lists.openembedded.org/g/openembedded-core/message/162779
Mute This Topic: https://lists.openembedded.org/mt/89568035/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] classes: add setuptools3_legacy

[Konrad Weihmann]

On 4 Mar 2022 21:14, Khem Raj  wrote:On Fri, Mar 4, 2022 at 11:20 AM Konrad Weihmann  wrote:

>

> If it's about meta-python why can't be placed there? I don't follow the argument that some edge case stuff outside of core requires the project to have additional (as of now not time limited) maintenance effort - if there would be a clear plan to remove that before kirkstone release I might agree, but with the release looming this all feels like abandoning the commitment for the upcoming release.



it is about many layers e.g. meta-oe, meta-xfce, meta-python and

perhaps many more that I have not come across yet. Then please provide a list what is failing, just so people, such as myself, can see the necessity of this change - as of now I didn't see any technically sound proof that this needs to be in coreWe want to enable a

wider ecosystem and we

do carry features in OE-core to do so from time to time. Core in

itself is a reference distribution, its not meant to carry just what

it needs to support a minimum distro

it serves as a baseline for many distros which may include other

layers. At times we do put specific features in specific layers, this

is not one of those.



>

> And just to emphasize my claim, if there's nothing in core that would require this patch it won't get tested, so it's another dead end.

>

> In the end it's the decision of the project leads, but I would be personally be disappointed to take the easy way out here.

>

> On 4 Mar 2022 19:01, Otavio Salvador  wrote:

>

>

>

> Em sex., 4 de mar. de 2022 às 13:56, Khem Raj  escreveu:

>

> There are recipes in meta-oe and meta-python which would need this

> and both layer only list core as sole dependency, if we put it in either of

> these layers, then one will depend on other which is not nice, so I rather

> would prefer it in core.

>

>

> Couldn't those recipes (depending on Python) go to meta-python?

>

> --

> Otavio Salvador O.S. Systems

> http://www.ossystems.com.br    http://code.ossystems.com.br

> Mobile: +55 (53) 9 9981-7854  Mobile: +1 (347) 903-9750

>

>



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162759): 
https://lists.openembedded.org/g/openembedded-core/message/162759
Mute This Topic: https://lists.openembedded.org/mt/89547022/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] classes: add setuptools3_legacy

[Konrad Weihmann]

If it's about meta-python why can't be placed there? I don't follow the argument that some edge case stuff outside of core requires the project to have additional (as of now not time limited) maintenance effort - if there would be a clear plan to remove that before kirkstone release I might agree, but with the release looming this all feels like abandoning the commitment for the upcoming release.And just to emphasize my claim, if there's nothing in core that would require this patch it won't get tested, so it's another dead end.In the end it's the decision of the project leads, but I would be personally be disappointed to take the easy way out here.On 4 Mar 2022 19:01, Otavio Salvador  wrote:Em sex., 4 de mar. de 2022 às 13:56, Khem Raj  escreveu:There are recipes in meta-oe and meta-python which would need this
and both layer only list core as sole dependency, if we put it in either of
these layers, then one will depend on other which is not nice, so I rather
would prefer it in core.Couldn't those recipes (depending on Python) go to meta-python?  -- Otavio Salvador                             O.S. Systemshttp://www.ossystems.com.br        http://code.ossystems.com.brMobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162755): 
https://lists.openembedded.org/g/openembedded-core/message/162755
Mute This Topic: https://lists.openembedded.org/mt/89547022/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] classes: add setuptools3_legacy

[Konrad Weihmann]

On 04.03.22 14:13, Ross Burton wrote:

On Fri, 4 Mar 2022 at 12:44, Konrad Weihmann  wrote:

For me that would be a candidate to be added to meta-python, or are
there any core recipes that currently break because of the new behavior?


This is restoring existing behaviour, so I believe it should be in
core.  There's going to be enough recipes using this that it is
justified in core. I expect we'll have deleted this by the time the
next LTS comes around too.


Furthermore the variables names are the same, so if people accidentally
(or due to complex injection trees/classes/distro settings/whatever have
new setuptools3 and this class here in the same recipe, things become
rather unpredictable - as of now there are the same but surely they will
start to diverge at one point.


That's intentional: if you have a recipe which needs the legacy
behaviour, just change the import from setuptool3 to
setuptools3_legacy.


Can you name any publicly available recipe that requires that?

For me this is the same situation as with the python2 > python3 
transition - there wasn't any fallback provided in core, but outside of 
core it was - so I would propose to do the same here.


If that legacy class will be part of core it's likely that it will 
remain till EOL of kirkstone which is somewhat in ~2025 (by current 
planning) - and meta-python2 showed that even then some recipes still 
haven't migrated (yes, chromium looking at you).
All in all I think core should just provide *one* way to do it while the 
legacy cases can easily live outside in a separate layer (meta-python 
for instance as the "old" distutils have been put there as well).


In additional everyone was worried so much about reputation lately that 
doing this long announced transition, while keeping the "old" way 
enabled (and creating more maintenance effort in the end), would 
definitely send the wrong signal to the community.





While we are at it pypi class needs a fallback too, because for me this
is the standard way of packaging python recipes and new class points to
the new setuptools implementation - so if we want to have this here, we
might need a fallback for pypi class as well


My understanding is that almost anything on pypi is already a pure
Python module and is mostly unaffected by the changes.

Ross

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162718): 
https://lists.openembedded.org/g/openembedded-core/message/162718
Mute This Topic: https://lists.openembedded.org/mt/89547022/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] classes: add setuptools3_legacy

[Konrad Weihmann]

For me that would be a candidate to be added to meta-python, or are 
there any core recipes that currently break because of the new behavior?


Furthermore the variables names are the same, so if people accidentally 
(or due to complex injection trees/classes/distro settings/whatever have 
new setuptools3 and this class here in the same recipe, things become 
rather unpredictable - as of now there are the same but surely they will 
start to diverge at one point.


While we are at it pypi class needs a fallback too, because for me this 
is the standard way of packaging python recipes and new class points to 
the new setuptools implementation - so if we want to have this here, we 
might need a fallback for pypi class as well



On 04.03.22 12:55, Ross Burton wrote:

Following a good discussion with PyPA upstream[1] the migration of the
setuptools3.bbclass to use bdist_wheel+pip turns out to be more complex
than thought.

Essentially, we're midway through a lot of changes: the future of Python
packaging is wheels and pip, but those by design are not as flexible as
traditional distutils and setup.py.

Specifically, with traditional distutils the package can implement its
own install task and write arbitrary files (such as init scripts).  With
wheels this is explicity impossible, so packages that do this cannot use
the new setuptools class and must continue to use the build/install tasks
as before.

This class is the old setuptools behaviour, bought back. However, as
distutils and the setuptools install task are both deprecated and will
soon be removed entirely, any users of this class should be moving to an
alternative build tool, be it a modern Python tool which works with
wheels, or a non-Pythonic tool such as Meson.

[1] https://github.com/pypa/packaging-problems/issues/576

Signed-off-by: Ross Burton 
---
  meta/classes/setuptools3_legacy.bbclass | 78 +
  1 file changed, 78 insertions(+)
  create mode 100644 meta/classes/setuptools3_legacy.bbclass

diff --git a/meta/classes/setuptools3_legacy.bbclass 
b/meta/classes/setuptools3_legacy.bbclass
new file mode 100644
index 00..5a99daadb5
--- /dev/null
+++ b/meta/classes/setuptools3_legacy.bbclass
@@ -0,0 +1,78 @@
+# This class is for packages which use the deprecated setuptools behaviour,
+# specifically custom install tasks which don't work correctly with 
bdist_wheel.
+# This behaviour is deprecated in setuptools[1] and won't work in the future, 
so
+# all users of this should consider their options: pure Python modules can use 
a
+# modern Python tool such as build[2], or packages which are doing more (such 
as
+# installing init scripts) should use a fully-featured build system such as 
Meson.
+#
+# [1] https://setuptools.pypa.io/en/latest/history.html#id142
+# [2] https://pypi.org/project/build/
+
+inherit setuptools3-base
+
+B = "${WORKDIR}/build"
+
+SETUPTOOLS_BUILD_ARGS ?= ""
+SETUPTOOLS_INSTALL_ARGS ?= "--root=${D} \
+--prefix=${prefix} \
+--install-lib=${PYTHON_SITEPACKAGES_DIR} \
+--install-data=${datadir}"
+
+SETUPTOOLS_PYTHON = "python3"
+SETUPTOOLS_PYTHON:class-native = "nativepython3"
+
+SETUPTOOLS_SETUP_PATH ?= "${S}"
+
+setuptools3_legacy_do_configure() {
+:
+}
+
+setuptools3_legacy_do_compile() {
+cd ${SETUPTOOLS_SETUP_PATH}
+NO_FETCH_BUILD=1 \
+STAGING_INCDIR=${STAGING_INCDIR} \
+STAGING_LIBDIR=${STAGING_LIBDIR} \
+${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py \
+build --build-base=${B} ${SETUPTOOLS_BUILD_ARGS} || \
+bbfatal_log "'${PYTHON_PN} setup.py build ${SETUPTOOLS_BUILD_ARGS}' 
execution failed."
+}
+setuptools3_legacy_do_compile[vardepsexclude] = "MACHINE"
+
+setuptools3_legacy_do_install() {
+cd ${SETUPTOOLS_SETUP_PATH}
+install -d ${D}${PYTHON_SITEPACKAGES_DIR}
+STAGING_INCDIR=${STAGING_INCDIR} \
+STAGING_LIBDIR=${STAGING_LIBDIR} \
+PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \
+${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py \
+build --build-base=${B} install --skip-build 
${SETUPTOOLS_INSTALL_ARGS} || \
+bbfatal_log "'${PYTHON_PN} setup.py install ${SETUPTOOLS_INSTALL_ARGS}' 
execution failed."
+
+# support filenames with *spaces*
+find ${D} -name "*.py" -exec grep -q ${D} {} \; \
+   -exec sed -i -e s:${D}::g {} \;
+
+for i in ${D}${bindir}/* ${D}${sbindir}/*; do
+if [ -f "$i" ]; then
+sed -i -e s:${PYTHON}:${USRBINPATH}/env\ 
${SETUPTOOLS_PYTHON}:g $i
+sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
+fi
+done
+
+rm -f ${D}${PYTHON_SITEPACKAGES_DIR}/easy-install.pth
+
+#
+# FIXME: Bandaid against wrong datadir computation
+#
+if [ -e ${D}${datadir}/share ]; then
+mv -f ${D}${datadir}/share/* ${D}${datadir}/
+rmdir ${D}${datadir}/share
+

Re: [OE-core] [PATCH 1/2] python3-native: Remove all pyc files

[Konrad Weihmann]

On 03.03.22 18:11, Konrad Weihmann wrote:



On 03.03.22 17:34, Richard Purdie wrote:

This removes a further 1600 files from sstate handling and lets python
create the ones it wants at runtime which is likely much better overall
for performance.

Signed-off-by: Richard Purdie 
---
  meta/recipes-devtools/python/python3_3.10.2.bb | 7 ++-
  1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3_3.10.2.bb 
b/meta/recipes-devtools/python/python3_3.10.2.bb

index b28aa6505a0..7ec443a531f 100644
--- a/meta/recipes-devtools/python/python3_3.10.2.bb
+++ b/meta/recipes-devtools/python/python3_3.10.2.bb
@@ -156,7 +156,12 @@ do_install:append:class-native() {
  # Remove the opt-1.pyc and opt-2.pyc files. There are over 
3,000 of them
  # and the overhead in each recipe-sysroot-native isn't worth 
it, particularly

  # when they're only used for python called with -O or -OO.
-    find ${D} -name *opt-*.pyc -delete
+    #find ${D} -name *opt-*.pyc -delete


This here looks like a leftover - guess that could be removed.

Thx for finally making that happen (I've done the same for quite a while 
in an extension in my setup) - I remember there was a bug about pyc 
leading to weird staging issues, I opened up like a year or two.

Guess that one could be closed when this is merged


For reference: it's this bug here 
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13868




+    # Remove all pyc files. There are a ton of them and it is 
probably faster to let
+    # python create the ones it wants at runtime rather than 
manage in the sstate

+    # tarballs and sysroot creation.
+    find ${D} -name *.pyc -delete
+
  }
  do_install:append() {











-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162667): 
https://lists.openembedded.org/g/openembedded-core/message/162667
Mute This Topic: https://lists.openembedded.org/mt/89528765/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] python3-native: Remove all pyc files

[Konrad Weihmann]

On 03.03.22 17:34, Richard Purdie wrote:

This removes a further 1600 files from sstate handling and lets python
create the ones it wants at runtime which is likely much better overall
for performance.

Signed-off-by: Richard Purdie 
---
  meta/recipes-devtools/python/python3_3.10.2.bb | 7 ++-
  1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3_3.10.2.bb 
b/meta/recipes-devtools/python/python3_3.10.2.bb
index b28aa6505a0..7ec443a531f 100644
--- a/meta/recipes-devtools/python/python3_3.10.2.bb
+++ b/meta/recipes-devtools/python/python3_3.10.2.bb
@@ -156,7 +156,12 @@ do_install:append:class-native() {
  # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of 
them
  # and the overhead in each recipe-sysroot-native isn't worth it, 
particularly
  # when they're only used for python called with -O or -OO.
-find ${D} -name *opt-*.pyc -delete
+#find ${D} -name *opt-*.pyc -delete


This here looks like a leftover - guess that could be removed.

Thx for finally making that happen (I've done the same for quite a while 
in an extension in my setup) - I remember there was a bug about pyc 
leading to weird staging issues, I opened up like a year or two.

Guess that one could be closed when this is merged


+# Remove all pyc files. There are a ton of them and it is probably 
faster to let
+# python create the ones it wants at runtime rather than manage in the 
sstate
+# tarballs and sysroot creation.
+find ${D} -name *.pyc -delete
+
  }
  
  do_install:append() {







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162666): 
https://lists.openembedded.org/g/openembedded-core/message/162666
Mute This Topic: https://lists.openembedded.org/mt/89528765/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] cve-check: get_cve_info should open the database read-only

[Konrad Weihmann]

This one is odd...

After scrolling through the log that I can get from a github pipeline 
(sorry no runner log available) - I noticed


2022-03-03T05:39:23.2334189Z WARNING: cve-update-db-native-1.0-r0 
do_fetch: Failed to fetch CVE data ([Errno 99] Cannot assign requested 
address)
2022-03-03T05:39:23.2569941Z NOTE: recipe cve-update-db-native-1.0-r0: 
task do_fetch: Succeeded


and then do_cve_check run on that recipe

2022-03-03T05:39:56.6239005Z NOTE: Running task 4428 of 5120 
(/opt/build/sources/poky/meta/recipes-core/meta/cve-update-db-native.bb:do_cve_check)


before failing with 2022-03-03T05:39:56.8063851Z Exception: 
sqlite3.OperationalError: attempt to write a readonly database


this doesn't look right to me - maybe there's something special about my 
settings - I will try to recreate that locally


On 03.03.22 12:27, Ross Burton wrote:

:tableflip:

Thanks, I guess!

Can you share the full bitbake log from that run? I'd like to see what
jobs are running in parallel.

Ross

On Thu, 3 Mar 2022 at 11:10, Konrad Weihmann  wrote:


Single from scratch build of core-image-minimal

On 03.03.22 12:07, Ross Burton wrote:

What's the scenario where this is happening?  Multiple build
directories using the same DL_DIR? Or was this a single build?

Ross

On Thu, 3 Mar 2022 at 08:10, Konrad Weihmann  wrote:


Sadly I encountered this here last night

NOTE: recipe cve-update-db-native-1.0-r0: task do_cve_check: Started
ERROR: cve-update-db-native-1.0-r0 do_cve_check: Error executing a
python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: 
0001:
*** 0002:do_cve_check(d)
0003:
File: '/opt/build/sources/poky/meta/classes/cve-check.bbclass', lineno:
104, function: do_cve_check
0100:try:
0101:patched_cves = get_patched_cves(d)
0102:except FileNotFoundError:
0103:bb.fatal("Failure in searching patches")
*** 0104:whitelisted, patched, unpatched = check_cves(d,
patched_cves)
0105:if patched or unpatched:
0106:cve_data = get_cve_info(d, patched + unpatched)
0107:cve_write_data(d, patched, unpatched,
whitelisted, cve_data)
0108:else:
File: '/opt/build/sources/poky/meta/classes/cve-check.bbclass', lineno:
198, function: check_cves
0194:else:
0195:vendor = "%"
0196:
0197:# Find all relevant CVE IDs.
*** 0198:for cverow in conn.execute("SELECT DISTINCT ID FROM
PRODUCTS WHERE PRODUCT IS ? AND VENDOR LIKE ?", (product, vendor)):
0199:cve = cverow[0]
0200:
0201:if cve in cve_whitelist:
0202:bb.note("%s-%s has been whitelisted for %s" %
(product, pv, cve))
Exception: sqlite3.OperationalError: attempt to write a readonly database

So it seems your patch doesn't solve the issue



On 23.02.22 13:54, Ross Burton wrote:

All of the function in cve-check should open the database read-only, as
the only writer is the fetch task in cve-update-db.  However,
get_cve_info() was failing to do this, which might be causing locking
issues with sqlite.

Signed-off-by: Ross Burton 
---
meta/classes/cve-check.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 2d69aeba4b..d715fbf4d8 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -265,7 +265,8 @@ def get_cve_info(d, cves):
import sqlite3

cve_data = {}
-conn = sqlite3.connect(d.getVar("CVE_CHECK_DB_FILE"))
+db_file = d.expand("file:${CVE_CHECK_DB_FILE}?mode=ro")
+conn = sqlite3.connect(db_file, uri=True)

for cve in cves:
for row in conn.execute("SELECT * FROM NVD WHERE ID IS ?", (cve,)):






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162653): 
https://lists.openembedded.org/g/openembedded-core/message/162653
Mute This Topic: https://lists.openembedded.org/mt/89339702/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] cve-check: get_cve_info should open the database read-only

[Konrad Weihmann]

Single from scratch build of core-image-minimal

On 03.03.22 12:07, Ross Burton wrote:

What's the scenario where this is happening?  Multiple build
directories using the same DL_DIR? Or was this a single build?

Ross

On Thu, 3 Mar 2022 at 08:10, Konrad Weihmann  wrote:


Sadly I encountered this here last night

NOTE: recipe cve-update-db-native-1.0-r0: task do_cve_check: Started
ERROR: cve-update-db-native-1.0-r0 do_cve_check: Error executing a
python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: 
   0001:
   *** 0002:do_cve_check(d)
   0003:
File: '/opt/build/sources/poky/meta/classes/cve-check.bbclass', lineno:
104, function: do_cve_check
   0100:try:
   0101:patched_cves = get_patched_cves(d)
   0102:except FileNotFoundError:
   0103:bb.fatal("Failure in searching patches")
   *** 0104:whitelisted, patched, unpatched = check_cves(d,
patched_cves)
   0105:if patched or unpatched:
   0106:cve_data = get_cve_info(d, patched + unpatched)
   0107:cve_write_data(d, patched, unpatched,
whitelisted, cve_data)
   0108:else:
File: '/opt/build/sources/poky/meta/classes/cve-check.bbclass', lineno:
198, function: check_cves
   0194:else:
   0195:vendor = "%"
   0196:
   0197:# Find all relevant CVE IDs.
   *** 0198:for cverow in conn.execute("SELECT DISTINCT ID FROM
PRODUCTS WHERE PRODUCT IS ? AND VENDOR LIKE ?", (product, vendor)):
   0199:cve = cverow[0]
   0200:
   0201:if cve in cve_whitelist:
   0202:bb.note("%s-%s has been whitelisted for %s" %
(product, pv, cve))
Exception: sqlite3.OperationalError: attempt to write a readonly database

So it seems your patch doesn't solve the issue



On 23.02.22 13:54, Ross Burton wrote:

All of the function in cve-check should open the database read-only, as
the only writer is the fetch task in cve-update-db.  However,
get_cve_info() was failing to do this, which might be causing locking
issues with sqlite.

Signed-off-by: Ross Burton 
---
   meta/classes/cve-check.bbclass | 3 ++-
   1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 2d69aeba4b..d715fbf4d8 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -265,7 +265,8 @@ def get_cve_info(d, cves):
   import sqlite3

   cve_data = {}
-conn = sqlite3.connect(d.getVar("CVE_CHECK_DB_FILE"))
+db_file = d.expand("file:${CVE_CHECK_DB_FILE}?mode=ro")
+conn = sqlite3.connect(db_file, uri=True)

   for cve in cves:
   for row in conn.execute("SELECT * FROM NVD WHERE ID IS ?", (cve,)):






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162649): 
https://lists.openembedded.org/g/openembedded-core/message/162649
Mute This Topic: https://lists.openembedded.org/mt/89339702/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] cve-check: get_cve_info should open the database read-only

[Konrad Weihmann]

Sadly I encountered this here last night

NOTE: recipe cve-update-db-native-1.0-r0: task do_cve_check: Started
ERROR: cve-update-db-native-1.0-r0 do_cve_check: Error executing a 
python function in exec_func_python() autogenerated:


The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: 
 0001:
 *** 0002:do_cve_check(d)
 0003:
File: '/opt/build/sources/poky/meta/classes/cve-check.bbclass', lineno: 
104, function: do_cve_check

 0100:try:
 0101:patched_cves = get_patched_cves(d)
 0102:except FileNotFoundError:
 0103:bb.fatal("Failure in searching patches")
 *** 0104:whitelisted, patched, unpatched = check_cves(d, 
patched_cves)

 0105:if patched or unpatched:
 0106:cve_data = get_cve_info(d, patched + unpatched)
 0107:cve_write_data(d, patched, unpatched, 
whitelisted, cve_data)

 0108:else:
File: '/opt/build/sources/poky/meta/classes/cve-check.bbclass', lineno: 
198, function: check_cves

 0194:else:
 0195:vendor = "%"
 0196:
 0197:# Find all relevant CVE IDs.
 *** 0198:for cverow in conn.execute("SELECT DISTINCT ID FROM 
PRODUCTS WHERE PRODUCT IS ? AND VENDOR LIKE ?", (product, vendor)):

 0199:cve = cverow[0]
 0200:
 0201:if cve in cve_whitelist:
 0202:bb.note("%s-%s has been whitelisted for %s" % 
(product, pv, cve))

Exception: sqlite3.OperationalError: attempt to write a readonly database

So it seems your patch doesn't solve the issue



On 23.02.22 13:54, Ross Burton wrote:

All of the function in cve-check should open the database read-only, as
the only writer is the fetch task in cve-update-db.  However,
get_cve_info() was failing to do this, which might be causing locking
issues with sqlite.

Signed-off-by: Ross Burton 
---
  meta/classes/cve-check.bbclass | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 2d69aeba4b..d715fbf4d8 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -265,7 +265,8 @@ def get_cve_info(d, cves):
  import sqlite3
  
  cve_data = {}

-conn = sqlite3.connect(d.getVar("CVE_CHECK_DB_FILE"))
+db_file = d.expand("file:${CVE_CHECK_DB_FILE}?mode=ro")
+conn = sqlite3.connect(db_file, uri=True)
  
  for cve in cves:

  for row in conn.execute("SELECT * FROM NVD WHERE ID IS ?", (cve,)):






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162637): 
https://lists.openembedded.org/g/openembedded-core/message/162637
Mute This Topic: https://lists.openembedded.org/mt/89339702/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] staging: FileExistsError

[Konrad Weihmann]

On 03.03.22 00:05, Richard Purdie wrote:

On Wed, 2022-03-02 at 21:55 +0100, Konrad Weihmann wrote:

An interesting behavior struck me once more after a long long time.

   *** 0002:extend_recipe_sysroot(d)
   0003:
File: '/layer/layer/poky/meta/classes/staging.bbclass', lineno: 584,
function: extend_recipe_sysroot
   0580:if "/bin/" in l or "/sbin/" in l:
   0581:# defer /*bin/* files until last in
case they need libs
   0582:binfiles[l] = (targetdir, dest)
   0583:else:
   *** 0584:staging_copyfile(l, targetdir, dest,
postinsts, seendirs)
   0585:
   0586:# Handle deferred binfiles
   0587:for l in binfiles:
   0588:(targetdir, dest) = binfiles[l]
File: '/layer/layer/poky/meta/classes/staging.bbclass', lineno: 158,
function: staging_copyfile
   0154:os.symlink(linkto, dest)
   0155:#bb.warn(c)
   0156:else:
   0157:try:
   *** 0158:os.link(c, dest)
   0159:except OSError as err:
   0160:if err.errno == errno.EXDEV:
   0161:bb.utils.copyfile(c, dest)
   0162:else:
Exception: FileExistsError: [Errno 17] File exists:
'/build/tmp/sysroots-components/x86_64/some/path/to/some/deeply/nested/file'
->
'/build/tmp/work/x86_64-linux/some-recipe/1.0-r0/recipe-sysroot-native/some/path/to/some/deeply/nested/file'

This happens in rare case when doing a differential build o a recipe
with hundreds or thousands of native dependencies.

The file mentioned is pretty much random and it's unique (no duplicates
among all the dependencies when it comes to filename+path).

So far I helped myself with ignoring this error by adding

  except FileExistsError:
  pass

to staging - I wonder if anyone else ever seen this behavior and what's
the best way to deal with it

What comes to my mind is

- ignoring it - but then no one can be sure it's the right file
- falling back to the OSError/EXDEV path and copying over the file requested
- a 3rd option

Any thoughts?

(Just to be sure, this happens like once in a thousand builds and only
on differential builds)


My initial thought is that is a horrible work around and we need to get to the
real issue.

The question is where is the duplicate file coming from and why.

Also, is this dunfell or master or something else? There were issues that were
fixed in these codepaths after dunfell.

Is there anything significant about the kinds of files this is being hit with.
For example are they generated pyc files? We have see that issue before. Is
there anything significant about the path lengths involved? It is hard to tell
from the information provided.


To shed some light on it - no it's current master (head of yesterday) 
and it's no regarding some pyc, as I normally remove them by a different 
class in my builds.
As I said, the error hits on pretty random file, one time it's a code 
file, one time it a LICENSE file - I don't see any pattern in there.


And I agree one need to get to the bottom of it, which indeed difficult 
as this hits me like once in every 3-4 months and usually a sstate clean 
fixes it for me.


TBH I have no idea how to get into this state, but once you're in there 
it can be reproduced with a 100% certainty.


The only thing I see is, that the more build time dependencies one 
injects the more likely it's going to hit.
In today incident I rebuild npm-native, which cause the rebuild of like 
~5000 dependencies to be setup for a particular workspace.
Anyway just to assure you it's not about npm, I encountered the very 
same on a recipe that had like ~250 python dependencies.


My question is, what should I do, next time this hits me.
What kind of trace and/or info you would need to analyze this, as I have 
admit this is beyond my understanding of bitbake/OE




Cheers,

Richard






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162630): 
https://lists.openembedded.org/g/openembedded-core/message/162630
Mute This Topic: https://lists.openembedded.org/mt/89511363/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] staging: FileExistsError

[Konrad Weihmann]

An interesting behavior struck me once more after a long long time.

 *** 0002:extend_recipe_sysroot(d)
 0003:
File: '/layer/layer/poky/meta/classes/staging.bbclass', lineno: 584, 
function: extend_recipe_sysroot

 0580:if "/bin/" in l or "/sbin/" in l:
 0581:# defer /*bin/* files until last in 
case they need libs

 0582:binfiles[l] = (targetdir, dest)
 0583:else:
 *** 0584:staging_copyfile(l, targetdir, dest, 
postinsts, seendirs)

 0585:
 0586:# Handle deferred binfiles
 0587:for l in binfiles:
 0588:(targetdir, dest) = binfiles[l]
File: '/layer/layer/poky/meta/classes/staging.bbclass', lineno: 158, 
function: staging_copyfile

 0154:os.symlink(linkto, dest)
 0155:#bb.warn(c)
 0156:else:
 0157:try:
 *** 0158:os.link(c, dest)
 0159:except OSError as err:
 0160:if err.errno == errno.EXDEV:
 0161:bb.utils.copyfile(c, dest)
 0162:else:
Exception: FileExistsError: [Errno 17] File exists: 
'/build/tmp/sysroots-components/x86_64/some/path/to/some/deeply/nested/file' 
-> 
'/build/tmp/work/x86_64-linux/some-recipe/1.0-r0/recipe-sysroot-native/some/path/to/some/deeply/nested/file'


This happens in rare case when doing a differential build o a recipe 
with hundreds or thousands of native dependencies.


The file mentioned is pretty much random and it's unique (no duplicates 
among all the dependencies when it comes to filename+path).


So far I helped myself with ignoring this error by adding

except FileExistsError:
pass

to staging - I wonder if anyone else ever seen this behavior and what's 
the best way to deal with it


What comes to my mind is

- ignoring it - but then no one can be sure it's the right file
- falling back to the OSError/EXDEV path and copying over the file requested
- a 3rd option

Any thoughts?

(Just to be sure, this happens like once in a thousand builds and only 
on differential builds)

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162624): 
https://lists.openembedded.org/g/openembedded-core/message/162624
Mute This Topic: https://lists.openembedded.org/mt/89511363/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling

[Konrad Weihmann]

On 02.03.22 20:04, Khem Raj wrote:

On Wed, Mar 2, 2022 at 11:01 AM Konrad Weihmann  wrote:




On 02.03.22 19:45, Khem Raj wrote:

v4 is definitely better than v3, I see one case it could address as well e.g.

python3-pymetno produces PyMetno-0.9.0-py3-none-any.whl, so if
guessing code could address the case-sensitive piece here would be
good.


I'm torn on this one, I agree that it would really nice to have in terms
of a seamless transition, but it would require either some bash/shell
hacks to disable case sensitive filename globs or some "after compile"
renaming - I'm not sure how to tackle this.
If everyone else agrees I would vote for "after compile" renaming to a
fully lower case filename - any thoughts?



I think the suggestion that Ross has is a good one to install the wheel file
intentionally to a known location using --dist-dir and then use a more
relaxed regexp to package it.


As I mentioned in the v4 patch dist-dir doesn't work for me - somehow it 
isn't respected by most of the python modules.
So my idea would be it remain on default "dist" directory, but run 
something line rename 'y/A-Z/a-z/' ${PYPA_WHEEL} after compile - also 
minding that in PIP_INSTALL_PACKAGE.


I'm happy to see dist-dir/bdist-dir work, but so far I failed to see any 
successful run on my setup (Ubuntu 20.04 btw)






On Wed, Mar 2, 2022 at 1:21 AM Konrad Weihmann  wrote:


v4 is out.
I tried to make it work with the dist-dir/bdist-dir option, but
apparently this isn't respected by setuptools in all of the tested recipes.
That's why I went with applying cleandirs on
${SETUPTOOLS_SETUP_PATH}/dist, that should do the trick as well.

v4 is tested against all core python recipes, all the python recipes in
my layers and a reasonable subset from meta-oe

On 02.03.22 09:06, Konrad Weihmann wrote:

My bad - one of the brackets in the name guessing slipped...
Will send a v4 soon

On 02.03.22 06:20, Khem Raj wrote:

this is causing 100+ packages to fail see

https://errors.yoctoproject.org/Errors/Build/142116/

I suggest to include meta-python for wider testing of such changes.

On Tue, Mar 1, 2022 at 12:14 PM Ross Burton  wrote:


On Tue, 1 Mar 2022 at 18:48, Khem Raj  wrote:

where the wheel file is called
pytest_runner-5.3.1-py3-none-any.whl


I'm thinking we should tell pip to install the wheels to a directory
we control and then just install *.whl.

IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
it's clean on builds.

Ross






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162623): 
https://lists.openembedded.org/g/openembedded-core/message/162623
Mute This Topic: https://lists.openembedded.org/mt/89472354/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling

[Konrad Weihmann]

On 02.03.22 19:45, Khem Raj wrote:

v4 is definitely better than v3, I see one case it could address as well e.g.

python3-pymetno produces PyMetno-0.9.0-py3-none-any.whl, so if
guessing code could address the case-sensitive piece here would be
good.


I'm torn on this one, I agree that it would really nice to have in terms 
of a seamless transition, but it would require either some bash/shell 
hacks to disable case sensitive filename globs or some "after compile" 
renaming - I'm not sure how to tackle this.
If everyone else agrees I would vote for "after compile" renaming to a 
fully lower case filename - any thoughts?




On Wed, Mar 2, 2022 at 1:21 AM Konrad Weihmann  wrote:


v4 is out.
I tried to make it work with the dist-dir/bdist-dir option, but
apparently this isn't respected by setuptools in all of the tested recipes.
That's why I went with applying cleandirs on
${SETUPTOOLS_SETUP_PATH}/dist, that should do the trick as well.

v4 is tested against all core python recipes, all the python recipes in
my layers and a reasonable subset from meta-oe

On 02.03.22 09:06, Konrad Weihmann wrote:

My bad - one of the brackets in the name guessing slipped...
Will send a v4 soon

On 02.03.22 06:20, Khem Raj wrote:

this is causing 100+ packages to fail see

https://errors.yoctoproject.org/Errors/Build/142116/

I suggest to include meta-python for wider testing of such changes.

On Tue, Mar 1, 2022 at 12:14 PM Ross Burton  wrote:


On Tue, 1 Mar 2022 at 18:48, Khem Raj  wrote:

where the wheel file is called
pytest_runner-5.3.1-py3-none-any.whl


I'm thinking we should tell pip to install the wheels to a directory
we control and then just install *.whl.

IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
it's clean on builds.

Ross






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162620): 
https://lists.openembedded.org/g/openembedded-core/message/162620
Mute This Topic: https://lists.openembedded.org/mt/89472354/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling

[Konrad Weihmann]

v4 is out.
I tried to make it work with the dist-dir/bdist-dir option, but 
apparently this isn't respected by setuptools in all of the tested recipes.
That's why I went with applying cleandirs on 
${SETUPTOOLS_SETUP_PATH}/dist, that should do the trick as well.


v4 is tested against all core python recipes, all the python recipes in 
my layers and a reasonable subset from meta-oe


On 02.03.22 09:06, Konrad Weihmann wrote:

My bad - one of the brackets in the name guessing slipped...
Will send a v4 soon

On 02.03.22 06:20, Khem Raj wrote:

this is causing 100+ packages to fail see

https://errors.yoctoproject.org/Errors/Build/142116/

I suggest to include meta-python for wider testing of such changes.

On Tue, Mar 1, 2022 at 12:14 PM Ross Burton  wrote:


On Tue, 1 Mar 2022 at 18:48, Khem Raj  wrote:

where the wheel file is called
pytest_runner-5.3.1-py3-none-any.whl


I'm thinking we should tell pip to install the wheels to a directory
we control and then just install *.whl.

IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
it's clean on builds.

Ross






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162601): 
https://lists.openembedded.org/g/openembedded-core/message/162601
Mute This Topic: https://lists.openembedded.org/mt/89472354/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4] pip_install_wheel: improve wheel handling

[Konrad Weihmann]

- replace python3 prefix when guessing the wheel name
  as there are still plenty of recipes out there that do use
  python3 prefixes
- remove all previously generated wheels matching the glob
  to avoid installing any outdated blob via cleandirs
  in setuptools3 class.
  Unfortunetaly proposed dist-dir or bdist-dir are not
  respected by setuptools, likely due because they
  are overridable by the setup script
- don't use PV in glob, as PV doesn't necessarily align with the
  version used inside of the setuptools configuration.
  this will avoid having the user set PYPA_WHEEL in a lot
  of recipes
- respect SETUPTOOLS_SETUP_PATH in PIP_INSTALL_DIST_PATH
  and use B as a fallback only (in case this class is inherited
  without setuptools3 class being there as well).
  recipes like python3-smbus run in a subfolder of the
  workspace and were failing in before this adjustment

Signed-off-by: Konrad Weihmann 
---
v4: restructure name guessing to make wrongly placed brackets
being spottable more easily.
fix wrongly placed bracket to make name guessing work again
with pypi_package settings.
remove compile:prepend and use setuptools:do_compile cleandirs
option (see commit message for further explanation)
v3: remove in prepend to compile, not in install for obvious reasons
v2: fix python3 prefix string

 meta/classes/pip_install_wheel.bbclass | 8 +---
 meta/classes/setuptools3.bbclass   | 1 +
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/meta/classes/pip_install_wheel.bbclass 
b/meta/classes/pip_install_wheel.bbclass
index 5b7e5cd706..3beff685bb 100644
--- a/meta/classes/pip_install_wheel.bbclass
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -1,12 +1,14 @@
 DEPENDS:append = " python3-pip-native"
 
 def guess_pip_install_package_name(d):
+import re
 '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
-return (d.getVar('PYPI_PACKAGE') or d.getVar('BPN')).replace('-', '_')
+name = d.getVar('PYPI_PACKAGE') or re.sub(r"^python3-", "", 
d.getVar('BPN'))
+return name.replace('-', '_')
 
 PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
-PIP_INSTALL_DIST_PATH ?= "${B}/dist"
-PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"
+PIP_INSTALL_DIST_PATH ?= "${@d.getVar('SETUPTOOLS_SETUP_PATH') or 
d.getVar('B')}/dist"
+PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-*-*.whl"
 
 PIP_INSTALL_ARGS ?= "\
 - \
diff --git a/meta/classes/setuptools3.bbclass b/meta/classes/setuptools3.bbclass
index 12561340b0..564996c556 100644
--- a/meta/classes/setuptools3.bbclass
+++ b/meta/classes/setuptools3.bbclass
@@ -28,6 +28,7 @@ setuptools3_do_compile() {
 bbfatal_log "'${PYTHON_PN} setup.py bdist_wheel 
${SETUPTOOLS_BUILD_ARGS}' execution failed."
 }
 setuptools3_do_compile[vardepsexclude] = "MACHINE"
+do_compile[cleandirs] += "${SETUPTOOLS_SETUP_PATH}/dist"
 
 setuptools3_do_install() {
 cd ${SETUPTOOLS_SETUP_PATH}
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162600): 
https://lists.openembedded.org/g/openembedded-core/message/162600
Mute This Topic: https://lists.openembedded.org/mt/89497642/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v3] pip_install_wheel: improve wheel handling

[Konrad Weihmann]

My bad - one of the brackets in the name guessing slipped...
Will send a v4 soon

On 02.03.22 06:20, Khem Raj wrote:

this is causing 100+ packages to fail see

https://errors.yoctoproject.org/Errors/Build/142116/

I suggest to include meta-python for wider testing of such changes.

On Tue, Mar 1, 2022 at 12:14 PM Ross Burton  wrote:


On Tue, 1 Mar 2022 at 18:48, Khem Raj  wrote:

where the wheel file is called
pytest_runner-5.3.1-py3-none-any.whl


I'm thinking we should tell pip to install the wheels to a directory
we control and then just install *.whl.

IIRC the option was --dist-dir, and we can [cleandirs] that to ensure
it's clean on builds.

Ross

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162595): 
https://lists.openembedded.org/g/openembedded-core/message/162595
Mute This Topic: https://lists.openembedded.org/mt/89472354/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3] pip_install_wheel: improve wheel handling

[Konrad Weihmann]

- replace python3 prefix when guessing the wheel name
  as there are still plenty of recipes out there that do use
  python3 prefixes
- remove all previously generated wheels matching the glob
  to avoid installing any outdated blob
- don't use PV in glob, as PV doesn't necessarily align with the
  version used inside of the setuptools configuration.
  this will avoid having the user set PYPA_WHEEL in a lot
  of recipes
- respect SETUPTOOLS_SETUP_PATH in PIP_INSTALL_DIST_PATH
  and use B as a fallback only (in case this class is inherited
  without setuptools3 class being there as well).
  recipes like python3-smbus run in a subfolder of the
  workspace and were failing in before this adjustment

Signed-off-by: Konrad Weihmann 
---
v3: remove in prepend to compile, not in install for obvious reasons
v2: fix python3 prefix string

 meta/classes/pip_install_wheel.bbclass | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/meta/classes/pip_install_wheel.bbclass 
b/meta/classes/pip_install_wheel.bbclass
index 5b7e5cd706..789c25b68c 100644
--- a/meta/classes/pip_install_wheel.bbclass
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -1,12 +1,13 @@
 DEPENDS:append = " python3-pip-native"
 
 def guess_pip_install_package_name(d):
+import re
 '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
-return (d.getVar('PYPI_PACKAGE') or d.getVar('BPN')).replace('-', '_')
+return (d.getVar('PYPI_PACKAGE') or re.sub(r"^python3-", "", 
d.getVar('BPN')).replace('-', '_'))
 
 PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
-PIP_INSTALL_DIST_PATH ?= "${B}/dist"
-PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"
+PIP_INSTALL_DIST_PATH ?= "${@d.getVar('SETUPTOOLS_SETUP_PATH') or 
d.getVar('B')}/dist"
+PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-*-*.whl"
 
 PIP_INSTALL_ARGS ?= "\
 - \
@@ -45,4 +46,9 @@ pip_install_wheel_do_install () {
 done
 }
 
+do_compile:prepend() {
+# cleanup previously generated files
+rm -f ${PYPA_WHEEL}
+}
+
 EXPORT_FUNCTIONS do_install
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162541): 
https://lists.openembedded.org/g/openembedded-core/message/162541
Mute This Topic: https://lists.openembedded.org/mt/89472354/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [meta-oe][PATCH v2] pip_install_wheel: improve wheel handling

[Konrad Weihmann]

- replace python3 prefix when guessing the wheel name
  as there are still plenty of recipes out there that do use
  python3 prefixes
- remove all previously generated wheels matching the glob
  to avoid installing any outdated blob
- don't use PV in glob, as PV doesn't necessarily align with the
  version used inside of the setuptools configuration.
  this will avoid having the user set PYPA_WHEEL in a lot
  of recipes
- respect SETUPTOOLS_SETUP_PATH in PIP_INSTALL_DIST_PATH
  and use B as a fallback only (in case this class is inherited
  without setuptools3 class being there as well).
  recipes like python3-smbus run in a subfolder of the
  workspace and were failing in before this adjustment

Signed-off-by: Konrad Weihmann 
---
v2: fix type in python3 prefix replacement

 meta/classes/pip_install_wheel.bbclass | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/meta/classes/pip_install_wheel.bbclass 
b/meta/classes/pip_install_wheel.bbclass
index 5b7e5cd706..1b6db91d6f 100644
--- a/meta/classes/pip_install_wheel.bbclass
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -1,12 +1,13 @@
 DEPENDS:append = " python3-pip-native"
 
 def guess_pip_install_package_name(d):
+import re
 '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
-return (d.getVar('PYPI_PACKAGE') or d.getVar('BPN')).replace('-', '_')
+return (d.getVar('PYPI_PACKAGE') or re.sub(r"^python3-", "", 
d.getVar('BPN')).replace('-', '_'))
 
 PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
-PIP_INSTALL_DIST_PATH ?= "${B}/dist"
-PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"
+PIP_INSTALL_DIST_PATH ?= "${@d.getVar('SETUPTOOLS_SETUP_PATH') or 
d.getVar('B')}/dist"
+PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-*-*.whl"
 
 PIP_INSTALL_ARGS ?= "\
 - \
@@ -28,6 +29,8 @@ PIP_INSTALL_PYTHON = "python3"
 PIP_INSTALL_PYTHON:class-native = "nativepython3"
 
 pip_install_wheel_do_install () {
+# cleanup previously generated files
+rm -f ${PYPA_WHEEL}
 nativepython3 -m pip install ${PIP_INSTALL_ARGS} ${PYPA_WHEEL} ||
 bbfatal_log "Failed to pip install wheel. Check the logs."
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162529): 
https://lists.openembedded.org/g/openembedded-core/message/162529
Mute This Topic: https://lists.openembedded.org/mt/89471480/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] pip_install_wheel: improve wheel handling

[Konrad Weihmann]

- replace python3 prefix when guessing the wheel name
  as there are still plenty of recipes out there that do use
  python3 prefixes
- remove all previously generated wheels matching the glob
  to avoid installing any outdated blob
- don't use PV in glob, as PV doesn't necessarily align with the
  version used inside of the setuptools configuration.
  this will avoid having the user set PYPA_WHEEL in a lot
  of recipes
- respect SETUPTOOLS_SETUP_PATH in PIP_INSTALL_DIST_PATH
  and use B as a fallback only (in case this class is inherited
  without setuptools3 class being there as well).
  recipes like python3-smbus run in a subfolder of the
  workspace and were failing in before this adjustment

Signed-off-by: Konrad Weihmann 
---
 meta/classes/pip_install_wheel.bbclass | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/meta/classes/pip_install_wheel.bbclass 
b/meta/classes/pip_install_wheel.bbclass
index 5b7e5cd706..e1fc25cbeb 100644
--- a/meta/classes/pip_install_wheel.bbclass
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -1,12 +1,13 @@
 DEPENDS:append = " python3-pip-native"
 
 def guess_pip_install_package_name(d):
+import re
 '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
-return (d.getVar('PYPI_PACKAGE') or d.getVar('BPN')).replace('-', '_')
+return (d.getVar('PYPI_PACKAGE') or re.sub(r"^python-3", "", 
d.getVar('BPN')).replace('-', '_'))
 
 PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
-PIP_INSTALL_DIST_PATH ?= "${B}/dist"
-PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"
+PIP_INSTALL_DIST_PATH ?= "${@d.getVar('SETUPTOOLS_SETUP_PATH') or 
d.getVar('B')}/dist"
+PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-*-*.whl"
 
 PIP_INSTALL_ARGS ?= "\
 - \
@@ -28,6 +29,8 @@ PIP_INSTALL_PYTHON = "python3"
 PIP_INSTALL_PYTHON:class-native = "nativepython3"
 
 pip_install_wheel_do_install () {
+# cleanup previously generated files
+rm -f ${PYPA_WHEEL}
 nativepython3 -m pip install ${PIP_INSTALL_ARGS} ${PYPA_WHEEL} ||
 bbfatal_log "Failed to pip install wheel. Check the logs."
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162528): 
https://lists.openembedded.org/g/openembedded-core/message/162528
Mute This Topic: https://lists.openembedded.org/mt/89471458/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] pip_install_wheel: Use BPN instead of PN to construct PYPI_PACKAGE default

[Konrad Weihmann]

Some more observations when using the current state of the pip/wheels 
patch series (from today's master)


- when using recipes that inherit setuptools (but not pypi) and do have 
a "python3-..." prefix in the recipe name - the name guessing fails. IMO 
it would be best to strip the "python3-" prefix in the guessing too


- when using recipes that do not have a version matching *exactly* the 
one from setup.py|pyproject.toml the wheel glob `PYPA_WHEEL ??= 
"${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"` fails

 - examples:
- PV = "1.2.3+${SRCREV}" for recipes that have additional patches on it
- recipe name is python3-foo_git.bb

In my opinion it doesn't make sense to have ${PV} in that glob - a 
simple asterisk would match all of them easily - or this there any real 
life example where a single python project builds several packages with 
different versions in the same workspace?


- another thing is SETUPTOOLS_INSTALL_ARGS ?= "--root=${D} 
--prefix=${prefix}" breaks a recipe that was previously installing files 
to "/etc" aka ${sysconfdir} - these are now installed into /usr/etc, 
which doesn't look right. For reference the project is 
https://github.com/HewlettPackard/reconbf


In a sense of a more painless transition these issues should be 
addressed before any release


On 26.02.22 09:21, Richard Purdie wrote:

This fixes the name for native and nativesdk recipes.

Signed-off-by: Richard Purdie 
---
  meta/classes/pip_install_wheel.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/pip_install_wheel.bbclass 
b/meta/classes/pip_install_wheel.bbclass
index 9f9feda6ee0..5b7e5cd7064 100644
--- a/meta/classes/pip_install_wheel.bbclass
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -2,7 +2,7 @@ DEPENDS:append = " python3-pip-native"
  
  def guess_pip_install_package_name(d):

  '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
-return (d.getVar('PYPI_PACKAGE') or d.getVar('PN')).replace('-', '_')
+return (d.getVar('PYPI_PACKAGE') or d.getVar('BPN')).replace('-', '_')
  
  PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"

  PIP_INSTALL_DIST_PATH ?= "${B}/dist"






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162432): 
https://lists.openembedded.org/g/openembedded-core/message/162432
Mute This Topic: https://lists.openembedded.org/mt/89406689/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2 2/2] pip_install_wheel: improved wheel filename guess

[Konrad Weihmann]

On 25.02.22 05:03, Tim Orling wrote:

From: Tim Orling 

Rather than only use PYPI_PACKAGE as a guess, fall back on PN for cases
where a recipe does not inherit pypi.

Wheels can only have alphanumeric characters in the 'distribution'
name [1]. Any other characters are replaced with an underscore. Provide a
function to replace dash with underscore.

[1] https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode

Signed-off-by: Tim Orling 
---
Changes in v2:
address review comments

  meta/classes/pip_install_wheel.bbclass | 6 +-
  1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/classes/pip_install_wheel.bbclass 
b/meta/classes/pip_install_wheel.bbclass
index 8a848c0ebab..9f9feda6ee0 100644
--- a/meta/classes/pip_install_wheel.bbclass
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -1,6 +1,10 @@
  DEPENDS:append = " python3-pip-native"
  
-PIP_INSTALL_PACKAGE ?= "${PYPI_PACKAGE}"

+def guess_pip_install_package_name(d):
+'''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
+return (d.getVar('PYPI_PACKAGE') or d.getVar('PN')).replace('-', '_')


In my opinion this needs to be BPN not PN - as if you will build 
python3-foo-native (for native only recipes) it will result in 
foo_native-1.2.3-*.whl, that just doesn't exit.
What disappoints me the most is that none of this comes with unit tests 
- it broke for a bunch of stuff I maintain this morning, making my 
morning coffee taste bitter :-(



+
+PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
  PIP_INSTALL_DIST_PATH ?= "${B}/dist"
  PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"
  







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162422): 
https://lists.openembedded.org/g/openembedded-core/message/162422
Mute This Topic: https://lists.openembedded.org/mt/89382270/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v3 00/32] Python PEP-517: build wheels and install with pip

[Konrad Weihmann]

On 25.02.22 14:16, Richard Purdie wrote:

On Thu, 2022-02-24 at 16:52 +0100, Konrad Weihmann wrote:

I got a kind of general question about this patch series and all the
followups: is this still considered to go into the next release?


It is still being considered, yes.


I'm a bit worried about the fallout of this pretty invasive change -
even though I see that at some point it needs to be done.

My understanding is that the "classic" way will stop with Python 3.12,
which doesn't apply to next LTS release - as this will likely remain on
3.10.
Only downside will be that manual helper files for updates of packages
that are lacking a setup.py needs to be provided (there are already a
few examples how to do it) - not a big deal if you'd ask me.

So what's the stand of the project regarding this issue - also keeping
in mind that I think it's already past feature freeze?


This is a tough one to make a decision on and I am conflicted. The change was
flagged up a while ago and has been regularly talked about. It is also something
we all agree will have to happen at some point.

The change is late and has issues but there was a base patchset sent before the
freeze deadline.

This isn't the final release point, it is the point where we stop taking new
invasive changes and stabilise and I think it important to keep that in mind.

Stepping back and thinking about the big picture (and e.g. the ability to take
security fixes into the LTS), I'm leaning towards trying to get it in. One other
consideration is having large delta between the LTS and onging development and
I'd prefer to minimise this particular difference if it is practical to do so.


Your argumentation does make sense, but I have to disagree on this 
particular point.
The using pip as the default installer and therefore wheels is something 
that will never (hopefully) get backported, so bringing this change in 
automatically builds up a huge delta to any other branch - thus here you 
would create a situation that (I agree) should be avoided.


Also moving around a few classes and recipe between core and 
meta-python, will either bind users to including meta-python in every 
setup or will create situation were people will try to work around these 
changes.


I see that this feature has been promised - and it might be bad for the 
project's reputation to drop it - still if one would ask me, I would 
prefer to delay it to the next release.
One potential option would be to offer that one (once mature and tested 
will a broad set of layers) as a mixin-layer, which then could be used 
with kirkstone LTS




I believe we have identified and fixed the majority of the issues that have
shown up in automated testing.

I haven't made a final decision but I am keeping an open mind on it and would
really prefer to get it merged. There are other issues being worked in parallel
which also would block the M3 build which does give time to resolve this one.

Cheers,

Richard





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162345): 
https://lists.openembedded.org/g/openembedded-core/message/162345
Mute This Topic: https://lists.openembedded.org/mt/89324642/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v3 00/32] Python PEP-517: build wheels and install with pip

[Konrad Weihmann]

I got a kind of general question about this patch series and all the 
followups: is this still considered to go into the next release?


I'm a bit worried about the fallout of this pretty invasive change - 
even though I see that at some point it needs to be done.


My understanding is that the "classic" way will stop with Python 3.12, 
which doesn't apply to next LTS release - as this will likely remain on 
3.10.
Only downside will be that manual helper files for updates of packages 
that are lacking a setup.py needs to be provided (there are already a 
few examples how to do it) - not a big deal if you'd ask me.


So what's the stand of the project regarding this issue - also keeping 
in mind that I think it's already past feature freeze?

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162325): 
https://lists.openembedded.org/g/openembedded-core/message/162325
Mute This Topic: https://lists.openembedded.org/mt/89324642/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] systemd: move systemd shared library into its own package

[Konrad Weihmann]

On 23.02.22 16:59, Stefan Herbrechtsmeier wrote:

From: Stefan Herbrechtsmeier 

Move the systemd shared library (libsystemd-shared.so) into its own
package to prevent a runtime dependency from udev package to systemd
package and thereby to a second init manager.

Signed-off-by: Stefan Herbrechtsmeier 

---

  meta/recipes-core/systemd/systemd_250.3.bb | 5 +
  1 file changed, 5 insertions(+)

diff --git a/meta/recipes-core/systemd/systemd_250.3.bb 
b/meta/recipes-core/systemd/systemd_250.3.bb
index 9121333f56..43ac66da62 100644
--- a/meta/recipes-core/systemd/systemd_250.3.bb
+++ b/meta/recipes-core/systemd/systemd_250.3.bb
@@ -371,6 +371,7 @@ PACKAGE_BEFORE_PN = "\
  ${PN}-journal-remote \
  ${PN}-extra-utils \
  ${PN}-udev-rules \
+libsystemd-shared \
  udev \
  udev-hwdb \
  "
@@ -387,6 +388,8 @@ DESCRIPTION:${PN}-journal-upload = "systemd-journal-upload 
uploads journal entri
  SUMMARY:${PN}-journal-remote = "Receive journal messages over the network"
  DESCRIPTION:${PN}-journal-remote = "systemd-journal-remote is a command to receive 
serialized journal events and store them to journal files."
  
+SUMMARY:${PN}-container = "Systemd shared library"


That doesn't look right to me - shouldn't that be SUMMARY:libsystemd-shared?


+
  SYSTEMD_PACKAGES = "${@bb.utils.contains('PACKAGECONFIG', 'binfmt', 
'${PN}-binfmt', '', d)} \
  ${@bb.utils.contains('PACKAGECONFIG', 'microhttpd', 
'${PN}-journal-gatewayd', '', d)} \
  ${@bb.utils.contains('PACKAGECONFIG', 'microhttpd', 
'${PN}-journal-remote', '', d)} \
@@ -654,6 +657,8 @@ INSANE_SKIP:${PN} += "dev-so libdir"
  INSANE_SKIP:${PN}-dbg += "libdir"
  INSANE_SKIP:${PN}-doc += " libdir"
  
+FILES:libsystemd-shared = "${rootlibexecdir}/systemd/libsystemd-shared*.so"

+
  RPROVIDES:udev = "hotplug"
  
  RDEPENDS:udev-hwdb += "udev"







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162269): 
https://lists.openembedded.org/g/openembedded-core/message/162269
Mute This Topic: https://lists.openembedded.org/mt/89343939/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH][RFC] cve-check: use local copy instead of global db

[Konrad Weihmann]

instead of using a global DB (default in DL_DIR) copy an existing
DB file into WORKDIR and use it from there.
This should avoid running into the reported "database is readonly" error
while at the same time there's no need to arbritrarily limit the
cve_check task to just one run at a time

Signed-off-by: Konrad Weihmann 
---
This patch should be tested by users that run cve-check on a regular
on hosts with as much as possible cores, before merging.
In local testing I haven't found any issues on a world build,
but as mentioned in the previous patch the issue is kind of hard to
reproduce.
So this patch aims at lifting the arbitrary task lock, while preventing
access by more than one thread/process at a time by sqlite.
Feedback through heavy local testing is very much appreciated

 meta/classes/cve-check.bbclass | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 21d3da7974..e4389b7001 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -28,6 +28,7 @@ CVE_VERSION ??= "${PV}"
 CVE_CHECK_DB_DIR ?= "${DL_DIR}/CVE_CHECK"
 CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_1.1.db"
 CVE_CHECK_DB_FILE_LOCK ?= "${CVE_CHECK_DB_FILE}.lock"
+CVE_CHECK_DB_FILE_LOCAL = 
"${WORKDIR}/${@os.path.basename(d.getVar('CVE_CHECK_DB_FILE'))}"
 
 CVE_CHECK_LOG ?= "${T}/cve.log"
 CVE_CHECK_TMP_FILE ?= "${TMPDIR}/cve_check"
@@ -94,9 +95,11 @@ python do_cve_check () {
 """
 Check recipe for patched and unpatched CVEs
 """
+import bb.utils
 from oe.cve_check import get_patched_cves
 
 if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):
+bb.utils.copyfile(d.getVar("CVE_CHECK_DB_FILE"), 
d.getVar("CVE_CHECK_DB_FILE_LOCAL"))
 try:
 patched_cves = get_patched_cves(d)
 except FileNotFoundError:
@@ -111,7 +114,6 @@ python do_cve_check () {
 }
 
 addtask cve_check before do_build after do_fetch
-do_cve_check[lockfiles] += "${CVE_CHECK_DB_FILE_LOCK}"
 do_cve_check[depends] = "cve-update-db-native:do_fetch"
 do_cve_check[nostamp] = "1"
 
@@ -185,7 +187,7 @@ def check_cves(d, patched_cves):
 cve_whitelist = d.getVar("CVE_CHECK_WHITELIST").split()
 
 import sqlite3
-db_file = d.expand("file:${CVE_CHECK_DB_FILE}?mode=ro")
+db_file = d.expand("file:${CVE_CHECK_DB_FILE_LOCAL}?mode=ro")
 conn = sqlite3.connect(db_file, uri=True)
 
 # For each of the known product names (e.g. curl has CPEs using curl and 
libcurl)...
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162190): 
https://lists.openembedded.org/g/openembedded-core/message/162190
Mute This Topic: https://lists.openembedded.org/mt/89323890/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH][RFC] cve-check: add lockfile to task

[Konrad Weihmann]

On 22.02.22 18:50, Ross Burton wrote:

On Tue, 22 Feb 2022 at 17:01, Konrad Weihmann  wrote:

This is somehow expected from my side tbh - as the lock file disables
any kind of parallelism - so just one instance of cve-check-task can run
at a time.
One thing that came to my mind is to move the created CVE_CHECK-sqlite
db into the recipe-sysroot before running the task (currently it resides
in DL_DIR), so each task would run on its own copy of the db - that
would avoid having multiple access call to the same file while enabling
us to remove the arbitrary task lock.


Hang on - surely the cve_check tasks are read-only, so why do we need
to lock the database?

Ross


TBH I don't know - I (and Ralph confirmed that) see occasionally this 
error from the original patch.
As I mentioned I lack the experience with sqlite to get to the bottom of 
it - all I can say is that since the task lock had been merged I'm error 
free in all the pipelines I monitor

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162183): 
https://lists.openembedded.org/g/openembedded-core/message/162183
Mute This Topic: https://lists.openembedded.org/mt/88258164/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH][RFC] cve-check: add lockfile to task

[Konrad Weihmann]

On 22.02.22 16:50, Ralph Siemsen wrote:

Hi Konrad,

On Fri, Jan 7, 2022 at 4:59 AM Konrad Weihmann <mailto:kweihm...@outlook.com>> wrote:


On 07.01.22 10:48, Konrad Weihmann wrote:
 > this should prevent running into the very rare error
 > sqlite3.OperationalError: attempt to write a readonly database

It's also possible that check_same_thread (that defaults to True ) in
the sqlite3.connect causes this (see
https://docs.python.org/3/library/sqlite3.html#sqlite3.connect
<https://docs.python.org/3/library/sqlite3.html#sqlite3.connect>) -
I have
too little experience with sqlite to provide any further input.

I'd be happy to get some feedback from users that do have that
experience with sqlite in a highly multithreaded/multiprocess
environment. Or by people that use cvecheck class in larger setups than
just core.


Afraid I am not well versed in the internals of sqlite, so I cannot 
comment on that part.


However, I do run cve-check on a larger setup than core, and after this 
commit recently landed in dunfell branch, I noticed that do_cve_check 
step is taking considerably longer than it used to. I have not 
benchmarked it carefully, but visually from watching the console during 
the build, it seems to be somewhere from 2 to 5 times slower. This is 
running a native build on an ext4 filesystem (not using containers etc).


This is somehow expected from my side tbh - as the lock file disables 
any kind of parallelism - so just one instance of cve-check-task can run 
at a time.
One thing that came to my mind is to move the created CVE_CHECK-sqlite 
db into the recipe-sysroot before running the task (currently it resides 
in DL_DIR), so each task would run on its own copy of the db - that 
would avoid having multiple access call to the same file while enabling 
us to remove the arbitrary task lock.




Are others seeing similar behaviour?

While I am writing, I should also mention that the CVE JSON feeds are 
apparently going to be changing (again).
https://www.cve.org/Media/News/item/news/2022/01/11/Changes-Coming-to-CVE-Record 
<https://www.cve.org/Media/News/item/news/2022/01/11/Changes-Coming-to-CVE-Record>

I wonder if we should perhaps move instead to using the git repo:
https://github.com/CVEProject/cvelist 
<https://github.com/CVEProject/cvelist>  (existing v4 format)
https://github.com/CVEProject/cvelistV5 
<https://github.com/CVEProject/cvelistV5> (preview of upcoming v5 format)
This would not solve the sqlite locking issue, though perhaps we could 
sidestep it and just search the json directly?

I guess I should really start a separate thread about this ;-)

Regards,
Ralph

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162179): 
https://lists.openembedded.org/g/openembedded-core/message/162179
Mute This Topic: https://lists.openembedded.org/mt/88258164/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][dunfell 20/20] ruby: correctly set native/target dependencies

[Konrad Weihmann]

This patch should be merged without this fix 
https://git.yoctoproject.org/poky/commit/?id=89004bc2480808576582001460e37d98143bf9a3


On 21.02.22 15:14, Steve Sakoman wrote:

From: Alexander Kanavin 

In particular libffi was missing from native, which
led to linking with host libffi instead.

Signed-off-by: Alexander Kanavin 
Signed-off-by: Richard Purdie 
(cherry picked from commit 293c9f879252a814107579542e8fca9af9dde599)
Signed-off-by: Steve Sakoman 
---
  meta/recipes-devtools/ruby/ruby.inc | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/ruby/ruby.inc 
b/meta/recipes-devtools/ruby/ruby.inc
index 7b6d4edc61..367cd98d09 100644
--- a/meta/recipes-devtools/ruby/ruby.inc
+++ b/meta/recipes-devtools/ruby/ruby.inc
@@ -14,8 +14,8 @@ LIC_FILES_CHKSUM = "\
  file://LEGAL;md5=2b6d62dc0d608f34d510ca3f428110ec \
  "
  
-DEPENDS = "ruby-native zlib openssl libyaml gdbm readline libffi"

-DEPENDS_class-native = "openssl-native libyaml-native readline-native 
zlib-native"
+DEPENDS = "zlib openssl libyaml gdbm readline libffi"
+DEPENDS_class-target:append = " ruby-native"
  
  SHRT_VER = "${@oe.utils.trim_version("${PV}", 2)}"

  SRC_URI = "http://cache.ruby-lang.org/pub/ruby/${SHRT_VER}/ruby-${PV}.tar.gz \






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162050): 
https://lists.openembedded.org/g/openembedded-core/message/162050
Mute This Topic: https://lists.openembedded.org/mt/89294107/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] gmp: fix EXTRA_OECONF for mipsarchr6

[Konrad Weihmann]

previously used mipsarchr6:append created an empty set for
EXTRA_OECONF and then appended --disable-assembly
while it should be just added to the existing var value.
Without this patch gmp will be configured without
--enable-cxx=detect for mipsarchr6 targets

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-support/gmp/gmp_6.2.1.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-support/gmp/gmp_6.2.1.bb 
b/meta/recipes-support/gmp/gmp_6.2.1.bb
index e61582afdf..33f439eca9 100644
--- a/meta/recipes-support/gmp/gmp_6.2.1.bb
+++ b/meta/recipes-support/gmp/gmp_6.2.1.bb
@@ -20,7 +20,7 @@ SRC_URI[sha256sum] = 
"eae9326beb4158c386e39a356818031bd28f3124cf915f8c5b1dc4c7a3
 acpaths = ""
 
 EXTRA_OECONF += " --enable-cxx=detect"
-EXTRA_OECONF:mipsarchr6:append = " --disable-assembly"
+EXTRA_OECONF:append:mipsarchr6 = " --disable-assembly"
 
 PACKAGES =+ "libgmpxx"
 FILES:libgmpxx = "${libdir}/libgmpxx${SOLIBS}"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161745): 
https://lists.openembedded.org/g/openembedded-core/message/161745
Mute This Topic: https://lists.openembedded.org/mt/89159391/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] ruby: fix DEPENDS append

[Konrad Weihmann]

recent change create a blank scope of DEPENDS for class-target,
basically leaving out all general dependencies, leading to the effect
that ruby will be shipped without the runtime dependencies of zlib,
openssl and libffi, making the corresponding gems unusable at runtime.

As the class-target scope should be appended only the correct override
is append:class-target

Signed-off-by: Konrad Weihmann 
---
 meta/recipes-devtools/ruby/ruby.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/ruby/ruby.inc 
b/meta/recipes-devtools/ruby/ruby.inc
index 7cb2181c33..35d3bfd344 100644
--- a/meta/recipes-devtools/ruby/ruby.inc
+++ b/meta/recipes-devtools/ruby/ruby.inc
@@ -14,7 +14,7 @@ LIC_FILES_CHKSUM = 
"file://COPYING;md5=5b8c87559868796979806100db3f3805 \
 "
 
 DEPENDS = "zlib openssl libyaml gdbm readline libffi"
-DEPENDS:class-target:append = " ruby-native"
+DEPENDS:append:class-target = " ruby-native"
 
 SHRT_VER = "${@oe.utils.trim_version("${PV}", 2)}"
 SRC_URI = "http://cache.ruby-lang.org/pub/ruby/${SHRT_VER}/ruby-${PV}.tar.gz \
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161744): 
https://lists.openembedded.org/g/openembedded-core/message/161744
Mute This Topic: https://lists.openembedded.org/mt/89157718/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v3] create-spdx: Get SPDX-License-Identifier from source

[Konrad Weihmann]

On 08.02.22 16:50, Saul Wold wrote:
> 
> 
> On 2/8/22 07:37, Konrad Weihmann wrote:
>> On 08.02.22 16:02, Saul Wold wrote:
>>> This patch will read the begining of source files and try to find
>>> the SPDX-License-Identifier to populate the licenseInfoInFiles
>>> field for each source file. This does not populate licenseConcluded
>>> at this time, nor rolls it up to package level.
>>>
>>> We read as binary file since some source code seem to have some
>>> binary characters, the license is then converted to ascii strings.
>>>
>>> Signed-off-by: Saul Wold 
>>> ---
>>> v2: Clean up commit message
>>> v3: Really fix up regex based on Peter's feedback!
>>>
>>>    meta/classes/create-spdx.bbclass | 22 ++
>>>    1 file changed, 22 insertions(+)
>>>
>>> diff --git a/meta/classes/create-spdx.bbclass 
>>> b/meta/classes/create-spdx.bbclass
>>> index 8b4203fdb5..64aada8593 100644
>>> --- a/meta/classes/create-spdx.bbclass
>>> +++ b/meta/classes/create-spdx.bbclass
>>> @@ -37,6 +37,23 @@ SPDX_SUPPLIER[doc] = "The SPDX PackageSupplier 
>>> field for SPDX packages created f
>>>    do_image_complete[depends] = "virtual/kernel:do_create_spdx"
>>> +def extract_licenses(filename):
>>> +    import re
>>> +
>>> +    lic_regex = re.compile(b'^\W*SPDX-License-Identifier:\s*([ 
>>> \w\d.()+-]+?)(?:\s+\W*)?$', re.MULTILINE)
>>
>> Taking inspiration from reuse-tool
>> (https://github.com/fsfe/reuse-tool/blob/master/src/reuse/_comment.py)
>> and the way they parse comment blocks the results with the updated regex
>> look good.
>>
> I was not aware of this parser.

It was advertised as **the** reference implementation of SPDX-Lic 
scanning a while back - so I would consider it good to get "inspired"

> 
>> Test sample set:
>> (* SPDX-License-Identifier: Foo-Bar *)
>> (* SPDX-License-Identifier: Foo-Bar *)
>> /* SPDX-License-Identifier: Foo-Bar */
>> 
>> <#-- SPDX-License-Identifier: Foo-Bar -->
>> <%-- SPDX-License-Identifier: Foo-Bar --%>
>> {# SPDX-License-Identifier: Foo-Bar #}
>> {/* SPDX-License-Identifier: Foo-Bar */}
>> {{!-- SPDX-License-Identifier: Foo-Bar --}}
>> @Comment{ SPDX-License-Identifier: Foo-Bar } ---> Only this one is
>> missed (which is bibtex syntax) - no idea if that is of importance for
>> anyone.
> 
>> Just wanted to highlight that this is not catching every possible 
>> input line
>>
> Do we need to pull in the complexity of the reuse-tool comment parser? 
> Let me know, might not make 3.5 if this is the case.

For me it this seems to be a good best effort approach and as long as no 
one claims that the output is technically a 100% correct, I'm good with it.
If one would want to improve it, I would say the usage of an external 
tokenizer like pygments would be good - extracting all the comments from 
a beginning of a file and then filtering by regex.
For now the implementation might miss out on a few edge cases (like the 
syntax I highlighted or comment blocks longer than 15k chars) but that 
is totally fine with me.

> 
> Sau!
> 
>>
>>> +
>>> +    try:
>>> +    with open(filename, 'rb') as f:
>>> +    size = min(15000, os.stat(filename).st_size)
>>> +    txt = f.read(size)
>>> +    licenses = re.findall(lic_regex, txt)
>>> +    if licenses:
>>> +    ascii_licenses = [lic.decode('ascii') for lic in 
>>> licenses]
>>> +    return ascii_licenses
>>> +    except Exception as e:
>>> +    bb.warn(f"Exception reading {filename}: {e}")
>>> +    return None
>>> +
>>>    def get_doc_namespace(d, doc):
>>>    import uuid
>>>    namespace_uuid = uuid.uuid5(uuid.NAMESPACE_DNS, 
>>> d.getVar("SPDX_UUID_NAMESPACE"))
>>> @@ -232,6 +249,11 @@ def add_package_files(d, doc, spdx_pkg, topdir, 
>>> get_spdxid, get_types, *, archiv
>>>    checksumValue=bb.utils.sha256_file(filepath),
>>>    ))
>>> +    if "SOURCE" in spdx_file.fileTypes:
>>> +    extracted_lics = extract_licenses(filepath)
>>> +    if extracted_lics:
>>> +    spdx_file.licenseInfoInFiles = extracted_lics
>>> +
>>>    doc.files.append(spdx_file)
>>>    doc.add_relationship(spdx_pkg, "CONTAINS", spdx_file)
>>>    spdx_pkg.hasFiles.append(spdx_file.SPDXID)
>>>
>>>
>>>
>>> 
>>>
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161522): 
https://lists.openembedded.org/g/openembedded-core/message/161522
Mute This Topic: https://lists.openembedded.org/mt/88997967/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v3] create-spdx: Get SPDX-License-Identifier from source

[Konrad Weihmann]

On 08.02.22 16:02, Saul Wold wrote:
> This patch will read the begining of source files and try to find
> the SPDX-License-Identifier to populate the licenseInfoInFiles
> field for each source file. This does not populate licenseConcluded
> at this time, nor rolls it up to package level.
> 
> We read as binary file since some source code seem to have some
> binary characters, the license is then converted to ascii strings.
> 
> Signed-off-by: Saul Wold 
> ---
> v2: Clean up commit message
> v3: Really fix up regex based on Peter's feedback!
> 
>   meta/classes/create-spdx.bbclass | 22 ++
>   1 file changed, 22 insertions(+)
> 
> diff --git a/meta/classes/create-spdx.bbclass 
> b/meta/classes/create-spdx.bbclass
> index 8b4203fdb5..64aada8593 100644
> --- a/meta/classes/create-spdx.bbclass
> +++ b/meta/classes/create-spdx.bbclass
> @@ -37,6 +37,23 @@ SPDX_SUPPLIER[doc] = "The SPDX PackageSupplier field for 
> SPDX packages created f
>   
>   do_image_complete[depends] = "virtual/kernel:do_create_spdx"
>   
> +def extract_licenses(filename):
> +import re
> +
> +lic_regex = re.compile(b'^\W*SPDX-License-Identifier:\s*([ 
> \w\d.()+-]+?)(?:\s+\W*)?$', re.MULTILINE)

Taking inspiration from reuse-tool 
(https://github.com/fsfe/reuse-tool/blob/master/src/reuse/_comment.py) 
and the way they parse comment blocks the results with the updated regex 
look good.

Test sample set:
(* SPDX-License-Identifier: Foo-Bar *)
(* SPDX-License-Identifier: Foo-Bar *)
/* SPDX-License-Identifier: Foo-Bar */

<#-- SPDX-License-Identifier: Foo-Bar -->
<%-- SPDX-License-Identifier: Foo-Bar --%>
{# SPDX-License-Identifier: Foo-Bar #}
{/* SPDX-License-Identifier: Foo-Bar */}
{{!-- SPDX-License-Identifier: Foo-Bar --}}
@Comment{ SPDX-License-Identifier: Foo-Bar } ---> Only this one is 
missed (which is bibtex syntax) - no idea if that is of importance for 
anyone.

Just wanted to highlight that this is not catching every possible input line


> +
> +try:
> +with open(filename, 'rb') as f:
> +size = min(15000, os.stat(filename).st_size)
> +txt = f.read(size)
> +licenses = re.findall(lic_regex, txt)
> +if licenses:
> +ascii_licenses = [lic.decode('ascii') for lic in licenses]
> +return ascii_licenses
> +except Exception as e:
> +bb.warn(f"Exception reading {filename}: {e}")
> +return None
> +
>   def get_doc_namespace(d, doc):
>   import uuid
>   namespace_uuid = uuid.uuid5(uuid.NAMESPACE_DNS, 
> d.getVar("SPDX_UUID_NAMESPACE"))
> @@ -232,6 +249,11 @@ def add_package_files(d, doc, spdx_pkg, topdir, 
> get_spdxid, get_types, *, archiv
>   checksumValue=bb.utils.sha256_file(filepath),
>   ))
>   
> +if "SOURCE" in spdx_file.fileTypes:
> +extracted_lics = extract_licenses(filepath)
> +if extracted_lics:
> +spdx_file.licenseInfoInFiles = extracted_lics
> +
>   doc.files.append(spdx_file)
>   doc.add_relationship(spdx_pkg, "CONTAINS", spdx_file)
>   spdx_pkg.hasFiles.append(spdx_file.SPDXID)
> 
> 
> 
> 
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161515): 
https://lists.openembedded.org/g/openembedded-core/message/161515
Mute This Topic: https://lists.openembedded.org/mt/88997967/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] valgrind: Enable network for ptest builds

[Konrad Weihmann]

My understanding is to add

--nonet

to XMLLINT_FLAGS = --noblanks --noout --xinclude --postvalid --noent

in docs/Makefile.in

and we should be good - haven't tried it yet but it should do the trick

On 13.01.22 18:39, Alexander Kanavin wrote:
Why wasn't this exposed by AB testing? And can there be a better 
solution (e.g. skip the validation step), since this would still break 
on hosts without internet.


Alex

On Thu, 13 Jan 2022 at 18:36, Khem Raj > wrote:


do_compile_ptest_base uses xmllint which accesses internet for
validating DTD

Signed-off-by: Khem Raj mailto:raj.k...@gmail.com>>
---
  meta/recipes-devtools/valgrind/valgrind_3.18.1.bb
 | 5 +
  1 file changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/valgrind/valgrind_3.18.1.bb

b/meta/recipes-devtools/valgrind/valgrind_3.18.1.bb

index 7837034f92e..5ebcae467fa 100644
--- a/meta/recipes-devtools/valgrind/valgrind_3.18.1.bb

+++ b/meta/recipes-devtools/valgrind/valgrind_3.18.1.bb

@@ -98,6 +98,11 @@ CACHED_CONFIGUREVARS +=
"ac_cv_path_PERL='/usr/bin/env perl'"
  # which fixes build path issue in DWARF.
  SELECTED_OPTIMIZATION = "${DEBUG_FLAGS}"

+# This is needed because xmllint needs access to network to
+# validate DTD 4.3 onwards it uses --dtdvalid
http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd

+# to xmllint, allow it to access internet
+do_compile_ptest_base[network] = "1"
+
  do_configure:prepend () {
      rm -rf ${S}/config.h
      sed -i -e 's:$(abs_top_builddir):$(pkglibdir)/ptest:g'
${S}/none/tests/Makefile.am
-- 
2.34.1










-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160528): 
https://lists.openembedded.org/g/openembedded-core/message/160528
Mute This Topic: https://lists.openembedded.org/mt/88401996/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [meta-oe][master][PATCH] nodejs_16.11.1.bb: only handle npm if configured

[Konrad Weihmann]

That needs to go to the openembedded-de...@list.openembedded.org as 
nodejs isn't part of core


On 11.01.22 17:23, Nisha Parrakat wrote:

Signed-off-by: Nisha Parrakat 
---
  meta-oe/recipes-devtools/nodejs/nodejs_16.11.1.bb | 9 ++---
  1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/meta-oe/recipes-devtools/nodejs/nodejs_16.11.1.bb 
b/meta-oe/recipes-devtools/nodejs/nodejs_16.11.1.bb
index 11034420a..c8d942538 100644
--- a/meta-oe/recipes-devtools/nodejs/nodejs_16.11.1.bb
+++ b/meta-oe/recipes-devtools/nodejs/nodejs_16.11.1.bb
@@ -175,8 +175,9 @@ do_install:append:class-native() {
  # /usr/bin/npm is symlink to /usr/lib/node_modules/npm/bin/npm-cli.js
  # use sed on npm-cli.js because otherwise symlink is replaced with normal 
file and
  # npm-cli.js continues to use old shebang
-sed "1s^.*^#\!/usr/bin/env node^g" -i 
${D}${exec_prefix}/lib/node_modules/npm/bin/npm-cli.js
-
+if [[ -f "${D}${exec_prefix}/lib/node_modules/npm/bin/npm-cli.js" ]]; then
+sed "1s^.*^#\!/usr/bin/env node^g" -i 
${D}${exec_prefix}/lib/node_modules/npm/bin/npm-cli.js
+fi
  # Install the native binaries to provide it within sysroot for the target 
compilation
  install -d ${D}${bindir}
  install -m 0755 ${S}/out/Release/torque ${D}${bindir}/torque
@@ -189,7 +190,9 @@ do_install:append:class-native() {
  }
  
  do_install:append:class-target() {

-sed "1s^.*^#\!${bindir}/env node^g" -i 
${D}${exec_prefix}/lib/node_modules/npm/bin/npm-cli.js
+if [[ -f "${D}${exec_prefix}/lib/node_modules/npm/bin/npm-cli.js" ]]; then
+sed "1s^.*^#\!${bindir}/env node^g" -i 
${D}${exec_prefix}/lib/node_modules/npm/bin/npm-cli.js
+fi
  }
  
  PACKAGES =+ "${PN}-npm"







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160419): 
https://lists.openembedded.org/g/openembedded-core/message/160419
Mute This Topic: https://lists.openembedded.org/mt/88352467/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] rootfs-postcommands.bbclass: Change the default.target location

[Konrad Weihmann]

On 10.01.22 21:33, Peter Kjellerstedt wrote:

-Original Message-
From: Richard Purdie 
Sent: den 10 januari 2022 16:44
To: Peter Kjellerstedt ; openembedded-
c...@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] rootfs-postcommands.bbclass: Change the
default.target location

On Mon, 2022-01-10 at 16:37 +0100, Peter Kjellerstedt wrote:

Rather than adding a default.target in /etc/systemd/system, modify the
one in /usr/lib/systemd/system.


Why?

(the commit message needs to say why as well as what it does)


I thought it did (avoid having two links where one will suffice).



Cheers,

Richard


Thinking a bit more about this though, I am not sure I can win. We have
had a corresponding local patch for many years, and it seems natural to
have the link only in /usr/lib/systemd/system. However, for a system that
uses package management (which we do not), it may not be as natural,
since then updating the systemd package in runtime would restore systemd's
default.target rather than the one that was installed with the system.

Would you accept a patch that instead introduces a variable for the path
to the default.target link, with the following definition:

SYSTEMD_DEFAULT_TARGET_PATH ?= "${@bb.utils.contains('IMAGE_FEATURES', 
'package-management', '${sysconfdir}/systemd/system', '${systemd_system_unitdir}', 
d}"


IMO that would look overly complicated.
Just pointing out that, with your patch, people can alter the default 
target by placing an appropriate link in /etc/systemd/... to override 
the default choices would be enough.
I guess changing things in /lib/systemd for an image makes more sense, 
it only has to be named in the commit message


Honestly I don't think it needs another variable to achieve the goal you 
had in mind


Konrad



That would make it reasonably sane by default, but still allow it to be
overridden if you have other ideas for your system.

//Peter
  







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160378): 
https://lists.openembedded.org/g/openembedded-core/message/160378
Mute This Topic: https://lists.openembedded.org/mt/88325758/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][PATCH] qemu.bbclass: drop OLDEST_KERNEL reference

[Konrad Weihmann]

On 08.01.22 16:13, Richard Purdie wrote:

On Sat, 2022-01-08 at 07:10 -0800, Matt Madison wrote:

On Sat, Jan 8, 2022 at 6:34 AM Richard Purdie
 wrote:


On Fri, 2022-01-07 at 06:44 -0800, Matt Madison wrote:

On Fri, Jan 7, 2022 at 4:56 AM Richard Purdie
 wrote:


On Fri, 2022-01-07 at 12:51 +, Richard Purdie via lists.openembedded.org
wrote:

On Fri, 2022-01-07 at 04:36 -0800, Matt Madison wrote:

On Fri, Jan 7, 2022 at 3:19 AM Richard Purdie
 wrote:


On Wed, 2021-12-15 at 12:16 -0800, Matt Madison wrote:

which is introducing task hash changes for some
allarch package builds, and should no longer
be needed with recent versions of qemu.

Signed-off-by: Matt Madison 
---
  meta/classes/qemu.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/qemu.bbclass b/meta/classes/qemu.bbclass
index 01a7b86ae1..333202b7c4 100644
--- a/meta/classes/qemu.bbclass
+++ b/meta/classes/qemu.bbclass
@@ -54,7 +54,7 @@ def qemu_run_binary(data, rootfs_path, binary):
  # this dance). For others (e.g. arm) a -cpu option is not necessary, since the
  # qemu-arm default CPU supports all required architecture levels.

-QEMU_OPTIONS = "-r ${OLDEST_KERNEL} ${@d.getVar("QEMU_EXTRAOPTIONS_%s" % 
d.getVar('PACKAGE_ARCH')) or ""}"
+QEMU_OPTIONS = "${@d.getVar("QEMU_EXTRAOPTIONS_%s" % d.getVar('PACKAGE_ARCH')) or 
""}"
  QEMU_OPTIONS[vardeps] += "QEMU_EXTRAOPTIONS_${PACKAGE_ARCH}"

  QEMU_EXTRAOPTIONS_ppce500v2 = " -cpu e500v2"


I've sent a revert for this since the commit message isn't true, this is causing
autobuilder failures and there was also a user report of issues with it.


Bummer.  I think this is because our OLDEST_KERNEL setting for aarch64
(3.14.0) is
higher than the UNAME_MINIMUM_RELEASE definition in QEMU (3.8.0).


https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/4559/steps/13/logs/stdio
(centos7 aarch64 build)

We'll have to find a different way to fix the allarch issue (more details
welcome on which package shows the issue).


I ran into the issue with cantarell-fonts and wayland-protocols, but
any recipe that inherits both allarch and meson should trigger it.

In my case, I have a layer check job for meta-tegra that runs
yocto-check-layer with the --machines option specifying qemux86 and
one of the (aarch64) machines in my layer, and that was reporting the
signature changes between the two architectures.


I wonder why we don't detect that with our standard tests? :/

There may be another fix:

diff --git a/meta/classes/allarch.bbclass b/meta/classes/allarch.bbclass
index 5bd5c44a276..0aadf7bfba6 100644
--- a/meta/classes/allarch.bbclass
+++ b/meta/classes/allarch.bbclass
@@ -43,6 +43,9 @@ python () {
  d.setVar("TARGET_LDFLAGS", "none")
  d.setVar("POPULATESYSROOTDEPS", "")

+if bb.data.inherits_class("meson", d):
+d.setVar("do_write_config", ":")
+
  # Avoid this being unnecessarily different due to nuances of
  # the target machine that aren't important for "all" arch
  # packages.




Sorry, that isn't quite right. We want to just remove the append to write_config
which generates the qemu wrapper which will probably need a little more invasive
juggling of the code to make it easier...


Your patch to disable the qemu wrapper for allarch looks like a good solution.
Hopefully, that will work OK across the board.


It seemed to be so I've merged it. Am hoping this resolves things!


Drat. Unfortunately, it only works if allarch is inherited after meson
(which inherits qemu).  So cantarell-fonts checks out OK,
but wayland-protocols doesn't, since it inherits allarch first.


Probably easy to fix but hard to spot new problems :/


would an insane check similar to native-last be of help here?



Cheers,

Richard






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160279): 
https://lists.openembedded.org/g/openembedded-core/message/160279
Mute This Topic: https://lists.openembedded.org/mt/87752526/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH][RFC] cve-check: add lockfile to task

[Konrad Weihmann]

On 07.01.22 10:48, Konrad Weihmann wrote:

this should prevent running into the very rare error
sqlite3.OperationalError: attempt to write a readonly database


It's also possible that check_same_thread (that defaults to True ) in 
the sqlite3.connect causes this (see 
https://docs.python.org/3/library/sqlite3.html#sqlite3.connect) - I have 
too little experience with sqlite to provide any further input.


I'd be happy to get some feedback from users that do have that 
experience with sqlite in a highly multithreaded/multiprocess 
environment. Or by people that use cvecheck class in larger setups than 
just core.




As highlighted by https://www.sqlite.org/faq.html#q5
it is likely that the adapter won't allow use multiple exec calls
at the same time.

So it's best to prevent multiple accesses at a time, by reusing
the already in place CVE_CHECK_DB_FILE_LOCK

YOCTO #14110

Signed-off-by: Konrad Weihmann 
---
  meta/classes/cve-check.bbclass | 1 +
  1 file changed, 1 insertion(+)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 70d1988a70..6c04ff9f09 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -111,6 +111,7 @@ python do_cve_check () {
  }
  
  addtask cve_check before do_build after do_fetch

+do_cve_check[lockfiles] += "${CVE_CHECK_DB_FILE_LOCK}"
  do_cve_check[depends] = "cve-update-db-native:do_fetch"
  do_cve_check[nostamp] = "1"
  







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160252): 
https://lists.openembedded.org/g/openembedded-core/message/160252
Mute This Topic: https://lists.openembedded.org/mt/88258164/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH][RFC] cve-check: add lockfile to task

[Konrad Weihmann]

this should prevent running into the very rare error
sqlite3.OperationalError: attempt to write a readonly database

As highlighted by https://www.sqlite.org/faq.html#q5
it is likely that the adapter won't allow use multiple exec calls
at the same time.

So it's best to prevent multiple accesses at a time, by reusing
the already in place CVE_CHECK_DB_FILE_LOCK

YOCTO #14110

Signed-off-by: Konrad Weihmann 
---
 meta/classes/cve-check.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 70d1988a70..6c04ff9f09 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -111,6 +111,7 @@ python do_cve_check () {
 }
 
 addtask cve_check before do_build after do_fetch
+do_cve_check[lockfiles] += "${CVE_CHECK_DB_FILE_LOCK}"
 do_cve_check[depends] = "cve-update-db-native:do_fetch"
 do_cve_check[nostamp] = "1"
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160251): 
https://lists.openembedded.org/g/openembedded-core/message/160251
Mute This Topic: https://lists.openembedded.org/mt/88258164/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] base/patch: Disable network for unpack/patch/configure/compile/install

[Konrad Weihmann]

On 27.12.21 13:54, Stefan Herbrechtsmeier wrote:

Hi Konrad,

Am 25.12.21 um 21:43 schrieb Konrad Weihmann:
What I so far don't really get is why increase in parsing time is such 
a big deal.
I admit when we're talking about npm it's some kind of a drastic 
increase in recipes one would have to maintain, just because some 
random project decides to use a trillion dependencies instead of 
writing two or three lines of code more.


Still I come to think this might be actually beneficial, as it shows 
how broken npm is from a distribution perspective - as it may be that 
some users actually start to access the situation when they are 
actually aware what monstrosity of a dep tree they are inheriting by 
just a single npm module.


So this is mind - and I don't want to sound radical - I would rather 
abandon npm and go support in core than to sacrifice closing the one 
main loophole in core that is preventing true accessibility at the 
moment.
Which is uncontrolled fetching outside of the fetching task, as this 
invalidates everything you will get in the end in terms of licensing, 
quality reports that haven't been done as part of the build and even 
CVE checking becomes pretty much worthless if one is allowed to inject 
random code on the fly into the build - and in the end everything that 
can't be assessed outside of the build is pretty much non-existing for 
most of the assessments I had to work with.


In the past I showed that npm and go can be made to work with these 
principles, even if they would introduce a different way of working 
and maybe the need for better tooling - *but* they can work with how 
oe-core works at the moment - to the expense of parsing time - and 
that's pretty much it from my point of view.


Still the gains outweigh it by far in my opinion, as it would make all 
of that accessible by common tooling already in place - including true 
reproducibility and builtin quality reports.


BTW I don't think rust and therefore cargo is heavily affected by it, 
as the cargo to bitbake scripting works kind in the way I would 
imagine for npm and go too - so far just npm and go are really really 
bad to handle in this case.


I'm happy to help on such scripting and tooling, but I don't see much 
worth in either accepting the fact that "modern languages" have to 
include some not validatable "magic" (which then would mean allowing 
uncontrolled network access) or working around the fact that a 
trillion dependencies is still a trillion dependencies no matter how 
you put it ( :) )


I fully agree with you and would be happy to add a 'recursive' option to 
devtool / recipetool to create multiple recipes on the fly instead of 
one big recipe with more than 100 different projects inclusive duplicate 
and different compatible versions.


That, yes - and a split between sources and recipes, I guess, is needed.
Like creation of a sharable inc-file for definition of sources (SRC_URI 
and dependencies) in addition to the actual recipe itself.

(One could also think about versioned inc-files here)
Nevertheless this is needed to untangle go (luckily npm is much more 
straight forward in this sense).


Like one has a go module "a" that consists of an inc-file, which defines 
the needed sources for "a" and the needed dependencies in terms of other 
go modules.
And then you have the recipe "a" which just includes this inc file and 
go.bbclass (plus the usual recipe specific meta data).
Now I could build recipe a or I could build any other recipe that 
depends on this go module with a version I could define (or tune).


I know that this isn't something that is not very welcome, as projects 
tend to define their versioned dependencies with lock-files (and every 
language/eco system in their very own way) - still I think the "freedom 
of choice" for a version is something that is a superb feature when 
using yocto - so I personally would rather ignore the maybe 
outdated/wrong information of a project as to not have it (this choice) 
at all.


So if one could make that happen with devtool (maybe using templates or 
so) I think the project would be prepared to fully support these 
"modern" languages without giving up on some of the principles like 
reproducibility


-- to get back to the responses from you other mail --

> Is it possible to create a recipe for the source and a recipe for the 
> binaries which depends on the source recipe? In this case the DEPENDS 
> is always the source package and the RDEPENDS the binary package.

> A BBCLASSEXTEND could be used to create the source recipe automatically.

That could be worth a shot, as I think this could actually work - 
basically the same idea as above but with recipes instead of inc-files.






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160023): 
https://lists.openembedded.org/g/openembedded-core/message/16

Re: [OE-core] [PATCH] base/patch: Disable network for unpack/patch/configure/compile/install

[Konrad Weihmann]

One more thing that came to my mind is that the project is missing demo 
recipes/real life examples in core that actually would show that one can 
achieve reproducibility in the sense of the project's definition for 
both npm and go.

So far there is the fetcher code but real life users are living elsewhere.

So one could add consumers to core or push the code to where the "real 
action" is (meta-virt for instance comes to my mind)


On 25.12.21 21:43, Konrad Weihmann wrote:
What I so far don't really get is why increase in parsing time is such a 
big deal.
I admit when we're talking about npm it's some kind of a drastic 
increase in recipes one would have to maintain, just because some random 
project decides to use a trillion dependencies instead of writing two or 
three lines of code more.


Still I come to think this might be actually beneficial, as it shows how 
broken npm is from a distribution perspective - as it may be that some 
users actually start to access the situation when they are actually 
aware what monstrosity of a dep tree they are inheriting by just a 
single npm module.


So this is mind - and I don't want to sound radical - I would rather 
abandon npm and go support in core than to sacrifice closing the one 
main loophole in core that is preventing true accessibility at the moment.
Which is uncontrolled fetching outside of the fetching task, as this 
invalidates everything you will get in the end in terms of licensing, 
quality reports that haven't been done as part of the build and even CVE 
checking becomes pretty much worthless if one is allowed to inject 
random code on the fly into the build - and in the end everything that 
can't be assessed outside of the build is pretty much non-existing for 
most of the assessments I had to work with.


In the past I showed that npm and go can be made to work with these 
principles, even if they would introduce a different way of working and 
maybe the need for better tooling - *but* they can work with how oe-core 
works at the moment - to the expense of parsing time - and that's pretty 
much it from my point of view.


Still the gains outweigh it by far in my opinion, as it would make all 
of that accessible by common tooling already in place - including true 
reproducibility and builtin quality reports.


BTW I don't think rust and therefore cargo is heavily affected by it, as 
the cargo to bitbake scripting works kind in the way I would imagine for 
npm and go too - so far just npm and go are really really bad to handle 
in this case.


I'm happy to help on such scripting and tooling, but I don't see much 
worth in either accepting the fact that "modern languages" have to 
include some not validatable "magic" (which then would mean allowing 
uncontrolled network access) or working around the fact that a trillion 
dependencies is still a trillion dependencies no matter how you put it ( 
:) )


Regards
Konrad





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160010): 
https://lists.openembedded.org/g/openembedded-core/message/160010
Mute This Topic: https://lists.openembedded.org/mt/87909311/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] base/patch: Disable network for unpack/patch/configure/compile/install

[Konrad Weihmann]

On 25.12.21 20:32, Stefan Herbrechtsmeier wrote:


Isn't the circular dependencies problem mainly a problem of the native 
packages? In many cases the dependency only exists at compile time of 
the main project and not at compile time of the dependency itself. In 
case of a native package all dependency sysroots need to be finished 
before the recipe sysroot itself.


Unfortunately it is not - it's happens that the go.mod already creates 
this circular dependency.


The classic example is that golang.org/x/tools depends on parts of 
golang.org/x/text (and guess what) golang.org/x/text depends solely on 
golang.org/x/tools :facepalm:


For go itself this isn't a big issue, as the compiler just pulls all the 
needed code into the compile workspace and build a binary out of it, 
which can be reused later on.


But for bitbake it's a big issue, as either you pull in

golang.org/x/text + golang.org/x/tools sources into the 
golang.org/x/text workspace, stripping you off the possibility to 
properly watch the revision of golang.org/x/tools (in terms of devtools 
and co) - or you simply end up with a circular dependency.


I scratched my head on this one for weeks, till I found a way to 
actually use the latest greatest of each go module each coming as a 
recipe of its own, while avoiding above mentioned situations -- and it's 
kind of a huge break to how things are done right now.


I'm pretty sure the same will happen in rust sooner or later.

And I personally don't see how to break it as I don't know any way go 
distinguishes between compile and runtime dependency - afaik they all 
need to be present at compile time

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160009): 
https://lists.openembedded.org/g/openembedded-core/message/160009
Mute This Topic: https://lists.openembedded.org/mt/87909311/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] base/patch: Disable network for unpack/patch/configure/compile/install

[Konrad Weihmann]

What I so far don't really get is why increase in parsing time is such a 
big deal.
I admit when we're talking about npm it's some kind of a drastic 
increase in recipes one would have to maintain, just because some random 
project decides to use a trillion dependencies instead of writing two or 
three lines of code more.


Still I come to think this might be actually beneficial, as it shows how 
broken npm is from a distribution perspective - as it may be that some 
users actually start to access the situation when they are actually 
aware what monstrosity of a dep tree they are inheriting by just a 
single npm module.


So this is mind - and I don't want to sound radical - I would rather 
abandon npm and go support in core than to sacrifice closing the one 
main loophole in core that is preventing true accessibility at the moment.
Which is uncontrolled fetching outside of the fetching task, as this 
invalidates everything you will get in the end in terms of licensing, 
quality reports that haven't been done as part of the build and even CVE 
checking becomes pretty much worthless if one is allowed to inject 
random code on the fly into the build - and in the end everything that 
can't be assessed outside of the build is pretty much non-existing for 
most of the assessments I had to work with.


In the past I showed that npm and go can be made to work with these 
principles, even if they would introduce a different way of working and 
maybe the need for better tooling - *but* they can work with how oe-core 
works at the moment - to the expense of parsing time - and that's pretty 
much it from my point of view.


Still the gains outweigh it by far in my opinion, as it would make all 
of that accessible by common tooling already in place - including true 
reproducibility and builtin quality reports.


BTW I don't think rust and therefore cargo is heavily affected by it, as 
the cargo to bitbake scripting works kind in the way I would imagine for 
npm and go too - so far just npm and go are really really bad to handle 
in this case.


I'm happy to help on such scripting and tooling, but I don't see much 
worth in either accepting the fact that "modern languages" have to 
include some not validatable "magic" (which then would mean allowing 
uncontrolled network access) or working around the fact that a trillion 
dependencies is still a trillion dependencies no matter how you put it ( 
:) )


Regards
Konrad

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160008): 
https://lists.openembedded.org/g/openembedded-core/message/160008
Mute This Topic: https://lists.openembedded.org/mt/87909311/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] base/patch: Disable network for unpack/patch/configure/compile/install

[Konrad Weihmann]

I had a look at the failures and most of them are fairly easy to fix - 
all but the go recipes like influxdb.


I'm not sure how to deal with that, so there aren't that many options here.

Either
- inject all the needed with a fixed revision, but that would prevent 
devtool from catching updates to them properly
- create recipes for all of them and pray that there won't be a circular 
dependency in any of them

- allow go to work with network (the least favorable option IMO)

in the case of influxdb for instance we are talking about

github.com/peterh/liner
golang.org/x/crypto
github.com/influxdata/influxql
github.com/influxdata/flux
github.com/BurntSushi/toml
github.com/influxdata/usage-client
golang.org/x/text
go.uber.org/zap
github.com/klauspost/pgzip
github.com/prometheus/client_golang
github.com/jsternberg/zap-logfmt
github.com/mattn/go-isatty
collectd.org
github.com/bmizerany/pat
github.com/dgrijalva/jwt-go/v4
github.com/gogo/protobuf
github.com/golang/snappy
github.com/tinylib/msgp
github.com/opentracing/opentracing-go
golang.org/x/sync
github.com/apache/arrow/go/arrow
github.com/pkg/errors
google.golang.org/grpc
github.com/kraj/xxhash
github.com/influxdata/roaring
github.com/xlab/treeprint
golang.org/x/time
golang.org/x/sys
github.com/jwilder/encoding
github.com/dgryski/go-bitstream

where especially the golang.org/x modules are known to have circular 
dependencies like x/a relies on x/b, while x/b requires x/c and x/c 
needs x/a for building.


Not sure how to tackle this - any thoughts?

On 24.12.21 09:30, Richard Purdie wrote:

On Thu, 2021-12-23 at 22:00 -0800, Khem Raj wrote:

is this list of failures due to this patch ?
https://errors.yoctoproject.org/Errors/Build/137579/


Looks likely, yes. Recipes shouldn't be attempting downloads in do_compile :(

Cheers,

Richard






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#15): 
https://lists.openembedded.org/g/openembedded-core/message/15
Mute This Topic: https://lists.openembedded.org/mt/87909311/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] base/patch: Disable network for unpack/patch/configure/compile/install

[Konrad Weihmann]

On 23.12.21 15:52, Andreas Müller wrote:

On Thu, Dec 23, 2021 at 2:19 PM Konrad Weihmann  wrote:




On 23.12.21 14:11, Richard Purdie wrote:

On Thu, 2021-12-23 at 12:31 +0100, Konrad Weihmann wrote:


On 23.12.21 11:49, Peter Kjellerstedt wrote:

-Original Message-
From: openembedded-core@lists.openembedded.org 
 On Behalf Of Richard Purdie
Sent: den 23 december 2021 00:21
To: openembedded-core@lists.openembedded.org
Subject: [OE-core] [PATCH] base/patch: Disable network for 
unpack/patch/configure/compile/install

Use the newly added nonetwork task flag to disable network access where
possible in unpack/patch/configure/compile/install tasks.

We can't disable networking in sstate tasks due to sstate downloads and
also so we can report hash equivalence to the server.


Since no tasks except fetch (and apparently the sstate tasks) are expected
to use the network, wouldn't it make more sense to reverse this flag? I.e.,
add do_fetch[network] = "1" instead. That way you don't get away with
adding some random task and using the network from it unless you explicitly
state that you will.


This is actually a brilliant idea, which would also make it easier to
control this behavior from a user's perspective


Part of me wonders if we really do want to make this "easy" for the user :/


"easier" may have sounded wrong - what I mean is that it would be then
explicit, with no network access being the default.

So for instance a simple grep on a new layer could confirm whether a
layer needs network access or not - while with the opt-out method of
your patch things may be a bit more difficult to check.

side note: this could also be put into the check-layer script (as I
think the official badge shall just go to layers that do not need
network access out of the scope defined by core)

And yes I got a ton of additional tasks in the layers I work with and
none of these would be affected by the opt-in model.

Either way, I really love to see this feature finally happen


Out of curiosity (will see fallout at least on libreoffice): Why is
this a feature to love?


Simply said because it makes validation of changes way easier.
In before I had to build a change in a normal environment (w network 
access) to verify that it actually builds, then populate some mirror and 
rebuild the same change in an environment without network access.
Now I only will need one run to be sure that it the same recipe will 
even build in 10yrs from now (assuming that a dl-mirror exists).




Andreas


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159987): 
https://lists.openembedded.org/g/openembedded-core/message/159987
Mute This Topic: https://lists.openembedded.org/mt/87909311/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] base/patch: Disable network for unpack/patch/configure/compile/install

[Konrad Weihmann]

On 23.12.21 14:11, Richard Purdie wrote:

On Thu, 2021-12-23 at 12:31 +0100, Konrad Weihmann wrote:


On 23.12.21 11:49, Peter Kjellerstedt wrote:

-Original Message-
From: openembedded-core@lists.openembedded.org 
 On Behalf Of Richard Purdie
Sent: den 23 december 2021 00:21
To: openembedded-core@lists.openembedded.org
Subject: [OE-core] [PATCH] base/patch: Disable network for 
unpack/patch/configure/compile/install

Use the newly added nonetwork task flag to disable network access where
possible in unpack/patch/configure/compile/install tasks.

We can't disable networking in sstate tasks due to sstate downloads and
also so we can report hash equivalence to the server.


Since no tasks except fetch (and apparently the sstate tasks) are expected
to use the network, wouldn't it make more sense to reverse this flag? I.e.,
add do_fetch[network] = "1" instead. That way you don't get away with
adding some random task and using the network from it unless you explicitly
state that you will.


This is actually a brilliant idea, which would also make it easier to
control this behavior from a user's perspective


Part of me wonders if we really do want to make this "easy" for the user :/


"easier" may have sounded wrong - what I mean is that it would be then 
explicit, with no network access being the default.


So for instance a simple grep on a new layer could confirm whether a 
layer needs network access or not - while with the opt-out method of 
your patch things may be a bit more difficult to check.


side note: this could also be put into the check-layer script (as I 
think the official badge shall just go to layers that do not need 
network access out of the scope defined by core)


And yes I got a ton of additional tasks in the layers I work with and 
none of these would be affected by the opt-in model.


Either way, I really love to see this feature finally happen



Cheers,

Richard


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159983): 
https://lists.openembedded.org/g/openembedded-core/message/159983
Mute This Topic: https://lists.openembedded.org/mt/87909311/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] base/patch: Disable network for unpack/patch/configure/compile/install

[Konrad Weihmann]

On 23.12.21 11:49, Peter Kjellerstedt wrote:

-Original Message-
From: openembedded-core@lists.openembedded.org 
 On Behalf Of Richard Purdie
Sent: den 23 december 2021 00:21
To: openembedded-core@lists.openembedded.org
Subject: [OE-core] [PATCH] base/patch: Disable network for 
unpack/patch/configure/compile/install

Use the newly added nonetwork task flag to disable network access where
possible in unpack/patch/configure/compile/install tasks.

We can't disable networking in sstate tasks due to sstate downloads and
also so we can report hash equivalence to the server.


Since no tasks except fetch (and apparently the sstate tasks) are expected
to use the network, wouldn't it make more sense to reverse this flag? I.e.,
add do_fetch[network] = "1" instead. That way you don't get away with
adding some random task and using the network from it unless you explicitly
state that you will.


This is actually a brilliant idea, which would also make it easier to 
control this behavior from a user's perspective






Signed-off-by: Richard Purdie 
---
  meta/classes/base.bbclass  | 4 
  meta/classes/patch.bbclass | 1 +
  2 files changed, 5 insertions(+)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index b709777f243..e4c6c983b59 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -214,6 +214,7 @@ python create_source_date_epoch_stamp() {
  oe.reproducible.epochfile_write(source_date_epoch, d.getVar('SDE_FILE'), 
d)
  }
  do_unpack[postfuncs] += "create_source_date_epoch_stamp"
+do_unpack[nonetwork] = "1"

  def get_source_date_epoch_value(d):
  return oe.reproducible.epochfile_read(d.getVar('SDE_FILE'), d)
@@ -358,6 +359,7 @@ base_do_configure() {
echo ${BB_TASKHASH} > ${CONFIGURESTAMPFILE}
fi
  }
+do_configure[nonetwork] = "1"

  addtask compile after do_configure
  do_compile[dirs] = "${B}"
@@ -368,11 +370,13 @@ base_do_compile() {
bbnote "nothing to compile"
fi
  }
+do_compile[nonetwork] = "1"

  addtask install after do_compile
  do_install[dirs] = "${B}"
  # Remove and re-create ${D} so that is it guaranteed to be empty
  do_install[cleandirs] = "${D}"
+do_install[nonetwork] = "1"

  base_do_install() {
:
diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass
index 8de70254919..57aaf7c31d1 100644
--- a/meta/classes/patch.bbclass
+++ b/meta/classes/patch.bbclass
@@ -164,6 +164,7 @@ patch_do_patch[vardepsexclude] = "PATCHRESOLVE"

  addtask patch after do_unpack
  do_patch[dirs] = "${WORKDIR}"
+do_patch[nonetwork] = "1"
  do_patch[depends] = "${PATCHDEPENDENCY}"

  EXPORT_FUNCTIONS do_patch
--
2.32.0


//Peter






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159976): 
https://lists.openembedded.org/g/openembedded-core/message/159976
Mute This Topic: https://lists.openembedded.org/mt/87909311/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [dunfell][PATCH v2] dropbear: Fix CVE-2020-36254

[Konrad Weihmann]

On 21.12.21 14:02, Ernst Sjöstrand wrote:

Dropbear shares a lot of code with other SSH implementations, so this is
a port of CVE-2018-20685 to dropbear.

Reference:
https://github.com/mkj/dropbear/commit/8f8a3dff705fad774a10864a2e3dbcfa9779ceff

Signed-off-by: Ernst Sjöstrand 


This is missing an Upstream-Status entry - in this case that should be 
"Upstream-Status: Backport"



---
Rename the patch file to make sure cve-check picks it up.

  meta/recipes-core/dropbear/dropbear.inc   |  4 +++-
  .../dropbear/dropbear/CVE-2020-36254.patch| 23 +++
  2 files changed, 26 insertions(+), 1 deletion(-)
  create mode 100644 meta/recipes-core/dropbear/dropbear/CVE-2020-36254.patch

diff --git a/meta/recipes-core/dropbear/dropbear.inc 
b/meta/recipes-core/dropbear/dropbear.inc
index d41e8b36dc..b949a9a337 100644
--- a/meta/recipes-core/dropbear/dropbear.inc
+++ b/meta/recipes-core/dropbear/dropbear.inc
@@ -22,7 +22,9 @@ SRC_URI = 
"http://matt.ucc.asn.au/dropbear/releases/dropbear-${PV}.tar.bz2 \
 file://dropbear.socket \
 file://dropbear.default \
 ${@bb.utils.contains('DISTRO_FEATURES', 'pam', '${PAM_SRC_URI}', 
'', d)} \
-   ${@bb.utils.contains('PACKAGECONFIG', 'disable-weak-ciphers', 
'file://dropbear-disable-weak-ciphers.patch', '', d)} "
+   ${@bb.utils.contains('PACKAGECONFIG', 'disable-weak-ciphers', 
'file://dropbear-disable-weak-ciphers.patch', '', d)} \
+   file://CVE-2020-36254.patch \
+   "
  
  PAM_SRC_URI = "file://0005-dropbear-enable-pam.patch \

 file://0006-dropbear-configuration-file.patch \
diff --git a/meta/recipes-core/dropbear/dropbear/CVE-2020-36254.patch 
b/meta/recipes-core/dropbear/dropbear/CVE-2020-36254.patch
new file mode 100644
index 00..72ac3e6630
--- /dev/null
+++ b/meta/recipes-core/dropbear/dropbear/CVE-2020-36254.patch
@@ -0,0 +1,23 @@
+From c96c48d62aefc372f2105293ddf8cff2d116dc3a Mon Sep 17 00:00:00 2001
+From: Haelwenn Monnier 
+Date: Mon, 25 May 2020 14:54:29 +0200
+Subject: [PATCH] scp.c: Port OpenSSH CVE-2018-20685 fix (#80)
+
+---
+ scp.c | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+diff --git a/scp.c b/scp.c
+index 742ae00..7b8e7d2 100644
+--- a/scp.c
 b/scp.c
+@@ -935,7 +935,8 @@ sink(int argc, char **argv)
+   size = size * 10 + (*cp++ - '0');
+   if (*cp++ != ' ')
+   SCREWUP("size not delimited");
+-  if ((strchr(cp, '/') != NULL) || (strcmp(cp, "..") == 0)) {
++  if (*cp == '\0' || strchr(cp, '/') != NULL ||
++  strcmp(cp, ".") == 0 || strcmp(cp, "..") == 0) {
+   run_err("error: unexpected filename: %s", cp);
+   exit(1);
+   }






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159917): 
https://lists.openembedded.org/g/openembedded-core/message/159917
Mute This Topic: https://lists.openembedded.org/mt/87876568/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 00/32] Add Inappropriate reasoining

[Konrad Weihmann]

Add reasoning to Inappropriate to all patches that are missing 
missing that entirely

To all maintainers please double check my proposals

The following changes since commit 31bc9cab25692fd0b07dbd994afd93802e93d6bf:

  bitbake: fetch: npm: Use temporary file for empty user config (2021-12-14 
22:47:04 +)

are available in the Git repository at:

  git://github.com/priv-kweihmann/poky inapp-add-min
  https://github.com/priv-kweihmann/poky/tree/inapp-add-min

Konrad Weihmann (32):
  connman: fix patch Inappropriate
  glib: fix patch Inappropriate
  ldconfig: fix patch Inappropriate
  readline: fix patch Inappropriate
  util-linux: fix patch Inappropriate
  autoconf: fix patch Inappropriate
  automake: fix patch Inappropriate
  binutils: fix patch Inappropriate
  e2fsprogs: fix patch Inappropriate
  flex: fix patch Inappropriate
  gcc: fix patch Inappropriate
  gdb: fix patch Inappropriate
  libtool: fix patch Inappropriate
  rsync: fix patch Inappropriate
  strace: fix patch Inappropriate
  tcf-agent: fix patch Inappropriate
  valgrind: fix patch Inappropriate
  xmlto: fix patch Inappropriate
  bc: fix patch Inappropriate
  ethtool: fix patch Inappropriate
  gawk: fix patch Inappropriate
  textinfo: fix patch Inappropriate
  gobject-intro: fix patch Inappropriate
  gtk-doc: fix patch Inappropriate
  libxcb: fix patch Inappropriate
  pcmanfm: fix patch Inappropriate
  boost: fix patch Inappropriate
  ca-certifactes: fix patch Inappropriate
  gmp: fix patch Inappropriate
  libtasn1: fix patch Inappropriate
  libffi: fix patch Inappropriate
  serf: fix patch Inappropriate

 .../connman/connman/no-version-scripts.patch| 2 +-
 ...1-Remove-the-warning-about-deprecated-paths-in-schemas.patch | 2 +-
 meta/recipes-core/glib-2.0/glib-2.0/relocate-modules.patch  | 2 +-
 .../glibc/ldconfig-native-2.12.1/no-aux-cache.patch | 2 +-
 meta/recipes-core/readline/readline-8.1/norpath.patch   | 2 +-
 .../util-linux/util-linux/avoid_parallel_tests.patch| 2 +-
 meta/recipes-devtools/autoconf/autoconf/man-host-perl.patch | 2 +-
 meta/recipes-devtools/autoconf/autoconf/no-man.patch| 2 +-
 meta/recipes-devtools/automake/automake/performance.patch   | 2 +-
 ...2-binutils-cross-Do-not-generate-linker-script-directo.patch | 2 +-
 .../binutils/binutils/0015-sync-with-OE-libtool-changes.patch   | 2 +-
 meta/recipes-devtools/e2fsprogs/e2fsprogs/ptest.patch   | 2 +-
 meta/recipes-devtools/e2fsprogs/e2fsprogs/quiet-debugfs.patch   | 2 +-
 meta/recipes-devtools/flex/flex/disable-tests.patch | 2 +-
 .../gcc/gcc/0024-handle-sysroot-support-for-nativesdk-gcc.patch | 2 +-
 meta/recipes-devtools/gdb/gdb/0007-Change-order-of-CFLAGS.patch | 2 +-
 .../libtool/libtool/dont-depend-on-help2man.patch   | 2 +-
 meta/recipes-devtools/libtool/libtool/multilib.patch| 2 +-
 meta/recipes-devtools/libtool/libtool/nohardcodepaths.patch | 2 +-
 meta/recipes-devtools/rsync/files/makefile-no-rebuild.patch | 2 +-
 meta/recipes-devtools/strace/strace/Makefile-ptest.patch| 2 +-
 meta/recipes-devtools/tcf-agent/tcf-agent/ldflags.patch | 2 +-
 .../valgrind-make-ld-XXX.so-strlen-intercept-optional.patch | 2 +-
 .../configure.in-drop-the-test-of-xmllint-and-xsltproc.patch| 2 +-
 meta/recipes-extended/bc/bc/no-gen-libmath.patch| 2 +-
 .../recipes-extended/ethtool/ethtool/avoid_parallel_tests.patch | 2 +-
 meta/recipes-extended/gawk/gawk/test-time.patch | 2 +-
 .../texinfo/texinfo/dont-depend-on-help2man.patch   | 2 +-
 ...01-Relocate-the-repository-directory-for-native-builds.patch | 2 +-
 meta/recipes-gnome/gtk-doc/files/pkg-config-native.patch| 2 +-
 meta/recipes-graphics/xorg-lib/libxcb/disable-check.patch   | 2 +-
 meta/recipes-sato/pcmanfm/files/no-desktop.patch| 2 +-
 ...1-Don-t-set-up-arch-instruction-set-flags-we-do-that-o.patch | 2 +-
 ...1-update-ca-certificates-don-t-use-Debianisms-in-run-p.patch | 2 +-
 ...1-Append-the-user-provided-flags-to-the-auto-detected-.patch | 2 +-
 .../gnutls/libtasn1/dont-depend-on-help2man.patch   | 2 +-
 meta/recipes-support/libffi/libffi/not-win32.patch  | 2 +-
 meta/recipes-support/serf/serf/env.patch| 2 +-
 meta/recipes-support/serf/serf/norpath.patch| 2 +-
 39 files changed, 39 insertions(+), 39 deletions(-)

-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159820): 
https://lists.openembedded.org/g/openembedded-core/message/159820
Mute This Topic: https://lists.openembedded.org/mt/87786835/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [meta-oe][RFC PATCH v2] insane: Inappropriate patch reasoning

[Konrad Weihmann]

On 16.12.21 15:54, Richard Purdie wrote:

On Thu, 2021-12-16 at 13:26 +0100, Konrad Weihmann wrote:

if a patch uses Upstream-Status: Inappropriate it should provide a machine
readable reasoning in square brackets.

According to latest wiki entry that would be

not author
native
licensing
configuration
enable feature
disable feature
bugfix .*
embedded specific
no upstream
other

a detailed reasoning could be provided as part of the commit message,
but format of the metadata line is fixed.

This patch adds a check to insane.bbclass and warns if there is a
non-compliant reasoning given, or none at all.

In a follow-up this should be turned into an error, as it was done
with missing Upstream-Status

Can be skipped with newly added patch-metadata key via INSANE_SKIP

Signed-off-by: Konrad Weihmann 
---
v2: add possibility to skip with patch-metadata in INSANE_SKIP

  meta/classes/insane.bbclass | 25 +
  1 file changed, 25 insertions(+)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 240f3aad62..eae8e0e549 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -1124,6 +1124,8 @@ python do_qa_staging() {
  python do_qa_patch() {
  import subprocess
  
+skip = (d.getVar('INSANE_SKIP') or "").split()

+
  
###
  # Check patch.log for fuzz warnings
  #
@@ -1191,6 +1193,29 @@ python do_qa_patch() {
 bb.error("Malformed Upstream-Status in patch\n%s\nPlease correct 
according to %s :\n%s" % (fullpath, guidelines, match_kinda.group(0)))
 else:
 bb.error("Missing Upstream-Status in patch\n%s\nPlease add 
according to %s ." % (fullpath, guidelines))
+
+   if 'patch-metadata' in skip:
+   continue
+
+   inappr_message_re = r'Inappropriate(\s+\[(?P.*)\])*'
+   inappr_reasons = [
+'not author',
+'native',
+'licensing',
+'configuration',
+'enable feature',
+'disable feature',
+'bugfix .*',
+'embedded specific',
+'no upstream',
+'other',
+   ]
+   for match_inappr in re.finditer(inappr_message_re, content, 
re.IGNORECASE | re.MULTILINE):
+
+   if 'reason' not in match_inappr.groupdict():
+   bb.warning("Missing Upstream-Status: Inappropriate reasoning in 
patch\n%s\nPlease add according to %s ." % (fullpath, guidelines))
+   elif not any(re.match(x, match_inappr.groupdict().get('reason', '') 
or '') for x in inappr_reasons):
+   bb.warning("Malformed Upstream-Status: Inappropriate in 
patch\n%s\nPlease correct according to %s :\n%s" % (fullpath, guidelines, 
match_inappr.group(0)))
  }
  
  python do_qa_configure() {


I appreciate the intent here but I think there are details we need to look at
first.

Whilst this list of inappropriate reasons looks like a good start, I'm not sure
it is exactly the list of things we want to encourage people to mark patches
with. The wiki was written a long time ago and I think we want to make sure it
is right before we go through the work of classifying a large number of patches.
I'm very much in favour of an approach which looks at actions we could take
based upon a given classification.

I'm also worried we actually lose information with any "forced" transition like
this. There may be links to discussion in the current [] field and I'd much
rather keep those links that force people into changing them and them being
lost. Common sense says people would move them to the patch description but that
isn't any guarantee that it would happen.

Also, this isn't really how we go about making transitions like this. Usually
we'd make some decision about a direction and then we'd migrate to it over time.
This patch will start generating hundreds of warnings on the autobuilder to the
point that all warnings would become meaningless. Currently it operates mostly
warning free and where we do hit them, usually intermittently, we get them
fixed. Once things start to fail, they "rot" quickly as one warning turns into
many more unnoticed.

In some ways I'd prefer we add a new field, something other than Inappropriate
and then over time we could classify patches. We'd use the approach we're using
with Pending where we gradually encourage movement over time.

Cheers,

Richard




Fine - I'm also in favor of a transition phase - I just wanted to 
highlight that Inappropriate lost almost all of its meaning as the grep 
output from an earlier mail showed.
One thing I could think about is to at least flag the patch that doesn't 
even provide a reasoning.


At this point we could do the very same for core as for the 
Upstream-Status and do a hard error when not even a one is given.


Then in a second round fix all the misspellings - and maybe

Re: [OE-core] [meta-oe][RFC PATCH v2 1/2] oe.lib.recipeutils: add get_layer_name method

[Konrad Weihmann]

On 16.12.21 15:41, Richard Purdie wrote:

On Wed, 2021-12-15 at 13:12 +0100, Konrad Weihmann wrote:

so one can get the layer name from a filepath

Signed-off-by: Konrad Weihmann 
---
v2: order by path length to correctly map nested layer

  meta/lib/oe/recipeutils.py | 11 ++-
  1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/recipeutils.py b/meta/lib/oe/recipeutils.py
index a0c6974f04..557e0f9bd5 100644
--- a/meta/lib/oe/recipeutils.py
+++ b/meta/lib/oe/recipeutils.py
@@ -21,7 +21,7 @@ import glob
  import bb.tinfoil
  
  from collections import OrderedDict, defaultdict

-from bb.utils import vercmp_string
+from bb.utils import vercmp_string, get_collection_res
  
  # Help us to find places to insert values

  recipe_progression = ['SUMMARY', 'DESCRIPTION', 'AUTHOR', 'HOMEPAGE', 
'BUGTRACKER', 'SECTION', 'LICENSE', 'LICENSE_FLAGS', 'LIC_FILES_CHKSUM', 
'PROVIDES', 'DEPENDS', 'PR', 'PV', 'SRCREV', 'SRCPV', 'SRC_URI', 'S', 
'do_fetch()', 'do_unpack()', 'do_patch()', 'EXTRA_OECONF', 'EXTRA_OECMAKE', 
'EXTRA_OESCONS', 'do_configure()', 'EXTRA_OEMAKE', 'do_compile()', 
'do_install()', 'do_populate_sysroot()', 'INITSCRIPT', 'USERADD', 'GROUPADD', 
'PACKAGES', 'FILES', 'RDEPENDS', 'RRECOMMENDS', 'RSUGGESTS', 'RPROVIDES', 
'RREPLACES', 'RCONFLICTS', 'ALLOW_EMPTY', 'populate_packages()', 
'do_package()', 'do_deploy()', 'BBCLASSEXTEND']
@@ -928,6 +928,15 @@ def find_layerdir(fn):
  return None
  return layerdir
  
+def get_layer_name(fn, d):

+""" Get the layer name from a filename """
+pth = os.path.abspath(fn)
+collection = get_collection_res(d)
+# reverse ordering by length to catch nested layers
+for k, v in dict(sorted(collection.items(), key=lambda item: len(item[1]), 
reverse=True)).items():
+if re.match(v, pth):
+return k
+return ""
  
  def replace_dir_vars(path, d):

  """Replace common directory paths with appropriate variable references (e.g. /etc 
becomes ${sysconfdir})"""



I suspect we should add something in bb.utils for this? I also think we should
probably do this at the bitbake level entirely, i.e. set some kind of variable
to the layername rather than having lots of metadata code trying to do it,
potentially badly?


That would indeed make sense, in the end the could be easily moved to 
bitbake - not sure about the variable per layer as we have already 
BBFILE_PATTERN and BBCOLLECTION (both used here indirectly) and what I 
wanted to have is a way to ask from what layer a particular file 
originates. In the current form it's pretty straight forward as it 
doesn't mind overloads from layers (and therefore no layer priority or 
order from bblayers.conf) - I'm not sure how complicated we would like 
something like this to be - so for now I would be fine if we take this 
piece, add something fully featured in bitbake and then removed this one 
here




Cheers,

Richard



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159790): 
https://lists.openembedded.org/g/openembedded-core/message/159790
Mute This Topic: https://lists.openembedded.org/mt/87742384/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [meta-oe][RFC PATCH v2] insane: Inappropriate patch reasoning

[Konrad Weihmann]

if a patch uses Upstream-Status: Inappropriate it should provide a machine
readable reasoning in square brackets.

According to latest wiki entry that would be

not author
native
licensing
configuration
enable feature
disable feature
bugfix .*
embedded specific
no upstream
other

a detailed reasoning could be provided as part of the commit message,
but format of the metadata line is fixed.

This patch adds a check to insane.bbclass and warns if there is a
non-compliant reasoning given, or none at all.

In a follow-up this should be turned into an error, as it was done
with missing Upstream-Status

Can be skipped with newly added patch-metadata key via INSANE_SKIP

Signed-off-by: Konrad Weihmann 
---
v2: add possibility to skip with patch-metadata in INSANE_SKIP

 meta/classes/insane.bbclass | 25 +
 1 file changed, 25 insertions(+)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 240f3aad62..eae8e0e549 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -1124,6 +1124,8 @@ python do_qa_staging() {
 python do_qa_patch() {
 import subprocess
 
+skip = (d.getVar('INSANE_SKIP') or "").split()
+
 ###
 # Check patch.log for fuzz warnings
 #
@@ -1191,6 +1193,29 @@ python do_qa_patch() {
bb.error("Malformed Upstream-Status in patch\n%s\nPlease 
correct according to %s :\n%s" % (fullpath, guidelines, match_kinda.group(0)))
else:
bb.error("Missing Upstream-Status in patch\n%s\nPlease add 
according to %s ." % (fullpath, guidelines))
+   
+   if 'patch-metadata' in skip:
+   continue
+   
+   inappr_message_re = r'Inappropriate(\s+\[(?P.*)\])*'
+   inappr_reasons = [
+'not author',
+'native',
+'licensing',
+'configuration',
+'enable feature',
+'disable feature',
+'bugfix .*',
+'embedded specific',
+'no upstream',
+'other',
+   ]
+   for match_inappr in re.finditer(inappr_message_re, content, 
re.IGNORECASE | re.MULTILINE):
+
+   if 'reason' not in match_inappr.groupdict():
+   bb.warning("Missing Upstream-Status: Inappropriate reasoning in 
patch\n%s\nPlease add according to %s ." % (fullpath, guidelines))
+   elif not any(re.match(x, match_inappr.groupdict().get('reason', '') 
or '') for x in inappr_reasons):
+   bb.warning("Malformed Upstream-Status: Inappropriate in 
patch\n%s\nPlease correct according to %s :\n%s" % (fullpath, guidelines, 
match_inappr.group(0)))
 }
 
 python do_qa_configure() {
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159784): 
https://lists.openembedded.org/g/openembedded-core/message/159784
Mute This Topic: https://lists.openembedded.org/mt/87765780/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [meta-oe][RFC PATCH v2 1/2] oe.lib.recipeutils: add get_layer_name method

[Konrad Weihmann]

so one can get the layer name from a filepath

Signed-off-by: Konrad Weihmann 
---
v2: order by path length to correctly map nested layer

 meta/lib/oe/recipeutils.py | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/recipeutils.py b/meta/lib/oe/recipeutils.py
index a0c6974f04..557e0f9bd5 100644
--- a/meta/lib/oe/recipeutils.py
+++ b/meta/lib/oe/recipeutils.py
@@ -21,7 +21,7 @@ import glob
 import bb.tinfoil
 
 from collections import OrderedDict, defaultdict
-from bb.utils import vercmp_string
+from bb.utils import vercmp_string, get_collection_res
 
 # Help us to find places to insert values
 recipe_progression = ['SUMMARY', 'DESCRIPTION', 'AUTHOR', 'HOMEPAGE', 
'BUGTRACKER', 'SECTION', 'LICENSE', 'LICENSE_FLAGS', 'LIC_FILES_CHKSUM', 
'PROVIDES', 'DEPENDS', 'PR', 'PV', 'SRCREV', 'SRCPV', 'SRC_URI', 'S', 
'do_fetch()', 'do_unpack()', 'do_patch()', 'EXTRA_OECONF', 'EXTRA_OECMAKE', 
'EXTRA_OESCONS', 'do_configure()', 'EXTRA_OEMAKE', 'do_compile()', 
'do_install()', 'do_populate_sysroot()', 'INITSCRIPT', 'USERADD', 'GROUPADD', 
'PACKAGES', 'FILES', 'RDEPENDS', 'RRECOMMENDS', 'RSUGGESTS', 'RPROVIDES', 
'RREPLACES', 'RCONFLICTS', 'ALLOW_EMPTY', 'populate_packages()', 
'do_package()', 'do_deploy()', 'BBCLASSEXTEND']
@@ -928,6 +928,15 @@ def find_layerdir(fn):
 return None
 return layerdir
 
+def get_layer_name(fn, d):
+""" Get the layer name from a filename """
+pth = os.path.abspath(fn)
+collection = get_collection_res(d)
+# reverse ordering by length to catch nested layers
+for k, v in dict(sorted(collection.items(), key=lambda item: len(item[1]), 
reverse=True)).items():
+if re.match(v, pth):
+return k
+return ""
 
 def replace_dir_vars(path, d):
 """Replace common directory paths with appropriate variable references 
(e.g. /etc becomes ${sysconfdir})"""
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159723): 
https://lists.openembedded.org/g/openembedded-core/message/159723
Mute This Topic: https://lists.openembedded.org/mt/87742384/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [meta-oe][RFC PATCH 2/2] insane: make patch upstream-status configurable

[Konrad Weihmann]

Picking up where Alex left it last time, this proposes a different way 
to make the layers where the Upstream-Status check is run configurable.


Please share your thoughs - esp. about the vardepsexclude, as I so far 
don't see any other option to get rid of the BBFILE_PATTERN creeping 
into the task hash


On 15.12.21 11:03, Konrad Weihmann wrote:

instead of hard coding /meta/ into a skip condition introduce a
variable called QA_PATCH_CHECK_LAYER containing a space
separated list of layer names to perform the upstrem-status check
against.

as this will add BBFILE_PATTERN_* and therefore host specific paths
to the task hash, we need to exclude do_qa_patch function from
task hashing.
As it is a postfunc it should run anyway, once do_patch needs to rerun

Signed-off-by: Konrad Weihmann 
---
  meta/classes/insane.bbclass | 10 --
  1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 240f3aad62..c19cc6f629 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -65,6 +65,8 @@ QA_EMPTY_DIRS ?= " \
  # message if the directory is not empty. If it is not specified for a 
directory,
  # then "but it is expected to be empty" will be used.
  
+QA_PATCH_CHECK_LAYER ?= "core"

+
  def package_qa_clean_path(path, d, pkg=None):
  """
  Remove redundant paths from the path for display.  If pkg isn't set then
@@ -1171,12 +1173,15 @@ python do_qa_patch() {
  # Check if the patch contains a correctly formatted and spelled 
Upstream-Status
  import re
  from oe import patch
+from oe.recipeutils import get_layer_name
+
+check_layer = (d.getVar("QA_PATCH_CHECK_LAYER") or "").split()
  
  for url in patch.src_patches(d):

 (_, _, fullpath, _, _, _) = bb.fetch.decodeurl(url)
  
-   # skip patches not in oe-core

-   if '/meta/' not in fullpath:
+   # skip patches in all but the configured ones
+   if get_layer_name(fullpath, d) not in check_layer:
 continue
  
 content = open(fullpath, encoding='utf-8', errors='ignore').read()

@@ -1298,6 +1303,7 @@ do_populate_sysroot[postfuncs] += "do_qa_staging "
  
  # Check for patch fuzz

  do_patch[postfuncs] += "do_qa_patch "
+do_patch[vardepsexclude] += "do_qa_patch "
  
  # Check broken config.log files, for packages requiring Gettext which

  # don't have it in DEPENDS.






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159719): 
https://lists.openembedded.org/g/openembedded-core/message/159719
Mute This Topic: https://lists.openembedded.org/mt/87741230/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [meta-oe][RFC PATCH 1/2] oe.lib.recipeutils: add get_layer_name method

[Konrad Weihmann]

so one can get the layer name from a filepath

Signed-off-by: Konrad Weihmann 
---
 meta/lib/oe/recipeutils.py | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/recipeutils.py b/meta/lib/oe/recipeutils.py
index a0c6974f04..a0e8840c8f 100644
--- a/meta/lib/oe/recipeutils.py
+++ b/meta/lib/oe/recipeutils.py
@@ -21,7 +21,7 @@ import glob
 import bb.tinfoil
 
 from collections import OrderedDict, defaultdict
-from bb.utils import vercmp_string
+from bb.utils import vercmp_string, get_collection_res
 
 # Help us to find places to insert values
 recipe_progression = ['SUMMARY', 'DESCRIPTION', 'AUTHOR', 'HOMEPAGE', 
'BUGTRACKER', 'SECTION', 'LICENSE', 'LICENSE_FLAGS', 'LIC_FILES_CHKSUM', 
'PROVIDES', 'DEPENDS', 'PR', 'PV', 'SRCREV', 'SRCPV', 'SRC_URI', 'S', 
'do_fetch()', 'do_unpack()', 'do_patch()', 'EXTRA_OECONF', 'EXTRA_OECMAKE', 
'EXTRA_OESCONS', 'do_configure()', 'EXTRA_OEMAKE', 'do_compile()', 
'do_install()', 'do_populate_sysroot()', 'INITSCRIPT', 'USERADD', 'GROUPADD', 
'PACKAGES', 'FILES', 'RDEPENDS', 'RRECOMMENDS', 'RSUGGESTS', 'RPROVIDES', 
'RREPLACES', 'RCONFLICTS', 'ALLOW_EMPTY', 'populate_packages()', 
'do_package()', 'do_deploy()', 'BBCLASSEXTEND']
@@ -928,6 +928,14 @@ def find_layerdir(fn):
 return None
 return layerdir
 
+def get_layer_name(fn, d):
+""" Get the layer name from a filename """
+pth = os.path.abspath(fn)
+collection = get_collection_res(d)
+for k, v in dict(sorted(collection.items(), key=lambda item: item[1], 
reverse=True)).items():
+if re.match(v, pth):
+return k
+return ""
 
 def replace_dir_vars(path, d):
 """Replace common directory paths with appropriate variable references 
(e.g. /etc becomes ${sysconfdir})"""
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159717): 
https://lists.openembedded.org/g/openembedded-core/message/159717
Mute This Topic: https://lists.openembedded.org/mt/87741229/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [meta-oe][RFC PATCH 2/2] insane: make patch upstream-status configurable

[Konrad Weihmann]

instead of hard coding /meta/ into a skip condition introduce a
variable called QA_PATCH_CHECK_LAYER containing a space
separated list of layer names to perform the upstrem-status check
against.

as this will add BBFILE_PATTERN_* and therefore host specific paths
to the task hash, we need to exclude do_qa_patch function from
task hashing.
As it is a postfunc it should run anyway, once do_patch needs to rerun

Signed-off-by: Konrad Weihmann 
---
 meta/classes/insane.bbclass | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 240f3aad62..c19cc6f629 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -65,6 +65,8 @@ QA_EMPTY_DIRS ?= " \
 # message if the directory is not empty. If it is not specified for a 
directory,
 # then "but it is expected to be empty" will be used.
 
+QA_PATCH_CHECK_LAYER ?= "core"
+
 def package_qa_clean_path(path, d, pkg=None):
 """
 Remove redundant paths from the path for display.  If pkg isn't set then
@@ -1171,12 +1173,15 @@ python do_qa_patch() {
 # Check if the patch contains a correctly formatted and spelled 
Upstream-Status
 import re
 from oe import patch
+from oe.recipeutils import get_layer_name
+
+check_layer = (d.getVar("QA_PATCH_CHECK_LAYER") or "").split()
 
 for url in patch.src_patches(d):
(_, _, fullpath, _, _, _) = bb.fetch.decodeurl(url)
 
-   # skip patches not in oe-core
-   if '/meta/' not in fullpath:
+   # skip patches in all but the configured ones
+   if get_layer_name(fullpath, d) not in check_layer:
continue
 
content = open(fullpath, encoding='utf-8', errors='ignore').read()
@@ -1298,6 +1303,7 @@ do_populate_sysroot[postfuncs] += "do_qa_staging "
 
 # Check for patch fuzz
 do_patch[postfuncs] += "do_qa_patch "
+do_patch[vardepsexclude] += "do_qa_patch "
 
 # Check broken config.log files, for packages requiring Gettext which
 # don't have it in DEPENDS.
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159718): 
https://lists.openembedded.org/g/openembedded-core/message/159718
Mute This Topic: https://lists.openembedded.org/mt/87741230/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][PATCH] meta/recipes-bsp: Add recipe for Slim Bootloader

[Konrad Weihmann]

On 15.12.21 08:16, Yongxin Liu wrote:

Slim Bootloader is an open-source boot firmware running on Intel x86
architecture.

Currently it supports qemu, apl(Apollo Lake), cfl(Coffee Lake),
cml(Comet Lake), tgl(Tiger Lake), and ehl(Elkhart Lake). You can set
"SLIMBOOT_TARGET" in .bb file or .bbappend file to specify or add the
target firmware you want, for example: SLIMBOOT_TARGET = "qemu apl".
The default target is qemu.

Generated firmware and security keys are installed in build directory:
 image
 `-- usr
 `-- libexec
 `-- slimboot
 |-- Outputs
 |   |-- qemu
 |   |-- apl
 |   `-- cfl
 `-- keys

Boot firmware for qemu can be used by command:
"qemu-system-x86_64 -machine q35 -nographic -serial mon:stdio -pflash 
SlimBootloader.bin"

Other boot firmware for real hardware cannot be programmed directly to
flash, please refer to https://slimbootloader.github.io/index.html for
more instructions.

Signed-off-by: Yongxin Liu 
---
  ...d-private-key-check-for-PKCS8-format.patch | 28 
  .../build-EDK2-BaseTools-with-bitbake.patch   | 70 +++
  ...limbootloader-BaseTools-with-bitbake.patch | 42 +++
  meta/recipes-bsp/slimboot/slimboot_git.bb | 57 +++
  4 files changed, 197 insertions(+)
  create mode 100644 
meta/recipes-bsp/slimboot/files/add-private-key-check-for-PKCS8-format.patch
  create mode 100644 
meta/recipes-bsp/slimboot/files/build-EDK2-BaseTools-with-bitbake.patch
  create mode 100644 
meta/recipes-bsp/slimboot/files/build-slimbootloader-BaseTools-with-bitbake.patch
  create mode 100644 meta/recipes-bsp/slimboot/slimboot_git.bb

diff --git 
a/meta/recipes-bsp/slimboot/files/add-private-key-check-for-PKCS8-format.patch 
b/meta/recipes-bsp/slimboot/files/add-private-key-check-for-PKCS8-format.patch
new file mode 100644
index 00..a7e9d96dfb
--- /dev/null
+++ 
b/meta/recipes-bsp/slimboot/files/add-private-key-check-for-PKCS8-format.patch
@@ -0,0 +1,28 @@
+From e31e9b34f5ffccaab3788da781e05a0622cf7d45 Mon Sep 17 00:00:00 2001
+From: Yongxin Liu 
+Date: Tue, 7 Dec 2021 16:18:43 +
+Subject: [PATCH] add private key check for PKCS8 format
+
+Upstream-Status: Pending


Pending patches are discouraged, please send them upstream


+
+Signed-off-by: Yongxin Liu 
+---
+ BootloaderCorePkg/Tools/SingleSign.py | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/BootloaderCorePkg/Tools/SingleSign.py 
b/BootloaderCorePkg/Tools/SingleSign.py
+index 2a550bf8..5baacbde 100644
+--- a/BootloaderCorePkg/Tools/SingleSign.py
 b/BootloaderCorePkg/Tools/SingleSign.py
+@@ -259,7 +259,7 @@ def single_sign_gen_pub_key (in_key, pub_key_file = None):
+ cmdline = [get_openssl_path(), 'rsa', '-pubout', '-text', '-noout', 
'-in', '%s' % in_key]
+ # Check if it is public key or private key
+ text = open(in_key, 'r').read()
+-if '-BEGIN RSA PRIVATE KEY-' in text:
++if '-BEGIN RSA PRIVATE KEY-' in text or '-BEGIN PRIVATE KEY-' in text:
+ is_prv_key = True
+ elif '-BEGIN PUBLIC KEY-' in text:
+ cmdline.extend (['-pubin'])
+--
+2.33.0
+
diff --git 
a/meta/recipes-bsp/slimboot/files/build-EDK2-BaseTools-with-bitbake.patch 
b/meta/recipes-bsp/slimboot/files/build-EDK2-BaseTools-with-bitbake.patch
new file mode 100644
index 00..3b9db74491
--- /dev/null
+++ b/meta/recipes-bsp/slimboot/files/build-EDK2-BaseTools-with-bitbake.patch
@@ -0,0 +1,70 @@
+From ce5ed81d9dd9c37e494560edb8f0fbae83713416 Mon Sep 17 00:00:00 2001
+From: Yongxin Liu 
+Date: Wed, 15 Dec 2021 02:19:42 +
+Subject: [PATCH] build EDK2/BaseTools with bitbake
+
+EDK2 basetools are needed by QEMU FSP. Due to CRLF line terminators
+in target file, the patch has to be set to binary format.
+
+Upstream-Status: Inappropriate [oe-core specific]


Why is this oe-specific, I'm missing a proper explanation in the commit 
message



+
+Signed-off-by: Yongxin Liu 
+---
+ .gitattributes   |   1 +
+ .../Tools/PrepareBuildComponentBin.py|   6 ++
+ .../build-edk2-BaseTools-with-bitbake.patch  | Bin 0 -> 1375 bytes
+ 3 files changed, 7 insertions(+)
+ create mode 100644 
Silicon/QemuSocPkg/FspBin/Patches/build-edk2-BaseTools-with-bitbake.patch
+
+diff --git a/.gitattributes b/.gitattributes
+index e372c4d9..d31d5dbb 100644
+--- a/.gitattributes
 b/.gitattributes
+@@ -17,3 +17,4 @@
+ # since the files that are being patched have a mixture of line
+ # endings. This means it must be treated as binary.
+ Silicon/QemuSocPkg/FspBin/Patches/0001-Build-QEMU-FSP-2.0-binaries.patch 
binary
++Silicon/QemuSocPkg/FspBin/Patches/build-edk2-BaseTools-with-bitbake.patch 
binary
+diff --git a/BootloaderCorePkg/Tools/PrepareBuildComponentBin.py 
b/BootloaderCorePkg/Tools/PrepareBuildComponentBin.py
+index 50f24736..2cd2f5d5 100644
+--- a/BootloaderCorePkg/Tools/PrepareBuildComponentBin.py
 

Re: [OE-core] [meta-oe][RFC PATCH] insane: Inappropriate patch reasoning

[Konrad Weihmann]

On 14.12.21 21:11, Alexander Kanavin wrote:
What I would like to see at least is correctly tagging patches with dead 
upstreams, now that we have the shiny new status for it, and a report 
where they can be seen clearly :)


The point of that patch is to have everything more machine readable, so 
we can have your requested stats and any stats one can think of :-) - 
and to my understanding that was the whole point of defining these 
metadata entries.


I think we should head down this road, even if the initial amount of 
changes might be quite high - but in the end we might have some pretty 
automated, fancy stats, that will really provide some insights.


If we don't, we might end up in defining more and more new ids for 
different things already covered by the current approach.




Alex

On Tue, 14 Dec 2021 at 19:48, Konrad Weihmann <mailto:kweihm...@outlook.com>> wrote:




On 14.12.21 19:38, Bruce Ashfield wrote:
 > On Tue, Dec 14, 2021 at 1:05 PM Konrad Weihmann
mailto:kweihm...@outlook.com>> wrote:
 >>
[snip]

 >>
 >>

meta/recipes-support/vte/vte/0001-Makefile.docs-correctly-substitute-gtkdoc-qemu-wrapp.patch:
 >>     6: Upstream-Status: Inappropriate [oe-core specific]
 >>
 >> or are simply ignoring it at all.
 >>
 >> In total that are 352 patches I could indentify in latest master.
 >>
 >> I don't want to open up another can of worms, but either we drop the
 >> reasoning from Inappropriate in the patch guideline completely or we
 >> need to fix that in core (I guess in other layers it is even
worse :) )
 >>
 >> Any thoughts?
 >
 > If we can opt out of the check either in a layer config or a recipe,
 > then the noise can at least be minimized. Since the ecosystem !=
 > oe-core, this level of checking needs to be something that can be
 > opted-out / disabled.
 >
 > Whatever way this goes, I'd suggest that it not merge until the start
 > of the new year .. luckily RP is in full control of when it merges.
 >
 > Because I for one, won't be spending the Chrismtas holidays queuing
 > other people's patches, or fixing status lines in the patches in my
 > layers :)
 >
 > Cheers,
 >
 > Bruce

Actually a very good idea, but I'm not sure how fine grained we want
these package-qa ids to be... @RP any thoughts?

Basically we can solve the issue either
- by keeping the status quo (dropping the reason from the docu)
- or having is according to docu
         - having it as a warning in the first place
         - turning it into an error once we are clean in core and
all layers
hosted on yocto project's infra

    But yeah sure that patch doesn't need to go in this year, as I don't
want to ruins everyone's holidays :-)

 >>
 >> On 14.12.21 18:53, Konrad Weihmann wrote:
 >>> if a patch uses Upstream-Status: Inappropriate it should
provide a machine
 >>> readable reasoning in square brackets.
 >>>
 >>> According to latest wiki entry that would be
 >>>
 >>> not author
 >>> native
 >>> licensing
 >>> configuration
 >>> enable feature
 >>> disable feature
 >>> bugfix .*
 >>> embedded specific
 >>> no upstream
 >>> other
 >>>
 >>> a detailed reasoning could be provided as part of the commit
message,
 >>> but format of the metadata line is fixed.
 >>>
 >>> This patch adds a check to insane.bbclass and warns if there is a
 >>> non-compliant reasoning given, or none at all.
 >>>
 >>> In a follow-up this should be turned into an error, as it was done
 >>> with missing Upstream-Status
 >>>
 >>> Signed-off-by: Konrad Weihmann mailto:kweihm...@outlook.com>>
 >>> ---
 >>>    meta/classes/insane.bbclass | 20 
 >>>    1 file changed, 20 insertions(+)
 >>>
 >>> diff --git a/meta/classes/insane.bbclass
b/meta/classes/insane.bbclass
 >>> index 240f3aad62..da26f4662c 100644
 >>> --- a/meta/classes/insane.bbclass
 >>> +++ b/meta/classes/insane.bbclass
 >>> @@ -1191,6 +1191,26 @@ python do_qa_patch() {
 >>>                   bb.error("Malformed Upstream-Status in
patch\n%s\nPlease correct according to %s :\n%s" % (fullpath,
guidelines, match_kinda.group(0)))
 >>>               else:
 >>>                   bb.erro

Re: [OE-core] [meta-oe][RFC PATCH] insane: Inappropriate patch reasoning

[Konrad Weihmann]

On 14.12.21 19:38, Bruce Ashfield wrote:

On Tue, Dec 14, 2021 at 1:05 PM Konrad Weihmann  wrote:



[snip]



meta/recipes-support/vte/vte/0001-Makefile.docs-correctly-substitute-gtkdoc-qemu-wrapp.patch:
6: Upstream-Status: Inappropriate [oe-core specific]

or are simply ignoring it at all.

In total that are 352 patches I could indentify in latest master.

I don't want to open up another can of worms, but either we drop the
reasoning from Inappropriate in the patch guideline completely or we
need to fix that in core (I guess in other layers it is even worse :) )

Any thoughts?


If we can opt out of the check either in a layer config or a recipe,
then the noise can at least be minimized. Since the ecosystem !=
oe-core, this level of checking needs to be something that can be
opted-out / disabled.

Whatever way this goes, I'd suggest that it not merge until the start
of the new year .. luckily RP is in full control of when it merges.

Because I for one, won't be spending the Chrismtas holidays queuing
other people's patches, or fixing status lines in the patches in my
layers :)

Cheers,

Bruce


Actually a very good idea, but I'm not sure how fine grained we want 
these package-qa ids to be... @RP any thoughts?


Basically we can solve the issue either
- by keeping the status quo (dropping the reason from the docu)
- or having is according to docu
- having it as a warning in the first place
	- turning it into an error once we are clean in core and all layers 
hosted on yocto project's infra


But yeah sure that patch doesn't need to go in this year, as I don't 
want to ruins everyone's holidays :-)




On 14.12.21 18:53, Konrad Weihmann wrote:

if a patch uses Upstream-Status: Inappropriate it should provide a machine
readable reasoning in square brackets.

According to latest wiki entry that would be

not author
native
licensing
configuration
enable feature
disable feature
bugfix .*
embedded specific
no upstream
other

a detailed reasoning could be provided as part of the commit message,
but format of the metadata line is fixed.

This patch adds a check to insane.bbclass and warns if there is a
non-compliant reasoning given, or none at all.

In a follow-up this should be turned into an error, as it was done
with missing Upstream-Status

Signed-off-by: Konrad Weihmann 
---
   meta/classes/insane.bbclass | 20 
   1 file changed, 20 insertions(+)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 240f3aad62..da26f4662c 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -1191,6 +1191,26 @@ python do_qa_patch() {
  bb.error("Malformed Upstream-Status in patch\n%s\nPlease correct 
according to %s :\n%s" % (fullpath, guidelines, match_kinda.group(0)))
  else:
  bb.error("Missing Upstream-Status in patch\n%s\nPlease add 
according to %s ." % (fullpath, guidelines))
+
+   inappr_message_re = r'Inappropriate(\s+\[(?P.*)\])*'
+   inappr_reasons = [
+'not author',
+'native',
+'licensing',
+'configuration',
+'enable feature',
+'disable feature',
+'bugfix .*',
+'embedded specific',
+'no upstream',
+'other',
+   ]
+   for match_inappr in re.finditer(inappr_message_re, content, 
re.IGNORECASE | re.MULTILINE):
+
+   if 'reason' not in match_inappr.groupdict():
+   bb.warning("Missing Upstream-Status: Inappropriate reasoning in 
patch\n%s\nPlease add according to %s ." % (fullpath, guidelines))
+   elif not any(re.match(x, match_inappr.groupdict().get('reason', '') 
or '') for x in inappr_reasons):
+   bb.warning("Malformed Upstream-Status: Inappropriate in 
patch\n%s\nPlease correct according to %s :\n%s" % (fullpath, guidelines, 
match_inappr.group(0)))
   }

   python do_qa_configure() {













-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159705): 
https://lists.openembedded.org/g/openembedded-core/message/159705
Mute This Topic: https://lists.openembedded.org/mt/87726727/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [meta-oe][RFC PATCH] insane: Inappropriate patch reasoning

[Konrad Weihmann]

: Inappropriate [openembedded specific]

meta/recipes-support/gnupg/gnupg/relocate.patch:
  7: Upstream-Status: Inappropriate [OE-specific]

meta/recipes-support/gnutls/libtasn1/dont-depend-on-help2man.patch:
  6: Upstream-Status: Inappropriate

meta/recipes-support/gpgme/gpgme/0001-Revert-build-Make-gpgme.m4-use-gpgrt-config-with-.pc.patch:
  10: Upstream-Status: Inappropriate [oe-core specific]

meta/recipes-support/gpgme/gpgme/0002-gpgme-lang-python-gpg-error-config-should-not-be-use.patch:
  10: Upstream-Status: Inappropriate [changes are specific to OE]

meta/recipes-support/gpgme/gpgme/0003-Correctly-install-python-modules.patch:
  6: Upstream-Status: Inappropriate [oe-core specific]

meta/recipes-support/gpgme/gpgme/0005-gpgme-config-skip-all-lib-or-usr-lib-directories-in-.patch:
  10: Upstream-Status: Inappropriate [oe-core specific]

meta/recipes-support/gpgme/gpgme/0008-do-not-auto-check-var-PYTHON.patch:
  12: Upstream-Status: Inappropriate [oe-core specific]

meta/recipes-support/icu/icu/0001-Disable-LDFLAGSICUDT-for-Linux.patch:
  6: Upstream-Status: Inappropriate [ OE Configuration ]

meta/recipes-support/itstool/itstool/0001-Native-Don-t-use-build-time-hardcoded-python-binary-.patch:
  16: Upstream-Status: Inappropriate [OE specific]

meta/recipes-support/itstool/itstool/0002-Don-t-use-build-time-hardcoded-python-binary-path.patch:
  12: Upstream-Status: Inappropriate [OE specific]

meta/recipes-support/libcap/files/0001-nativesdk-libcap-Raise-the-size-of-arrays-containing.patch:
  14: Upstream-Status: Inappropriate [SDK specific]

meta/recipes-support/libcap/files/0002-tests-do-not-run-target-executables.patch:
  6: Upstream-Status: Inappropriate [oe-core specific]

meta/recipes-support/libevent/libevent/0001-test-regress_dns.c-patch-out-tests-that-require-a-wo.patch:
  10: Upstream-Status: Inappropriate [oe-core specific]

meta/recipes-support/libevent/libevent/Makefile-missing-test-dir.patch:
  10: Upstream-Status: Inappropriate [Other]

meta/recipes-support/libffi/libffi/not-win32.patch:
  16: Upstream-Status: Inappropriate

meta/recipes-support/libgcrypt/files/0001-libgcrypt-fix-m4-file-for-oe-core.patch:
  11: Upstream-Status: Inappropriate [oe-specific]

meta/recipes-support/libgcrypt/files/0003-tests-bench-slope.c-workaround-ICE-failure-on-mips-w.patch:
  47: Upstream-Status: Inappropriate [oe specific]

meta/recipes-support/libgpg-error/libgpg-error/0001-Do-not-fail-when-testing-config-scripts.patch:
  10: Upstream-Status: Inappropriate [oe-core specific]

meta/recipes-support/libgpg-error/libgpg-error/pkgconfig.patch:
  6: Upstream-Status: Inappropriate [gnupg upstream will not take this, 
it has been tried with other components]


meta/recipes-support/libnl/files/enable-serial-tests.patch:
  8: Upstream-Status: Inappropriate [oe-specific]

meta/recipes-support/libssh2/files/0001-Don-t-let-host-enviroment-to-decide-if-a-test-is-bui.patch:
  9: Upstream-Status: Inappropriate[oe specific]

meta/recipes-support/libunwind/libunwind/musl-header-conflict.patch:
  24: Upstream-Status: Inappropriate [really need to fix gcc]

meta/recipes-support/nettle/nettle/Add-target-to-only-build-tests-not-run-them.patch:
  6: Upstream-Status: Inappropriate [not a complete solution]

meta/recipes-support/nghttp2/nghttp2/0001-fetch-ocsp-response-use-python3.patch:
  9: Upstream-Status: Inappropriate [oe specific]

meta/recipes-support/pinentry/pinentry-1.2.0/gpg-error_pkconf.patch:
  9: Upstream-Status: Inappropriate [OE specific]

meta/recipes-support/pinentry/pinentry-1.2.0/libassuan_pkgconf.patch:
  9: Upstream-Status: Inappropriate [OE specific]

meta/recipes-support/serf/serf/env.patch:
  14: Upstream-Status: Inappropriate

meta/recipes-support/serf/serf/norpath.patch:
  4: Upstream-Status: Inappropriate

meta/recipes-support/vim/files/0001-src-Makefile-improve-reproducibility.patch:
  12: Upstream-Status: Inappropriate [oe specific]

meta/recipes-support/vte/vte/0001-Makefile.docs-correctly-substitute-gtkdoc-qemu-wrapp.patch:
  6: Upstream-Status: Inappropriate [oe-core specific]

or are simply ignoring it at all.

In total that are 352 patches I could indentify in latest master.

I don't want to open up another can of worms, but either we drop the 
reasoning from Inappropriate in the patch guideline completely or we

need to fix that in core (I guess in other layers it is even worse :) )

Any thoughts?

On 14.12.21 18:53, Konrad Weihmann wrote:

if a patch uses Upstream-Status: Inappropriate it should provide a machine
readable reasoning in square brackets.

According to latest wiki entry that would be

not author
native
licensing
configuration
enable feature
disable feature
bugfix .*
embedded specific
no upstream
other

a detailed reasoning could be provided as part of the commit message,
but format of the metadata line is fixed.

This patch adds a check to insane.bbclass and warns if there is a
non-compliant reasoning given, or none at all.

In a follow-up this should be turned into an error

[OE-core] [meta-oe][RFC PATCH] insane: Inappropriate patch reasoning

[Konrad Weihmann]

if a patch uses Upstream-Status: Inappropriate it should provide a machine
readable reasoning in square brackets.

According to latest wiki entry that would be

not author
native
licensing
configuration
enable feature
disable feature
bugfix .*
embedded specific
no upstream
other

a detailed reasoning could be provided as part of the commit message,
but format of the metadata line is fixed.

This patch adds a check to insane.bbclass and warns if there is a
non-compliant reasoning given, or none at all.

In a follow-up this should be turned into an error, as it was done
with missing Upstream-Status

Signed-off-by: Konrad Weihmann 
---
 meta/classes/insane.bbclass | 20 
 1 file changed, 20 insertions(+)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 240f3aad62..da26f4662c 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -1191,6 +1191,26 @@ python do_qa_patch() {
bb.error("Malformed Upstream-Status in patch\n%s\nPlease 
correct according to %s :\n%s" % (fullpath, guidelines, match_kinda.group(0)))
else:
bb.error("Missing Upstream-Status in patch\n%s\nPlease add 
according to %s ." % (fullpath, guidelines))
+   
+   inappr_message_re = r'Inappropriate(\s+\[(?P.*)\])*'
+   inappr_reasons = [
+'not author',
+'native',
+'licensing',
+'configuration',
+'enable feature',
+'disable feature',
+'bugfix .*',
+'embedded specific',
+'no upstream',
+'other',
+   ]
+   for match_inappr in re.finditer(inappr_message_re, content, 
re.IGNORECASE | re.MULTILINE):
+
+   if 'reason' not in match_inappr.groupdict():
+   bb.warning("Missing Upstream-Status: Inappropriate reasoning in 
patch\n%s\nPlease add according to %s ." % (fullpath, guidelines))
+   elif not any(re.match(x, match_inappr.groupdict().get('reason', '') 
or '') for x in inappr_reasons):
+   bb.warning("Malformed Upstream-Status: Inappropriate in 
patch\n%s\nPlease correct according to %s :\n%s" % (fullpath, guidelines, 
match_inappr.group(0)))
 }
 
 python do_qa_configure() {
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159701): 
https://lists.openembedded.org/g/openembedded-core/message/159701
Mute This Topic: https://lists.openembedded.org/mt/87726727/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-