Re: [OE-core] [qa-build-notification] QA notification for completed autobuilder build (yocto-4.3.rc2)
Hi All, QA for yocto-4.3.rc2 is completed. This is the full report for this release: https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/tree/?h=intel-yocto-testresults === Summary No high milestone defects. No new issue found. Thanks, Jing Hui > -Original Message- > From: qa-build-notificat...@lists.yoctoproject.org notificat...@lists.yoctoproject.org> On Behalf Of Pokybuild User > Sent: Friday, October 20, 2023 2:56 PM > To: yo...@lists.yoctoproject.org > Cc: qa-build-notificat...@lists.yoctoproject.org > Subject: [qa-build-notification] QA notification for completed autobuilder > build (yocto-4.3.rc2) > > > A build flagged for QA (yocto-4.3.rc2) was completed on the autobuilder > and is available at: > > > https://autobuilder.yocto.io/pub/releases/yocto-4.3.rc2 > > > Build URL: > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6080 > > Build hash information: > > bitbake: 5419a8473d6d4cd1d01537de68ad8d72cf5be0b2 > meta-agl: 4063b4f9a712e32337c1d9678b2f2481dde2a346 > meta-arm: 4bf3246b7b1147e29d42a835631c89279a6ce520 > meta-aws: a30a2b66f1447dc5abdbca6c5de743e39c08b99b > meta-intel: 1bca60610c597571769edc4a057a04bfdbd2f994 > meta-mingw: 65ef95a74f6ae815f63f636ed53e140a26a014ce > meta-openembedded: c66ec8776ec6f467937918a114bdefa28ac036a9 > meta-virtualization: 827092c2ec925ea3a024dcda9ccfd738e351e6ba > oecore: 4c261f8cbdf0c7196a74daad041d04eb093015f3 > poky: 15b576c4101231d248fda7ae0824e1780e1a8901 > > > > This is an automated message from the Yocto Project Autobuilder > Git: git://git.yoctoproject.org/yocto-autobuilder2 > Email: richard.pur...@linuxfoundation.org > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189697): https://lists.openembedded.org/g/openembedded-core/message/189697 Mute This Topic: https://lists.openembedded.org/mt/102127279/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell][PATCH] kexec-tools: Ignore Fedora/RedHat specific CVE-2021-20269
From: Lee Chee Yang
Signed-off-by: Lee Chee Yang
---
meta/recipes-kernel/kexec/kexec-tools_2.0.20.bb | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/recipes-kernel/kexec/kexec-tools_2.0.20.bb
b/meta/recipes-kernel/kexec/kexec-tools_2.0.20.bb
index 871b36440f..206c6ccae7 100644
--- a/meta/recipes-kernel/kexec/kexec-tools_2.0.20.bb
+++ b/meta/recipes-kernel/kexec/kexec-tools_2.0.20.bb
@@ -30,6 +30,9 @@ inherit autotools update-rc.d systemd
export LDFLAGS = "-L${STAGING_LIBDIR}"
EXTRA_OECONF = " --with-zlib=yes"
+# affects kexec-tools shipped by Fedora versions prior to 2.0.21-8 and RHEL
versions prior to 2.0.20-47.
+CVE_CHECK_WHITELIST += "CVE-2021-20269"
+
do_compile_prepend() {
# Remove the prepackaged config.h from the source tree as it overrides
# the same file generated by configure and placed in the build tree
--
2.37.3
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189696):
https://lists.openembedded.org/g/openembedded-core/message/189696
Mute This Topic: https://lists.openembedded.org/mt/102194515/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] kernel: Commit without running hooks
On Wed, Oct 25, 2023 at 6:37 PM William A. Kennington III via
lists.openembedded.org wrote:
>
> The hooks are pulled from the impure environment and are often broken in
> our environments. There is no reason to add extra metadata or verify the
> commit message as its arbitrary to turn the tarball into a git repo.
But what about the other uses of git ? If the hooks are broken during
the creation step, other uses of git should also be broken.
We've had the ability to check the git config for quite some time, we
could take it further an inhibit the use of the host config if we are
worried about breakage like this.
Bruce
>
> Signed-off-by: William A. Kennington III
> ---
> meta/classes-recipe/kernel-yocto.bbclass | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-recipe/kernel-yocto.bbclass
> b/meta/classes-recipe/kernel-yocto.bbclass
> index 4ac977b122..cb9cd26b09 100644
> --- a/meta/classes-recipe/kernel-yocto.bbclass
> +++ b/meta/classes-recipe/kernel-yocto.bbclass
> @@ -408,7 +408,7 @@ do_kernel_checkout() {
> git init
> check_git_config
> git add .
> - git commit -q -m "baseline commit: creating repo for
> ${PN}-${PV}"
> + git commit -q -n -m "baseline commit: creating repo for
> ${PN}-${PV}"
> git clean -d -f
> fi
>
> --
> 2.42.0.820.g83a721a137-goog
>
>
>
>
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189695):
https://lists.openembedded.org/g/openembedded-core/message/189695
Mute This Topic: https://lists.openembedded.org/mt/102189231/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] kernel: Commit without running hooks
The hooks are pulled from the impure environment and are often broken in
our environments. There is no reason to add extra metadata or verify the
commit message as its arbitrary to turn the tarball into a git repo.
Signed-off-by: William A. Kennington III
---
meta/classes-recipe/kernel-yocto.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes-recipe/kernel-yocto.bbclass
b/meta/classes-recipe/kernel-yocto.bbclass
index 4ac977b122..cb9cd26b09 100644
--- a/meta/classes-recipe/kernel-yocto.bbclass
+++ b/meta/classes-recipe/kernel-yocto.bbclass
@@ -408,7 +408,7 @@ do_kernel_checkout() {
git init
check_git_config
git add .
- git commit -q -m "baseline commit: creating repo for
${PN}-${PV}"
+ git commit -q -n -m "baseline commit: creating repo for
${PN}-${PV}"
git clean -d -f
fi
--
2.42.0.820.g83a721a137-goog
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189694):
https://lists.openembedded.org/g/openembedded-core/message/189694
Mute This Topic: https://lists.openembedded.org/mt/102189231/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] kernel.bbclass: Use strip utility used for kernel build in do_package
On Tue, 2023-10-24 at 19:17 -0400, Bruce Ashfield wrote:
> On Tue, Oct 24, 2023 at 6:07 PM Khem Raj wrote:
> >
> > os.environ does not pass this down to runstrip() function and in
> > strip_execs() its using STRIP bitbake variable to find the strip utility
> > to use. Since there might be a trailing whitespace in KERNEL_STRIP
> > remove that otherwise python is not able to launch it.
> > e.g.
> >
> > FileNotFoundError: [Errno 2] No such file or directory:
> > 'riscv64-yoe-linux-strip '
> >
> > This is more evident when STRIP and KERNEL_STRIP are different utilities
> > e.g. when using clang as default toolchain but using gcc+binutils only for
> > kernel build.
> >
> > Signed-off-by: Khem Raj
> > Cc: Bruce Ashfield
> > ---
> > meta/classes-recipe/kernel.bbclass | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/meta/classes-recipe/kernel.bbclass
> > b/meta/classes-recipe/kernel.bbclass
> > index 2ec9ea2091e..16b85dbca48 100644
> > --- a/meta/classes-recipe/kernel.bbclass
> > +++ b/meta/classes-recipe/kernel.bbclass
> > @@ -336,7 +336,7 @@ kernel_do_transform_bundled_initramfs() {
> > do_transform_bundled_initramfs[dirs] = "${B}"
> >
> > python do_package:prepend () {
> > -os.environ['STRIP'] = d.getVar('KERNEL_STRIP')
> > +d.setVar('STRIP', d.getVar('KERNEL_STRIP').strip())
>
> Looks fine to me.
>
> I can't say that I know the mechanics if every setVar'd variable makes it into
> the environment to be available for the build process .. but since it is
> working
> for you, I assume it does :)
FWIW, only variables that have their export flag set or which exported
at some point, which STRIP presumably does or is.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189693):
https://lists.openembedded.org/g/openembedded-core/message/189693
Mute This Topic: https://lists.openembedded.org/mt/102167569/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core][PATCH] patchtest/selftest: add XSKIP, update test files
Since we are skipping the merge test, two of the selftests now report
SKIP instead of XPASS/XFAIL as expected. Adjust the two files to have
the right endings for XSKIP, and add the category so that it can be used
for more extensive testing in the future.
Signed-off-by: Trevor Gamblin
---
...ad.pass => Merge.test_series_merge_on_head.1.skip} | 0
...ad.fail => Merge.test_series_merge_on_head.2.skip} | 0
meta/lib/patchtest/selftest/selftest | 11 ---
3 files changed, 8 insertions(+), 3 deletions(-)
rename meta/lib/patchtest/selftest/files/{Merge.test_series_merge_on_head.pass
=> Merge.test_series_merge_on_head.1.skip} (100%)
rename meta/lib/patchtest/selftest/files/{Merge.test_series_merge_on_head.fail
=> Merge.test_series_merge_on_head.2.skip} (100%)
diff --git
a/meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.pass
b/meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.1.skip
similarity index 100%
rename from
meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.pass
rename to
meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.1.skip
diff --git
a/meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.fail
b/meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.2.skip
similarity index 100%
rename from
meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.fail
rename to
meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.2.skip
diff --git a/meta/lib/patchtest/selftest/selftest
b/meta/lib/patchtest/selftest/selftest
index 006d1badf83..6fad50ce616 100755
--- a/meta/lib/patchtest/selftest/selftest
+++ b/meta/lib/patchtest/selftest/selftest
@@ -18,14 +18,15 @@ parentdir = os.path.dirname(topdir)
# path to the repo root
repodir = os.path.dirname(os.path.dirname(parentdir))
-def print_results(passcount, skipcount, failcount, xpasscount, xfailcount,
errorcount):
-total = passcount + skipcount + failcount + xpasscount + xfailcount +
errorcount
+def print_results(passcount, failcount, skipcount, xpasscount, xfailcount,
xskipcount, errorcount):
+total = passcount + skipcount + failcount + xpasscount + xfailcount +
xskipcount + errorcount
print("")
print("Testsuite summary for %s" % os.path.basename(topdir))
print("")
print("# TOTAL: %s" % str(total))
print("# XPASS: %s" % str(xpasscount))
print("# XFAIL: %s" % str(xfailcount))
+print("# XSKIP: %s" % str(xskipcount))
print("# PASS: %s" % str(passcount))
print("# FAIL: %s" % str(failcount))
print("# SKIP: %s" % str(skipcount))
@@ -48,6 +49,7 @@ if __name__ == '__main__':
skipcount = 0
xpasscount = 0
xfailcount = 0
+xskipcount = 0
errorcount = 0
results = None
@@ -71,6 +73,9 @@ if __name__ == '__main__':
elif expected_result.upper() == "PASS" and result.upper()
== "PASS":
xpasscount = xpasscount + 1
print("XPASS: %s (file: %s)" % (testid.strip("."),
os.path.basename(patch)))
+elif expected_result.upper() == "SKIP" and result.upper()
== "SKIP":
+xskipcount = xskipcount + 1
+print("XSKIP: %s (file: %s)" % (testid.strip("."),
os.path.basename(patch)))
else:
print("%s: %s (%s)" % (result.upper(),
testid.strip("."), os.path.basename(patch)))
if result.upper() == "PASS":
@@ -86,4 +91,4 @@ if __name__ == '__main__':
else:
print ("No test for=%s" % patch)
-print_results(passcount, skipcount, failcount, xpasscount, xfailcount,
errorcount)
+print_results(passcount, failcount, skipcount, xpasscount, xfailcount,
xskipcount, errorcount)
--
2.41.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189692):
https://lists.openembedded.org/g/openembedded-core/message/189692
Mute This Topic: https://lists.openembedded.org/mt/102182400/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [RESEND PATCH 2/2] oeqa/selftest/devtool: remove spaces on empty line
Le mer. 25 oct. 2023 à 17:01, Luca Ceresoli
a écrit :
>
> Hello Julien,
>
> On Tue, 24 Oct 2023 16:17:54 +0200
> "Julien Stephan" wrote:
>
> > Signed-off-by: Julien Stephan
> > ---
> > meta/lib/oeqa/selftest/cases/devtool.py | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/meta/lib/oeqa/selftest/cases/devtool.py
> > b/meta/lib/oeqa/selftest/cases/devtool.py
> > index c36b1efa78b..b5c488be8e8 100644
> > --- a/meta/lib/oeqa/selftest/cases/devtool.py
> > +++ b/meta/lib/oeqa/selftest/cases/devtool.py
> > @@ -575,7 +575,7 @@ class DevtoolAddTests(DevtoolBase):
> > checkvars['S'] = '${WORKDIR}/MarkupSafe-%s' % testver
> > checkvars['SRC_URI'] = url
> > self._test_recipe_contents(recipefile, checkvars, [])
> > -
> > +
>
> This patch does not apply due to the '-' line above having no spaces
> after '-'. Perhaps your mailer mangled the content removing training
> whitespace (which, ironically, is what the patch itself aims at doing!).
>
> Using 'git send-email' is usually the best way to send patches without
> this sort of issues.
>
Hi Luca
My editor mangled it.. I just resent it again.
Thank you
Cheers
Julien
> Luca
>
> --
> Luca Ceresoli, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189691):
https://lists.openembedded.org/g/openembedded-core/message/189691
Mute This Topic: https://lists.openembedded.org/mt/102158568/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core][RESEND][PATCH 2/2] oeqa/selftest/devtool: remove spaces on empty line
Signed-off-by: Julien Stephan
---
meta/lib/oeqa/selftest/cases/devtool.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py
b/meta/lib/oeqa/selftest/cases/devtool.py
index c36b1efa78b..b5c488be8e8 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -575,7 +575,7 @@ class DevtoolAddTests(DevtoolBase):
checkvars['S'] = '${WORKDIR}/MarkupSafe-%s' % testver
checkvars['SRC_URI'] = url
self._test_recipe_contents(recipefile, checkvars, [])
-
+
def test_devtool_add_fetch_git(self):
tempdir = tempfile.mkdtemp(prefix='devtoolqa')
self.track_for_cleanup(tempdir)
--
2.42.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189690):
https://lists.openembedded.org/g/openembedded-core/message/189690
Mute This Topic: https://lists.openembedded.org/mt/102181567/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core][RESEND][PATCH 1/2] oeqa/selftest/devtool: abort if a local workspace already exist
if user run devtool selftests with a local workspacelayer
the tests fail with various error such as:
- devtool.DevtoolAddTests.test_devtool_add just hangs
- devtool.DevtoolModifyTests.* fail with the following error:
ERROR: Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check
bblayers.conf or layer.conf to fix it.
Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or
layer.conf to fix it.
Check if a workspacelayer exists, warn the user and abort the tests
Signed-off-by: Julien Stephan
---
meta/lib/oeqa/selftest/cases/devtool.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py
b/meta/lib/oeqa/selftest/cases/devtool.py
index b577f6d62a1..c36b1efa78b 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -27,6 +27,9 @@ def setUpModule():
corecopydir = os.path.join(templayerdir, 'core-copy')
bblayers_conf = os.path.join(os.environ['BUILDDIR'], 'conf',
'bblayers.conf')
edited_layers = []
+# make sure user doesn't have a local workspace
+result = runCmd('bitbake-layers show-layers')
+assert "workspacelayer" not in result.output, "Devtool test suite cannot
be run with a local workspace directory"
# We need to take a copy of the meta layer so we can modify it and not
# have any races against other tests that might be running in parallel
--
2.42.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189689):
https://lists.openembedded.org/g/openembedded-core/message/189689
Mute This Topic: https://lists.openembedded.org/mt/102181566/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v4 5/5] oeqa/selftest/recipetool: add selftest for PEP-517 recipe creation
Add 3 tests to check the creation of PEP-517 project using the 3
backends supported by bitbake:
- setuptools.build_meta
- poetry.core.masonry.api
- flit_core.buildapi
Theses tests requires the tomllib python module, so skip theses tests
if module is not present. tomllib module is part of python starting from 3.11
Signed-off-by: Julien Stephan
---
meta/lib/oeqa/selftest/cases/recipetool.py | 102 +
1 file changed, 102 insertions(+)
diff --git a/meta/lib/oeqa/selftest/cases/recipetool.py
b/meta/lib/oeqa/selftest/cases/recipetool.py
index d3aea74228f..8e0fc995f7e 100644
--- a/meta/lib/oeqa/selftest/cases/recipetool.py
+++ b/meta/lib/oeqa/selftest/cases/recipetool.py
@@ -474,6 +474,108 @@ class RecipetoolCreateTests(RecipetoolBase):
inherits = ['setuptools3']
self._test_recipe_contents(recipefile, checkvars, inherits)
+def test_recipetool_create_python3_pep517_setuptools_build_meta(self):
+# This test require python 3.11 or above for the tomllib module
+# or tomli module to be installed
+try:
+import tomllib
+except ImportError:
+try:
+import tomli
+except ImportError:
+self.skipTest('Test requires python 3.11 or above for tomllib
module or tomli module')
+
+# Test creating python3 package from tarball (using
setuptools.build_meta class)
+temprecipe = os.path.join(self.tempdir, 'recipe')
+os.makedirs(temprecipe)
+pn = 'webcolors'
+pv = '1.13'
+recipefile = os.path.join(temprecipe, 'python3-%s_%s.bb' % (pn, pv))
+srcuri =
'https://files.pythonhosted.org/packages/a1/fb/f95560c6a5d4469d9c49e24cf1b5d4d21ffab5608251c6020a965fb7791c/%s-%s.tar.gz'
% (pn, pv)
+result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
+self.assertTrue(os.path.isfile(recipefile))
+checkvars = {}
+checkvars['SUMMARY'] = 'A library for working with the color formats
defined by HTML and CSS.'
+checkvars['LICENSE'] = set(['BSD-3-Clause'])
+checkvars['LIC_FILES_CHKSUM'] =
'file://LICENSE;md5=702b1ef12cf66832a88f24c8f2ee9c19'
+checkvars['SRC_URI'] =
'https://files.pythonhosted.org/packages/a1/fb/f95560c6a5d4469d9c49e24cf1b5d4d21ffab5608251c6020a965fb7791c/webcolors-${PV}.tar.gz'
+checkvars['SRC_URI[md5sum]'] = 'c9be30c5b0cf1cad32e4cbacbb2229e9'
+checkvars['SRC_URI[sha1sum]'] =
'c90b84fb65eed9b4c9dea7f08c657bfac0e820a5'
+checkvars['SRC_URI[sha256sum]'] =
'c225b674c83fa923be93d235330ce0300373d02885cef23238813b0d5668304a'
+checkvars['SRC_URI[sha384sum]'] =
'45652af349660f19f68d01361dd5bda287789e5ea63608f52a8cea526ac04465614db2ea236103fb8456b1fcaea96ed7'
+checkvars['SRC_URI[sha512sum]'] =
'074aaf135ac6b0025b88b731d1d6dfa4c539b4fff7195658cc58a4326bb9f0449a231685d312b4a1ec48ca535a838bfa5c680787fe0e61473a2a092c448937d0'
+inherits = ['python_setuptools_build_meta']
+
+self._test_recipe_contents(recipefile, checkvars, inherits)
+
+def test_recipetool_create_python3_pep517_poetry_core_masonry_api(self):
+# This test require python 3.11 or above for the tomllib module
+# or tomli module to be installed
+try:
+import tomllib
+except ImportError:
+try:
+import tomli
+except ImportError:
+self.skipTest('Test requires python 3.11 or above for tomllib
module or tomli module')
+
+# Test creating python3 package from tarball (using
poetry.core.masonry.api class)
+temprecipe = os.path.join(self.tempdir, 'recipe')
+os.makedirs(temprecipe)
+pn = 'iso8601'
+pv = '2.1.0'
+recipefile = os.path.join(temprecipe, 'python3-%s_%s.bb' % (pn, pv))
+srcuri =
'https://files.pythonhosted.org/packages/b9/f3/ef59cee614d5e0accf6fd0cbba025b93b272e626ca89fb70a3e9187c5d15/%s-%s.tar.gz'
% (pn, pv)
+result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
+self.assertTrue(os.path.isfile(recipefile))
+checkvars = {}
+checkvars['SUMMARY'] = 'Simple module to parse ISO 8601 dates'
+checkvars['LICENSE'] = set(['MIT'])
+checkvars['LIC_FILES_CHKSUM'] =
'file://LICENSE;md5=aab31f2ef7ba214a5a341eaa47a7f367'
+checkvars['SRC_URI'] =
'https://files.pythonhosted.org/packages/b9/f3/ef59cee614d5e0accf6fd0cbba025b93b272e626ca89fb70a3e9187c5d15/iso8601-${PV}.tar.gz'
+checkvars['SRC_URI[md5sum]'] = '6e33910eba87066b3be7fcf3d59d16b5'
+checkvars['SRC_URI[sha1sum]'] =
'efd225b2c9fa7d9e4a1ec6ad94f3295cee982e61'
+checkvars['SRC_URI[sha256sum]'] =
'6b1d3829ee8921c4301998c909f7829fa9ed3cbdac0d3b16af2d743aed1ba8df'
+checkvars['SRC_URI[sha384sum]'] =
'255002433fe65c19adfd6b91494271b613cb25ef6a35ac77436de1e03d60cc07bf89fd716451b917f1435e4384860ef6'
+checkvars['SRC_URI[sha512sum]'] =
Re: [OE-core] [PATCH v3 0/4] devtool/recipetool: add support of PEP-517
Le mer. 25 oct. 2023 à 16:11, Richard Purdie a écrit : > > On Tue, 2023-10-24 at 16:40 +0200, Julien Stephan wrote: > > Hello all, > > > > This series adds basic support for PEP-517 in recipetool/devtool, plus somes > > others fixes. > > > > Recipetool/devtool selftest passed locally > > > > Missing features: > > - extra or optionnal dependencies are not handled > > - version number for dependencies are not handled (issue already present > > for the legacy setuptool) > > - some projects can declare a minimal pyproject.toml file, and put all > > the metadata in setup.py/setup.cfg/requirements.txt .. theses cases > > are not handled. If a pyproject.toml file is found, assumes it has all > > necessary metadata. > > > > Changes from v2: > > - try to use tomllib from python core (since 3.11) and if not available, > > try to import tomli (instead of toml, as tomllib and tomli have both the > > same api) > > - improve support for poetry: poetry looks for metadata into the > > "tools.poetry" section instead of the "project" section > > - add the selftests for the 3 supported backends. Skip the tests if > > tomliib (python > 3.11) or tomli is not installed > > - fix self test failing with the new "python3-" prefix added to > > recipetool for python recipes > > This patch series still doesn't seem to apply against master. Could you > check your rebase is ok or if there is a patch missing or something? Hi Richard, Yes, I forgot the first patch of the series. Just sent a v4 fixing this. Thank you Cheers Julien > > Cheers, > > Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189688): https://lists.openembedded.org/g/openembedded-core/message/189688 Mute This Topic: https://lists.openembedded.org/mt/102158953/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v4 4/5] scripts:recipetool:create_buildsys_python: add PEP517 support
Add basic support for PEP517 [1] for the 3 following backends that are
supported by bitbake:
- setuptools.build_meta
- poetry.core.masonry.api
- flit_core.buildapi
If a pyproject.toml file is found, use it to create the recipe,
otherwise fallback to the old setup.py method.
Some projects can declare a minimal pyproject.toml file, and put all
the metadata in setup.py/setup.cfg/requirements.txt .. theses cases
are not handled. If a pyproject.toml file is found, assumes it has all
necessary metadata.
As for the old setup.py method, version numbers for dependencies are not
handled.
Some features may be missing, such as the extra dependencies.
[YOCTO #14737]
[1]: https://peps.python.org/pep-0517/
Signed-off-by: Julien Stephan
---
.../lib/recipetool/create_buildsys_python.py | 268 +-
1 file changed, 267 insertions(+), 1 deletion(-)
diff --git a/scripts/lib/recipetool/create_buildsys_python.py
b/scripts/lib/recipetool/create_buildsys_python.py
index 69f6f5ca511..9e7f22c0db0 100644
--- a/scripts/lib/recipetool/create_buildsys_python.py
+++ b/scripts/lib/recipetool/create_buildsys_python.py
@@ -656,6 +656,270 @@ class PythonSetupPyRecipeHandler(PythonRecipeHandler):
handled.append('buildsystem')
+class PythonPyprojectTomlRecipeHandler(PythonRecipeHandler):
+"""Base class to support PEP517 and PEP518
+
+PEP517 https://peps.python.org/pep-0517/#source-trees
+PEP518 https://peps.python.org/pep-0518/#build-system-table
+"""
+# bitbake currently support the 3 following backends
+build_backend_map = {
+"setuptools.build_meta": "python_setuptools_build_meta",
+"poetry.core.masonry.api": "python_poetry_core",
+"flit_core.buildapi": "python_flit_core",
+}
+
+# setuptools.build_meta and flit declare project metadata into the
"project" section of pyproject.toml
+# according to PEP-621:
https://packaging.python.org/en/latest/specifications/declaring-project-metadata/#declaring-project-metadata
+# while poetry uses the "tool.poetry" section according to its official
documentation: https://python-poetry.org/docs/pyproject/
+# keys from "project" and "tool.poetry" sections are almost the same
except for the HOMEPAGE which is "homepage" for tool.poetry
+# and "Homepage" for "project" section. So keep both
+bbvar_map = {
+"name": "PN",
+"version": "PV",
+"Homepage": "HOMEPAGE",
+"homepage": "HOMEPAGE",
+"description": "SUMMARY",
+"license": "LICENSE",
+"dependencies": "RDEPENDS:${PN}",
+"requires": "DEPENDS",
+}
+
+replacements = [
+("license", r" +$", ""),
+("license", r"^ +", ""),
+("license", r" ", "-"),
+("license", r"^GNU-", ""),
+("license", r"-[Ll]icen[cs]e(,?-[Vv]ersion)?", ""),
+("license", r"^UNKNOWN$", ""),
+# Remove currently unhandled version numbers from these variables
+("requires", r"\[[^\]]+\]$", ""),
+("requires", r"^([^><= ]+).*", r"\1"),
+("dependencies", r"\[[^\]]+\]$", ""),
+("dependencies", r"^([^><= ]+).*", r"\1"),
+]
+
+excluded_native_pkgdeps = [
+# already provided by python_setuptools_build_meta.bbclass
+"python3-setuptools-native",
+"python3-wheel-native",
+# already provided by python_poetry_core.bbclass
+"python3-poetry-core-native",
+# already provided by python_flit_core.bbclass
+"python3-flit-core-native",
+]
+
+# add here a list of known and often used packages and the corresponding
bitbake package
+known_deps_map = {
+"setuptools": "python3-setuptools",
+"wheel": "python3-wheel",
+"poetry-core": "python3-poetry-core",
+"flit_core": "python3-flit-core",
+"setuptools-scm": "python3-setuptools-scm",
+}
+
+def __init__(self):
+pass
+
+def process(self, srctree, classes, lines_before, lines_after, handled,
extravalues):
+info = {}
+
+if 'buildsystem' in handled:
+return False
+
+# Check for non-zero size setup.py files
+setupfiles = RecipeHandler.checkfiles(srctree, ["pyproject.toml"])
+for fn in setupfiles:
+if os.path.getsize(fn):
+break
+else:
+return False
+
+setupscript = os.path.join(srctree, "pyproject.toml")
+
+try:
+try:
+import tomllib
+except ImportError:
+try:
+import tomli as tomllib
+except ImportError:
+logger.exception("Neither 'tomllib' nor 'tomli' could be
imported. Please use python3.11 or above or install tomli module")
+return False
+except Exception:
+logger.exception("Failed to parse pyproject.toml")
+return False
+
+with open(setupscript, "rb") as
[OE-core] [PATCH v4 0/5] devtool/recipetool: add support of PEP-517
Hello all, This series adds basic support for PEP-517 in recipetool/devtool, plus somes others fixes. Recipetool/devtool selftest passed locally Missing features: - extra or optionnal dependencies are not handled - version number for dependencies are not handled (issue already present for the legacy setuptool) - some projects can declare a minimal pyproject.toml file, and put all the metadata in setup.py/setup.cfg/requirements.txt .. theses cases are not handled. If a pyproject.toml file is found, assumes it has all necessary metadata. Changes from v3 - add missing patch Changes from v2: - try to use tomllib from python core (since 3.11) and if not available, try to import tomli (instead of toml, as tomllib and tomli have both the same api) - improve support for poetry: poetry looks for metadata into the "tools.poetry" section instead of the "project" section - add the selftests for the 3 supported backends. Skip the tests if tomliib (python > 3.11) or tomli is not installed - fix self test failing with the new "python3-" prefix added to recipetool for python recipes Julien Stephan (5): scripts:recipetool:create_buildsys_python: fix license note scripts:recipetool:create_buildsys_python: prefix created recipes with python3- scripts:recipetool:create_buildsys_python: refactor code for futur PEP517 addition scripts:recipetool:create_buildsys_python: add PEP517 support oeqa/selftest/recipetool: add selftest for PEP-517 recipe creation meta/lib/oeqa/selftest/cases/recipetool.py| 106 +- .../lib/recipetool/create_buildsys_python.py | 1019 +++-- 2 files changed, 760 insertions(+), 365 deletions(-) -- 2.42.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189684): https://lists.openembedded.org/g/openembedded-core/message/189684 Mute This Topic: https://lists.openembedded.org/mt/102181148/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v4 1/5] scripts:recipetool:create_buildsys_python: fix license note
License field of setup is not always standardized, so we usually use the
classifier to determine the correct license format to use in the recipe.
A warning note is added above the LICENSE field of the create recipe
in case a license is provided in setup. But when the plugin is called,
"LICENSE =" is not yet present so we can never display this note.
Replace the "LICENSE =" condition with "##LICENSE_PLACEHOLDER##"
to actually be able to display the note message
Signed-off-by: Julien Stephan
---
scripts/lib/recipetool/create_buildsys_python.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/lib/recipetool/create_buildsys_python.py
b/scripts/lib/recipetool/create_buildsys_python.py
index 92468b22546..321d0ba257d 100644
--- a/scripts/lib/recipetool/create_buildsys_python.py
+++ b/scripts/lib/recipetool/create_buildsys_python.py
@@ -254,7 +254,7 @@ class PythonRecipeHandler(RecipeHandler):
if license_str:
for i, line in enumerate(lines_before):
-if line.startswith('LICENSE = '):
+if line.startswith('##LICENSE_PLACEHOLDER##'):
lines_before.insert(i, '# NOTE: License in
setup.py/PKGINFO is: %s' % license_str)
break
--
2.42.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189682):
https://lists.openembedded.org/g/openembedded-core/message/189682
Mute This Topic: https://lists.openembedded.org/mt/102181146/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v4 3/5] scripts:recipetool:create_buildsys_python: refactor code for futur PEP517 addition
In order to prepare the support for pyproject.toml (PEP517 [1]) enabled
projects, refactor the code and move setup.py specific code into a
specific class in order to allow sharing the PythonRecipeHandler class
No functionnal changes expected
[1]: https://peps.python.org/pep-0517/#source-tree
Signed-off-by: Julien Stephan
---
.../lib/recipetool/create_buildsys_python.py | 748 +-
1 file changed, 385 insertions(+), 363 deletions(-)
diff --git a/scripts/lib/recipetool/create_buildsys_python.py
b/scripts/lib/recipetool/create_buildsys_python.py
index 502e1dfbc3d..69f6f5ca511 100644
--- a/scripts/lib/recipetool/create_buildsys_python.py
+++ b/scripts/lib/recipetool/create_buildsys_python.py
@@ -37,63 +37,8 @@ class PythonRecipeHandler(RecipeHandler):
assume_provided = ['builtins', 'os.path']
# Assumes that the host python3 builtin_module_names is sane for target too
assume_provided = assume_provided + list(sys.builtin_module_names)
+excluded_fields = []
-bbvar_map = {
-'Name': 'PN',
-'Version': 'PV',
-'Home-page': 'HOMEPAGE',
-'Summary': 'SUMMARY',
-'Description': 'DESCRIPTION',
-'License': 'LICENSE',
-'Requires': 'RDEPENDS:${PN}',
-'Provides': 'RPROVIDES:${PN}',
-'Obsoletes': 'RREPLACES:${PN}',
-}
-# PN/PV are already set by recipetool core & desc can be extremely long
-excluded_fields = [
-'Description',
-]
-setup_parse_map = {
-'Url': 'Home-page',
-'Classifiers': 'Classifier',
-'Description': 'Summary',
-}
-setuparg_map = {
-'Home-page': 'url',
-'Classifier': 'classifiers',
-'Summary': 'description',
-'Description': 'long-description',
-}
-# Values which are lists, used by the setup.py argument based metadata
-# extraction method, to determine how to process the setup.py output.
-setuparg_list_fields = [
-'Classifier',
-'Requires',
-'Provides',
-'Obsoletes',
-'Platform',
-'Supported-Platform',
-]
-setuparg_multi_line_values = ['Description']
-replacements = [
-('License', r' +$', ''),
-('License', r'^ +', ''),
-('License', r' ', '-'),
-('License', r'^GNU-', ''),
-('License', r'-[Ll]icen[cs]e(,?-[Vv]ersion)?', ''),
-('License', r'^UNKNOWN$', ''),
-
-# Remove currently unhandled version numbers from these variables
-('Requires', r' *\([^)]*\)', ''),
-('Provides', r' *\([^)]*\)', ''),
-('Obsoletes', r' *\([^)]*\)', ''),
-('Install-requires', r'^([^><= ]+).*', r'\1'),
-('Extras-require', r'^([^><= ]+).*', r'\1'),
-('Tests-require', r'^([^><= ]+).*', r'\1'),
-
-# Remove unhandled dependency on particular features (e.g. foo[PDF])
-('Install-requires', r'\[[^\]]+\]$', ''),
-]
classifier_license_map = {
'License :: OSI Approved :: Academic Free License (AFL)': 'AFL',
@@ -166,122 +111,34 @@ class PythonRecipeHandler(RecipeHandler):
def __init__(self):
pass
-def process(self, srctree, classes, lines_before, lines_after, handled,
extravalues):
-if 'buildsystem' in handled:
-return False
-
-# Check for non-zero size setup.py files
-setupfiles = RecipeHandler.checkfiles(srctree, ['setup.py'])
-for fn in setupfiles:
-if os.path.getsize(fn):
-break
-else:
-return False
-
-# setup.py is always parsed to get at certain required information,
such as
-# distutils vs setuptools
-#
-# If egg info is available, we use it for both its PKG-INFO metadata
-# and for its requires.txt for install_requires.
-# If PKG-INFO is available but no egg info is, we use that for
metadata in preference to
-# the parsed setup.py, but use the install_requires info from the
-# parsed setup.py.
-
-setupscript = os.path.join(srctree, 'setup.py')
-try:
-setup_info, uses_setuptools, setup_non_literals, extensions =
self.parse_setup_py(setupscript)
-except Exception:
-logger.exception("Failed to parse setup.py")
-setup_info, uses_setuptools, setup_non_literals, extensions = {},
True, [], []
-
-egginfo = glob.glob(os.path.join(srctree, '*.egg-info'))
-if egginfo:
-info = self.get_pkginfo(os.path.join(egginfo[0], 'PKG-INFO'))
-requires_txt = os.path.join(egginfo[0], 'requires.txt')
-if os.path.exists(requires_txt):
-with codecs.open(requires_txt) as f:
-inst_req = []
-extras_req = collections.defaultdict(list)
-current_feature = None
-for line in f.readlines():
-line = line.rstrip()
-if not line:
-
[OE-core] [PATCH v4 2/5] scripts:recipetool:create_buildsys_python: prefix created recipes with python3-
By convention, all python recipes start with "python3-" so update
create_buildsys_python to do this
This rule doesn't apply for packages already starting with "python"
Update recipetool's selftest accordingly
Signed-off-by: Julien Stephan
---
meta/lib/oeqa/selftest/cases/recipetool.py | 4 ++--
scripts/lib/recipetool/create_buildsys_python.py | 5 +
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/recipetool.py
b/meta/lib/oeqa/selftest/cases/recipetool.py
index 48661bee6f2..d3aea74228f 100644
--- a/meta/lib/oeqa/selftest/cases/recipetool.py
+++ b/meta/lib/oeqa/selftest/cases/recipetool.py
@@ -445,7 +445,7 @@ class RecipetoolCreateTests(RecipetoolBase):
# Basic test to see if github URL mangling works
temprecipe = os.path.join(self.tempdir, 'recipe')
os.makedirs(temprecipe)
-recipefile = os.path.join(temprecipe, 'meson_git.bb')
+recipefile = os.path.join(temprecipe, 'python3-meson_git.bb')
srcuri = 'https://github.com/mesonbuild/meson;rev=0.32.0'
result = runCmd(['recipetool', 'create', '-o', temprecipe, srcuri])
self.assertTrue(os.path.isfile(recipefile))
@@ -479,7 +479,7 @@ class RecipetoolCreateTests(RecipetoolBase):
temprecipe = os.path.join(self.tempdir, 'recipe')
os.makedirs(temprecipe)
pv = '0.32.0'
-recipefile = os.path.join(temprecipe, 'meson_%s.bb' % pv)
+recipefile = os.path.join(temprecipe, 'python3-meson_%s.bb' % pv)
srcuri =
'https://github.com/mesonbuild/meson/releases/download/%s/meson-%s.tar.gz' %
(pv, pv)
result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
self.assertTrue(os.path.isfile(recipefile))
diff --git a/scripts/lib/recipetool/create_buildsys_python.py
b/scripts/lib/recipetool/create_buildsys_python.py
index 321d0ba257d..502e1dfbc3d 100644
--- a/scripts/lib/recipetool/create_buildsys_python.py
+++ b/scripts/lib/recipetool/create_buildsys_python.py
@@ -297,6 +297,11 @@ class PythonRecipeHandler(RecipeHandler):
value = ' '.join(str(v) for v in values if v)
bbvar = self.bbvar_map[field]
+if bbvar == "PN":
+# by convention python recipes start with "python3-"
+if not value.startswith('python'):
+value = 'python3-' + value
+
if bbvar not in extravalues and value:
extravalues[bbvar] = value
--
2.42.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189683):
https://lists.openembedded.org/g/openembedded-core/message/189683
Mute This Topic: https://lists.openembedded.org/mt/102181147/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [RESEND PATCH 2/2] oeqa/selftest/devtool: remove spaces on empty line
Hello Julien,
On Tue, 24 Oct 2023 16:17:54 +0200
"Julien Stephan" wrote:
> Signed-off-by: Julien Stephan
> ---
> meta/lib/oeqa/selftest/cases/devtool.py | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/lib/oeqa/selftest/cases/devtool.py
> b/meta/lib/oeqa/selftest/cases/devtool.py
> index c36b1efa78b..b5c488be8e8 100644
> --- a/meta/lib/oeqa/selftest/cases/devtool.py
> +++ b/meta/lib/oeqa/selftest/cases/devtool.py
> @@ -575,7 +575,7 @@ class DevtoolAddTests(DevtoolBase):
> checkvars['S'] = '${WORKDIR}/MarkupSafe-%s' % testver
> checkvars['SRC_URI'] = url
> self._test_recipe_contents(recipefile, checkvars, [])
> -
> +
This patch does not apply due to the '-' line above having no spaces
after '-'. Perhaps your mailer mangled the content removing training
whitespace (which, ironically, is what the patch itself aims at doing!).
Using 'git send-email' is usually the best way to send patches without
this sort of issues.
Luca
--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189681):
https://lists.openembedded.org/g/openembedded-core/message/189681
Mute This Topic: https://lists.openembedded.org/mt/102158568/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core][PATCH] patchtest-send-results: check max line length, simplify responses
Check that the maximum line length of the testresult file is less than
220 characters, to help guard against malicious changes being sent in
email responses. If any line exceeds this length, replace the normal
testresults used in the response with a line stating that tests failed,
but the results could not be processed. Also clean up the respone
substrings slightly to go along with the change.
Signed-off-by: Trevor Gamblin
---
scripts/patchtest-send-results | 17 +
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/scripts/patchtest-send-results b/scripts/patchtest-send-results
index bb4604aeaf1..078651ec381 100755
--- a/scripts/patchtest-send-results
+++ b/scripts/patchtest-send-results
@@ -25,12 +25,8 @@ more information:\n\n---\n"""
suggestions = """\n---\n\nPlease address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
-email with an explanation of why the patch format should be accepted.
-Note that patchtest may report failures in the merge-on-head test for
-patches that are part of a series if they rely on changes from
-preceeding entries.
-
-If you believe these results are due to an error in patchtest, please
+email with an explanation of why the patch format should be accepted. If
+you believe these results are due to an error in patchtest, please
submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest'
category under 'Yocto Project Subprojects'). Thank you!"""
@@ -52,8 +48,6 @@ testresult = None
with open(result_file, "r") as f:
testresult = f.read()
-reply_contents = greeting + testresult + suggestions
-
# we know these patch files will only contain a single patch, so only
# worry about the first element for getting the subject
mbox = mailbox.mbox(args.patch)
@@ -61,6 +55,13 @@ mbox_subject = mbox[0]['subject']
subject_line = f"Patchtest results for {mbox_subject}"
if "FAIL" in testresult:
+reply_contents = None
+if len(max(open(result_file, 'r'), key=len)) > 220:
+warning = "Tests failed for the patch, but the results log could not
be processed due to excessive result line length."
+reply_contents = greeting + warning + suggestions
+else:
+reply_contents = greeting + testresult + suggestions
+
ses_client = boto3.client('ses', region_name='us-west-2')
response = ses_client.send_email(
Source='patcht...@automation.yoctoproject.org',
--
2.41.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189680):
https://lists.openembedded.org/g/openembedded-core/message/189680
Mute This Topic: https://lists.openembedded.org/mt/102179782/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v3 0/4] devtool/recipetool: add support of PEP-517
On Tue, 2023-10-24 at 16:40 +0200, Julien Stephan wrote: > Hello all, > > This series adds basic support for PEP-517 in recipetool/devtool, plus somes > others fixes. > > Recipetool/devtool selftest passed locally > > Missing features: > - extra or optionnal dependencies are not handled > - version number for dependencies are not handled (issue already present > for the legacy setuptool) > - some projects can declare a minimal pyproject.toml file, and put all > the metadata in setup.py/setup.cfg/requirements.txt .. theses cases > are not handled. If a pyproject.toml file is found, assumes it has all > necessary metadata. > > Changes from v2: > - try to use tomllib from python core (since 3.11) and if not available, > try to import tomli (instead of toml, as tomllib and tomli have both the > same api) > - improve support for poetry: poetry looks for metadata into the > "tools.poetry" section instead of the "project" section > - add the selftests for the 3 supported backends. Skip the tests if > tomliib (python > 3.11) or tomli is not installed > - fix self test failing with the new "python3-" prefix added to > recipetool for python recipes This patch series still doesn't seem to apply against master. Could you check your rebase is ok or if there is a patch missing or something? Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189679): https://lists.openembedded.org/g/openembedded-core/message/189679 Mute This Topic: https://lists.openembedded.org/mt/102158953/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2] cve-check: Classify patched CVEs into 3 statuses
Hello Marko, I think that we will need to go back to the drawing board and have a look what we want to report from the CVE check. I'm not totally happy with the solution proposed here, because it is adding high-level states. However, it is a step forward to be able to map our status to VEX. In the RFQ work on SPDX3 (which includes VEX in the "security" profile) we had a look at that and realized that mapping of the information is not possible, we're missing information. The current state of "Ignore" is not great either. You have added additional context, but still some statuses do not make sense in Ignore. It is effectively used as an "override" of everything. I also have a multi-fetcher in the works, which downloads CVE data from other sources that NVD. Results are sometimes in conflict and we'll have to handle that too. And there are other sources lurking behind too. So, when the RFQ season is finished (Oct 31st), we'll get back to the drawing board to design what statuses we really want and need. I understand you'd like to be a part of this discussion. Kind regards, Marta On Wed, Oct 25, 2023 at 3:09 PM Marko, Peter wrote: > > Hello Marta, > > > > Major reason why we introduced CVE_STATUS was exactly to avoid patch like > this. > > There were ideas to introduce 5 or 10 or 15 different statuses and we decided > to keep 3 and introduce “sub-statuses”. > > These sub-statuses are listed in cve reports, too. > > > > Currently we have three main statuses: > > Patched – common status for all sub-statuses which indicate that component is > not vulnerable > > Unpatched - common status for all sub-statuses which indicate that component > is vulnerable > > Ignored - common status for all sub-statuses which indicate that component is > vulnerable but not in yocto configuration context > > > > If we don’t like “Patched” we can rename it (e.g. to “Unaffected”) and have > additional sub-statuses under this new name. > > Otherwise we start exploding the statuses as someone will “need” additional > one soon. > > > > If we really want to introduce these new statues (I hope not), please modify > this patch to handle its CVE_STATUS flags, too. > > Additionally, I’d drop “Undecidable” and map it to “Unpatched” (so someone > needs to analyze as any other open vulnerability report) > > > > Best Regards, > > Peter > > > > From: Marta Rybczynska > Sent: Wednesday, October 25, 2023 14:44 > To: Andrej Valek > Cc: Matsunaga-Shinji ; Richard Purdie > ; OE-core > ; Shunsuke Tokumoto > ; Marko, Peter (ADV D EU SK BFS1) > > Subject: Re: [OE-core] [PATCH v2] cve-check: Classify patched CVEs into 3 > statuses > > > > Hello Andrej, > > This patch is splitting the Patched state, not the ignore one. This is not > incorrect CPE or anything else. > > > > Currently Patched means one of two situations: either this issue has never > affected the code base (example: we have version 1.0, issue was introduced in > 2.0 and fixed in 2.1), or the issue has been fixed. > > > > Yes, another reason to say ignore, not affected is a manual analysis showing > a thing like: the issue affects only windows. > > > > Regards, > > Marta > > > > On Wed, 25 Oct 2023, 12:18 Andrej Valek, wrote: > > Hi Marta, > > That's fine, as I said we designed the "ignore" with status > "cpe-incorrect" or "ignored" exactly for those purposes. Extending the > option with "not affected" doesn't make any sense. > > You have to set the status to "why is not affected" = "ignored". Which > completely covers the requested case. > > Regards, > Andrej > > On 25.10.2023 11:33, Marta Rybczynska wrote: > > Hi Andrej, > > This is more complex. "Not affected" is also an issue that isn't present in > > the > > code - like when we have a version that has never had the vulnerability. > > Those are also currently 'Patched' in cve-check. > > > > This work is in sync with what VEX is doing, is it the use-case > > Matsanaga-Shinji? > > > > Regards, > > Marta > > > > On Wed, Oct 25, 2023 at 8:44 AM Andrej Valek wrote: > >> Hi all, > >> > >> Do we really need a new "not_affected" state? I guess the ignore state > >> is exactly designed for those purposes. > >> > >> Regards, > >> Andrej > >> > >> On 25.10.2023 07:13, Matsunaga-Shinji wrote: > >>> CVEs that are currently considered "Patched" are classified into the > >>> following 3 statuses: > >>> 1. "Patched" - means that a patch file that fixed the vulnerability > >>> has been applied > >>> 2. "Not affected" - means that the package version (PV) is not affected > >>> by the vulnerability > >>> 3. "Undecidable" - means that versions cannot be compared to determine > >>> if they are affected by the vulnerability > >>> > >>> Signed-off-by: Shinji Matsunaga > >>> Signed-off-by: Shunsuke Tokumoto > >>> --- > >>> > >>> Changes for v2: > >>> - Fix the status "Out of range" to "Not affected" > >>> > >>>meta/classes/cve-check.bbclass | 55 +++--- > >>>1 file changed, 38
Re: [OE-core] [PATCH v2] cve-check: Classify patched CVEs into 3 statuses
Hello Marta,
Major reason why we introduced CVE_STATUS was exactly to avoid patch like this.
There were ideas to introduce 5 or 10 or 15 different statuses and we decided
to keep 3 and introduce “sub-statuses”.
These sub-statuses are listed in cve reports, too.
Currently we have three main statuses:
Patched – common status for all sub-statuses which indicate that component is
not vulnerable
Unpatched - common status for all sub-statuses which indicate that component is
vulnerable
Ignored - common status for all sub-statuses which indicate that component is
vulnerable but not in yocto configuration context
If we don’t like “Patched” we can rename it (e.g. to “Unaffected”) and have
additional sub-statuses under this new name.
Otherwise we start exploding the statuses as someone will “need” additional one
soon.
If we really want to introduce these new statues (I hope not), please modify
this patch to handle its CVE_STATUS flags, too.
Additionally, I’d drop “Undecidable” and map it to “Unpatched” (so someone
needs to analyze as any other open vulnerability report)
Best Regards,
Peter
From: Marta Rybczynska
Sent: Wednesday, October 25, 2023 14:44
To: Andrej Valek
Cc: Matsunaga-Shinji ; Richard Purdie
; OE-core
; Shunsuke Tokumoto
; Marko, Peter (ADV D EU SK BFS1)
Subject: Re: [OE-core] [PATCH v2] cve-check: Classify patched CVEs into 3
statuses
Hello Andrej,
This patch is splitting the Patched state, not the ignore one. This is not
incorrect CPE or anything else.
Currently Patched means one of two situations: either this issue has never
affected the code base (example: we have version 1.0, issue was introduced in
2.0 and fixed in 2.1), or the issue has been fixed.
Yes, another reason to say ignore, not affected is a manual analysis showing a
thing like: the issue affects only windows.
Regards,
Marta
On Wed, 25 Oct 2023, 12:18 Andrej Valek,
mailto:andre...@skyrain.eu>> wrote:
Hi Marta,
That's fine, as I said we designed the "ignore" with status
"cpe-incorrect" or "ignored" exactly for those purposes. Extending the
option with "not affected" doesn't make any sense.
You have to set the status to "why is not affected" = "ignored". Which
completely covers the requested case.
Regards,
Andrej
On 25.10.2023 11:33, Marta Rybczynska wrote:
> Hi Andrej,
> This is more complex. "Not affected" is also an issue that isn't present in
> the
> code - like when we have a version that has never had the vulnerability.
> Those are also currently 'Patched' in cve-check.
>
> This work is in sync with what VEX is doing, is it the use-case
> Matsanaga-Shinji?
>
> Regards,
> Marta
>
> On Wed, Oct 25, 2023 at 8:44 AM Andrej Valek
> mailto:andre...@skyrain.eu>> wrote:
>> Hi all,
>>
>> Do we really need a new "not_affected" state? I guess the ignore state
>> is exactly designed for those purposes.
>>
>> Regards,
>> Andrej
>>
>> On 25.10.2023 07:13, Matsunaga-Shinji wrote:
>>> CVEs that are currently considered "Patched" are classified into the
>>> following 3 statuses:
>>> 1. "Patched" - means that a patch file that fixed the vulnerability
>>> has been applied
>>> 2. "Not affected" - means that the package version (PV) is not affected by
>>> the vulnerability
>>> 3. "Undecidable" - means that versions cannot be compared to determine if
>>> they are affected by the vulnerability
>>>
>>> Signed-off-by: Shinji Matsunaga
>>> mailto:shin.matsun...@fujitsu.com>>
>>> Signed-off-by: Shunsuke Tokumoto
>>> mailto:s-tokum...@fujitsu.com>>
>>> ---
>>>
>>> Changes for v2:
>>> - Fix the status "Out of range" to "Not affected"
>>>
>>>meta/classes/cve-check.bbclass | 55 +++---
>>>1 file changed, 38 insertions(+), 17 deletions(-)
>>>
>>> diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
>>> index b55f4299da..502db324df 100644
>>> --- a/meta/classes/cve-check.bbclass
>>> +++ b/meta/classes/cve-check.bbclass
>>> @@ -185,10 +185,10 @@ python do_cve_check () {
>>>patched_cves = get_patched_cves(d)
>>>except FileNotFoundError:
>>>bb.fatal("Failure in searching patches")
>>> -ignored, patched, unpatched, status = check_cves(d,
>>> patched_cves)
>>> -if patched or unpatched or (d.getVar("CVE_CHECK_COVERAGE") ==
>>> "1" and status):
>>> -cve_data = get_cve_info(d, patched + unpatched + ignored)
>>> -cve_write_data(d, patched, unpatched, ignored, cve_data,
>>> status)
>>> +ignored, patched, unpatched, not_affected, undecidable, status
>>> = check_cves(d, patched_cves)
>>> +if patched or unpatched or not_affected or undecidable or
>>> (d.getVar("CVE_CHECK_COVERAGE") == "1" and status):
>>> +cve_data = get_cve_info(d, patched + unpatched + ignored +
>>> not_affected + undecidable)
>>> +cve_write_data(d, patched, unpatched, ignored,
>>> not_affected, undecidable, cve_data,
Re: [OE-core] [PATCH v2] cve-check: Classify patched CVEs into 3 statuses
Hello Andrej,
This patch is splitting the Patched state, not the ignore one. This is not
incorrect CPE or anything else.
Currently Patched means one of two situations: either this issue has never
affected the code base (example: we have version 1.0, issue was introduced
in 2.0 and fixed in 2.1), or the issue has been fixed.
Yes, another reason to say ignore, not affected is a manual analysis
showing a thing like: the issue affects only windows.
Regards,
Marta
On Wed, 25 Oct 2023, 12:18 Andrej Valek, wrote:
> Hi Marta,
>
> That's fine, as I said we designed the "ignore" with status
> "cpe-incorrect" or "ignored" exactly for those purposes. Extending the
> option with "not affected" doesn't make any sense.
>
> You have to set the status to "why is not affected" = "ignored". Which
> completely covers the requested case.
>
> Regards,
> Andrej
>
> On 25.10.2023 11:33, Marta Rybczynska wrote:
> > Hi Andrej,
> > This is more complex. "Not affected" is also an issue that isn't present
> in the
> > code - like when we have a version that has never had the vulnerability.
> > Those are also currently 'Patched' in cve-check.
> >
> > This work is in sync with what VEX is doing, is it the use-case
> > Matsanaga-Shinji?
> >
> > Regards,
> > Marta
> >
> > On Wed, Oct 25, 2023 at 8:44 AM Andrej Valek
> wrote:
> >> Hi all,
> >>
> >> Do we really need a new "not_affected" state? I guess the ignore state
> >> is exactly designed for those purposes.
> >>
> >> Regards,
> >> Andrej
> >>
> >> On 25.10.2023 07:13, Matsunaga-Shinji wrote:
> >>> CVEs that are currently considered "Patched" are classified into the
> following 3 statuses:
> >>> 1. "Patched" - means that a patch file that fixed the
> vulnerability has been applied
> >>> 2. "Not affected" - means that the package version (PV) is not
> affected by the vulnerability
> >>> 3. "Undecidable" - means that versions cannot be compared to
> determine if they are affected by the vulnerability
> >>>
> >>> Signed-off-by: Shinji Matsunaga
> >>> Signed-off-by: Shunsuke Tokumoto
> >>> ---
> >>>
> >>> Changes for v2:
> >>> - Fix the status "Out of range" to "Not affected"
> >>>
> >>>meta/classes/cve-check.bbclass | 55
> +++---
> >>>1 file changed, 38 insertions(+), 17 deletions(-)
> >>>
> >>> diff --git a/meta/classes/cve-check.bbclass
> b/meta/classes/cve-check.bbclass
> >>> index b55f4299da..502db324df 100644
> >>> --- a/meta/classes/cve-check.bbclass
> >>> +++ b/meta/classes/cve-check.bbclass
> >>> @@ -185,10 +185,10 @@ python do_cve_check () {
> >>>patched_cves = get_patched_cves(d)
> >>>except FileNotFoundError:
> >>>bb.fatal("Failure in searching patches")
> >>> -ignored, patched, unpatched, status = check_cves(d,
> patched_cves)
> >>> -if patched or unpatched or
> (d.getVar("CVE_CHECK_COVERAGE") == "1" and status):
> >>> -cve_data = get_cve_info(d, patched + unpatched +
> ignored)
> >>> -cve_write_data(d, patched, unpatched, ignored,
> cve_data, status)
> >>> +ignored, patched, unpatched, not_affected, undecidable,
> status = check_cves(d, patched_cves)
> >>> +if patched or unpatched or not_affected or undecidable or
> (d.getVar("CVE_CHECK_COVERAGE") == "1" and status):
> >>> +cve_data = get_cve_info(d, patched + unpatched +
> ignored + not_affected + undecidable)
> >>> +cve_write_data(d, patched, unpatched, ignored,
> not_affected, undecidable, cve_data, status)
> >>>else:
> >>>bb.note("No CVE database found, skipping CVE check")
> >>>
> >>> @@ -308,13 +308,13 @@ def check_cves(d, patched_cves):
> >>>products = d.getVar("CVE_PRODUCT").split()
> >>># If this has been unset then we're not scanning for CVEs here
> (for example, image recipes)
> >>>if not products:
> >>> -return ([], [], [], [])
> >>> +return ([], [], [], [], [], [])
> >>>pv = d.getVar("CVE_VERSION").split("+git")[0]
> >>>
> >>># If the recipe has been skipped/ignored we return empty lists
> >>>if pn in d.getVar("CVE_CHECK_SKIP_RECIPE").split():
> >>>bb.note("Recipe has been skipped by cve-check")
> >>> -return ([], [], [], [])
> >>> +return ([], [], [], [], [], [])
> >>>
> >>># Convert CVE_STATUS into ignored CVEs and check validity
> >>>cve_ignore = []
> >>> @@ -328,6 +328,8 @@ def check_cves(d, patched_cves):
> >>>conn = sqlite3.connect(db_file, uri=True)
> >>>
> >>># For each of the known product names (e.g. curl has CPEs using
> curl and libcurl)...
> >>> +cves_not_affected = []
> >>> +cves_undecidable = []
> >>>for product in products:
> >>>cves_in_product = False
> >>>if ":" in product:
> >>> @@ -355,6 +357,7 @@ def check_cves(d, patched_cves):
> >>>
> >>>vulnerable = False
> >>>
[OE-core][PATCH] make-mod-scripts: exclude rm_work
- Add the recipe to the RM_WORK_EXCLUDE
With rm_work active, external module signing throws an error:
scripts/sign-file: error while loading shared libraries: libcrypto.so.3: can
not open shared object file: No such file or directory
Preserve libraries that sign-file script needs during runtime.
Some solutions [1][2] for this problem have already been submitted
on the mailing list but none of them get merged:
[1] Fix using RM_WORK_EXCLUDE_ITEMS
https://lists.openembedded.org/g/openembedded-core/message/180113
[2] Fix using static linking
https://lists.openembedded.org/g/openembedded-core/message/182024
Signed-off-by: Jose Quaresma
---
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb
b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb
index f6f47cfff5..6dcca738a8 100644
--- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb
+++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb
@@ -10,6 +10,8 @@ PACKAGE_ARCH = "${MACHINE_ARCH}"
S = "${WORKDIR}"
+RM_WORK_EXCLUDE += "${PN}"
+
do_configure[depends] += "virtual/kernel:do_shared_workdir
openssl-native:do_populate_sysroot"
do_compile[depends] += "virtual/kernel:do_compile_kernelmodules"
--
2.42.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189675):
https://lists.openembedded.org/g/openembedded-core/message/189675
Mute This Topic: https://lists.openembedded.org/mt/102176240/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2] cve-check: Classify patched CVEs into 3 statuses
Hi Marta,
That's fine, as I said we designed the "ignore" with status
"cpe-incorrect" or "ignored" exactly for those purposes. Extending the
option with "not affected" doesn't make any sense.
You have to set the status to "why is not affected" = "ignored". Which
completely covers the requested case.
Regards,
Andrej
On 25.10.2023 11:33, Marta Rybczynska wrote:
Hi Andrej,
This is more complex. "Not affected" is also an issue that isn't present in the
code - like when we have a version that has never had the vulnerability.
Those are also currently 'Patched' in cve-check.
This work is in sync with what VEX is doing, is it the use-case
Matsanaga-Shinji?
Regards,
Marta
On Wed, Oct 25, 2023 at 8:44 AM Andrej Valek wrote:
Hi all,
Do we really need a new "not_affected" state? I guess the ignore state
is exactly designed for those purposes.
Regards,
Andrej
On 25.10.2023 07:13, Matsunaga-Shinji wrote:
CVEs that are currently considered "Patched" are classified into the following
3 statuses:
1. "Patched" - means that a patch file that fixed the vulnerability has
been applied
2. "Not affected" - means that the package version (PV) is not affected by the
vulnerability
3. "Undecidable" - means that versions cannot be compared to determine if they
are affected by the vulnerability
Signed-off-by: Shinji Matsunaga
Signed-off-by: Shunsuke Tokumoto
---
Changes for v2:
- Fix the status "Out of range" to "Not affected"
meta/classes/cve-check.bbclass | 55 +++---
1 file changed, 38 insertions(+), 17 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index b55f4299da..502db324df 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -185,10 +185,10 @@ python do_cve_check () {
patched_cves = get_patched_cves(d)
except FileNotFoundError:
bb.fatal("Failure in searching patches")
-ignored, patched, unpatched, status = check_cves(d, patched_cves)
-if patched or unpatched or (d.getVar("CVE_CHECK_COVERAGE") == "1"
and status):
-cve_data = get_cve_info(d, patched + unpatched + ignored)
-cve_write_data(d, patched, unpatched, ignored, cve_data,
status)
+ignored, patched, unpatched, not_affected, undecidable, status =
check_cves(d, patched_cves)
+if patched or unpatched or not_affected or undecidable or
(d.getVar("CVE_CHECK_COVERAGE") == "1" and status):
+cve_data = get_cve_info(d, patched + unpatched + ignored +
not_affected + undecidable)
+cve_write_data(d, patched, unpatched, ignored, not_affected,
undecidable, cve_data, status)
else:
bb.note("No CVE database found, skipping CVE check")
@@ -308,13 +308,13 @@ def check_cves(d, patched_cves):
products = d.getVar("CVE_PRODUCT").split()
# If this has been unset then we're not scanning for CVEs here (for
example, image recipes)
if not products:
-return ([], [], [], [])
+return ([], [], [], [], [], [])
pv = d.getVar("CVE_VERSION").split("+git")[0]
# If the recipe has been skipped/ignored we return empty lists
if pn in d.getVar("CVE_CHECK_SKIP_RECIPE").split():
bb.note("Recipe has been skipped by cve-check")
-return ([], [], [], [])
+return ([], [], [], [], [], [])
# Convert CVE_STATUS into ignored CVEs and check validity
cve_ignore = []
@@ -328,6 +328,8 @@ def check_cves(d, patched_cves):
conn = sqlite3.connect(db_file, uri=True)
# For each of the known product names (e.g. curl has CPEs using curl and
libcurl)...
+cves_not_affected = []
+cves_undecidable = []
for product in products:
cves_in_product = False
if ":" in product:
@@ -355,6 +357,7 @@ def check_cves(d, patched_cves):
vulnerable = False
ignored = False
+undecidable = False
product_cursor = conn.execute("SELECT * FROM PRODUCTS WHERE ID IS ?
AND PRODUCT IS ? AND VENDOR LIKE ?", (cve, product, vendor))
for row in product_cursor:
@@ -376,7 +379,7 @@ def check_cves(d, patched_cves):
except:
bb.warn("%s: Failed to compare %s %s %s for %s" %
(product, pv, operator_start,
version_start, cve))
-vulnerable_start = False
+undecidable = True
else:
vulnerable_start = False
@@ -387,10 +390,15 @@ def check_cves(d, patched_cves):
except:
bb.warn("%s: Failed to compare %s %s %s for %s" %
(product, pv, operator_end, version_end,
cve))
-vulnerable_end = False
+
Re: [OE-core] [PATCH V1] cairo:Add ptest support
On Wed, 25 Oct 2023 at 10:47, qi...@fujitsu.com wrote: > If you can do the cairo update to 1.18.0 first, I would really appreciate > that. > > I'm trying to do the cairo update. > Which images and tests should be check? There is no definite and specific answer to this, but a few things you could do, in order: - make sure it builds for qemux86_64 / glibc - make sure it builds for qemuarm (32 bit) / musl - make sure all dependent recipes still build (grep -ilr cairo * in poky/meta/ will show which ones, or you can simply run bitbake world if you have a powerful machine) - run '-c testimage' for core-image-sato-sdk and core-image weston (use qemux86_64 with kvm for speed) - boot those images with qemu and visually check that they still show correct graphical output There are further tests, e.g. binary reproducibility, but those will be run on the autobuilder, and you aren't expected to do them. If they fail, integrators will tell you. Thanks, Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189673): https://lists.openembedded.org/g/openembedded-core/message/189673 Mute This Topic: https://lists.openembedded.org/mt/102130424/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2] cve-check: Classify patched CVEs into 3 statuses
Hi Andrej,
This is more complex. "Not affected" is also an issue that isn't present in the
code - like when we have a version that has never had the vulnerability.
Those are also currently 'Patched' in cve-check.
This work is in sync with what VEX is doing, is it the use-case
Matsanaga-Shinji?
Regards,
Marta
On Wed, Oct 25, 2023 at 8:44 AM Andrej Valek wrote:
>
> Hi all,
>
> Do we really need a new "not_affected" state? I guess the ignore state
> is exactly designed for those purposes.
>
> Regards,
> Andrej
>
> On 25.10.2023 07:13, Matsunaga-Shinji wrote:
> > CVEs that are currently considered "Patched" are classified into the
> > following 3 statuses:
> > 1. "Patched" - means that a patch file that fixed the vulnerability
> > has been applied
> > 2. "Not affected" - means that the package version (PV) is not affected by
> > the vulnerability
> > 3. "Undecidable" - means that versions cannot be compared to determine if
> > they are affected by the vulnerability
> >
> > Signed-off-by: Shinji Matsunaga
> > Signed-off-by: Shunsuke Tokumoto
> > ---
> >
> > Changes for v2:
> > - Fix the status "Out of range" to "Not affected"
> >
> > meta/classes/cve-check.bbclass | 55 +++---
> > 1 file changed, 38 insertions(+), 17 deletions(-)
> >
> > diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
> > index b55f4299da..502db324df 100644
> > --- a/meta/classes/cve-check.bbclass
> > +++ b/meta/classes/cve-check.bbclass
> > @@ -185,10 +185,10 @@ python do_cve_check () {
> > patched_cves = get_patched_cves(d)
> > except FileNotFoundError:
> > bb.fatal("Failure in searching patches")
> > -ignored, patched, unpatched, status = check_cves(d,
> > patched_cves)
> > -if patched or unpatched or (d.getVar("CVE_CHECK_COVERAGE") ==
> > "1" and status):
> > -cve_data = get_cve_info(d, patched + unpatched + ignored)
> > -cve_write_data(d, patched, unpatched, ignored, cve_data,
> > status)
> > +ignored, patched, unpatched, not_affected, undecidable, status
> > = check_cves(d, patched_cves)
> > +if patched or unpatched or not_affected or undecidable or
> > (d.getVar("CVE_CHECK_COVERAGE") == "1" and status):
> > +cve_data = get_cve_info(d, patched + unpatched + ignored +
> > not_affected + undecidable)
> > +cve_write_data(d, patched, unpatched, ignored,
> > not_affected, undecidable, cve_data, status)
> > else:
> > bb.note("No CVE database found, skipping CVE check")
> >
> > @@ -308,13 +308,13 @@ def check_cves(d, patched_cves):
> > products = d.getVar("CVE_PRODUCT").split()
> > # If this has been unset then we're not scanning for CVEs here (for
> > example, image recipes)
> > if not products:
> > -return ([], [], [], [])
> > +return ([], [], [], [], [], [])
> > pv = d.getVar("CVE_VERSION").split("+git")[0]
> >
> > # If the recipe has been skipped/ignored we return empty lists
> > if pn in d.getVar("CVE_CHECK_SKIP_RECIPE").split():
> > bb.note("Recipe has been skipped by cve-check")
> > -return ([], [], [], [])
> > +return ([], [], [], [], [], [])
> >
> > # Convert CVE_STATUS into ignored CVEs and check validity
> > cve_ignore = []
> > @@ -328,6 +328,8 @@ def check_cves(d, patched_cves):
> > conn = sqlite3.connect(db_file, uri=True)
> >
> > # For each of the known product names (e.g. curl has CPEs using curl
> > and libcurl)...
> > +cves_not_affected = []
> > +cves_undecidable = []
> > for product in products:
> > cves_in_product = False
> > if ":" in product:
> > @@ -355,6 +357,7 @@ def check_cves(d, patched_cves):
> >
> > vulnerable = False
> > ignored = False
> > +undecidable = False
> >
> > product_cursor = conn.execute("SELECT * FROM PRODUCTS WHERE
> > ID IS ? AND PRODUCT IS ? AND VENDOR LIKE ?", (cve, product, vendor))
> > for row in product_cursor:
> > @@ -376,7 +379,7 @@ def check_cves(d, patched_cves):
> > except:
> > bb.warn("%s: Failed to compare %s %s %s for
> > %s" %
> > (product, pv, operator_start,
> > version_start, cve))
> > -vulnerable_start = False
> > +undecidable = True
> > else:
> > vulnerable_start = False
> >
> > @@ -387,10 +390,15 @@ def check_cves(d, patched_cves):
> > except:
> > bb.warn("%s: Failed to compare %s %s %s for
> > %s" %
> > (product, pv, operator_end,
> > version_end, cve))
> > -vulnerable_end = False
> > +
Re: [OE-core] [PATCH V1] cairo:Add ptest support
Hi，Alex > > If you can do the cairo update to 1.18.0 first, I would really appreciate > that. I'm trying to do the cairo update. Which images and tests should be check? -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189671): https://lists.openembedded.org/g/openembedded-core/message/189671 Mute This Topic: https://lists.openembedded.org/mt/102130424/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2] cve-check: Classify patched CVEs into 3 statuses
Hi all,
Do we really need a new "not_affected" state? I guess the ignore state
is exactly designed for those purposes.
Regards,
Andrej
On 25.10.2023 07:13, Matsunaga-Shinji wrote:
CVEs that are currently considered "Patched" are classified into the following
3 statuses:
1. "Patched" - means that a patch file that fixed the vulnerability has
been applied
2. "Not affected" - means that the package version (PV) is not affected by the
vulnerability
3. "Undecidable" - means that versions cannot be compared to determine if they
are affected by the vulnerability
Signed-off-by: Shinji Matsunaga
Signed-off-by: Shunsuke Tokumoto
---
Changes for v2:
- Fix the status "Out of range" to "Not affected"
meta/classes/cve-check.bbclass | 55 +++---
1 file changed, 38 insertions(+), 17 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index b55f4299da..502db324df 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -185,10 +185,10 @@ python do_cve_check () {
patched_cves = get_patched_cves(d)
except FileNotFoundError:
bb.fatal("Failure in searching patches")
-ignored, patched, unpatched, status = check_cves(d, patched_cves)
-if patched or unpatched or (d.getVar("CVE_CHECK_COVERAGE") == "1"
and status):
-cve_data = get_cve_info(d, patched + unpatched + ignored)
-cve_write_data(d, patched, unpatched, ignored, cve_data,
status)
+ignored, patched, unpatched, not_affected, undecidable, status =
check_cves(d, patched_cves)
+if patched or unpatched or not_affected or undecidable or
(d.getVar("CVE_CHECK_COVERAGE") == "1" and status):
+cve_data = get_cve_info(d, patched + unpatched + ignored +
not_affected + undecidable)
+cve_write_data(d, patched, unpatched, ignored, not_affected,
undecidable, cve_data, status)
else:
bb.note("No CVE database found, skipping CVE check")
@@ -308,13 +308,13 @@ def check_cves(d, patched_cves):
products = d.getVar("CVE_PRODUCT").split()
# If this has been unset then we're not scanning for CVEs here (for
example, image recipes)
if not products:
-return ([], [], [], [])
+return ([], [], [], [], [], [])
pv = d.getVar("CVE_VERSION").split("+git")[0]
# If the recipe has been skipped/ignored we return empty lists
if pn in d.getVar("CVE_CHECK_SKIP_RECIPE").split():
bb.note("Recipe has been skipped by cve-check")
-return ([], [], [], [])
+return ([], [], [], [], [], [])
# Convert CVE_STATUS into ignored CVEs and check validity
cve_ignore = []
@@ -328,6 +328,8 @@ def check_cves(d, patched_cves):
conn = sqlite3.connect(db_file, uri=True)
# For each of the known product names (e.g. curl has CPEs using curl and libcurl)...
+cves_not_affected = []
+cves_undecidable = []
for product in products:
cves_in_product = False
if ":" in product:
@@ -355,6 +357,7 @@ def check_cves(d, patched_cves):
vulnerable = False
ignored = False
+undecidable = False
product_cursor = conn.execute("SELECT * FROM PRODUCTS WHERE ID IS ? AND PRODUCT IS ? AND VENDOR LIKE ?", (cve, product, vendor))
for row in product_cursor:
@@ -376,7 +379,7 @@ def check_cves(d, patched_cves):
except:
bb.warn("%s: Failed to compare %s %s %s for %s" %
(product, pv, operator_start,
version_start, cve))
-vulnerable_start = False
+undecidable = True
else:
vulnerable_start = False
@@ -387,10 +390,15 @@ def check_cves(d, patched_cves):
except:
bb.warn("%s: Failed to compare %s %s %s for %s" %
(product, pv, operator_end, version_end,
cve))
-vulnerable_end = False
+undecidable = True
else:
vulnerable_end = False
+if undecidable:
+bb.note("%s-%s is undecidable to %s" % (pn, real_pv,
cve))
+cves_undecidable.append(cve)
+break
+
if operator_start and operator_end:
vulnerable = vulnerable_start and vulnerable_end
else:
@@ -406,9 +414,9 @@ def check_cves(d, patched_cves):
break
product_cursor.close()
-if not vulnerable:
+if not undecidable and not vulnerable:
bb.note("%s-%s is not vulnerable to
