[OE-core] [PATCH 2/3] systemd: Add link-udev-shared PACKAGECONFIG

[Stefan Herbrechtsmeier]

From: Stefan Herbrechtsmeier 

Add a PACKAGECONFIG to link systemd-udev and its helpers to
libsystemd-shared.so. If enabled the udev package depends on the systemd
package.

Signed-off-by: Stefan Herbrechtsmeier 
---

 meta/recipes-core/systemd/systemd_249.7.bb | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-core/systemd/systemd_249.7.bb 
b/meta/recipes-core/systemd/systemd_249.7.bb
index 53e7f6738d..739f3aae86 100644
--- a/meta/recipes-core/systemd/systemd_249.7.bb
+++ b/meta/recipes-core/systemd/systemd_249.7.bb
@@ -156,6 +156,9 @@ PACKAGECONFIG[kmod] = "-Dkmod=true,-Dkmod=false,kmod"
 PACKAGECONFIG[ldconfig] = "-Dldconfig=true,-Dldconfig=false,,ldconfig"
 PACKAGECONFIG[libidn] = "-Dlibidn=true,-Dlibidn=false,libidn,,libidn"
 PACKAGECONFIG[libidn2] = "-Dlibidn2=true,-Dlibidn2=false,libidn2,,libidn2"
+# Link udev shared with systemd helper library.
+# If enabled the udev package depends on the systemd package (which has the 
needed shared library).
+PACKAGECONFIG[link-udev-shared] = 
"-Dlink-udev-shared=true,-Dlink-udev-shared=false"
 PACKAGECONFIG[localed] = "-Dlocaled=true,-Dlocaled=false"
 PACKAGECONFIG[logind] = "-Dlogind=true,-Dlogind=false"
 PACKAGECONFIG[lz4] = "-Dlz4=true,-Dlz4=false,lz4"
@@ -215,11 +218,6 @@ rootprefix ?= "${root_prefix}"
 rootlibdir ?= "${base_libdir}"
 rootlibexecdir = "${rootprefix}/lib"
 
-# This links udev statically with systemd helper library.
-# Otherwise udev package would depend on systemd package (which has the needed 
shared library),
-# and always pull it into images.
-EXTRA_OEMESON += "-Dlink-udev-shared=false"
-
 EXTRA_OEMESON += "-Dnobody-user=nobody \
   -Dnobody-group=nobody \
   -Drootlibdir=${rootlibdir} \
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161160): 
https://lists.openembedded.org/g/openembedded-core/message/161160
Mute This Topic: https://lists.openembedded.org/mt/88836330/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] recipetool/create: Scan for SDPX-License-Identifier

[Stefan Herbrechtsmeier]

Hi Richard,

Am 03.02.2022 um 22:24 schrieb Richard Purdie via lists.openembedded.org:

On Thu, 2022-02-03 at 09:07 -0800, Saul Wold wrote:

When a file can not be identified by checksum and they contain an SPDX
License-Identifier tag, use it as the found license.

[YOCTO #14529]

Tested with LICENSE files that contain 1 or more SPDX-License-Identifier tags

Signed-off-by: Saul Wold 
---
  scripts/lib/recipetool/create.py | 16 +++-
  1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 507a230511..9149c2d94f 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1221,14 +1221,20 @@ def guess_license(srctree, d):
  for licfile in sorted(licfiles):
  md5value = bb.utils.md5_file(licfile)
  license = md5sums.get(md5value, None)
+license_list = []
  if not license:
  license, crunched_md5, lictext = crunch_license(licfile)
  if lictext and not license:
-license = 'Unknown'
-logger.info("Please add the following line for '%s' to a 
'lib/recipetool/licenses.csv' " \
-"and replace `Unknown` with the license:\n" \
-"%s,Unknown" % (os.path.relpath(licfile, srctree), 
md5value))
-if license:
+spdx_re = re.compile('SPDX-License-Identifier:\s+([-A-Za-z\d. 
]+)[ |\n|\r\n]*?')
+license_list = re.findall(spdx_re, "\n".join(lictext))
+if not license_list:
+license_list.append('Unknown')
+logger.info("Please add the following line for '%s' to a 
'lib/recipetool/licenses.csv' " \
+"and replace `Unknown` with the license:\n" \
+"%s,Unknown" % (os.path.relpath(licfile, srctree), 
md5value))
+else:
+license_list.append(license)
+for license in license_list:
  licenses.append((license, os.path.relpath(licfile, srctree), 
md5value))
  
  # FIXME should we grab at least one source file with a license header and add that too?


I think to close this bug the code may need to go one step further and
effectively grep over the source tree.


Please keep in mind that we need a full license text and not only the 
license name for license compliance. The current function only search 
for license files with license text.



We'd probably want to list the value of any SPDX-License-Identifier: header
found in any of the source files for the user to then decide upon?


I think this is an other feature like a license checker because if you 
have a SPDX-License-Identifier without a license text you have a license 
violation.


This brings us to the problem that this code will interpret a file with 
only a SPDX-License-Identifier as a license file with license text.


Regards
  Stefan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161342): 
https://lists.openembedded.org/g/openembedded-core/message/161342
Mute This Topic: https://lists.openembedded.org/mt/7504/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] recipetool/create: Scan for SDPX-License-Identifier

[Stefan Herbrechtsmeier]

Hi Saul,

Am 03.02.2022 um 18:07 schrieb Saul Wold via lists.openembedded.org:

When a file can not be identified by checksum and they contain an SPDX
License-Identifier tag, use it as the found license.

[YOCTO #14529]

Tested with LICENSE files that contain 1 or more SPDX-License-Identifier tags


Can you please give an example for an project with use a 
SPDX-License-Identifier inside a license file.




Signed-off-by: Saul Wold 
---
  scripts/lib/recipetool/create.py | 16 +++-
  1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 507a230511..9149c2d94f 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1221,14 +1221,20 @@ def guess_license(srctree, d):
  for licfile in sorted(licfiles):
  md5value = bb.utils.md5_file(licfile)
  license = md5sums.get(md5value, None)
+license_list = []


Could you please use an other name. We already have licenses and it is 
hard to distinguish the difference between licenses and license_list.



  if not license:
  license, crunched_md5, lictext = crunch_license(licfile)
  if lictext and not license:
-license = 'Unknown'
-logger.info("Please add the following line for '%s' to a 
'lib/recipetool/licenses.csv' " \
-"and replace `Unknown` with the license:\n" \
-"%s,Unknown" % (os.path.relpath(licfile, srctree), 
md5value))
-if license:
+spdx_re = re.compile('SPDX-License-Identifier:\s+([-A-Za-z\d. 
]+)[ |\n|\r\n]*?')
+license_list = re.findall(spdx_re, "\n".join(lictext))
+if not license_list:
+license_list.append('Unknown')
+logger.info("Please add the following line for '%s' to a 
'lib/recipetool/licenses.csv' " \
+"and replace `Unknown` with the license:\n" \
+"%s,Unknown" % (os.path.relpath(licfile, srctree), 
md5value))
+else:
+license_list.append(license)
+for license in license_list:
  licenses.append((license, os.path.relpath(licfile, srctree), 
md5value))
  
  # FIXME should we grab at least one source file with a license header and add that too?


Regards
  Stefan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161340): 
https://lists.openembedded.org/g/openembedded-core/message/161340
Mute This Topic: https://lists.openembedded.org/mt/7504/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] recipetool/create: Scan for SDPX-License-Identifier

[Stefan Herbrechtsmeier]

Am 04.02.2022 um 14:41 schrieb Richard Purdie:

On Fri, 2022-02-04 at 10:05 +0100, Stefan Herbrechtsmeier wrote:

Am 03.02.2022 um 22:24 schrieb Richard Purdie via lists.openembedded.org:

On Thu, 2022-02-03 at 09:07 -0800, Saul Wold wrote:

When a file can not be identified by checksum and they contain an SPDX
License-Identifier tag, use it as the found license.

[YOCTO #14529]

Tested with LICENSE files that contain 1 or more SPDX-License-Identifier tags

Signed-off-by: Saul Wold 
---
   scripts/lib/recipetool/create.py | 16 +++-
   1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 507a230511..9149c2d94f 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1221,14 +1221,20 @@ def guess_license(srctree, d):
   for licfile in sorted(licfiles):
   md5value = bb.utils.md5_file(licfile)
   license = md5sums.get(md5value, None)
+license_list = []
   if not license:
   license, crunched_md5, lictext = crunch_license(licfile)
   if lictext and not license:
-license = 'Unknown'
-logger.info("Please add the following line for '%s' to a 
'lib/recipetool/licenses.csv' " \
-"and replace `Unknown` with the license:\n" \
-"%s,Unknown" % (os.path.relpath(licfile, srctree), 
md5value))
-if license:
+spdx_re = re.compile('SPDX-License-Identifier:\s+([-A-Za-z\d. 
]+)[ |\n|\r\n]*?')
+license_list = re.findall(spdx_re, "\n".join(lictext))
+if not license_list:
+license_list.append('Unknown')
+logger.info("Please add the following line for '%s' to a 
'lib/recipetool/licenses.csv' " \
+"and replace `Unknown` with the license:\n" \
+"%s,Unknown" % (os.path.relpath(licfile, srctree), 
md5value))
+else:
+license_list.append(license)
+for license in license_list:
   licenses.append((license, os.path.relpath(licfile, srctree), 
md5value))
   
   # FIXME should we grab at least one source file with a license header and add that too?


I think to close this bug the code may need to go one step further and
effectively grep over the source tree.


Please keep in mind that we need a full license text and not only the
license name for license compliance. The current function only search
for license files with license text.


We'd probably want to list the value of any SPDX-License-Identifier: header
found in any of the source files for the user to then decide upon?


I think this is an other feature like a license checker because if you
have a SPDX-License-Identifier without a license text you have a license
violation.

This brings us to the problem that this code will interpret a file with
only a SPDX-License-Identifier as a license file with license text.


As I understand it the tool is there to help write a recipe so filling out
LICENSE and highlighting a missing full license text would be a valid approach
for the tool and helpful to the user?


Yes, but we should distinguish between license files which are guess via 
hash of the content and SPDX-License-Identifier which labels the source 
code’s license. In this case the SPDX-License-Identifier is non-material 
text from a license file and should be filtered out inside 
crunch_license function.


The collection of all used licenses via SPDX-License-Identifier is an 
additional feature and we need a warning if a SPDX-License-Identifier 
exists without license file.



It certainly isn't intended as full validation, just intended to assist the
creation of a recipe.


But this patch is an regress because it doesn't distinguish between a 
license file with a known hash and a mostly empty file with a 
SPDX-License-Identifier.


Regards
  Stefan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161366): 
https://lists.openembedded.org/g/openembedded-core/message/161366
Mute This Topic: https://lists.openembedded.org/mt/7504/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2 3/3] systemd: Minimize udev package size if DISTRO_FEATURES contains systemd

[Stefan Herbrechtsmeier]

Am 06.02.22 um 10:53 schrieb Richard Purdie:

On Sun, 2022-02-06 at 08:10 +, Richard Purdie via lists.openembedded.org
wrote:

On Wed, 2022-02-02 at 08:35 +0100, Stefan Herbrechtsmeier wrote:

From: Stefan Herbrechtsmeier 

Link udev shared with systemd helper to minimize the udev package size
if DISTRO_FEATURES contains systemd.

It is only usefull to link udev static with systemd helper if udev
should be installed without systemd.

Signed-off-by: Stefan Herbrechtsmeier 

---

(no changes since v1)

  meta/recipes-core/systemd/systemd_249.7.bb | 1 +
  1 file changed, 1 insertion(+)



This seems to break some of our tests:

https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/4708
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3121
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3085
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/3089
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3127

Presumably it causes extra dependencies to be pulled in causing the file overlap
where there was previously none.


Thinking a bit more, I think this needs to be:

diff --git a/meta/recipes-core/systemd/systemd_250.3.bb 
b/meta/recipes-core/systemd/systemd_250.3.bb
index 2b6bfcc2c25..7851c4c16a7 100644
--- a/meta/recipes-core/systemd/systemd_250.3.bb
+++ b/meta/recipes-core/systemd/systemd_250.3.bb
@@ -69,7 +69,7 @@ PACKAGECONFIG ??= " \
  ${@bb.utils.filter('DISTRO_FEATURES', 'acl audit efi ldconfig pam selinux 
smack usrmerge polkit seccomp', d)} \
  ${@bb.utils.contains('DISTRO_FEATURES', 'wifi', 'rfkill', '', d)} \
  ${@bb.utils.contains('DISTRO_FEATURES', 'x11', 'xkbcommon', '', d)} \
-${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'link-udev-shared', '', 
d)} \
+${@bb.utils.contains('DISTRO_FEATURES', 'sysvinit', '', 
'link-udev-shared', d)} \
  backlight \
  binfmt \
  gshadow \


which I'll test...


Maybe the systemd naming for the option is sub optimal. The 
link-udev-shared option create a dependency between udev and systemd 
package. It links udev against the libsystemd-shared library. Without it 
udev is linked statically against this library. The option should only 
be enabled if udev is always used together with systemd.


Regards
  Stefan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161403): 
https://lists.openembedded.org/g/openembedded-core/message/161403
Mute This Topic: https://lists.openembedded.org/mt/88853822/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2 3/3] systemd: Minimize udev package size if DISTRO_FEATURES contains systemd

[Stefan Herbrechtsmeier]

Am 06.02.22 um 09:10 schrieb Richard Purdie:

On Wed, 2022-02-02 at 08:35 +0100, Stefan Herbrechtsmeier wrote:

From: Stefan Herbrechtsmeier 

Link udev shared with systemd helper to minimize the udev package size
if DISTRO_FEATURES contains systemd.

It is only usefull to link udev static with systemd helper if udev
should be installed without systemd.

Signed-off-by: Stefan Herbrechtsmeier 

---

(no changes since v1)

  meta/recipes-core/systemd/systemd_249.7.bb | 1 +
  1 file changed, 1 insertion(+)



This seems to break some of our tests:

https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/4708
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3121
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3085
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/3089
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3127

Presumably it causes extra dependencies to be pulled in causing the file overlap
where there was previously none.


Sorry, I was told sysvinit with systemd isn't a valid use case anymore.

Would it be okay to enable it only if sysvinit isn't in distro?

Otherwise reject the change because it is impossible to know if a 
dependency between udev and systemd is okay or not.


Regads
  Stefan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161402): 
https://lists.openembedded.org/g/openembedded-core/message/161402
Mute This Topic: https://lists.openembedded.org/mt/88853822/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/4] wic: partition: Support valueless keys in sourceparams

[Stefan Herbrechtsmeier]

From: Stefan Herbrechtsmeier 

Accept valueless keys in sourceparams without equals sign (=) to match
the comment and support Boolean entries.

Signed-off-by: Stefan Herbrechtsmeier 
---

 scripts/lib/wic/partition.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index a25834048e..09e491dd49 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -171,7 +171,7 @@ class Partition():
 # Split sourceparams string of the form key1=val1[,key2=val2,...]
 # into a dict.  Also accepts valueless keys i.e. without =
 splitted = self.sourceparams.split(',')
-srcparams_dict = dict(par.split('=', 1) for par in splitted if par)
+srcparams_dict = dict((par.split('=', 1) + [None])[:2] for par in 
splitted if par)
 
 plugin = PluginMgr.get_plugins('source')[self.source]
 plugin.do_configure_partition(self, srcparams_dict, creator,
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161617): 
https://lists.openembedded.org/g/openembedded-core/message/161617
Mute This Topic: https://lists.openembedded.org/mt/89049084/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/4] wic: rawcopy: Add support for packed images

[Stefan Herbrechtsmeier]

From: Stefan Herbrechtsmeier 

Add support for packed images to wic rawcopy handler do minimize disk
usage in deploy directory and reuse of packed images between wic and
swupdate. Add `unpack` to sourceparams to unpack an bz2, gz and xz
archives.

Example:
part / --source rawcopy 
--sourceparams="file=core-image-minimal-qemu.ext4.gz,unpack"

Signed-off-by: Stefan Herbrechtsmeier 

---

 scripts/lib/wic/plugins/source/rawcopy.py | 28 ++-
 1 file changed, 27 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/wic/plugins/source/rawcopy.py 
b/scripts/lib/wic/plugins/source/rawcopy.py
index fa7b1eb8ac..7816e00e49 100644
--- a/scripts/lib/wic/plugins/source/rawcopy.py
+++ b/scripts/lib/wic/plugins/source/rawcopy.py
@@ -4,6 +4,8 @@
 
 import logging
 import os
+import signal
+import subprocess
 
 from wic import WicError
 from wic.pluginbase import SourcePlugin
@@ -38,6 +40,24 @@ class RawCopyPlugin(SourcePlugin):
 
 exec_cmd(cmd)
 
+@staticmethod
+def do_image_uncompression(src, dst, workdir):
+def subprocess_setup():
+# Python installs a SIGPIPE handler by default. This is usually 
not what
+# non-Python subprocesses expect.
+# SIGPIPE errors are known issues with gzip/bash
+signal.signal(signal.SIGPIPE, signal.SIG_DFL)
+
+decompressor = {
+".bz2": "bzip2",
+".gz": "gzip",
+".xz": "xz"
+}.get(os.path.splitext(src)[1])
+if not decompressor:
+raise WicError("Compression not support")
+cmd = "%s -dc %s > %s" % (decompressor, src, dst)
+subprocess.call(cmd, preexec_fn=subprocess_setup, shell=True, 
cwd=workdir)
+
 @classmethod
 def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
  oe_builddir, bootimg_dir, kernel_dir,
@@ -56,7 +76,13 @@ class RawCopyPlugin(SourcePlugin):
 if 'file' not in source_params:
 raise WicError("No file specified")
 
-src = os.path.join(kernel_dir, source_params['file'])
+if 'unpack' in source_params:
+img = os.path.join(kernel_dir, source_params['file'])
+src = os.path.join(cr_workdir, 
os.path.splitext(source_params['file'])[0])
+RawCopyPlugin.do_image_uncompression(img, src, cr_workdir)
+else:
+src = os.path.join(kernel_dir, source_params['file'])
+
 dst = os.path.join(cr_workdir, "%s.%s" % 
(os.path.basename(source_params['file']), part.lineno))
 
 if not os.path.exists(os.path.dirname(dst)):
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161618): 
https://lists.openembedded.org/g/openembedded-core/message/161618
Mute This Topic: https://lists.openembedded.org/mt/89049086/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 3/4] selftest: wic: Remove requirement of syslinux from test_rawcopy_plugin

[Stefan Herbrechtsmeier]

From: Stefan Herbrechtsmeier 

Remove bootimg-pcbios from wks to eliminate requirement of syslinux from
test_rawcopy_plugin to avoid the following error.

ERROR: Couldn't find correct bootimg_dir, exiting

Signed-off-by: Stefan Herbrechtsmeier 
---

 meta/lib/oeqa/selftest/cases/wic.py | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/wic.py 
b/meta/lib/oeqa/selftest/cases/wic.py
index 5fc8e65142..96b3e1b6a5 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -1070,10 +1070,8 @@ class Wic2(WicTestCase):
 img = 'core-image-minimal'
 machine = get_bb_var('MACHINE', img)
 with NamedTemporaryFile("w", suffix=".wks") as wks:
-wks.writelines(['part /boot --active --source bootimg-pcbios\n',
-'part / --source rawcopy 
--sourceparams="file=%s-%s.ext4" --use-uuid\n'\
- % (img, machine),
-'bootloader --timeout=0 
--append="console=ttyS0,115200n8"\n'])
+wks.write('part / --source rawcopy 
--sourceparams="file=%s-%s.ext4"\n'\
+  % (img, machine))
 wks.flush()
 cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir)
 runCmd(cmd)
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161619): 
https://lists.openembedded.org/g/openembedded-core/message/161619
Mute This Topic: https://lists.openembedded.org/mt/89049088/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 4/4] selftest: wic: Add rawcopy plugin unpack test

[Stefan Herbrechtsmeier]

From: Stefan Herbrechtsmeier 

Signed-off-by: Stefan Herbrechtsmeier 

---

 meta/lib/oeqa/selftest/cases/wic.py | 18 +++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/wic.py 
b/meta/lib/oeqa/selftest/cases/wic.py
index 96b3e1b6a5..a021f8d84b 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -1065,13 +1065,14 @@ class Wic2(WicTestCase):
 self.assertEqual(1, status, 'Failed to run command "%s": %s' % 
(cmd, output))
 self.assertEqual(output, '2')
 
-def test_rawcopy_plugin(self):
+def _rawcopy_plugin(self, fstype):
 """Test rawcopy plugin"""
 img = 'core-image-minimal'
 machine = get_bb_var('MACHINE', img)
+params = ',unpack' if fstype.endswith('.gz') else ''
 with NamedTemporaryFile("w", suffix=".wks") as wks:
-wks.write('part / --source rawcopy 
--sourceparams="file=%s-%s.ext4"\n'\
-  % (img, machine))
+wks.write('part / --source rawcopy 
--sourceparams="file=%s-%s.%s%s"\n'\
+  % (img, machine, fstype, params))
 wks.flush()
 cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir)
 runCmd(cmd)
@@ -1079,6 +1080,17 @@ class Wic2(WicTestCase):
 out = glob(self.resultdir + "%s-*direct" % wksname)
 self.assertEqual(1, len(out))
 
+def test_rawcopy_plugin(self):
+self._rawcopy_plugin('ext4')
+
+def test_rawcopy_plugin_unpack(self):
+fstype = 'ext4.gz'
+config = 'IMAGE_FSTYPES = "%s"\n' % fstype
+self.append_config(config)
+self.assertEqual(0, bitbake('core-image-minimal').status)
+self.remove_config(config)
+self._rawcopy_plugin(fstype)
+
 def test_empty_plugin(self):
 """Test empty plugin"""
 config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "test_empty_plugin.wks"\n'
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161620): 
https://lists.openembedded.org/g/openembedded-core/message/161620
Mute This Topic: https://lists.openembedded.org/mt/89049089/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/2] gcc-target: fix glob to remove gcc- binary

[Stefan Herbrechtsmeier]

From: Stefan Herbrechtsmeier 

The glob to remove the gcc- binary expects a single-digit major
version which is no longer true.

Signed-off-by: Stefan Herbrechtsmeier 
---

 meta/recipes-devtools/gcc/gcc-target.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/gcc/gcc-target.inc 
b/meta/recipes-devtools/gcc/gcc-target.inc
index bf55e692e6..bcea75b2fa 100644
--- a/meta/recipes-devtools/gcc/gcc-target.inc
+++ b/meta/recipes-devtools/gcc/gcc-target.inc
@@ -193,7 +193,7 @@ do_install () {
rm -f *c++*
 
# We don't care about the gcc- ones for this
-   rm -f *gcc-?.?*
+   rm -f *gcc-?*.?*
 
# Not sure why we end up with these but we don't want them...
rm -f ${TARGET_PREFIX}${TARGET_PREFIX}*
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161615): 
https://lists.openembedded.org/g/openembedded-core/message/161615
Mute This Topic: https://lists.openembedded.org/mt/89048989/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/2] gcc-target: move cc1plus to g++ package

[Stefan Herbrechtsmeier]

From: Stefan Herbrechtsmeier 

Move cc1plus from gcc to g++ package. Therefor, remove the duplicate
FILES entry from gcc package and keep the entry in g++ package.

Signed-off-by: Stefan Herbrechtsmeier 

---

 meta/recipes-devtools/gcc/gcc-target.inc | 1 -
 1 file changed, 1 deletion(-)

diff --git a/meta/recipes-devtools/gcc/gcc-target.inc 
b/meta/recipes-devtools/gcc/gcc-target.inc
index bcea75b2fa..66f737c9dc 100644
--- a/meta/recipes-devtools/gcc/gcc-target.inc
+++ b/meta/recipes-devtools/gcc/gcc-target.inc
@@ -44,7 +44,6 @@ FILES:${PN} = "\
 ${bindir}/${TARGET_PREFIX}gcc* \
 ${bindir}/${TARGET_PREFIX}lto* \
 ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/collect2* \
-${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1plus \
 ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/g++-mapper-server \
 ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/lto* \
 ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/lib*${SOLIBS} \
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161616): 
https://lists.openembedded.org/g/openembedded-core/message/161616
Mute This Topic: https://lists.openembedded.org/mt/89048993/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2 3/3] systemd: Minimize udev package size if DISTRO_FEATURES contains systemd

[Stefan Herbrechtsmeier]

Am 06.02.22 um 17:43 schrieb Stefan Herbrechtsmeier:

Am 06.02.22 um 10:53 schrieb Richard Purdie:
On Sun, 2022-02-06 at 08:10 +, Richard Purdie via 
lists.openembedded.org

wrote:

On Wed, 2022-02-02 at 08:35 +0100, Stefan Herbrechtsmeier wrote:

From: Stefan Herbrechtsmeier 

Link udev shared with systemd helper to minimize the udev package size
if DISTRO_FEATURES contains systemd.

It is only usefull to link udev static with systemd helper if udev
should be installed without systemd.

Signed-off-by: Stefan Herbrechtsmeier 



---

(no changes since v1)

  meta/recipes-core/systemd/systemd_249.7.bb | 1 +
  1 file changed, 1 insertion(+)



This seems to break some of our tests:

https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/4708
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3121
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3085
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/3089
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3127

Presumably it causes extra dependencies to be pulled in causing the 
file overlap

where there was previously none.


Thinking a bit more, I think this needs to be:

diff --git a/meta/recipes-core/systemd/systemd_250.3.bb 
b/meta/recipes-core/systemd/systemd_250.3.bb

index 2b6bfcc2c25..7851c4c16a7 100644
--- a/meta/recipes-core/systemd/systemd_250.3.bb
+++ b/meta/recipes-core/systemd/systemd_250.3.bb
@@ -69,7 +69,7 @@ PACKAGECONFIG ??= " \
  ${@bb.utils.filter('DISTRO_FEATURES', 'acl audit efi ldconfig 
pam selinux smack usrmerge polkit seccomp', d)} \

  ${@bb.utils.contains('DISTRO_FEATURES', 'wifi', 'rfkill', '', d)} \
  ${@bb.utils.contains('DISTRO_FEATURES', 'x11', 'xkbcommon', '', 
d)} \
-    ${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 
'link-udev-shared', '', d)} \
+    ${@bb.utils.contains('DISTRO_FEATURES', 'sysvinit', '', 
'link-udev-shared', d)} \

  backlight \
  binfmt \
  gshadow \


which I'll test...


Maybe the systemd naming for the option is sub optimal. The 
link-udev-shared option create a dependency between udev and systemd 
package. It links udev against the libsystemd-shared library. Without it 
udev is linked statically against this library. The option should only 
be enabled if udev is always used together with systemd.


Sorry, I should read the patch careful.

I we can assume that udev from systemd is only used with sysvinit or 
systemd, this is okay.


Regards
  Stefan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161404): 
https://lists.openembedded.org/g/openembedded-core/message/161404
Mute This Topic: https://lists.openembedded.org/mt/88853822/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2 3/3] systemd: Minimize udev package size if DISTRO_FEATURES contains systemd

[Stefan Herbrechtsmeier]

Am 06.02.2022 um 20:05 schrieb Mark Hatle:

On 2/6/22 11:35 AM, Richard Purdie wrote:

On Sun, 2022-02-06 at 18:31 +0100, Alexander Kanavin wrote:

On Sun, 6 Feb 2022 at 18:27, Mark Hatle 
wrote:

It definitely works in Honister (I'm actively using it.)

We produce a single package set, then allow for multiple image 
recipes to

select
which init manager to use.  This simplifies the eSDK, sstate-cache and
binary
package feed -- while still giving the end user the ability to switch.




Selection of init manager is a distro setting, it can be used by 
component
recipes to make decisions, and overriding it from images is neither 
tested nor

supported.


There is a hybrid where you can enable sysvinit and systemd at the 
same time and
use them in different situations. We do test that and it is that test 
which

showed issues on the autobuilder with this patch.

It isn't as easy as pick and chose either in a given image but handy for
initramfs and some other uses.


I've got the following in my distro:

# Create packages that support both systemd and sysvinit
# but only on arm/arm64 systems, microblaze is sysvinit only
DISTRO_FEATURES_BACKFILL_CONSIDERED:remove:arm = "sysvinit systemd"
DISTRO_FEATURES_BACKFILL_CONSIDERED:remove:aarch64 = "sysvinit systemd"
DISTRO_FEATURES_BACKFILL:append:arm = " systemd"
DISTRO_FEATURES_BACKFILL:append:aarch64 = " systemd”

INIT_MANAGER_DEFAULT = "systemd"
INIT_MANAGER_DEFAULT:microblaze = "sysvinit"
INIT_MANAGER_DEFAULT:zynq = "sysvinit"

INIT_MANAGER ?= "${INIT_MANAGER_DEFAULT}"

Then a few configurations change the INIT_MANAGER, and (almost) all of 
the sstate is re-usable and things don't need to be re-built.


The issue I have, microblaze if the filesystem is over about 80mb it 
doesn't fit into memory.  On Arm, some devices are memory limited the 
same way microblaze is, while some have full access to flash storage and 
thus can use systemd.  And on aarch64 - most defined (but not all) have 
flash storage.


This is all about sstate-cache re-use (time to build the image) and 
ability to have a SINGLE binary distribution that supports the image 
size requirements of the user.


After the change from Richard the optimization will only take place if 
DISTRO_FEATURES doesn't contain sysvinit and you will get the 
independent systemd and udev packages for all your platforms.


Regards
  Stefan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161459): 
https://lists.openembedded.org/g/openembedded-core/message/161459
Mute This Topic: https://lists.openembedded.org/mt/88853822/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] bluez5: install /var/lib/bluetooth directory

[Stefan Herbrechtsmeier]

Am 08.04.2022 um 22:03 schrieb Markus Volk via lists.openembedded.org:
also had this issue and found out, that the bluetooth service did come 
up, once bluetooth was enabled


My fix was to edit like this

ReadWritePaths=:/var/lib/bluetooth

The colon ensures that the service won't fail if the directory doesn't 
exist


Regarding the documentation the ReadOnlyPaths and ReadWritePaths makes 
not sense because ProtectSystem=full mounts /usr, boot loader and /etc 
directories read-only.


Have somebody report the problem to the bluez project?

Regards
  Stefan





Am 08.04.22 um 13:01 schrieb Yi Zhao:


On 4/8/22 18:51, Stefan Herbrechtsmeier wrote:

Am 08.04.2022 um 11:23 schrieb Yi Zhao:


On 4/8/22 16:34, Stefan Herbrechtsmeier wrote:

Am 08.04.2022 um 10:15 schrieb Yi Zhao via lists.openembedded.org:

There is a bluetooth service startup failure:
bluetooth.service: Failed at step NAMESPACE spawning 
/usr/libexec/bluetooth/bluetoothd: No such file or directory
bluetooth.service: Failed to set up mount namespacing: 
/run/systemd/unit-root/var/lib/bluetooth: No such file or directory


This is because the directory /var/lib/bluetooth has been listed in
ReadWritePaths= in the service unit file but does not exist. We 
need to

create it before service startup.

Signed-off-by: Yi Zhao 
---
  meta/recipes-connectivity/bluez5/bluez5.inc | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/meta/recipes-connectivity/bluez5/bluez5.inc 
b/meta/recipes-connectivity/bluez5/bluez5.inc

index 79d4645ca8..ee2cdbcc59 100644
--- a/meta/recipes-connectivity/bluez5/bluez5.inc
+++ b/meta/recipes-connectivity/bluez5/bluez5.inc
@@ -102,6 +102,8 @@ do_install:append() {
  # Patch python tools to use Python 3; they should be source 
compatible, but

  # still refer to Python 2 in the shebang
  sed -i -e '1s,#!.*python.*,#!${bindir}/python3,' 
${D}${libdir}/bluez/test/*

+
+    install -d ${D}${localstatedir}/lib/bluetooth


Please use systemd StateDirectory= [1]. Maybe it is reasonable to 
replace the ReadWritePaths= with StateDirectory=.


[1] 
https://www.freedesktop.org/software/systemd/man/systemd.exec.html#RuntimeDirectory= 




Thanks Stefank, but the ReadWritePaths= is set by bluez:

https://github.com/bluez/bluez/commit/442d211b5f30f00d5ddd69b43385a03c1428ac45 



And who creates the directory before the patch?


There is no need this directory before the patch. The service can 
startup without this directory.





The systemd solution to create a state directory is StateDirectory= 
(or tmpfiles.d for older versions of systemd). Systemd supports 
stateless system [1] and therefore every service must work with an 
empty /var/lib.



I'll try adding StateDirecotry to service unit file to  see if this 
can solve the problem.  Thanks.



//Yi



This patch hide a bug in bluez and the problem should be fixed in bluez.

[1] https://0pointer.net/blog/projects/stateless.html










-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164217): 
https://lists.openembedded.org/g/openembedded-core/message/164217
Mute This Topic: https://lists.openembedded.org/mt/90331367/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] bluez5: install /var/lib/bluetooth directory

[Stefan Herbrechtsmeier]

Hi,

Am 11.04.2022 um 09:17 schrieb Stefan Herbrechtsmeier via 
lists.openembedded.org:

Am 08.04.2022 um 22:03 schrieb Markus Volk via lists.openembedded.org:
also had this issue and found out, that the bluetooth service did come 
up, once bluetooth was enabled


My fix was to edit like this

ReadWritePaths=:/var/lib/bluetooth

The colon ensures that the service won't fail if the directory doesn't 
exist


Regarding the documentation the ReadOnlyPaths and ReadWritePaths makes 
not sense because ProtectSystem=full mounts /usr, boot loader and /etc 
directories read-only.


Have somebody report the problem to the bluez project?


I have open an issue:
https://github.com/bluez/bluez/issues/329

I will post a patch when we come to a solution.

Regards
  Stefan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164308): 
https://lists.openembedded.org/g/openembedded-core/message/164308
Mute This Topic: https://lists.openembedded.org/mt/90331367/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] bluez5: install /var/lib/bluetooth directory

[Stefan Herbrechtsmeier]

Am 08.04.2022 um 10:15 schrieb Yi Zhao via lists.openembedded.org:

There is a bluetooth service startup failure:
bluetooth.service: Failed at step NAMESPACE spawning 
/usr/libexec/bluetooth/bluetoothd: No such file or directory
bluetooth.service: Failed to set up mount namespacing: 
/run/systemd/unit-root/var/lib/bluetooth: No such file or directory

This is because the directory /var/lib/bluetooth has been listed in
ReadWritePaths= in the service unit file but does not exist. We need to
create it before service startup.

Signed-off-by: Yi Zhao 
---
  meta/recipes-connectivity/bluez5/bluez5.inc | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/meta/recipes-connectivity/bluez5/bluez5.inc 
b/meta/recipes-connectivity/bluez5/bluez5.inc
index 79d4645ca8..ee2cdbcc59 100644
--- a/meta/recipes-connectivity/bluez5/bluez5.inc
+++ b/meta/recipes-connectivity/bluez5/bluez5.inc
@@ -102,6 +102,8 @@ do_install:append() {
# Patch python tools to use Python 3; they should be source compatible, 
but
# still refer to Python 2 in the shebang
sed -i -e '1s,#!.*python.*,#!${bindir}/python3,' 
${D}${libdir}/bluez/test/*
+
+   install -d ${D}${localstatedir}/lib/bluetooth


Please use systemd StateDirectory= [1]. Maybe it is reasonable to 
replace the ReadWritePaths= with StateDirectory=.


[1] 
https://www.freedesktop.org/software/systemd/man/systemd.exec.html#RuntimeDirectory=



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164156): 
https://lists.openembedded.org/g/openembedded-core/message/164156
Mute This Topic: https://lists.openembedded.org/mt/90331367/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] recipetool: Do not use mutable default arguments in Python

[Stefan Herbrechtsmeier]

From: Stefan Herbrechtsmeier 

Remove mutable default arguments in Python because they can lead to all
sorts of nasty and horrible bugs.

https://florimond.dev/en/posts/2018/08/python-mutable-defaults-are-the-source-of-all-evil/

Revert `recipetool: Change default paramter fallback_licenses of
function split_pkg_licenses from None to []` and instead check
fallback_licenses before use.

Signed-off-by: Stefan Herbrechtsmeier 

---

 scripts/lib/recipetool/create.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 220465ed2f..824ac6350d 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1235,7 +1235,7 @@ def guess_license(srctree, d):
 
 return licenses
 
-def split_pkg_licenses(licvalues, packages, outlines, fallback_licenses=[], 
pn='${PN}'):
+def split_pkg_licenses(licvalues, packages, outlines, fallback_licenses=None, 
pn='${PN}'):
 """
 Given a list of (license, path, md5sum) as returned by guess_license(),
 a dict of package name to path mappings, write out a set of
@@ -1258,7 +1258,7 @@ def split_pkg_licenses(licvalues, packages, outlines, 
fallback_licenses=[], pn='
 for pkgname in packages:
 # Assume AND operator between license files
 license = ' & '.join(list(set(pkglicenses.get(pkgname, ['Unknown'] 
or 'Unknown'
-if license == 'Unknown' and pkgname in fallback_licenses:
+if license == 'Unknown' and fallback_licenses and pkgname in 
fallback_licenses:
 license = fallback_licenses[pkgname]
 licenses = tidy_licenses(license)
 license = ' & '.join(licenses)
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164407): 
https://lists.openembedded.org/g/openembedded-core/message/164407
Mute This Topic: https://lists.openembedded.org/mt/90461493/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2] bluez5: Add fix for startup issues under systemd

[Stefan Herbrechtsmeier]

Hi Richard,

should I update your patch or only the Upstream-Status or will you 
prefer to do it yourself?


Regards
  Stefan

Am 14.04.2022 um 22:52 schrieb Richard Purdie via lists.openembedded.org:

The systemd bluetooth service failed to start. Add a workaround for this whilst 
the
final fix is discussed upstream, https://github.com/bluez/bluez/issues/329.

Signed-off-by: Richard Purdie 
---

v2: Add ProtectSystem=strict
  meta/recipes-connectivity/bluez5/bluez5.inc   |  1 +
  .../bluez5/bluez5/fix_service.patch   | 30 +++
  2 files changed, 31 insertions(+)
  create mode 100644 meta/recipes-connectivity/bluez5/bluez5/fix_service.patch

diff --git a/meta/recipes-connectivity/bluez5/bluez5.inc 
b/meta/recipes-connectivity/bluez5/bluez5.inc
index 79d4645ca89..22dd07b3480 100644
--- a/meta/recipes-connectivity/bluez5/bluez5.inc
+++ b/meta/recipes-connectivity/bluez5/bluez5.inc
@@ -53,6 +53,7 @@ SRC_URI = 
"${KERNELORG_MIRROR}/linux/bluetooth/bluez-${PV}.tar.xz \
 ${@bb.utils.contains('DISTRO_FEATURES', 'systemd', '', 
'file://0001-Allow-using-obexd-without-systemd-in-the-user-sessio.patch', d)} \
 
file://0001-tests-add-a-target-for-building-tests-without-runnin.patch \
 file://0001-test-gatt-Fix-hung-issue.patch \
+   file://fix_service.patch \
 "
  S = "${WORKDIR}/bluez-${PV}"
  
diff --git a/meta/recipes-connectivity/bluez5/bluez5/fix_service.patch b/meta/recipes-connectivity/bluez5/bluez5/fix_service.patch

new file mode 100644
index 000..96fdf6b299c
--- /dev/null
+++ b/meta/recipes-connectivity/bluez5/bluez5/fix_service.patch
@@ -0,0 +1,30 @@
+The systemd bluetooth service failed to start because the /var/lib/bluetooth
+path of ReadWritePaths= is created by the bluetooth daemon itself.
+
+The commit systemd: Add more filesystem lockdown (442d211) add 
ReadWritePaths=/etc/bluetooth
+and ReadOnlyPaths=/var/lib/bluetooth options to the bluetooth systemd service.
+The existing ProtectSystem=full option mounts the /usr, the boot loader
+directories and /etc read-only. This means the two option are useless and 
could be removed.
+
+Upstream-Status: Submitted [https://github.com/bluez/bluez/issues/329]
+
+Index: bluez-5.64/src/bluetooth.service.in
+===
+--- bluez-5.64.orig/src/bluetooth.service.in
 bluez-5.64/src/bluetooth.service.in
+@@ -15,12 +15,12 @@ LimitNPROC=1
+
+ # Filesystem lockdown
+ ProtectHome=true
+-ProtectSystem=full
++ProtectSystem=strict
+ PrivateTmp=true
+ ProtectKernelTunables=true
+ ProtectControlGroups=true
+-ReadWritePaths=@statedir@
+-ReadOnlyPaths=@confdir@
++ConfigurationDirectory=bluetooth
++StateDirectory=bluetooth
+
+ # Execute Mappings
+ MemoryDenyWriteExecute=true






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164813): 
https://lists.openembedded.org/g/openembedded-core/message/164813
Mute This Topic: https://lists.openembedded.org/mt/90474022/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] bluez5: install /var/lib/bluetooth directory

[Stefan Herbrechtsmeier]

Am 08.04.2022 um 11:23 schrieb Yi Zhao:


On 4/8/22 16:34, Stefan Herbrechtsmeier wrote:

Am 08.04.2022 um 10:15 schrieb Yi Zhao via lists.openembedded.org:

There is a bluetooth service startup failure:
bluetooth.service: Failed at step NAMESPACE spawning 
/usr/libexec/bluetooth/bluetoothd: No such file or directory
bluetooth.service: Failed to set up mount namespacing: 
/run/systemd/unit-root/var/lib/bluetooth: No such file or directory


This is because the directory /var/lib/bluetooth has been listed in
ReadWritePaths= in the service unit file but does not exist. We need to
create it before service startup.

Signed-off-by: Yi Zhao 
---
  meta/recipes-connectivity/bluez5/bluez5.inc | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/meta/recipes-connectivity/bluez5/bluez5.inc 
b/meta/recipes-connectivity/bluez5/bluez5.inc

index 79d4645ca8..ee2cdbcc59 100644
--- a/meta/recipes-connectivity/bluez5/bluez5.inc
+++ b/meta/recipes-connectivity/bluez5/bluez5.inc
@@ -102,6 +102,8 @@ do_install:append() {
  # Patch python tools to use Python 3; they should be source 
compatible, but

  # still refer to Python 2 in the shebang
  sed -i -e '1s,#!.*python.*,#!${bindir}/python3,' 
${D}${libdir}/bluez/test/*

+
+    install -d ${D}${localstatedir}/lib/bluetooth


Please use systemd StateDirectory= [1]. Maybe it is reasonable to 
replace the ReadWritePaths= with StateDirectory=.


[1] 
https://www.freedesktop.org/software/systemd/man/systemd.exec.html#RuntimeDirectory= 




Thanks Stefank, but the ReadWritePaths= is set by bluez:

https://github.com/bluez/bluez/commit/442d211b5f30f00d5ddd69b43385a03c1428ac45


And who creates the directory before the patch?

The systemd solution to create a state directory is StateDirectory= (or 
tmpfiles.d for older versions of systemd). Systemd supports stateless 
system [1] and therefore every service must work with an empty /var/lib.


This patch hide a bug in bluez and the problem should be fixed in bluez.

[1] https://0pointer.net/blog/projects/stateless.html

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164158): 
https://lists.openembedded.org/g/openembedded-core/message/164158
Mute This Topic: https://lists.openembedded.org/mt/90331367/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] Easyish reproducibility issue for someone?

[Stefan Herbrechtsmeier]

Hi Richard,

Am 22.06.2022 um 15:10 schrieb Richard Purdie via lists.openembedded.org:

There is a reproducibility issue which shouldn't be too hard to fix if
someone had a few minutes to help out:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=14845

It is in llvm and is a cmake configuration issue. Rather than setting
that option based on the presence of python modules (with python from
the host), we should add a configuration option to force it to be
disabled in our case. Should be upstreamable with the right patch.


I'm not able to test it at the moment but it should be possible to set 
the variables (PY_PYGMENTS_FOUND, PY_PYGMENTS_LEXERS_C_CPP_FOUND and 
PY_YAML_FOUND) to OFF via EXTRA_OECMAKE:


-DPY_PYGMENTS_FOUND=OFF

The find_python_module function in config-ix.cmake returns early if the 
variable PY_${module_upper}_FOUND is defined.


Regards
  Stefan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167237): 
https://lists.openembedded.org/g/openembedded-core/message/167237
Mute This Topic: https://lists.openembedded.org/mt/91921251/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] Easyish reproducibility issue for someone?

[Stefan Herbrechtsmeier]

Hi Markus,

Am 22.06.2022 um 16:18 schrieb Markus Volk via lists.openembedded.org:

Hi,

couldn't we just do it like this?

PACKAGECONFIG ?= ""
PACKAGECONFIG[optviewer] = 
"-DLLVM_HAVE_OPT_VIEWER_MODULES=ON,-DLLVM_HAVE_OPT_VIEWER_MODULES=OFF,,python3-pygments 
python3-pyyaml"


Depending on wether the PACKAGECONFIG option is taken it looks like this 
in CMakeCache.txt:


//No help, variable specified on the command line.
LLVM_HAVE_OPT_VIEWER_MODULES:UNINITIALIZED=OFF

//No help, variable specified on the command line.
LLVM_HAVE_OPT_VIEWER_MODULES:UNINITIALIZED=ON


Have you test it? The config-ix.cmake file sets the 
LLVM_HAVE_OPT_VIEWER_MODULES variable [1].


Instead of LLVM_HAVE_OPT_VIEWER_MODULES I would set PY_PYGMENTS_FOUND, 
PY_PYGMENTS_LEXERS_C_CPP_FOUND and PY_YAML_FOUND.


[1] 
https://github.com/llvm/llvm-project/blob/main/llvm/cmake/config-ix.cmake#L693


Regards
  Stefan


Am 22.06.22 um 15:54 schrieb Stefan Herbrechtsmeier:

Hi Richard,

Am 22.06.2022 um 15:10 schrieb Richard Purdie via lists.openembedded.org:

There is a reproducibility issue which shouldn't be too hard to fix if
someone had a few minutes to help out:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=14845

It is in llvm and is a cmake configuration issue. Rather than setting
that option based on the presence of python modules (with python from
the host), we should add a configuration option to force it to be
disabled in our case. Should be upstreamable with the right patch.


I'm not able to test it at the moment but it should be possible to set 
the variables (PY_PYGMENTS_FOUND, PY_PYGMENTS_LEXERS_C_CPP_FOUND and 
PY_YAML_FOUND) to OFF via EXTRA_OECMAKE:


-DPY_PYGMENTS_FOUND=OFF

The find_python_module function in config-ix.cmake returns early if 
the variable PY_${module_upper}_FOUND is defined.


Regards
  Stefan


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167241): 
https://lists.openembedded.org/g/openembedded-core/message/167241
Mute This Topic: https://lists.openembedded.org/mt/91921251/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 3/5] poky-meta: add go vendor class for offline builds

[Stefan Herbrechtsmeier]

From: Lukas Funke 

Signed-off-by: Lukas Funke 
Signed-off-by: Stefan Herbrechtsmeier 
---

 meta/classes/go-vendor.bbclass | 68 ++
 1 file changed, 68 insertions(+)
 create mode 100644 meta/classes/go-vendor.bbclass

diff --git a/meta/classes/go-vendor.bbclass b/meta/classes/go-vendor.bbclass
new file mode 100644
index 00..a1a740a1fc
--- /dev/null
+++ b/meta/classes/go-vendor.bbclass
@@ -0,0 +1,68 @@
+# Copyright 2022 (C) Weidmueller GmbH & Co KG
+# Author: Lukas Funke 
+#
+# Handle Go vendor support for offline builds
+#
+# When importing Go modules, Go downloads the imported module using
+# a network (proxy) connection ahead of the compile stage. This contradicts 
+# the yocto build concept of fetching every source ahead of build-time
+# and supporting offline builds.
+#
+# To support offline builds, we use Go 'vendoring': module dependencies are 
+# downloaded during the fetch-phase and unpacked into the modules 'vendor'
+# folder. Additinally a manifest file is generated for the 'vendor' folder
+# 
+
+inherit go-mod
+
+def go_src_uri(repo, path=None, subdir=None, vcs='git', destsuffix_prefix = 
'git/src/import/vendor.fetch'):
+module_path = repo if not path else path
+src_uri = "{}://{};name={};destsuffix={}/{}".format(vcs, repo, \
+module_path.replace('/', '.'), \
+destsuffix_prefix, module_path)
+
+src_uri += ";subdir={}".format(subdir) if subdir else ""
+src_uri += ";nobranch=1;protocol=https" if vcs == "git" else ""
+
+return src_uri
+
+def go_generate_vendor_manifest(d):
+
+vendor_dir = os.path.join(os.path.basename(d.getVar('S')),
+'src', d.getVar('GO_IMPORT'), "vendor")
+dst = os.path.join(vendor_dir, "modules.txt")
+
+go_modules = d.getVarFlags("GO_MODULE_PATH")
+with open(dst, "w") as manifest:
+for go_module in go_modules:
+module_path = d.getVarFlag("GO_MODULE_PATH", go_module)
+module_version = d.getVarFlag("GO_MODULE_VERSION", go_module)
+if module_path and module_version:
+manifest.write("# %s %s\n" % (module_path, module_version))
+manifest.write("## explicit\n")
+exclude = set(['vendor'])
+for subdir, dirs, files in os.walk(os.path.join(vendor_dir, 
module_path), topdown=True):
+dirs[:] = [d for d in dirs if d not in exclude]
+for file in files:
+if file.endswith(".go"):
+manifest.write(subdir[len(vendor_dir)+1:] + "\n")
+break
+
+python go_do_unpack:append() {
+src_uri = (d.getVar('SRC_URI') or "").split()
+if len(src_uri) == 0:
+return
+
+try:
+fetcher = bb.fetch2.Fetch(src_uri, d)
+src_folder = os.path.join(os.path.basename(d.getVar('S')),
+'src', d.getVar('GO_IMPORT'))
+vendor_src = os.path.join(src_folder, "vendor")
+vendor_dst = os.path.join(d.getVar('S'), "src", "import", 
"vendor.fetch")
+
+os.symlink(os.path.relpath(vendor_dst, src_folder), vendor_src)
+go_generate_vendor_manifest(d)
+
+except bb.fetch2.BBFetchException as e:
+raise bb.build.FuncFailed(e)
+}
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165324): 
https://lists.openembedded.org/g/openembedded-core/message/165324
Mute This Topic: https://lists.openembedded.org/mt/90928683/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/5] recipetool-create: add ensure_native_cmd function

[Stefan Herbrechtsmeier]

From: Lukas Funke 

Signed-off-by: Lukas Funke 
Signed-off-by: Stefan Herbrechtsmeier 
---

 scripts/lib/recipetool/create.py | 23 +++
 1 file changed, 23 insertions(+)

diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 824ac6350d..efcb82173e 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1375,6 +1375,29 @@ def convert_rpm_xml(xmlfile):
 values[varname] = child[0].text
 return values
 
+def ensure_native_cmd(tinfoil, cmd):
+"""Check if the command is available in the recipes"""
+if not tinfoil.recipes_parsed:
+tinfoil.parse_recipes()
+
+try:
+d = tinfoil.parse_recipe("%s-native" % cmd)
+except bb.providers.NoProvider:
+bb.error("Nothing provides '%s-native' which is required for the 
build" % cmd)
+bb.note("You will likely need to add a layer that provides %s" % cmd)
+sys.exit(14)
+
+bindir = d.getVar("STAGING_BINDIR_NATIVE")
+cmdpath = os.path.join(bindir, cmd)
+
+if not os.path.exists(cmdpath):
+tinfoil.build_targets("%s-native" % cmd, "addto_recipe_sysroot")
+
+if not os.path.exists(cmdpath):
+bb.error("Failed to add '%s' to sysroot" % cmd)
+sys.exit(14)
+
+return bindir
 
 def register_commands(subparsers):
 parser_create = subparsers.add_parser('create',
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165323): 
https://lists.openembedded.org/g/openembedded-core/message/165323
Mute This Topic: https://lists.openembedded.org/mt/90928682/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/5] create_npm: reuse ensure_native_cmd from create.py

[Stefan Herbrechtsmeier]

From: Lukas Funke 

Signed-off-by: Lukas Funke 
Signed-off-by: Stefan Herbrechtsmeier 
---

 scripts/lib/recipetool/create_npm.py | 28 ++--
 1 file changed, 2 insertions(+), 26 deletions(-)

diff --git a/scripts/lib/recipetool/create_npm.py 
b/scripts/lib/recipetool/create_npm.py
index 3394a89970..3aed59252c 100644
--- a/scripts/lib/recipetool/create_npm.py
+++ b/scripts/lib/recipetool/create_npm.py
@@ -18,6 +18,7 @@ from recipetool.create import RecipeHandler
 from recipetool.create import get_license_md5sums
 from recipetool.create import guess_license
 from recipetool.create import split_pkg_licenses
+from recipetool.create import ensure_native_cmd
 logger = logging.getLogger('recipetool')
 
 TINFOIL = None
@@ -54,31 +55,6 @@ class NpmRecipeHandler(RecipeHandler):
 
 return registry
 
-@staticmethod
-def _ensure_npm():
-"""Check if the 'npm' command is available in the recipes"""
-if not TINFOIL.recipes_parsed:
-TINFOIL.parse_recipes()
-
-try:
-d = TINFOIL.parse_recipe("nodejs-native")
-except bb.providers.NoProvider:
-bb.error("Nothing provides 'nodejs-native' which is required for 
the build")
-bb.note("You will likely need to add a layer that provides nodejs")
-sys.exit(14)
-
-bindir = d.getVar("STAGING_BINDIR_NATIVE")
-npmpath = os.path.join(bindir, "npm")
-
-if not os.path.exists(npmpath):
-TINFOIL.build_targets("nodejs-native", "addto_recipe_sysroot")
-
-if not os.path.exists(npmpath):
-bb.error("Failed to add 'npm' to sysroot")
-sys.exit(14)
-
-return bindir
-
 @staticmethod
 def _npm_global_configs(dev):
 """Get the npm global configuration"""
@@ -190,7 +166,7 @@ class NpmRecipeHandler(RecipeHandler):
 # npm version is high enough to ensure an efficient dependency tree
 # resolution and avoid issue with the shrinkwrap file format.
 # Moreover the native npm is mandatory for the build.
-bindir = self._ensure_npm()
+bindir = ensure_native_cmd(TINFOIL, "npm")
 
 d = bb.data.createCopy(TINFOIL.config_data)
 d.prependVar("PATH", bindir + ":")
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165325): 
https://lists.openembedded.org/g/openembedded-core/message/165325
Mute This Topic: https://lists.openembedded.org/mt/90928684/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 4/5] recipetool: add go recipe generator

[Stefan Herbrechtsmeier]

From: Lukas Funke 

Signed-off-by: Lukas Funke 
Signed-off-by: Stefan Herbrechtsmeier 
---

 scripts/lib/recipetool/create_go.py | 394 
 1 file changed, 394 insertions(+)
 create mode 100644 scripts/lib/recipetool/create_go.py

diff --git a/scripts/lib/recipetool/create_go.py 
b/scripts/lib/recipetool/create_go.py
new file mode 100644
index 00..4552e9b470
--- /dev/null
+++ b/scripts/lib/recipetool/create_go.py
@@ -0,0 +1,394 @@
+# Recipe creation tool - go support plugin
+#
+# Copyright (C) 2022 Weidmueller GmbH & Co KG
+# Author: Lukas Funke 
+#
+# Copyright (c) 2009 The Go Authors. All rights reserved.
+#
+#  SPDX-License-Identifier: BSD-3-Clause AND GPL-2.0-only
+#
+import bb.utils
+from collections import namedtuple
+from enum import Enum
+from html.parser import HTMLParser
+import json
+import logging
+import os
+import re
+import subprocess
+import sys
+import tempfile
+import shutil
+from urllib.error import URLError, HTTPError
+import urllib.parse
+import urllib.request
+
+from recipetool.create import RecipeHandler, handle_license_vars, 
ensure_native_cmd
+
+GoImport = namedtuple('GoImport', 'reporoot vcs repourl suffix')
+logger = logging.getLogger('recipetool')
+
+tinfoil = None
+
+re_pseudo_semver = 
re.compile(r"v([0-9]+)\.([0-9]+).([0-9]+|\([0-9]+\+1\))-(pre\.[0-9]+\.)?([0-9]+\.)?(?P[0-9]+)-(?P[0-9Aa-zA-Z]+)")
+re_semver = 
re.compile(r"^v(?P0|[1-9]\d*)\.(?P0|[1-9]\d*)\.(?P0|[1-9]\d*)(?:-(?P(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+(?P[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$")
+
+def tinfoil_init(instance):
+global tinfoil
+tinfoil = instance
+
+class GoRecipeHandler(RecipeHandler):
+
+def _resolve_repository_static(self, modulepath):
+_rootpath = None
+_vcs = None
+_repourl = None
+_suffix = None
+
+host, _, path = modulepath.partition('/')
+
+class vcs(Enum):
+pathprefix = "pathprefix"
+regexp = "regexp"
+vcs = "vcs"
+repo = "repo"
+check = "check"
+schemelessRepo = "schemelessRepo"
+
+# GitHub
+vcsGitHub = {}
+vcsGitHub[vcs.pathprefix] = "github.com"
+vcsGitHub[vcs.regexp] = 
re.compile(r'^(?Pgithub\.com/[A-Za-z0-9_.\-]+/[A-Za-z0-9_.\-]+)(/(?P[A-Za-z0-9_.\-]+))*$')
+vcsGitHub[vcs.vcs] = "git"
+vcsGitHub[vcs.repo] = "https://\g"
+
+# Bitbucket
+vcsBitbucket = {}
+vcsBitbucket[vcs.pathprefix] = "bitbucket.org"
+vcsBitbucket[vcs.regexp] = 
re.compile(r'^(?Pbitbucket\.org/(?P[A-Za-z0-9_.\-]+/[A-Za-z0-9_.\-]+))(/(?P[A-Za-z0-9_.\-]+))*$')
+vcsBitbucket[vcs.vcs] = "git"
+vcsBitbucket[vcs.repo] = "https://\g"
+
+# IBM DevOps Services (JazzHub)
+vcsIBMDevOps = {}
+vcsIBMDevOps[vcs.pathprefix] = "hub.jazz.net/git"
+vcsIBMDevOps[vcs.regexp] = 
re.compile(r'^(?Phub\.jazz\.net/git/[a-z0-9]+/[A-Za-z0-9_.\-]+)(/(?P[A-Za-z0-9_.\-]+))*$')
+vcsIBMDevOps[vcs.vcs] = "git"
+vcsIBMDevOps[vcs.repo] = "https://\g"
+
+# Git at Apache
+vcsApacheGit = {}
+vcsApacheGit[vcs.pathprefix] = "git.apache.org"
+vcsApacheGit[vcs.regexp] = 
re.compile(r'^(?Pgit\.apache\.org/[a-z0-9_.\-]+\.git)(/(?P[A-Za-z0-9_.\-]+))*$')
+vcsApacheGit[vcs.vcs] = "git"
+vcsApacheGit[vcs.repo] = "https://\g"
+
+# Git at OpenStack
+vcsOpenStackGit = {}
+vcsOpenStackGit[vcs.pathprefix] = "git.openstack.org"
+vcsOpenStackGit[vcs.regexp] = 
re.compile(r'^(?Pgit\.openstack\.org/[A-Za-z0-9_.\-]+/[A-Za-z0-9_.\-]+)(\.git)?(/(?P[A-Za-z0-9_.\-]+))*$')
+vcsOpenStackGit[vcs.vcs] = "git"
+vcsOpenStackGit[vcs.repo] = "https://\g"
+
+# chiselapp.com for fossil
+vcsChiselapp = {}
+vcsChiselapp[vcs.pathprefix] = "chiselapp.com"
+vcsChiselapp[vcs.regexp] = 
re.compile(r'^(?Pchiselapp\.com/user/[A-Za-z0-9]+/repository/[A-Za-z0-9_.\-]+)$')
+vcsChiselapp[vcs.vcs] = "fossil"
+vcsChiselapp[vcs.repo] = "https://\g"
+
+# General syntax for any server.
+# Must be last.
+vcsGeneralServer = {}
+vcsGeneralServer[vcs.regexp] = 
re.compile("(?P(?P([a-z0-9.\-]+\.)+[a-z0-9.\-]+(:[0-9]+)?(/~?[A-Za-z0-9_.\-]+)+?)\.(?Pbzr|fossil|git|hg|svn))(/~?(?P[A-Za-z0-9_.\-]+))*$")
+vcsGeneralServer[vcs.schemelessRepo] = True
+
+vcsPaths = [vcsGitHub, vcsBitbucket, vcsIBMDevOps, vcsApacheGit, 
vcsOpenStackGit, vcsChiselapp, vcsGeneralServer]
+
+if modulepath.startswith("example.net") or modulepath == "rsc.io":
+logger.warning

[OE-core] [PATCH 5/5] oe-selftest: add go recipe create selftest

[Stefan Herbrechtsmeier]

From: Lukas Funke 

Signed-off-by: Lukas Funke 

Signed-off-by: Stefan Herbrechtsmeier 
---

 meta/lib/oeqa/selftest/cases/recipetool.py | 88 ++
 1 file changed, 88 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/recipetool.py 
b/meta/lib/oeqa/selftest/cases/recipetool.py
index 510dae6bad..bb36e7b6d5 100644
--- a/meta/lib/oeqa/selftest/cases/recipetool.py
+++ b/meta/lib/oeqa/selftest/cases/recipetool.py
@@ -530,6 +530,94 @@ class RecipetoolTests(RecipetoolBase):
 libpath = os.path.join(get_bb_var('COREBASE'), 'scripts', 'lib', 
'recipetool')
 sys.path.insert(0, libpath)
 
+def test_recipetool_create_go(self):
+# Basic test to check go recipe generation
+def urifiy(url, version, path = None, subdir = None):
+path = ",path='%s'" % path if path else ''
+subdir = ",subdir='%s'" % subdir if subdir else ''
+return "${@go_src_uri('%s','%s'%s%s)}" % (url, path, subdir)
+
+temprecipe = os.path.join(self.tempdir, 'recipe')
+os.makedirs(temprecipe)
+recipefile = os.path.join(temprecipe, 'edgex-go_git.bb')
+srcuri = 'https://github.com/edgexfoundry/edgex-go.git'
+srcrev = "v2.2.0-dev.54"
+result = runCmd('recipetool create -o %s %s -S %s' % (temprecipe, 
srcuri, srcrev))
+self.assertTrue(os.path.isfile(recipefile))
+checkvars = {}
+src_uri = ['git://${GO_IMPORT};nobranch=1;name=${BPN}']
+checkvars['LIC_FILES_CHKSUM'] = 
set(['file://src/${GO_IMPORT}/LICENSE;md5=71a6955f3cd81a809549da266346dc59'])
+checkvars['GO_IMPORT'] = "github.com/edgexfoundry/edgex-go"
+inherits = ['go-vendor']
+dependencies = \
+[
+('bitbucket.org/bertimus9/systemstat'),
+
('github.com/edgexfoundry/go-mod-bootstrap','github.com/edgexfoundry/go-mod-bootstrap/v2'),
+
('github.com/edgexfoundry/go-mod-core-contracts''github.com/edgexfoundry/go-mod-core-contracts/v2'),
+
('github.com/edgexfoundry/go-mod-messaging','github.com/edgexfoundry/go-mod-messaging/v2'),
+
('github.com/edgexfoundry/go-mod-registry','github.com/edgexfoundry/go-mod-registry/v2'),
+
('github.com/edgexfoundry/go-mod-secrets','github.com/edgexfoundry/go-mod-secrets/v2'),
+('github.com/fxamacker/cbor','github.com/fxamacker/cbor/v2'),
+('github.com/golang-jwt/jwt','github.com/golang-jwt/jwt/v4'),
+('github.com/gomodule/redigo'),
+('github.com/google/uuid'),
+('github.com/gorilla/mux'),
+('github.com/lib/pq'),
+('github.com/pelletier/go-toml'),
+
('github.com/spiffe/go-spiffe','github.com/spiffe/go-spiffe/v2'),
+('github.com/stretchr/testify'),
+('go.googlesource.com/crypto','golang.org/x/crypto'),
+('gopkg.in/eapache/queue.v1'),
+('gopkg.in/yaml.v3'),
+('github.com/armon/go-metrics'),
+('github.com/cenkalti/backoff'),
+('github.com/davecgh/go-spew'),
+('github.com/eclipse/paho.mqtt.golang'),
+
('github.com/edgexfoundry/go-mod-configuration','github.com/edgexfoundry/go-mod-configuration/v2'),
+('github.com/fatih/color'),
+('github.com/go-kit/log'),
+('github.com/go-logfmt/logfmt'),
+('github.com/go-playground/locales'),
+('github.com/go-playground/universal-translator'),
+
('github.com/go-playground/validator','github.com/go-playground/validator/v10'),
+('github.com/go-redis/redis','github.com/go-redis/redis/v7'),
+('github.com/golang/protobuf'),
+('github.com/gorilla/websocket'),
+('github.com/hashicorp/consul','api'),
+('github.com/hashicorp/errwrap'),
+('github.com/hashicorp/go-cleanhttp'),
+('github.com/hashicorp/go-hclog'),
+('github.com/hashicorp/go-immutable-radix'),
+('github.com/hashicorp/go-multierror'),
+('github.com/hashicorp/go-rootcerts'),
+('github.com/hashicorp/golang-lru'),
+('github.com/hashicorp/serf'),
+('github.com/leodido/go-urn'),
+('github.com/mattn/go-colorable'),
+('github.com/mattn/go-isatty'),
+('github.com/mitchellh/consulstructure'),
+('github.com/mitchellh/copystructure'),
+('github.com/mitchellh/go-homedir'),
+('github.com/mitchellh/mapstructure'),
+('github.com/mitchellh/reflectwalk'),
+('github.com/pebbe/zmq4'),
+('github.com/pmezard/go-difflib'),
+('github.com/stretchr/

Re: [OE-core] [PATCH 1/5] recipetool-create: add ensure_native_cmd function

[Stefan Herbrechtsmeier]

Hi Luca,

Lukas will check the logs and fix the problem.

Regards
  Stefan

Am 09.05.2022 um 23:46 schrieb Luca Ceresoli:

Hello Stefan,

Il giorno Fri,  6 May 2022 08:59:13 +0200
"Stefan Herbrechtsmeier" 
ha scritto:


From: Lukas Funke 

Signed-off-by: Lukas Funke 
Signed-off-by: Stefan Herbrechtsmeier



Testing builds with your series trigger many build failures related to
recipetool. Can you check these logs?

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3557/steps/15/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3506/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/3527/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3566/steps/14/logs/stdio




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165487): 
https://lists.openembedded.org/g/openembedded-core/message/165487
Mute This Topic: https://lists.openembedded.org/mt/90928682/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Stefan Herbrechtsmeier]

Hi Martin,

Am 26.12.2022 um 10:59 schrieb Martin Jansa via lists.openembedded.org:
On Mon, Dec 26, 2022 at 10:38 AM Vyacheslav Yurkov 
 wrote:


Martin, I did see that patch series, but I was not sure about its
state, therefore I didn't try it. If I understood correctly, that
partially solves the issue, but the proper solution should involve
the fetcher, that's why it's not yet merged to master. Am I
missing something?


That's not my understanding.

The proposed changes for recipetool had some cosmetic and functional 
review comments and it doesn't work correctly for every possible go 
dependency (e.g. the 3 issues I've mentioned from telegram recipe), 
but it's very good starting point and we should be able to get it 
merged in master (and then everybody can more easily contribute fixes 
for the corner cases where it might not work well yet).


Even missing documentation IMHO shouldn't block merging it at this 
point, because there is a clear need this (as there are already 3 
partial implementations to fix this issue, neither 100% complete and 
nothing merged in master - the recipetool changes are IMHO the best of 
3 as it integrates with devtool as well and seems to be most complete).


go fetcher in bitbake is another possibility for future, but 
generating the long deps in SRC_URI is much better solution then 
enabling network in every go-mod recipe.


Does an agreement exist if we should use a package manager class which 
creates a wget fetcher URI (ex. pipi) or a specific fetcher (ex. crate 
and npm)? At the moment both specific fetchers have different URI 
styles, create their specific tar commands and doesn't reuse the wget 
fetcher. Furthermore, the crate fetcher doesn't check the integrity 
during fetch. The fetcher inside bitbake have the advantage that you can 
fetch the latest version (ex. AUTOREV), but it brings the risk of code 
duplication and missing features. If we have an agreement and a common 
style its easy to replace the function with a specific fetcher.


Regards
  Stefan


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#175317): 
https://lists.openembedded.org/g/openembedded-core/message/175317
Mute This Topic: https://lists.openembedded.org/mt/95827446/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 01/13] cargo-update-recipe-crates.bbclass: add a class to generate SRC_URI crate lists from Cargo.lock

[Stefan Herbrechtsmeier]

Hi Alex,

Am 07.12.2022 um 17:21 schrieb Alexander Kanavin:
On Wed 7. Dec 2022 at 16.28, Stefan Herbrechtsmeier 
 wrote:


Hi Alex,

Am 07.12.2022 um 09:53 schrieb Alexander Kanavin via
lists.openembedded.org <http://lists.openembedded.org>:
> On Wed, 7 Dec 2022 at 01:21, Sergey Bostandzhyan
 wrote:
>> I think this should be documented more prominently, as missing
information
>> on how to get going was the biggest obstacle, at least for me.
The benefit of
>> this workflow is, that no additional layer is needed and that
everything seems
>> to more or less work out of the box within the usual checkout.
> Thanks for the report, I totally agree that the rust recipe workflow
> should be documented somewhere in the official manuals, I'm just not
> sure where. Perhaps the 'common tasks' in
> https://docs.yoctoproject.org/dev-manual/index.html ?
>
> Can you write and propose a patch to the documentation repository?

Is the recipetool obsolete or why we have yust an other tool to
create a
recipe in oe-core?


Which another tool are you referring to? Cargo bitbake is a 3rd party 
project, one that I am not recommending to anyone in any way. And 
having a class to update or create a list of crates does not preclude 
its use in recipetool. Patches welcome.


A bit of politeness would be welcome too, Stefan, seriously. Watch 
your tone.


Sorry for the harsh tone but you ignore existing tools and add just 
another tool to oe-core without mention any reason or document it. Do 
you really expect that somebody will patch existing tools if main 
developers ignore existing tools and provide new tool for their use 
case? Why should somebody improve an existing tool, extend the 
documentation, add tests or even upstream its work if these same 
requirements don’t exist for the main developers. It looks like the 
requirements for foreign and main contributors are different and this 
doesn't encourage people to participant. Maybe this is only my personal 
feeling and I apologize my harsh tone, but the acceptance of patches 
should be comprehensible, and expectations should be the same for 
everyone. Should others answer to your comments that their solution 
doesn’t preclude your suggestion and that they welcome patches? For sure 
it takes more time to add rust support to recipetool but I think a 
second tool without a clear reason in oe-core hurts more in long term 
because its now unclear if new features (like checksums or licenses 
support) should be added to this tool or if this tool is only a 
temporary solution and should be replaced by recipetool in long term. 
Furthermore, this class is marked as a class for a recipe but shouldn’t 
be inherit by a recipe and manipulates a file inside the meta data.


Regards
  Stefan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17): 
https://lists.openembedded.org/g/openembedded-core/message/17
Mute This Topic: https://lists.openembedded.org/mt/94683148/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 01/13] cargo-update-recipe-crates.bbclass: add a class to generate SRC_URI crate lists from Cargo.lock

[Stefan Herbrechtsmeier]

Hi Alex,

Am 09.12.2022 um 13:09 schrieb Alexander Kanavin:

On Fri, 9 Dec 2022 at 11:39, Stefan Herbrechtsmeier
 wrote:

Sorry for the harsh tone but you ignore existing tools and add just another 
tool to oe-core without mention any reason or document it.

The reason to add the class was to be able to update existing
rust-based recipes without having to manually update long lists of
crates in those recipes, which is a rather urgent issue, more urgent
than having rust support in recipetool. The class does not replace
recipetool, in fact it can, and should be used by devtool/recipetool
once someone finds time and motivation to sit down and add (currently
missing) rust support to it (class functionality can be executed from
those tools via tinfoil.build_targets()). I do not have that time, and
I am not getting paid for doing that either. Oh, and we do not have a
maintainer for those tools. Would you like to volunteer for that?


Yes, I'm happy to maintain recipetool. We need to go on with our npm und 
go support anyway.


Regards
  Stefan


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#174492): 
https://lists.openembedded.org/g/openembedded-core/message/174492
Mute This Topic: https://lists.openembedded.org/mt/94683148/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 01/13] cargo-update-recipe-crates.bbclass: add a class to generate SRC_URI crate lists from Cargo.lock

[Stefan Herbrechtsmeier]

Hi Richard,

Am 09.12.2022 um 13:01 schrieb Richard Purdie:

On Fri, 2022-12-09 at 11:39 +0100, Stefan Herbrechtsmeier wrote:

Sorry for the harsh tone but you ignore existing tools and add just
another tool to oe-core without mention any reason or document it. Do
you really expect that somebody will patch existing tools if main
developers ignore existing tools and provide new tool for their use
case?

I'm not sure that is entirely fair criticism. We have some challenges
with some of the new languages. Rust support was in a separate layer.
We took some of those recipes into core, then ended up making
significant changes to them. We never took the external tool.

The hope in doing that was that we'd find a better way to make things
work. I think what Alex has done is an improvement over that external
tool and it experiments with a different way of handling things. I've
had generally quite positive feedback on the approach itself. Yes,
there are some issues with documentation and some people using it have
struggled with some usability issues. None of those look like they're
unfixable.


  Why should somebody improve an existing tool, extend the
documentation, add tests or even upstream its work if these same
requirements don’t exist for the main developers. It looks like the
requirements for foreign and main contributors are different and this
doesn't encourage people to participant.

I don't see us treating developers differently and I am concerned you
think we/I do. Any given solution that is proposed is evaluated on it's
pros and cons. In this case the solution is quite self contained and
allowed the approach to be experimented with whilst solving a real
world issue. If it doesn't work out we can easily drop it. The risk
from taking it is therefore low. Yes, I should probably insist on
documentation. In this case it is relatively simple code which is
relatively easily understood so I've been less worried about that up
front.

If it worked out well, we can integrate it further and document it. If
it doesn't it can be removed. FWIW I have heard people saying they like
the approach and that we should use it for some of the other languages
with challenges like this.


  Maybe this is only my personal feeling and I apologize my harsh
tone, but the acceptance of patches should be comprehensible, and
expectations should be the same for everyone.

I do try to ensure that. My "algorithm" for accepting patches is
probably not easily documented but I think the factors here which are
the standalone nature of the change and the easy with which we could
drop it if needed. As such it is in my "low risk" category of patches.

I'd note Alex has another patch which has been sitting for months
unmerged as it is in my "high risk" to the project category. I suspect
that one will not actually merge but I need to find the time to explain
why, right now it is more based on a feeling it is the wrong direction.


  Should others answer to your comments that their solution doesn’t
preclude your suggestion and that they welcome patches? For sure it
takes more time to add rust support to recipetool but I think a
second tool without a clear reason in oe-core hurts more in long term
because its now unclear if new features (like checksums or licenses
support) should be added to this tool or if this tool is only a
temporary solution and should be replaced by recipetool in long term.
Furthermore, this class is marked as a class for a recipe but
shouldn’t be inherit by a recipe and manipulates a file inside the
meta data.

We do have precedence for classes that help updates like this. Both
python and perl have code that adds tasks that function a bit like
this, in those cases for the core recipe.

Personally, I would ultimately like to see these operations handled by
recipetool and I suspect natural evolution of the code may head that
way. Both devtool and recipetool have used classes as a way to help
them perform operations so in that sense, this is actually a logical
development path for those tools.


Thanks for you feedback and the clarification. This helps me a lot.

Stefan


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#174461): 
https://lists.openembedded.org/g/openembedded-core/message/174461
Mute This Topic: https://lists.openembedded.org/mt/94683148/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 01/13] cargo-update-recipe-crates.bbclass: add a class to generate SRC_URI crate lists from Cargo.lock

[Stefan Herbrechtsmeier]

Hi Alex,

Am 07.12.2022 um 09:53 schrieb Alexander Kanavin via lists.openembedded.org:

On Wed, 7 Dec 2022 at 01:21, Sergey Bostandzhyan  wrote:

I think this should be documented more prominently, as missing information
on how to get going was the biggest obstacle, at least for me. The benefit of
this workflow is, that no additional layer is needed and that everything seems
to more or less work out of the box within the usual checkout.

Thanks for the report, I totally agree that the rust recipe workflow
should be documented somewhere in the official manuals, I'm just not
sure where. Perhaps the 'common tasks' in
https://docs.yoctoproject.org/dev-manual/index.html ?

Can you write and propose a patch to the documentation repository?


Is the recipetool obsolete or why we have yust an other tool to create a 
recipe in oe-core?


Regards
  Stefan


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#174359): 
https://lists.openembedded.org/g/openembedded-core/message/174359
Mute This Topic: https://lists.openembedded.org/mt/94683148/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-