Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Khem Raj]

I am seeing bunch of failures on meta-oe and meta-atmel

https://errors.yoctoproject.org/Errors/Details/600064/
https://errors.yoctoproject.org/Errors/Details/600065/
https://errors.yoctoproject.org/Errors/Details/600071/

On Fri, Aug 6, 2021 at 2:04 PM Tony Battersby  wrote:
>
> On 8/6/21 12:44 PM, Tony Battersby wrote:
> > On 8/6/21 1:33 AM, Richard Purdie wrote:
> >> On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote:
> >>> On 8/5/21 6:21 AM, Richard Purdie wrote:
>  On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
> > On 2021-07-27 9:34 a.m., Tony Battersby wrote:
> >> Currently, enabling Link Time Optimization via:
> >>
> >> require conf/distro/include/lto.inc
> >> DISTRO_FEATURES_append = " lto"
> >>
> >> makes the build non-reproducible (as in
> >> https://reproducible-builds.org/).  This patch set fixes some of the
> >> problems, but is not yet a complete solution.
> >>
> >> This patch set addresses the following bugs:
> >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
> >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
> >>
> >> openembedded-core patches
> >> 1/4: lto.inc: disable LTO for grub
> >> 2/4: gcc: Backport patch to make LTO builds more reproducible
> >> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
> >>
> >> openembedded-devel patches
> >> 4/4: net-snmp: fix QA Issue after LDFLAGS change
> >>
> >> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
> >> TARGET_CFLAGS).  This could possibly be done differently, such as by
> >> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
> >> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me 
> >> know
> >> if there is a better/preferred way to do it.
> >>
> >> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I 
> >> have
> >> included a patch to fix that.  Other packages might have been broken as
> >> well, although I haven't found any others.
> >>
> >> With all these patches, there are still many shared libraries in /lib
> >> and /usr/lib that are still not reproducible with LTO enabled because
> >> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
> >> patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
> >> the libbz2 package from the bzip2 recipe is a good example.  I do not
> >> have a patch for that problem.  I will leave that to someone else.
> >>
> >> So this is what we need to achieve reproducible LTO builds:
> >> *) This patch set
> >> *) A fix for libtool
> >> *) Add LTO to reproducible builds testing if appropriate (e.g.
> >> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
> >> *) Fixes for individual packages, if needed
> >> *) Once everything works, update the documentation again (I previously
> >> requested a documentation change for this bug, which will eventually
> >> show up at
> >> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
> > This is certainly a few steps in the right direction with a good outline
> > of what's next.
> >
> > I suspect that Richard has been busy with other things like overrides
> > and the update to glibc-2.34.
> >
> > Do you have any suggestions or questions about what we should do
> > with this patch series Richard?
>  Thanks for the reminder, I've been meaning to queue/test this which I 
>  will
>  do. The gcc version has been bumped but I ported the patch forward to it
>  and the patches are in master-next for testing. It is great to see this
>  support moving forward.
> 
>  Much as it pains me to admit having any libtool knowledge, I may be able 
>  to
>  help with that bit. There is an untested patch below which may address 
>  that
>  issue, or at least give a good hint on how to do it if it doesn't work. I
>  don't have an environment to test easily right now.
> 
>  Cheers,
> 
>  Richard
> >>> I tested your libtool patch in master-next with the bzip2 recipe.  It
> >>> works as intended; libbz2.so.1.0.6 and all the other output files are
> >>> now reproducible with LTO enabled.  I also verified that libtool is
> >>> passing the -f*-prefix-map flags to the link command in log.do_compile
> >>> as intended.
> >>>
> >>> I also verified that core-image-minimal builds successfully with the
> >>> libtool patch.
> >> Thanks for testing that, sounds like it helps that issue. I'll queue that 
> >> patch.
> >>
> >> Unfortunately even with the ruby fix I queued, we still see reproduciblity
> >> issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and 
> >> ruby-dbg
> >> packages:
> >>
> >> https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio
> >>
> >> We'll need to resolve those 

Re: [OE-core] [poky][dunfell][PATCH] lighttpd: Add patch for reuse large memory chunks

[Steve Sakoman]

On Thu, Aug 5, 2021 at 8:55 PM Purushottam choudhary
 wrote:
>
> Added 0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch
> to fix large memory usage for large file downloads
> from dynamic backends reuse or release large memory chunks.
>
> This issue is caused by a bug in the lighttpd 1.4.55 version and
> has been fixed in lighttpd 1.4.58. Hence, it is not needed for
> master and hardknott branch because lighttpd has 1.4.59 version.
>
> Link: 
> https://redmine.lighttpd.net/projects/lighttpd/repository/14/revisions/7ba521ffb4959f6f74a609d5d4acafc29a038337

This patch is throwing warnings on the autobuilder:

Applying patch 0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch
patching file src/chunk.c
Hunk #2 succeeded at 124 with fuzz 1.
patching file src/chunk.h
patching file src/http-header-glue.c

The context lines in the patches can be updated with devtool:

devtool modify lighttpd
devtool finish --force-patch-refresh lighttpd 

Could you fix this and send a V2 so that we can have clean builds on
the autobuilder?

Thanks!

Steve


> Signed-off-by: Purushottam Choudhary 
> ---
>  ...large-mem-chunks-fix-mem-usage-fixes-3033.patch | 225 
> +
>  meta/recipes-extended/lighttpd/lighttpd_1.4.55.bb  |   1 +
>  2 files changed, 226 insertions(+)
>  create mode 100644 
> meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch
>
> diff --git 
> a/meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch
>  
> b/meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch
> new file mode 100644
> index 000..c4e4a60
> --- /dev/null
> +++ 
> b/meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch
> @@ -0,0 +1,225 @@
> +From 40dadb863fd105c4a66b598c2423b1eb11b1f26e Mon Sep 17 00:00:00 2001
> +From: Glenn Strauss 
> +Date: Wed, 23 Dec 2020 23:14:47 -0500
> +Subject: [PATCH] [core] reuse large mem chunks (fix mem usage) (fixes #3033)
> +
> +(cherry picked from commit 7ba521ffb4959f6f74a609d5d4acafc29a038337)
> +
> +(thx flynn)
> +
> +fix large memory usage for large file downloads from dynamic backends
> +
> +reuse or release large memory chunks
> +
> +x-ref:
> +  "Memory Growth with PUT and full buffered streams"
> +  https://redmine.lighttpd.net/issues/3033
> +
> +Upstream-Status: Backport
> +Comment: No hunk refreshed
> +https://redmine.lighttpd.net/projects/lighttpd/repository/14/revisions/7ba521ffb4959f6f74a609d5d4acafc29a038337
> +Signed-off-by: Purushottam Choudhary 
> +---
> + src/chunk.c| 99 +-
> + src/chunk.h|  2 +
> + src/http-header-glue.c |  2 +-
> + 3 files changed, 82 insertions(+), 21 deletions(-)
> +
> +diff --git a/src/chunk.c b/src/chunk.c
> +index 09dd3f14..ffb37951 100644
> +--- a/src/chunk.c
>  b/src/chunk.c
> +@@ -28,16 +28,20 @@
> + static size_t chunk_buf_sz = 4096;
> + static chunk *chunks, *chunks_oversized;
> + static chunk *chunk_buffers;
> ++static int chunks_oversized_n;
> + static array *chunkqueue_default_tempdirs = NULL;
> + static off_t chunkqueue_default_tempfile_size = DEFAULT_TEMPFILE_SIZE;
> +
> + void chunkqueue_set_chunk_size (size_t sz)
> + {
> +-chunk_buf_sz = sz > 0 ? ((sz + 1023) & ~1023uL) : 4096;
> ++size_t x = 1024;
> ++while (x < sz && x < (1u << 30)) x <<= 1;
> ++chunk_buf_sz = sz > 0 ? x : 4096;
> + }
> +
> + void chunkqueue_set_tempdirs_default_reset (void)
> + {
> ++chunk_buf_sz = 8192;
> + chunkqueue_default_tempdirs = NULL;
> + chunkqueue_default_tempfile_size = DEFAULT_TEMPFILE_SIZE;
> + }
> +@@ -120,15 +124,49 @@ static void chunk_free(chunk *c) {
> +   free(c);
> + }
> +
> +-buffer * chunk_buffer_acquire(void) {
> ++static chunk * chunk_pop_oversized(size_t sz) {
> ++/* future: might have buckets of certain sizes, up to socket buf sizes 
> */
> ++if (chunks_oversized && chunks_oversized->mem->size >= sz) {
> ++--chunks_oversized_n;
> ++chunk *c = chunks_oversized;
> ++chunks_oversized = c->next;
> ++return c;
> ++}
> ++return NULL;
> ++}
> ++
> ++static void chunk_push_oversized(chunk * const c, const size_t sz) {
> ++if (chunks_oversized_n < 64 && chunk_buf_sz >= 4096) {
> ++++chunks_oversized_n;
> ++chunk **co = _oversized;
> ++while (*co && sz < (*co)->mem->size) co = &(*co)->next;
> ++c->next = *co;
> ++*co = c;
> ++}
> ++else
> ++chunk_free(c);
> ++}
> ++
> ++static buffer * chunk_buffer_acquire_sz(size_t sz) {
> + chunk *c;
> + buffer *b;
> +-if (chunks) {
> +-c = chunks;
> +-chunks = c->next;
> ++if (sz <= chunk_buf_sz) {
> ++if (chunks) {
> ++c = chunks;
> ++chunks = c->next;
> ++}
> ++else
> ++c = 

Re: [OE-core] [meta][dunfell][PATCH v2] systemd: Add fix for CVE-2020-13529 and CVE-2021-33910

[Steve Sakoman]

On Wed, Aug 4, 2021 at 7:54 PM Ranjitsinh Rathod
 wrote:
>
> Added fix for below CVEs from below Link
> http://archive.ubuntu.com/ubuntu/pool/main/s/systemd/systemd_237-3ubuntu10.50.debian.tar.xz
>
> 1. CVE-2020-13529
> Upstream-Status: Backport 
> [https://github.com/systemd/systemd/commit/38e980a6a5a3442c2f48b1f827284388096d8ca5]
>
> 2. CVE-2021-33910
> Upstream-Status: Backport 
> [https://github.com/systemd/systemd/pull/20256/commits/441e0115646d54f080e5c3bb0ba477c892861ab9]

This patch is throwing warnings on the autobuilder:

WARNING: systemd-1_244.5-r0 do_patch: Fuzz detected:

Applying patch CVE-2020-13529.patch
patching file src/libsystemd-network/sd-dhcp-client.c
Hunk #1 succeeded at 1392 with fuzz 1 (offset 87 lines).

The context lines in the patches can be updated with devtool:

devtool modify systemd
devtool finish --force-patch-refresh systemd 

Could you fix this and send a V3 so that we have clean builds on the
autobuilder?

Steve

> Signed-off-by: Ranjitsinh Rathod 
> ---
>  .../systemd/systemd/CVE-2020-13529.patch  | 42 
>  .../systemd/systemd/CVE-2021-33910.patch  | 67 +++
>  meta/recipes-core/systemd/systemd_244.5.bb|  2 +
>  3 files changed, 111 insertions(+)
>  create mode 100644 meta/recipes-core/systemd/systemd/CVE-2020-13529.patch
>  create mode 100644 meta/recipes-core/systemd/systemd/CVE-2021-33910.patch
>
> diff --git a/meta/recipes-core/systemd/systemd/CVE-2020-13529.patch 
> b/meta/recipes-core/systemd/systemd/CVE-2020-13529.patch
> new file mode 100644
> index 00..4c013e2532
> --- /dev/null
> +++ b/meta/recipes-core/systemd/systemd/CVE-2020-13529.patch
> @@ -0,0 +1,42 @@
> +From 38e980a6a5a3442c2f48b1f827284388096d8ca5 Mon Sep 17 00:00:00 2001
> +From: Yu Watanabe 
> +Date: Thu, 24 Jun 2021 01:22:07 +0900
> +Subject: [PATCH] sd-dhcp-client: tentatively ignore FORCERENEW command
> +
> +This makes DHCP client ignore FORCERENEW requests, as unauthenticated
> +FORCERENEW requests causes a security issue (TALOS-2020-1142, 
> CVE-2020-13529).
> +
> +Let's re-enable this after RFC3118 (Authentication for DHCP Messages)
> +and/or RFC6704 (Forcerenew Nonce Authentication) are implemented.
> +
> +Fixes #16774.
> +
> +Upstream-Status: Backport 
> [https://github.com/systemd/systemd/commit/38e980a6a5a3442c2f48b1f827284388096d8ca5]
> +CVE: CVE-2020-13529
> +
> +Signed-off-by: Ranjitsinh Rathod 
> +---
> + src/libsystemd-network/sd-dhcp-client.c | 8 
> + 1 file changed, 8 insertions(+)
> +
> +--- a/src/libsystemd-network/sd-dhcp-client.c
>  b/src/libsystemd-network/sd-dhcp-client.c
> +@@ -1305,9 +1305,17 @@ static int client_handle_forcerenew(sd_d
> + if (r != DHCP_FORCERENEW)
> + return -ENOMSG;
> +
> ++#if 0
> + log_dhcp_client(client, "FORCERENEW");
> +
> + return 0;
> ++#else
> ++/* FIXME: Ignore FORCERENEW requests until we implement RFC3118 
> (Authentication for DHCP
> ++ * Messages) and/or RFC6704 (Forcerenew Nonce Authentication), as 
> unauthenticated FORCERENEW
> ++ * requests causes a security issue (TALOS-2020-1142, 
> CVE-2020-13529). */
> ++log_dhcp_client(client, "Received FORCERENEW, ignoring.");
> ++return -ENOMSG;
> ++#endif
> + }
> +
> + static int client_handle_ack(sd_dhcp_client *client, DHCPMessage *ack, 
> size_t len) {
> +
> diff --git a/meta/recipes-core/systemd/systemd/CVE-2021-33910.patch 
> b/meta/recipes-core/systemd/systemd/CVE-2021-33910.patch
> new file mode 100644
> index 00..be042165a0
> --- /dev/null
> +++ b/meta/recipes-core/systemd/systemd/CVE-2021-33910.patch
> @@ -0,0 +1,67 @@
> +Backport of:
> +
> +From 441e0115646d54f080e5c3bb0ba477c892861ab9 Mon Sep 17 00:00:00 2001
> +From: =?UTF-8?q?Zbigniew=20J=C4=99drzejewski-Szmek?= 
> +Date: Wed, 23 Jun 2021 11:46:41 +0200
> +Subject: [PATCH 1/2] basic/unit-name: do not use strdupa() on a path
> +
> +The path may have unbounded length, for example through a fuse mount.
> +
> +CVE-2021-33910: attacked controlled alloca() leads to crash in systemd and
> +ultimately a kernel panic. Systemd parses the content of /proc/self/mountinfo
> +and each mountpoint is passed to mount_setup_unit(), which calls
> +unit_name_path_escape() underneath. A local attacker who is able to mount a
> +filesystem with a very long path can crash systemd and the whole system.
> +
> +https://bugzilla.redhat.com/show_bug.cgi?id=1970887
> +
> +The resulting string length is bounded by UNIT_NAME_MAX, which is 256. But we
> +can't easily check the length after simplification before doing the
> +simplification, which in turns uses a copy of the string we can write to.
> +So we can't reject paths that are too long before doing the duplication.
> +Hence the most obvious solution is to switch back to strdup(), as before
> +7410616cd9dbbec97cf98d75324da5cda2b2f7a2.
> +
> +Upstream-Status: Backport 
> 

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Tony Battersby]

On 8/6/21 12:44 PM, Tony Battersby wrote:
> On 8/6/21 1:33 AM, Richard Purdie wrote:
>> On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote:
>>> On 8/5/21 6:21 AM, Richard Purdie wrote:
 On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
> On 2021-07-27 9:34 a.m., Tony Battersby wrote:
>> Currently, enabling Link Time Optimization via:
>>
>> require conf/distro/include/lto.inc
>> DISTRO_FEATURES_append = " lto"
>>
>> makes the build non-reproducible (as in
>> https://reproducible-builds.org/).  This patch set fixes some of the
>> problems, but is not yet a complete solution.
>>
>> This patch set addresses the following bugs:
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
>>
>> openembedded-core patches
>> 1/4: lto.inc: disable LTO for grub
>> 2/4: gcc: Backport patch to make LTO builds more reproducible
>> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
>>
>> openembedded-devel patches
>> 4/4: net-snmp: fix QA Issue after LDFLAGS change
>>
>> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
>> TARGET_CFLAGS).  This could possibly be done differently, such as by
>> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
>> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
>> if there is a better/preferred way to do it.
>>
>> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
>> included a patch to fix that.  Other packages might have been broken as
>> well, although I haven't found any others.
>>
>> With all these patches, there are still many shared libraries in /lib
>> and /usr/lib that are still not reproducible with LTO enabled because
>> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
>> patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
>> the libbz2 package from the bzip2 recipe is a good example.  I do not
>> have a patch for that problem.  I will leave that to someone else.
>>
>> So this is what we need to achieve reproducible LTO builds:
>> *) This patch set
>> *) A fix for libtool
>> *) Add LTO to reproducible builds testing if appropriate (e.g.
>> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
>> *) Fixes for individual packages, if needed
>> *) Once everything works, update the documentation again (I previously
>> requested a documentation change for this bug, which will eventually
>> show up at
>> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
> This is certainly a few steps in the right direction with a good outline
> of what's next.
>
> I suspect that Richard has been busy with other things like overrides
> and the update to glibc-2.34.
>
> Do you have any suggestions or questions about what we should do
> with this patch series Richard?
 Thanks for the reminder, I've been meaning to queue/test this which I will
 do. The gcc version has been bumped but I ported the patch forward to it
 and the patches are in master-next for testing. It is great to see this
 support moving forward.

 Much as it pains me to admit having any libtool knowledge, I may be able to
 help with that bit. There is an untested patch below which may address that
 issue, or at least give a good hint on how to do it if it doesn't work. I
 don't have an environment to test easily right now.

 Cheers,

 Richard
>>> I tested your libtool patch in master-next with the bzip2 recipe.  It
>>> works as intended; libbz2.so.1.0.6 and all the other output files are
>>> now reproducible with LTO enabled.  I also verified that libtool is
>>> passing the -f*-prefix-map flags to the link command in log.do_compile
>>> as intended.
>>>
>>> I also verified that core-image-minimal builds successfully with the
>>> libtool patch.
>> Thanks for testing that, sounds like it helps that issue. I'll queue that 
>> patch.
>>
>> Unfortunately even with the ruby fix I queued, we still see reproduciblity
>> issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and ruby-dbg 
>> packages:
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio
>>
>> We'll need to resolve those before I can merge the flags change.
>>
>> Diffoscope output:
>>
>> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210805-8jfd55ki/packages/diff-html/
>>
>> Basically LDFLAGS get encoded into a couple of scripts and the ruby binary.
>>
>> Cheers,
>>
>> Richard
>>
> It looks like the ruby problem is being caused by MJIT_DLDFLAGS in
> build/tmp/work/core2-64-poky-linux/ruby/3.0.2-r0/build/mjit_config.h. 
> There is already a patch
> 

Re: [OE-core] [PATCH] systemd: Fix build on musl

[Andre McCurdy]

On Fri, Aug 6, 2021 at 7:12 AM Luca Boccassi  wrote:
>
> On Thu, 29 Jul 2021 at 20:11, Andre McCurdy  wrote:
> >
> > On Thu, Jul 29, 2021 at 6:49 AM Luca Bocassi  
> > wrote:
> > >
> > > Having a look at the patches, a few comments:
> > >
> > > - 0012-don-t-pass-AT_SYMLINK_NOFOLLOW-flag-to-faccessat.patch I find
> > > quite worrying, as it fundamentally changes access patterns, some of
> > > which are done for security reasons. At best, this will cause
> > > completely different runtime behaviours for the same filesystem
> > > depending on the libc implementation, which doesn't sound great?
> >
> > I wrote a long and verbose comment when I created the patch which
> > tries to document any differences in runtime behaviour.
> >
> >   
> >   Avoid using AT_SYMLINK_NOFOLLOW flag. It doesn't seem like the right 
> > thing to
> >   do and it's not portable (not supported by musl). See:
> >
> > 
> > http://lists.landley.net/pipermail/toybox-landley.net/2014-September/003610.html
> > http://www.openwall.com/lists/musl/2015/02/05/2
> >
> >   Note that laccess() is never passing AT_EACCESS so a lot of the 
> > discussion in
> >   the links above doesn't apply. Note also that (currently) all systemd 
> > callers
> >   of laccess() pass mode as F_OK, so only check for existence of a file, not
> >   access permissions. Therefore, in this case, the only distiction between
> >   faccessat() with (flag == 0) and (flag == AT_SYMLINK_NOFOLLOW) is the
> >   behaviour for broken symlinks; laccess() on a broken symlink will succeed
> >   with (flag == AT_SYMLINK_NOFOLLOW) and fail (flag == 0).
> >
> >   The laccess() macros was added to systemd some time ago and it's not 
> > clear if
> >   or why it needs to return success for broken symlinks. Maybe just 
> > historical
> >   and not actually necessary or desired behaviour?
> >   
> >
> > If that comment is now out of date or something is missing then please
> > send a patch to update it.
> >
> > However looking at this patch again now, it appears to have got broken
> > during a past rebase:
> >
> >   
> > https://git.openembedded.org/openembedded-core/commit/?id=e8dd5a36bf2f1e645fb2ff15eb3b5e97c04776e6
> >
> > The upstream code changed from:
> >
> >   #define laccess(path, mode) faccessat(AT_FDCWD, (path), (mode),
> > AT_SYMLINK_NOFOLLOW)
> >
> > to
> >
> >   #define laccess(path, mode) \
> >   (faccessat(AT_FDCWD, (path), (mode), AT_SYMLINK_NOFOLLOW) <
> > 0 ? -errno : 0)
> >
> > but the replacement version in the patch still returns the raw result
> > from faccessat(). That looks like an issue.
>
> If you think the flag is unnecessary (I don't, we use these for a
> reason, but that's not important right now), the correct action is to
> send a PR upstream to discuss removing it. Patching it out for one
> build case of many is just going to be a source of incompatibility and
> surprises for users, as the behaviour on the same filesystem changes
> depending on the build option. Having said that, I don't use musl so
> all of this is really not a problem for me, just providing some
> feedback as upstream maintainer, in case it can be useful.

I don't have any interest in systemd + musl anymore either. I did an
initial port as a proof of concept and sent patches to Khem off
list... and was somewhat surprised when they showed up some time later
in oe-core.

Note that there have been long discussions here previously about
whether OE should claim to support systemd + musl. The effort to
support it properly (including clarifying questions like this with
upstream as you suggest) doesn't seem huge but so far no one seems to
care enough about systemd + musl to do it. We rebase and tweak the
patches but guidance to potential users should still be "use at your
own risk".

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154573): 
https://lists.openembedded.org/g/openembedded-core/message/154573
Mute This Topic: https://lists.openembedded.org/mt/84490599/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3] python3-scons{-native}: upgrade 4.1.0 -> 4.2.0

[Tim Orling]

Update patch as setup.cfg changes are now in upstream.

License-Update: Update Copyright years

NOTE: The 4.2.0 Release of SCons will deprecate Python 3.5 Support.
Python 3.5 support will be dropped in the next major release.

RELEASE 4.2.0 - Sat, 31 Jul 2021 18:12:46 -0700

  From Byron Platt:
- Fix Install() issue when copytree recursion gives bad arguments
  that can lead to install side-effects including keeping dangling
  symlinks and silently failing to copy directories (and their
  subdirectories) when the directory already exists in the target.

  From Joseph Brill:
- Internal MSVS update: Remove unnecessary calls to find all
  installed versions of msvc when constructing the installed visual
  studios list.

  From William Deegan:
- Improve Subst()'s logic to check for proper callable function or
  class's argument list. It will now allow callables with expected
  args, and any extra args as long as they have default arguments.
  Additionally functions with no defaults for extra arguments as
  long as they are set using functools.partial to create a new
  callable which set them.
- Fix Issue #3035 - mingw with SHLIBVERSION set fails with either
  not a dll error or "Multiple ways to build the same target were
  specified for:".  Now mingw will disable creating the symlinks
  (and adding version string to ) dlls.  It sets
  SHLIBNOVERSIONSYMLINKS, IMPLIBNOVERSIONSYMLINKS and
  LDMODULENOVERSIONSYMLINKS to True.
- Added --experimental flag, to enable various experimental
  features/tools.  You can specify 'all', 'none', or any combination
  of available experimental features.
- Fix Issue #3933 - Remove unguarded print of debug information in
  SharedLibrary logic when SHLIBVERSION is specified.
- Fix versioned shared library naming for MacOS platform.
  (Previously was libxyz.dylib.1.2.3, has been fixed to
  libxyz.1.2.3.dylib. Additionally the sonamed symlink had the same
  issue, that is now resolved as well)
- Add experimental ninja builder. (Contributed by MongoDB, Daniel
  Moody and many others).
- Fix #3955 - _LIBDIRFLAGS leaving $( and $) in *COMSTR output.
  Added affect_signature flag to _concat function.  If set to False,
  it will prepend and append $( and $). That way the various
  Environment variables can use that rather than
  "$( _concat(...)$)".
- Fix issue with exparimental ninja tool which would fail on windows
  or when ninja package wasn't installed but --experimental=ninja was
  specified.
- As part of experimental ninja tool, allow SetOption() to set both
  disable_execute_ninja and disable_ninja.

  From David H:
- Fix Issue #3906 - `IMPLICIT_COMMAND_DEPENDENCIES` was not properly
  disabled when set to any string value (For example ['none','false',
  'no','off']) Also previously 'All' wouldn't have the desired affect.

  From Ivan Kravets:
- Provide a custom argument escape function for `TempFileMunge`
  using a new `TEMPFILEARGESCFUNC` variable. Useful if you need to
  apply extra operations on a command argument before writing to a
  temporary file (fix Windows slashes, normalize paths, etc.)

  From Henrik Maier:
   - DocbookXslt tool: The XSLT stylesheet file is now initialized to an
 env.File() Node, such that dependencies work correctly in hierarchical
 builds (eg when using DocbookXslt in SConscript('subdir/SConscript')
 context.

  From Daniel Moody:
- Update CacheDir to use uuid for tmpfile uniqueness instead of pid.
  This fixes cases for shared cache where two systems write to the
  same cache tmpfile at the same time because the happened to get the
  same pid.
- Added support for passing custom CacheDir derived classes to
  SCons. Moved copy_from_cache attribute from the Environment class to
  CacheDir class. Code contributed by MongoDB.
- Update BuildTask to pass all targets to the progress object fixing
  an issue where multi-target build nodes only got the first target
  passed to the progress object.
- Fix a potential race condition in shared cache environments where
  the permissions are not writeable for a moment after the file has
  been renamed and other builds (users) will copy it out of the cache.
  Small reorganization of logic to copy files from cachedir. Moved
  CacheDir writeable permission code for copy to cache behind the atomic
  rename operation.
- Added marking of intermediate and and multi target nodes generated
  from SConf tests so that is_conftest() is more accurate.
- Added test for configure check failing to ensure it didn't break
  generating and running ninja.

  From Mats Wichmann:
- Initial support in tests for Python 3.10 - expected bytecode and
  one changed expected exception message. Change some more regexes
  to be specified as rawstrings in 

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Tony Battersby]

On 8/6/21 1:33 AM, Richard Purdie wrote:
> On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote:
>> On 8/5/21 6:21 AM, Richard Purdie wrote:
>>> On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
 On 2021-07-27 9:34 a.m., Tony Battersby wrote:
> Currently, enabling Link Time Optimization via:
>
> require conf/distro/include/lto.inc
> DISTRO_FEATURES_append = " lto"
>
> makes the build non-reproducible (as in
> https://reproducible-builds.org/).  This patch set fixes some of the
> problems, but is not yet a complete solution.
>
> This patch set addresses the following bugs:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
>
> openembedded-core patches
> 1/4: lto.inc: disable LTO for grub
> 2/4: gcc: Backport patch to make LTO builds more reproducible
> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
>
> openembedded-devel patches
> 4/4: net-snmp: fix QA Issue after LDFLAGS change
>
> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
> TARGET_CFLAGS).  This could possibly be done differently, such as by
> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
> if there is a better/preferred way to do it.
>
> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
> included a patch to fix that.  Other packages might have been broken as
> well, although I haven't found any others.
>
> With all these patches, there are still many shared libraries in /lib
> and /usr/lib that are still not reproducible with LTO enabled because
> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
> patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
> the libbz2 package from the bzip2 recipe is a good example.  I do not
> have a patch for that problem.  I will leave that to someone else.
>
> So this is what we need to achieve reproducible LTO builds:
> *) This patch set
> *) A fix for libtool
> *) Add LTO to reproducible builds testing if appropriate (e.g.
> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
> *) Fixes for individual packages, if needed
> *) Once everything works, update the documentation again (I previously
> requested a documentation change for this bug, which will eventually
> show up at
> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
 This is certainly a few steps in the right direction with a good outline
 of what's next.

 I suspect that Richard has been busy with other things like overrides
 and the update to glibc-2.34.

 Do you have any suggestions or questions about what we should do
 with this patch series Richard?
>>> Thanks for the reminder, I've been meaning to queue/test this which I will
>>> do. The gcc version has been bumped but I ported the patch forward to it
>>> and the patches are in master-next for testing. It is great to see this
>>> support moving forward.
>>>
>>> Much as it pains me to admit having any libtool knowledge, I may be able to
>>> help with that bit. There is an untested patch below which may address that
>>> issue, or at least give a good hint on how to do it if it doesn't work. I
>>> don't have an environment to test easily right now.
>>>
>>> Cheers,
>>>
>>> Richard
>> I tested your libtool patch in master-next with the bzip2 recipe.  It
>> works as intended; libbz2.so.1.0.6 and all the other output files are
>> now reproducible with LTO enabled.  I also verified that libtool is
>> passing the -f*-prefix-map flags to the link command in log.do_compile
>> as intended.
>>
>> I also verified that core-image-minimal builds successfully with the
>> libtool patch.
> Thanks for testing that, sounds like it helps that issue. I'll queue that 
> patch.
>
> Unfortunately even with the ruby fix I queued, we still see reproduciblity
> issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and ruby-dbg 
> packages:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio
>
> We'll need to resolve those before I can merge the flags change.
>
> Diffoscope output:
>
> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210805-8jfd55ki/packages/diff-html/
>
> Basically LDFLAGS get encoded into a couple of scripts and the ruby binary.
>
> Cheers,
>
> Richard
>
It looks like the ruby problem is being caused by MJIT_DLDFLAGS in
build/tmp/work/core2-64-poky-linux/ruby/3.0.2-r0/build/mjit_config.h. 
There is already a patch
poky/meta/recipes-devtools/ruby/ruby/0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch
that removes some of these items for the sake of reproducibiltiy.  But I
do not know ruby, so I do not know the 

Re: [OE-core] [PATCH v2] python3-scons{-native}: upgrade 4.1.0 -> 4.2.0

[Tim Orling]

Disregard.
V3 in process

On Fri, Aug 6, 2021 at 9:22 AM Tim Orling via lists.openembedded.org
 wrote:

> License-Update: Update Copyright years
>
> NOTE: The 4.2.0 Release of SCons will deprecate Python 3.5 Support.
> Python 3.5 support will be dropped in the next major release.
>
> RELEASE 4.2.0 - Sat, 31 Jul 2021 18:12:46 -0700
>
>   From Byron Platt:
> - Fix Install() issue when copytree recursion gives bad arguments
>   that can lead to install side-effects including keeping dangling
>   symlinks and silently failing to copy directories (and their
>   subdirectories) when the directory already exists in the target.
>
>   From Joseph Brill:
> - Internal MSVS update: Remove unnecessary calls to find all
>   installed versions of msvc when constructing the installed visual
>   studios list.
>
>   From William Deegan:
> - Improve Subst()'s logic to check for proper callable function or
>   class's argument list. It will now allow callables with expected
>   args, and any extra args as long as they have default arguments.
>   Additionally functions with no defaults for extra arguments as
>   long as they are set using functools.partial to create a new
>   callable which set them.
> - Fix Issue #3035 - mingw with SHLIBVERSION set fails with either
>   not a dll error or "Multiple ways to build the same target were
>   specified for:".  Now mingw will disable creating the symlinks
>   (and adding version string to ) dlls.  It sets
>   SHLIBNOVERSIONSYMLINKS, IMPLIBNOVERSIONSYMLINKS and
>   LDMODULENOVERSIONSYMLINKS to True.
> - Added --experimental flag, to enable various experimental
>   features/tools.  You can specify 'all', 'none', or any combination
>   of available experimental features.
> - Fix Issue #3933 - Remove unguarded print of debug information in
>   SharedLibrary logic when SHLIBVERSION is specified.
> - Fix versioned shared library naming for MacOS platform.
>   (Previously was libxyz.dylib.1.2.3, has been fixed to
>   libxyz.1.2.3.dylib. Additionally the sonamed symlink had the same
>   issue, that is now resolved as well)
> - Add experimental ninja builder. (Contributed by MongoDB, Daniel
>   Moody and many others).
> - Fix #3955 - _LIBDIRFLAGS leaving $( and $) in *COMSTR output.
>   Added affect_signature flag to _concat function.  If set to False,
>   it will prepend and append $( and $). That way the various
>   Environment variables can use that rather than
>   "$( _concat(...)$)".
> - Fix issue with exparimental ninja tool which would fail on windows
>   or when ninja package wasn't installed but --experimental=ninja was
>   specified.
> - As part of experimental ninja tool, allow SetOption() to set both
>   disable_execute_ninja and disable_ninja.
>
>   From David H:
> - Fix Issue #3906 - `IMPLICIT_COMMAND_DEPENDENCIES` was not properly
>   disabled when set to any string value (For example ['none','false',
>   'no','off']) Also previously 'All' wouldn't have the desired affect.
>
>   From Ivan Kravets:
> - Provide a custom argument escape function for `TempFileMunge`
>   using a new `TEMPFILEARGESCFUNC` variable. Useful if you need to
>   apply extra operations on a command argument before writing to a
>   temporary file (fix Windows slashes, normalize paths, etc.)
>
>   From Henrik Maier:
>- DocbookXslt tool: The XSLT stylesheet file is now initialized to an
>  env.File() Node, such that dependencies work correctly in hierarchical
>  builds (eg when using DocbookXslt in SConscript('subdir/SConscript')
>  context.
>
>   From Daniel Moody:
> - Update CacheDir to use uuid for tmpfile uniqueness instead of pid.
>   This fixes cases for shared cache where two systems write to the
>   same cache tmpfile at the same time because the happened to get the
>   same pid.
> - Added support for passing custom CacheDir derived classes to
>   SCons. Moved copy_from_cache attribute from the Environment class to
>   CacheDir class. Code contributed by MongoDB.
> - Update BuildTask to pass all targets to the progress object fixing
>   an issue where multi-target build nodes only got the first target
>   passed to the progress object.
> - Fix a potential race condition in shared cache environments where
>   the permissions are not writeable for a moment after the file has
>   been renamed and other builds (users) will copy it out of the cache.
>   Small reorganization of logic to copy files from cachedir. Moved
>   CacheDir writeable permission code for copy to cache behind the
> atomic
>   rename operation.
> - Added marking of intermediate and and multi target nodes generated
>   from SConf tests so that is_conftest() is more accurate.
> - Added test for configure check failing to ensure it didn't break
>   generating 

[OE-core] [PATCH v2] python3-scons{-native}: upgrade 4.1.0 -> 4.2.0

[Tim Orling]

License-Update: Update Copyright years

NOTE: The 4.2.0 Release of SCons will deprecate Python 3.5 Support.
Python 3.5 support will be dropped in the next major release.

RELEASE 4.2.0 - Sat, 31 Jul 2021 18:12:46 -0700

  From Byron Platt:
- Fix Install() issue when copytree recursion gives bad arguments
  that can lead to install side-effects including keeping dangling
  symlinks and silently failing to copy directories (and their
  subdirectories) when the directory already exists in the target.

  From Joseph Brill:
- Internal MSVS update: Remove unnecessary calls to find all
  installed versions of msvc when constructing the installed visual
  studios list.

  From William Deegan:
- Improve Subst()'s logic to check for proper callable function or
  class's argument list. It will now allow callables with expected
  args, and any extra args as long as they have default arguments.
  Additionally functions with no defaults for extra arguments as
  long as they are set using functools.partial to create a new
  callable which set them.
- Fix Issue #3035 - mingw with SHLIBVERSION set fails with either
  not a dll error or "Multiple ways to build the same target were
  specified for:".  Now mingw will disable creating the symlinks
  (and adding version string to ) dlls.  It sets
  SHLIBNOVERSIONSYMLINKS, IMPLIBNOVERSIONSYMLINKS and
  LDMODULENOVERSIONSYMLINKS to True.
- Added --experimental flag, to enable various experimental
  features/tools.  You can specify 'all', 'none', or any combination
  of available experimental features.
- Fix Issue #3933 - Remove unguarded print of debug information in
  SharedLibrary logic when SHLIBVERSION is specified.
- Fix versioned shared library naming for MacOS platform.
  (Previously was libxyz.dylib.1.2.3, has been fixed to
  libxyz.1.2.3.dylib. Additionally the sonamed symlink had the same
  issue, that is now resolved as well)
- Add experimental ninja builder. (Contributed by MongoDB, Daniel
  Moody and many others).
- Fix #3955 - _LIBDIRFLAGS leaving $( and $) in *COMSTR output.
  Added affect_signature flag to _concat function.  If set to False,
  it will prepend and append $( and $). That way the various
  Environment variables can use that rather than
  "$( _concat(...)$)".
- Fix issue with exparimental ninja tool which would fail on windows
  or when ninja package wasn't installed but --experimental=ninja was
  specified.
- As part of experimental ninja tool, allow SetOption() to set both
  disable_execute_ninja and disable_ninja.

  From David H:
- Fix Issue #3906 - `IMPLICIT_COMMAND_DEPENDENCIES` was not properly
  disabled when set to any string value (For example ['none','false',
  'no','off']) Also previously 'All' wouldn't have the desired affect.

  From Ivan Kravets:
- Provide a custom argument escape function for `TempFileMunge`
  using a new `TEMPFILEARGESCFUNC` variable. Useful if you need to
  apply extra operations on a command argument before writing to a
  temporary file (fix Windows slashes, normalize paths, etc.)

  From Henrik Maier:
   - DocbookXslt tool: The XSLT stylesheet file is now initialized to an
 env.File() Node, such that dependencies work correctly in hierarchical
 builds (eg when using DocbookXslt in SConscript('subdir/SConscript')
 context.

  From Daniel Moody:
- Update CacheDir to use uuid for tmpfile uniqueness instead of pid.
  This fixes cases for shared cache where two systems write to the
  same cache tmpfile at the same time because the happened to get the
  same pid.
- Added support for passing custom CacheDir derived classes to
  SCons. Moved copy_from_cache attribute from the Environment class to
  CacheDir class. Code contributed by MongoDB.
- Update BuildTask to pass all targets to the progress object fixing
  an issue where multi-target build nodes only got the first target
  passed to the progress object.
- Fix a potential race condition in shared cache environments where
  the permissions are not writeable for a moment after the file has
  been renamed and other builds (users) will copy it out of the cache.
  Small reorganization of logic to copy files from cachedir. Moved
  CacheDir writeable permission code for copy to cache behind the atomic
  rename operation.
- Added marking of intermediate and and multi target nodes generated
  from SConf tests so that is_conftest() is more accurate.
- Added test for configure check failing to ensure it didn't break
  generating and running ninja.

  From Mats Wichmann:
- Initial support in tests for Python 3.10 - expected bytecode and
  one changed expected exception message. Change some more regexes
  to be specified as rawstrings in response to DeprecationWarnings.
- Add an example of 

[OE-core][PATCH] kernel-fitimage: images should not be signed with the same keys as the configurations

[Thomas Perrot]

Otherwise the "required" property, from UBOOT_DTB_BINARY, will be set to "conf"
and no error will be raised in case of error.

Signed-off-by: Thomas Perrot 
---
 meta/classes/kernel-fitimage.bbclass | 40 
 1 file changed, 35 insertions(+), 5 deletions(-)

diff --git a/meta/classes/kernel-fitimage.bbclass 
b/meta/classes/kernel-fitimage.bbclass
index a9d1002200c9..72f692e40e63 100644
--- a/meta/classes/kernel-fitimage.bbclass
+++ b/meta/classes/kernel-fitimage.bbclass
@@ -60,6 +60,14 @@ FIT_DESC ?= "Kernel fitImage for 
${DISTRO_NAME}/${PV}/${MACHINE}"
 # Sign individual images as well
 FIT_SIGN_INDIVIDUAL ?= "0"
 
+# Keys used to sign individually images nodes.
+# The keys to sign images nodes must be different from those used to sign
+# configurations nodes, otherwise the "required" property, from
+# UBOOT_DTB_BINARY, will be set to "conf", because "conf" prevails on "image".
+# Then images signature checking will not be mandatory and no error will be
+# raised.
+# UBOOT_SIGN_IMG_KEYNAME = "dev2" # keys name in keydir (eg. "dev2.crt", 
"dev2.key")
+
 #
 # Emit the fitImage ITS header
 #
@@ -121,7 +129,7 @@ fitimage_emit_section_kernel() {
 
kernel_csum="${FIT_HASH_ALG}"
kernel_sign_algo="${FIT_SIGN_ALG}"
-   kernel_sign_keyname="${UBOOT_SIGN_KEYNAME}"
+   kernel_sign_keyname="${UBOOT_SIGN_IMG_KEYNAME}"
 
ENTRYPOINT="${UBOOT_ENTRYPOINT}"
if [ -n "${UBOOT_ENTRYSYMBOL}" ]; then
@@ -167,7 +175,7 @@ fitimage_emit_section_dtb() {
 
dtb_csum="${FIT_HASH_ALG}"
dtb_sign_algo="${FIT_SIGN_ALG}"
-   dtb_sign_keyname="${UBOOT_SIGN_KEYNAME}"
+   dtb_sign_keyname="${UBOOT_SIGN_IMG_KEYNAME}"
 
dtb_loadline=""
dtb_ext=${DTB##*.}
@@ -214,7 +222,7 @@ fitimage_emit_section_boot_script() {
 
 bootscr_csum="${FIT_HASH_ALG}"
bootscr_sign_algo="${FIT_SIGN_ALG}"
-   bootscr_sign_keyname="${UBOOT_SIGN_KEYNAME}"
+   bootscr_sign_keyname="${UBOOT_SIGN_IMG_KEYNAME}"
 
 cat << EOF >> ${1}
 bootscr-${2} {
@@ -278,7 +286,7 @@ fitimage_emit_section_ramdisk() {
 
ramdisk_csum="${FIT_HASH_ALG}"
ramdisk_sign_algo="${FIT_SIGN_ALG}"
-   ramdisk_sign_keyname="${UBOOT_SIGN_KEYNAME}"
+   ramdisk_sign_keyname="${UBOOT_SIGN_IMG_KEYNAME}"
ramdisk_loadline=""
ramdisk_entryline=""
 
@@ -475,6 +483,10 @@ fitimage_assemble() {
bootscr_id=""
rm -f ${1} arch/${ARCH}/boot/${2}
 
+   if [ "${UBOOT_SIGN_KEYNAME}" = "${UBOOT_SIGN_IMG_KEYNAME}" ]; then
+   bbfatal "Keys used to sign images and configuration nodes must 
be different."
+   fi
+
fitimage_emit_fit_header ${1}
 
#
@@ -674,7 +686,7 @@ do_kernel_generate_rsa_keys() {
 
if [ "${UBOOT_SIGN_ENABLE}" = "1" ] && [ "${FIT_GENERATE_KEYS}" = "1" 
]; then
 
-   # Generate keys only if they don't already exist
+   # Generate keys to sign configuration nodes, only if they don't 
already exist
if [ ! -f "${UBOOT_SIGN_KEYDIR}/${UBOOT_SIGN_KEYNAME}".key ] || 
\
[ ! -f "${UBOOT_SIGN_KEYDIR}/${UBOOT_SIGN_KEYNAME}".crt 
]; then
 
@@ -691,6 +703,24 @@ do_kernel_generate_rsa_keys() {
-key 
"${UBOOT_SIGN_KEYDIR}/${UBOOT_SIGN_KEYNAME}".key \
-out 
"${UBOOT_SIGN_KEYDIR}/${UBOOT_SIGN_KEYNAME}".crt
fi
+
+   # Generate keys to sign image nodes, only if they don't already 
exist
+   if [ ! -f "${UBOOT_SIGN_KEYDIR}/${UBOOT_SIGN_IMG_KEYNAME}".key 
] || \
+   [ ! -f 
"${UBOOT_SIGN_KEYDIR}/${UBOOT_SIGN_IMG_KEYNAME}".crt ]; then
+
+   # make directory if it does not already exist
+   mkdir -p "${UBOOT_SIGN_KEYDIR}"
+
+   echo "Generating RSA private key for signing fitImage"
+   openssl genrsa ${FIT_KEY_GENRSA_ARGS} -out \
+   
"${UBOOT_SIGN_KEYDIR}/${UBOOT_SIGN_IMG_KEYNAME}".key \
+   "${FIT_SIGN_NUMBITS}"
+
+   echo "Generating certificate for signing fitImage"
+   openssl req ${FIT_KEY_REQ_ARGS} "${FIT_KEY_SIGN_PKCS}" \
+   -key 
"${UBOOT_SIGN_KEYDIR}/${UBOOT_SIGN_IMG_KEYNAME}".key \
+   -out 
"${UBOOT_SIGN_KEYDIR}/${UBOOT_SIGN_IMG_KEYNAME}".crt
+   fi
fi
 }
 
-- 
2.31.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154568): 
https://lists.openembedded.org/g/openembedded-core/message/154568
Mute This Topic: https://lists.openembedded.org/mt/84711813/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][dunfell 07/33] curl: Fix CVE-2021-22924 and CVE-2021-22925

[Mike Crowe via lists.openembedded.org]

On Thursday 05 August 2021 at 05:33:44 -1000, Steve Sakoman wrote:
> From: Mike Crowe 
> 
> curl v7.78 contained fixes for five CVEs:
> 
> CVE-2021-22922[1] and CVE-2021-22923[2] are only present when support
> for metalink is enabled. EXTRA_OECONF contains "--without-libmetalink"
> so these fixes are unnecessary.
> 
> CVE-2021-22926[3] only affects builds for MacOS.
> 
> CVE-2021-22924[4] and CVE-2021-22925[5] are both applicable. Take the
> patches from Ubuntu 20.04 curl_7.68.0-1ubuntu2.6 package which is close
> enough that the patch for CVE-2021-22924 applies without conflicts..

Now that you've added back the "== 2", I believe the final sentence is now
true for both patches. That may not be worth worrying about.

> 
> [1] https://curl.se/docs/CVE-2021-22922.html
> [2] https://curl.se/docs/CVE-2021-22923.html
> [3] https://curl.se/docs/CVE-2021-22926.html
> [4] https://curl.se/docs/CVE-2021-22924.html
> [5] https://curl.se/docs/CVE-2021-22925.html
> 
> Signed-off-by: Mike Crowe 
> Signed-off-by: Steve Sakoman 

Mike.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154567): 
https://lists.openembedded.org/g/openembedded-core/message/154567
Mute This Topic: https://lists.openembedded.org/mt/84687225/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[v2][oe-core][PATCH 1/1] icu: increase command buffer size

[Joe Slater]

Make cmd in pkg_installLibrary() LARGE_BUFFER_SIZE to avoid
pathname truncation if the install path is longer than about
150 characters.  For default compilation, the U_ASSERT does
not detect buffer overflow, but that issue is not addressed.

Signed-off-by: Joe Slater 
---
 ...pkgdata-increase-command-buffer-size.patch | 43 +++
 meta/recipes-support/icu/icu_69.1.bb  |  1 +
 2 files changed, 44 insertions(+)
 create mode 100644 
meta/recipes-support/icu/icu/0001-pkgdata-increase-command-buffer-size.patch

diff --git 
a/meta/recipes-support/icu/icu/0001-pkgdata-increase-command-buffer-size.patch 
b/meta/recipes-support/icu/icu/0001-pkgdata-increase-command-buffer-size.patch
new file mode 100644
index 00..ea68e4be9f
--- /dev/null
+++ 
b/meta/recipes-support/icu/icu/0001-pkgdata-increase-command-buffer-size.patch
@@ -0,0 +1,43 @@
+From ab6b1acdeed76899f8227c38ab7e7675c7673ff1 Mon Sep 17 00:00:00 2001
+From: Joe Slater 
+Date: Thu, 5 Aug 2021 09:23:48 -0700
+Subject: [PATCH 1/1] pkgdata: increase command buffer size
+
+Make cmd LARGE_BUFFER_SIZE to avoid pathname truncation
+when install paths are longer than about 150 characters.
+
+Upstream-Status: Pending
+
+Signed-off-by: Joe Slater 
+---
+ source/tools/pkgdata/pkgdata.cpp | 6 +++---
+ 1 file changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/source/tools/pkgdata/pkgdata.cpp 
b/source/tools/pkgdata/pkgdata.cpp
+index 5ab682e..d4b70ef 100644
+--- a/tools/pkgdata/pkgdata.cpp
 b/tools/pkgdata/pkgdata.cpp
+@@ -1131,17 +1131,17 @@ normal_symlink_mode:
+ 
+ static int32_t pkg_installLibrary(const char *installDir, const char 
*targetDir, UBool noVersion) {
+ int32_t result = 0;
+-char cmd[SMALL_BUFFER_MAX_SIZE];
++char cmd[LARGE_BUFFER_MAX_SIZE];
+ 
+ auto ret = snprintf(cmd,
+-SMALL_BUFFER_MAX_SIZE,
++LARGE_BUFFER_MAX_SIZE,
+ "cd %s && %s %s %s%s%s",
+ targetDir,
+ pkgDataFlags[INSTALL_CMD],
+ libFileNames[LIB_FILE_VERSION],
+ installDir, PKGDATA_FILE_SEP_STRING, 
libFileNames[LIB_FILE_VERSION]);
+ (void)ret;
+-U_ASSERT(0 <= ret && ret < SMALL_BUFFER_MAX_SIZE);
++U_ASSERT(0 <= ret && ret < LARGE_BUFFER_MAX_SIZE);
+ 
+ result = runCommand(cmd);
+ 
+-- 
+2.29.2
+
diff --git a/meta/recipes-support/icu/icu_69.1.bb 
b/meta/recipes-support/icu/icu_69.1.bb
index bfeea8d2f0..00c22372b5 100644
--- a/meta/recipes-support/icu/icu_69.1.bb
+++ b/meta/recipes-support/icu/icu_69.1.bb
@@ -107,6 +107,7 @@ SRC_URI = "${BASE_SRC_URI};name=code \
file://filter.json \
file://fix-install-manx.patch \
file://0001-icu-Added-armeb-support.patch \
+   file://0001-pkgdata-increase-command-buffer-size.patch \
"
 
 SRC_URI_append_class-target = "\
-- 
2.29.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154566): 
https://lists.openembedded.org/g/openembedded-core/message/154566
Mute This Topic: https://lists.openembedded.org/mt/84711017/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] systemd: Fix build on musl

[Luca Bocassi]

On Thu, 29 Jul 2021 at 20:11, Andre McCurdy  wrote:
>
> On Thu, Jul 29, 2021 at 6:49 AM Luca Bocassi  wrote:
> >
> > Having a look at the patches, a few comments:
> >
> > - 0012-don-t-pass-AT_SYMLINK_NOFOLLOW-flag-to-faccessat.patch I find
> > quite worrying, as it fundamentally changes access patterns, some of
> > which are done for security reasons. At best, this will cause
> > completely different runtime behaviours for the same filesystem
> > depending on the libc implementation, which doesn't sound great?
>
> I wrote a long and verbose comment when I created the patch which
> tries to document any differences in runtime behaviour.
>
>   
>   Avoid using AT_SYMLINK_NOFOLLOW flag. It doesn't seem like the right thing 
> to
>   do and it's not portable (not supported by musl). See:
>
> 
> http://lists.landley.net/pipermail/toybox-landley.net/2014-September/003610.html
> http://www.openwall.com/lists/musl/2015/02/05/2
>
>   Note that laccess() is never passing AT_EACCESS so a lot of the discussion 
> in
>   the links above doesn't apply. Note also that (currently) all systemd 
> callers
>   of laccess() pass mode as F_OK, so only check for existence of a file, not
>   access permissions. Therefore, in this case, the only distiction between
>   faccessat() with (flag == 0) and (flag == AT_SYMLINK_NOFOLLOW) is the
>   behaviour for broken symlinks; laccess() on a broken symlink will succeed
>   with (flag == AT_SYMLINK_NOFOLLOW) and fail (flag == 0).
>
>   The laccess() macros was added to systemd some time ago and it's not clear 
> if
>   or why it needs to return success for broken symlinks. Maybe just historical
>   and not actually necessary or desired behaviour?
>   
>
> If that comment is now out of date or something is missing then please
> send a patch to update it.
>
> However looking at this patch again now, it appears to have got broken
> during a past rebase:
>
>   
> https://git.openembedded.org/openembedded-core/commit/?id=e8dd5a36bf2f1e645fb2ff15eb3b5e97c04776e6
>
> The upstream code changed from:
>
>   #define laccess(path, mode) faccessat(AT_FDCWD, (path), (mode),
> AT_SYMLINK_NOFOLLOW)
>
> to
>
>   #define laccess(path, mode) \
>   (faccessat(AT_FDCWD, (path), (mode), AT_SYMLINK_NOFOLLOW) <
> 0 ? -errno : 0)
>
> but the replacement version in the patch still returns the raw result
> from faccessat(). That looks like an issue.

If you think the flag is unnecessary (I don't, we use these for a
reason, but that's not important right now), the correct action is to
send a PR upstream to discuss removing it. Patching it out for one
build case of many is just going to be a source of incompatibility and
surprises for users, as the behaviour on the same filesystem changes
depending on the build option. Having said that, I don't use musl so
all of this is really not a problem for me, just providing some
feedback as upstream maintainer, in case it can be useful.

Kind regards,
Luca Boccassi

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154565): 
https://lists.openembedded.org/g/openembedded-core/message/154565
Mute This Topic: https://lists.openembedded.org/mt/84490599/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] override syntax for VIRTUAL-RUNTIME and PREFERRED_PROVIDER

[Richard Purdie]

On Fri, 2021-08-06 at 06:24 -0700, Devendra Tewari wrote:
> Does the override syntax change affect VIRTUAL-RUNTIME and 
> PREFERRED_PROVIDER? If yes, 
> will convert-overrides.py support changes to these?

It does not so it does not need changing.

Cheers,

Richard




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154564): 
https://lists.openembedded.org/g/openembedded-core/message/154564
Mute This Topic: https://lists.openembedded.org/mt/84707911/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v4 2/7] overlayfs.bbclass: generate overlayfs mount units

[Michael Opdenacker]

On 8/6/21 3:13 PM, Vyacheslav Yurkov wrote:
> I've already updated the docs and will submit the patch as soon as
> this series is approved and merged.


Fantastic. Many thanks :)

Cheers
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154563): 
https://lists.openembedded.org/g/openembedded-core/message/154563
Mute This Topic: https://lists.openembedded.org/mt/84706478/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][RFC][PATCH][dunfell] wic: Add --use-stock-fstab option

[Daniel Gomez]

On Fri, 6 Aug 2021 at 12:29, Paul Barker  wrote:
>
> On Fri,  6 Aug 2021 11:23:37 +0200
> Daniel Gomez  wrote:
>
> > When embeddding rootfs image (e.g. 'rootfs-dir') as a partition we
> > might want to keep the stock fstab for that image. In such case, use
> > this option to not update the fstab and use the stock one instead.
> >
> > This option allows to specify which partitions get the updated fstab
> > and which get the stock fstab.
> >
> > Example:
> > part /export --source rootfs --rootfs-dir=hockeycam-image
> > --fstype=ext4 --label export --align 1024 --use-stock-fstab
> >
> > part / --source rootfs --fstype=ext4 --label rootfs --align 1024
> >
> > Signed-off-by: Daniel Gomez 
> > ---
> >
> > The commit-id ce682a73b7447652f898ce1d1d0416a456df5416 reverted the
> > functionality to restore the stock/original fstab file in a rootfs
> > partition image (wic plugin).
> >
> > Add back such functionality with an option to the user to select which
> > fstab will be used for each partition by adding the --use-stock-fstab.
> > In this case, the user will select whether or not the stock fstab will
> > be updated or kept.
> >
> > If you agree with the patch I'll adapt it to the master branch.
> >
> > Daniel
> >
> >  scripts/lib/wic/ksparser.py  | 1 +
> >  scripts/lib/wic/partition.py | 3 ++-
> >  2 files changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> > index 452a160232..64933e0a5b 100644
> > --- a/scripts/lib/wic/ksparser.py
> > +++ b/scripts/lib/wic/ksparser.py
> > @@ -184,6 +184,7 @@ class KickStart():
> >  part.add_argument('--use-uuid', action='store_true')
> >  part.add_argument('--uuid')
> >  part.add_argument('--fsuuid')
> > +part.add_argument('--use-stock-fstab', action='store_true')
> >
> >  bootloader = subparsers.add_parser('bootloader')
> >  bootloader.add_argument('--append')
> > diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> > index 85f9847047..c9e45773de 100644
> > --- a/scripts/lib/wic/partition.py
> > +++ b/scripts/lib/wic/partition.py
> > @@ -54,6 +54,7 @@ class Partition():
> >  self.uuid = args.uuid
> >  self.fsuuid = args.fsuuid
> >  self.type = args.type
> > +self.use_stock_fstab = args.use_stock_fstab
> >  self.updated_fstab_path = None
> >  self.has_fstab = False
> >  self.update_fstab_in_rootfs = False
> > @@ -286,7 +287,7 @@ class Partition():
> >  (self.fstype, extraopts, rootfs, label_str, self.fsuuid, 
> > rootfs_dir)
> >  exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
> >
> > -if self.updated_fstab_path and self.has_fstab:
> > +if self.updated_fstab_path and self.has_fstab and not 
> > self.use_stock_fstab:
> >  debugfs_script_path = os.path.join(cr_workdir, 
> > "debugfs_script")
> >  with open(debugfs_script_path, "w") as f:
> >  f.write("cd etc\n")
> > --
> > 2.30.2
> >
>
> You're only addressing ext2/3/4 partitions here. The patch needs to be
> extended to cover the msdos partition case and the
> update_fstab_in_rootfs case in the rootfs source plugin.
>
> I also recommend changing the option in ksparser to '--no-fstab-update'
> to match the argument you can pass to wic itself. That way you can pass
> '--no-fstab-update' to wic to set the option for all partitions, or
> include it in the wks file to set the option per-partition.

Thanks Richard and Paul for the suggestions and the missing tasks.
I'll send a patch as soon as I have all prepared.

Daniel
>
> Thanks,
>
> --
> Paul Barker
> https://pbarker.dev/

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154562): 
https://lists.openembedded.org/g/openembedded-core/message/154562
Mute This Topic: https://lists.openembedded.org/mt/84704596/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] override syntax for VIRTUAL-RUNTIME and PREFERRED_PROVIDER

[Devendra Tewari]

Does the override syntax change affect VIRTUAL-RUNTIME and PREFERRED_PROVIDER? 
If yes, will convert-overrides.py support changes to these?

Thanks,
Devendra

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154561): 
https://lists.openembedded.org/g/openembedded-core/message/154561
Mute This Topic: https://lists.openembedded.org/mt/84707911/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v4 2/7] overlayfs.bbclass: generate overlayfs mount units

[Vyacheslav Yurkov]

I've already updated the docs and will submit the patch as soon as this
series is approved and merged.

Vyacheslav

On Fri, Aug 6, 2021, 16:06 Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

> On Fri, 2021-08-06 at 15:00 +0200, Michael Opdenacker wrote:
> > Slava, Richard,
> >
> > On 8/6/21 2:06 PM, Vyacheslav Yurkov wrote:
> > > It's often desired in Embedded System design to have a read-only
> rootfs.
> > > But a lot of different applications might want to have a read-write
> access
> > > to some parts of a filesystem. It can be especially useful when your
> update
> > > mechanism overwrites the whole rootfs, but you want your application
> data
> > > to be preserved between updates. This class provides a way to achieve
> that
> > > by means of overlayfs and at the same time keeping the base rootfs
> read-only.
> > >
> > > Signed-off-by: Vyacheslav Yurkov 
> > > ---
> > >  meta/classes/overlayfs.bbclass | 111 +
> > >  1 file changed, 111 insertions(+)
> > >  create mode 100644 meta/classes/overlayfs.bbclass
> > >
> > > diff --git a/meta/classes/overlayfs.bbclass
> b/meta/classes/overlayfs.bbclass
> > > new file mode 100644
> > > index 00..8d9b59c9bf
> > > --- /dev/null
> > > +++ b/meta/classes/overlayfs.bbclass
> > > @@ -0,0 +1,111 @@
> > > +# Class for generation of overlayfs mount units
> > > +#
> > > +# It's often desired in Embedded System design to have a read-only
> rootfs.
> > > +# But a lot of different applications might want to have a read-write
> access to
> > > +# some parts of a filesystem. It can be especially useful when your
> update mechanism
> > > +# overwrites the whole rootfs, but you want your application data to
> be preserved
> > > +# between updates. This class provides a way to achieve that by means
> > > +# of overlayfs and at the same time keeping the base rootfs read-only.
> > > +#
> > > +# Usage example.
> > > +#
> > > +# Set a mount point for a partition overlayfs is going to use as
> upper layer
> > > +# in your machine configuration. Underlying file system can be
> anything that
> > > +# is supported by overlayfs. This has to be done in your machine
> configuration.
> > > +# QA check fails to catch file existence if you redefine this
> variable in your recipe!
> > > +#
> > > +#   OVERLAYFS_MOUNT_POINT[data] ?= "/data"
> > > +#
> > > +# The class assumes you have a data.mount systemd unit defined in your
> > > +# systemd-machine-units recipe and installed to the image.
> > > +#
> > > +# Then you can specify writable directories on a recipe base
> > > +#
> > > +#   OVERLAYFS_WRITABLE_PATHS[data] =
> "/usr/share/my-custom-application"
> > > +#
> > > +# To support several mount points you can use a different variable
> flag. Assume we
> > > +# want to have a writable location on the file system, but not
> interested where the data
> > > +# survive a reboot. Then we could have a mnt-overlay.mount unit for a
> tmpfs file system:
> > > +#
> > > +#   OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
> > > +#   OVERLAYFS_WRITABLE_PATHS[mnt-overlay] =
> "/usr/share/another-application"
> > > +#
> > > +# Note: the class does not support /etc directory itself, because
> systemd depends on it
> >
> >
> > Many thanks for this new class and the corresponding documentation.
> > Richard, I guess that's something we'll need to document in the manual,
> > once it's merge, right?
>
> Ideally, yes, correct.
>
> Cheers,
>
> Richard
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154560): 
https://lists.openembedded.org/g/openembedded-core/message/154560
Mute This Topic: https://lists.openembedded.org/mt/84706478/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v4 2/7] overlayfs.bbclass: generate overlayfs mount units

[Richard Purdie]

On Fri, 2021-08-06 at 15:00 +0200, Michael Opdenacker wrote:
> Slava, Richard,
> 
> On 8/6/21 2:06 PM, Vyacheslav Yurkov wrote:
> > It's often desired in Embedded System design to have a read-only rootfs.
> > But a lot of different applications might want to have a read-write access
> > to some parts of a filesystem. It can be especially useful when your update
> > mechanism overwrites the whole rootfs, but you want your application data
> > to be preserved between updates. This class provides a way to achieve that
> > by means of overlayfs and at the same time keeping the base rootfs 
> > read-only.
> > 
> > Signed-off-by: Vyacheslav Yurkov 
> > ---
> >  meta/classes/overlayfs.bbclass | 111 +
> >  1 file changed, 111 insertions(+)
> >  create mode 100644 meta/classes/overlayfs.bbclass
> > 
> > diff --git a/meta/classes/overlayfs.bbclass b/meta/classes/overlayfs.bbclass
> > new file mode 100644
> > index 00..8d9b59c9bf
> > --- /dev/null
> > +++ b/meta/classes/overlayfs.bbclass
> > @@ -0,0 +1,111 @@
> > +# Class for generation of overlayfs mount units
> > +#
> > +# It's often desired in Embedded System design to have a read-only rootfs.
> > +# But a lot of different applications might want to have a read-write 
> > access to
> > +# some parts of a filesystem. It can be especially useful when your update 
> > mechanism
> > +# overwrites the whole rootfs, but you want your application data to be 
> > preserved
> > +# between updates. This class provides a way to achieve that by means
> > +# of overlayfs and at the same time keeping the base rootfs read-only.
> > +#
> > +# Usage example.
> > +#
> > +# Set a mount point for a partition overlayfs is going to use as upper 
> > layer
> > +# in your machine configuration. Underlying file system can be anything 
> > that
> > +# is supported by overlayfs. This has to be done in your machine 
> > configuration.
> > +# QA check fails to catch file existence if you redefine this variable in 
> > your recipe!
> > +#
> > +#   OVERLAYFS_MOUNT_POINT[data] ?= "/data"
> > +#
> > +# The class assumes you have a data.mount systemd unit defined in your
> > +# systemd-machine-units recipe and installed to the image.
> > +#
> > +# Then you can specify writable directories on a recipe base
> > +#
> > +#   OVERLAYFS_WRITABLE_PATHS[data] = "/usr/share/my-custom-application"
> > +#
> > +# To support several mount points you can use a different variable flag. 
> > Assume we
> > +# want to have a writable location on the file system, but not interested 
> > where the data
> > +# survive a reboot. Then we could have a mnt-overlay.mount unit for a 
> > tmpfs file system:
> > +#
> > +#   OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
> > +#   OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = 
> > "/usr/share/another-application"
> > +#
> > +# Note: the class does not support /etc directory itself, because systemd 
> > depends on it
> 
> 
> Many thanks for this new class and the corresponding documentation.
> Richard, I guess that's something we'll need to document in the manual,
> once it's merge, right?

Ideally, yes, correct.

Cheers,

Richard


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154559): 
https://lists.openembedded.org/g/openembedded-core/message/154559
Mute This Topic: https://lists.openembedded.org/mt/84706478/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v4 7/7] MAINTAINERS: add overlayfs maintainer

[Michael Opdenacker]

On 8/6/21 2:06 PM, Vyacheslav Yurkov wrote:
> Signed-off-by: Vyacheslav Yurkov 
> ---
>  MAINTAINERS.md | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/MAINTAINERS.md b/MAINTAINERS.md
> index 2ddcde6878..36a9bde90c 100644
> --- a/MAINTAINERS.md
> +++ b/MAINTAINERS.md
> @@ -40,6 +40,7 @@ Component/Subsystem Maintainers
>  * opkg: Alex Stewart
>  * devtool: Saul Wold
>  * eSDK: Saul Wold
> +* overlayfs: Vyacheslav Yurkov


Reviewed-by: Michael Opdenacker 

;-)

Michael.
-- 

Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154558): 
https://lists.openembedded.org/g/openembedded-core/message/154558
Mute This Topic: https://lists.openembedded.org/mt/84706484/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v4 2/7] overlayfs.bbclass: generate overlayfs mount units

[Michael Opdenacker]

Slava, Richard,

On 8/6/21 2:06 PM, Vyacheslav Yurkov wrote:
> It's often desired in Embedded System design to have a read-only rootfs.
> But a lot of different applications might want to have a read-write access
> to some parts of a filesystem. It can be especially useful when your update
> mechanism overwrites the whole rootfs, but you want your application data
> to be preserved between updates. This class provides a way to achieve that
> by means of overlayfs and at the same time keeping the base rootfs read-only.
>
> Signed-off-by: Vyacheslav Yurkov 
> ---
>  meta/classes/overlayfs.bbclass | 111 +
>  1 file changed, 111 insertions(+)
>  create mode 100644 meta/classes/overlayfs.bbclass
>
> diff --git a/meta/classes/overlayfs.bbclass b/meta/classes/overlayfs.bbclass
> new file mode 100644
> index 00..8d9b59c9bf
> --- /dev/null
> +++ b/meta/classes/overlayfs.bbclass
> @@ -0,0 +1,111 @@
> +# Class for generation of overlayfs mount units
> +#
> +# It's often desired in Embedded System design to have a read-only rootfs.
> +# But a lot of different applications might want to have a read-write access 
> to
> +# some parts of a filesystem. It can be especially useful when your update 
> mechanism
> +# overwrites the whole rootfs, but you want your application data to be 
> preserved
> +# between updates. This class provides a way to achieve that by means
> +# of overlayfs and at the same time keeping the base rootfs read-only.
> +#
> +# Usage example.
> +#
> +# Set a mount point for a partition overlayfs is going to use as upper layer
> +# in your machine configuration. Underlying file system can be anything that
> +# is supported by overlayfs. This has to be done in your machine 
> configuration.
> +# QA check fails to catch file existence if you redefine this variable in 
> your recipe!
> +#
> +#   OVERLAYFS_MOUNT_POINT[data] ?= "/data"
> +#
> +# The class assumes you have a data.mount systemd unit defined in your
> +# systemd-machine-units recipe and installed to the image.
> +#
> +# Then you can specify writable directories on a recipe base
> +#
> +#   OVERLAYFS_WRITABLE_PATHS[data] = "/usr/share/my-custom-application"
> +#
> +# To support several mount points you can use a different variable flag. 
> Assume we
> +# want to have a writable location on the file system, but not interested 
> where the data
> +# survive a reboot. Then we could have a mnt-overlay.mount unit for a tmpfs 
> file system:
> +#
> +#   OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
> +#   OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/another-application"
> +#
> +# Note: the class does not support /etc directory itself, because systemd 
> depends on it


Many thanks for this new class and the corresponding documentation.
Richard, I guess that's something we'll need to document in the manual,
once it's merge, right?

Thanks again,
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154557): 
https://lists.openembedded.org/g/openembedded-core/message/154557
Mute This Topic: https://lists.openembedded.org/mt/84706478/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 7/7] MAINTAINERS: add overlayfs maintainer

[Vyacheslav Yurkov]

Signed-off-by: Vyacheslav Yurkov 
---
 MAINTAINERS.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS.md b/MAINTAINERS.md
index 2ddcde6878..36a9bde90c 100644
--- a/MAINTAINERS.md
+++ b/MAINTAINERS.md
@@ -40,6 +40,7 @@ Component/Subsystem Maintainers
 * opkg: Alex Stewart
 * devtool: Saul Wold
 * eSDK: Saul Wold
+* overlayfs: Vyacheslav Yurkov
 
 Maintainers needed
 --
-- 
2.28.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154556): 
https://lists.openembedded.org/g/openembedded-core/message/154556
Mute This Topic: https://lists.openembedded.org/mt/84706484/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 6/7] oeqa/selftest: overlayfs unit tests

[Vyacheslav Yurkov]

Unit tests for overlayfs.bbclass

Signed-off-by: Vyacheslav Yurkov 
---
 meta/lib/oeqa/selftest/cases/overlayfs.py | 171 ++
 1 file changed, 171 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/overlayfs.py

diff --git a/meta/lib/oeqa/selftest/cases/overlayfs.py 
b/meta/lib/oeqa/selftest/cases/overlayfs.py
new file mode 100644
index 00..0184d52494
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/overlayfs.py
@@ -0,0 +1,171 @@
+#
+# SPDX-License-Identifier: MIT
+#
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu
+
+class OverlayFSTests(OESelftestTestCase):
+"""Overlayfs class usage tests"""
+
+def getline(self, res, line):
+for l in res.output.split('\n'):
+if line in l:
+return l
+
+def add_overlay_conf_to_machine(self):
+machine_inc = """
+OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
+"""
+self.set_machine_config(machine_inc)
+
+def test_distro_features_missing(self):
+"""
+Summary:   Check that required DISTRO_FEATURES are set
+Expected:  Fail when either systemd or overlayfs are not in 
DISTRO_FEATURES
+Author:Vyacheslav Yurkov 
+"""
+
+config = """
+IMAGE_INSTALL:append = " overlayfs-user"
+"""
+overlayfs_recipe_append = """
+inherit overlayfs
+"""
+self.write_config(config)
+self.add_overlay_conf_to_machine()
+self.write_recipeinc('overlayfs-user', overlayfs_recipe_append)
+
+res = bitbake('core-image-minimal', ignore_status=True)
+line = self.getline(res, "overlayfs-user was skipped: missing required 
distro features")
+self.assertTrue("overlayfs" in res.output, msg=res.output)
+self.assertTrue("systemd" in res.output, msg=res.output)
+self.assertTrue("ERROR: Required build target 'core-image-minimal' has 
no buildable providers." in res.output, msg=res.output)
+
+def test_not_all_units_installed(self):
+"""
+Summary:   Test QA check that we have required mount units in the image
+Expected:  Fail because mount unit for overlay partition is not 
installed
+Author:Vyacheslav Yurkov 
+"""
+
+config = """
+IMAGE_INSTALL:append = " overlayfs-user"
+DISTRO_FEATURES += "systemd overlayfs"
+"""
+
+self.write_config(config)
+self.add_overlay_conf_to_machine()
+
+res = bitbake('core-image-minimal', ignore_status=True)
+line = self.getline(res, "Unit name mnt-overlay.mount not found in 
systemd unit directories")
+self.assertTrue(line and line.startswith("WARNING:"), msg=res.output)
+line = self.getline(res, "Not all mount units are installed by the 
BSP")
+self.assertTrue(line and line.startswith("ERROR:"), msg=res.output)
+
+def test_mount_unit_not_set(self):
+"""
+Summary:   Test whether mount unit was set properly
+Expected:  Fail because mount unit was not set
+Author:Vyacheslav Yurkov 
+"""
+
+config = """
+IMAGE_INSTALL:append = " overlayfs-user"
+DISTRO_FEATURES += "systemd overlayfs"
+"""
+
+self.write_config(config)
+
+res = bitbake('core-image-minimal', ignore_status=True)
+line = self.getline(res, "A recipe uses overlayfs class but there is 
no OVERLAYFS_MOUNT_POINT set in your MACHINE configuration")
+self.assertTrue(line and line.startswith("Parsing recipes...ERROR:"), 
msg=res.output)
+
+def test_wrong_mount_unit_set(self):
+"""
+Summary:   Test whether mount unit was set properly
+Expected:  Fail because not the correct flag used for mount unit
+Author:Vyacheslav Yurkov 
+"""
+
+config = """
+IMAGE_INSTALL:append = " overlayfs-user"
+DISTRO_FEATURES += "systemd overlayfs"
+"""
+
+wrong_machine_config = """
+OVERLAYFS_MOUNT_POINT[usr-share-overlay] = "/usr/share/overlay"
+"""
+
+self.write_config(config)
+self.set_machine_config(wrong_machine_config)
+
+res = bitbake('core-image-minimal', ignore_status=True)
+line = self.getline(res, "Missing required mount point for 
OVERLAYFS_MOUNT_POINT[mnt-overlay] in your MACHINE configuration")
+self.assertTrue(line and line.startswith("Parsing recipes...ERROR:"), 
msg=res.output)
+
+def test_correct_image(self):
+"""
+Summary:   Check that we can create an image when all parameters are
+   set correctly
+Expected:  Image is created successfully
+Author:Vyacheslav Yurkov 
+"""
+
+config = """
+IMAGE_INSTALL:append = " overlayfs-user systemd-machine-units"
+DISTRO_FEATURES += "systemd overlayfs"
+
+# Use systemd as init manager
+VIRTUAL-RUNTIME_init_manager = "systemd"
+
+# enable overlayfs in the kernel
+KERNEL_EXTRA_FEATURES:append = " features/overlayfs/overlayfs.scc"
+"""
+
+ 

[OE-core] [PATCH v4 5/7] overlayfs: meta-selftest recipe

[Vyacheslav Yurkov]

The recipe demonstrates example usage of overlayfs bbclass

Signed-off-by: Vyacheslav Yurkov 
---
 .../overlayfs-user/overlayfs-user.bb| 17 +
 1 file changed, 17 insertions(+)
 create mode 100644 meta-selftest/recipes-test/overlayfs-user/overlayfs-user.bb

diff --git a/meta-selftest/recipes-test/overlayfs-user/overlayfs-user.bb 
b/meta-selftest/recipes-test/overlayfs-user/overlayfs-user.bb
new file mode 100644
index 00..60405067de
--- /dev/null
+++ b/meta-selftest/recipes-test/overlayfs-user/overlayfs-user.bb
@@ -0,0 +1,17 @@
+SUMMARY = "Overlayfs class unit test"
+DESCRIPTION = "Contains an overlayfs configuration"
+LICENSE = "MIT"
+
+INHIBIT_DEFAULT_DEPS = "1"
+EXCLUDE_FROM_WORLD = "1"
+
+inherit ${@bb.utils.contains("DISTRO_FEATURES", "overlayfs", "overlayfs", "", 
d)}
+include test_recipe.inc
+
+OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/my-application"
+
+do_install() {
+install -d ${D}/usr/share/my-application
+}
+
+FILES:${PN} += "/usr"
-- 
2.28.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154554): 
https://lists.openembedded.org/g/openembedded-core/message/154554
Mute This Topic: https://lists.openembedded.org/mt/84706482/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 4/7] systemd-machine-units: add bbappend for meta-selftest

[Vyacheslav Yurkov]

Signed-off-by: Vyacheslav Yurkov 
---
 .../systemd-machine-units/systemd-machine-units_%.bbappend  | 2 ++
 1 file changed, 2 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/systemd-machine-units/systemd-machine-units_%.bbappend

diff --git 
a/meta-selftest/recipes-test/systemd-machine-units/systemd-machine-units_%.bbappend
 
b/meta-selftest/recipes-test/systemd-machine-units/systemd-machine-units_%.bbappend
new file mode 100644
index 00..205720982c
--- /dev/null
+++ 
b/meta-selftest/recipes-test/systemd-machine-units/systemd-machine-units_%.bbappend
@@ -0,0 +1,2 @@
+# This bbappend is used to alter the recipe using the test_recipe.inc file 
created by tests.
+include test_recipe.inc
-- 
2.28.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154553): 
https://lists.openembedded.org/g/openembedded-core/message/154553
Mute This Topic: https://lists.openembedded.org/mt/84706481/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 3/7] rootfs-postcommands: add QA check for overlayfs

[Vyacheslav Yurkov]

The check is conditional and only enabled when overlayfs is set in
DISTRO_FEATURES

Signed-off-by: Vyacheslav Yurkov 
---
 meta/classes/rootfs-postcommands.bbclass | 25 
 1 file changed, 25 insertions(+)

diff --git a/meta/classes/rootfs-postcommands.bbclass 
b/meta/classes/rootfs-postcommands.bbclass
index fbfa63fcb3..c5746eba13 100644
--- a/meta/classes/rootfs-postcommands.bbclass
+++ b/meta/classes/rootfs-postcommands.bbclass
@@ -39,6 +39,8 @@ ROOTFS_POSTPROCESS_COMMAND += 
'${@bb.utils.contains("DISTRO_FEATURES", "systemd"
 
 ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;'
 
+ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", 
"overlayfs", "overlayfs_qa_check;", "", d)}'
+
 inherit image-artifact-names
 
 # Sort the user and group entries in /etc by ID in order to make the content
@@ -373,3 +375,26 @@ rootfs_reproducible () {
fi
fi
 }
+
+python overlayfs_qa_check() {
+from oe.overlayfs import mountUnitName
+
+# this is a dumb check for unit existence, not its validity
+overlayMountPoints = d.getVarFlags("OVERLAYFS_MOUNT_POINT")
+imagepath = d.getVar("IMAGE_ROOTFS")
+searchpaths = [oe.path.join(imagepath, d.getVar("sysconfdir"), "systemd", 
"system"),
+   oe.path.join(imagepath, d.getVar("systemd_system_unitdir"))]
+
+allUnitExist = True;
+for mountPoint in overlayMountPoints:
+path = d.getVarFlag('OVERLAYFS_MOUNT_POINT', mountPoint)
+unit = mountUnitName(path)
+
+if not any(os.path.isfile(oe.path.join(dirpath, unit))
+   for dirpath in searchpaths):
+bb.warn('Unit name %s not found in systemd unit directories' % 
unit)
+allUnitExist = False;
+
+if not allUnitExist:
+bb.fatal('Not all mount units are installed by the BSP')
+}
-- 
2.28.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154552): 
https://lists.openembedded.org/g/openembedded-core/message/154552
Mute This Topic: https://lists.openembedded.org/mt/84706480/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 1/7] lib/oe: add generic functions for overlayfs

[Vyacheslav Yurkov]

Signed-off-by: Vyacheslav Yurkov 
---
 meta/lib/oe/overlayfs.py | 43 
 1 file changed, 43 insertions(+)
 create mode 100644 meta/lib/oe/overlayfs.py

diff --git a/meta/lib/oe/overlayfs.py b/meta/lib/oe/overlayfs.py
new file mode 100644
index 00..21ef710509
--- /dev/null
+++ b/meta/lib/oe/overlayfs.py
@@ -0,0 +1,43 @@
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# This file contains common functions for overlayfs and its QA check
+
+# this function is based on 
https://github.com/systemd/systemd/blob/main/src/basic/unit-name.c
+def escapeSystemdUnitName(path):
+escapeMap = {
+'/': '-',
+'-': "\\x2d",
+'\\': "\\x5d"
+}
+return "".join([escapeMap.get(c, c) for c in path.strip('/')])
+
+def strForBash(s):
+return s.replace('\\', '')
+
+def mountUnitName(unit):
+return escapeSystemdUnitName(unit) + '.mount'
+
+def helperUnitName(unit):
+return escapeSystemdUnitName(unit) + '-create-upper-dir.service'
+
+def unitFileList(d):
+fileList = []
+overlayMountPoints = d.getVarFlags("OVERLAYFS_MOUNT_POINT")
+
+if not overlayMountPoints:
+bb.fatal("A recipe uses overlayfs class but there is no 
OVERLAYFS_MOUNT_POINT set in your MACHINE configuration")
+
+# check that we have required mount points set first
+requiredMountPoints = d.getVarFlags('OVERLAYFS_WRITABLE_PATHS')
+for mountPoint in requiredMountPoints:
+if mountPoint not in overlayMountPoints:
+bb.fatal("Missing required mount point for 
OVERLAYFS_MOUNT_POINT[%s] in your MACHINE configuration" % mountPoint)
+
+for mountPoint in overlayMountPoints:
+for path in d.getVarFlag('OVERLAYFS_WRITABLE_PATHS', 
mountPoint).split():
+fileList.append(mountUnitName(path))
+fileList.append(helperUnitName(path))
+
+return fileList
+
-- 
2.28.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154550): 
https://lists.openembedded.org/g/openembedded-core/message/154550
Mute This Topic: https://lists.openembedded.org/mt/84706477/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 2/7] overlayfs.bbclass: generate overlayfs mount units

[Vyacheslav Yurkov]

It's often desired in Embedded System design to have a read-only rootfs.
But a lot of different applications might want to have a read-write access
to some parts of a filesystem. It can be especially useful when your update
mechanism overwrites the whole rootfs, but you want your application data
to be preserved between updates. This class provides a way to achieve that
by means of overlayfs and at the same time keeping the base rootfs read-only.

Signed-off-by: Vyacheslav Yurkov 
---
 meta/classes/overlayfs.bbclass | 111 +
 1 file changed, 111 insertions(+)
 create mode 100644 meta/classes/overlayfs.bbclass

diff --git a/meta/classes/overlayfs.bbclass b/meta/classes/overlayfs.bbclass
new file mode 100644
index 00..8d9b59c9bf
--- /dev/null
+++ b/meta/classes/overlayfs.bbclass
@@ -0,0 +1,111 @@
+# Class for generation of overlayfs mount units
+#
+# It's often desired in Embedded System design to have a read-only rootfs.
+# But a lot of different applications might want to have a read-write access to
+# some parts of a filesystem. It can be especially useful when your update 
mechanism
+# overwrites the whole rootfs, but you want your application data to be 
preserved
+# between updates. This class provides a way to achieve that by means
+# of overlayfs and at the same time keeping the base rootfs read-only.
+#
+# Usage example.
+#
+# Set a mount point for a partition overlayfs is going to use as upper layer
+# in your machine configuration. Underlying file system can be anything that
+# is supported by overlayfs. This has to be done in your machine configuration.
+# QA check fails to catch file existence if you redefine this variable in your 
recipe!
+#
+#   OVERLAYFS_MOUNT_POINT[data] ?= "/data"
+#
+# The class assumes you have a data.mount systemd unit defined in your
+# systemd-machine-units recipe and installed to the image.
+#
+# Then you can specify writable directories on a recipe base
+#
+#   OVERLAYFS_WRITABLE_PATHS[data] = "/usr/share/my-custom-application"
+#
+# To support several mount points you can use a different variable flag. 
Assume we
+# want to have a writable location on the file system, but not interested 
where the data
+# survive a reboot. Then we could have a mnt-overlay.mount unit for a tmpfs 
file system:
+#
+#   OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
+#   OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/another-application"
+#
+# Note: the class does not support /etc directory itself, because systemd 
depends on it
+
+REQUIRED_DISTRO_FEATURES += "systemd overlayfs"
+
+inherit systemd features_check
+
+python do_create_overlayfs_units() {
+CreateDirsUnitTemplate = """[Unit]
+Description=Overlayfs directories setup
+Requires={DATA_MOUNT_UNIT}
+After={DATA_MOUNT_UNIT}
+DefaultDependencies=no
+
+[Service]
+Type=oneshot
+ExecStart=mkdir -p {DATA_MOUNT_POINT}/workdir{LOWERDIR} && mkdir -p 
{DATA_MOUNT_POINT}/upper{LOWERDIR}
+RemainAfterExit=true
+StandardOutput=journal
+
+[Install]
+WantedBy=multi-user.target
+"""
+MountUnitTemplate = """[Unit]
+Description=Overlayfs mount unit
+Requires={CREATE_DIRS_SERVICE}
+After={CREATE_DIRS_SERVICE}
+
+[Mount]
+What=overlay
+Where={LOWERDIR}
+Type=overlay
+Options=lowerdir={LOWERDIR},upperdir={DATA_MOUNT_POINT}/upper{LOWERDIR},workdir={DATA_MOUNT_POINT}/workdir{LOWERDIR}
+
+[Install]
+WantedBy=multi-user.target
+"""
+
+def prepareUnits(data, lower):
+from oe.overlayfs import mountUnitName, helperUnitName
+
+args = {
+'DATA_MOUNT_POINT': data,
+'DATA_MOUNT_UNIT': mountUnitName(data),
+'CREATE_DIRS_SERVICE': helperUnitName(lower),
+'LOWERDIR': lower,
+}
+
+with open(os.path.join(d.getVar('WORKDIR'), mountUnitName(lower)), 
'w') as f:
+f.write(MountUnitTemplate.format(**args))
+
+with open(os.path.join(d.getVar('WORKDIR'), helperUnitName(lower)), 
'w') as f:
+f.write(CreateDirsUnitTemplate.format(**args))
+
+overlayMountPoints = d.getVarFlags("OVERLAYFS_MOUNT_POINT")
+for mountPoint in overlayMountPoints:
+for lower in d.getVarFlag('OVERLAYFS_WRITABLE_PATHS', 
mountPoint).split():
+prepareUnits(d.getVarFlag('OVERLAYFS_MOUNT_POINT', mountPoint), 
lower)
+}
+
+# we need to generate file names early during parsing stage
+python () {
+from oe.overlayfs import strForBash, unitFileList
+
+unitList = unitFileList(d)
+for unit in unitList:
+d.appendVar('SYSTEMD_SERVICE:' + d.getVar('PN'), ' ' + unit);
+d.appendVar('FILES:' + d.getVar('PN'), ' ' + strForBash(unit))
+
+d.setVar('OVERLAYFS_UNIT_LIST', ' '.join([strForBash(s) for s in 
unitList]))
+}
+
+do_install:append() {
+install -d ${D}${systemd_system_unitdir}
+for unit in ${OVERLAYFS_UNIT_LIST}; do
+install -m 0444 ${WORKDIR}/${unit} ${D}${systemd_system_unitdir}
+done
+}
+
+addtask create_overlayfs_units before do_install
-- 
2.28.0

[OE-core] [PATCH 2/2] curl: Fix reproducibility issue with LDFLAGS

[Richard Purdie]

If we add DEBUG_PREFIX_MAP into LDFLAGS, curl-dev is no longer reproducible.
Fix this.

Signed-off-by: Richard Purdie 
---
 meta/recipes-support/curl/curl_7.78.0.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-support/curl/curl_7.78.0.bb 
b/meta/recipes-support/curl/curl_7.78.0.bb
index 2e2be661024..dece0babb2f 100644
--- a/meta/recipes-support/curl/curl_7.78.0.bb
+++ b/meta/recipes-support/curl/curl_7.78.0.bb
@@ -73,6 +73,7 @@ do_install:append:class-target() {
-e 's,--sysroot=${STAGING_DIR_TARGET},,g' \
-e 's,--with-libtool-sysroot=${STAGING_DIR_TARGET},,g' \
-e 's|${DEBUG_PREFIX_MAP}||g' \
+   -e 's|${@" ".join(d.getVar("DEBUG_PREFIX_MAP").split())}||g' \
${D}${bindir}/curl-config
 }
 
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154549): 
https://lists.openembedded.org/g/openembedded-core/message/154549
Mute This Topic: https://lists.openembedded.org/mt/84705768/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/2] gettext: Fix reproducibility issue with LDFLAGS

[Richard Purdie]

If we add DEBUG_PREFIX_MAP into LDFLAGS, gettext-ptest is no longer 
reproducible.
Fix this.

Signed-off-by: Richard Purdie 
---
 meta/recipes-core/gettext/gettext_0.21.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-core/gettext/gettext_0.21.bb 
b/meta/recipes-core/gettext/gettext_0.21.bb
index 4247b487ab2..5ada7099375 100644
--- a/meta/recipes-core/gettext/gettext_0.21.bb
+++ b/meta/recipes-core/gettext/gettext_0.21.bb
@@ -171,6 +171,7 @@ do_install_ptest() {
 find ${D}${PTEST_PATH}/ -name "*.o" -exec rm {} \;
 chmod 0755 ${D}${PTEST_PATH}/tests/lang-vala 
${D}${PTEST_PATH}/tests/plural-1 ${D}${PTEST_PATH}/tests/xgettext-tcl-4 \
${D}${PTEST_PATH}/tests/xgettext-vala-1  
${D}${PTEST_PATH}/tests/xgettext-po-2
+sed -i -e 's|${DEBUG_PREFIX_MAP}||g' ${D}${PTEST_PATH}/tests/init-env
 fi
 }
 
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154548): 
https://lists.openembedded.org/g/openembedded-core/message/154548
Mute This Topic: https://lists.openembedded.org/mt/84705767/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][PATCH 2/2] convert-overrides.py: skip patches

[Richard Purdie]

On Fri, 2021-08-06 at 02:03 -0700, Chen Qi wrote:
> If a line specifies a patch, skip it.
> 
> Signed-off-by: Chen Qi 
> ---
>  scripts/contrib/convert-overrides.py | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/scripts/contrib/convert-overrides.py 
> b/scripts/contrib/convert-overrides.py
> index 4d41a4c475..387e29d66d 100755
> --- a/scripts/contrib/convert-overrides.py
> +++ b/scripts/contrib/convert-overrides.py
> @@ -63,6 +63,7 @@ skips = skips + ["run_loaddata_poky", 
> "determine_if_poky_env", "do_populate_poky
>  skips = skips + ["get_appends_for_files", "test_doubleref_remove", 
> "test_bitbakelayers_add_remove", "elf32_x86_64", "colour_remove", 
> "revmap_remove"]
>  skips = skips + ["test_rpm_remove", "test_bitbakelayers_add_remove", 
> "recipe_append_file", "log_data_removed", "recipe_append", 
> "systemd_machine_unit_append"]
>  skips = skips + ["recipetool_append", "changetype_remove", 
> "try_appendfile_wc", "test_qemux86_directdisk", "test_layer_appends", 
> "tgz_removed"]
> +skips = skips + ["file://"]

This is never simple since this would now skip:

SRC_URI_append = " file://"

I'm not sure which issue is better/worse...

Cheers,

Richard



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154547): 
https://lists.openembedded.org/g/openembedded-core/message/154547
Mute This Topic: https://lists.openembedded.org/mt/84704327/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] cve-check: remove deprecated CVE_CHECK_CVE_WHITELIST

[Michael Opdenacker]

This variable has been deprecated since Yocto Project version 3.0.

Signed-off-by: Michael Opdenacker 
---
 meta/classes/cve-check.bbclass | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 6582f97151..04d19f53fd 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -238,9 +238,6 @@ def check_cves(d, patched_cves):
 bb.note("Recipe has been whitelisted, skipping check")
 return ([], [], [])
 
-old_cve_whitelist =  d.getVar("CVE_CHECK_CVE_WHITELIST")
-if old_cve_whitelist:
-bb.warn("CVE_CHECK_CVE_WHITELIST is deprecated, please use 
CVE_CHECK_WHITELIST.")
 cve_whitelist = d.getVar("CVE_CHECK_WHITELIST").split()
 
 import sqlite3
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154546): 
https://lists.openembedded.org/g/openembedded-core/message/154546
Mute This Topic: https://lists.openembedded.org/mt/84705697/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][RFC][PATCH][dunfell] wic: Add --use-stock-fstab option

[Paul Barker]

On Fri,  6 Aug 2021 11:23:37 +0200
Daniel Gomez  wrote:

> When embeddding rootfs image (e.g. 'rootfs-dir') as a partition we
> might want to keep the stock fstab for that image. In such case, use
> this option to not update the fstab and use the stock one instead.
> 
> This option allows to specify which partitions get the updated fstab
> and which get the stock fstab.
> 
> Example:
> part /export --source rootfs --rootfs-dir=hockeycam-image
> --fstype=ext4 --label export --align 1024 --use-stock-fstab
> 
> part / --source rootfs --fstype=ext4 --label rootfs --align 1024
> 
> Signed-off-by: Daniel Gomez 
> ---
> 
> The commit-id ce682a73b7447652f898ce1d1d0416a456df5416 reverted the
> functionality to restore the stock/original fstab file in a rootfs
> partition image (wic plugin).
> 
> Add back such functionality with an option to the user to select which
> fstab will be used for each partition by adding the --use-stock-fstab.
> In this case, the user will select whether or not the stock fstab will
> be updated or kept.
> 
> If you agree with the patch I'll adapt it to the master branch.
> 
> Daniel
> 
>  scripts/lib/wic/ksparser.py  | 1 +
>  scripts/lib/wic/partition.py | 3 ++-
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> index 452a160232..64933e0a5b 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -184,6 +184,7 @@ class KickStart():
>  part.add_argument('--use-uuid', action='store_true')
>  part.add_argument('--uuid')
>  part.add_argument('--fsuuid')
> +part.add_argument('--use-stock-fstab', action='store_true')
> 
>  bootloader = subparsers.add_parser('bootloader')
>  bootloader.add_argument('--append')
> diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> index 85f9847047..c9e45773de 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -54,6 +54,7 @@ class Partition():
>  self.uuid = args.uuid
>  self.fsuuid = args.fsuuid
>  self.type = args.type
> +self.use_stock_fstab = args.use_stock_fstab
>  self.updated_fstab_path = None
>  self.has_fstab = False
>  self.update_fstab_in_rootfs = False
> @@ -286,7 +287,7 @@ class Partition():
>  (self.fstype, extraopts, rootfs, label_str, self.fsuuid, 
> rootfs_dir)
>  exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
> 
> -if self.updated_fstab_path and self.has_fstab:
> +if self.updated_fstab_path and self.has_fstab and not 
> self.use_stock_fstab:
>  debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
>  with open(debugfs_script_path, "w") as f:
>  f.write("cd etc\n")
> --
> 2.30.2
> 

You're only addressing ext2/3/4 partitions here. The patch needs to be
extended to cover the msdos partition case and the
update_fstab_in_rootfs case in the rootfs source plugin.

I also recommend changing the option in ksparser to '--no-fstab-update'
to match the argument you can pass to wic itself. That way you can pass
'--no-fstab-update' to wic to set the option for all partitions, or
include it in the wks file to set the option per-partition.

Thanks,

-- 
Paul Barker
https://pbarker.dev/


pgpnp1YzDVrHE.pgp
Description: OpenPGP digital signature

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154545): 
https://lists.openembedded.org/g/openembedded-core/message/154545
Mute This Topic: https://lists.openembedded.org/mt/84704596/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][RFC][PATCH][dunfell] wic: Add --use-stock-fstab option

[Richard Purdie]

On Fri, 2021-08-06 at 11:23 +0200, Daniel Gomez wrote:
> When embeddding rootfs image (e.g. 'rootfs-dir') as a partition we
> might want to keep the stock fstab for that image. In such case, use
> this option to not update the fstab and use the stock one instead.
> 
> This option allows to specify which partitions get the updated fstab
> and which get the stock fstab.
> 
> Example:
> part /export --source rootfs --rootfs-dir=hockeycam-image
> --fstype=ext4 --label export --align 1024 --use-stock-fstab
> 
> part / --source rootfs --fstype=ext4 --label rootfs --align 1024
> 
> Signed-off-by: Daniel Gomez 
> ---
> 
> The commit-id ce682a73b7447652f898ce1d1d0416a456df5416 reverted the
> functionality to restore the stock/original fstab file in a rootfs
> partition image (wic plugin).
> 
> Add back such functionality with an option to the user to select which
> fstab will be used for each partition by adding the --use-stock-fstab.
> In this case, the user will select whether or not the stock fstab will
> be updated or kept.
> 
> If you agree with the patch I'll adapt it to the master branch.

It seems reasonable to me. It would need to go to master first and
ideally we'd add some kind of test to oe-selftest -r wic (see 
meta/lib/oeqa/selftest/cases/wic.py).

Cheers,

Richard


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154544): 
https://lists.openembedded.org/g/openembedded-core/message/154544
Mute This Topic: https://lists.openembedded.org/mt/84704596/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [qa-build-notification] QA notification for completed autobuilder build (yocto-3.1.10.rc1)

[Sangeeta Jain]

Hello All,

This is the full report for yocto-3.1.10.rc1:  
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/tree/?h=intel-yocto-testresults

=== Summary 
No high milestone defects.

One issue observed in this release:

Bug 14488 - AB-INT PTEST: tcl socket.test intermittent failure

=== Bugs 
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14488

Thanks,
Sangeeta

> -Original Message-
> From: qa-build-notificat...@lists.yoctoproject.org  notificat...@lists.yoctoproject.org> On Behalf Of Richard Purdie
> Sent: Wednesday, 28 July, 2021 6:48 AM
> To:  
> Cc: qa-build-notification 
> Subject: [qa-build-notification] QA notification for completed autobuilder 
> build
> (yocto-3.1.10.rc1)
> 
> A build flagged for QA (yocto-3.1.10.rc1) was completed on the autobuilder and
> is available at:
> 
> 
> https://autobuilder.yocto.io/pub/releases/yocto-3.1.10.rc1
> 
> 
> Build hash information:
> 
> bitbake: 0e0af15b84e07e6763300dcd092b980086b9b9c4
> meta-agl: 68c0029602e539d005c33b6c36baa32e34bf67bf
> meta-arm: b79b9122e0dc66ca03542f82ff4fa8eb05174f61
> meta-aws: d66de6ed82fe6b89e3b8b2626f4db6f5d7d5713d
> meta-gplv2: 60b251c25ba87e946a0ca4cdc8d17b1cb09292ac
> meta-intel: d8bf86ae6288ae520b8ddd7209a0b448b9693f48
> meta-mingw: 524de686205b5d6736661d4532f5f98fee8589b7
> meta-openembedded: 8fbcfb9f023e0eab7f122eb19607a1c757b0d245
> oecore: 9ae339ace9274be71bfd3b5e5da64dceac9fa963
> poky: 2a848e95074318f3a243df7b3f40513a13173a82
> 
> 
> 
> This is an automated message from the Yocto Project Autobuilder
> Git: git://git.yoctoproject.org/yocto-autobuilder2
> Email: richard.pur...@linuxfoundation.org
> 
> 
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154543): 
https://lists.openembedded.org/g/openembedded-core/message/154543
Mute This Topic: https://lists.openembedded.org/mt/84704966/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][RFC][PATCH][dunfell] wic: direct: Use stock fstab in partitions

[Daniel Gomez]

Hi Richard,

On Thu, 5 Aug 2021 at 17:31, Richard Purdie
 wrote:
>
> On Thu, 2021-08-05 at 17:03 +0200, Daniel Gomez wrote:
> > Be able to get the stock fstab in partitions.
> >
> > Functionality reverted at commit id:
> > ce682a73b7447652f898ce1d1d0416a456df5416
> >
> > Signed-off-by: Daniel Gomez 
> > ---
> >
> > Hi,
> >
> > I'm trying to get back the stock fstab file in a rootfs partition
> > image (wic plugin) and I'd like to know what you think about it.
> >
> > Functionality was included before commit-id
> > ce682a73b7447652f898ce1d1d0416a456df5416 and removed afterwards. Would
> > it be possible to add it again? What would be needed to re-add it?
> >
>
> The commit you mention above shouldn't have changed the output, it just used a
> different method to replace the fstab file. The question becomes why isn't the
> modified file being used in the way you're generating an image. I don't know
> wic well enough to know the answer to that though.

Thanks for your comments. I did some debugging and the fstab in use
for each partition will always be updated/replaced in the
'prepare_rootfs_ext' function (when 'update_fstab' in direct.py
updates it):
http://git.openembedded.org/openembedded-core/tree/scripts/lib/wic/partition.py?h=dunfell#n289

To fix it, I made this patch for dunfell so it gets updated if the user wants.
https://lists.openembedded.org/g/openembedded-core/message/154537

The user now decides with --use-stock-fstab if the stock fstab is kept
or replaced/updated.

As said in the other patch/message, I'll update it to master if you agree.

Thanks,
Daniel

>
> Cheers,
>
> Richard
>
>
> > Thanks in advance,
> > Daniel
> >
> > Here the relevant information:
> >
> > wks definition:
> > part /boot/bootloader --source qbootimg-pcbios --active --align 1024 
> > --use-uuid --fsoptions defaults,ro
> > part /export --source rootfs --rootfs-dir=hockeycam-image  --fstype=ext4 
> > --label export --align 1024 --use-uuid --fsoptions defaults,ro 
> > --extra-space 400M
> > part /media/rw --fstype=ext4 --label rwfs --align 1024 --use-uuid --size 200
> > part / --source rootfs --fstype=ext4 --label rootfs --align 1024 --use-uuid 
> > --fsoptions defaults,ro --extra-space 400M
> >
> > bootloader --timeout=1 --append="ro rootfstype=ext4 qtec_mem.size=64" 
> > --source qbootimg-pcbios
> >
> > /export/etc/fstab before:
> > # stock fstab - you probably want to override this with a machine specific 
> > one
> >
> > /dev/root/auto   ro  1  0
> > proc /procproc   defaults  
> > 0  0
> > devpts   /dev/pts devpts 
> > mode=0620,ptmxmode=0666,gid=5  0  0
> > tmpfs/run tmpfs  
> > mode=0755,nodev,nosuid,strictatime 0  0
> > tmpfs/var/volatiletmpfs  defaults  
> > 0  0
> >
> > # uncomment this if your device has a SD/MMC/Transflash slot
> > #/dev/mmcblk0p1   /media/card  auto   defaults,sync,noauto  
> > 0  0
> >
> > /dev/sdb /mnt auto defaults 0 0
> > UUID=D339-7B77  /boot/bootloadervfatdefaults,ro 0   0
> > UUID=31f46be1-d9fb-4081-bd53-2c2727e7854b   /export ext4defaults,ro 
> > 0   0
> > UUID=d8e8f5d4-897e-48cd-9188-33598363f706   /media/rw   ext4
> > defaults0   0
> >
> > /export/etc/fstab after:
> > # stock fstab - you probably want to override this with a machine specific 
> > one
> >
> > /dev/root/auto   ro  1  0
> > proc /procproc   defaults  
> > 0  0
> > devpts   /dev/pts devpts 
> > mode=0620,ptmxmode=0666,gid=5  0  0
> > tmpfs/run tmpfs  
> > mode=0755,nodev,nosuid,strictatime 0  0
> > tmpfs/var/volatiletmpfs  defaults  
> > 0  0
> >
> > # uncomment this if your device has a SD/MMC/Transflash slot
> > #/dev/mmcblk0p1   /media/card  auto   defaults,sync,noauto  
> > 0  0
> >
> > 10.100.10.100:/mnt/rw /media/rw   nfsdefaults   0 0
> >
> >
> >  scripts/lib/wic/plugins/imager/direct.py | 20 
> >  1 file changed, 16 insertions(+), 4 deletions(-)
> >
> > diff --git a/scripts/lib/wic/plugins/imager/direct.py 
> > b/scripts/lib/wic/plugins/imager/direct.py
> > index 7e1c1c03ab..1bb9d38524 100644
> > --- a/scripts/lib/wic/plugins/imager/direct.py
> > +++ b/scripts/lib/wic/plugins/imager/direct.py
> > @@ -111,7 +111,15 @@ class DirectPlugin(ImagerPlugin):
> >
> >  with open(fstab_path) as fstab:
> >  fstab_lines = fstab.readlines()
> > +self.original_fstab = fstab_lines.copy()
> >
> > +if self._update_fstab(fstab_lines, self.parts):
> > +with open(fstab_path, "w") as fstab:
> > +fstab.writelines(fstab_lines)
> > +else:
> > +  

Re: [OE-core][PATCH 1/2] convert-overrides.py: also convert comments without a leading whitespace

[Chen Qi]

Got it. I've sent out V2.

Regards,
Qi

On 08/06/2021 05:25 PM, Martin Jansa wrote:

Can you please bump the script version as well?

On Fri, Aug 6, 2021 at 11:03 AM Chen Qi > wrote:


Currently lines like below are converted.
  e.g.
  # IMAGE_INSTALL_append = " A"

But lines without a leading whitespace are not converted.
  e.g.
  #IMAGE_INSTALL_append = " A"

We should be converting both.

Signed-off-by: Chen Qi mailto:qi.c...@windriver.com>>
---
 scripts/contrib/convert-overrides.py


| 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/scripts/contrib/convert-overrides.py


b/scripts/contrib/convert-overrides.py


index e4a310d1d6..4d41a4c475 100755
--- a/scripts/contrib/convert-overrides.py


+++ b/scripts/contrib/convert-overrides.py


@@ -69,15 +69,15 @@ packagevars = packagevars + imagevars

 vars_re = {}
 for exp in vars:
-vars_re[exp] =
(re.compile('((^|[\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + exp),
r"\1:" + exp)
+vars_re[exp] =
(re.compile('((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + exp),
r"\1:" + exp)

 shortvars_re = {}
 for exp in shortvars:
-shortvars_re[exp] =
(re.compile('((^|[\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + exp +
'([\(\'"\s:])'), r"\1:" + exp + r"\3")
+shortvars_re[exp] =
(re.compile('((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + exp +
'([\(\'"\s:])'), r"\1:" + exp + r"\3")

 package_re = {}
 for exp in packagevars:
-package_re[exp] = (re.compile('(^|[\'"\s\-\+]+)' + exp + '_'
+ '([$a-z"\'\s%\[<{\\\*].)'), r"\1" + exp + r":\2")
+package_re[exp] = (re.compile('(^|[#\'"\s\-\+]+)' + exp + '_'
+ '([$a-z"\'\s%\[<{\\\*].)'), r"\1" + exp + r":\2")

 # Other substitutions to make
 subs = {
-- 
2.30.2








-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154541): 
https://lists.openembedded.org/g/openembedded-core/message/154541
Mute This Topic: https://lists.openembedded.org/mt/84704328/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core][PATCH V2] convert-overrides.py: also convert comments without a leading whitespace

[Chen Qi]

Currently lines like below are converted.
  e.g.
  # IMAGE_INSTALL_append = " A"

But lines without a leading whitespace are not converted.
  e.g.
  #IMAGE_INSTALL_append = " A"

We should be converting both.

Also bump the script's version.

Signed-off-by: Chen Qi 
---
 scripts/contrib/convert-overrides.py | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/contrib/convert-overrides.py 
b/scripts/contrib/convert-overrides.py
index 4647f29cab..7085f239db 100755
--- a/scripts/contrib/convert-overrides.py
+++ b/scripts/contrib/convert-overrides.py
@@ -70,15 +70,15 @@ packagevars = packagevars + imagevars
 
 vars_re = {}
 for exp in vars:
-vars_re[exp] = (re.compile('((^|[\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + 
exp), r"\1:" + exp)
+vars_re[exp] = (re.compile('((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + 
exp), r"\1:" + exp)
 
 shortvars_re = {}
 for exp in shortvars:
-shortvars_re[exp] = (re.compile('((^|[\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' 
+ exp + '([\(\'"\s:])'), r"\1:" + exp + r"\3")
+shortvars_re[exp] = (re.compile('((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' 
+ exp + '([\(\'"\s:])'), r"\1:" + exp + r"\3")
 
 package_re = {}
 for exp in packagevars:
-package_re[exp] = (re.compile('(^|[\'"\s\-\+]+)' + exp + '_' + 
'([$a-z"\'\s%\[<{\\\*].)'), r"\1" + exp + r":\2")
+package_re[exp] = (re.compile('(^|[#\'"\s\-\+]+)' + exp + '_' + 
'([$a-z"\'\s%\[<{\\\*].)'), r"\1" + exp + r":\2")
 
 # Other substitutions to make
 subs = {
@@ -123,7 +123,7 @@ def processfile(fn):
 pass
 
 ourname = os.path.basename(sys.argv[0])
-ourversion = "0.9.3"
+ourversion = "0.9.4"
 
 if os.path.isfile(sys.argv[1]):
 processfile(sys.argv[1])
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154540): 
https://lists.openembedded.org/g/openembedded-core/message/154540
Mute This Topic: https://lists.openembedded.org/mt/84704757/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core][PATCH] meta: use new override syntax in comments

[Chen Qi]

Signed-off-by: Chen Qi 
---
 meta/conf/bitbake.conf   | 4 ++--
 meta/conf/machine/qemuppc64.conf | 4 ++--
 meta/recipes-core/dbus/dbus-glib_0.112.bb| 2 +-
 meta/recipes-devtools/elfutils/elfutils_0.185.bb | 2 +-
 meta/recipes-devtools/qemu/qemu.inc  | 2 +-
 5 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 1d5f5b7fcd..e5b31d0b8b 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -236,8 +236,8 @@ DESCRIPTION ?= "${SUMMARY}."
 # The following two are commented out because they result in a recursive
 # definition of the variable in some corner cases.  These are left in
 # to illustrate the intended behavior.
-#SUMMARY_${PN} ?= "${SUMMARY}"
-#DESCRIPTION_${PN} ?= "${DESCRIPTION}"
+#SUMMARY:${PN} ?= "${SUMMARY}"
+#DESCRIPTION:${PN} ?= "${DESCRIPTION}"
 
 SUMMARY:${PN}-src ?= "${SUMMARY} - Source files"
 DESCRIPTION:${PN}-src ?= "${DESCRIPTION}  \
diff --git a/meta/conf/machine/qemuppc64.conf b/meta/conf/machine/qemuppc64.conf
index 0682e752be..a5270e98ed 100644
--- a/meta/conf/machine/qemuppc64.conf
+++ b/meta/conf/machine/qemuppc64.conf
@@ -18,7 +18,7 @@ QB_KERNEL_CMDLINE_APPEND = "console=hvc0"
 QB_OPT_APPEND = "-usb -device usb-tablet"
 
 #prelink broken on ppc64
-#USER_CLASSES_remove = "image-prelink"
-#IMAGE_PREPROCESS_COMMAND_remove = "prelink_image;"
+#USER_CLASSES:remove = "image-prelink"
+#IMAGE_PREPROCESS_COMMAND:remove = "prelink_image;"
 
 MACHINE_EXTRA_RRECOMMENDS += " kernel-modules"
diff --git a/meta/recipes-core/dbus/dbus-glib_0.112.bb 
b/meta/recipes-core/dbus/dbus-glib_0.112.bb
index a03b2addfd..99b0a20004 100644
--- a/meta/recipes-core/dbus/dbus-glib_0.112.bb
+++ b/meta/recipes-core/dbus/dbus-glib_0.112.bb
@@ -20,7 +20,7 @@ SRC_URI[sha256sum] = 
"7d550dccdfcd286e33895501829ed971eeb65c614e73aadb4a08aeef71
 inherit autotools pkgconfig gettext bash-completion gtk-doc
 
 #default disable regression tests, some unit test code in non testing code
-#PACKAGECONFIG_pn-${PN} = "tests" enable regression tests local.conf
+#PACKAGECONFIG:pn-${PN} = "tests" enable regression tests local.conf
 PACKAGECONFIG ??= ""
 PACKAGECONFIG[tests] = "--enable-tests,,,"
 
diff --git a/meta/recipes-devtools/elfutils/elfutils_0.185.bb 
b/meta/recipes-devtools/elfutils/elfutils_0.185.bb
index 5031ab97e7..7a88c5259f 100644
--- a/meta/recipes-devtools/elfutils/elfutils_0.185.bb
+++ b/meta/recipes-devtools/elfutils/elfutils_0.185.bb
@@ -127,7 +127,7 @@ FILES:libdw  = "${libdir}/libdw-${PV}.so 
${libdir}/libdw.so.* ${libdir}/elfutils
 FILES:libdebuginfod = "${libdir}/libdebuginfod-${PV}.so 
${libdir}/libdebuginfod.so.*"
 # Some packages have the version preceeding the .so instead properly
 # versioned .so., so we need to reorder and repackage.
-#FILES_${PN} += "${libdir}/*-${PV}.so ${base_libdir}/*-${PV}.so"
+#FILES:${PN} += "${libdir}/*-${PV}.so ${base_libdir}/*-${PV}.so"
 #FILES_SOLIBSDEV = "${libdir}/libasm.so ${libdir}/libdw.so ${libdir}/libelf.so"
 
 # The package contains symlinks that trip up insane
diff --git a/meta/recipes-devtools/qemu/qemu.inc 
b/meta/recipes-devtools/qemu/qemu.inc
index 4de8a9098b..0bbc4b1187 100644
--- a/meta/recipes-devtools/qemu/qemu.inc
+++ b/meta/recipes-devtools/qemu/qemu.inc
@@ -110,7 +110,7 @@ export LIBTOOL="${HOST_SYS}-libtool"
 
 B = "${WORKDIR}/build"
 
-#EXTRA_OECONF_append = " --python=${HOSTTOOLS_DIR}/python3"
+#EXTRA_OECONF:append = " --python=${HOSTTOOLS_DIR}/python3"
 
 do_configure:prepend:class-native() {
# Append build host pkg-config paths for native target since the host 
may provide sdl
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154539): 
https://lists.openembedded.org/g/openembedded-core/message/154539
Mute This Topic: https://lists.openembedded.org/mt/84704740/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][PATCH 1/2] convert-overrides.py: also convert comments without a leading whitespace

[Martin Jansa]

Can you please bump the script version as well?

On Fri, Aug 6, 2021 at 11:03 AM Chen Qi  wrote:

> Currently lines like below are converted.
>   e.g.
>   # IMAGE_INSTALL_append = " A"
>
> But lines without a leading whitespace are not converted.
>   e.g.
>   #IMAGE_INSTALL_append = " A"
>
> We should be converting both.
>
> Signed-off-by: Chen Qi 
> ---
>  scripts/contrib/convert-overrides.py | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/scripts/contrib/convert-overrides.py
> b/scripts/contrib/convert-overrides.py
> index e4a310d1d6..4d41a4c475 100755
> --- a/scripts/contrib/convert-overrides.py
> +++ b/scripts/contrib/convert-overrides.py
> @@ -69,15 +69,15 @@ packagevars = packagevars + imagevars
>
>  vars_re = {}
>  for exp in vars:
> -vars_re[exp] = (re.compile('((^|[\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_'
> + exp), r"\1:" + exp)
> +vars_re[exp] = (re.compile('((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_'
> + exp), r"\1:" + exp)
>
>  shortvars_re = {}
>  for exp in shortvars:
> -shortvars_re[exp] =
> (re.compile('((^|[\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + exp +
> '([\(\'"\s:])'), r"\1:" + exp + r"\3")
> +shortvars_re[exp] =
> (re.compile('((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + exp +
> '([\(\'"\s:])'), r"\1:" + exp + r"\3")
>
>  package_re = {}
>  for exp in packagevars:
> -package_re[exp] = (re.compile('(^|[\'"\s\-\+]+)' + exp + '_' +
> '([$a-z"\'\s%\[<{\\\*].)'), r"\1" + exp + r":\2")
> +package_re[exp] = (re.compile('(^|[#\'"\s\-\+]+)' + exp + '_' +
> '([$a-z"\'\s%\[<{\\\*].)'), r"\1" + exp + r":\2")
>
>  # Other substitutions to make
>  subs = {
> --
> 2.30.2
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154538): 
https://lists.openembedded.org/g/openembedded-core/message/154538
Mute This Topic: https://lists.openembedded.org/mt/84704328/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core][RFC][PATCH][dunfell] wic: Add --use-stock-fstab option

[Daniel Gomez]

When embeddding rootfs image (e.g. 'rootfs-dir') as a partition we
might want to keep the stock fstab for that image. In such case, use
this option to not update the fstab and use the stock one instead.

This option allows to specify which partitions get the updated fstab
and which get the stock fstab.

Example:
part /export --source rootfs --rootfs-dir=hockeycam-image
--fstype=ext4 --label export --align 1024 --use-stock-fstab

part / --source rootfs --fstype=ext4 --label rootfs --align 1024

Signed-off-by: Daniel Gomez 
---

The commit-id ce682a73b7447652f898ce1d1d0416a456df5416 reverted the
functionality to restore the stock/original fstab file in a rootfs
partition image (wic plugin).

Add back such functionality with an option to the user to select which
fstab will be used for each partition by adding the --use-stock-fstab.
In this case, the user will select whether or not the stock fstab will
be updated or kept.

If you agree with the patch I'll adapt it to the master branch.

Daniel

 scripts/lib/wic/ksparser.py  | 1 +
 scripts/lib/wic/partition.py | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index 452a160232..64933e0a5b 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -184,6 +184,7 @@ class KickStart():
 part.add_argument('--use-uuid', action='store_true')
 part.add_argument('--uuid')
 part.add_argument('--fsuuid')
+part.add_argument('--use-stock-fstab', action='store_true')

 bootloader = subparsers.add_parser('bootloader')
 bootloader.add_argument('--append')
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 85f9847047..c9e45773de 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -54,6 +54,7 @@ class Partition():
 self.uuid = args.uuid
 self.fsuuid = args.fsuuid
 self.type = args.type
+self.use_stock_fstab = args.use_stock_fstab
 self.updated_fstab_path = None
 self.has_fstab = False
 self.update_fstab_in_rootfs = False
@@ -286,7 +287,7 @@ class Partition():
 (self.fstype, extraopts, rootfs, label_str, self.fsuuid, 
rootfs_dir)
 exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)

-if self.updated_fstab_path and self.has_fstab:
+if self.updated_fstab_path and self.has_fstab and not 
self.use_stock_fstab:
 debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
 with open(debugfs_script_path, "w") as f:
 f.write("cd etc\n")
--
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154537): 
https://lists.openembedded.org/g/openembedded-core/message/154537
Mute This Topic: https://lists.openembedded.org/mt/84704596/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core][PATCH 1/2] convert-overrides.py: also convert comments without a leading whitespace

[Chen Qi]

Currently lines like below are converted.
  e.g.
  # IMAGE_INSTALL_append = " A"

But lines without a leading whitespace are not converted.
  e.g.
  #IMAGE_INSTALL_append = " A"

We should be converting both.

Signed-off-by: Chen Qi 
---
 scripts/contrib/convert-overrides.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/scripts/contrib/convert-overrides.py 
b/scripts/contrib/convert-overrides.py
index e4a310d1d6..4d41a4c475 100755
--- a/scripts/contrib/convert-overrides.py
+++ b/scripts/contrib/convert-overrides.py
@@ -69,15 +69,15 @@ packagevars = packagevars + imagevars
 
 vars_re = {}
 for exp in vars:
-vars_re[exp] = (re.compile('((^|[\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + 
exp), r"\1:" + exp)
+vars_re[exp] = (re.compile('((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + 
exp), r"\1:" + exp)
 
 shortvars_re = {}
 for exp in shortvars:
-shortvars_re[exp] = (re.compile('((^|[\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' 
+ exp + '([\(\'"\s:])'), r"\1:" + exp + r"\3")
+shortvars_re[exp] = (re.compile('((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' 
+ exp + '([\(\'"\s:])'), r"\1:" + exp + r"\3")
 
 package_re = {}
 for exp in packagevars:
-package_re[exp] = (re.compile('(^|[\'"\s\-\+]+)' + exp + '_' + 
'([$a-z"\'\s%\[<{\\\*].)'), r"\1" + exp + r":\2")
+package_re[exp] = (re.compile('(^|[#\'"\s\-\+]+)' + exp + '_' + 
'([$a-z"\'\s%\[<{\\\*].)'), r"\1" + exp + r":\2")
 
 # Other substitutions to make
 subs = {
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154536): 
https://lists.openembedded.org/g/openembedded-core/message/154536
Mute This Topic: https://lists.openembedded.org/mt/84704328/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core][PATCH 2/2] convert-overrides.py: skip patches

[Chen Qi]

If a line specifies a patch, skip it.

Signed-off-by: Chen Qi 
---
 scripts/contrib/convert-overrides.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/contrib/convert-overrides.py 
b/scripts/contrib/convert-overrides.py
index 4d41a4c475..387e29d66d 100755
--- a/scripts/contrib/convert-overrides.py
+++ b/scripts/contrib/convert-overrides.py
@@ -63,6 +63,7 @@ skips = skips + ["run_loaddata_poky", 
"determine_if_poky_env", "do_populate_poky
 skips = skips + ["get_appends_for_files", "test_doubleref_remove", 
"test_bitbakelayers_add_remove", "elf32_x86_64", "colour_remove", 
"revmap_remove"]
 skips = skips + ["test_rpm_remove", "test_bitbakelayers_add_remove", 
"recipe_append_file", "log_data_removed", "recipe_append", 
"systemd_machine_unit_append"]
 skips = skips + ["recipetool_append", "changetype_remove", 
"try_appendfile_wc", "test_qemux86_directdisk", "test_layer_appends", 
"tgz_removed"]
+skips = skips + ["file://"]
 
 imagevars = ["IMAGE_CMD", "EXTRA_IMAGECMD", "IMAGE_TYPEDEP", "CONVERSION_CMD", 
"COMPRESS_CMD"]
 packagevars = packagevars + imagevars
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154535): 
https://lists.openembedded.org/g/openembedded-core/message/154535
Mute This Topic: https://lists.openembedded.org/mt/84704327/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] libtool: Fix lto option passing for reproducible builds

[Richard Purdie]

If lto is enabled, we need the prefix-map variables to be passed to the linker.
Add these to the list of options libtool passes through.

Signed-off-by: Richard Purdie 
---
 .../libtool/libtool-2.4.6.inc |  1 +
 .../libtool/libtool/lto-prefix.patch  | 22 +++
 2 files changed, 23 insertions(+)
 create mode 100644 meta/recipes-devtools/libtool/libtool/lto-prefix.patch

diff --git a/meta/recipes-devtools/libtool/libtool-2.4.6.inc 
b/meta/recipes-devtools/libtool/libtool-2.4.6.inc
index c1cbceb51fd..6748d7468e7 100644
--- a/meta/recipes-devtools/libtool/libtool-2.4.6.inc
+++ b/meta/recipes-devtools/libtool/libtool-2.4.6.inc
@@ -23,6 +23,7 @@ SRC_URI = "${GNU_MIRROR}/libtool/libtool-${PV}.tar.gz \

file://0001-libtool-Check-for-static-libs-for-internal-compiler-.patch \

file://0001-Makefile.am-make-sure-autoheader-run-before-autoconf.patch \

file://0001-Makefile.am-make-sure-autoheader-run-before-automake.patch \
+   file://lto-prefix.patch \
   "
 
 SRC_URI[md5sum] = "addf44b646ddb4e3919805aa88fa7c5e"
diff --git a/meta/recipes-devtools/libtool/libtool/lto-prefix.patch 
b/meta/recipes-devtools/libtool/libtool/lto-prefix.patch
new file mode 100644
index 000..2bd010b8e48
--- /dev/null
+++ b/meta/recipes-devtools/libtool/libtool/lto-prefix.patch
@@ -0,0 +1,22 @@
+If lto is enabled, we need the prefix-map variables to be passed to the linker.
+Add these to the list of options libtool passes through.
+
+Upstream-Status: Pending
+Signed-off-by: Richard Purdie 
+
+Index: libtool-2.4.6/build-aux/ltmain.in
+===
+--- libtool-2.4.6.orig/build-aux/ltmain.in
 libtool-2.4.6/build-aux/ltmain.in
+@@ -5424,9 +5424,10 @@ func_mode_link ()
+   # --sysroot=*  for sysroot support
+   # -O*, -g*, -flto*, -fwhopr*, -fuse-linker-plugin GCC link-time 
optimization
+   # -stdlib=*select c++ std lib with clang
++  # -f*-prefix-map*  needed for lto linking
+   -64|-mips[0-9]|-r[0-9][0-9]*|-xarch=*|-xtarget=*|+DA*|+DD*|-q*|-m*| \
+   
-t[45]*|-txscale*|-p|-pg|--coverage|-fprofile-*|-F*|@*|-tp=*|--sysroot=*| \
+-  
-O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*)
++  
-O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*|-f*-prefix-map*)
+ func_quote_for_eval "$arg"
+   arg=$func_quote_for_eval_result
+ func_append compile_command " $arg"
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154534): 
https://lists.openembedded.org/g/openembedded-core/message/154534
Mute This Topic: https://lists.openembedded.org/mt/84704269/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [oe-core][PATCH 1/1] icu: increase command buffer size

[Alexander Kanavin]

Upstream-Status is missing, and this looks like something that should be
sent upstream first for review.

Alex

On Thu, 5 Aug 2021 at 22:22, Joe Slater  wrote:

> Make cmd in pkg_installLibrary() LARGE_BUFFER_SIZE to avoid
> pathname truncation if the install path is longer than about
> 150 characters.  For default compilation, the U_ASSERT does
> not detect buffer overflow.
>
> Signed-off-by: Joe Slater 
> ---
>  ...pkgdata-increase-command-buffer-size.patch | 41 +++
>  meta/recipes-support/icu/icu_69.1.bb  |  1 +
>  2 files changed, 42 insertions(+)
>  create mode 100644
> meta/recipes-support/icu/icu/0001-pkgdata-increase-command-buffer-size.patch
>
> diff --git
> a/meta/recipes-support/icu/icu/0001-pkgdata-increase-command-buffer-size.patch
> b/meta/recipes-support/icu/icu/0001-pkgdata-increase-command-buffer-size.patch
> new file mode 100644
> index 00..14d17f9742
> --- /dev/null
> +++
> b/meta/recipes-support/icu/icu/0001-pkgdata-increase-command-buffer-size.patch
> @@ -0,0 +1,41 @@
> +From ab6b1acdeed76899f8227c38ab7e7675c7673ff1 Mon Sep 17 00:00:00 2001
> +From: Joe Slater 
> +Date: Thu, 5 Aug 2021 09:23:48 -0700
> +Subject: [oe-core][PATCH 1/1] pkgdata: increase command buffer size
> +
> +Make cmd LARGE_BUFFER_SIZE to avoid pathname truncation
> +when install paths are longer than about 150 characters.
> +
> +Signed-off-by: Joe Slater 
> +---
> + source/tools/pkgdata/pkgdata.cpp | 6 +++---
> + 1 file changed, 3 insertions(+), 3 deletions(-)
> +
> +diff --git a/source/tools/pkgdata/pkgdata.cpp
> b/source/tools/pkgdata/pkgdata.cpp
> +index 5ab682e..d4b70ef 100644
> +--- a/tools/pkgdata/pkgdata.cpp
>  b/tools/pkgdata/pkgdata.cpp
> +@@ -1131,17 +1131,17 @@ normal_symlink_mode:
> +
> + static int32_t pkg_installLibrary(const char *installDir, const char
> *targetDir, UBool noVersion) {
> + int32_t result = 0;
> +-char cmd[SMALL_BUFFER_MAX_SIZE];
> ++char cmd[LARGE_BUFFER_MAX_SIZE];
> +
> + auto ret = snprintf(cmd,
> +-SMALL_BUFFER_MAX_SIZE,
> ++LARGE_BUFFER_MAX_SIZE,
> + "cd %s && %s %s %s%s%s",
> + targetDir,
> + pkgDataFlags[INSTALL_CMD],
> + libFileNames[LIB_FILE_VERSION],
> + installDir, PKGDATA_FILE_SEP_STRING,
> libFileNames[LIB_FILE_VERSION]);
> + (void)ret;
> +-U_ASSERT(0 <= ret && ret < SMALL_BUFFER_MAX_SIZE);
> ++U_ASSERT(0 <= ret && ret < LARGE_BUFFER_MAX_SIZE);
> +
> + result = runCommand(cmd);
> +
> +--
> +2.29.2
> +
> diff --git a/meta/recipes-support/icu/icu_69.1.bb
> b/meta/recipes-support/icu/icu_69.1.bb
> index bfeea8d2f0..00c22372b5 100644
> --- a/meta/recipes-support/icu/icu_69.1.bb
> +++ b/meta/recipes-support/icu/icu_69.1.bb
> @@ -107,6 +107,7 @@ SRC_URI = "${BASE_SRC_URI};name=code \
> file://filter.json \
> file://fix-install-manx.patch \
> file://0001-icu-Added-armeb-support.patch \
> +   file://0001-pkgdata-increase-command-buffer-size.patch \
> "
>
>  SRC_URI_append_class-target = "\
> --
> 2.29.2
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154533): 
https://lists.openembedded.org/g/openembedded-core/message/154533
Mute This Topic: https://lists.openembedded.org/mt/84693849/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] cve-check: Add allowlist that is same function of whitelist.

[ito-yui...@fujitsu.com]

HI, Armin

Sorry to bother you again.
Could you please tell me what should I do for the variables to be approved in 
wiki?

Thanks,

Yuichi Ito

> -Original Message-
> From: Ito, Yuichi
> Sent: Tuesday, July 20, 2021 3:49 PM
> To: 'akuster808' 
> Cc: Richard Purdie ;
> openembedded-core@lists.openembedded.org
> Subject: RE: [OE-core] [PATCH] cve-check: Add allowlist that is same
> function of whitelist.
> 
> HI, Armin
> 
> Thanks for information of wiki.
> I created account and add CVE_CHECK_WHITELIST in Variables.
> 
> According to this message, CVE_CHECK_PN_WHITELIST is deprecate and
> should remove it.
> https://lists.openembedded.org/g/openembedded-core/message/142505
> 
> Thanks,
> 
> Yuichi Ito
> 
> > -Original Message-
> > From: akuster808 
> > Sent: Tuesday, July 6, 2021 4:09 AM
> > To: Richard Purdie ; Ito,
> > Yuichi/伊藤 有
> > 一 ;
> openembedded-core@lists.openembedded.org
> > Subject: Re: [OE-core] [PATCH] cve-check: Add allowlist that is same
> > function of whitelist.
> >
> >
> >
> > On 6/26/21 3:04 AM, Richard Purdie wrote:
> > > On Wed, 2021-06-23 at 17:56 +0900, ito-yui...@fujitsu.com wrote:
> > >> The Linux team plan to removed references to racially-charged
> > >> jargon from their code for more neutral and inclusive language.
> > >> So replace use of "whitelist" with "allowlist" in cve-check.
> > >>
> > >> First, we add CVE_CHECK_ALLOWLIST and it is considered patched as
> > >> well as CVE_CHECK_WHITELIST.
> > >> We plan to replace about other word later and eventualy, replace
> > >> all "whitelist" to "allowlist".
> > >>
> > >> Signed-off-by: Yuichi Ito 
> > > The TSC did discuss this and proposed a plan on how we should go
> > > about addressing these issues:
> > >
> > >
> >
> https://lists.openembedded.org/g/openembedded-architecture/topic/inclu
> > > sive_language_summary/75821819
> > >
> > > I appreciate this patch has good intent but I would really like to
> > > see a wider plan on how we address this rather than changing single
> > > variables piecemeal.
> > >
> > > For example we may want to standardise on the term "IGNORE" rather
> > > than "ALLOW" or even "FILTER" or "VERIFIED" or something more
> > > specific to the meaning of CVEs and CVE checking.
> > >
> > > There is an opportunity to try and make the metadata and variable
> > > names more consistent and understandable but if we just change
> > > single things at a time this opportunity would be missed.
> >
> > I have created a wiki page to collate our thoughts and agreements to
> > help us move foreword in a coherent manner to address renaming some
> > troubling variable names.
> > https://wiki.yoctoproject.org/wiki/Inclusive_language
> >
> > I logged some variable names along with a few renaming examples. Maybe
> > this can turn into a plan???
> >
> > Since it's a wiki, please edit accordingly.
> >
> > - armin
> >
> >
> > Cheers,
> > > Richard
> > >
> > >
> > > 
> > >


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154532): 
https://lists.openembedded.org/g/openembedded-core/message/154532
Mute This Topic: https://lists.openembedded.org/mt/83733796/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] Issue after override syntax change

[Richard Purdie]

On Thu, 2021-08-05 at 21:07 +0100, Richard Purdie via lists.openembedded.org 
wrote:
> Hi,
> 
> On Thu, 2021-08-05 at 18:59 +0200, Konrad Weihmann wrote:
> > I'm currently seeing the following behavior after the latest override 
> > syntax change (tested up to today's poky and meta-openembedded)
> > 
> > Take an image and install any kind of packagegroup, which references 
> > several recipes, which pull in dependencies by their own.
> > 
> > image:
> > 
> > IMAGE_INSTALL:append = " packagegroup-foo"
> > 
> > pkggroup:
> > 
> > RDEPENDS:${PN} += "recipe"
> > 
> > recipe:
> > 
> > RDEPENDS:${PN} += "libxml2"
> > 
> > pkgdata for the recipe
> > 
> > RDEPENDS:recipe: libxml2 (>= 2.9.12)
> > 
> > and the pkggroup are correct
> > 
> > RDEPENDS:packagegroup-foo: recipe
> > 
> > still I don't get libxml2 installed into the final image - there is no 
> > sign of that in log.do_rootfs or anywhere else.
> > 
> > The same thing worked pretty well the week before.
> > 
> > Thoughts?
> 
> There should be no change in the behaviour so that is a little concerning.
> 
> Is there an OE-Core example of this I could look at?

I did debug this a bit with Konrad and locally. The packagedata fix I sent out
addresses the issue. I merged it straight into master as it is a nasty issue
and needs fixing ASAP.

Cheers,

Richard





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154531): 
https://lists.openembedded.org/g/openembedded-core/message/154531
Mute This Topic: https://lists.openembedded.org/mt/84689297/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] packagedata: Fix after override syntax change

[Richard Purdie]

Fix a reference that should have been part of the override syntax change
causing packages to be written out incorrectly.

Signed-off-by: Richard Purdie 
---
 meta/classes/packagedata.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/packagedata.bbclass b/meta/classes/packagedata.bbclass
index 3b962f29b53..c2760e2bf0c 100644
--- a/meta/classes/packagedata.bbclass
+++ b/meta/classes/packagedata.bbclass
@@ -27,7 +27,7 @@ python read_subpackage_metadata () {
 # of that variable, e.g. DESCRIPTION could clobber 
DESCRIPTION:
 # We therefore don't clobber for the unsuffixed variable versions
 #
-if key.endswith("_" + pkg):
+if key.endswith(":" + pkg):
 d.setVar(key, sdata[key])
 else:
 d.setVar(key, sdata[key], parsing=True)
-- 
2.30.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154530): 
https://lists.openembedded.org/g/openembedded-core/message/154530
Mute This Topic: https://lists.openembedded.org/mt/84703957/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [poky][dunfell][PATCH] lighttpd: Add patch for reuse large memory chunks

[Purushottam choudhary]

Added 0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch
to fix large memory usage for large file downloads
from dynamic backends reuse or release large memory chunks.

This issue is caused by a bug in the lighttpd 1.4.55 version and
has been fixed in lighttpd 1.4.58. Hence, it is not needed for
master and hardknott branch because lighttpd has 1.4.59 version.

Link: 
https://redmine.lighttpd.net/projects/lighttpd/repository/14/revisions/7ba521ffb4959f6f74a609d5d4acafc29a038337

Signed-off-by: Purushottam Choudhary 
---
 ...large-mem-chunks-fix-mem-usage-fixes-3033.patch | 225 +
 meta/recipes-extended/lighttpd/lighttpd_1.4.55.bb  |   1 +
 2 files changed, 226 insertions(+)
 create mode 100644 
meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch

diff --git 
a/meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch
 
b/meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch
new file mode 100644
index 000..c4e4a60
--- /dev/null
+++ 
b/meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch
@@ -0,0 +1,225 @@
+From 40dadb863fd105c4a66b598c2423b1eb11b1f26e Mon Sep 17 00:00:00 2001
+From: Glenn Strauss 
+Date: Wed, 23 Dec 2020 23:14:47 -0500
+Subject: [PATCH] [core] reuse large mem chunks (fix mem usage) (fixes #3033)
+
+(cherry picked from commit 7ba521ffb4959f6f74a609d5d4acafc29a038337)
+
+(thx flynn)
+
+fix large memory usage for large file downloads from dynamic backends
+
+reuse or release large memory chunks
+
+x-ref:
+  "Memory Growth with PUT and full buffered streams"
+  https://redmine.lighttpd.net/issues/3033
+
+Upstream-Status: Backport
+Comment: No hunk refreshed
+https://redmine.lighttpd.net/projects/lighttpd/repository/14/revisions/7ba521ffb4959f6f74a609d5d4acafc29a038337
+Signed-off-by: Purushottam Choudhary 
+---
+ src/chunk.c| 99 +-
+ src/chunk.h|  2 +
+ src/http-header-glue.c |  2 +-
+ 3 files changed, 82 insertions(+), 21 deletions(-)
+
+diff --git a/src/chunk.c b/src/chunk.c
+index 09dd3f14..ffb37951 100644
+--- a/src/chunk.c
 b/src/chunk.c
+@@ -28,16 +28,20 @@
+ static size_t chunk_buf_sz = 4096;
+ static chunk *chunks, *chunks_oversized;
+ static chunk *chunk_buffers;
++static int chunks_oversized_n;
+ static array *chunkqueue_default_tempdirs = NULL;
+ static off_t chunkqueue_default_tempfile_size = DEFAULT_TEMPFILE_SIZE;
+
+ void chunkqueue_set_chunk_size (size_t sz)
+ {
+-chunk_buf_sz = sz > 0 ? ((sz + 1023) & ~1023uL) : 4096;
++size_t x = 1024;
++while (x < sz && x < (1u << 30)) x <<= 1;
++chunk_buf_sz = sz > 0 ? x : 4096;
+ }
+
+ void chunkqueue_set_tempdirs_default_reset (void)
+ {
++chunk_buf_sz = 8192;
+ chunkqueue_default_tempdirs = NULL;
+ chunkqueue_default_tempfile_size = DEFAULT_TEMPFILE_SIZE;
+ }
+@@ -120,15 +124,49 @@ static void chunk_free(chunk *c) {
+   free(c);
+ }
+
+-buffer * chunk_buffer_acquire(void) {
++static chunk * chunk_pop_oversized(size_t sz) {
++/* future: might have buckets of certain sizes, up to socket buf sizes */
++if (chunks_oversized && chunks_oversized->mem->size >= sz) {
++--chunks_oversized_n;
++chunk *c = chunks_oversized;
++chunks_oversized = c->next;
++return c;
++}
++return NULL;
++}
++
++static void chunk_push_oversized(chunk * const c, const size_t sz) {
++if (chunks_oversized_n < 64 && chunk_buf_sz >= 4096) {
++++chunks_oversized_n;
++chunk **co = _oversized;
++while (*co && sz < (*co)->mem->size) co = &(*co)->next;
++c->next = *co;
++*co = c;
++}
++else
++chunk_free(c);
++}
++
++static buffer * chunk_buffer_acquire_sz(size_t sz) {
+ chunk *c;
+ buffer *b;
+-if (chunks) {
+-c = chunks;
+-chunks = c->next;
++if (sz <= chunk_buf_sz) {
++if (chunks) {
++c = chunks;
++chunks = c->next;
++}
++else
++c = chunk_init(chunk_buf_sz);
++/* future: might choose to pop from chunks_oversized, if available
++ * (even if larger than sz) rather than allocating new chunk
++ * (and if doing so, might replace chunks_oversized_n) */
+ }
+ else {
+-c = chunk_init(chunk_buf_sz);
++/*(round up to nearest chunk_buf_sz)*/
++sz = (sz + (chunk_buf_sz-1)) & ~(chunk_buf_sz-1);
++c = chunk_pop_oversized(sz);
++if (NULL == c)
++c = chunk_init(sz);
+ }
+ c->next = chunk_buffers;
+ chunk_buffers = c;
+@@ -137,21 +175,47 @@ buffer * chunk_buffer_acquire(void) {
+ return b;
+ }
+
++buffer * chunk_buffer_acquire(void) {
++return chunk_buffer_acquire_sz(chunk_buf_sz);
++}
++
+ void chunk_buffer_release(buffer *b) {
+ if (NULL == b) return;