Re: [OE-core] [PATCH] git: set --with-gitconfig=/etc/gitconfig for -native builds

2024-05-16 Thread Rasmus Villemoes via lists.openembedded.org
Polite ping.

On 23/04/2024 11.57, Rasmus Villemoes via lists.openembedded.org wrote:
> From: Rasmus Villemoes 
> 
> Commit 6c2ae2346db0 (kern-tools: depend on git-replacement-native)
> broke our kernel builds. For saving space and time, we have a DL_DIR
> shared between multiple users/buildbots, not all of which run with the
> same uid (and with appropriate sticky bits set so that files
> downloaded by one user become owned by a common group and are readable
> by others). This works fine also for git sources because the docker
> images we use all have a /etc/gitconfig with
> 
>   [safe]
> directory = *
> 
> But with the mentioned commit, the host's git is no longer used for
> do_unpack (nor for do_fetch if re-building and sysroot has already
> been populated by a previous build), causing spurious "fatal: detected
> dubious ownership..." failures.
> 
> Currently, the path where the git-native binary searches for system
> gitconfig is the sysroot from it was built, which obviously doesn't
> contain a /etc/gitconfig. As for the nativesdk variant, respect the
> host's /etc/gitconfig if present.
> 
> Signed-off-by: Rasmus Villemoes 
> ---
>  meta/recipes-devtools/git/git_2.44.0.bb | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/recipes-devtools/git/git_2.44.0.bb 
> b/meta/recipes-devtools/git/git_2.44.0.bb
> index 90e555eba7..78b00dd19f 100644
> --- a/meta/recipes-devtools/git/git_2.44.0.bb
> +++ b/meta/recipes-devtools/git/git_2.44.0.bb
> @@ -40,6 +40,7 @@ EXTRA_OECONF = 
> "--with-perl=${STAGING_BINDIR_NATIVE}/perl-native/perl \
>   --without-iconv \
>  "
>  EXTRA_OECONF:append:class-nativesdk = " --with-gitconfig=/etc/gitconfig "
> +EXTRA_OECONF:append:class-native = " --with-gitconfig=/etc/gitconfig "
>  
>  # Needs brokensep as this doesn't use automake
>  inherit autotools-brokensep perlnative bash-completion manpages
> 
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#199445): 
https://lists.openembedded.org/g/openembedded-core/message/199445
Mute This Topic: https://lists.openembedded.org/mt/105686820/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] systemd: sed ROOT_HOME only if sysusers PACKAGECONFIG is set

2024-04-24 Thread Rasmus Villemoes via lists.openembedded.org
On 24/04/2024 10.36, Alexander Kanavin via lists.openembedded.org wrote:
> On Wed, 24 Apr 2024 at 10:19, Christian B. Sørensen via
> lists.openembedded.org 
> wrote:
> 
>> Any input on the patchset ? Would appreciate if it could be included in the 
>> scarthgap release.
> 
> A little patience would help. The patch is fine (and so there's not
> input), it's in a staging branch, 

How would one know? It's not in master-next, and I don't see any other
branches containing it. Is there some git repo other than
https://git.openembedded.org/openembedded-core/ one could follow to see
that a patch has actually been picked up? The complete radio silence
when submitting to oe-core makes it extremely hard to know if anybody
has even seen a given patch. Ideally, one would get an email
notification (not sent to the list), but just being able to peek at some
staging repo would help avoid "impatient" pings.

> but integration to master would happen after scarthgap is out.

So the plan is to release scarthgap with a known bug and a known fix for
it? Or are you saying this will go into the scarthgap branch before
release, and after that be applied to master?

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198685): 
https://lists.openembedded.org/g/openembedded-core/message/198685
Mute This Topic: https://lists.openembedded.org/mt/105593219/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] git: set --with-gitconfig=/etc/gitconfig for -native builds

2024-04-23 Thread Rasmus Villemoes via lists.openembedded.org
From: Rasmus Villemoes 

Commit 6c2ae2346db0 (kern-tools: depend on git-replacement-native)
broke our kernel builds. For saving space and time, we have a DL_DIR
shared between multiple users/buildbots, not all of which run with the
same uid (and with appropriate sticky bits set so that files
downloaded by one user become owned by a common group and are readable
by others). This works fine also for git sources because the docker
images we use all have a /etc/gitconfig with

  [safe]
directory = *

But with the mentioned commit, the host's git is no longer used for
do_unpack (nor for do_fetch if re-building and sysroot has already
been populated by a previous build), causing spurious "fatal: detected
dubious ownership..." failures.

Currently, the path where the git-native binary searches for system
gitconfig is the sysroot from it was built, which obviously doesn't
contain a /etc/gitconfig. As for the nativesdk variant, respect the
host's /etc/gitconfig if present.

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-devtools/git/git_2.44.0.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-devtools/git/git_2.44.0.bb 
b/meta/recipes-devtools/git/git_2.44.0.bb
index 90e555eba7..78b00dd19f 100644
--- a/meta/recipes-devtools/git/git_2.44.0.bb
+++ b/meta/recipes-devtools/git/git_2.44.0.bb
@@ -40,6 +40,7 @@ EXTRA_OECONF = 
"--with-perl=${STAGING_BINDIR_NATIVE}/perl-native/perl \
--without-iconv \
 "
 EXTRA_OECONF:append:class-nativesdk = " --with-gitconfig=/etc/gitconfig "
+EXTRA_OECONF:append:class-native = " --with-gitconfig=/etc/gitconfig "
 
 # Needs brokensep as this doesn't use automake
 inherit autotools-brokensep perlnative bash-completion manpages
-- 
2.40.1.1.g1c60b9335d


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198611): 
https://lists.openembedded.org/g/openembedded-core/message/198611
Mute This Topic: https://lists.openembedded.org/mt/105686820/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] bmaptool: add PROVIDES bmap-tools for compatibility

2024-04-19 Thread Rasmus Villemoes via lists.openembedded.org
On 19/04/2024 23.00, Alexander Kanavin wrote:
> This was already proposed, and rejected.
> https://lists.openembedded.org/g/openembedded-core/topic/104753355
> 
> You need to fix the metadata that refers to the old name.

So, how can a layer then be compatible with both nanbield and scarthgap?
Could this perhaps be applied just to the scarthgap branch, but not master?

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198565): 
https://lists.openembedded.org/g/openembedded-core/message/198565
Mute This Topic: https://lists.openembedded.org/mt/105625834/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] bmaptool: add PROVIDES bmap-tools for compatibility

2024-04-19 Thread Rasmus Villemoes via lists.openembedded.org
From: Rasmus Villemoes 

This is very often a build dependency, such as in my case using a
class from meta-ptx, which fails with

ERROR: Nothing PROVIDES 'bmap-tools-native'. Close matches:
  bmaptool-native
  bpftool-native
  mtools-native

due to the renaming.

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-support/bmaptool/bmaptool_git.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-support/bmaptool/bmaptool_git.bb 
b/meta/recipes-support/bmaptool/bmaptool_git.bb
index fd53c21c06..b4b3fa1fbd 100644
--- a/meta/recipes-support/bmaptool/bmaptool_git.bb
+++ b/meta/recipes-support/bmaptool/bmaptool_git.bb
@@ -23,6 +23,7 @@ RDEPENDS:${PN} = "python3-core python3-compression 
python3-misc python3-mmap pyt
 inherit setuptools3
 
 # For compatibility with layers before scarthgap
+PROVIDES = "bmap-tools"
 RREPLACES:${PN} = "bmap-tools"
 RCONFLICTS:${PN} = "bmap-tools"
 
-- 
2.40.1.1.g1c60b9335d


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198562): 
https://lists.openembedded.org/g/openembedded-core/message/198562
Mute This Topic: https://lists.openembedded.org/mt/105625834/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] openssh: add After dependencies on nss-user-lookup.target

2024-04-17 Thread Rasmus Villemoes via lists.openembedded.org
From: Rasmus Villemoes 

Quoting 'man systemd.special':

  nss-user-lookup.target

A target that should be used as synchronization point for all
regular UNIX user/group name service lookups. [...] All services
for which the availability of the full user/group database is
essential should be ordered after this target, but not pull it
in. All services which provide parts of the user/group database
should be ordered before this target, and pull it in.

When no service providing parts of the user/group database exists and
thus pulls in the nss-user-lookup.target, this added dependency is a
no-op.

However, when such a service does exist, and e.g. modifies /etc/shadow
to change password or enable/disable certain accounts, it is essential
that no ssh connections are accepted until those changes are made.

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-connectivity/openssh/openssh/sshd.service | 1 +
 meta/recipes-connectivity/openssh/openssh/sshd.socket  | 1 +
 2 files changed, 2 insertions(+)

diff --git a/meta/recipes-connectivity/openssh/openssh/sshd.service 
b/meta/recipes-connectivity/openssh/openssh/sshd.service
index 2a997b656a..3e570ab1e5 100644
--- a/meta/recipes-connectivity/openssh/openssh/sshd.service
+++ b/meta/recipes-connectivity/openssh/openssh/sshd.service
@@ -2,6 +2,7 @@
 Description=OpenSSH server daemon
 Wants=sshdgenkeys.service
 After=sshdgenkeys.service
+After=nss-user-lookup.target
 
 [Service]
 Environment="SSHD_OPTS="
diff --git a/meta/recipes-connectivity/openssh/openssh/sshd.socket 
b/meta/recipes-connectivity/openssh/openssh/sshd.socket
index 8d76d62309..7dd2ed0626 100644
--- a/meta/recipes-connectivity/openssh/openssh/sshd.socket
+++ b/meta/recipes-connectivity/openssh/openssh/sshd.socket
@@ -1,6 +1,7 @@
 [Unit]
 Conflicts=sshd.service
 Wants=sshdgenkeys.service
+After=nss-user-lookup.target
 
 [Socket]
 ExecStartPre=@BASE_BINDIR@/mkdir -p /var/run/sshd
-- 
2.40.1.1.g1c60b9335d


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198463): 
https://lists.openembedded.org/g/openembedded-core/message/198463
Mute This Topic: https://lists.openembedded.org/mt/105575252/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH v2] perf: add jevents PACKAGECONFIG item

2023-11-07 Thread Rasmus Villemoes via lists.openembedded.org
From: Rasmus Villemoes 

Building for an arm64 target, e.g. qemuarm64 or a raspberrypi3,
without "python" in PACKAGECONFIG, results in

| Makefile.config:892: *** ERROR: No python interpreter needed for jevents 
generation. Install python or build with NO_JEVENTS=1..  Stop.

Signed-off-by: Rasmus Villemoes 
---
v2: add comment on (non-)relationship to 'python' PC item.

 meta/recipes-kernel/perf/perf.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index a392166e73..5672f7d338 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -28,6 +28,9 @@ PACKAGECONFIG[audit] = ",NO_LIBAUDIT=1,audit"
 PACKAGECONFIG[manpages] = ",,xmlto-native asciidoc-native"
 PACKAGECONFIG[cap] = ",,libcap"
 PACKAGECONFIG[libtraceevent] = ",NO_LIBTRACEEVENT=1,libtraceevent"
+# jevents requires host python for generating a .c file, but is
+# unrelated to the python item.
+PACKAGECONFIG[jevents] = ",NO_JEVENTS=1,python3-native"
 # Arm CoreSight
 PACKAGECONFIG[coresight] = "CORESIGHT=1,,opencsd"
 PACKAGECONFIG[pfm4] = ",NO_LIBPFM4=1,libpfm4"
-- 
2.40.1.1.g1c60b9335d


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190260): 
https://lists.openembedded.org/g/openembedded-core/message/190260
Mute This Topic: https://lists.openembedded.org/mt/102439731/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] perf: add jevents PACKAGECONFIG item

2023-11-07 Thread Rasmus Villemoes via lists.openembedded.org
On 06/11/2023 16.07, Richard Purdie wrote:
> On Mon, 2023-11-06 at 16:00 +0100, Rasmus Villemoes wrote:
>>
>> When I set PACKAGECONFIG for perf in a perf.bbappend file, without
>> python being in there, the build breaks:
>>
[...]
>> This difference is, to put it mildly, quite confusing and surprising
>> (and took quite some time to track down). I don't know if there's
>> anything to do about it. Is setting PACKAGECONFIG via :pn-foo overrides
>> the only supported method, or is this just a quirk of the perf recipe
>> that one has to live with?
> 
> That is a pretty horrible interaction of things. As you say, that
> inherit is effectively an immediate expansion of the data and wouldn't
> "see" changes in any bbappend.
> 
> Could you file a bug please?

Done. https://bugzilla.yoctoproject.org/show_bug.cgi?id=15276

I now see that the same problem probably exists for the perl item (just
the other way around because perl is not initially in PACKAGECONFIG);
the line

include ${@bb.utils.contains('PACKAGECONFIG', 'perl', 'perf-perl.inc',
'', d)}

won't have effect if perl is only set in PACKAGECONFIG in a .bbappend.

> I'm wondering if we should have some other form of inherit which
> applies at the end of parsing which would avoid that issue. Sometimes
> order is important, sometimes it is less important with the inherits.

Dunno. Maybe a new bitbake keyword "defer" (or "late" or whatever):

defer inherit ${@something(PACKAGECONFIG)}

that would parse the rest of the line as usual, but instead of adding
directly to the statements list, instead to some new deferred_statements
(which StatementGroup could track itself, it would probably just need to
be taught .append_deferred and .eval_deferred methods and have a
.deferred member).

I don't really know much about bitbake internals, it doesn't sound too
hard to do, but I'm probably missing something. Also, perhaps it's too
subtle.

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190258): 
https://lists.openembedded.org/g/openembedded-core/message/190258
Mute This Topic: https://lists.openembedded.org/mt/102420698/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] perf: add jevents PACKAGECONFIG item

2023-11-06 Thread Rasmus Villemoes via lists.openembedded.org
On 06/11/2023 15.49, Bruce Ashfield wrote:
> On Mon, Nov 6, 2023 at 9:22 AM Rasmus Villemoes
>  wrote:
>>
>> From: Rasmus Villemoes 
>>
>> Building for an arm64 target, e.g. qemuarm64 or a raspberrypi3,
>> without "python" in PACKAGECONFIG, results in
>>
>> | Makefile.config:892: *** ERROR: No python interpreter needed for jevents 
>> generation. Install python or build with NO_JEVENTS=1..  Stop.
>>
>> Signed-off-by: Rasmus Villemoes 
>> ---
>>  meta/recipes-kernel/perf/perf.bb | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/meta/recipes-kernel/perf/perf.bb 
>> b/meta/recipes-kernel/perf/perf.bb
>> index a392166e73..37b53ee7df 100644
>> --- a/meta/recipes-kernel/perf/perf.bb
>> +++ b/meta/recipes-kernel/perf/perf.bb
>> @@ -28,6 +28,7 @@ PACKAGECONFIG[audit] = ",NO_LIBAUDIT=1,audit"
>>  PACKAGECONFIG[manpages] = ",,xmlto-native asciidoc-native"
>>  PACKAGECONFIG[cap] = ",,libcap"
>>  PACKAGECONFIG[libtraceevent] = ",NO_LIBTRACEEVENT=1,libtraceevent"
>> +PACKAGECONFIG[jevents] = ",NO_JEVENTS=1,python3-native"
> 
> Since we already have a python packageconfig and it takes care of
> the dependencies that we need, we shouldn't be repeating the python3-native
> in this one. We'll end up having to maintain that dependency in
> two locations.
> 
> i.e. we don't have python3-native as a dependency of our main python
> packageconfig .. is that an error (the python packageconfig) or should
> this be depending on python3 python3-setuptools-native, just like the
> python one ?

The two are as I understand it entirely unrelated. The existing python
knob controls whether some stuff that requires python at runtime is
built/installed (and that build may of course itself require some native
python, in this case in the guise of setuptools-native).

The jevents stuff is purely about generating some tables [as a .c file]
at build time, and that requires native python, but does not impose a
dependency on python on target.

> Since there's no way to make packageconfig 'a' depend on
> packageconfig 'b' that I know of (RP will correct me if I'm missing
> or forgetting something here), we could always add a test on the
> final PACKAGECONFIG and error/warn if jevents was enabled and
> python wasn't.

I don't think it's correct to force jevents pc to include python pc, as
the latter will make the perf binary pull in the whole python
interpreter etc.

If feature A depends on foo-native and foo and bar, and feature B
requires foo-native, I don't see why one should be forced to enable
feature A in order to get B. [Conversely, even if the dependencies of
feature A today happen to be a superset of those of feature B, I don't
think selecting feature A should automatically imply feature B.]

> It is always a bit annoying that perf turns these on, and we have
> to explicitly opt-out .. but that is out of our control :)

Indeed.

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190227): 
https://lists.openembedded.org/g/openembedded-core/message/190227
Mute This Topic: https://lists.openembedded.org/mt/102420698/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] perf: add jevents PACKAGECONFIG item

2023-11-06 Thread Rasmus Villemoes via lists.openembedded.org
Somewhat related, but maybe not really:

When I set PACKAGECONFIG for perf in a perf.bbappend file, without
python being in there, the build breaks:

| Makefile.config:277: ***
.../tmp/work/qemuarm64-poky-linux/perf/1.0/recipe-sysroot-native/usr/bin/python3-native/python3-config
not found.  Stop.
| make[1]: *** [Makefile.perf:242: sub-make] Error 2

If I instead set the same PACKAGECONFIG value, but by doing

PACKAGECONFIG:pn-perf = "tui libunwind libtraceevent"

in some global configuration file, it works as expected.



I think the difference is due to the

inherit ${@bb.utils.contains('PACKAGECONFIG', 'python',
'python3targetconfig', '', d)}

line in the perf recipe. At the time where this line is evaluated, if
PACKAGECONFIG is set via a .bbappend, this ends up evaluated using the
??= value which does include python, and hence we end up inheriting
python3targetconfig, which in turn inherits python3native, which does

export PYTHON

And since PYTHON is now exported, perf's logic ends up in this fragment
in tools/perf/Makefile.config:

# If PYTHON is defined but PYTHON_CONFIG isn't, then take
$(PYTHON)-config as if it was the user
# supplied value for PYTHON_CONFIG. Because it's "user supplied", error
out if it doesn't exist.
ifdef PYTHON
  ifndef PYTHON_CONFIG
PYTHON_CONFIG_AUTO := $(call get-executable,$(PYTHON)-config)
PYTHON_CONFIG := $(if $(PYTHON_CONFIG_AUTO),$(PYTHON_CONFIG_AUTO),\
  $(call $(error $(PYTHON)-config not found)))
  endif
endif

which breaks because the value of PACKAGECONFIG that ended up
determining buildtime dependencies did not pull in that python3-config.



This difference is, to put it mildly, quite confusing and surprising
(and took quite some time to track down). I don't know if there's
anything to do about it. Is setting PACKAGECONFIG via :pn-foo overrides
the only supported method, or is this just a quirk of the perf recipe
that one has to live with?

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190222): 
https://lists.openembedded.org/g/openembedded-core/message/190222
Mute This Topic: https://lists.openembedded.org/mt/102420698/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] perf: add jevents PACKAGECONFIG item

2023-11-06 Thread Rasmus Villemoes via lists.openembedded.org
From: Rasmus Villemoes 

Building for an arm64 target, e.g. qemuarm64 or a raspberrypi3,
without "python" in PACKAGECONFIG, results in

| Makefile.config:892: *** ERROR: No python interpreter needed for jevents 
generation. Install python or build with NO_JEVENTS=1..  Stop.

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-kernel/perf/perf.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index a392166e73..37b53ee7df 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -28,6 +28,7 @@ PACKAGECONFIG[audit] = ",NO_LIBAUDIT=1,audit"
 PACKAGECONFIG[manpages] = ",,xmlto-native asciidoc-native"
 PACKAGECONFIG[cap] = ",,libcap"
 PACKAGECONFIG[libtraceevent] = ",NO_LIBTRACEEVENT=1,libtraceevent"
+PACKAGECONFIG[jevents] = ",NO_JEVENTS=1,python3-native"
 # Arm CoreSight
 PACKAGECONFIG[coresight] = "CORESIGHT=1,,opencsd"
 PACKAGECONFIG[pfm4] = ",NO_LIBPFM4=1,libpfm4"
-- 
2.40.1.1.g1c60b9335d


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190219): 
https://lists.openembedded.org/g/openembedded-core/message/190219
Mute This Topic: https://lists.openembedded.org/mt/102420698/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] valgrind: split helper scripts to separate packages, update dependencies

2023-11-03 Thread Rasmus Villemoes via lists.openembedded.org
From: Rasmus Villemoes 

The cachegrind scripts have been rewritten in python3, so the RDEPENDS
on perl is no longer sufficient. This is unfortunately not caught by
QA checks since the scripts use

  #! /usr/bin/env python3

as shebang line.

Since the valgrind binary by itself can be quite useful on target,
allow including that in the rootfs without also having to pull in the
whole perl and python interpreters and stdlibs.

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-devtools/valgrind/valgrind_3.21.0.bb | 14 --
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/valgrind/valgrind_3.21.0.bb 
b/meta/recipes-devtools/valgrind/valgrind_3.21.0.bb
index 717d82830e..68ab319a1b 100644
--- a/meta/recipes-devtools/valgrind/valgrind_3.21.0.bb
+++ b/meta/recipes-devtools/valgrind/valgrind_3.21.0.bb
@@ -92,6 +92,18 @@ CACHED_CONFIGUREVARS += "ac_cv_path_PERL='/usr/bin/env perl'"
 # which fixes build path issue in DWARF.
 SELECTED_OPTIMIZATION = "${DEBUG_FLAGS}"
 
+# Split out various helper scripts to separate packages to avoid the
+# main package depending on perl and python.
+PACKAGES =+ "${PN}-cachegrind ${PN}-massif ${PN}-callgrind"
+
+FILES:${PN}-cachegrind = "${bindir}/cg_*"
+FILES:${PN}-massif = "${bindir}/ms_*"
+FILES:${PN}-callgrind = "${bindir}/callgrind_*"
+
+RDEPENDS:${PN}-cachegrind = "${PN} python3-core"
+RDEPENDS:${PN}-massif = "${PN} perl"
+RDEPENDS:${PN}-callgrind = "${PN} perl"
+
 do_configure:prepend () {
 rm -rf ${S}/config.h
 sed -i -e 's:$(abs_top_builddir):$(pkglibdir)/ptest:g' 
${S}/none/tests/Makefile.am
@@ -116,8 +128,6 @@ VALGRINDARCH:powerpc64le = "ppc64le"
 
 INHIBIT_PACKAGE_STRIP_FILES = 
"${PKGD}${libexecdir}/valgrind/vgpreload_memcheck-${VALGRINDARCH}-linux.so"
 
-RDEPENDS:${PN} += "perl"
-
 # valgrind needs debug information for ld.so at runtime in order to
 # redirect functions like strlen.
 RRECOMMENDS:${PN} += "${TCLIBC}-dbg"
-- 
2.40.1.1.g1c60b9335d


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190163): 
https://lists.openembedded.org/g/openembedded-core/message/190163
Mute This Topic: https://lists.openembedded.org/mt/102365327/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH v2] perf: lift TARGET_CC_ARCH modification out of security_flags.inc

2023-10-23 Thread Rasmus Villemoes via lists.openembedded.org
From: Rasmus Villemoes 

Building perf without security_flags.inc being included in one's
distro results in the buildpaths warning

WARNING: perf-1.0-r9 do_package_qa: QA Issue: File /usr/bin/trace in
package perf contains reference to TMPDIR

because the ${DEBUG_PREFIX_MAP} does not get used. Most recipes get
that from CFLAGS, but the perf recipe explicitly unsets that.

Now ${SELECTED_OPTIMIZATION} of course contains more than just
${DEBUG_FLAGS}/${DEBUG_PREFIX_MAP}. For most TUs, perf's build system
adds its own optimization flags (-O6 for odd reasons), so for those
including the -O2 or -Og doesn't change anything. But looking at the
.o.cmd files show that there are some TUs which currently get built
without any -O flag. So for those adding the distro's
SELECTED_OPTIMIZATION seem to be the right thing to do.

Signed-off-by: Rasmus Villemoes 
---
v2: add comment

 meta/conf/distro/include/security_flags.inc | 1 -
 meta/recipes-kernel/perf/perf.bb| 9 +
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/conf/distro/include/security_flags.inc 
b/meta/conf/distro/include/security_flags.inc
index 2972f05b4e..d97a6edb0f 100644
--- a/meta/conf/distro/include/security_flags.inc
+++ b/meta/conf/distro/include/security_flags.inc
@@ -69,4 +69,3 @@ SECURITY_LDFLAGS:pn-xserver-xorg = "${SECURITY_X_LDFLAGS}"
 TARGET_CC_ARCH:append:pn-binutils = " ${SELECTED_OPTIMIZATION}"
 TARGET_CC_ARCH:append:pn-gcc = " ${SELECTED_OPTIMIZATION}"
 TARGET_CC_ARCH:append:pn-gdb = " ${SELECTED_OPTIMIZATION}"
-TARGET_CC_ARCH:append:pn-perf = " ${SELECTED_OPTIMIZATION}"
diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 675acfaf26..a392166e73 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -73,6 +73,15 @@ SPDX_S = "${S}/tools/perf"
 # supported kernel.
 LDFLAGS="-ldl -lutil"
 
+# Perf's build system adds its own optimization flags for most TUs,
+# overriding the flags included here. But for some, perf does not add
+# any -O option, so ensure the distro's chosen optimization gets used
+# for those. Since ${SELECTED_OPTIMIZATION} always includes
+# ${DEBUG_FLAGS} which in turn includes ${DEBUG_PREFIX_MAP}, this also
+# ensures perf is built with appropriate -f*-prefix-map options,
+# avoiding the 'buildpaths' QA warning.
+TARGET_CC_ARCH += "${SELECTED_OPTIMIZATION}"
+
 EXTRA_OEMAKE = '\
 V=1 \
 VF=1 \
-- 
2.40.1.1.g1c60b9335d


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189615): 
https://lists.openembedded.org/g/openembedded-core/message/189615
Mute This Topic: https://lists.openembedded.org/mt/102130591/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] perf: lift TARGET_CC_ARCH modification out of security_flags.inc

2023-10-20 Thread Rasmus Villemoes via lists.openembedded.org
On 20/10/2023 12.13, Richard Purdie wrote:
> On Fri, 2023-10-20 at 12:03 +0200, Rasmus Villemoes wrote:
>> On 20/10/2023 11.38, Richard Purdie wrote:
>>> On Fri, 2023-10-20 at 10:10 +0200, Rasmus Villemoes wrote:
>>>> On 19/10/2023 14.48, Richard Purdie wrote:
>>
>>>>> The fact this works suggests perf is ignoring TARGET_CFLAGS. Is there
>>>>> anything in the perf build system where we should be passing in cflags
>>>>> we really want used?
>>>>
>>>> Well, IIUC, the normal way TARGET_CFLAGS would take effect is via the
>>>> 'export CFLAGS = "${TARGET_CFLAGS}'. But in the perf recipe, both
>>>> do_compile and do_install start with
>>>>
>>>> # Linux kernel build system is expected to do the right thing
>>>> unset CFLAGS
>>>>
>>>> And perf's build system does indeed add lots of flags of its own, at
>>>> least for most TUs, but nowhere is any -f(macro/debug/file)-prefix being
>>>> set.
>>>>
>>>> So I do think that TARGET_CC_ARCH is the best place for flags that we
>>>> really want used. At least as an initial step, because this is known to
>>>> work when using the security_flags.inc file. Maybe there's some cleaner
>>>> way where we don't unset CFLAGS, but that could be a giant can of worms.
>>>>
>>>> And yes, a comment should be added. Is something like
>>>>
>>>> # Perf's build system adds its own optimization flags for most TUs,
>>>> # overriding the flags included here. But for some, perf does not add
>>>> # any -O option, so ensure the distro's chosen optimization gets used
>>>> # for those. Since ${SELECTED_OPTIMIZATION} always includes
>>>> # ${DEBUG_FLAGS} which in turn includes ${DEBUG_PREFIX_MAP}, this also
>>>> # ensures perf is built with appropriate -f*-prefix-map options,
>>>> # avoiding the 'buildpaths' QA warning.
>>>>
>>>> too verbose?
>>>
>>> If it were me I'd probably write:
>>>
>>> # The perf makefile has "unset CFLAGS" as "Linux kernel build system is
>>> # expected to do the right thing". 
>>
>> It's not the perf makefile, it's right there in our recipe itself. I
>> don't think upstream perf is to blame.
> 
> Sorry, I misunderstood. I thought this was the perf environment that
> was doing this so in that case your comment does make much more sense
> than mine.

OK. Bruce, should I resend?

> Out of interest did you see what happens if you allow our CFLAGS in and
> remove that unset? 

I don't think I've tried that, but I've tried a few other things that I
thought would be cleaner but which then exploded, so in the end I
decided that just lifting exactly what the poky distro does should be
safest and have the highest likelyhood of being accepted.

> I also wondered if EXTRA_CFLAGS in EXTRA_OEMAKE
> might be an option?

Based on quick reading of perf build documentation, yes, EXTRA_CFLAGS
seems like a reasonable approach. What I'm a little worried about is the
TUs where perf apparently does not by itself add all its cflags (-O6 and
whatnot), so I'm not sure if they would be built with what is passed via
EXTRA_CFLAGS.

I will experiment sometime next week if I remember and nothing else
preempts that plan.

> I'm a bit worried that the current situation all feels a little
> fragile.

Agree. But the QA checks do help quite a lot here.

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189518): 
https://lists.openembedded.org/g/openembedded-core/message/189518
Mute This Topic: https://lists.openembedded.org/mt/102058904/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] perf: lift TARGET_CC_ARCH modification out of security_flags.inc

2023-10-20 Thread Rasmus Villemoes via lists.openembedded.org
On 20/10/2023 11.38, Richard Purdie wrote:
> On Fri, 2023-10-20 at 10:10 +0200, Rasmus Villemoes wrote:
>> On 19/10/2023 14.48, Richard Purdie wrote:

>>> The fact this works suggests perf is ignoring TARGET_CFLAGS. Is there
>>> anything in the perf build system where we should be passing in cflags
>>> we really want used?
>>
>> Well, IIUC, the normal way TARGET_CFLAGS would take effect is via the
>> 'export CFLAGS = "${TARGET_CFLAGS}'. But in the perf recipe, both
>> do_compile and do_install start with
>>
>> # Linux kernel build system is expected to do the right thing
>> unset CFLAGS
>>
>> And perf's build system does indeed add lots of flags of its own, at
>> least for most TUs, but nowhere is any -f(macro/debug/file)-prefix being
>> set.
>>
>> So I do think that TARGET_CC_ARCH is the best place for flags that we
>> really want used. At least as an initial step, because this is known to
>> work when using the security_flags.inc file. Maybe there's some cleaner
>> way where we don't unset CFLAGS, but that could be a giant can of worms.
>>
>> And yes, a comment should be added. Is something like
>>
>> # Perf's build system adds its own optimization flags for most TUs,
>> # overriding the flags included here. But for some, perf does not add
>> # any -O option, so ensure the distro's chosen optimization gets used
>> # for those. Since ${SELECTED_OPTIMIZATION} always includes
>> # ${DEBUG_FLAGS} which in turn includes ${DEBUG_PREFIX_MAP}, this also
>> # ensures perf is built with appropriate -f*-prefix-map options,
>> # avoiding the 'buildpaths' QA warning.
>>
>> too verbose?
> 
> If it were me I'd probably write:
> 
> # The perf makefile has "unset CFLAGS" as "Linux kernel build system is
> # expected to do the right thing". 

It's not the perf makefile, it's right there in our recipe itself. I
don't think upstream perf is to blame.

> It doesn't and we need our prefix 
> # map options 

Yes.

> and security flags amongst other things so force our 
> # cflags in.

This has nothing to do with the security flags. That file still contains

TARGET_CC_ARCH:append:class-target = " ${SECURITY_CFLAGS}"

and TARGET_CC_ARCH is included in the CC value we pass to perf's make,
with or without this patch. So if the distro includes
security_flags.inc, perf gets built with those flags as expected.

But when that file is _not_ included, we do miss the prefix-map flags,
which should be passed regardless of any security distro settings. I
think the line I'm moving out of security_flags.inc have never really
had any business being there in the first place, regardless of whether
adding ${SELECTED_OPTIMIZATION} to TARGET_CC_ARCH is the most
appropriate method to get those prefix-map options passed. But since
that's how the poky distro has achieved that until now, it seems to be
the least invasive and least risky fix for distros that don't use
security_flags.inc.

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189513): 
https://lists.openembedded.org/g/openembedded-core/message/189513
Mute This Topic: https://lists.openembedded.org/mt/102058904/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] perf: lift TARGET_CC_ARCH modification out of security_flags.inc

2023-10-20 Thread Rasmus Villemoes via lists.openembedded.org
On 19/10/2023 14.48, Richard Purdie wrote:
> On Thu, 2023-10-19 at 14:32 +0200, Rasmus Villemoes via
> lists.openembedded.org wrote:
>> From: Rasmus Villemoes 
>>
>> Building perf without security_flags.inc being included in one's
>> distro results in the buildpaths warning
>>
>> WARNING: perf-1.0-r9 do_package_qa: QA Issue: File /usr/bin/trace in
>> package perf contains reference to TMPDIR
>>
>> because the ${DEBUG_PREFIX_MAP} does not get used. Most recipes get
>> that from CFLAGS, but the perf recipe explicitly unsets that.
>>
>> Now ${SELECTED_OPTIMIZATION} of course contains more than just
>> ${DEBUG_FLAGS}/${DEBUG_PREFIX_MAP}. For most TUs, perf's build system
>> adds its own optimization flags (-O6 for odd reasons), so for those
>> including the -O2 or -Og doesn't change anything. But looking at the
>> .o.cmd files show that there are some TUs which currently get built
>> without any -O flag. So for those adding the distro's
>> SELECTED_OPTIMIZATION seem to be the right thing to do.
>>
>> Signed-off-by: Rasmus Villemoes 
>> ---
>>  meta/conf/distro/include/security_flags.inc | 1 -
>>  meta/recipes-kernel/perf/perf.bb| 1 +
>>  2 files changed, 1 insertion(+), 1 deletion(-)
> 
> The fact this works suggests perf is ignoring TARGET_CFLAGS. Is there
> anything in the perf build system where we should be passing in cflags
> we really want used?

Well, IIUC, the normal way TARGET_CFLAGS would take effect is via the
'export CFLAGS = "${TARGET_CFLAGS}'. But in the perf recipe, both
do_compile and do_install start with

# Linux kernel build system is expected to do the right thing
unset CFLAGS

And perf's build system does indeed add lots of flags of its own, at
least for most TUs, but nowhere is any -f(macro/debug/file)-prefix being
set.

So I do think that TARGET_CC_ARCH is the best place for flags that we
really want used. At least as an initial step, because this is known to
work when using the security_flags.inc file. Maybe there's some cleaner
way where we don't unset CFLAGS, but that could be a giant can of worms.

And yes, a comment should be added. Is something like

# Perf's build system adds its own optimization flags for most TUs,
# overriding the flags included here. But for some, perf does not add
# any -O option, so ensure the distro's chosen optimization gets used
# for those. Since ${SELECTED_OPTIMIZATION} always includes
# ${DEBUG_FLAGS} which in turn includes ${DEBUG_PREFIX_MAP}, this also
# ensures perf is built with appropriate -f*-prefix-map options,
# avoiding the 'buildpaths' QA warning.

too verbose?

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189508): 
https://lists.openembedded.org/g/openembedded-core/message/189508
Mute This Topic: https://lists.openembedded.org/mt/102058904/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] perf: lift TARGET_CC_ARCH modification out of security_flags.inc

2023-10-19 Thread Rasmus Villemoes via lists.openembedded.org
From: Rasmus Villemoes 

Building perf without security_flags.inc being included in one's
distro results in the buildpaths warning

WARNING: perf-1.0-r9 do_package_qa: QA Issue: File /usr/bin/trace in
package perf contains reference to TMPDIR

because the ${DEBUG_PREFIX_MAP} does not get used. Most recipes get
that from CFLAGS, but the perf recipe explicitly unsets that.

Now ${SELECTED_OPTIMIZATION} of course contains more than just
${DEBUG_FLAGS}/${DEBUG_PREFIX_MAP}. For most TUs, perf's build system
adds its own optimization flags (-O6 for odd reasons), so for those
including the -O2 or -Og doesn't change anything. But looking at the
.o.cmd files show that there are some TUs which currently get built
without any -O flag. So for those adding the distro's
SELECTED_OPTIMIZATION seem to be the right thing to do.

Signed-off-by: Rasmus Villemoes 
---
 meta/conf/distro/include/security_flags.inc | 1 -
 meta/recipes-kernel/perf/perf.bb| 1 +
 2 files changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/distro/include/security_flags.inc 
b/meta/conf/distro/include/security_flags.inc
index 2972f05b4e..d97a6edb0f 100644
--- a/meta/conf/distro/include/security_flags.inc
+++ b/meta/conf/distro/include/security_flags.inc
@@ -69,4 +69,3 @@ SECURITY_LDFLAGS:pn-xserver-xorg = "${SECURITY_X_LDFLAGS}"
 TARGET_CC_ARCH:append:pn-binutils = " ${SELECTED_OPTIMIZATION}"
 TARGET_CC_ARCH:append:pn-gcc = " ${SELECTED_OPTIMIZATION}"
 TARGET_CC_ARCH:append:pn-gdb = " ${SELECTED_OPTIMIZATION}"
-TARGET_CC_ARCH:append:pn-perf = " ${SELECTED_OPTIMIZATION}"
diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 675acfaf26..a6110dedc9 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -72,6 +72,7 @@ SPDX_S = "${S}/tools/perf"
 # symbols and this is preferred than requiring patches to every old
 # supported kernel.
 LDFLAGS="-ldl -lutil"
+TARGET_CC_ARCH += "${SELECTED_OPTIMIZATION}"
 
 EXTRA_OEMAKE = '\
 V=1 \
-- 
2.40.1.1.g1c60b9335d


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189451): 
https://lists.openembedded.org/g/openembedded-core/message/189451
Mute This Topic: https://lists.openembedded.org/mt/102058904/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH 04/15] perf: fix buildpaths QA warning in 6.4+

2023-10-10 Thread Rasmus Villemoes via lists.openembedded.org
On 10/10/2023 14.31, Bruce Ashfield wrote:
> 
> 
> On Tue, Oct 10, 2023 at 7:44 AM Rasmus Villemoes
> It seems we can fix it with
> 
> $ git diff
> diff --git a/meta/recipes-kernel/perf/perf.bb <http://perf.bb>
> b/meta/recipes-kernel/perf/perf.bb <http://perf.bb>
> index 420286e..59c0c10 100644
> --- a/meta/recipes-kernel/perf/perf.bb <http://perf.bb>
> +++ b/meta/recipes-kernel/perf/perf.bb <http://perf.bb>
> @@ -81,7 +81,7 @@ EXTRA_OEMAKE = '\
>      LDSHARED="${CC} -shared" \
>      AR="${AR}" \
>      LD="${LD}" \
> -    EXTRA_CFLAGS="-ldw -I${S}" \
> +    EXTRA_CFLAGS="-ldw -I${S} ${DEBUG_PREFIX_MAP}" \
>      YFLAGS='-y
> --file-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}' 
> \
>      EXTRA_LDFLAGS="${PERF_EXTRA_LDFLAGS}" \
>      perfexecdir=${libexecdir} \
> 
> but I'm quite curious what the difference between our setups is, since
> apparently the only problem oe-core has/had was that new file introduced
> in v6.4.
> 
> 
> You can see any patches that we have against perf in the kernel-cache
> repository (https://git.yoctoproject.org/yocto-kernel-cache/
> <https://git.yoctoproject.org/yocto-kernel-cache/>). That being
> said, we unfortunately cannot really patch perf, since the recipe is
> supported against more than our reference kernels.

I didn't suggest to patch perf, I suggested to patch the perf recipe.
> 
> Other than that, there's no special setup (in particular no setup that
> I have) as it is the autobuilder infrastructure that typically finds and
> reports these issues.

So I found the difference, and those autobuilder builds do in fact make
the compiler use all the right -f*-prefix-map options. It's because
meta/conf/distro/include/security_flags.inc contains

TARGET_CC_ARCH:append:pn-perf = " ${SELECTED_OPTIMIZATION}"

and that includes the DEBUG_PREFIX_MAP setting, and in turn gets
included in the CC setting via HOST_CC_ARCH. And of course the poky
distro includes that security_flags.inc file, while our custom distro
does not (up until five minutes ago, I never knew that file existed).

Of course one wonders why those :append:pn-* settings that don't
actually append any security related flags, but do affect stuff like the
recipes passing QA, are hidden away in a file called security_flags.inc
instead of those settings just being directly in the recipes themselves.
There's no explanation in oe-core e93765ffb57 ; it removed a whole bunch
of append_pn-* stuff which had apparently become redundant (the commit
log hints as much), but doesn't offer any explanation as to why pn-perf
should grow that setting. And AFAICT that long predates the invention of
the -f*-prefix-map flags, so it seems to be mostly by accident that
perf, with the poky distro, actually does build with those flags.

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188910): 
https://lists.openembedded.org/g/openembedded-core/message/188910
Mute This Topic: https://lists.openembedded.org/mt/100051819/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH 04/15] perf: fix buildpaths QA warning in 6.4+

2023-10-10 Thread Rasmus Villemoes via lists.openembedded.org
On 10/07/2023 05.20, Bruce Ashfield via lists.openembedded.org wrote:
> From: Bruce Ashfield 
> 
> kernel version 6.4 introduces a new file that need to have
> absolute paths removed, so we can avoid the buildpaths QA
> warning and have relocatable packages.
> 
> We add pmu-flex.h to the processing, and the issue is resolved.

Hi Bruce

We've seen those buildpaths qa warning when building perf for a long
time, and they happen whether we build against a 5.15, 6.2 or 6.5
kernel. More specifically, what we see is

WARNING: perf-1.0-r9 do_package_qa: QA Issue: File /usr/bin/.debug/perf
in package perf-dbg contains reference to TMPDIR [buildpaths]
WARNING: perf-1.0-r9 do_package_qa: QA Issue: File /usr/bin/trace in
package perf contains reference to TMPDIR
File /usr/bin/perf in package perf contains reference to TMPDIR [buildpaths]

Looking at the .o.cmd files, there is indeed nowhere a
-fmacro-prefix-map or -fdebug-prefix-map, and apart from the bison/yacc
stuff, I also don't see where the perf recipe attempts to pass such flags.

It seems we can fix it with

$ git diff
diff --git a/meta/recipes-kernel/perf/perf.bb
b/meta/recipes-kernel/perf/perf.bb
index 420286e..59c0c10 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -81,7 +81,7 @@ EXTRA_OEMAKE = '\
 LDSHARED="${CC} -shared" \
 AR="${AR}" \
 LD="${LD}" \
-EXTRA_CFLAGS="-ldw -I${S}" \
+EXTRA_CFLAGS="-ldw -I${S} ${DEBUG_PREFIX_MAP}" \
 YFLAGS='-y
--file-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}' \
 EXTRA_LDFLAGS="${PERF_EXTRA_LDFLAGS}" \
 perfexecdir=${libexecdir} \

but I'm quite curious what the difference between our setups is, since
apparently the only problem oe-core has/had was that new file introduced
in v6.4.

FWIW, what we build are vanilla -stable kernels, i.e. "6.2" means
v6.2.16 as tagged by Greg. The only modification of the perf recipe we
do is removing "scripting" from PACKAGECONFIG.

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#11): 
https://lists.openembedded.org/g/openembedded-core/message/11
Mute This Topic: https://lists.openembedded.org/mt/100051819/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] openssh: update sshd_check_keys script to make use of 'sshd -G'

2023-09-29 Thread Rasmus Villemoes via lists.openembedded.org
From: Rasmus Villemoes 

Parsing sshd's config file with 'sed' does not work in for example the
case where somebody has made use of the new ability to add a config
fragment in /etc/ssh/sshd_config.d/ with one or more HostKey
stanzas. Also, sshd_config keywords are case-insensitive, but the
current sed pattern only matches the CamelCase spelling of HostKey.

In openssh 9.3, sshd learnt a new command line flag '-G', which causes
sshd to parse the given configuration file and print the resulting
effective configuration on stdout. So use that instead.

Furthermore, since that "effective configuration" includes the default
set of host keys if the configuration file has no HostKey stanzas, we
also avoid the script needing to know what sshd's default is - that
could plausibly change with some future release.

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-connectivity/openssh/openssh/sshd_check_keys | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/recipes-connectivity/openssh/openssh/sshd_check_keys 
b/meta/recipes-connectivity/openssh/openssh/sshd_check_keys
index ef117de897..606d1894b5 100644
--- a/meta/recipes-connectivity/openssh/openssh/sshd_check_keys
+++ b/meta/recipes-connectivity/openssh/openssh/sshd_check_keys
@@ -57,8 +57,7 @@ while true ; do
 esac
 done
 
-HOST_KEYS=$(sed -n 's/^[ \t]*HostKey[ \t]\+\(.*\)/\1/p' "${sshd_config}")
-[ -z "${HOST_KEYS}" ] && HOST_KEYS="$SYSCONFDIR/ssh_host_rsa_key 
$SYSCONFDIR/ssh_host_ecdsa_key $SYSCONFDIR/ssh_host_ed25519_key"
+HOST_KEYS=$(sshd -G -f "${sshd_config}" | grep -i '^hostkey ' | cut -f2 -d' ')
 
 for key in ${HOST_KEYS} ; do
 [ -f $key ] && continue
-- 
2.40.1.1.g1c60b9335d


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188396): 
https://lists.openembedded.org/g/openembedded-core/message/188396
Mute This Topic: https://lists.openembedded.org/mt/101653899/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] Dilemma on changes - merge or not to merge (e.g. 6.4)

2023-08-16 Thread Rasmus Villemoes via lists.openembedded.org
On 15/08/2023 15.08, Paul Gortmaker via lists.openembedded.org wrote:
> [Dilemma on changes - merge or not to merge (e.g. 6.4)] On 14/08/2023 (Mon 
> 10:54) Richard Purdie wrote:
> 
>> Remaining are:
>>   * an error upon boot on preempt-rt on qemux86-64
>>  (e.g. 
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/7616/steps/36/logs/stdio)
>>  We'll probably just have to ignore it in parselogs as it has been??
>>  around for a while and nobody seems interested in fixing it upstream.
> 
> Just back from vacation and I see an internal report of 10-ish at boot

it seems to be rate-limited to 10 per boot, so it should never appear
more than those 10ish times:

static bool report_idle_softirq(void)
{
...
if (ratelimit >= 10)
return false;
...
ratelimit++;
...
}

(it's all non-atomic/lockfree, so ofc it could just happen to get
emitted 11 or 12 times if the stars align just right...)

> 
>   NOHZ tick-stop error: local softirq work is pending, handler #80!!!
> 
> ..on the 6.1.43-rt10-yocto-preempt-rt kernel, on real hardware.  So it
> seems we can't blame that one entirely on v6.4 kernel (or qemu).
> 
> We used to get (late 3.x and 4.x era) pretty common "NOHZ: local softirq
> pending" messages even on common/popular distro kernels.  But I haven't
> seen those for a long time and they didn't scream "error" or have the
> alarmist three exclamation marks either.

FWIW, we're also seeing exactly that "NOHZ tick-stop error" message on
6.4.6-rt8 running on a couple of different imx8mp based boards.

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#186117): 
https://lists.openembedded.org/g/openembedded-core/message/186117
Mute This Topic: https://lists.openembedded.org/mt/100733646/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [RFC] [PATCH] Allow fitimage + initramfs rebuild to be accelerated

2022-11-11 Thread Rasmus Villemoes via lists.openembedded.org
On 10/11/2022 21.32, Paul Eggleton via lists.openembedded.org wrote:
> Hi Richard
> 
> On Thursday, 10 November 2022 11:14:24 NZDT Richard Purdie wrote:

>> Adding in yet further if/else paths with magic variables to control
>> them isn't filling me with confidence.
> 
> I understand that. I was hoping to figure out a way to avoid adding 
> significant 
> extra complexity.
>

FWIW, we've long since given up on using kernel-fitimage.bbclass, it is
way to inflexible (e.g. there's no way to build two FIT images with the
same kernel but different initramfs).

It is much simpler to just have a separate small kernel-fit.bb recipe
which pulls vmlinux or Image or whatnot, plus the .dtbs, from
DEPLOY_DIR_IMAGE along with the initramfs images, provide our own .its
file(s), and build the FIT image(s) there. That automatically avoids
rebuilding the kernel when only something in initramfs has changed.

Building the kernel and packaging the binary into some container format
really doesn't need to be done in the same recipe.

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173128): 
https://lists.openembedded.org/g/openembedded-core/message/173128
Mute This Topic: https://lists.openembedded.org/mt/94747626/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] bitbake.conf: set BB_DEFAULT_UMASK using ??=

2022-08-26 Thread Rasmus Villemoes via lists.openembedded.org
Currently, there's no way for the user's site.conf, local.conf or
similar to set BB_DEFAULT_UMASK, because those files are included by
bitbake.conf prior to the unconditional assignment of
BB_DEFAULT_UMASK. To make that possible, use a weak default assignment
instead. This is also consistent with most other variable assignments
in the lower half of bitbake.conf.

I believe the risk of a regression is very small; it would require
something like somebody having a definition of BB_DEFAULT_UMASK in a
local configuration file, and having been relying on that _not_ taking
effect.

Signed-off-by: Rasmus Villemoes 
---
 meta/conf/bitbake.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index dd2df8a552..52a36d788b 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -928,7 +928,7 @@ SHELL[unexport] = "1"
 TRANSLATED_TARGET_ARCH ??= "${@d.getVar('TARGET_ARCH').replace("_", "-")}"
 
 # Set a default umask to use for tasks for determinism
-BB_DEFAULT_UMASK = "022"
+BB_DEFAULT_UMASK ??= "022"
 
 # Complete output from bitbake
 BB_CONSOLELOG ?= "${LOG_DIR}/cooker/${MACHINE}/${DATETIME}.log"
-- 
2.37.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169915): 
https://lists.openembedded.org/g/openembedded-core/message/169915
Mute This Topic: https://lists.openembedded.org/mt/93268429/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] Should BB_DEFAULT_UMASK be set with ?=

2022-08-24 Thread Rasmus Villemoes via lists.openembedded.org
On 30/05/2022 11.19, Rasmus Villemoes via lists.openembedded.org wrote:
> Hi,
> 
> Suppose I want to define BB_DEFAULT_UMASK in site.conf or local.conf (I
> do have a use case for setting it to 002). But bitbake.conf includes
> those files before it then goes on to use an unconditional assignment
> 
> BB_DEFAULT_UMASK = "022"
> 
> So how/where is one supposed to be able to set BB_DEFAULT_UMASK? Should
> the assignment in bitbake.conf be ?= or perhaps ??= instead? Or should
> the assigment instead be moved above the inclusion of the user's
> configuration files?

ping (and cc += RP)

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169815): 
https://lists.openembedded.org/g/openembedded-core/message/169815
Mute This Topic: https://lists.openembedded.org/mt/93226320/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] e2fsprogs: add alternatives handling of lsattr as well

2022-06-09 Thread Rasmus Villemoes via lists.openembedded.org
On 09/06/2022 09.20, Luca Ceresoli wrote:
> Hi Rasmus,
> 
> On Wed,  8 Jun 2022 15:12:05 +0200
> "Rasmus Villemoes via lists.openembedded.org"
>  wrote:
>  ^^^
> 
> As you can see above, your sender address is getting mangled. This is
> not your fault, but it makes applying your patches annoying.
> 
> Can you please try to work around that by setting the sendemail.from
> parameter in your git config?

Sorry about that, didn't know it.

It's not completely clear if I should put just the email address or the
full "Name " in that configuration item, i.e. whether it's

[sendemail]
from = rasmus.villem...@prevas.dk

or

[sendemail]
from = Rasmus Villemoes 

For now I've gone with the latter, please let me know if that's correct.
In any case we'll see if it works the next time I submit a patch.

Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166769): 
https://lists.openembedded.org/g/openembedded-core/message/166769
Mute This Topic: https://lists.openembedded.org/mt/91622646/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] e2fsprogs: add alternatives handling of lsattr as well

2022-06-08 Thread Rasmus Villemoes via lists.openembedded.org
Building busybox with CONFIG_LSATTR=y and installing that in the same
filesystem as e2fsprogs breaks:

  ERROR: ... do_rootfs: Postinstall scriptlets of ['busybox'] have failed. If 
the intention is to defer them to first boot,
  then please place them into pkg_postinst_ontarget:${PN} ().
  Deferring to first boot via 'exit 1' is no longer supported.

Fix that by also alternatifying lsattr just as chattr already is.

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-devtools/e2fsprogs/e2fsprogs_1.46.5.bb | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.46.5.bb 
b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.46.5.bb
index ec48f419c7a..5b2d1921f08 100644
--- a/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.46.5.bb
+++ b/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.46.5.bb
@@ -51,6 +51,7 @@ do_install () {
oe_multilib_header ext2fs/ext2_types.h
install -d ${D}${base_bindir}
mv ${D}${bindir}/chattr ${D}${base_bindir}/chattr.e2fsprogs
+   mv ${D}${bindir}/lsattr ${D}${base_bindir}/lsattr.e2fsprogs
 
install -v -m 755 ${S}/contrib/populate-extfs.sh ${D}${base_sbindir}/
 
@@ -99,10 +100,12 @@ FILES:libe2p = "${base_libdir}/libe2p.so.*"
 FILES:libext2fs = "${libdir}/e2initrd_helper ${base_libdir}/libext2fs.so.*"
 FILES:${PN}-dev += "${datadir}/*/*.awk ${datadir}/*/*.sed ${base_libdir}/*.so 
${bindir}/compile_et ${bindir}/mk_cmds"
 
-ALTERNATIVE:${PN} = "chattr"
+ALTERNATIVE:${PN} = "chattr lsattr"
 ALTERNATIVE_PRIORITY = "100"
 ALTERNATIVE_LINK_NAME[chattr] = "${base_bindir}/chattr"
 ALTERNATIVE_TARGET[chattr] = "${base_bindir}/chattr.e2fsprogs"
+ALTERNATIVE_LINK_NAME[lsattr] = "${base_bindir}/lsattr"
+ALTERNATIVE_TARGET[lsattr] = "${base_bindir}/lsattr.e2fsprogs"
 
 ALTERNATIVE:${PN}-doc = "fsck.8"
 ALTERNATIVE_LINK_NAME[fsck.8] = "${mandir}/man8/fsck.8"
-- 
2.31.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166707): 
https://lists.openembedded.org/g/openembedded-core/message/166707
Mute This Topic: https://lists.openembedded.org/mt/91622646/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] vim: put xxd in its own package

2022-06-08 Thread Rasmus Villemoes via lists.openembedded.org
On 08/06/2022 14.30, Paulo Neves wrote:
> On 6/8/22 14:13, Rasmus Villemoes wrote:
> 
>> On 08/06/2022 13.58, Paulo Neves via lists.openembedded.org wrote:
> I mean that your patch allows for a new standalone package ${PN}-xxd.
> Were somebody to use this package standalone, meaning without installing vim,
> the ncurses-terminfo-base would not be installed, possibly breaking xxd.
> 
> Before your patch this risk did not occur due to xxd being include in the 
> ${PN}
> which had an RDEPENDS to ncurses-terminfo-base.

Thanks, now I understand what you were saying. And no, xxd does not in
any way depend on those terminfo files; it's a simple utility for
converting binary files to hex (or the reverse, which is what I happen
to need, taking input in the form of ascii hex chars and producing
binary output). Yes, I've tested it on target as well.

Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166705): 
https://lists.openembedded.org/g/openembedded-core/message/166705
Mute This Topic: https://lists.openembedded.org/mt/91618936/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] vim: put xxd in its own package

2022-06-08 Thread Rasmus Villemoes via lists.openembedded.org
On 08/06/2022 13.58, Paulo Neves via lists.openembedded.org wrote:
> Forgive me if it is a stupid question, but does xxd not rdepend on
> ncurses-terminfo-base as well?

No, it does not, it's a trivial standalone utility whose only dynamic
dependency is libc.

> I ask because before your patch it could
> was implied it that it did, s part of PN.

I don't understand what you're trying to say. I just put the extra
RDEPENDS from vim to vim-xxd into the existing RDEPENDS:vim line. That
doesn't imply any dependency either way between the two items listed on
the RHS.

Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166702): 
https://lists.openembedded.org/g/openembedded-core/message/166702
Mute This Topic: https://lists.openembedded.org/mt/91618936/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] vim: put xxd in its own package

2022-06-08 Thread Rasmus Villemoes via lists.openembedded.org
The xxd tool can be quite handy by itself, and doesn't have anything
to do with vim per se. Make it possible to include the rather tiny xxd
in a rootfs without pulling in the several MB vim binary and
associated data.

For backwards compatibility, add an RDEPENDS from the main package to
the new vim-xxd package.

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-support/vim/vim_8.2.bb | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-support/vim/vim_8.2.bb 
b/meta/recipes-support/vim/vim_8.2.bb
index f358e61132b..fee9f055e9a 100644
--- a/meta/recipes-support/vim/vim_8.2.bb
+++ b/meta/recipes-support/vim/vim_8.2.bb
@@ -2,14 +2,18 @@ require vim.inc
 
 PROVIDES = "xxd"
 
-RDEPENDS:${PN} = "ncurses-terminfo-base"
+RDEPENDS:${PN} = "ncurses-terminfo-base ${PN}-xxd"
 # Recommend that runtime data is installed along with vim
 RRECOMMENDS:${PN} = "${PN}-syntax ${PN}-help ${PN}-tutor ${PN}-vimrc 
${PN}-common"
 
 PACKAGECONFIG:class-native = ""
 BBCLASSEXTEND = "native nativesdk"
 
-ALTERNATIVE:${PN}:append = " xxd"
+PACKAGES =+ "${PN}-xxd"
+FILES:${PN}-xxd = "${bindir}/xxd"
+RPROVIDES:${PN}-xxd = "xxd"
+
+ALTERNATIVE:${PN}-xxd = "xxd"
 ALTERNATIVE_TARGET[xxd] = "${bindir}/xxd"
 ALTERNATIVE_LINK_NAME[xxd] = "${bindir}/xxd"
 
-- 
2.31.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166693): 
https://lists.openembedded.org/g/openembedded-core/message/166693
Mute This Topic: https://lists.openembedded.org/mt/91618936/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] Should BB_DEFAULT_UMASK be set with ?=

2022-05-30 Thread Rasmus Villemoes via lists.openembedded.org
Hi,

Suppose I want to define BB_DEFAULT_UMASK in site.conf or local.conf (I
do have a use case for setting it to 002). But bitbake.conf includes
those files before it then goes on to use an unconditional assignment

BB_DEFAULT_UMASK = "022"

So how/where is one supposed to be able to set BB_DEFAULT_UMASK? Should
the assignment in bitbake.conf be ?= or perhaps ??= instead? Or should
the assigment instead be moved above the inclusion of the user's
configuration files?

Rasmus


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166274): 
https://lists.openembedded.org/g/openembedded-core/message/166274
Mute This Topic: https://lists.openembedded.org/mt/91427323/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] git: make expat and curl into PACKAGECONFIG items

2022-03-30 Thread Rasmus Villemoes via lists.openembedded.org
It can be useful to use git on target (e.g. with some wrapper like
etckeeper for keeping track of changes to /etc), and for such cases,
it is likely one has no need for pulling from/pushing to http[s]
repositories. From the INSTALL file:

- "libcurl" library ... If you do not use http:// or https://
  repositories, and do not want to put patches into an IMAP
  mailbox, you do not have to have them (use NO_CURL).

- "expat" library; git-http-push uses it for remote lock
  management over DAV.  Similar to "curl" above, this is
  optional (with NO_EXPAT).

Setting --without-expat and --without-curl reduces the size of the
installed "git" package from 18M to 12M, in addition to avoiding
pulling those libraries into the rootfs.

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-devtools/git/git_2.35.1.bb | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/git/git_2.35.1.bb 
b/meta/recipes-devtools/git/git_2.35.1.bb
index 0cff3ce1001..47c22118640 100644
--- a/meta/recipes-devtools/git/git_2.35.1.bb
+++ b/meta/recipes-devtools/git/git_2.35.1.bb
@@ -3,7 +3,7 @@ HOMEPAGE = "http://git-scm.com;
 DESCRIPTION = "Git is a free and open source distributed version control 
system designed to handle everything from small to very large projects with 
speed and efficiency."
 SECTION = "console/utils"
 LICENSE = "GPL-2.0-only"
-DEPENDS = "openssl curl zlib expat"
+DEPENDS = "openssl zlib"
 
 PROVIDES:append:class-native = " git-replacement-native"
 
@@ -18,10 +18,12 @@ LIC_FILES_CHKSUM = 
"file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1"
 
 CVE_PRODUCT = "git-scm:git"
 
-PACKAGECONFIG ??= ""
+PACKAGECONFIG ??= "expat curl"
 PACKAGECONFIG[cvsserver] = ""
 PACKAGECONFIG[svn] = ""
 PACKAGECONFIG[manpages] = ",,asciidoc-native xmlto-native"
+PACKAGECONFIG[curl] = "--with-curl,--without-curl,curl"
+PACKAGECONFIG[expat] = "--with-expat,--without-expat,expat"
 
 EXTRA_OECONF = "--with-perl=${STAGING_BINDIR_NATIVE}/perl-native/perl \
--without-tcltk \
-- 
2.31.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#163781): 
https://lists.openembedded.org/g/openembedded-core/message/163781
Mute This Topic: https://lists.openembedded.org/mt/90128519/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] bitbake.conf: add BB_NUMBER_THREADS to BB_HASHEXCLUDE_COMMON

2022-02-28 Thread Rasmus Villemoes via lists.openembedded.org
On 28/02/2022 14.41, Richard Purdie wrote:
> On Mon, 2022-02-28 at 09:42 +0100, Rasmus Villemoes wrote:
>> The imx-gpu-sdk recipe in the meta-imx layer references
>> ${BB_NUMBER_THREADS} in its do_compile function. Changing
>> BB_NUMBER_THREADS between bitbake invocations leads to the well-known
>>
>>   When reparsing 
>> ...meta-imx/meta-sdk/recipes-graphics/imx-gpu-sdk/imx-gpu-sdk_5.8.0.bb:do_compile,
>>  the basehash value changed from 
>> 69be88cf220840ff2203e11cfe65681880b0bf9b88db67d50c1ba772b883bd18 to 
>> 5e6d5029fac8d7856ada4c2eca359568298f82cdb64567d7dd4deda503d9f83a. The 
>> metadata is not deterministic and this needs to be fixed.
>>
>> And I'm not the first to hit this problem with that recipe:
>> https://community.nxp.com/t5/i-MX-Processors/imx-gpu-sdk-compile-error-IMX8MP-IMX8MM-IMX8MQ/td-p/1217864
>>
>> This happens because BB_NUMBER_THREADS is in BB_HASHCONFIG_IGNORE_VARS,
>> so changing it does not cause the recipe to be reparsed, but it is not
>> included in BB_HASHEXCLUDE_COMMON and thus
>> BB_BASEHASH_IGNORE_VARS. This is inconsistent with and in contrast to
>> both PARALLEL_MAKE and OMP_NUM_THREADS, the latter of which even has
>> ${BB_NUMBER_THREADS} as default value.
> 
> Technically imx-gpu-sdk is incorrect. BB_NUMBER_THREADS is the number of tasks
> bitbake should run. PARALLEL_MAKE is what is used for parallelism in 
> do_compile.
> 
> I appreciate that has -j in but you can use: ${@oe.utils.parallel_make(d)} to
> obtain the value that recipe needs.

Oh, I'm not saying that that recipe isn't buggy, but is still seems
awfully user-unfriendly and inconsistent to treat BB_NUMBER_THREADS
different from PARALLEL_MAKE wrt. hash computations.

But now I've raised the issue, and I don't care much what happens next;
I now know the root cause of the error I saw and rarely build using
meta-imx in the first place.

Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162498): 
https://lists.openembedded.org/g/openembedded-core/message/162498
Mute This Topic: https://lists.openembedded.org/mt/89446852/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] bitbake.conf: add BB_NUMBER_THREADS to BB_HASHEXCLUDE_COMMON

2022-02-28 Thread Rasmus Villemoes via lists.openembedded.org
The imx-gpu-sdk recipe in the meta-imx layer references
${BB_NUMBER_THREADS} in its do_compile function. Changing
BB_NUMBER_THREADS between bitbake invocations leads to the well-known

  When reparsing 
...meta-imx/meta-sdk/recipes-graphics/imx-gpu-sdk/imx-gpu-sdk_5.8.0.bb:do_compile,
 the basehash value changed from 
69be88cf220840ff2203e11cfe65681880b0bf9b88db67d50c1ba772b883bd18 to 
5e6d5029fac8d7856ada4c2eca359568298f82cdb64567d7dd4deda503d9f83a. The metadata 
is not deterministic and this needs to be fixed.

And I'm not the first to hit this problem with that recipe:
https://community.nxp.com/t5/i-MX-Processors/imx-gpu-sdk-compile-error-IMX8MP-IMX8MM-IMX8MQ/td-p/1217864

This happens because BB_NUMBER_THREADS is in BB_HASHCONFIG_IGNORE_VARS,
so changing it does not cause the recipe to be reparsed, but it is not
included in BB_HASHEXCLUDE_COMMON and thus
BB_BASEHASH_IGNORE_VARS. This is inconsistent with and in contrast to
both PARALLEL_MAKE and OMP_NUM_THREADS, the latter of which even has
${BB_NUMBER_THREADS} as default value.

Signed-off-by: Rasmus Villemoes 
---
 meta/conf/bitbake.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 6fb7bfeb23c..6bc18016056 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -942,7 +942,7 @@ BB_HASHEXCLUDE_COMMON ?= "TMPDIR FILE PATH PWD BB_TASKHASH 
BBPATH BBSERVER DL_DI
 BB_WORKERCONTEXT BB_LIMITEDDEPS BB_UNIHASH extend_recipe_sysroot 
DEPLOY_DIR \
 SSTATE_HASHEQUIV_METHOD SSTATE_HASHEQUIV_REPORT_TASKDATA \
 SSTATE_HASHEQUIV_OWNER CCACHE_TOP_DIR BB_HASHSERVE GIT_CEILING_DIRECTORIES 
\
-OMP_NUM_THREADS BB_CURRENTTASK"
+OMP_NUM_THREADS BB_CURRENTTASK BB_NUMBER_THREADS"
 BB_BASEHASH_IGNORE_VARS ?= "${BB_HASHEXCLUDE_COMMON} PSEUDO_IGNORE_PATHS 
BUILDHISTORY_DIR \
 SSTATE_DIR SOURCE_DATE_EPOCH"
 BB_HASHCONFIG_IGNORE_VARS ?= "${BB_HASHEXCLUDE_COMMON} DATE TIME SSH_AGENT_PID 
\
-- 
2.31.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162452): 
https://lists.openembedded.org/g/openembedded-core/message/162452
Mute This Topic: https://lists.openembedded.org/mt/89446852/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] kernel.bbclass: remove unnecessary dead code

2021-09-23 Thread Rasmus Villemoes via lists.openembedded.org
The grep pattern seems to have been wrong ever since we stopped adding
the -ffile-prefix-map via a patch in commit 20aea61385e, because the
actual upstream gcc produces

  -ffile-prefix-map==

and not

  -ffile-prefix-map=

Besides, these *-prefix-map options are already used when building the
kernel due to the setting of KERNEL_CC, and the kernel build system
itself also probes for and uses at least -fmacro-prefix-map (hence
taking care of __FILE__ etc., but not necessarily things that go in
.debug_info sections).

Signed-off-by: Rasmus Villemoes 
---
 meta/classes/kernel.bbclass | 15 ++-
 1 file changed, 2 insertions(+), 13 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index d13c38fb02..4acec1877e 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -322,15 +322,6 @@ python do_devshell:prepend () {
 
 addtask bundle_initramfs after do_install before do_deploy
 
-get_cc_option () {
-   # Check if KERNEL_CC supports the option "file-prefix-map".
-   # This option allows us to build images with __FILE__ values 
that do not
-   # contain the host build path.
-   if ${KERNEL_CC} -Q --help=joined | grep -q 
"\-ffile-prefix-map="; then
-   echo "-ffile-prefix-map=${S}=/kernel-source/"
-   fi
-}
-
 kernel_do_compile() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
@@ -361,9 +352,8 @@ kernel_do_compile() {
copy_initramfs

use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE_NAME}.cpio
fi
-   cc_extra=$(get_cc_option)
for typeformake in ${KERNEL_IMAGETYPE_FOR_MAKE} ; do
-   oe_runmake ${typeformake} CC="${KERNEL_CC} $cc_extra " 
LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
+   oe_runmake ${typeformake} CC="${KERNEL_CC}" LD="${KERNEL_LD}" 
${KERNEL_EXTRA_ARGS} $use_alternate_initrd
done
# vmlinux.gz is not built by kernel
if (echo "${KERNEL_IMAGETYPES}" | grep -wq "vmlinux\.gz"); then
@@ -390,8 +380,7 @@ do_compile_kernelmodules() {
bbnote "KBUILD_BUILD_TIMESTAMP: $ts"
fi
if (grep -q -i -e '^CONFIG_MODULES=y$' ${B}/.config); then
-   cc_extra=$(get_cc_option)
-   oe_runmake -C ${B} ${PARALLEL_MAKE} modules CC="${KERNEL_CC} 
$cc_extra " LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
+   oe_runmake -C ${B} ${PARALLEL_MAKE} modules CC="${KERNEL_CC}" 
LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
 
# Module.symvers gets updated during the 
# building of the kernel modules. We need to
-- 
2.31.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156260): 
https://lists.openembedded.org/g/openembedded-core/message/156260
Mute This Topic: https://lists.openembedded.org/mt/85810078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] RRECOMMENDS "masking" unsatisfiable IMAGE_INSTALL

2021-07-26 Thread Rasmus Villemoes via lists.openembedded.org
On 06/07/2021 21.39, Alex Stewart wrote:
> Hey Rasmus,
> 
> Sorry for the delay; I was OOO for the holidays and I'm just now working
> through my inbox.

No need to apologize; as it happens I've just now returned from vacation.

> On 6/28/21 4:17 AM, Rasmus Villemoes wrote:
>> On 25/06/2021 18.13, Richard Purdie wrote:
>>> That is probably a bug that needs opening against opkg in bugzilla then.
>>> Added Alex to Cc: (opkg maintainer).
>> Indeed, thanks. Alex, do you have enough context from the above, or do I
>> need to do anything explicitly to file a bug?
>>
>> Rasmus
> 
> Could you expand a bit on how OE is supposed to fail when the kernel
> module package is unsatisfiable? Does it propagate a opkg satisfaction
> error, or is the misconfiguration caught in the OE python modules?
> 
> I ask because it seems most likely to me that the inconsistency here is
> in the OE opkg interface modules, rather than opkg itself - which
> obviously has no concept of IMAGE_INSTALL or the kernel configuration.
> 
> Can you identify the set of packages that OE is requesting opkg to
> install, with and without the RRECOMMENDS? 

Looking at log.do_rootfs, it seems that opkg gets invoked in exactly the
same way with and without the RRECOMMENDS in play:

With RRECOMMENDS, where the build succeeds:

NOTE: Installing the following packages: [...] glib-2.0 iproute2
kernel-image kernel-module-abcd less libgettextlib [...]
NOTE: [...]/recipe-sysroot-native/usr/bin/opkg --volatile-cache -f
[...]/work/rpi3-oe-linux/pil-rootfs/1.0-r0/opkg.conf -t
[...]/work/rpi3-oe-linux/pil-rootfs/1.0-r0/temp/ipktemp/ -o
[...]/work/rpi3-oe-linux/pil-rootfs/1.0-r0/rootfs  --force_postinstall
--prefer-arch-to-version   --add-ignore-recommends busybox-syslog
--add-ignore-recommends busybox-udhcpc --add-ignore-recommends
kernel-vmlinux --add-ignore-recommends udev-cache install [...] glib-2.0
iproute2 kernel-image kernel-module-abcd less libgettextlib [...]

And no further mention of kernel-module-abcd.

Without the RRECOMMENDS:

NOTE: Installing the following packages: [...] glib-2.0 iproute2
kernel-image kernel-module-abcd less libgettextlib [...]
NOTE: [...]/recipe-sysroot-native/usr/bin/opkg --volatile-cache -f
[...]/work/rpi3-oe-linux/pil-rootfs/1.0-r0/opkg.conf -t
[...]/work/rpi3-oe-linux/pil-rootfs/1.0-r0/temp/ipktemp/ -o
[...]/work/rpi3-oe-linux/pil-rootfs/1.0-r0/rootfs  --force_postinstall
--prefer-arch-to-version   --add-ignore-recommends busybox-syslog
--add-ignore-recommends busybox-udhcpc --add-ignore-recommends
kernel-vmlinux --add-ignore-recommends udev-cache install [...] glib-2.0
iproute2 kernel-image kernel-module-abcd less libgettextlib [...]
ERROR: Unable to install packages. Command
'[...]/recipe-sysroot-native/usr/bin/opkg --volatile-cache -f
[...]/work/rpi3-oe-linux/pil-rootfs/1.0-r0/opkg.conf -t
[...]/work/rpi3-oe-linux/pil-rootfs/1.0-r0/temp/ipktemp/ -o
[...]/work/rpi3-oe-linux/pil-rootfs/1.0-r0/rootfs  --force_postinstall
--prefer-arch-to-version   --add-ignore-recommends busybox-syslog
--add-ignore-recommends busybox-udhcpc --add-ignore-recommends
kernel-vmlinux --add-ignore-recommends udev-cache install [...] glib-2.0
iproute2 kernel-image kernel-module-abcd less libgettextlib [...]'
returned 255:
 * opkg_prepare_url_for_install: Couldn't find anything to satisfy
'kernel-module-abcd'.


So I assume the difference must be in some of the metadata files
(opkg.conf or the ipktemp/ dir argument to the -t flag), but I know
nothing about how opkg/ipk works internally or what to look for.

Thanks,
Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154143): 
https://lists.openembedded.org/g/openembedded-core/message/154143
Mute This Topic: https://lists.openembedded.org/mt/83779506/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] recipes-kernel: vdsotest: new recipe

2021-07-01 Thread Rasmus Villemoes via lists.openembedded.org
On 30/06/2021 23.19, Alexander Kanavin wrote:

> Should this be in meta-oe rather than oe-core?

Perhaps. Could you share some of your thought process that led you to
ask that question? I honestly didn't think much about it, but it did
seem to fit well with the other "userspace tools closely tied to how the
linux kernel works".

Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153450): 
https://lists.openembedded.org/g/openembedded-core/message/153450
Mute This Topic: https://lists.openembedded.org/mt/83902323/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] recipes-kernel: vdsotest: new recipe

2021-06-30 Thread Rasmus Villemoes via lists.openembedded.org
vdsotest is a handy tool for testing and benchmarking the vDSO,
e.g. to measure the overhead of clock_gettime() done via the vDSO
compared to an actual system call.

Signed-off-by: Rasmus Villemoes 
---
Resending because the list rejected the first attempt, sent from a
wrong email address. Apologies to anyone receiving this again.

 meta/recipes-kernel/vdsotest/vdsotest_0.3.bb | 11 +++
 1 file changed, 11 insertions(+)
 create mode 100644 meta/recipes-kernel/vdsotest/vdsotest_0.3.bb

diff --git a/meta/recipes-kernel/vdsotest/vdsotest_0.3.bb 
b/meta/recipes-kernel/vdsotest/vdsotest_0.3.bb
new file mode 100644
index 00..5f94909102
--- /dev/null
+++ b/meta/recipes-kernel/vdsotest/vdsotest_0.3.bb
@@ -0,0 +1,11 @@
+SUMMARY = "vdsotest is a utility for testing and benchmarking a Linux VDSO"
+HOMEPAGE = "https://github.com/nlynch-mentor/vdsotest;
+LICENSE = "GPL-2.0"
+LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
+
+SRC_URI = "git://github.com/nlynch-mentor/vdsotest.git"
+SRCREV = "4ad733fb25364e4afaf4060e57c2429a93b686c5"
+
+S = "${WORKDIR}/git"
+
+inherit autotools
-- 
2.31.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153446): 
https://lists.openembedded.org/g/openembedded-core/message/153446
Mute This Topic: https://lists.openembedded.org/mt/83902323/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] RRECOMMENDS "masking" unsatisfiable IMAGE_INSTALL

2021-06-28 Thread Rasmus Villemoes via lists.openembedded.org
On 25/06/2021 18.13, Richard Purdie wrote:
> On Fri, 2021-06-25 at 17:45 +0200, Rasmus Villemoes wrote:
>> On 25/06/2021 14.16, Richard Purdie wrote:
>>> On Fri, 2021-06-25 at 09:37 +0200, Rasmus Villemoes via 
>>> lists.openembedded.org wrote:
>>>> I noticed that if I have an image recipe that says
>>>>
>>>> IMAGE_INSTALL += "kernel-module-foo"
>>>>
>>>> it fails as expected when the kernel hasn't been built with CONFIG_FOO=m.
>>>>
>>>> However, if at the same time some other package which happens to get
>>>> installed in the same image says
>>>>
>>>> RRECOMMENDS_${PN} += "kernel-module-foo"
>>>>
>>>> the image build silently succeeds, obviously without any foo.ko
>>>> included. This doesn't seem right. [I'm well aware that the RRECOMMENDS
>>>> line alone works as expected, this is about the case where both the
>>>> above lines are in play.]
>>>
>>> Which package backend are you using and does this happen if you
>>> use a different one, particularly swapping ipk and rpm?
>>>
>>> That might tell us if this is a package manager issue or a bitbake
>>> metadata one.
>>
>> I was using ipk. It seems to work as expected (i.e. fail the build) when
>> using rpm.
> 
> That is probably a bug that needs opening against opkg in bugzilla then.
> Added Alex to Cc: (opkg maintainer).

Indeed, thanks. Alex, do you have enough context from the above, or do I
need to do anything explicitly to file a bug?

Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153352): 
https://lists.openembedded.org/g/openembedded-core/message/153352
Mute This Topic: https://lists.openembedded.org/mt/83779506/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] RRECOMMENDS "masking" unsatisfiable IMAGE_INSTALL

2021-06-25 Thread Rasmus Villemoes via lists.openembedded.org
On 25/06/2021 14.16, Richard Purdie wrote:
> On Fri, 2021-06-25 at 09:37 +0200, Rasmus Villemoes via 
> lists.openembedded.org wrote:
>> I noticed that if I have an image recipe that says
>>
>> IMAGE_INSTALL += "kernel-module-foo"
>>
>> it fails as expected when the kernel hasn't been built with CONFIG_FOO=m.
>>
>> However, if at the same time some other package which happens to get
>> installed in the same image says
>>
>> RRECOMMENDS_${PN} += "kernel-module-foo"
>>
>> the image build silently succeeds, obviously without any foo.ko
>> included. This doesn't seem right. [I'm well aware that the RRECOMMENDS
>> line alone works as expected, this is about the case where both the
>> above lines are in play.]
> 
> Which package backend are you using and does this happen if you
> use a different one, particularly swapping ipk and rpm?
> 
> That might tell us if this is a package manager issue or a bitbake
> metadata one.

I was using ipk. It seems to work as expected (i.e. fail the build) when
using rpm.

Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153280): 
https://lists.openembedded.org/g/openembedded-core/message/153280
Mute This Topic: https://lists.openembedded.org/mt/83779506/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] RRECOMMENDS "masking" unsatisfiable IMAGE_INSTALL

2021-06-25 Thread Rasmus Villemoes via lists.openembedded.org
I noticed that if I have an image recipe that says

IMAGE_INSTALL += "kernel-module-foo"

it fails as expected when the kernel hasn't been built with CONFIG_FOO=m.

However, if at the same time some other package which happens to get
installed in the same image says

RRECOMMENDS_${PN} += "kernel-module-foo"

the image build silently succeeds, obviously without any foo.ko
included. This doesn't seem right. [I'm well aware that the RRECOMMENDS
line alone works as expected, this is about the case where both the
above lines are in play.]

=

So I discovered this while trying to figure out how to be able to say
IMAGE_INSTALL += "kernel-module-ext4" , and have that succeed whenever
ext4 is builtin or modular, and have it be a build time error if the
kernel was built with EXT4=n - so the recommends approach is out.  It
obviously doesn't work out-of-the-box for the EXT4=y case. So now I'm
contemplating doing

python kernel_base_provide_builtin () {
kname = d.getVar("KERNEL_PACKAGE_NAME") or "kernel"
builtin = os.path.join(d.getVar("PKGD"), "lib/modules",
d.getVar("KERNEL_VERSION"), "modules.builtin")
if not os.path.exists(builtin):
return
builtin_mods = []
with open(builtin) as f:
for mod in f:
mod = mod.strip()
mod = os.path.basename(mod)
if not mod.endswith(".ko"):
continue
mod = mod[:-3]
mod = mod.lower().replace("_", "-")
builtin_mods.append("%s-module-%s" % (kname, mod))

basepkg = kname + "-base"
rprovides = (d.getVar("RPROVIDES_" + basepkg) or "").split()
rprovides += builtin_mods
d.setVar("RPROVIDES_" + basepkg, " ".join(rprovides))
}
PACKAGESPLITFUNCS += "kernel_base_provide_builtin"

i.e., make sure that the kernel-base package is an rprovider of
kernel-module-ext4 for the EXT4=y case [*]. That seems to work as
expected in the cases I've tried (provided nobody rrecommends ext4, per
above). Does anybody see anything wrong with that approach? EXT4 is just
an example, I'm gonna use it for other modules/subsystems with more
complicated dependencies that might not be satisfied in a base .config,
so a fragment saying CONFIG_FOO=m might have been ignored due to that,
and I'd prefer catching that no later than at image build time.

I'm not necessarily suggesting the above for oe-core, I can live with it
in my own kernel recipes.

[*] kernel-base and not the package containing the actual kernel binary,
because kernel-base contains the modules.builtin file which is what
modprobe consults first to see if it actually has to do anything, so it
makes sense to have kernel-module-ext4 pull in that file and the other
/lib/modules/ metadata when EXT4=y. The kernel binary is as often as not
not actually in the rootfs image, and wouldn't be useful to modprobe anyway.

Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153266): 
https://lists.openembedded.org/g/openembedded-core/message/153266
Mute This Topic: https://lists.openembedded.org/mt/83779506/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [RFC PATCH 0/4] fix externalsrc task dependency issues

2020-09-25 Thread Rasmus Villemoes
On 25/09/2020 15.14, Rasmus Villemoes via lists.openembedded.org wrote:

> You've convinced me that changing deltask to noexec is too
> wide-reaching, which means that I think just patch 1/4 should be
> applied. Please consider doing that.

Or rather, please apply the v2 I just sent to update the commit log to
reflect my new understanding.

Thanks,
Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142789): 
https://lists.openembedded.org/g/openembedded-core/message/142789
Mute This Topic: https://lists.openembedded.org/mt/77015064/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH v2] kernel.bbclass: ensure symlink_kernsrc task gets run even with externalsrc

2020-09-25 Thread Rasmus Villemoes
Commit c5dfc2586b41 (kernel.bbclass: run do_symlink_kernsrc before
do_patch) fixed a race between do_symlink_kernsrc and
do_populate_lic. However, I missed the fact that when
externalsrc.bbclass is in use, the do_patch task doesn't exist,
meaning that do_symlink_kernsrc now doesn't get run at all, breaking
the build.

We need this task to run regardless of whether do_patch exists or not,
so reinstate the configure->symlink_kernsrc dependency explicitly.

Fixes: c5dfc2586b41 (kernel.bbclass: run do_symlink_kernsrc before
do_patch)
Reported-by: Chanho Park 

Signed-off-by: Rasmus Villemoes 
---
v2: update commit log to not say that externalsrc.bbclass is at fault
for using deltask over [noexec]. Rest of the series is dropped.

 meta/classes/kernel.bbclass | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 48135b3d41..78def5bbc1 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -172,7 +172,10 @@ python do_symlink_kernsrc () {
 shutil.move(s, kernsrc)
 os.symlink(kernsrc, s)
 }
-addtask symlink_kernsrc before do_patch after do_unpack
+# do_patch is normally ordered before do_configure, but
+# externalsrc.bbclass deletes do_patch, breaking the dependency of
+# do_configure on do_symlink_kernsrc.
+addtask symlink_kernsrc before do_patch do_configure after do_unpack
 
 inherit kernel-arch deploy
 
-- 
2.23.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142788): 
https://lists.openembedded.org/g/openembedded-core/message/142788
Mute This Topic: https://lists.openembedded.org/mt/77080010/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [RFC PATCH 0/4] fix externalsrc task dependency issues

2020-09-25 Thread Rasmus Villemoes
On 25/09/2020 13.43, Richard Purdie wrote:
> On Fri, 2020-09-25 at 12:31 +0200, Rasmus Villemoes wrote:
>> Well, perhaps listtasks could be taught to say "[noexec flag set]" in
>> the description - currently it seems to be somewhat random whether a
>> class uses noexec or deltask.
> 
> No. That would just confuse things even more. That would hide do_build
> for example so the default target would be missing?

I did not say "don't display noexec tasks". I said 'say "[noexec flag
set]" in the description' - i.e., have the output include an indication
that that task will not be actually be executed. Perhaps a slightly
longer prose would be helpful, say something like "[disabled for this
recipe due to noexec flag being set]" appended after the current
description. There are already lots of noexec tasks, e.g. all image
recipes have a lot of ordinary tasks disabled via noexec, so that should
help address your concern that

> ... If
> someone can "see" the task in listtasks and trys to run it but nothing
> happens, its very confusing for the user.

>> Oh, and for debugging task ordering issues, it would also be nice if
>> listtasks showed tasks in topological rather than alphabetical order -
>> that won't necessarily always show an implicit dependency that went
>> missing due to a deltask, but it would at least reveal some of them.
> 
> I think showing half of some information would be much more confusing
> that clearly not showing it. We'd just get bug reports that the order
> was "wrong" and it can't be fixed in all cases.

Can we then (perhaps via some listtasksfull or whatnot) get bitbake to
display the tasks along with their dependencies. Or how would I, today,
go about figuring out what (intra-recipe) dependencies
some-recipe:do_configure actually has?

>>> I think there were technical issues as well, such as recipes which
>>> inject other tasks between unpack and patch. We needed to stop those
>>> other tasks running too which deltasks does effectively, noexec would
>>> not.
>>
>> Eh, but the other side of that coin is that there are obviously tasks
>> injected between unpack and patch, or some other combination, which must
>> _not_ be prevented from running
> 
> Some need to be prevented, for example where they do extra unpacking
> type operations on files. Those would actively break externalsrc.
> 
>>  - cf. at least the two or three workarounds that seem to become unnecessary.
> 
> Some would be unnecessary, it would break other use cases.
> 
>> So can you give examples of
>> such tasks that _should_ implicitly not run when externalsrc disables
>> (one way or another) the SRCTREECOVEREDTASKS tasks? That's actually the
>> RFC part of this, I hope somebody with far more knowledge of various
>> corner cases and odd recipes can demonstrate why deltask is more
>> appropriate - and then, as I said, some comments to that effect in
>> externalsrc.bbclass would be much appreciated.
> 
> I will try my best to find them. I don't have a list of these things I
> can just recite from memory. One that did come up in a quick search is:
> 
> classes/dos2unix.bbclass:addtask convert_crlf_to_lf after do_unpack before 
> do_patch
> 
> I would note that the class doesn't appear used in OE-Core any longer.

OK, that counts, even if that class appears unused - I can see how such
things can exist.

> Proving OE-Core is "safe" for such a change is only part of the
> challenge too, there is the wider ecosystem to consider.

Absolutely.

>> I think it would be weird for a recipe to inherit externalsrc and at the
>> same time define a task ordered before do_patch, then not expect that
>> task to run at all. So I'm genuinely curious how such examples would look.
> 
> devtool takes recipes and adds externalsrc behind the scenes to them to
> enable it to do things. Its expected these scenarios work, its used for
> example by the automated upgrade helper. We're not risking breaking all
> these tools just because you think the code might be cleaner.

See, that's the kind of explanation I'm looking for - and again, I'd
like to see in externalsrc.bbclass itself. The thing is, I never knew
about the existence of the externalsrc class until I noticed the bug
reported on c5dfc2586b. So when I tried to figure out how to fix that
bug, I had to understand why externalsrc removed the do_patch task,
breaking the configure->patch->symlink_kernsrc chain.

> Your approach also concerns me as you are basically asking me to
> "prove" your code has issues and it seems aren't interested in testing
> even the patch you did send.

That is not how I intended this to be interpreted. But I was r

Re: [OE-core] [RFC PATCH 0/4] fix externalsrc task dependency issues

2020-09-25 Thread Rasmus Villemoes
On 25/09/2020 11.22, Richard Purdie wrote:
> On Fri, 2020-09-25 at 11:06 +0200, Rasmus Villemoes wrote:
>> On 22/09/2020 17.15, Rasmus Villemoes wrote:
>>> This is all entirely untested, hence RFC. But if this doesn't work
>>> for
>>> some reason, I think externalsrc.bbclass should grow some comments
>>> as
>>> to why deltask is used instead of the noexec flag.
>>>
>>> Patch 4 is a revert of patch 1, but I did it that way to facilitate
>>> backporting, in case patch 2 is too dangerous for backporting. If
>>> that
>>> is considered safe enough, it should be enough to apply patch 2 to
>>> avoid the churn. In any case, patch 3 is just a little
>>> simplification
>>> made possible by patch 2.
>>>
>>>
>>
>> Ping. Chanho, can you tell if this would fix the externalsrc problem?
>> Richard, any comments on the deltask vs noexec usage?
> 
> You started this patchset with "This is all entirely untested" which
> I'm afraid made me rather nervous. Much as I'd love to, I don't have
> time to debug every patch series people send.
> 
> I'm fairly sure the deltask vs. noexec was deliberate and Paul and I
> went around in circles on this several times.
> 
> I have some memory that at least part of the problem was usability. If
> someone can "see" the task in listtasks and trys to run it but nothing
> happens, its very confusing for the user. By deleting it, we make it
> very clear its not there.

Well, perhaps listtasks could be taught to say "[noexec flag set]" in
the description - currently it seems to be somewhat random whether a
class uses noexec or deltask.

Oh, and for debugging task ordering issues, it would also be nice if
listtasks showed tasks in topological rather than alphabetical order -
that won't necessarily always show an implicit dependency that went
missing due to a deltask, but it would at least reveal some of them.

> I think there were technical issues as well, such as recipes which
> inject other tasks between unpack and patch. We needed to stop those
> other tasks running too which deltasks does effectively, noexec would
> not.

Eh, but the other side of that coin is that there are obviously tasks
injected between unpack and patch, or some other combination, which must
_not_ be prevented from running - cf. at least the two or three
workarounds that seem to become unnecessary. So can you give examples of
such tasks that _should_ implicitly not run when externalsrc disables
(one way or another) the SRCTREECOVEREDTASKS tasks? That's actually the
RFC part of this, I hope somebody with far more knowledge of various
corner cases and odd recipes can demonstrate why deltask is more
appropriate - and then, as I said, some comments to that effect in
externalsrc.bbclass would be much appreciated.

I think it would be weird for a recipe to inherit externalsrc and at the
same time define a task ordered before do_patch, then not expect that
task to run at all. So I'm genuinely curious how such examples would look.

Thanks,
Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142779): 
https://lists.openembedded.org/g/openembedded-core/message/142779
Mute This Topic: https://lists.openembedded.org/mt/77015064/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [RFC PATCH 0/4] fix externalsrc task dependency issues

2020-09-25 Thread Rasmus Villemoes
On 22/09/2020 17.15, Rasmus Villemoes wrote:
> This is all entirely untested, hence RFC. But if this doesn't work for
> some reason, I think externalsrc.bbclass should grow some comments as
> to why deltask is used instead of the noexec flag.
> 
> Patch 4 is a revert of patch 1, but I did it that way to facilitate
> backporting, in case patch 2 is too dangerous for backporting. If that
> is considered safe enough, it should be enough to apply patch 2 to
> avoid the churn. In any case, patch 3 is just a little simplification
> made possible by patch 2.
> 
> 

Ping. Chanho, can you tell if this would fix the externalsrc problem?
Richard, any comments on the deltask vs noexec usage?

Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142773): 
https://lists.openembedded.org/g/openembedded-core/message/142773
Mute This Topic: https://lists.openembedded.org/mt/77015064/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [RFC PATCH 0/4] fix externalsrc task dependency issues

2020-09-22 Thread Rasmus Villemoes
This is all entirely untested, hence RFC. But if this doesn't work for
some reason, I think externalsrc.bbclass should grow some comments as
to why deltask is used instead of the noexec flag.

Patch 4 is a revert of patch 1, but I did it that way to facilitate
backporting, in case patch 2 is too dangerous for backporting. If that
is considered safe enough, it should be enough to apply patch 2 to
avoid the churn. In any case, patch 3 is just a little simplification
made possible by patch 2.


Rasmus Villemoes (4):
  kernel.bbclass: ensure symlink_kernsrc task gets run even with
externalsrc
  externalsrc.bbclass: make SRCTREECOVEREDTASKS noexec instead of
deleting them
  kernel-yocto.bbclass: remove now unnecessary externalsrc workaround
  Revert "kernel.bbclass: ensure symlink_kernsrc task gets run even with
externalsrc"

 meta/classes/externalsrc.bbclass  | 5 +
 meta/classes/kernel-yocto.bbclass | 6 --
 2 files changed, 1 insertion(+), 10 deletions(-)

-- 
2.23.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142720): 
https://lists.openembedded.org/g/openembedded-core/message/142720
Mute This Topic: https://lists.openembedded.org/mt/77015064/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [RFC PATCH 4/4] Revert "kernel.bbclass: ensure symlink_kernsrc task gets run even with externalsrc"

2020-09-22 Thread Rasmus Villemoes
With the changes to externalsrc.bbclass to make tasks noexec instead
of deleting them, we no longer need to explicitly make
do_symlink_kernsrc a dependency of do_configure.

Signed-off-by: Rasmus Villemoes 
---
 meta/classes/kernel.bbclass | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 78def5bbc1..48135b3d41 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -172,10 +172,7 @@ python do_symlink_kernsrc () {
 shutil.move(s, kernsrc)
 os.symlink(kernsrc, s)
 }
-# do_patch is normally ordered before do_configure, but
-# externalsrc.bbclass deletes do_patch, breaking the dependency of
-# do_configure on do_symlink_kernsrc.
-addtask symlink_kernsrc before do_patch do_configure after do_unpack
+addtask symlink_kernsrc before do_patch after do_unpack
 
 inherit kernel-arch deploy
 
-- 
2.23.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142724): 
https://lists.openembedded.org/g/openembedded-core/message/142724
Mute This Topic: https://lists.openembedded.org/mt/77015068/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [RFC PATCH 2/4] externalsrc.bbclass: make SRCTREECOVEREDTASKS noexec instead of deleting them

2020-09-22 Thread Rasmus Villemoes
The preferred way to prevent a task from running is not to deltask it,
but setting the noexec flag. That way various implicit dependencies
through the task in question are preserved, avoiding ad hoc
workarounds, such as the one right here.

Signed-off-by: Rasmus Villemoes 
---
 meta/classes/externalsrc.bbclass | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
index d200129987..7b4d8b369f 100644
--- a/meta/classes/externalsrc.bbclass
+++ b/meta/classes/externalsrc.bbclass
@@ -100,14 +100,11 @@ python () {
 d.setVarFlag(task, 'cleandirs', ' '.join(cleandirs))
 
 fetch_tasks = ['do_fetch', 'do_unpack']
-# If we deltask do_patch, there's no dependency to ensure do_unpack 
gets run, so add one
-# Note that we cannot use d.appendVarFlag() here because deps is 
expected to be a list object, not a string
-d.setVarFlag('do_configure', 'deps', (d.getVarFlag('do_configure', 
'deps', False) or []) + ['do_unpack'])
 
 for task in d.getVar("SRCTREECOVEREDTASKS").split():
 if local_srcuri and task in fetch_tasks:
 continue
-bb.build.deltask(task, d)
+d.setVarFlag(task, 'noexec', '1')
 
 d.prependVarFlag('do_compile', 'prefuncs', 
"externalsrc_compile_prefunc ")
 d.prependVarFlag('do_configure', 'prefuncs', 
"externalsrc_configure_prefunc ")
-- 
2.23.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142722): 
https://lists.openembedded.org/g/openembedded-core/message/142722
Mute This Topic: https://lists.openembedded.org/mt/77015066/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [RFC PATCH 3/4] kernel-yocto.bbclass: remove now unnecessary externalsrc workaround

2020-09-22 Thread Rasmus Villemoes
Now that SRCTREECOVEREDTASKS gets marked as noexec rather than being
deltask'ed, do_kernel_configme is still ordered after do_patch (and
recursively, after do_unpack), so this workaround is no longer needed.

Signed-off-by: Rasmus Villemoes 
---
 meta/classes/kernel-yocto.bbclass | 6 --
 1 file changed, 6 deletions(-)

diff --git a/meta/classes/kernel-yocto.bbclass 
b/meta/classes/kernel-yocto.bbclass
index a35c5923df..741b0b961c 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -656,12 +656,6 @@ python () {
 # If diffconfig is available, ensure it runs after kernel_configme
 if 'do_diffconfig' in d:
 bb.build.addtask('do_diffconfig', None, 'do_kernel_configme', d)
-
-externalsrc = d.getVar('EXTERNALSRC')
-if externalsrc:
-# If we deltask do_patch, do_kernel_configme is left without
-# dependencies and runs too early
-d.setVarFlag('do_kernel_configme', 'deps', 
(d.getVarFlag('do_kernel_configme', 'deps', False) or []) + ['do_unpack'])
 }
 
 # extra tasks
-- 
2.23.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142723): 
https://lists.openembedded.org/g/openembedded-core/message/142723
Mute This Topic: https://lists.openembedded.org/mt/77015067/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [RFC PATCH 1/4] kernel.bbclass: ensure symlink_kernsrc task gets run even with externalsrc

2020-09-22 Thread Rasmus Villemoes
Commit c5dfc2586b41 (kernel.bbclass: run do_symlink_kernsrc before
do_patch) fixed a race between do_symlink_kernsrc and
do_populate_lic. However, I missed the fact that when
externalsrc.bbclass is in use, the do_patch task doesn't exist,
meaning that do_symlink_kernsrc now doesn't get run at all, breaking
the build.

While I think the proper fix is to stop deltask'ing do_patch and
instead make it noexec in externalsrc.bbclass, a more targeted fix
suitable for backporting is to simply make this a dependency of both
do_patch and do_configure.

Fixes: c5dfc2586b41 (kernel.bbclass: run do_symlink_kernsrc before
do_patch)
Reported-by: Chanho Park 

Signed-off-by: Rasmus Villemoes 
---
 meta/classes/kernel.bbclass | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 48135b3d41..78def5bbc1 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -172,7 +172,10 @@ python do_symlink_kernsrc () {
 shutil.move(s, kernsrc)
 os.symlink(kernsrc, s)
 }
-addtask symlink_kernsrc before do_patch after do_unpack
+# do_patch is normally ordered before do_configure, but
+# externalsrc.bbclass deletes do_patch, breaking the dependency of
+# do_configure on do_symlink_kernsrc.
+addtask symlink_kernsrc before do_patch do_configure after do_unpack
 
 inherit kernel-arch deploy
 
-- 
2.23.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142721): 
https://lists.openembedded.org/g/openembedded-core/message/142721
Mute This Topic: https://lists.openembedded.org/mt/77015065/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core][dunfell 23/25] kernel.bbclass: run do_symlink_kernsrc before do_patch

2020-09-21 Thread Rasmus Villemoes
On 16/09/2020 17.18, Steve Sakoman wrote:
> Since there is an upcoming dunfell release and we don't have a fix for
> this issue I am going to revert this patch.
> 
> When it is fixed in master I will reconsider taking this patch and the
> fix for dunfell.

Sorry for not seeing this sooner, and for the bug in the first place.

However, looking into externalsrc.bbclass, isn't the bug there,
considering the "if we delete do_patch, do_unpack doesn't run" fixup
done - kernel-yocto.bbclass has a similar fixup for do_kernel_configme.
Why doesn't that class just set [noexec]=1 instead of deleting the tasks
outright, killing all those random fixups?

But if that's too invasive, I suppose just ordering the symlink task
before both patch and configure should work, i.e.

--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -172,7 +172,7 @@ python do_symlink_kernsrc () {
 shutil.move(s, kernsrc)
 os.symlink(kernsrc, s)
 }
-addtask symlink_kernsrc before do_patch after do_unpack
+addtask symlink_kernsrc before do_patch do_configure after do_unpack

 inherit kernel-arch deploy




Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142672): 
https://lists.openembedded.org/g/openembedded-core/message/142672
Mute This Topic: https://lists.openembedded.org/mt/76518932/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] rootfs-postcommands.bbclass: make /etc/timestamp path configurable

2020-09-08 Thread Rasmus Villemoes
On 04/09/2020 13.51, Richard Purdie wrote:
> On Fri, 2020-09-04 at 09:08 +0200, Rasmus Villemoes wrote:
>> Not every image necessarily wants a timestamp file,
> 
> Wouldn't you just not have rootfs_update_timestamp in
> ROOTFS_POSTPROCESS_COMMAND in that case?

Perhaps, but removing stuff from ROOTFS_POSTPROCESS_COMMAND is a little
tedious and error-prone, 8c835f63 isn't all that old. And with the other
use case in mind, it really isn't a lot of extra logic to assign a
meaning to ROOTFS_TIMESTAMP_FILE being empty.

> 
> I have a feeling there are other consumers of this file which would
> also need to be changed to reference ROOTFS_TIMESTAMP_FILE?

Well, there are two files associated to the initscripts recipe
mentioning it, and a mention in devtool. But I don't imagine
ROOTFS_TIMESTAMP_FILE being set globally, it's really something one can
set in individual image recipes for images that don't create an ordinary
linux rootfs (e.g. bootfiles for RPi). Dunno, if somebody sets
ROOTFS_TIMESTAMP_FILE for their proper rootfs and also include the
initscripts package in that rootfs, I suppose they should match, but I
don't know where one should set the variable.

Btw., is there some good reason that by default, with reprocible_builds
turned on, the rootfs gets two files with the exact same content
(/etc/timestamp and /etc/version)? Because we'll need to fix (or avoid)
the creation of /etc/version in the same way for some of our images.

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#142250): 
https://lists.openembedded.org/g/openembedded-core/message/142250
Mute This Topic: https://lists.openembedded.org/mt/76624850/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] [PATCH] rootfs-postcommands.bbclass: make /etc/timestamp path configurable

2020-09-04 Thread Rasmus Villemoes
Not every image necessarily wants a timestamp file, and even if it
does, it's possible to create filesystem images that do not have a
/etc directory or where some other location for the file is more
appropriate. So add a ROOTFS_TIMESTAMP_FILE variable weakly defaulting
to /etc/timestamp - setting that to the empty string opts out of
creating the timestamp file.

I wasn't sure whether to spell /etc as ${sysconfdir}, but chose /etc
to stay completely backwards compatible.

Signed-off-by: Rasmus Villemoes 
---
 meta/classes/rootfs-postcommands.bbclass | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/classes/rootfs-postcommands.bbclass 
b/meta/classes/rootfs-postcommands.bbclass
index 984730ebe8..cf54f22566 100644
--- a/meta/classes/rootfs-postcommands.bbclass
+++ b/meta/classes/rootfs-postcommands.bbclass
@@ -12,6 +12,7 @@ ROOTFS_POSTPROCESS_COMMAND += 
'${@bb.utils.contains_any("IMAGE_FEATURES", [ 'deb
 ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains_any("IMAGE_FEATURES", [ 
'debug-tweaks', 'post-install-logging' ], "postinst_enable_logging; ", "",d)}'
 
 # Create /etc/timestamp during image construction to give a reasonably sane 
default time setting
+ROOTFS_TIMESTAMP_FILE ??= "/etc/timestamp"
 ROOTFS_POSTPROCESS_COMMAND += "rootfs_update_timestamp; "
 
 # Tweak the mount options for rootfs in /etc/fstab if read-only-rootfs is 
enabled
@@ -275,14 +276,18 @@ python write_image_manifest () {
 # Can be used to create /etc/timestamp during image construction to give a 
reasonably
 # sane default time setting
 rootfs_update_timestamp () {
+   if [ -z "${ROOTFS_TIMESTAMP_FILE}" ]; then
+   return
+   fi
if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
# Convert UTC into %4Y%2m%2d%2H%2M%2S
sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} 
+%4Y%2m%2d%2H%2M%2S`
else
sformatted=`date -u +%4Y%2m%2d%2H%2M%2S`
fi
-   echo $sformatted > ${IMAGE_ROOTFS}/etc/timestamp
-   bbnote "rootfs_update_timestamp: set /etc/timestamp to $sformatted"
+   mkdir -p $(dirname ${IMAGE_ROOTFS}${ROOTFS_TIMESTAMP_FILE})
+   echo $sformatted > ${IMAGE_ROOTFS}${ROOTFS_TIMESTAMP_FILE}
+   bbnote "rootfs_update_timestamp: set ${ROOTFS_TIMESTAMP_FILE} to 
$sformatted"
 }
 
 # Prevent X from being started
-- 
2.23.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#142182): 
https://lists.openembedded.org/g/openembedded-core/message/142182
Mute This Topic: https://lists.openembedded.org/mt/76624850/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] [PATCH] kernel.bbclass: run do_symlink_kernsrc before do_patch

2020-08-25 Thread Rasmus Villemoes
There's a race between do_symlink_kernsrc and do_populate_lic, since
the latter is ordered "after do_patch"; so the two may run in
parallel. In some cases, that actually causes do_populate_lic to fail
if it happens to look for a license file somewhere under ${S} in the
short window after shutil.move and before the symlink has been
created.

Fix that by simply ordering symlink_kernsrc before do_patch. Any task
that pokes around in ${S} looking for files should be ordered after
do_patch, so this should also fix similar latent races with other ad
hoc tasks.

Signed-off-by: Rasmus Villemoes 
---
 meta/classes/kernel.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index e2ceb6a333..cba77daa7a 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -172,7 +172,7 @@ python do_symlink_kernsrc () {
 shutil.move(s, kernsrc)
 os.symlink(kernsrc, s)
 }
-addtask symlink_kernsrc before do_configure after do_unpack
+addtask symlink_kernsrc before do_patch after do_unpack
 
 inherit kernel-arch deploy
 
-- 
2.23.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#141825): 
https://lists.openembedded.org/g/openembedded-core/message/141825
Mute This Topic: https://lists.openembedded.org/mt/76407816/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] do_populate_lic problems

2020-08-25 Thread Rasmus Villemoes
There are a couple of problems with do_populate_lic, especially around
the kernel and u-boot recipes.

First, we have a race condition in kernel recipes:

addtask populate_lic after do_patch before do_build
addtask symlink_kernsrc before do_configure after do_unpack

so there's no ordering between those two tasks. I have in fact seen
do_populate_lic fail because (part of) it runs just between the two lines

shutil.move(s, kernsrc)
os.symlink(kernsrc, s)

WARNING: Could not copy license file /mnt/...
...
ERROR: QA Issue: ...: LIC_FILES_CHKSUM points to an invalid file:

and it's fairly easy to reproduce with a monkey-patch:

@@ -148,7 +148,9 @@ python do_symlink_kernsrc () {
 os.symlink(s, kernsrc)
 else:
 import shutil
+import time
 shutil.move(s, kernsrc)
+time.sleep(15)
 os.symlink(kernsrc, s)
 }

I think this could be fixed by making symlink_kernsrc run before
do_patch rather than do_configure - in fact, AFAICT, there's the exact
same lack of ordering with do_patch, it's just that nobody usually has
any .patch files for the kernel when it's maintained in git.

Second, as I've reported previously [1], creating hard-links to files in
kernel or u-boot source repositories, as do_populate_lic does, leads to
scripts/setlocalversion falsely appending a "-dirty" string. It doesn't
happen if one is building a kernel containing ff64dd485730 or a backport
of it (e.g. v5.0+, v4.19.82+) _and_ one has git >= 2.14. U-Boot hasn't
copied ff64dd485730 yet, so it can always happen for U-Boot. Since
there's no ordering between do_populate_lic and do_compile, it's not
even deterministic whether that -dirty string is appended. So while
appropriate tools and patched upstream projects will make the problem go
away, I'm wondering the hardlink optimization is really worth the
trouble it causes.

[1] https://lists.openembedded.org/g/openembedded-core/message/137702

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#141806): 
https://lists.openembedded.org/g/openembedded-core/message/141806
Mute This Topic: https://lists.openembedded.org/mt/76403349/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] cleansstate

2020-08-25 Thread Rasmus Villemoes
On 24/08/2020 23.08, Richard Purdie wrote:
> On Mon, 2020-08-24 at 23:05 +0200, Rasmus Villemoes wrote:
>> On 22/08/2020 09.27, Richard Purdie via lists.openembedded.org wrote:
>>> On Fri, 2020-08-21 at 17:32 -0700, Tom King wrote:
>>>> What would be a use case for cleanstate?
>>>

>>> I understand why people want it although it should never really
>>> need to
>>> be used.
>>
>> Well, except when you discover that your sstate is effectively
>> corrupted/polluted by objects that happen to have the same hashes as
>> those computed from current recipes, but the build artifacts
>> would/should be different. Such as when using a file://foo/* glob
>> pattern in SRC_URI.
> 
> Where such issues are reported, we do try and fix them. I hadn't quite
> connected that the globbing issue was actually breaking sstate hashes,
> that is more worrying :(

It does, and you can reproduce yourself using upstream core recipes. Try
building connman-gnome. Then alter one of the png files under images/,
and build again. The recipe does not get rebuilt. Then cleansstate,
build, and yes, then the modified png does get picked up.

Of course, those pngs don't change often, if ever. But in our case the
recipe was for including some firmware blobs for some I/O modules. We
found out when the CI builds that used sstate cache failed to produce
BSPs containing the updated firmware, while the nightly builds that
build from scratch did.

>>
>> Richard, I'm still hoping you'd respond in the
>> https://lists.openembedded.org/g/openembedded-core/topic/75235594
>> thread. As I see it, half-way supporting glob patterns in file://
>> URIs is a giant footgun.
> 
> My leaning is to remove them. You can just do file://foo and I think
> that actually works?

In some of the cases where we had recipes using globs, yes, that was the
solution. In a few other cases we had more complicated patterns (e.g.
*.sh or something else that wasn't just including a whole dir).

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#141805): 
https://lists.openembedded.org/g/openembedded-core/message/141805
Mute This Topic: https://lists.openembedded.org/mt/76298145/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] cleansstate

2020-08-24 Thread Rasmus Villemoes
On 22/08/2020 09.27, Richard Purdie via lists.openembedded.org wrote:
> On Fri, 2020-08-21 at 17:32 -0700, Tom King wrote:
>> What would be a use case for cleanstate?
> 
> I never really wanted to add it at all. There is/was some case for
> wanting to remove sstate objects and ensure something really does
> rebuild from source. When sstate was new and had more issues, people
> really wanted it and I was pushed into having it exist.
> 
> I understand why people want it although it should never really need to
> be used.

Well, except when you discover that your sstate is effectively
corrupted/polluted by objects that happen to have the same hashes as
those computed from current recipes, but the build artifacts
would/should be different. Such as when using a file://foo/* glob
pattern in SRC_URI.

Richard, I'm still hoping you'd respond in the
https://lists.openembedded.org/g/openembedded-core/topic/75235594
thread. As I see it, half-way supporting glob patterns in file:// URIs
is a giant footgun.

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#141789): 
https://lists.openembedded.org/g/openembedded-core/message/141789
Mute This Topic: https://lists.openembedded.org/mt/76298145/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] file checksums versus SRC_URI globs

2020-08-05 Thread Rasmus Villemoes
On 09/07/2020 21.31, Rasmus Villemoes via lists.openembedded.org wrote:
> On 02/07/2020 08.42, Rasmus Villemoes via lists.openembedded.org wrote:
>> On 01/07/2020 16.03, Quentin Schulz wrote:
>>> Hi Rasmus,
>>>
>>> On Wed, Jul 01, 2020 at 03:51:19PM +0200, Rasmus Villemoes wrote:
>>>> Hi,
>>>>
>>>> We have a recipe that uses
>>>>
>>>>   SRC_URI += "file://somedir/*"
>>>>
>>>
>>> Glob aren't supported. Use "file://somedir/" instead.
>>
>> Thanks, that actually works for one of the cases we have (there are
>> others that use globs which cannot be solved quite that simply, but for
>> now I'm just listing files explicitly instead).
>>
>> However, I'm not sure that "globs aren't supported". The commit I
>> referenced clearly tried to make that work (better), it also "works" in
>> the sense of unpacking the expected things when building from scratch -
>> there's even
>>
>> def test_local_wildcard(self):
>> tree = self.fetchUnpack(['file://a', 'file://dir/*'])
>> self.assertEqual(tree, ['a',  'dir/c', 'dir/d', 'dir/subdir/e'])
>>
>> in bitbake/lib/bb/tests/fetch.py. And the two upstream recipes
>> connman-gnome_0.7.bb and matchbox-desktop_2.2.bb both use that exact
>> pattern.
>>
>> So either
>>
>> - this is a plain bug in the signature computation,
> 
> I'm guessing the culprit is
> 
> commit 6c0706a28d72c591f1b75b6e3f3b645859387c7e
> Author: Richard Purdie 
> Date:   Mon Dec 8 21:25:23 2014 +
> 
> cache/fetch2/siggen: Ensure we track include history for file checksums
> 
> which was the one that introduced the ":True: or ":False" suffixing via
> 
> +filelist.append(f + ":" + str(os.path.exists(f)))
> 
> and then also did
> 
> @@ -981,6 +980,10 @@ def get_file_checksums(filelist, pn):
> 
>  checksums = []
>  for pth in filelist.split():
> +exist = pth.split(":")[1]
> +if exist == "False":
> +continue
> +pth = pth.split(":")[0]
>  if '*' in pth:
>  # Handle globs
>  for f in glob.glob(pth):
> 
> which practically guaranteed that the "if '*' in pth" would be dead code.
> 
> Richard, do you agree that this is a bug in the signature computation?

Richard, ping?

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#141199): 
https://lists.openembedded.org/g/openembedded-core/message/141199
Mute This Topic: https://lists.openembedded.org/mt/75235594/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] [PATCH] move find_cfgs() helper to cml1.bbclass

2020-07-09 Thread Rasmus Villemoes
u-boot.inc and busybox.inc contain identical copies of this little
helper. They both inherit the cml1 class and use cml1_do_configure
right after having used this helper, and other recipes that want to
write similar logic for doing Kconfig via fragments will also need it
or something equivalent.

Signed-off-by: Rasmus Villemoes 
---
Those other recipes can certainly also just copy-paste find_cfgs(),
but I think it's nice to avoid even small pieces of code duplication
(it all adds up).

What does cml1 stand for? I can't find anything in git history or
comments, and "basic support for the Linux kernel style build
configuration system" doesn't provide much of a clue either.

 meta/classes/cml1.bbclass | 10 ++
 meta/recipes-bsp/u-boot/u-boot.inc| 10 --
 meta/recipes-core/busybox/busybox.inc | 10 --
 3 files changed, 10 insertions(+), 20 deletions(-)

diff --git a/meta/classes/cml1.bbclass b/meta/classes/cml1.bbclass
index c7f6723cb3..8ab240589a 100644
--- a/meta/classes/cml1.bbclass
+++ b/meta/classes/cml1.bbclass
@@ -1,3 +1,13 @@
+# returns all the elements from the src uri that are .cfg files
+def find_cfgs(d):
+sources=src_patches(d, True)
+sources_list=[]
+for s in sources:
+if s.endswith('.cfg'):
+sources_list.append(s)
+
+return sources_list
+
 cml1_do_configure() {
set -e
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
diff --git a/meta/recipes-bsp/u-boot/u-boot.inc 
b/meta/recipes-bsp/u-boot/u-boot.inc
index c91da02959..5ec3d90764 100644
--- a/meta/recipes-bsp/u-boot/u-boot.inc
+++ b/meta/recipes-bsp/u-boot/u-boot.inc
@@ -70,16 +70,6 @@ UBOOT_EXTLINUX_INSTALL_DIR ?= "/boot/extlinux"
 UBOOT_EXTLINUX_CONF_NAME ?= "extlinux.conf"
 UBOOT_EXTLINUX_SYMLINK ?= "${UBOOT_EXTLINUX_CONF_NAME}-${MACHINE}-${PR}"
 
-# returns all the elements from the src uri that are .cfg files
-def find_cfgs(d):
-sources=src_patches(d, True)
-sources_list=[]
-for s in sources:
-if s.endswith('.cfg'):
-sources_list.append(s)
-
-return sources_list
-
 do_configure () {
 if [ -n "${UBOOT_CONFIG}" ]; then
 unset i j
diff --git a/meta/recipes-core/busybox/busybox.inc 
b/meta/recipes-core/busybox/busybox.inc
index 6cfdcd7344..45aaa2b41c 100644
--- a/meta/recipes-core/busybox/busybox.inc
+++ b/meta/recipes-core/busybox/busybox.inc
@@ -133,16 +133,6 @@ do_prepare_config () {
fi
 }
 
-# returns all the elements from the src uri that are .cfg files
-def find_cfgs(d):
-sources=src_patches(d, True)
-sources_list=[]
-for s in sources:
-if s.endswith('.cfg'):
-sources_list.append(s)
-
-return sources_list
-
 do_configure () {
set -x
do_prepare_config
-- 
2.23.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#140493): 
https://lists.openembedded.org/g/openembedded-core/message/140493
Mute This Topic: https://lists.openembedded.org/mt/75405917/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] file checksums versus SRC_URI globs

2020-07-09 Thread Rasmus Villemoes
On 02/07/2020 08.42, Rasmus Villemoes via lists.openembedded.org wrote:
> On 01/07/2020 16.03, Quentin Schulz wrote:
>> Hi Rasmus,
>>
>> On Wed, Jul 01, 2020 at 03:51:19PM +0200, Rasmus Villemoes wrote:
>>> Hi,
>>>
>>> We have a recipe that uses
>>>
>>>   SRC_URI += "file://somedir/*"
>>>
>>
>> Glob aren't supported. Use "file://somedir/" instead.
> 
> Thanks, that actually works for one of the cases we have (there are
> others that use globs which cannot be solved quite that simply, but for
> now I'm just listing files explicitly instead).
> 
> However, I'm not sure that "globs aren't supported". The commit I
> referenced clearly tried to make that work (better), it also "works" in
> the sense of unpacking the expected things when building from scratch -
> there's even
> 
> def test_local_wildcard(self):
> tree = self.fetchUnpack(['file://a', 'file://dir/*'])
> self.assertEqual(tree, ['a',  'dir/c', 'dir/d', 'dir/subdir/e'])
> 
> in bitbake/lib/bb/tests/fetch.py. And the two upstream recipes
> connman-gnome_0.7.bb and matchbox-desktop_2.2.bb both use that exact
> pattern.
> 
> So either
> 
> - this is a plain bug in the signature computation,

I'm guessing the culprit is

commit 6c0706a28d72c591f1b75b6e3f3b645859387c7e
Author: Richard Purdie 
Date:   Mon Dec 8 21:25:23 2014 +

cache/fetch2/siggen: Ensure we track include history for file checksums

which was the one that introduced the ":True: or ":False" suffixing via

+filelist.append(f + ":" + str(os.path.exists(f)))

and then also did

@@ -981,6 +980,10 @@ def get_file_checksums(filelist, pn):

 checksums = []
 for pth in filelist.split():
+exist = pth.split(":")[1]
+if exist == "False":
+continue
+pth = pth.split(":")[0]
 if '*' in pth:
 # Handle globs
 for f in glob.glob(pth):

which practically guaranteed that the "if '*' in pth" would be dead code.

Richard, do you agree that this is a bug in the signature computation?

As I wrote previously, there's no warning anywhere that using globs in
SRC_URI will fail to take the contents of the referenced file into
account in hashes, but there's obviously still a some code that makes
file://*.c work wrt. unpacking (and a test case for that).

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#140490): 
https://lists.openembedded.org/g/openembedded-core/message/140490
Mute This Topic: https://lists.openembedded.org/mt/75235594/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] [PATCH] [dunfell] coreutils: don't split stdbuf to own package with single-binary

2020-07-09 Thread Rasmus Villemoes
Commit 992cec44 (coreutils: Move stdbuf into an own package
coreutils-stdbuf) breaks package-qa when the single-binary
PACKAGECONFIG is used:

ERROR: coreutils-8.32-r0 do_package_qa: QA Issue: /usr/bin/stdbuf contained in 
package coreutils-stdbuf requires /usr/bin/coreutils, but no providers found in 
RDEPENDS_coreutils-stdbuf? [file-rdeps]
ERROR: coreutils-8.32-r0 do_package_qa: QA run found fatal errors. Please 
consider fixing them.

With that PACKAGECONFIG, /usr/bin/stdbuf is just a simple "script"
containing the single line

  #!/usr/bin/coreutils --coreutils-prog-shebang=stdbuf

Since there's no point splitting stdbuf to its own package when all
the functionality is in the single big coreutils binary anyway, fix
this by not creating the separate stdbuf package for the single-binary
case. But also make sure that the coreutils-stdbuf item always exists
so recipes can always RDEPEND on coreutils-stdbuf.

Signed-off-by: Rasmus Villemoes 
Signed-off-by: Richard Purdie 
(cherry picked from commit 74d24b5b895198898944260136d05e991a203c11)
---
 meta/recipes-core/coreutils/coreutils_8.31.bb | 15 +--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/coreutils/coreutils_8.31.bb 
b/meta/recipes-core/coreutils/coreutils_8.31.bb
index 2b196b7512..0c8452da98 100644
--- a/meta/recipes-core/coreutils/coreutils_8.31.bb
+++ b/meta/recipes-core/coreutils/coreutils_8.31.bb
@@ -59,9 +59,20 @@ base_bindir_progs = "cat chgrp chmod chown cp date dd echo 
false hostname kill l
 
 sbindir_progs= "chroot"
 
-PACKAGE_BEFORE_PN_class-target += "coreutils-stdbuf"
+# Split stdbuf into its own package, so one can include
+# coreutils-stdbuf without getting the rest of coreutils, but make
+# coreutils itself pull in stdbuf, so IMAGE_INSTALL += "coreutils"
+# always provides all coreutils
+PACKAGE_BEFORE_PN_class-target += "${@bb.utils.contains('PACKAGECONFIG', 
'single-binary', '', 'coreutils-stdbuf', d)}"
 FILES_coreutils-stdbuf = "${bindir}/stdbuf ${libdir}/coreutils/libstdbuf.so"
-RDEPENDS_coreutils_class-target += "coreutils-stdbuf"
+RDEPENDS_coreutils_class-target += "${@bb.utils.contains('PACKAGECONFIG', 
'single-binary', '', 'coreutils-stdbuf', d)}"
+
+# However, when the single-binary PACKAGECONFIG is used, stdbuf
+# functionality is built into the single coreutils binary, so there's
+# no point splitting /usr/bin/stdbuf to its own package. Instead, add
+# an RPROVIDE so that rdepending on coreutils-stdbuf will work
+# regardless of whether single-binary is in effect.
+RPROVIDES_coreutils += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', 
'coreutils-stdbuf', '', d)}"
 
 # Let aclocal use the relative path for the m4 file rather than the
 # absolute since coreutils has a lot of m4 files, otherwise there might
-- 
2.23.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#140465): 
https://lists.openembedded.org/g/openembedded-core/message/140465
Mute This Topic: https://lists.openembedded.org/mt/75392946/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] [PATCH v2] coreutils: don't split stdbuf to own package with single-binary

2020-07-06 Thread Rasmus Villemoes
Commit 992cec44 (coreutils: Move stdbuf into an own package
coreutils-stdbuf) breaks package-qa when the single-binary
PACKAGECONFIG is used:

ERROR: coreutils-8.32-r0 do_package_qa: QA Issue: /usr/bin/stdbuf contained in 
package coreutils-stdbuf requires /usr/bin/coreutils, but no providers found in 
RDEPENDS_coreutils-stdbuf? [file-rdeps]
ERROR: coreutils-8.32-r0 do_package_qa: QA run found fatal errors. Please 
consider fixing them.

With that PACKAGECONFIG, /usr/bin/stdbuf is just a simple "script"
containing the single line

  #!/usr/bin/coreutils --coreutils-prog-shebang=stdbuf

Since there's no point splitting stdbuf to its own package when all
the functionality is in the single big coreutils binary anyway, fix
this by not creating the separate stdbuf package for the single-binary
case. But also make sure that the coreutils-stdbuf item always exists
so recipes can always RDEPEND on coreutils-stdbuf.

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-core/coreutils/coreutils_8.32.bb | 15 +--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/coreutils/coreutils_8.32.bb 
b/meta/recipes-core/coreutils/coreutils_8.32.bb
index 2422f8634e..f8ae3fd94c 100644
--- a/meta/recipes-core/coreutils/coreutils_8.32.bb
+++ b/meta/recipes-core/coreutils/coreutils_8.32.bb
@@ -59,9 +59,20 @@ base_bindir_progs = "cat chgrp chmod chown cp date dd echo 
false hostname kill l
 
 sbindir_progs= "chroot"
 
-PACKAGE_BEFORE_PN_class-target += "coreutils-stdbuf"
+# Split stdbuf into its own package, so one can include
+# coreutils-stdbuf without getting the rest of coreutils, but make
+# coreutils itself pull in stdbuf, so IMAGE_INSTALL += "coreutils"
+# always provides all coreutils
+PACKAGE_BEFORE_PN_class-target += "${@bb.utils.contains('PACKAGECONFIG', 
'single-binary', '', 'coreutils-stdbuf', d)}"
 FILES_coreutils-stdbuf = "${bindir}/stdbuf ${libdir}/coreutils/libstdbuf.so"
-RDEPENDS_coreutils_class-target += "coreutils-stdbuf"
+RDEPENDS_coreutils_class-target += "${@bb.utils.contains('PACKAGECONFIG', 
'single-binary', '', 'coreutils-stdbuf', d)}"
+
+# However, when the single-binary PACKAGECONFIG is used, stdbuf
+# functionality is built into the single coreutils binary, so there's
+# no point splitting /usr/bin/stdbuf to its own package. Instead, add
+# an RPROVIDE so that rdepending on coreutils-stdbuf will work
+# regardless of whether single-binary is in effect.
+RPROVIDES_coreutils += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', 
'coreutils-stdbuf', '', d)}"
 
 # Let aclocal use the relative path for the m4 file rather than the
 # absolute since coreutils has a lot of m4 files, otherwise there might
-- 
2.23.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#140322): 
https://lists.openembedded.org/g/openembedded-core/message/140322
Mute This Topic: https://lists.openembedded.org/mt/75328937/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] [PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

2020-07-06 Thread Rasmus Villemoes
On 04/07/2020 23.10, Adrian Bunk wrote:
> On Fri, Jul 03, 2020 at 11:48:36AM +0100, Richard Purdie wrote:
>> ...
>> I hadn't realised there was already the reverse dependency. I think we
>> need to let the package exist in both cases so the two dependencies
>> need to reverse direction depending on whether single-binary is set.
>> ...
> 
> What about no separate coreutils-stdbuf package with single-binary,
> instead an RPROVIDES of coreutils-stdbuf?

That would probably work. That's essentially the first patch, plus a

RPROVIDES_coreutils += "${@bb.utils.contains('PACKAGECONFIG',
'single-binary', 'coreutils-stdbuf', '', d)}"

line, right?

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#140320): 
https://lists.openembedded.org/g/openembedded-core/message/140320
Mute This Topic: https://lists.openembedded.org/mt/75275273/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] [PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

2020-07-03 Thread Rasmus Villemoes
On 03/07/2020 12.48, Richard Purdie wrote:

>>> Whilst I realise there is a problem here is the correct fix not:
>>>
>>> RDEPENDS_coreutils-stdbuf_class-target += 
>>> "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils', 
>>> d)}"
>>>
>>> ?
>>
>> [Well, the coreutils should be in the true branch of that.] I dunno, it
>> creates a cyclic rdepends between coreutils and coreutils-stdbuf. Is
>> that ok? Seems a bit ugly to me, even if it would work.
> 
> I hadn't realised there was already the reverse dependency. I think we
> need to let the package exist in both cases so the two dependencies
> need to reverse direction depending on whether single-binary is set.
> 
> So something like:
> 
> RDEPENDS_coreutils_class-target += "${@bb.utils.contains('PACKAGECONFIG', 
> 'single-binary', '', 'coreutils-stdbuf', d)}"
> RDEPENDS_coreutils-stdbuf_class-target += 
> "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', 'coreutils', '', d)}"
> 
> ?

The argument for creating the coreutils->coreutils-stdbuf RDEPENDS was
that people would get the same stuff in their rootfs as they used to
(but they could also just get stdbuf if they needed that but use busybox
for all the rest).

The above would mean people who set single-binary and include coreutils
in their rootfs no longer get the stdbuf utility (the functionality is
in the binary, but they no longer get the /usr/bin/stdbuf wrapper
script). But I suppose few enough people use single-binary (or the qa
issue would surely have surfaced sometime in the past half year) that we
can live with having to explicitly add coreutils-stdbuf. But then
perhaps we can just unconditionally drop the coreutils->coreutils-stdbuf
dependency, and have anybody who need stdbuf include it explicitly?

In general, I do support splitting things to their own packages without
automatically pulling them back in by the main package; it can be a
little annoying for downstreams, but surely everybody have tests that
ensure the utilities they need are in their images. [I'm not being
sarcastic.]

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#140280): 
https://lists.openembedded.org/g/openembedded-core/message/140280
Mute This Topic: https://lists.openembedded.org/mt/75275273/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] [PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

2020-07-03 Thread Rasmus Villemoes
On 03/07/2020 11.19, Richard Purdie wrote:
> On Fri, 2020-07-03 at 09:36 +0200, Rasmus Villemoes wrote:
>> Commit 992cec44 (coreutils: Move stdbuf into an own package
>> coreutils-stdbuf) breaks package-qa when the single-binary
>> PACKAGECONFIG is used:
>>
>> ERROR: coreutils-8.31-r0 do_package_qa: QA Issue: /usr/bin/stdbuf
>> contained in package coreutils-stdbuf requires /usr/bin/coreutils,
>> but no providers found in RDEPENDS_coreutils-stdbuf? [file-rdeps]
>> ERROR: coreutils-8.31-r0 do_package_qa: QA run found fatal errors.
>> Please consider fixing them.
> 
> Does this problem exist for other packages in coreutils?

What other packages? stdbuf is the only one being split to its own package:

coreutils/8.31-r0$ ls -l packages-split/
total 32
drwxr-xr-x 4 ravi abcdef 4096 Jul  3 09:49 coreutils
drwxr-xr-x 3 ravi abcdef 4096 Jul  3 09:50 coreutils-dbg
drwxr-xr-x 2 ravi abcdef 4096 Jul  3 09:49 coreutils-dev
drwxr-xr-x 3 ravi abcdef 4096 Jul  3 09:49 coreutils-doc
drwxr-xr-x 2 ravi abcdef 4096 Jul  3 09:49 coreutils-locale
-rw-r--r-- 1 ravi abcdef   48 Jul  3 09:49 coreutils.shlibdeps
drwxr-xr-x 3 ravi abcdef 4096 Jul  3 09:50 coreutils-src
drwxr-xr-x 2 ravi abcdef 4096 Jul  3 09:49 coreutils-staticdev


> I'd suspect it
> does in which case why is stdbuf special?

Because it's the only util that gets special packaging treatment. Unless
there's some magic 'please auto-split all utils to their own packages'
that I don't know about and which we don't happen to set.

> Whilst I realise there is a problem here is the correct fix not:
> 
> RDEPENDS_coreutils-stdbuf_class-target += 
> "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils', d)}"
> 
> ?

[Well, the coreutils should be in the true branch of that.] I dunno, it
creates a cyclic rdepends between coreutils and coreutils-stdbuf. Is
that ok? Seems a bit ugly to me, even if it would work.

> As Alex says, this would need to be merged in master before we can even
> consider it for dunfell.

I'll do it for master once I know which way you want to fix it.

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#140256): 
https://lists.openembedded.org/g/openembedded-core/message/140256
Mute This Topic: https://lists.openembedded.org/mt/75275273/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] [PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

2020-07-03 Thread Rasmus Villemoes
Commit 992cec44 (coreutils: Move stdbuf into an own package
coreutils-stdbuf) breaks package-qa when the single-binary
PACKAGECONFIG is used:

ERROR: coreutils-8.31-r0 do_package_qa: QA Issue: /usr/bin/stdbuf contained in 
package coreutils-stdbuf requires /usr/bin/coreutils, but no providers found in 
RDEPENDS_coreutils-stdbuf? [file-rdeps]
ERROR: coreutils-8.31-r0 do_package_qa: QA run found fatal errors. Please 
consider fixing them.
ERROR: Logfile of failure stored in: 
/mnt/ext4/projects/deif/pdsystems/yocto/tmp-glibc/work/ppce300c3-oe-linux/coreutils/8.31-r0/temp/log.do_package_qa.5594
ERROR: Task 
(/mnt/ext4/projects/deif/pdsystems/yocto/meta-poky/meta/recipes-core/coreutils/coreutils_8.31.bb:do_package_qa)
 failed with exit code '1'

With that PACKAGECONFIG, /usr/bin/stdbuf is just a simple "script"
containing the single line

  #!/usr/bin/coreutils --coreutils-prog-shebang=stdbuf

Since there's no point splitting stdbuf to its own package when all
the functionality is in the single big coreutils binary anyway, fix
this by not creating the separate stdbuf package for the single-binary
case.

Signed-off-by: Rasmus Villemoes 
---
The same issue exists in master, which has been bumped to 8.32, but
other than that the patch should apply cleanly there as well.

 meta/recipes-core/coreutils/coreutils_8.31.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/coreutils/coreutils_8.31.bb 
b/meta/recipes-core/coreutils/coreutils_8.31.bb
index 57b2c1bdba..80e8161367 100644
--- a/meta/recipes-core/coreutils/coreutils_8.31.bb
+++ b/meta/recipes-core/coreutils/coreutils_8.31.bb
@@ -58,9 +58,9 @@ base_bindir_progs = "cat chgrp chmod chown cp date dd echo 
false hostname kill l
 
 sbindir_progs= "chroot"
 
-PACKAGE_BEFORE_PN_class-target += "coreutils-stdbuf"
+PACKAGE_BEFORE_PN_class-target += "${@bb.utils.contains('PACKAGECONFIG', 
'single-binary', '', 'coreutils-stdbuf', d)}"
 FILES_coreutils-stdbuf = "${bindir}/stdbuf ${libdir}/coreutils/libstdbuf.so"
-RDEPENDS_coreutils_class-target += "coreutils-stdbuf"
+RDEPENDS_coreutils_class-target += "${@bb.utils.contains('PACKAGECONFIG', 
'single-binary', '', 'coreutils-stdbuf', d)}"
 
 # Let aclocal use the relative path for the m4 file rather than the
 # absolute since coreutils has a lot of m4 files, otherwise there might
-- 
2.23.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#140251): 
https://lists.openembedded.org/g/openembedded-core/message/140251
Mute This Topic: https://lists.openembedded.org/mt/75275273/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] file checksums versus SRC_URI globs

2020-07-02 Thread Rasmus Villemoes
On 01/07/2020 16.03, Quentin Schulz wrote:
> Hi Rasmus,
> 
> On Wed, Jul 01, 2020 at 03:51:19PM +0200, Rasmus Villemoes wrote:
>> Hi,
>>
>> We have a recipe that uses
>>
>>   SRC_URI += "file://somedir/*"
>>
> 
> Glob aren't supported. Use "file://somedir/" instead.

Thanks, that actually works for one of the cases we have (there are
others that use globs which cannot be solved quite that simply, but for
now I'm just listing files explicitly instead).

However, I'm not sure that "globs aren't supported". The commit I
referenced clearly tried to make that work (better), it also "works" in
the sense of unpacking the expected things when building from scratch -
there's even

def test_local_wildcard(self):
tree = self.fetchUnpack(['file://a', 'file://dir/*'])
self.assertEqual(tree, ['a',  'dir/c', 'dir/d', 'dir/subdir/e'])

in bitbake/lib/bb/tests/fetch.py. And the two upstream recipes
connman-gnome_0.7.bb and matchbox-desktop_2.2.bb both use that exact
pattern.

So either

- this is a plain bug in the signature computation,
- documentation lacks a big fat warning that using globs in file://
should be avoided unless one has a fetish for tracking down bugs due to
confused sstate, or
- all code that makes glob patterns work should be ripped out so that it
is indeed not supported, thus eliminating this footgun.

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#140215): 
https://lists.openembedded.org/g/openembedded-core/message/140215
Mute This Topic: https://lists.openembedded.org/mt/75235594/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] file checksums versus SRC_URI globs

2020-07-01 Thread Rasmus Villemoes
Hi,

We have a recipe that uses

  SRC_URI += "file://somedir/*"

and we have noticed that changing one of the files inside somedir does
not cause the recipe to be rebuilt - in fact, no task hashes are
affected; we discovered this because a clean build worked correctly,
while one that uses an sstate mirror would end up with an old version of
those files on the target.

Digging around, it _seems_ that this should be supported, e.g.

commit bb66113bde5361b869dce2bdaece5b938f077ea8
Author: Richard Purdie 
Date:   Sun Apr 6 11:08:22 2014 +0100

bitbake: fetch2: Fix bug in file checksum generation

However, it also seems that something since then has broken the
checksumming of files specified via a glob pattern. At least if I take
the recipe mentioned in that commit and run "bitbake-dumpsig -t
connman-gnome do_fetch", I get

Variable SRC_URI value is
git://github.com/connectivity/connman-gnome.git
file://0001-Removed-icon-from-connman-gnome-about-applet.patch
 file://null_check_for_ipv4_config.patchfile://images/*
  file://connman-gnome-fix-dbus-interface-name.patch
file://0001-Port-to-Gtk3.patch
...
This task depends on the checksums of files:
[('0001-Removed-icon-from-connman-gnome-about-applet.patch',
'4c1b4d89f92949f34d5c232d85398bb4'),
('connman-gnome-fix-dbus-interface-name.patch',
'51f2ebbc91ec35184876dfdf8aec35fd'),
('null_check_for_ipv4_config.patch',
'a23271e41c9fe81551244d875106af10'), ('0001-Port-to-Gtk3.patch',
'fa8ffcd77c330fd5063c7a7c62dd7c69')]

i.e., none of the files in images/ are mentioned, contrary to what was
apparently the case back in 2014.

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#140182): 
https://lists.openembedded.org/g/openembedded-core/message/140182
Mute This Topic: https://lists.openembedded.org/mt/75235594/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] [PATCH] curl: add debug info

2020-06-17 Thread Rasmus Villemoes
Currently, curl (and libcurl) is built without debug info, making the
curl-dbg package rather useless. Since debug symbols are automatically
stripped and put in that package by the build system, making sure that
curl is built with -g shouldn't hurt anything, but will help those
that try to debug a libcurl-using application and hence explicitly
include curl-dbg in their rootfs.

Unfortunately, setting --enable-debug then changes the default value
of the optimize option from (assume yes) to (assume no), while also
changing the default value of the curldebug option [which is a
separate thing that actually changes generated code to add some memory
tracking] from (assume no) to (assume yes). So explicitly pass the
appropriate options that make those two have the same value as they
used to have by default.

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-support/curl/curl_7.70.0.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-support/curl/curl_7.70.0.bb 
b/meta/recipes-support/curl/curl_7.70.0.bb
index baf72f8e75..bb25677ef7 100644
--- a/meta/recipes-support/curl/curl_7.70.0.bb
+++ b/meta/recipes-support/curl/curl_7.70.0.bb
@@ -57,6 +57,9 @@ EXTRA_OECONF = " \
 --with-ca-bundle=${sysconfdir}/ssl/certs/ca-certificates.crt \
 --without-libmetalink \
 --without-libpsl \
+--enable-debug \
+--enable-optimize \
+--disable-curldebug \
 "
 
 do_install_append_class-target() {
-- 
2.23.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#139587): 
https://lists.openembedded.org/g/openembedded-core/message/139587
Mute This Topic: https://lists.openembedded.org/mt/74933778/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] [PATCH v3] glibc: move ld.so.conf back to main package

2020-06-05 Thread Rasmus Villemoes
On 04/06/2020 22.05, Andre McCurdy wrote:

> but I can try to summarise the details related to the actual patch...

[snip]

> There's a genuine bug and I don't see an obviously better fix, so I
> think the patch should be merged.

Thank you for that summary, that's much better than what I could have
written, and also thanks for the conclusion.

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#139258): 
https://lists.openembedded.org/g/openembedded-core/message/139258
Mute This Topic: https://lists.openembedded.org/mt/74625854/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] [PATCH v3] glibc: move ld.so.conf back to main package

2020-06-04 Thread Rasmus Villemoes
On 03/06/2020 22.44, Andre McCurdy wrote:
> On Wed, Jun 3, 2020 at 12:38 AM Rasmus Villemoes
>  wrote:
>> On 02/06/2020 23.46, Andre McCurdy wrote:
>>> On Tue, Jun 2, 2020 at 2:15 PM Phil Blundell via
>>> lists.openembedded.org  wrote:
>>>>
>>>
>>> It seems like a weird corner case to me. Where do these libraries come
>>> from and why can't they be moved or symlinked into a standard path?
>>
>> Pre-compiled proprietary binaries/libraries that go in
>> /opt//..., just as /opt is meant for (and no, we can't just
>> install them to /usr/lib, /usr/bin etc - partly because they contain
>> hard-coded absolute paths, but also due to some not-entirely-techical
>> reasons). So there's an /etc/ld.so.conf.d/.conf that needs to be
>> picked up.
>>
>>> If they are somehow special and only used by special applications then
>>> setting rpath or using LD_LIBRARY_PATH just for those applications
>>> would seem to be a better solution than enabling them globally.
>>
>> We don't control the compilation, so rpath is out. We did consider
>> LD_LIBRARY_PATH, but figured that the ld.conf solution is more robust
>> (no problems with setuid binaries or an application that misguidedly
>> sanitizes the environment for subprocesses).
> 
> Unless the supplier of your binaries is completely unresponsive (if
> so, bad luck) then educating them on the usage of rpath and asking for
> a new release is probably the cleanest solution.

For various reasons that is not a viable path.

> If you start your special application via a wrapper script which sets
> LD_LIBRARY_PATH then it won't be seen by anything else, so the
> concerns about other apps messing with it etc should not apply?

"Other apps" could be the apps in /opt themselves, that start other
binaries from under /opt, perhaps via some maze of shell scripts or
whatnot (the usual mess one sees with proprietary stuff) - which is also
the reason I'm worried they might have done some "value-add"
sanitization somewhere.

> Relying on /etc/ld.so.conf.d/*.conf will certainly work too, but has
> two disadvantages: it will add the custom library path globally
> instead of just for the special apps and it's not portable (it's not
> supported by musl etc - although if you only care about your
> proprietary binaries pre-compiled for glibc then that won't matter to
> you).

One could also ask the question: If one is not supposed to use the
drop-in dir ld.so.conf.d/ for exactly what I'm using it for, then why
does Yocto provide a ld.so.conf that mentions it? Sure, for glibc-based
distros (of course, it's from the glibc recipe), but that's fine.

Please remember that this is all just about ensuring that the ldconfig
run at image build time produces the same result, regardless of whether
one has opted to have ldconfig-the-binary in the rootfs or not (and
everybody seems to agree there are valid reasons not to have that).

> Anyway, the original patch has been merged, so this now just a side 
> discussion.

Indeed.

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#139206): 
https://lists.openembedded.org/g/openembedded-core/message/139206
Mute This Topic: https://lists.openembedded.org/mt/74625854/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] [PATCH v3] glibc: move ld.so.conf back to main package

2020-06-03 Thread Rasmus Villemoes
On 02/06/2020 23.46, Andre McCurdy wrote:
> On Tue, Jun 2, 2020 at 2:15 PM Phil Blundell via
> lists.openembedded.org  wrote:
>>
>> On Tue, Jun 02, 2020 at 09:17:44PM +0100, Richard Purdie wrote:
>>> I understand the concern, I am a little torn on this as adding in too
>>> many different controls and options complicates the test matrix and
>>> makes things harder for users.
>>>
>>> You're effectively suggesting a new DISTRO_FEATURE to control this? or
>>> maybe better, perhaps a glibc PACKAGECONFIG?
>>
>> Yes, right.  I don't think it need be a DISTRO_FEATURE because nothing
>> outside glibc needs to care.
>>
>> Here are the scenarios that I think matter, ordered from simplest to
>> most complex:
>>
>> 1. Immutable filesystem, every library installed in the place where
>> ld.so would first look for it anyway (i.e. everything in {/usr}/lib).
>> In this case, we don't want ldconfig (because it can never do anything
>> useful), we don't want ld.so.cache and we don't want ld.so.conf
>> because they would cause ld.so to do extra file loads and computation
>> but end up with the same result that it would anyway.
>>
>> 2. Immutable filesystem but some libraries are in places that ld.so
>> wouldn't automatically know about.  In this case we do want ld.so.conf
>> and ld.so.cache, but we still don't want ldconfig.
>>
>> 3. Mutable filesystem where arbitrary binaries can be installed in
>> arbitrary places.  It's probably debatable whether ldconfig is needed
>> in all these cases, but clearly it's needed in some and I think at
>> this point it can be left to a DISTRO decision how exactly they want
>> to optimize things.
>>
>> I think right now oe-core supports #1 and #3.  The proposed patch
>> seems to be aimed at #2, which is a completely valid usecase,
> 
> It seems like a weird corner case to me. Where do these libraries come
> from and why can't they be moved or symlinked into a standard path?

Pre-compiled proprietary binaries/libraries that go in
/opt//..., just as /opt is meant for (and no, we can't just
install them to /usr/lib, /usr/bin etc - partly because they contain
hard-coded absolute paths, but also due to some not-entirely-techical
reasons). So there's an /etc/ld.so.conf.d/.conf that needs to be
picked up.

> If they are somehow special and only used by special applications then
> setting rpath or using LD_LIBRARY_PATH just for those applications
> would seem to be a better solution than enabling them globally.

We don't control the compilation, so rpath is out. We did consider
LD_LIBRARY_PATH, but figured that the ld.conf solution is more robust
(no problems with setuid binaries or an application that misguidedly
sanitizes the environment for subprocesses).

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#139149): 
https://lists.openembedded.org/g/openembedded-core/message/139149
Mute This Topic: https://lists.openembedded.org/mt/74625854/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] [PATCH v3] glibc: move ld.so.conf back to main package

2020-06-03 Thread Rasmus Villemoes
On 02/06/2020 23.15, Phil Blundell wrote:
> On Tue, Jun 02, 2020 at 09:17:44PM +0100, Richard Purdie wrote:
>> I understand the concern, I am a little torn on this as adding in too
>> many different controls and options complicates the test matrix and
>> makes things harder for users.
>>
>> You're effectively suggesting a new DISTRO_FEATURE to control this? or
>> maybe better, perhaps a glibc PACKAGECONFIG?
> 
> Yes, right.  I don't think it need be a DISTRO_FEATURE because nothing
> outside glibc needs to care.
> 
> Here are the scenarios that I think matter, ordered from simplest to
> most complex:
> 
> 1. Immutable filesystem, every library installed in the place where
> ld.so would first look for it anyway (i.e. everything in {/usr}/lib).
> In this case, we don't want ldconfig (because it can never do anything
> useful), we don't want ld.so.cache and we don't want ld.so.conf
> because they would cause ld.so to do extra file loads and computation
> but end up with the same result that it would anyway.
> 
> I think right now oe-core supports #1 and #3.  The proposed patch
> seems to be aimed at #2, which is a completely valid usecase, but
> my concern is that we don't want to pessimize #1 in the process.

With or without my patch, there's an ldconfig invocation at image build
time, and lacking an /etc/ld.so.conf in the -r filesystem, ldconfig just
looks at the default locations (/lib and /usr/lib, perhaps a few more
depending on $arch). But you still get an ld.so.cache generated listing
the libs in /lib and /usr/lib. So I don't understand this performance
concern at all - at the very most, the presence of the config file
causes a microscopic performance hit at image build time when ldconfig
goes looking for the /etc/ld.so.conf.d/* files and (usually) find none.

Also, IME, the cache file usually speed things up since ld.so avoids a
filesystem access to figure out that
/{usr,}/lib//libfoo.so doesn't exist (of course
there's an upfront cost of mmapping and parsing the ld.so.cache, so for
programs with very few dynlib dependencies, it might be faster to not
have the cache). See below.

If you really don't want the ld.so.cache, ldconfig has the -N option.
But adding some knob to put that in is completely orthogonal to this patch.

Rasmus

Here's the first few lines of "strace -o /tmp/cache.txt factor 12345":

execve("/usr/bin/factor", ["factor", "12345"], 0xbff6cf3c /* 14 vars */) = 0
brk(NULL)   = 0x10173000
access("/etc/ld.so.preload", R_OK)  = -1 ENOENT (No such file or
directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=6527, ...}) = 0
mmap2(NULL, 6527, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7ef2000
close(3)= 0
openat(AT_FDCWD, "/lib/librt.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3,
"\177ELF\1\2\1\3\0\0\0\0\0\0\0\0\0\3\0\24\0\0\0\1\0\0\36\300\0\0\0004"...,
512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=38596, ...}) = 0
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7ef
mmap2(0xffdf000, 65768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE,
3, 0) = 0xffdf000
mprotect(0xffe7000, 28672, PROT_NONE)   = 0
mmap2(0xffee000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x8000) = 0xffee000
close(3)= 0
openat(AT_FDCWD, "/lib/libcap.so.2", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3,
"\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\24\0\0\0\1\0\0\21P\0\0\0004"...,
512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=22060, ...}) = 0
mmap2(0xffbe000, 65820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE,
3, 0) = 0xffbe000
mprotect(0xffc3000, 40960, PROT_NONE)   = 0
mmap2(0xffcd000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4000) = 0xffcd000
close(3)= 0
openat(AT_FDCWD, "/usr/lib/libgmp.so.10",
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3, "\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\24\0\0\0\1\0\0\226
\0\0\0004"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=551740, ...}) = 0
mmap2(0xff1d000, 591268, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE,
3, 0) = 0xff1d000
mprotect(0xffa3000, 36864, PROT_NONE)   = 0
mmap2(0xffac000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x85000) = 0xffac000
close(3)= 0

Compare that to what happens when I move the ld.so.cache out of the way:

execve("/usr/bin/factor", ["factor", "12345"], 0xbfc7c3bc /* 14 vars */) = 0
brk(NULL)   = 0x10173000
access("/etc/ld.so.preload", R_OK)  = -1 ENOENT (No such file or
directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_LARGEFILE|O_CLOEXEC) =
-1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/tls/ppc603/librt.so.1",
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat64("/lib/tls/ppc603", 0xbf98df78)   = -1 ENOENT (No such file or

Re: [OE-core] [PATCH v3] glibc: move ld.so.conf back to main package

2020-06-02 Thread Rasmus Villemoes
On 02/06/2020 19.58, Khem Raj wrote:
> On Tue, Jun 2, 2020 at 5:17 AM Rasmus Villemoes
>  wrote:
>>
>> There are cases where one doesn't want ldconfig on target (e.g. for
>> read-only root filesystems, it's rather pointless), yet one still
>> needs ld.so.conf to be present at image build time:
>>
>> When some recipe installs libraries to a non-standard location, and
>> dutifully drops in a file in /etc/ld.so.conf.d/foo.conf, we need the
>> ld.so.conf containing the
>>
>>   include /etc/ld.so.conf.d/*.conf
>>
>> stanza to get those other locations picked up.
>>
>> So change the packaging logic so that there's always an ld.so.conf
>> present when the build-time ldconfig runs.
>>
>> The ld.so.conf and ld.so.conf.d/*.conf files don't take up much
>> room (at least not compared to the 700K binary ldconfig), and they
>> might be needed in case ldconfig is installable, so leave them
>> alone.
>>
>> In case of a read-only rootfs, one could add some logic to remove them
>> if one really wants to shave those few dozens of bytes off.
>>
>> While here, fix typos in the bb.note (add spaces) so one can just
>> copy-paste the line from the log-file and redo the command.
>>
> 
> The patch, in essence, looks ok, have you considered the feeds based
> distro's case
> will it keep working for them, or if there are changes e.g. they must
> include ldconfig
> compulsory etc. then it should be documented.

I don't think I understand the question. All I'm doing (now) is making
sure the rootfs always (well, it comes bundled with glibc, so for all
glibc-based systems) has a ld.so.conf. If there's an update of that
ld.so.conf, that will change the hash of the glibc recipe (it's a
SRC_URI) and hence of both the glibc and ldconfig packages. A change of
the provided ld.so.conf will lead to glibc being rebuilt (no?), which I
expect in turn will cause glibc on target to (need to) be updated (no?).
That seems to be the case with or without this patch.

But maybe I have no clue what a "feeds based distro" is, I've only ever
done wholesale replace-the-entire-image (in an A/B setup).

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#139119): 
https://lists.openembedded.org/g/openembedded-core/message/139119
Mute This Topic: https://lists.openembedded.org/mt/74625854/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] [PATCH v3] glibc: move ld.so.conf back to main package

2020-06-02 Thread Rasmus Villemoes
There are cases where one doesn't want ldconfig on target (e.g. for
read-only root filesystems, it's rather pointless), yet one still
needs ld.so.conf to be present at image build time:

When some recipe installs libraries to a non-standard location, and
dutifully drops in a file in /etc/ld.so.conf.d/foo.conf, we need the
ld.so.conf containing the

  include /etc/ld.so.conf.d/*.conf

stanza to get those other locations picked up.

So change the packaging logic so that there's always an ld.so.conf
present when the build-time ldconfig runs.

The ld.so.conf and ld.so.conf.d/*.conf files don't take up much
room (at least not compared to the 700K binary ldconfig), and they
might be needed in case ldconfig is installable, so leave them
alone.

In case of a read-only rootfs, one could add some logic to remove them
if one really wants to shave those few dozens of bytes off.

While here, fix typos in the bb.note (add spaces) so one can just
copy-paste the line from the log-file and redo the command.

Signed-off-by: Rasmus Villemoes 
---

v3: rebase to master

v2: don't delete the configuration file(s) - that's also a much
simpler patch.

 meta/lib/oe/rootfs.py | 2 +-
 meta/recipes-core/glibc/glibc-package.inc | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index cd65e62030..a0ac33ada6 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -297,7 +297,7 @@ class Rootfs(object, metaclass=ABCMeta):
 
 def _run_ldconfig(self):
 if self.d.getVar('LDCONFIGDEPEND'):
-bb.note("Executing: ldconfig -r" + self.image_rootfs + "-c new -v")
+bb.note("Executing: ldconfig -r " + self.image_rootfs + " -c new 
-v")
 self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
   'new', '-v'])
 
diff --git a/meta/recipes-core/glibc/glibc-package.inc 
b/meta/recipes-core/glibc/glibc-package.inc
index ff25fd4187..5f1ab60dc6 100644
--- a/meta/recipes-core/glibc/glibc-package.inc
+++ b/meta/recipes-core/glibc/glibc-package.inc
@@ -23,9 +23,9 @@ ARCH_DYNAMIC_LOADER_aarch64 = "ld-linux-${TARGET_ARCH}.so.1"
 libc_baselibs_append = " ${@oe.utils.conditional('ARCH_DYNAMIC_LOADER', '', 
'', '${root_prefix}/lib/${ARCH_DYNAMIC_LOADER}', d)}"
 INSANE_SKIP_${PN}_append_aarch64 = " libdir"
 
-FILES_${PN} = "${libc_baselibs} ${libexecdir}/*"
+FILES_${PN} = "${libc_baselibs} ${libexecdir}/* ${sysconfdir}/ld.so.conf"
 RRECOMMENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'ldconfig', 
'${MLPREFIX}ldconfig', '', d)}"
-FILES_ldconfig = "${base_sbindir}/ldconfig ${sysconfdir}/ld.so.conf"
+FILES_ldconfig = "${base_sbindir}/ldconfig"
 FILES_ldd = "${bindir}/ldd"
 FILES_libsegfault = "${base_libdir}/libSegFault*"
 FILES_libcidn = "${base_libdir}/libcidn-*.so ${base_libdir}/libcidn.so.*"
-- 
2.23.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#139091): 
https://lists.openembedded.org/g/openembedded-core/message/139091
Mute This Topic: https://lists.openembedded.org/mt/74625854/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] [PATCH v2] glibc: move ld.so.conf back to main package

2020-06-02 Thread Rasmus Villemoes
There are cases where one doesn't want ldconfig on target (e.g. for
read-only root filesystems, it's rather pointless), yet one still
needs ld.so.conf to be present at image build time:

When some recipe installs libraries to a non-standard location, and
dutifully drops in a file in /etc/ld.so.conf.d/foo.conf, we need the
ld.so.conf containing the

  include /etc/ld.so.conf.d/*.conf

stanza to get those other locations picked up.

So change the packaging logic so that there's always an ld.so.conf
present when the build-time ldconfig runs.

The ld.so.conf and ld.so.conf.d/*.conf files don't take up much
room (at least not compared to the 700K binary ldconfig), and they
might be needed in case ldconfig is installable, so leave them
alone.

In case of a read-only rootfs, one could add some logic to remove them
if one really wants to shave those few dozens of bytes off.

While here, fix typos in the bb.note (add spaces) so one can just
copy-paste the line from the log-file and redo the command.

Signed-off-by: Rasmus Villemoes 
---
v2: don't delete the configuration file(s) - that's also a much
simpler patch.

 meta/lib/oe/rootfs.py | 2 +-
 meta/recipes-core/glibc/glibc-package.inc | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index cd65e62030..a0ac33ada6 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -297,7 +297,7 @@ class Rootfs(object, metaclass=ABCMeta):
 
 def _run_ldconfig(self):
 if self.d.getVar('LDCONFIGDEPEND'):
-bb.note("Executing: ldconfig -r" + self.image_rootfs + "-c new -v")
+bb.note("Executing: ldconfig -r " + self.image_rootfs + " -c new 
-v")
 self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
   'new', '-v'])
 
diff --git a/meta/recipes-core/glibc/glibc-package.inc 
b/meta/recipes-core/glibc/glibc-package.inc
index aa8e059216..387e90a0ab 100644
--- a/meta/recipes-core/glibc/glibc-package.inc
+++ b/meta/recipes-core/glibc/glibc-package.inc
@@ -23,9 +23,9 @@ ARCH_DYNAMIC_LOADER_aarch64 = "ld-linux-${TARGET_ARCH}.so.1"
 libc_baselibs_append = " ${@oe.utils.conditional('ARCH_DYNAMIC_LOADER', '', 
'', '${root_prefix}/lib/${ARCH_DYNAMIC_LOADER}', d)}"
 INSANE_SKIP_${PN}_append_aarch64 = " libdir"
 
-FILES_${PN} = "${libc_baselibs} ${libexecdir}/*"
+FILES_${PN} = "${libc_baselibs} ${libexecdir}/* ${sysconfdir}/ld.so.conf"
 RRECOMMENDS_${PN} = "${@bb.utils.filter('DISTRO_FEATURES', 'ldconfig', d)}"
-FILES_ldconfig = "${base_sbindir}/ldconfig ${sysconfdir}/ld.so.conf"
+FILES_ldconfig = "${base_sbindir}/ldconfig"
 FILES_ldd = "${bindir}/ldd"
 FILES_libsegfault = "${base_libdir}/libSegFault*"
 FILES_libcidn = "${base_libdir}/libcidn-*.so ${base_libdir}/libcidn.so.*"
-- 
2.23.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#139087): 
https://lists.openembedded.org/g/openembedded-core/message/139087
Mute This Topic: https://lists.openembedded.org/mt/74624263/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] [PATCH] glibc: move ld.so.conf back to main package

2020-05-18 Thread Rasmus Villemoes
On 18/05/2020 14.29, Andreas Oberritter wrote:
> Hello Rasmus,
> 
> On Mon, 18 May 2020 14:12:43 +0200
> Rasmus Villemoes  wrote:
> 
>> I'm certainly open to other ways of solving this. But can we agree that
>> it is a bug that the ldconfig done at build-time does not take
>> /etc/ld.so.conf.d/* into account, and that that should not depend on
>> whether one has ldconfig-the-binary on target?
> 
> have you tried installing ldconfig and adding it to ROOTFS_RO_UNNEEDED? It 
> might be an improvement to include it in ROOTFS_RO_UNNEEDED by default.

Thanks, but I don't think that will work:

self._uninstall_unneeded()

if self.progress_reporter:
self.progress_reporter.next_stage()

self._insert_feed_uris()

self._run_ldconfig()

so if the ldconfig package (including the ld.so.conf file) is just added
to ROOTFS_RO_UNNEEDED, it will be gone by the time we get to doing the
build-time ld.so.cache generation. And I think it has to be done in this
order - if some of the packages removed by _uninstall_unneeded remove
shared libraries, one doesn't want stale entries in ld.so.cache
referring to those.

> Please consider use-cases where writable filesystems ship without ldconfig, 
> but a user installs it from a package feed when needed.

Do you mean I should leave /etc/ld.so.conf.d/ alone? I can do that. But
I'd say that also in that case the current behaviour is buggy - if any
package in the rootfs ships with an /etc/ld.so.conf.d/ entry, that entry
should be taken into account at image build time, regardless of whether
ldconfig is there at image build time or can usefully be added later via
a package installer.

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#138408): 
https://lists.openembedded.org/g/openembedded-core/message/138408
Mute This Topic: https://lists.openembedded.org/mt/74289052/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] [PATCH] glibc: move ld.so.conf back to main package

2020-05-18 Thread Rasmus Villemoes
On 18/05/2020 13.55, Martin Jansa wrote:
> This won't work, because as soon as glibc is upgraded with package
> management, the ldconfig files will be restored and there won't be any
> do_rootfs hook to remove it.
> 
> Why isn't ldconfig installed in your setup in first place? RRECOMMENDS
> should be enough to pull it into every image by default.

[At the risk of sounding like the lady in the Monty Python spam sketch:]
I don't _want_ it in my setup - it's a gigantic 700K binary that is
useless on a squashfs rootfs where for obvious reasons no new libraries
(or new versions) will ever appear. So no, I don't have ldconfig in my
distro features (so it's not even rrecommended), and I also don't
install recommended packages.

I just need a ld.so.cache built at image build time. Which works just
fine, except that the drop-in dir gets ignored due to lack of
ld.so.conf. After ld.so.cache has been built, the configuration files
are welcome to go the way of the dodo.

And I'm not removing the ldconfig files if ldconfig is in distro
features (or if someone has explicitly RDEPEND'ed on the ldconfig
package), so there's no issue of those files getting "restored" - if you
have ldconfig on target, they never disappear, so them getting replaced
during a package upgrade is not a problem.

I'm certainly open to other ways of solving this. But can we agree that
it is a bug that the ldconfig done at build-time does not take
/etc/ld.so.conf.d/* into account, and that that should not depend on
whether one has ldconfig-the-binary on target?

Rasmus

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#138403): 
https://lists.openembedded.org/g/openembedded-core/message/138403
Mute This Topic: https://lists.openembedded.org/mt/74289052/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] [PATCH] glibc: move ld.so.conf back to main package

2020-05-18 Thread Rasmus Villemoes
There are cases where one doesn't want ldconfig on target (e.g. for
read-only root filesystems, it's rather pointless), yet one still
needs ld.so.conf to be present at image build time:

When some recipe installs libraries to a non-standard location, and
dutifully drops in a file in /etc/ld.so.conf.d/foo.conf, we need the
ld.so.conf containing the

  include /etc/ld.so.conf.d/*.conf

stanza to get those other locations picked up.

So change the packaging logic so that there's always an ld.so.conf
present when the build-time ldconfig runs, then delete it (and the .d
directory) when they would serve no purpose at run-time.

While here, fix a typo in the bb.note so one can just copy-paste the
line from the log-file and redo the command.

Signed-off-by: Rasmus Villemoes 
---
 meta/lib/oe/rootfs.py | 11 ++-
 meta/recipes-core/glibc/glibc-package.inc |  4 ++--
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index cd65e62030..0ff4278d15 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -297,9 +297,18 @@ class Rootfs(object, metaclass=ABCMeta):
 
 def _run_ldconfig(self):
 if self.d.getVar('LDCONFIGDEPEND'):
-bb.note("Executing: ldconfig -r" + self.image_rootfs + "-c new -v")
+bb.note("Executing: ldconfig -r " + self.image_rootfs + " -c new 
-v")
 self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
   'new', '-v'])
+# If DISTRO_FEATURES doesn't contain ldconfig, and the
+# ldconfig binary hasn't been pulled in explicitly, remove the
+# configuration file(s) from the image.
+keep_conf = bb.utils.contains("DISTRO_FEATURES", "ldconfig", True, 
False, self.d)
+keep_conf = keep_conf or 
os.path.exists(os.path.join(self.image_rootfs, self.d.getVar('base_sbindir'), 
'ldconfig'))
+if not keep_conf:
+sysconfdir = os.path.join(self.image_rootfs, 
self.d.getVar('sysconfdir'))
+bb.utils.remove(os.path.join(sysconfdir, 'ld.so.conf'))
+bb.utils.remove(os.path.join(sysconfdir, 'ld.so.conf.d'), 
recurse=True)
 
 def _check_for_kernel_modules(self, modules_dir):
 for root, dirs, files in os.walk(modules_dir, topdown=True):
diff --git a/meta/recipes-core/glibc/glibc-package.inc 
b/meta/recipes-core/glibc/glibc-package.inc
index aa8e059216..387e90a0ab 100644
--- a/meta/recipes-core/glibc/glibc-package.inc
+++ b/meta/recipes-core/glibc/glibc-package.inc
@@ -23,9 +23,9 @@ ARCH_DYNAMIC_LOADER_aarch64 = "ld-linux-${TARGET_ARCH}.so.1"
 libc_baselibs_append = " ${@oe.utils.conditional('ARCH_DYNAMIC_LOADER', '', 
'', '${root_prefix}/lib/${ARCH_DYNAMIC_LOADER}', d)}"
 INSANE_SKIP_${PN}_append_aarch64 = " libdir"
 
-FILES_${PN} = "${libc_baselibs} ${libexecdir}/*"
+FILES_${PN} = "${libc_baselibs} ${libexecdir}/* ${sysconfdir}/ld.so.conf"
 RRECOMMENDS_${PN} = "${@bb.utils.filter('DISTRO_FEATURES', 'ldconfig', d)}"
-FILES_ldconfig = "${base_sbindir}/ldconfig ${sysconfdir}/ld.so.conf"
+FILES_ldconfig = "${base_sbindir}/ldconfig"
 FILES_ldd = "${bindir}/ldd"
 FILES_libsegfault = "${base_libdir}/libSegFault*"
 FILES_libcidn = "${base_libdir}/libcidn-*.so ${base_libdir}/libcidn.so.*"
-- 
2.23.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#138401): 
https://lists.openembedded.org/g/openembedded-core/message/138401
Mute This Topic: https://lists.openembedded.org/mt/74289052/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


[OE-core] Hard-linking to u-boot source files considered harmful

2020-05-01 Thread Rasmus Villemoes
I was wondering why my U-Boot build would always report as coming from a
-dirty directory. The /git source directory was completely clean and
matched the desired commit.

But the -dirty is appended by the scripts/setlocalversion (linux uses
the same script) if

  git diff-index --name-only HEAD

reports anything (mind you, you have to run that command before running
"git status" - the latter will update git's index so the diff-index may
no longer print anything). Indeed, that prints

  Licenses/gpl-2.0.txt

The reason diff-index thinks that might be changed is that, since the
git checkout, that file has had its nlinks (and consequently ctime)
modified:

  File: Licenses/gpl-2.0.txt
  Size: 18092   Blocks: 40 IO Block: 4096   regular file
Device: 811h/2065d  Inode: 41445215Links: 3
Access: (0644/-rw-r--r--)  Uid: ( 1000/ravi)   Gid: ( 1001/  abcdef)
Access: 2020-05-01 12:04:47.708081401 +0200
Modify: 2020-05-01 12:04:45.756078017 +0200
Change: 2020-05-01 12:04:48.032081963 +0200
 Birth: -

And who creates those extra links? do_populate_lic (and perhaps some
other task later yet). Since that is just "after patch before build", it
is almost certain to get run before the compile task gets around to
invoking setlocalversion and thus create the
include/config/uboot.release file. A crude hack that just forces
do_populate_lic after do_install indeed makes the -dirty go away. Or, an
even cruder hack, prepending a "git status" done in ${S} to do_compile
causes git to refresh its index.

I have tons of ways of working around this, but the question is, is the
optimization of using hard links really worth it? If it is, can we at
least have a "do not ever create hard links to stuff under ${S}"
per-recipe variable, defaulting to 1 if the "primary" SRC_URI (yeah,
that concept probably doesn't exist) is a git:// one?

do_populate_lic is not the only offender, btw., there's apparently also
some create-debug-package-with-sources that will do the same thing, but
I think that will by its nature always run after any configure/compile
step that uses git to create some version.

Rasmus
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#137702): 
https://lists.openembedded.org/g/openembedded-core/message/137702
Mute This Topic: https://lists.openembedded.org/mt/73394646/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: 
https://lists.openembedded.org/g/openembedded-core/leave/8023207/1426099254/xyzzy
  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [OE-core] [PATCH v2] glibc: don't ignore global LDFLAGS settings

2020-02-03 Thread Rasmus Villemoes
ping

On 20/01/2020 16.42, Khem Raj wrote:
> this patch is ok but I have reservations since  += now means global
> ldflags will be applied
> so it would need some testing to ensure it works well.
> 
> On Mon, Jan 20, 2020 at 1:23 AM Rasmus Villemoes
>  wrote:
>>
>> Two things: The -Wl,-rpath-link comment is stale (due to per-recipe
>> staging), so we no longer need to set our own LDFLAGS to avoid
>> -Wl,-rpath-link being in there. Second, overriding LDFLAGS should be
>> done at the recipe level so "bitbake -e" can show what is going on.
>> Otherwise debugging why one's global LDFLAGS tweaks are being ignored
>> is needlessly painful.
>>
>> So pull out the LDFLAGS setting from do_compile, and change it to an
>> append instead of assignment.
>>
>> For the benefit of future git blame: the -fuse-ld=bfd setting was
>> added by ac64c3b96b (glibc: always use bfd linker).
>>
>> Signed-off-by: Rasmus Villemoes 
>> ---
>> v2: Rebase to real upstream master. Note to self: "git pull" before rebasing 
>> to master.
>>
>>  meta/recipes-core/glibc/glibc_2.31.bb | 3 +--
>>  1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/meta/recipes-core/glibc/glibc_2.31.bb 
>> b/meta/recipes-core/glibc/glibc_2.31.bb
>> index b8c570db52..cfba4de49b 100644
>> --- a/meta/recipes-core/glibc/glibc_2.31.bb
>> +++ b/meta/recipes-core/glibc/glibc_2.31.bb
>> @@ -93,9 +93,8 @@ do_configure () {
>>  CPPFLAGS="" oe_runconf
>>  }
>>
>> +LDFLAGS += "-fuse-ld=bfd"
>>  do_compile () {
>> -   # -Wl,-rpath-link /lib in LDFLAGS can cause breakage if 
>> another glibc is in staging
>> -   LDFLAGS="-fuse-ld=bfd"
>> base_do_compile
>> echo "Adjust ldd script"
>> if [ -n "${RTLDLIST}" ]
>> --
>> 2.23.0
>>


-- 
Rasmus Villemoes
Software Developer
Prevas A/S
Hedeager 3
DK-8200 Aarhus N
+45 51210274
rasmus.villem...@prevas.dk
www.prevas.dk
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH v2] glibc: don't ignore global LDFLAGS settings

2020-01-20 Thread Rasmus Villemoes
On 20/01/2020 16.42, Khem Raj wrote:
> this patch is ok but I have reservations since  += now means global
> ldflags will be applied

Indeed, that's half the point of the patch.

> so it would need some testing to ensure it works well.

Yes, but I don't think there's any way to figure out if it breaks
something other than trying it. If it does, I can certainly make it just
LDFLAGS = "..." outside do_compile, and then I'll have to .bbappend or
_pn-glibc_append my custom LDFLAGS, which is ok for a single rather
special recipe, but would be a hassle to maintain if this was a common
thing.

Rasmus
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH v2] glibc: don't ignore global LDFLAGS settings

2020-01-20 Thread Rasmus Villemoes
Two things: The -Wl,-rpath-link comment is stale (due to per-recipe
staging), so we no longer need to set our own LDFLAGS to avoid
-Wl,-rpath-link being in there. Second, overriding LDFLAGS should be
done at the recipe level so "bitbake -e" can show what is going on.
Otherwise debugging why one's global LDFLAGS tweaks are being ignored
is needlessly painful.

So pull out the LDFLAGS setting from do_compile, and change it to an
append instead of assignment.

For the benefit of future git blame: the -fuse-ld=bfd setting was
added by ac64c3b96b (glibc: always use bfd linker).

Signed-off-by: Rasmus Villemoes 
---
v2: Rebase to real upstream master. Note to self: "git pull" before rebasing to 
master.

 meta/recipes-core/glibc/glibc_2.31.bb | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/recipes-core/glibc/glibc_2.31.bb 
b/meta/recipes-core/glibc/glibc_2.31.bb
index b8c570db52..cfba4de49b 100644
--- a/meta/recipes-core/glibc/glibc_2.31.bb
+++ b/meta/recipes-core/glibc/glibc_2.31.bb
@@ -93,9 +93,8 @@ do_configure () {
 CPPFLAGS="" oe_runconf
 }
 
+LDFLAGS += "-fuse-ld=bfd"
 do_compile () {
-   # -Wl,-rpath-link /lib in LDFLAGS can cause breakage if 
another glibc is in staging
-   LDFLAGS="-fuse-ld=bfd"
base_do_compile
echo "Adjust ldd script"
if [ -n "${RTLDLIST}" ]
-- 
2.23.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH] glibc: don't ignore global LDFLAGS settings

2020-01-19 Thread Rasmus Villemoes
Two things: The -Wl,-rpath-link comment is stale (due to per-recipe
staging), so we no longer need to set our own LDFLAGS to avoid
-Wl,-rpath-link being in there. Second, overriding LDFLAGS should be
done at the recipe level so "bitbake -e" can show what is going on.
Otherwise debugging why one's global LDFLAGS tweaks are being ignored
is needlessly painful.

So pull out the LDFLAGS setting from do_compile, and change it to an
append instead of assignment.

For the benefit of future git blame: the -fuse-ld=bfd setting was
added by ac64c3b96b (glibc: always use bfd linker).

Signed-off-by: Rasmus Villemoes 
---
 meta/recipes-core/glibc/glibc_2.30.bb | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/recipes-core/glibc/glibc_2.30.bb 
b/meta/recipes-core/glibc/glibc_2.30.bb
index 7913bc2812..c6dd78ca8b 100644
--- a/meta/recipes-core/glibc/glibc_2.30.bb
+++ b/meta/recipes-core/glibc/glibc_2.30.bb
@@ -93,9 +93,8 @@ do_configure () {
 CPPFLAGS="" oe_runconf
 }
 
+LDFLAGS += "-fuse-ld=bfd"
 do_compile () {
-   # -Wl,-rpath-link /lib in LDFLAGS can cause breakage if 
another glibc is in staging
-   LDFLAGS="-fuse-ld=bfd"
base_do_compile
echo "Adjust ldd script"
if [ -n "${RTLDLIST}" ]
-- 
2.23.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] glibc and LDFLAGS

2020-01-18 Thread Rasmus Villemoes
I've added a custom setting to TARGET_LDFLAGS, but then I noticed that
the glibc build ignored that. The glibc do_compile did

 LDFLAGS = ""

for a very long time, with the explanation that "# -Wl,-rpath-link
/lib in LDFLAGS can cause breakage if another glibc is in
staging". Is that actually still relevant with per-recipe staging?

If that is indeed moot, can we do

 CPPFLAGS="" oe_runconf
 }

+LDFLAGS += "-fuse-ld=bfd"
 do_compile () {
-   # -Wl,-rpath-link /lib in LDFLAGS can cause breakage if
another glibc is in staging
-   LDFLAGS="-fuse-ld=bfd"
base_do_compile
echo "Adjust ldd script"
if [ -n "${RTLDLIST}" ]

?

And if not, at least move the LDFLAGS definition out to recipe level so
"bitbake -e" will show the value actually in effect? (And that would
also allow a .bbappend to tweak glibc's LDFLAGS with my custom setting,
though it's annoying to have to maintain that, so the best is if glibc
can just be built with the global LDFLAGS setting).

Rasmus
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH v2] iproute2: drop pointless configure-cross.patch

2019-07-24 Thread Rasmus Villemoes
The configure script has already found IPT_LIB_DIR via pkgconfig, so
the configure-cross.patch really just introduces dead and broken
fallback code.

Broken, because the SYSROOT variable does not actually get set to a
sensible value - the argument $1 passed to the configure script when
invoked from do_configure is ${STAGING_INCDIR}. Obviously that
directory does not have /lib or /usr subdirectories, so we're not
really helping the fallback logic in check_ipt_lib_dir() - in fact,
we're more or less guaranteeing that we won't find those .so files.

Signed-off-by: Rasmus Villemoes 
---
v2: apply to master instead of warrior, update patch description

 .../iproute2/iproute2/configure-cross.patch   | 39 ---
 .../iproute2/iproute2_5.2.0.bb|  1 -
 2 files changed, 40 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/iproute2/iproute2/configure-cross.patch

diff --git a/meta/recipes-connectivity/iproute2/iproute2/configure-cross.patch 
b/meta/recipes-connectivity/iproute2/iproute2/configure-cross.patch
deleted file mode 100644
index b0c4a088fa..00
--- a/meta/recipes-connectivity/iproute2/iproute2/configure-cross.patch
+++ /dev/null
@@ -1,39 +0,0 @@
-From 3835b1cc4a55361443c2b37dea688da652217635 Mon Sep 17 00:00:00 2001
-From: Koen Kooi 
-Date: Sun, 31 Mar 2019 17:32:07 +0200
-Subject: [PATCH] make configure cross compile safe
-
-According to Kevin Tian:
-Upstream-Status: Pending
-
-Signed-off-by: Koen Kooi 
-Signed-off-by: Shane Wang 
-

- configure | 3 ++-
- 1 file changed, 2 insertions(+), 1 deletion(-)
-
-diff --git a/configure b/configure
-index 45fcffb6..0c5c9146 100755
 a/configure
-+++ b/configure
-@@ -3,6 +3,7 @@
- # This is not an autoconf generated configure
- #
- INCLUDE=${1:-"$PWD/include"}
-+SYSROOT=$1
- 
- # Output file which is input to Makefile
- CONFIG=config.mk
-@@ -162,7 +163,7 @@ check_ipt_lib_dir()
-   return
-   fi
- 
--  for dir in /lib /usr/lib /usr/local/lib; do
-+  for dir in $SYSROOT/lib $SYSROOT/usr/lib $SYSROOT/usr/local/lib; do
-   for file in "xtables" "iptables"; do
-   file="$dir/$file/lib*t_*so"
-   if [ -f $file ]; then
--- 
-2.17.1
-
diff --git a/meta/recipes-connectivity/iproute2/iproute2_5.2.0.bb 
b/meta/recipes-connectivity/iproute2/iproute2_5.2.0.bb
index 8ef529d91a..1728cd69a1 100644
--- a/meta/recipes-connectivity/iproute2/iproute2_5.2.0.bb
+++ b/meta/recipes-connectivity/iproute2/iproute2_5.2.0.bb
@@ -1,7 +1,6 @@
 require iproute2.inc
 
 SRC_URI = "${KERNELORG_MIRROR}/linux/utils/net/${BPN}/${BP}.tar.xz \
-   file://configure-cross.patch \
file://0001-libc-compat.h-add-musl-workaround.patch \
   "
 
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH] iproute2: drop pointless configure-cross.patch

2019-07-24 Thread Rasmus Villemoes
The configure script has already found IPT_LIB_DIR via pkgconfig, so
the configure-cross.patch really just introduces dead and broken
fallback code.

Broken, because the SYSROOT variable does not actually get set to a
sensible value - the argument $1 passed to the configure script when
invoked from the Makefile is KERNEL_INCLUDE, which we set to
${STAGING_INCDIR} in EXTRA_OEMAKE. Obviously that directory does not
have /lib or /usr subdirectories, so we're not really helping the
fallback logic in check_ipt_lib_dir() - in fact, we're more or less
guaranteeing that we won't find those .so files.

Signed-off-by: Rasmus Villemoes 
---
 .../iproute2/iproute2/configure-cross.patch   | 32 ---
 .../iproute2/iproute2_4.19.0.bb   |  1 -
 2 files changed, 33 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/iproute2/iproute2/configure-cross.patch

diff --git a/meta/recipes-connectivity/iproute2/iproute2/configure-cross.patch 
b/meta/recipes-connectivity/iproute2/iproute2/configure-cross.patch
deleted file mode 100644
index 8b75a2ada4..00
--- a/meta/recipes-connectivity/iproute2/iproute2/configure-cross.patch
+++ /dev/null
@@ -1,32 +0,0 @@
-From 85b0589b4843c03e8e6fd9416d71ea449a73c5c0 Mon Sep 17 00:00:00 2001
-From: Koen Kooi 
-Date: Thu, 3 Nov 2011 10:46:16 +0100
-Subject: [PATCH] make configure cross compile safe
-
-According to Kevin Tian:
-Upstream-Status: Pending
-
-Signed-off-by: Koen Kooi 
-Signed-off-by: Shane Wang 
-
-Index: iproute2-4.14.1/configure
-===
 iproute2-4.14.1.orig/configure
-+++ iproute2-4.14.1/configure
-@@ -2,6 +2,7 @@
- # This is not an autoconf generated configure
- #
- INCLUDE=${1:-"$PWD/include"}
-+SYSROOT=$1
- 
- # Output file which is input to Makefile
- CONFIG=config.mk
-@@ -195,7 +196,7 @@ check_ipt_lib_dir()
-   return
-   fi
- 
--  for dir in /lib /usr/lib /usr/local/lib
-+  for dir in $SYSROOT/lib $SYSROOT/usr/lib $SYSROOT/usr/local/lib
-   do
-   for file in $dir/{xtables,iptables}/lib*t_*so ; do
-   if [ -f $file ]; then
diff --git a/meta/recipes-connectivity/iproute2/iproute2_4.19.0.bb 
b/meta/recipes-connectivity/iproute2/iproute2_4.19.0.bb
index 6db4062d68..b1db40885a 100644
--- a/meta/recipes-connectivity/iproute2/iproute2_4.19.0.bb
+++ b/meta/recipes-connectivity/iproute2/iproute2_4.19.0.bb
@@ -1,7 +1,6 @@
 require iproute2.inc
 
 SRC_URI = "${KERNELORG_MIRROR}/linux/utils/net/${BPN}/${BP}.tar.xz \
-   file://configure-cross.patch \
file://0001-libc-compat.h-add-musl-workaround.patch \
file://0001-ip-Remove-unneed-header.patch \
   "
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 2/2] openssh: usable sshd depends on rngd from rng-tools

2019-05-09 Thread Rasmus Villemoes
On 08/05/2019 16.22, mikko.rap...@bmw.de wrote:
> On Wed, May 08, 2019 at 05:07:08PM +0300, Adrian Bunk wrote:
>> On Wed, May 08, 2019 at 04:26:09PM +0300, Mikko Rapeli wrote:
>>> Since openssl 1.1.1 and openssh which uses it, sshd
>>> startup is delayed. The delays range from few seconds
>>> to minutes and even to hours. The delays are visible
>>> in host keys generation and when sshd process is started
>>> in response to incoming TCP connection but is failing
>>> to provide SSH version string and clients or tests time out.
>>>
>>> In all cases traces show that sshd is waiting for getentropy()
>>> system call to return from Linux kernel, which returns only
>>> after kernel side random number pool is initialized. The pool
>>> is initialized via various entropy source which may be
>>> missing on embedded development boards or via rngd from
>>> rng-tools package from userspace. HW random number generation
>>> and kernel support help but rngd is till needed to feed that data
>>> back to the Linux kernel.
>>>
>>> Example from an NXP imx8 board shows that kernel random number pool
>>> initialization can take over 400 seconds without rngd,
>>> and with rngd it is initialized at around 4 seconds after boot.
>>> The completion of initialization is visible in kernel dmesg with line
>>> "random: crng init done".
>>> ...
>>> --- a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
>>> +++ b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
>>> @@ -148,6 +148,7 @@ FILES_${PN}-keygen = "${bindir}/ssh-keygen"
>>>  
>>>  RDEPENDS_${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen"
>>>  RDEPENDS_${PN}-sshd += "${PN}-keygen 
>>> ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit 
>>> pam-plugin-loginuid', '', d)}"
>>> +RDEPENDS_${PN}-sshd += "rng-tools"
>>> ...
>>
>> This should only be an RRECOMMENDS so that people can opt out of it.
>>
>> E.g. CONFIG_RANDOM_TRUST_CPU in the kernel can solve the same 
>> problem without using rng-tools on some platforms.
> 
> I think this is a stronger dependency than just RRECOMMENDS. We build
> images and disable recommends but we care that sshd starts as fast as in
> sumo and earlier yocto releases for testing etc purposes.

But why should boards without a hwrng be forced to spend disk space and
run-time resources on a daemon which they don't benefit from at all?

I don't think RANDOM_TRUST_CPU works, though. That's just for stuff like
rdrand(), i.e. instructions built into the CPU - not for some other
on-chip hwrng. Whether those are used for seeding early on (i.e.,
without rngd doing its thing) depends on the ->quality parameter set by
the individual hwrng drivers. Very few set one, so they get assigned the
default_quality, which is a module parameter that defaults to 0.

IOW, I think (but I haven't got around to testing this) one should set
rng_core.default_quality=512 (or something) on the kernel command line
to make the kernel start the hwrng_fill thread that will seed the
entropy pool from the hwrng. At least if I'm reading
drivers/char/hw_random/core.c correctly.

Rasmus




-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] rng-tools: Fix crazy defaults

2019-03-05 Thread Rasmus Villemoes
On 09/11/2018 09.54, Hongxu Jia wrote:
> Since commit [f1dc9ac rng-tools: Fix crazy defaults] fixed
> init based on sysvinit, this fix rngd.service based on systemd.
> 
> Signed-off-by: Hongxu Jia 
> ---
>  meta/recipes-support/rng-tools/rng-tools/rngd.service | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-support/rng-tools/rng-tools/rngd.service 
> b/meta/recipes-support/rng-tools/rng-tools/rngd.service
> index cb81024..f0355db 100644
> --- a/meta/recipes-support/rng-tools/rng-tools/rngd.service
> +++ b/meta/recipes-support/rng-tools/rng-tools/rngd.service
> @@ -5,7 +5,7 @@ After=systemd-udev-settle.service
>  Before=sysinit.target
>  
>  [Service]
> -ExecStart=@SBINDIR@/rngd -f -r /dev/urandom
> +ExecStart=@SBINDIR@/rngd -f -r /dev/hwrng
>  SuccessExitStatus=66
>  
>  [Install]
> 

This has been applied to master, but please apply this to thud as well -
feeding the kernel's entropy pool from that same entropy pool is of
course utter nonsense, but since rngd is also _crediting those bytes
with providing real bits of entropy_ (ioctl RNDADDENTROPY) makes it a
security issue.

Rasmus
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] rng-tools: Fix crazy defaults

2019-03-02 Thread Rasmus Villemoes
On 09/11/2018 09.54, Hongxu Jia wrote:
> Since commit [f1dc9ac rng-tools: Fix crazy defaults] fixed
> init based on sysvinit, this fix rngd.service based on systemd.
> 
> Signed-off-by: Hongxu Jia 
> ---
>  meta/recipes-support/rng-tools/rng-tools/rngd.service | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-support/rng-tools/rng-tools/rngd.service 
> b/meta/recipes-support/rng-tools/rng-tools/rngd.service
> index cb81024..f0355db 100644
> --- a/meta/recipes-support/rng-tools/rng-tools/rngd.service
> +++ b/meta/recipes-support/rng-tools/rng-tools/rngd.service
> @@ -5,7 +5,7 @@ After=systemd-udev-settle.service
>  Before=sysinit.target
>  
>  [Service]
> -ExecStart=@SBINDIR@/rngd -f -r /dev/urandom
> +ExecStart=@SBINDIR@/rngd -f -r /dev/hwrng
>  SuccessExitStatus=66
>  
>  [Install]
> 

This has been applied to master, but please apply this to thud as well -
feeding the kernel's entropy pool from that same entropy pool is of
course utter nonsense, but since rngd is also _crediting those bytes
with providing real bits of entropy_ (ioctl RNDADDENTROPY) makes it a
security issue.

Rasmus
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH resend^2] wic: allow bitbake variables in kickstarter files

2019-01-14 Thread Rasmus Villemoes
ping ping

On 29/12/2018 01.06, Rasmus Villemoes wrote:
> image_types_wic.bbclass has a mechanism for doing variable substitution
> on .wks files by simply letting the input file be called
> .wks.in. However, that doesn't allow using variables in files included
> via the include directive. This is unfortunate, because lacking either
> the ability to include other files or variable substitution leads to
> fragile and error-prone duplication between kickstarter files and
> recipes/configuration files used for various boards.
> 
> This adds (somewhat naive) support for variable substitution in all
> files parsed by wic. The user should add all required variables to
> WICVARS to get them exported appropriately.
> 
> Signed-off-by: Rasmus Villemoes 
> ---
> scripts/lib/wic/ksparser.py | 17 +
>  1 file changed, 17 insertions(+)
> 
> diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> index 7e5a9c5092..08baf76123 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -28,14 +28,30 @@
>  import os
>  import shlex
>  import logging
> +import re
>  
>  from argparse import ArgumentParser, ArgumentError, ArgumentTypeError
>  
>  from wic.engine import find_canned
>  from wic.partition import Partition
> +from wic.misc import get_bitbake_var
>  
>  logger = logging.getLogger('wic')
>  
> +__expand_var_regexp__ = re.compile(r"\${[^{}@\n\t :]+}")
> +
> +def expand_line(line):
> +while True:
> +m = __expand_var_regexp__.search(line)
> +if not m:
> +return line
> +key = m.group()[2:-1]
> +val = get_bitbake_var(key)
> +if val is None:
> +logger.warning("cannot expand variable %s" % key)
> +return line
> +line = line[:m.start()] + val + line[m.end():]
> +
>  class KickStartError(Exception):
>  """Custom exception."""
>  pass
> @@ -190,6 +206,7 @@ class KickStart():
>  line = line.strip()
>  lineno += 1
>  if line and line[0] != '#':
> +line = expand_line(line)
>  try:
>  line_args = shlex.split(line)
>  parsed = parser.parse_args(line_args)
> 


-- 
Rasmus Villemoes
Software Developer
Prevas A/S
Hedeager 3
DK-8200 Aarhus N
+45 51210274
rasmus.villem...@prevas.dk
www.prevas.dk
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH resend^2] wic: allow bitbake variables in kickstarter files

2019-01-09 Thread Rasmus Villemoes
ping

On 29/12/2018 01.06, Rasmus Villemoes wrote:
> image_types_wic.bbclass has a mechanism for doing variable substitution
> on .wks files by simply letting the input file be called
> .wks.in. However, that doesn't allow using variables in files included
> via the include directive. This is unfortunate, because lacking either
> the ability to include other files or variable substitution leads to
> fragile and error-prone duplication between kickstarter files and
> recipes/configuration files used for various boards.
> 
> This adds (somewhat naive) support for variable substitution in all
> files parsed by wic. The user should add all required variables to
> WICVARS to get them exported appropriately.
> 
> Signed-off-by: Rasmus Villemoes 
> ---
> scripts/lib/wic/ksparser.py | 17 +
>  1 file changed, 17 insertions(+)
> 
> diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> index 7e5a9c5092..08baf76123 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -28,14 +28,30 @@
>  import os
>  import shlex
>  import logging
> +import re
>  
>  from argparse import ArgumentParser, ArgumentError, ArgumentTypeError
>  
>  from wic.engine import find_canned
>  from wic.partition import Partition
> +from wic.misc import get_bitbake_var
>  
>  logger = logging.getLogger('wic')
>  
> +__expand_var_regexp__ = re.compile(r"\${[^{}@\n\t :]+}")
> +
> +def expand_line(line):
> +while True:
> +m = __expand_var_regexp__.search(line)
> +if not m:
> +return line
> +key = m.group()[2:-1]
> +val = get_bitbake_var(key)
> +if val is None:
> +logger.warning("cannot expand variable %s" % key)
> +return line
> +line = line[:m.start()] + val + line[m.end():]
> +
>  class KickStartError(Exception):
>  """Custom exception."""
>  pass
> @@ -190,6 +206,7 @@ class KickStart():
>  line = line.strip()
>  lineno += 1
>  if line and line[0] != '#':
> +line = expand_line(line)
>  try:
>  line_args = shlex.split(line)
>  parsed = parser.parse_args(line_args)
> 


-- 
Rasmus Villemoes
Software Developer
Prevas A/S
Hedeager 3
DK-8200 Aarhus N
+45 51210274
rasmus.villem...@prevas.dk
www.prevas.dk
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH resend^2] wic: allow bitbake variables in kickstarter files

2018-12-29 Thread Rasmus Villemoes
image_types_wic.bbclass has a mechanism for doing variable substitution
on .wks files by simply letting the input file be called
.wks.in. However, that doesn't allow using variables in files included
via the include directive. This is unfortunate, because lacking either
the ability to include other files or variable substitution leads to
fragile and error-prone duplication between kickstarter files and
recipes/configuration files used for various boards.

This adds (somewhat naive) support for variable substitution in all
files parsed by wic. The user should add all required variables to
WICVARS to get them exported appropriately.

Signed-off-by: Rasmus Villemoes 
---
scripts/lib/wic/ksparser.py | 17 +
 1 file changed, 17 insertions(+)

diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index 7e5a9c5092..08baf76123 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -28,14 +28,30 @@
 import os
 import shlex
 import logging
+import re
 
 from argparse import ArgumentParser, ArgumentError, ArgumentTypeError
 
 from wic.engine import find_canned
 from wic.partition import Partition
+from wic.misc import get_bitbake_var
 
 logger = logging.getLogger('wic')
 
+__expand_var_regexp__ = re.compile(r"\${[^{}@\n\t :]+}")
+
+def expand_line(line):
+while True:
+m = __expand_var_regexp__.search(line)
+if not m:
+return line
+key = m.group()[2:-1]
+val = get_bitbake_var(key)
+if val is None:
+logger.warning("cannot expand variable %s" % key)
+return line
+line = line[:m.start()] + val + line[m.end():]
+
 class KickStartError(Exception):
 """Custom exception."""
 pass
@@ -190,6 +206,7 @@ class KickStart():
 line = line.strip()
 lineno += 1
 if line and line[0] != '#':
+line = expand_line(line)
 try:
 line_args = shlex.split(line)
 parsed = parser.parse_args(line_args)
-- 
2.19.1.6.gbde171bbf5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/2] lib/oe/package.py: use bb.utils.break_hardlinks helper

2018-08-21 Thread Rasmus Villemoes
This does the same thing, but is more efficient in case st_nlinks
is (already) 1.

Depends on bitbake commit 7ae93cf40ab91965147055100432961436bce46c .

Signed-off-by: Rasmus Villemoes 
---
 meta/lib/oe/package.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index 21c80aaa38..efd36b3758 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -159,7 +159,7 @@ def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, 
d, qa_already_stripp
 else:
 # break hardlinks so that we do not strip the original.
 inodes[inodecache[file]] = file
-bb.utils.copyfile(file, file)
+bb.utils.break_hardlinks(file)
 elffiles[file] = elf_file
 
 #
-- 
2.16.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/2] package.bbclass: use bb.utils.break_hardlinks helper

2018-08-21 Thread Rasmus Villemoes
This does the same thing, but is more efficient in case st_nlinks
is (already) 1.

Depends on bitbake commit 7ae93cf40ab91965147055100432961436bce46c .

Signed-off-by: Rasmus Villemoes 
---
 meta/classes/package.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 4ce9de2f57..8dca3b0191 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1029,7 +1029,7 @@ python split_and_strip_files () {
 else:
 inodes[file_reference] = [file]
 # break hardlink
-bb.utils.copyfile(file, file)
+bb.utils.break_hardlinks(file)
 elffiles[file] = elf_file
 # Modified the file so clear the cache
 cpath.updatecache(file)
-- 
2.16.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [pseudo] [RFC 1/2] pseudo_ipc.c: Use MSG_NOSIGNAL if available

2018-08-15 Thread Rasmus Villemoes
On 2018-07-11 16:30, Rasmus Villemoes wrote:
> MSG_NOSIGNAL has been in Linux since 2.2, and has been standardized in
> POSIX 2008. Using that when available avoids the overhead of the two
> syscalls to set and restore the SIGPIPE handler. Moreover, we can
> eliminate one write() call by making use of sendmsg() to do
> scatter-gather I/O.

Ping. The README in the pseudo repo says to use the openembedded-core
list for pseudo patches, so I assume this is the right place. I have
other ideas for improving pseudo's performance on both client and server
side, but there's no point sending patches to /dev/null.

Rasmus
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [RFC PATCH] package.bbclass: improve -dbg and -src package ordering

2018-08-15 Thread Rasmus Villemoes
On 2018-07-17 10:13, Rasmus Villemoes wrote:
> ping...
> 

ping2

> On 2018-07-11 13:38, Rasmus Villemoes wrote:
>> nativesdk-gpgme fails package_qa when setting PACKAGE_DEBUG_SPLIT_STYLE
>> = "debug-with-srcpkg".
>>
>> ERROR: nativesdk-gpgme-1.10.0-r0 do_package_qa: QA Issue: non debug package 
>> contains .debug directory: nativesdk-python3-gpg path 
>> /work/x86_64-nativesdk-oesdk-linux/nativesdk-gpgme/1.10.0-r0/packages-split/nativesdk-python3-gpg/usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/lib/python3.5/site-packages/gpg/.debug/_gpgme.cpython-35m-x86_64-linux-gnu.so
>>  [debug-files]
>>
>> This turns out to be because the automatic moving of the -dbg package to
>> the beginning of the package list is disabled in that case, so the
>> python3-gpg packages that the recipe prepends to PACKAGES ends up before
>> the -dbg package.
>>
>> It's not clear why the "and not split_source_package" was added when
>> debug-with-srcpkg was introduced. Presumably the intention was to
>> prevent the -dbg package to end up before the -src package, which we of
>> course need to. But at the same time, we still need -dbg packages to end
>> up before all other packages.
>>
>> Using list.insert(0, ...) also means that if there happens to more than
>> one -dbg package, their relative ordering gets inverted in the new list.
>>
>> This tries to fix these issues by sorting the packages by (priority,
>> original position), where priority is 10 for -src, 30 for -dbg and 50
>> for everything else. That guarantees that packages of the same "type"
>> preserve their relative ordering, while also ensuring that -dbg always
>> preceed other packages. This scheme is also quite extensible, and,
>> should the need arise, one could even expose the priorities as a knob
>> the recipe author could use to ensure specific orderings of packages
>> instead of the somewhat fragile and coarse-grained method of "prepend or
>> append, and ensure you do that in a proper order".
>>
>> Probably the autodebug condition needs to stay, but I think the
>> split_source_package condition in the preceding elif should be removed,
>> so that that logic applies to all packages called -src, not just the one
>> we might have created a few lines above.
>>
>> Signed-off-by: Rasmus Villemoes 
>> ---
>>  meta/classes/package.bbclass | 15 ---
>>  1 file changed, 8 insertions(+), 7 deletions(-)
>>
>> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
>> index 8276046369..72b9c66c3b 100644
>> --- a/meta/classes/package.bbclass
>> +++ b/meta/classes/package.bbclass
>> @@ -1151,21 +1151,22 @@ python populate_packages () {
>>  
>>  # Sanity check PACKAGES for duplicates
>>  # Sanity should be moved to sanity.bbclass once we have the 
>> infrastructure
>> -package_list = []
>> +package_dict = {}
>>  
>> -for pkg in packages.split():
>> -if pkg in package_list:
>> +for i, pkg in enumerate(packages.split()):
>> +if pkg in package_dict:
>>  msg = "%s is listed in PACKAGES multiple times, this leads to 
>> packaging errors." % pkg
>>  package_qa_handle_error("packages-list", msg, d)
>>  # If debug-with-srcpkg mode is enabled then the src package will 
>> have
>>  # priority over dbg package when assigning the files.
>>  # This allows src package to include source files and remove them 
>> from dbg.
>>  elif split_source_package and pkg.endswith("-src"):
>> -package_list.insert(0, pkg)
>> -elif autodebug and pkg.endswith("-dbg") and not 
>> split_source_package:
>> -package_list.insert(0, pkg)
>> +package_dict[pkg] = (10, i)
>> +elif autodebug and pkg.endswith("-dbg"):
>> +package_dict[pkg] = (30, i)
>>  else:
>> -package_list.append(pkg)
>> +package_dict[pkg] = (50, i)
>> +package_list = sorted(package_dict.keys(), key=package_dict.get)
>>  d.setVar('PACKAGES', ' '.join(package_list))
>>  pkgdest = d.getVar('PKGDEST')
>>  
>>
> 


-- 
Rasmus Villemoes
Software Developer
Prevas A/S
Hedeager 3
DK-8200 Aarhus N
+45 51210274
rasmus.villem...@prevas.dk
www.prevas.dk
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH] busybox: udhcpd: fix not dying on SIGTERM

2018-07-17 Thread Rasmus Villemoes
busybox 1.27.2 udhcpd does not respond to SIGTERM. Backport the upstream
fix. I've verified that this makes our local automated test suite pass
again.

Signed-off-by: Rasmus Villemoes 
---
 .../busybox/udhcpd-fix-not-dying-on-SIGTERM.patch  | 273 +
 meta/recipes-core/busybox/busybox_1.27.2.bb|   1 +
 2 files changed, 274 insertions(+)
 create mode 100644 
meta/recipes-core/busybox/busybox/udhcpd-fix-not-dying-on-SIGTERM.patch

diff --git 
a/meta/recipes-core/busybox/busybox/udhcpd-fix-not-dying-on-SIGTERM.patch 
b/meta/recipes-core/busybox/busybox/udhcpd-fix-not-dying-on-SIGTERM.patch
new file mode 100644
index 00..1ea0385a8f
--- /dev/null
+++ b/meta/recipes-core/busybox/busybox/udhcpd-fix-not-dying-on-SIGTERM.patch
@@ -0,0 +1,273 @@
+From 1384459d88cb0f5f47b6a36b8346dcf425a643f5 Mon Sep 17 00:00:00 2001
+From: Denys Vlasenko 
+Date: Sat, 10 Mar 2018 19:01:48 +0100
+Subject: [PATCH] udhcpd: fix "not dying on SIGTERM"
+
+Upstream-status: Backport 
[https://git.busybox.net/busybox/commit/?id=3293bc146985c56790033409facc0ad64a471084]
+
+Fixes:
+   commit 52a515d18724bbb34e3ccbbb0218efcc4eccc0a8
+   "udhcp: use poll() instead of select()"
+   Feb 16 2017
+
+udhcp_sp_read() is meant to check whether signal pipe indeed has some data to 
read.
+In the above commit, it was changed as follows:
+
+-  if (!FD_ISSET(signal_pipe.rd, rfds))
++  if (!pfds[0].revents)
+   return 0;
+
+The problem is, the check was working for select() purely by accident.
+Caught signal interrupts select()/poll() syscalls, they return with EINTR
+(regardless of SA_RESTART flag in sigaction). _Then_ signal handler is invoked.
+IOW: they can't see any changes to fd state caused by signal haldler
+(in our case, signal handler makes signal pipe ready to be read).
+
+For select(), it means that rfds[] bit array is unmodified, bit of signal
+pipe's read fd is still set, and the above check "works": it thinks select()
+says there is data to read.
+
+This accident does not work for poll(): .revents stays clear, and we do not
+try reading signal pipe as we should. In udhcpd, we fall through and block
+in socket read. Further SIGTERM signals simply cause socket read to be
+interrupted and then restarted (since SIGTERM handler has SA_RESTART=1).
+
+Fixing this as follows: remove the check altogether. Set signal pipe read fd
+to nonblocking mode. Always read it in udhcp_sp_read().
+If read fails, assume it's EAGAIN and return 0 ("no signal seen").
+
+udhcpd avoids reading signal pipe on every recvd packet by looping if EINTR
+(using safe_poll()) - thus ensuring we have correct .revents for all fds -
+and calling udhcp_sp_read() only if pfds[0].revents!=0.
+
+udhcpc performs much fewer reads (typically it sleeps >99.999% of the time),
+there is no need to optimize it: can call udhcp_sp_read() after each poll
+unconditionally.
+
+To robustify socket reads, unconditionally set pfds[1].revents=0
+in udhcp_sp_fd_set() (which is before poll), and check it before reading
+network socket in udhcpd.
+
+TODO:
+This might still fail: if pfds[1].revents=POLLIN, socket read may still block.
+There are rare cases when select/poll indicates that data can be read,
+but then actual read still blocks (one such case is UDP packets with
+wrong checksum). General advise is, if you use a poll/select loop,
+keep all your fds nonblocking.
+Maybe we should also do that to our network sockets?
+
+function old new   delta
+udhcp_sp_setup55  65 +10
+udhcp_sp_fd_set   54  60  +6
+udhcp_sp_read 46  36 -10
+udhcpd_main 14511437 -14
+udhcpc_main 27232708 -15
+--
+(add/remove: 0/0 grow/shrink: 2/3 up/down: 16/-39)Total: -23 bytes
+
+Signed-off-by: Denys Vlasenko 
+---
+ examples/var_service/dhcpd_if/run  |  4 +--
+ .../dhcpd_if/{udhcpc.conf => udhcpd.conf}  |  0
+ networking/udhcp/common.h  |  2 +-
+ networking/udhcp/d6_dhcpc.c|  9 +++---
+ networking/udhcp/dhcpc.c   |  9 +++---
+ networking/udhcp/dhcpd.c   | 34 --
+ networking/udhcp/signalpipe.c  | 11 +++
+ 7 files changed, 35 insertions(+), 34 deletions(-)
+ rename examples/var_service/dhcpd_if/{udhcpc.conf => udhcpd.conf} (100%)
+
+diff --git a/examples/var_service/dhcpd_if/run 
b/examples/var_service/dhcpd_if/run
+index de85dece0..a603bdc71 100755
+--- a/examples/var_service/dhcpd_if/run
 b/examples/var_service/dhcpd_if/run
+@@ -11,13 +11,13 @@ echo "* Upping iface $if&q

Re: [OE-core] [PATCH resend] wic: allow bitbake variables in kickstarter files

2018-07-17 Thread Rasmus Villemoes
On 2018-07-03 14:54, Rasmus Villemoes wrote:
> image_types_wic.bbclass has a mechanism for doing variable substitution
> on .wks files by simply letting the input file be called
> .wks.in. However, that doesn't allow using variables in files included
> via the include directive. This is unfortunate, because lacking either
> the ability to include other files or variable substitution leads to
> fragile and error-prone duplication between kickstarter files and
> recipes/configuration files used for various boards.
> 
> This adds (somewhat naive) support for variable substitution in all
> files parsed by wic. The user should add all required variables to
> WICVARS to get them exported appropriately.

ping^n
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [RFC PATCH] package.bbclass: improve -dbg and -src package ordering

2018-07-17 Thread Rasmus Villemoes
ping...

On 2018-07-11 13:38, Rasmus Villemoes wrote:
> nativesdk-gpgme fails package_qa when setting PACKAGE_DEBUG_SPLIT_STYLE
> = "debug-with-srcpkg".
> 
> ERROR: nativesdk-gpgme-1.10.0-r0 do_package_qa: QA Issue: non debug package 
> contains .debug directory: nativesdk-python3-gpg path 
> /work/x86_64-nativesdk-oesdk-linux/nativesdk-gpgme/1.10.0-r0/packages-split/nativesdk-python3-gpg/usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/lib/python3.5/site-packages/gpg/.debug/_gpgme.cpython-35m-x86_64-linux-gnu.so
>  [debug-files]
> 
> This turns out to be because the automatic moving of the -dbg package to
> the beginning of the package list is disabled in that case, so the
> python3-gpg packages that the recipe prepends to PACKAGES ends up before
> the -dbg package.
> 
> It's not clear why the "and not split_source_package" was added when
> debug-with-srcpkg was introduced. Presumably the intention was to
> prevent the -dbg package to end up before the -src package, which we of
> course need to. But at the same time, we still need -dbg packages to end
> up before all other packages.
> 
> Using list.insert(0, ...) also means that if there happens to more than
> one -dbg package, their relative ordering gets inverted in the new list.
> 
> This tries to fix these issues by sorting the packages by (priority,
> original position), where priority is 10 for -src, 30 for -dbg and 50
> for everything else. That guarantees that packages of the same "type"
> preserve their relative ordering, while also ensuring that -dbg always
> preceed other packages. This scheme is also quite extensible, and,
> should the need arise, one could even expose the priorities as a knob
> the recipe author could use to ensure specific orderings of packages
> instead of the somewhat fragile and coarse-grained method of "prepend or
> append, and ensure you do that in a proper order".
> 
> Probably the autodebug condition needs to stay, but I think the
> split_source_package condition in the preceding elif should be removed,
> so that that logic applies to all packages called -src, not just the one
> we might have created a few lines above.
> 
> Signed-off-by: Rasmus Villemoes 
> ---
>  meta/classes/package.bbclass | 15 ---
>  1 file changed, 8 insertions(+), 7 deletions(-)
> 
> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
> index 8276046369..72b9c66c3b 100644
> --- a/meta/classes/package.bbclass
> +++ b/meta/classes/package.bbclass
> @@ -1151,21 +1151,22 @@ python populate_packages () {
>  
>  # Sanity check PACKAGES for duplicates
>  # Sanity should be moved to sanity.bbclass once we have the 
> infrastructure
> -package_list = []
> +package_dict = {}
>  
> -for pkg in packages.split():
> -if pkg in package_list:
> +for i, pkg in enumerate(packages.split()):
> +if pkg in package_dict:
>  msg = "%s is listed in PACKAGES multiple times, this leads to 
> packaging errors." % pkg
>  package_qa_handle_error("packages-list", msg, d)
>  # If debug-with-srcpkg mode is enabled then the src package will have
>  # priority over dbg package when assigning the files.
>  # This allows src package to include source files and remove them 
> from dbg.
>  elif split_source_package and pkg.endswith("-src"):
> -package_list.insert(0, pkg)
> -elif autodebug and pkg.endswith("-dbg") and not split_source_package:
> -package_list.insert(0, pkg)
> +package_dict[pkg] = (10, i)
> +elif autodebug and pkg.endswith("-dbg"):
> +package_dict[pkg] = (30, i)
>  else:
> -package_list.append(pkg)
> +package_dict[pkg] = (50, i)
> +package_list = sorted(package_dict.keys(), key=package_dict.get)
>  d.setVar('PACKAGES', ' '.join(package_list))
>  pkgdest = d.getVar('PKGDEST')
>  
> 

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


  1   2   >