Re: [OE-core] [PATCH v7 3/3] nfs-utils: Configure nfsv4 ID mapping & Kerberos

[Enrico Scholz via lists.openembedded.org]

"Dan McGregor"  writes:

> Add support for the nfsv4 user ID mapping daemon, configured with
> a sensible default, and add a packageconfig for Kerberos support.
>
> This is reasonably tested in production in our environment, but only
> systemd support. There'll be some more work to do to get GSSAPI and
> NFS idmapd support integreated into that.

Just for interest: how do you solve GSSAPI?  I made it work for me, put
it requires ugly changes (e.g. see [1])

There is an early dependency loop in libtirpc -> python -> kerberos ->
openssl -> python.  I solved it by

- creating a throw-away krb5-initial package [2]; installs libraries in
  /opt/krb5 and is used only at a few places.

- splitting libtirpc; this required to touch only a few other packages
  some weeks/months ago.  But now, [file-rdeps] error is triggered and
  every

Both is ugly



Enrico

Footnotes:
[1]  
https://gitlab.com/ensc-groups/bpi-router/de.ensc.bpi-router/-/tree/nanbield-next/meta-core/recipes-extended/libtirpc?ref_type=heads

 (there exists a newer version but I do have not pushed it yet...)

[2]  
https://gitlab.com/ensc-groups/bpi-router/de.ensc.bpi-router/-/tree/nanbield-next/meta-core/recipes-connectivity/krb5?ref_type=heads


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198010): 
https://lists.openembedded.org/g/openembedded-core/message/198010
Mute This Topic: https://lists.openembedded.org/mt/105279025/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] shadow: fix copydir operation with 'pseudo'

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Calling 'useradd' through pseudo on (at least) Ubuntu 20 creates
filesystem objects (.bashrc, .profile) with invalid attributes.  It
manifests as

| tar: ./home/.../.bashrc: Unknown file type; file ignored

or

| Copying files into the device: __populate_fs: ignoring entry ".bashrc"
| .bashrc: File not found by ext2_lookup while looking up ".bashrc"

when building the image.

This happens due to a bug in shadow which is caused by clobbering
fstatat() results.

Signed-off-by: Enrico Scholz 
---
 ...copy_entry-use-temporary-stat-buffer.patch | 39 +++
 meta/recipes-extended/shadow/shadow.inc   |  1 +
 2 files changed, 40 insertions(+)
 create mode 100644 
meta/recipes-extended/shadow/files/0001-lib-copydir-copy_entry-use-temporary-stat-buffer.patch

diff --git 
a/meta/recipes-extended/shadow/files/0001-lib-copydir-copy_entry-use-temporary-stat-buffer.patch
 
b/meta/recipes-extended/shadow/files/0001-lib-copydir-copy_entry-use-temporary-stat-buffer.patch
new file mode 100644
index ..d278a4cda3ff
--- /dev/null
+++ 
b/meta/recipes-extended/shadow/files/0001-lib-copydir-copy_entry-use-temporary-stat-buffer.patch
@@ -0,0 +1,39 @@
+From af4b8cb780587aa736692a3baa76b60474f19c5d Mon Sep 17 00:00:00 2001
+From: Enrico Scholz 
+Date: Mon, 18 Mar 2024 12:14:21 +0100
+Subject: [PATCH] lib/copydir:copy_entry(): use temporary stat buffer
+
+There are no guarantees that fstatat() does not clobber the stat
+buffer on errors.
+
+Use a temporary buffer so that the following code sees correct
+attributes of the source entry.
+
+Upstream-Status: Submitted [https://github.com/shadow-maint/shadow/pull/974]
+
+Signed-off-by: Enrico Scholz 
+---
+ lib/copydir.c | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+Index: shadow-4.14.2/lib/copydir.c
+===
+--- shadow-4.14.2.orig/lib/copydir.c
 shadow-4.14.2/lib/copydir.c
+@@ -415,6 +415,7 @@ static int copy_entry (const struct path
+ {
+   int err = 0;
+   struct stat sb;
++  struct stat tmp_sb;
+   struct link_name *lp;
+   struct timespec mt[2];
+ 
+@@ -436,7 +437,7 @@ static int copy_entry (const struct path
+* If the destination already exists do nothing.
+* This is after the copy_dir above to still iterate into 
subdirectories.
+*/
+-  if (fstatat(dst->dirfd, dst->name, , AT_SYMLINK_NOFOLLOW) != 
-1) {
++  if (fstatat(dst->dirfd, dst->name, _sb, 
AT_SYMLINK_NOFOLLOW) != -1) {
+   return 0;
+   }
+ 
diff --git a/meta/recipes-extended/shadow/shadow.inc 
b/meta/recipes-extended/shadow/shadow.inc
index 16b99a0b60e8..a164b8347f8c 100644
--- a/meta/recipes-extended/shadow/shadow.inc
+++ b/meta/recipes-extended/shadow/shadow.inc
@@ -12,6 +12,7 @@ DEPENDS = "virtual/crypt"
 
 GITHUB_BASE_URI = "https://github.com/shadow-maint/shadow/releases;
 SRC_URI = "${GITHUB_BASE_URI}/download/${PV}/${BP}.tar.gz \
+   file://0001-lib-copydir-copy_entry-use-temporary-stat-buffer.patch \
${@bb.utils.contains('PACKAGECONFIG', 'pam', '${PAM_SRC_URI}', '', 
d)} \
file://useradd \
"
-- 
2.44.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#197307): 
https://lists.openembedded.org/g/openembedded-core/message/197307
Mute This Topic: https://lists.openembedded.org/mt/105003196/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] '.bashrc': Unknown file type; file ignored

[Enrico Scholz via lists.openembedded.org]

Enrico Scholz  writes:

> 4.14.2/shadow-4.14.2/lib/copydir.c
>
> | 443 if (fstatat(dst->dirfd, dst->name, , AT_SYMLINK_NOFOLLOW) != 
> -1) {
> | return 0;
> | }

Bug in shadow-utils; when fstatat() fails, content of 'sb' will still be
used.

  
https://github.com/shadow-maint/shadow/blob/f4293f9fbc2b855878f549d9124bdd638fb08c60/lib/copydir.c#L439-L447

pseudo might trigger it because it calls the __fxstatat64() with an
uninitialized 64 bit buffer and copies its content to the 32 bit buffer.


Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#197302): 
https://lists.openembedded.org/g/openembedded-core/message/197302
Mute This Topic: https://lists.openembedded.org/mt/104950372/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] '.bashrc': Unknown file type; file ignored

[Enrico Scholz via lists.openembedded.org]

Enrico Scholz  writes:

> Seen on scarthgap bc793fa9d1fe24c102d91e97b7002b6e637cbfa5 and it seems
> to be very reproducible (both the failure on ubuntu-20.04 and the success
> on fedora-39).

Can be reproduced manually by setting the PSEUDO environment and
invoking

| pseudo useradd -R `pwd`/rootfs bar1
| pseudo ls -la rootfs/home/bar1


Problem is somewhere in the interpretation of the fstatat() result in

4.14.2/shadow-4.14.2/lib/copydir.c

| 443   if (fstatat(dst->dirfd, dst->name, , AT_SYMLINK_NOFOLLOW) != 
-1) {
|   return 0;
|   }
| 
| + printf("fstatat(%d, %s) -> dev=%d, ino=%d, mode=%o\n",
| +dst->dirfd, dst->name, (int)sb.st_dev, (int)sb.st_ino, 
sb.st_mode);


Shows here (content is from /etc/skel)

| fstatat(5, .profile) -> dev=64514, ino=1787061952, mode=15241063300
| fstatat(5, .bashrc) -> dev=64514, ino=1787061952, mode=15241063300

While

| $ stat rootfs/etc/skel/. 
|   File: rootfs/etc/skel/.bashrc
|   Size: 410 Blocks: 8  IO Block: 4096   regular file
| Device: 252,2   Inode: 19552578Links: 1

is expected.



Pseudo log shows for this call

1914825: wrapper called: __fxstatat
1914825: __fxstatat - signals blocked, obtaining lock
1914825: paes: newpath .../rootfs/etc/skel, element list <.bashrc>
1914825: element to add: '.bashrc'
1914825: pae: '.../rootfs/etc/skel', + '.bashrc', is_dir 1
1914825: paes: append_element gave us '.../rootfs/etc/skel/.bashrc', current ''
1914825: .../rootfs/etc/skel + .bashrc => <.../rootfs/etc/skel/.bashrc>
1914825: base_path[nofollow]: .../rootfs/etc/skel.bashrc => 
.../rootfs/etc/skel/.bashrc
1914825: root_path [__fxstatat, 235]: '.../rootfs/etc/skel/.bashrc' from 
'.bashrc'
1914825: statat .../rootfs/etc/skel/.bashrc (+buf) (0100755): processing 
request [ino 19552578]
1914825: sending request [ino 19552578]
1914825: sending a message: ino 19552578
1914825: msg type 3 (stat), external path .../rootfs/etc/skel/.bashrc, mode 
0100755
1914825: wrote 182 bytes
1914825: sent!



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#197248): 
https://lists.openembedded.org/g/openembedded-core/message/197248
Mute This Topic: https://lists.openembedded.org/mt/104950372/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] '.bashrc': Unknown file type; file ignored

[Enrico Scholz via lists.openembedded.org]

Hello,

with ubuntu-20.04 based builders I get problems like

| DEBUG: Executing shell function do_image_tar
| tar: ./home/.../.bashrc: Unknown file type; file ignored
| tar: Exiting with failure status due to previous errors
| WARNING: exit code 1 from a shell command.
| NOTE: recipe ...-image-core-1.0-r0: task do_image_tar: Failed

or

| Creating journal (8192 blocks): done
| Copying files into the device: __populate_fs: ignoring entry ".bashrc"
| .bashrc: File not found by ext2_lookup while looking up ".bashrc"
| mkfs.ext4: File not found by ext2_lookup while populating file system
| 
| + bb_sh_exit_handler
| + ret=1
| + [ 1 != 0 ]
| + echo WARNING: exit code 1 from a shell command.
| + exit 1
| WARNING: exit code 1 from a shell command.
| NOTE: recipe ...-image-1.0-r0: task do_image_wic: Failed


Build works fine under fedora-39 based builders.

I see related discussions at 2024-01-18 ("shadow: link executables
statically for -native variant") but not after.


Seen on scarthgap bc793fa9d1fe24c102d91e97b7002b6e637cbfa5 and it seems
to be very reproducible (both the failure on ubuntu-20.04 and the success
on fedora-39).



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#197140): 
https://lists.openembedded.org/g/openembedded-core/message/197140
Mute This Topic: https://lists.openembedded.org/mt/104950372/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 0/7] Replace sshd_config patching by snippets

[Enrico Scholz via lists.openembedded.org]

Richard Purdie  writes:

> Some further related warnings:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/23/builds/9031/steps/11/logs/warnins

| stdio: WARNING: Nothing RPROVIDES 'nativesdk-openssh-config' ...

ok

I will reduce dependency from

|-RRECOMMENDS:${PN} += "openssh-config"

to

|+RRECOMMENDS:${PN}-sshd:class-target += "openssh-config"


| stdio: WARNING: Nothing RPROVIDES 'nativesdk-openssh-scp' ...
| stdio: WARNING: Nothing RPROVIDES 'nativesdk-openssh-sshd' ...
| stdio: WARNING: Nothing RPROVIDES 'nativesdk-openssh-ssh' ...

are these warnings really new?



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#197111): 
https://lists.openembedded.org/g/openembedded-core/message/197111
Mute This Topic: https://lists.openembedded.org/mt/104868003/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 0/7] Replace sshd_config patching by snippets

[Enrico Scholz via lists.openembedded.org]

Richard Purdie  writes:

> Thanks for sending this. I suspect something like this might be
> desirable however unfortunately the timing is a little tricky as we're
> just past the feature freeze point for 5.0.

ok; my fault.  I delayed it too much.

Would it be possible to communicate such schedules more visibly?
E.g. put a link on https://wiki.yoctoproject.org/wiki/Releases ?


> https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/8649/steps/14/logs/warnings

ok


> so I squashed a fix in for that. The second run had this:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/6390/steps/12/logs/stdio

It is difficulty for me to analyze this from the outside.  The patchset
requires that the 'openssh-sshd' IMAGE_FEATURE is set; installing the
package manually does not suffice anymore.

I can not detect such an IMAGE_FEATURE configuration in step 10; but I
might miss other setup.



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#197109): 
https://lists.openembedded.org/g/openembedded-core/message/197109
Mute This Topic: https://lists.openembedded.org/mt/104868003/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/7, v3] openssh: replace complete configuration files by patch

[Enrico Scholz via lists.openembedded.org]

Alexander Kanavin  writes:

> Why is the patch inappropriate for upstream submission? To me it looks
> like it should be at least proposed.
>
> ++Include /etc/ssh/sshd_config.d/*.conf

Underlying feature exists for 4 years and nearly every major linux
distribution (including OE) has such a line.  So I assume there is a
reason that it is not upstream yet.

I added an upstream report
(https://bugzilla.mindrot.org/show_bug.cgi?id=3672) and will update
patch status when resubmitting.



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196991): 
https://lists.openembedded.org/g/openembedded-core/message/196991
Mute This Topic: https://lists.openembedded.org/mt/104882201/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/7, v3] openssh: replace complete configuration files by patch

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Instead of shipping the whole configuration files for openssh, add
small patch includes configuration snippets from subdirectories.

This allows us to keep the original upstream configuration which is
mainly useful for documentation purposes.  It makes it more easy to
identify OE specific setup.

Signed-off-by: Enrico Scholz 
---
 .../openssh/openssh/include-conf.patch|  34 +
 .../openssh/openssh/ssh_config|  48 ---
 .../openssh/openssh/sshd_config   | 119 --
 .../openssh/openssh_9.6p1.bb  |   5 +-
 4 files changed, 35 insertions(+), 171 deletions(-)
 create mode 100644 meta/recipes-connectivity/openssh/openssh/include-conf.patch
 delete mode 100644 meta/recipes-connectivity/openssh/openssh/ssh_config
 delete mode 100644 meta/recipes-connectivity/openssh/openssh/sshd_config

diff --git a/meta/recipes-connectivity/openssh/openssh/include-conf.patch 
b/meta/recipes-connectivity/openssh/openssh/include-conf.patch
new file mode 100644
index ..0a3f6839f838
--- /dev/null
+++ b/meta/recipes-connectivity/openssh/openssh/include-conf.patch
@@ -0,0 +1,34 @@
+Include configuration snippets from subdirectory.
+
+NOTE: first configuration option wins.
+
+Upstream-Status: Inappropriate [configuration]
+
+Signed-off-by: Enrico Scholz 
+
+Index: openssh-9.5p1/ssh_config
+===
+--- openssh-9.5p1.orig/ssh_config
 openssh-9.5p1/ssh_config
+@@ -5,6 +5,8 @@
+ # users, and the values can be changed in per-user configuration files
+ # or on the command line.
+
++Include /etc/ssh/ssh_config.d/*.conf
++
+ # Configuration data is parsed as follows:
+ #  1. command line options
+ #  2. user-specific file
+Index: openssh-9.5p1/sshd_config
+===
+--- openssh-9.5p1.orig/sshd_config
 openssh-9.5p1/sshd_config
+@@ -10,6 +10,8 @@
+ # possible, but leave them commented.  Uncommented options override the
+ # default value.
+
++Include /etc/ssh/sshd_config.d/*.conf
++
+ #Port 22
+ #AddressFamily any
+ #ListenAddress 0.0.0.0
diff --git a/meta/recipes-connectivity/openssh/openssh/ssh_config 
b/meta/recipes-connectivity/openssh/openssh/ssh_config
deleted file mode 100644
index cb2774a163ed..
--- a/meta/recipes-connectivity/openssh/openssh/ssh_config
+++ /dev/null
@@ -1,48 +0,0 @@
-#  $OpenBSD: ssh_config,v 1.35 2020/07/17 03:43:42 dtucker Exp $
-
-# This is the ssh client system-wide configuration file.  See
-# ssh_config(5) for more information.  This file provides defaults for
-# users, and the values can be changed in per-user configuration files
-# or on the command line.
-
-# Configuration data is parsed as follows:
-#  1. command line options
-#  2. user-specific file
-#  3. system-wide file
-# Any configuration value is only changed the first time it is set.
-# Thus, host-specific definitions should be at the beginning of the
-# configuration file, and defaults at the end.
-
-# Site-wide defaults for some commonly used options.  For a comprehensive
-# list of available options, their meanings and defaults, please see the
-# ssh_config(5) man page.
-
-Include /etc/ssh/ssh_config.d/*.conf
-
-# Host *
-#   ForwardAgent no
-#   ForwardX11 no
-#   PasswordAuthentication yes
-#   HostbasedAuthentication no
-#   GSSAPIAuthentication no
-#   GSSAPIDelegateCredentials no
-#   BatchMode no
-#   CheckHostIP yes
-#   AddressFamily any
-#   ConnectTimeout 0
-#   StrictHostKeyChecking ask
-#   IdentityFile ~/.ssh/id_rsa
-#   IdentityFile ~/.ssh/id_dsa
-#   IdentityFile ~/.ssh/id_ecdsa
-#   IdentityFile ~/.ssh/id_ed25519
-#   Port 22
-#   Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc
-#   MACs hmac-md5,hmac-sha1,umac...@openssh.com
-#   EscapeChar ~
-#   Tunnel no
-#   TunnelDevice any:any
-#   PermitLocalCommand no
-#   VisualHostKey no
-#   ProxyCommand ssh -q -W %h:%p gateway.example.com
-#   RekeyLimit 1G 1h
-#   UserKnownHostsFile ~/.ssh/known_hosts.d/%k
diff --git a/meta/recipes-connectivity/openssh/openssh/sshd_config 
b/meta/recipes-connectivity/openssh/openssh/sshd_config
deleted file mode 100644
index e9eaf9315775..
--- a/meta/recipes-connectivity/openssh/openssh/sshd_config
+++ /dev/null
@@ -1,119 +0,0 @@
-#  $OpenBSD: sshd_config,v 1.104 2021/07/02 05:11:21 dtucker Exp $
-
-# This is the sshd server system-wide configuration file.  See
-# sshd_config(5) for more information.
-
-# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin
-
-# The strategy used for options in the default sshd_config shipped with
-# OpenSSH is to specify options with their default value where
-# possible, but leave them commented.  Uncommented options override the
-# default value.
-
-Include /etc/ssh/sshd_config.d/*.conf
-
-#Port 22
-#AddressFamily any
-#ListenAddress 0.0.0.0
-#ListenAddress ::
-
-#HostKey /etc/ssh/ssh_host_rsa_key
-#HostKey /etc/ssh

[OE-core] [PATCH 1/7, v2] openssh: replace complete configuration files by patch

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Instead of shipping the whole configuration files for openssh, add
small patch includes configuration snippets from subdirectories.

This allows us to keep the original upstream configuration which is
mainly useful for documentation purposes.  It makes it more easy to
identify OE specific setup.

Signed-off-by: Enrico Scholz 
---
 .../openssh/openssh/include-conf.patch|  32 +
 .../openssh/openssh/ssh_config|  48 ---
 .../openssh/openssh/sshd_config   | 119 --
 .../openssh/openssh_9.6p1.bb  |   5 +-
 4 files changed, 33 insertions(+), 171 deletions(-)
 create mode 100644 meta/recipes-connectivity/openssh/openssh/include-conf.patch
 delete mode 100644 meta/recipes-connectivity/openssh/openssh/ssh_config
 delete mode 100644 meta/recipes-connectivity/openssh/openssh/sshd_config

diff --git a/meta/recipes-connectivity/openssh/openssh/include-conf.patch 
b/meta/recipes-connectivity/openssh/openssh/include-conf.patch
new file mode 100644
index ..3e0bd2957e25
--- /dev/null
+++ b/meta/recipes-connectivity/openssh/openssh/include-conf.patch
@@ -0,0 +1,32 @@
+Include configuration snippets from subdirectory.
+
+NOTE: first configuration option wins.
+
+Upstream-Status: Inappropriate [configuration]
+
+Index: openssh-9.5p1/ssh_config
+===
+--- openssh-9.5p1.orig/ssh_config
 openssh-9.5p1/ssh_config
+@@ -5,6 +5,8 @@
+ # users, and the values can be changed in per-user configuration files
+ # or on the command line.
+ 
++Include /etc/ssh/ssh_config.d/*.conf
++
+ # Configuration data is parsed as follows:
+ #  1. command line options
+ #  2. user-specific file
+Index: openssh-9.5p1/sshd_config
+===
+--- openssh-9.5p1.orig/sshd_config
 openssh-9.5p1/sshd_config
+@@ -10,6 +10,8 @@
+ # possible, but leave them commented.  Uncommented options override the
+ # default value.
+ 
++Include /etc/ssh/sshd_config.d/*.conf
++
+ #Port 22
+ #AddressFamily any
+ #ListenAddress 0.0.0.0
diff --git a/meta/recipes-connectivity/openssh/openssh/ssh_config 
b/meta/recipes-connectivity/openssh/openssh/ssh_config
deleted file mode 100644
index cb2774a163ed..
--- a/meta/recipes-connectivity/openssh/openssh/ssh_config
+++ /dev/null
@@ -1,48 +0,0 @@
-#  $OpenBSD: ssh_config,v 1.35 2020/07/17 03:43:42 dtucker Exp $
-
-# This is the ssh client system-wide configuration file.  See
-# ssh_config(5) for more information.  This file provides defaults for
-# users, and the values can be changed in per-user configuration files
-# or on the command line.
-
-# Configuration data is parsed as follows:
-#  1. command line options
-#  2. user-specific file
-#  3. system-wide file
-# Any configuration value is only changed the first time it is set.
-# Thus, host-specific definitions should be at the beginning of the
-# configuration file, and defaults at the end.
-
-# Site-wide defaults for some commonly used options.  For a comprehensive
-# list of available options, their meanings and defaults, please see the
-# ssh_config(5) man page.
-
-Include /etc/ssh/ssh_config.d/*.conf
-
-# Host *
-#   ForwardAgent no
-#   ForwardX11 no
-#   PasswordAuthentication yes
-#   HostbasedAuthentication no
-#   GSSAPIAuthentication no
-#   GSSAPIDelegateCredentials no
-#   BatchMode no
-#   CheckHostIP yes
-#   AddressFamily any
-#   ConnectTimeout 0
-#   StrictHostKeyChecking ask
-#   IdentityFile ~/.ssh/id_rsa
-#   IdentityFile ~/.ssh/id_dsa
-#   IdentityFile ~/.ssh/id_ecdsa
-#   IdentityFile ~/.ssh/id_ed25519
-#   Port 22
-#   Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc
-#   MACs hmac-md5,hmac-sha1,umac...@openssh.com
-#   EscapeChar ~
-#   Tunnel no
-#   TunnelDevice any:any
-#   PermitLocalCommand no
-#   VisualHostKey no
-#   ProxyCommand ssh -q -W %h:%p gateway.example.com
-#   RekeyLimit 1G 1h
-#   UserKnownHostsFile ~/.ssh/known_hosts.d/%k
diff --git a/meta/recipes-connectivity/openssh/openssh/sshd_config 
b/meta/recipes-connectivity/openssh/openssh/sshd_config
deleted file mode 100644
index e9eaf9315775..
--- a/meta/recipes-connectivity/openssh/openssh/sshd_config
+++ /dev/null
@@ -1,119 +0,0 @@
-#  $OpenBSD: sshd_config,v 1.104 2021/07/02 05:11:21 dtucker Exp $
-
-# This is the sshd server system-wide configuration file.  See
-# sshd_config(5) for more information.
-
-# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin
-
-# The strategy used for options in the default sshd_config shipped with
-# OpenSSH is to specify options with their default value where
-# possible, but leave them commented.  Uncommented options override the
-# default value.
-
-Include /etc/ssh/sshd_config.d/*.conf
-
-#Port 22
-#AddressFamily any
-#ListenAddress 0.0.0.0
-#ListenAddress ::
-
-#HostKey /etc/ssh/ssh_host_rsa_key
-#HostKey /etc/ssh/ssh_host_ecdsa_key
-#HostKey /etc/ssh

Re: [OE-core] Patchtest results for [PATCH 2/7] openssh-config: initial checkin

[Enrico Scholz via lists.openembedded.org]

patcht...@automation.yoctoproject.org writes:

> FAIL: test lic files chksum modified not mentioned: LIC_FILES_CHKSUM changed 
> without "License-Update:" tag and description in commit message 
> (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)

This failure seems to be bogus; patch creates a new recipe with a new
LIC_FILES_CHKSUM.  It does not change/update an existing tag.


Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196963): 
https://lists.openembedded.org/g/openembedded-core/message/196963
Mute This Topic: https://lists.openembedded.org/mt/104868543/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 0/7] Replace sshd_config patching by snippets

[Enrico Scholz via lists.openembedded.org]

To deal with system setups, sshd was configured in the following way:

 - sshd_config is shipped completely by OE and DISTRO_FEATURES (pam,
   x11) are patched in during do_install

   --> this is difficulty to maintain; e.g. sshd_config must be
   synchronized between OpenSSH releases and OE adaptations
   manually inserted

 - two different configuration files (sshd_config + sshd_config_readonly)
   are created; IMAGE_FEATURES decides which one is used and it is patched
   in a ROOTFS_COMMAND in the system

   --> this make it difficult for third party recipes to incorporate
   their changes (they have to go over both files)

   --> the readonly HostKey locations and algorithms are hardcoded
   which makes it difficult to place them e.g. on a persistent
   /opt partition and disable e.g. ecdsa

 - depending on IMAGE_FEATURES (empty passwords, root login), both
   files are patched by a ROOTFS_POSTCOMMAND

   --> these changes are lost when pkgmgmt is used for the image and
   openssh being updated


The patchset:

 - reduces changes to sshd_config to

   | Include /etc/ssh/sshd_config.d/*.conf

   --> This is already the done in current recipe and most mainline
   Linux distributions are doing it

 - moves configuration in new openssh-config recipe which is a weak
   dependency of openssh (and can be replaced by another IMAGE_INSTALL)

   Recipe ships configuration as small snippets which might contain
   dynamically created content (e.g. 'UsePAM yes')

 - IMAGE_FEATURE based setup is done by creating subpackages with
   the corresponding options.  These subpackages are added to
   FEATURE_PACKAGES_ssh-server-openssh

 - readonly rootfs setup has been enhanced by

   | RO_KEYDIR ??= "/var/run/ssh"
   | KEY_ALGORITHMS ??= "rsa ecdsa ed25519"

   parameters which can be overridden.


Enrico Scholz (7):
  openssh: replace complete configuration files by patch
  openssh-config: initial checkin
  openssh: move configuration tweaking in configuration recipe
  image: prepare openssh configuration
  openssh: replace 'allow-empty-password' rootfs scipt by configuration
  openssh: replace 'allow-root-login' rootfs scipt by configuration
  openssh: move read-only-rootfs setup in configuration snippet

 meta/classes-recipe/core-image.bbclass|  19 ++-
 .../rootfs-postcommands.bbclass   |  25 +---
 .../openssh/openssh-config.bb |  51 
 .../60-allow-empty-password.conf  |   1 +
 .../openssh-config/60-allow-root-login.conf   |   1 +
 .../openssh/openssh-config/80-oe.conf |   5 +
 .../openssh/openssh/include-conf.patch|  32 +
 .../openssh/openssh/ssh_config|  48 ---
 .../openssh/openssh/sshd_config   | 119 --
 .../openssh/openssh_9.6p1.bb  |  20 +--
 10 files changed, 112 insertions(+), 209 deletions(-)
 create mode 100644 meta/recipes-connectivity/openssh/openssh-config.bb
 create mode 100644 
meta/recipes-connectivity/openssh/openssh-config/60-allow-empty-password.conf
 create mode 100644 
meta/recipes-connectivity/openssh/openssh-config/60-allow-root-login.conf
 create mode 100644 meta/recipes-connectivity/openssh/openssh-config/80-oe.conf
 create mode 100644 meta/recipes-connectivity/openssh/openssh/include-conf.patch
 delete mode 100644 meta/recipes-connectivity/openssh/openssh/ssh_config
 delete mode 100644 meta/recipes-connectivity/openssh/openssh/sshd_config

-- 
2.44.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196955): 
https://lists.openembedded.org/g/openembedded-core/message/196955
Mute This Topic: https://lists.openembedded.org/mt/104868003/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 7/7] openssh: move read-only-rootfs setup in configuration snippet

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

This patch replaces the duplicate 'sshd_config_readonly' configuration
file and logic behind by an extra packages which is installed when
corresponding IMAGE_FEATURES are set.

**NOTE**: this causes a regression when host keys are added manually
to the image.  Users have to do

| OPENSSH_FEATURE_CONFIGURATION:remove = "openssh-config-read-only-rootfs"

in this case, or create a .bbappend for openssh-config which sets
RO_KEYDIR.

Signed-off-by: Enrico Scholz 
---
 meta/classes-recipe/core-image.bbclass  | 12 
 meta/classes-recipe/rootfs-postcommands.bbclass | 13 +
 meta/recipes-connectivity/openssh/openssh-config.bb |  7 +++
 meta/recipes-connectivity/openssh/openssh_9.6p1.bb  |  7 +--
 4 files changed, 21 insertions(+), 18 deletions(-)

diff --git a/meta/classes-recipe/core-image.bbclass 
b/meta/classes-recipe/core-image.bbclass
index 10a2905d9a27..477b6ba47042 100644
--- a/meta/classes-recipe/core-image.bbclass
+++ b/meta/classes-recipe/core-image.bbclass
@@ -84,8 +84,20 @@ CORE_IMAGE_EXTRA_INSTALL ?= ""
 IMAGE_INSTALL ?= "${CORE_IMAGE_BASE_INSTALL}"
 
 OPENSSH_FEATURE_CONFIGURATION = "\
+${@'openssh-config-read-only-rootfs' if etc_is_readonly(d) else ''} \
 ${@bb.utils.contains_any('IMAGE_FEATURES', [ 'debug-tweaks', 
'allow-empty-password' ], 'openssh-config-allow-empty-password', '',d)} \
 ${@bb.utils.contains_any('IMAGE_FEATURES', [ 'debug-tweaks', 
'allow-root-login' ], 'openssh-config-allow-root-login', '',d)} \
 "
 
+def etc_is_readonly(d):
+features = (d.getVar('IMAGE_FEATURES') or "").split()
+
+if 'read-only-rootfs' not in features:
+return False
+
+if 'stateless-rootfs' in features:
+return True
+
+return 'overlayfs-etc' not in features
+
 inherit image
diff --git a/meta/classes-recipe/rootfs-postcommands.bbclass 
b/meta/classes-recipe/rootfs-postcommands.bbclass
index 633f88de6ec8..ba6eb84e4055 100644
--- a/meta/classes-recipe/rootfs-postcommands.bbclass
+++ b/meta/classes-recipe/rootfs-postcommands.bbclass
@@ -188,21 +188,10 @@ read_only_rootfs_hook () {
fi
 
# If we're using openssh and the /etc/ssh directory has no 
pre-generated keys,
-   # we should configure openssh to use the configuration file 
/etc/ssh/sshd_config_readonly
-   # and the keys under /var/run/ssh.
+   # we should configure dropbear to use the keys under /var/lib/dropbear
# If overlayfs-etc is used this is not done as /etc is treated as 
writable
# If stateless-rootfs is enabled this is always done as we don't want 
to save keys then
if ${@ 'true' if not bb.utils.contains('IMAGE_FEATURES', 
'overlayfs-etc', True, False, d) or bb.utils.contains('IMAGE_FEATURES', 
'stateless-rootfs', True, False, d) else 'false'}; then
-   if [ -d ${IMAGE_ROOTFS}/etc/ssh ]; then
-   if [ -e ${IMAGE_ROOTFS}/etc/ssh/ssh_host_rsa_key ]; then
-   echo "SYSCONFDIR=\${SYSCONFDIR:-/etc/ssh}" >> 
${IMAGE_ROOTFS}/etc/default/ssh
-   echo "SSHD_OPTS=" >> 
${IMAGE_ROOTFS}/etc/default/ssh
-   else
-   echo "SYSCONFDIR=\${SYSCONFDIR:-/var/run/ssh}" 
>> ${IMAGE_ROOTFS}/etc/default/ssh
-   echo "SSHD_OPTS='-f 
/etc/ssh/sshd_config_readonly'" >> ${IMAGE_ROOTFS}/etc/default/ssh
-   fi
-   fi
-
# Also tweak the key location for dropbear in the same way.
if [ -d ${IMAGE_ROOTFS}/etc/dropbear ]; then
if [ ! -e 
${IMAGE_ROOTFS}/etc/dropbear/dropbear_rsa_host_key ]; then
diff --git a/meta/recipes-connectivity/openssh/openssh-config.bb 
b/meta/recipes-connectivity/openssh/openssh-config.bb
index d4ed661d8299..d2d0d9f4ad0d 100644
--- a/meta/recipes-connectivity/openssh/openssh-config.bb
+++ b/meta/recipes-connectivity/openssh/openssh-config.bb
@@ -9,6 +9,9 @@ SRC_URI = "\
 file://80-oe.conf \
 "
 
+RO_KEYDIR ??= "/var/run/ssh"
+KEY_ALGORITHMS ??= "rsa ecdsa ed25519"
+
 do_install() {
 d=${D}${sysconfdir}/ssh/sshd_config.d
 install -d "$d"
@@ -27,6 +30,10 @@ do_install() {
 if [ "${@bb.utils.filter('DISTRO_FEATURES', 'x11', d)}" ]; then
 echo "X11Forwarding yes" >> "$f"
 fi
+
+for alg in ${KEY_ALGORITHMS}; do
+printf 'HostKey %s/ssh_host_%s_key\n' '${RO_KEYDIR}' "$alg"
+done > $d/60-read-only-rootfs.conf
 }
 
 python populate_packages:prepend() {
diff --git a/meta/recipes-connectivity/openssh/openssh_9.6p1.bb 
b/meta/recipes-connectivity/openssh/openssh_9.6p1.bb
index e792b459d838..29bc132de8ef 100644
--- a/meta/recipes-connectivity/openssh/openssh_9.6p1.bb
+++ b/meta/recipes-connectivity/openssh/openssh_9.6p1

[OE-core] [PATCH 4/7] image: prepare openssh configuration

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Add an OPENSSH_FEATURE_CONFIGURATION variable which will hold openssh
configuration packages.

Signed-off-by: Enrico Scholz 
---
 meta/classes-recipe/core-image.bbclass | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/classes-recipe/core-image.bbclass 
b/meta/classes-recipe/core-image.bbclass
index 40fc15cb04f2..adf236e0693e 100644
--- a/meta/classes-recipe/core-image.bbclass
+++ b/meta/classes-recipe/core-image.bbclass
@@ -55,7 +55,7 @@ FEATURE_PACKAGES_tools-sdk = "packagegroup-core-sdk 
packagegroup-core-standalone
 FEATURE_PACKAGES_nfs-server = "packagegroup-core-nfs-server"
 FEATURE_PACKAGES_nfs-client = "packagegroup-core-nfs-client"
 FEATURE_PACKAGES_ssh-server-dropbear = "packagegroup-core-ssh-dropbear"
-FEATURE_PACKAGES_ssh-server-openssh = "packagegroup-core-ssh-openssh"
+FEATURE_PACKAGES_ssh-server-openssh = "packagegroup-core-ssh-openssh 
${OPENSSH_FEATURE_CONFIGURATION}"
 FEATURE_PACKAGES_hwcodecs = "${MACHINE_HWCODECS}"
 
 
@@ -83,4 +83,7 @@ CORE_IMAGE_EXTRA_INSTALL ?= ""
 
 IMAGE_INSTALL ?= "${CORE_IMAGE_BASE_INSTALL}"
 
+OPENSSH_FEATURE_CONFIGURATION = "\
+"
+
 inherit image
-- 
2.44.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196959): 
https://lists.openembedded.org/g/openembedded-core/message/196959
Mute This Topic: https://lists.openembedded.org/mt/104868009/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/7] openssh: replace complete configuration files by patch

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Instead of shipping the whole configuration files for openssh, add
small patch includes configuration snippets from subdirectories.

This allows us to keep the original upstream configuration which is
mainly useful for documentation purposes.  It makes it more easy to
identify OE specific setup.

Signed-off-by: Enrico Scholz 
---
 .../openssh/openssh/include-conf.patch|  32 +
 .../openssh/openssh/ssh_config|  48 ---
 .../openssh/openssh/sshd_config   | 119 --
 .../openssh/openssh_9.6p1.bb  |   5 +-
 4 files changed, 33 insertions(+), 171 deletions(-)
 create mode 100644 meta/recipes-connectivity/openssh/openssh/include-conf.patch
 delete mode 100644 meta/recipes-connectivity/openssh/openssh/ssh_config
 delete mode 100644 meta/recipes-connectivity/openssh/openssh/sshd_config

diff --git a/meta/recipes-connectivity/openssh/openssh/include-conf.patch 
b/meta/recipes-connectivity/openssh/openssh/include-conf.patch
new file mode 100644
index ..3e0bd2957e25
--- /dev/null
+++ b/meta/recipes-connectivity/openssh/openssh/include-conf.patch
@@ -0,0 +1,32 @@
+Include configuration snippets from subdirectory.
+
+NOTE: first configuration option wins.
+
+Upstream-Status: Inappropriate
+
+Index: openssh-9.5p1/ssh_config
+===
+--- openssh-9.5p1.orig/ssh_config
 openssh-9.5p1/ssh_config
+@@ -5,6 +5,8 @@
+ # users, and the values can be changed in per-user configuration files
+ # or on the command line.
+ 
++Include /etc/ssh/ssh_config.d/*.conf
++
+ # Configuration data is parsed as follows:
+ #  1. command line options
+ #  2. user-specific file
+Index: openssh-9.5p1/sshd_config
+===
+--- openssh-9.5p1.orig/sshd_config
 openssh-9.5p1/sshd_config
+@@ -10,6 +10,8 @@
+ # possible, but leave them commented.  Uncommented options override the
+ # default value.
+ 
++Include /etc/ssh/sshd_config.d/*.conf
++
+ #Port 22
+ #AddressFamily any
+ #ListenAddress 0.0.0.0
diff --git a/meta/recipes-connectivity/openssh/openssh/ssh_config 
b/meta/recipes-connectivity/openssh/openssh/ssh_config
deleted file mode 100644
index cb2774a163ed..
--- a/meta/recipes-connectivity/openssh/openssh/ssh_config
+++ /dev/null
@@ -1,48 +0,0 @@
-#  $OpenBSD: ssh_config,v 1.35 2020/07/17 03:43:42 dtucker Exp $
-
-# This is the ssh client system-wide configuration file.  See
-# ssh_config(5) for more information.  This file provides defaults for
-# users, and the values can be changed in per-user configuration files
-# or on the command line.
-
-# Configuration data is parsed as follows:
-#  1. command line options
-#  2. user-specific file
-#  3. system-wide file
-# Any configuration value is only changed the first time it is set.
-# Thus, host-specific definitions should be at the beginning of the
-# configuration file, and defaults at the end.
-
-# Site-wide defaults for some commonly used options.  For a comprehensive
-# list of available options, their meanings and defaults, please see the
-# ssh_config(5) man page.
-
-Include /etc/ssh/ssh_config.d/*.conf
-
-# Host *
-#   ForwardAgent no
-#   ForwardX11 no
-#   PasswordAuthentication yes
-#   HostbasedAuthentication no
-#   GSSAPIAuthentication no
-#   GSSAPIDelegateCredentials no
-#   BatchMode no
-#   CheckHostIP yes
-#   AddressFamily any
-#   ConnectTimeout 0
-#   StrictHostKeyChecking ask
-#   IdentityFile ~/.ssh/id_rsa
-#   IdentityFile ~/.ssh/id_dsa
-#   IdentityFile ~/.ssh/id_ecdsa
-#   IdentityFile ~/.ssh/id_ed25519
-#   Port 22
-#   Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc
-#   MACs hmac-md5,hmac-sha1,umac...@openssh.com
-#   EscapeChar ~
-#   Tunnel no
-#   TunnelDevice any:any
-#   PermitLocalCommand no
-#   VisualHostKey no
-#   ProxyCommand ssh -q -W %h:%p gateway.example.com
-#   RekeyLimit 1G 1h
-#   UserKnownHostsFile ~/.ssh/known_hosts.d/%k
diff --git a/meta/recipes-connectivity/openssh/openssh/sshd_config 
b/meta/recipes-connectivity/openssh/openssh/sshd_config
deleted file mode 100644
index e9eaf9315775..
--- a/meta/recipes-connectivity/openssh/openssh/sshd_config
+++ /dev/null
@@ -1,119 +0,0 @@
-#  $OpenBSD: sshd_config,v 1.104 2021/07/02 05:11:21 dtucker Exp $
-
-# This is the sshd server system-wide configuration file.  See
-# sshd_config(5) for more information.
-
-# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin
-
-# The strategy used for options in the default sshd_config shipped with
-# OpenSSH is to specify options with their default value where
-# possible, but leave them commented.  Uncommented options override the
-# default value.
-
-Include /etc/ssh/sshd_config.d/*.conf
-
-#Port 22
-#AddressFamily any
-#ListenAddress 0.0.0.0
-#ListenAddress ::
-
-#HostKey /etc/ssh/ssh_host_rsa_key
-#HostKey /etc/ssh/ssh_host_ecdsa_key
-#HostKey /etc/ssh/ssh_host_ed25519_key

[OE-core] [PATCH 6/7] openssh: replace 'allow-root-login' rootfs scipt by configuration

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Install 'openssh-config-allow-root-login' when corresponding
IMAGE_FEATURES are active.

Signed-off-by: Enrico Scholz 
---
 meta/classes-recipe/core-image.bbclass  | 1 +
 meta/classes-recipe/rootfs-postcommands.bbclass | 6 --
 meta/recipes-connectivity/openssh/openssh-config.bb | 2 ++
 .../openssh/openssh-config/60-allow-root-login.conf | 1 +
 4 files changed, 4 insertions(+), 6 deletions(-)
 create mode 100644 
meta/recipes-connectivity/openssh/openssh-config/60-allow-root-login.conf

diff --git a/meta/classes-recipe/core-image.bbclass 
b/meta/classes-recipe/core-image.bbclass
index 63e0e99b2a56..10a2905d9a27 100644
--- a/meta/classes-recipe/core-image.bbclass
+++ b/meta/classes-recipe/core-image.bbclass
@@ -85,6 +85,7 @@ IMAGE_INSTALL ?= "${CORE_IMAGE_BASE_INSTALL}"
 
 OPENSSH_FEATURE_CONFIGURATION = "\
 ${@bb.utils.contains_any('IMAGE_FEATURES', [ 'debug-tweaks', 
'allow-empty-password' ], 'openssh-config-allow-empty-password', '',d)} \
+${@bb.utils.contains_any('IMAGE_FEATURES', [ 'debug-tweaks', 
'allow-root-login' ], 'openssh-config-allow-root-login', '',d)} \
 "
 
 inherit image
diff --git a/meta/classes-recipe/rootfs-postcommands.bbclass 
b/meta/classes-recipe/rootfs-postcommands.bbclass
index 88f88505b5ed..633f88de6ec8 100644
--- a/meta/classes-recipe/rootfs-postcommands.bbclass
+++ b/meta/classes-recipe/rootfs-postcommands.bbclass
@@ -268,12 +268,6 @@ ssh_allow_empty_password () {
 # allow dropbear/openssh to accept root logins
 #
 ssh_allow_root_login () {
-   for config in sshd_config sshd_config_readonly; do
-   if [ -e ${IMAGE_ROOTFS}${sysconfdir}/ssh/$config ]; then
-   sed -i 
's/^[#[:space:]]*PermitRootLogin.*/PermitRootLogin yes/' 
${IMAGE_ROOTFS}${sysconfdir}/ssh/$config
-   fi
-   done
-
if [ -e ${IMAGE_ROOTFS}${sbindir}/dropbear ] ; then
if grep -q DROPBEAR_EXTRA_ARGS 
${IMAGE_ROOTFS}${sysconfdir}/default/dropbear 2>/dev/null ; then
sed -i '/^DROPBEAR_EXTRA_ARGS=/ s/-w//' 
${IMAGE_ROOTFS}${sysconfdir}/default/dropbear
diff --git a/meta/recipes-connectivity/openssh/openssh-config.bb 
b/meta/recipes-connectivity/openssh/openssh-config.bb
index 20dfe086f8ab..d4ed661d8299 100644
--- a/meta/recipes-connectivity/openssh/openssh-config.bb
+++ b/meta/recipes-connectivity/openssh/openssh-config.bb
@@ -5,6 +5,7 @@ LIC_FILES_CHKSUM = 
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384
 
 SRC_URI = "\
 file://60-allow-empty-password.conf \
+file://60-allow-root-login.conf \
 file://80-oe.conf \
 "
 
@@ -13,6 +14,7 @@ do_install() {
 install -d "$d"
 install -p -m 0644 \
 ${WORKDIR}/60-allow-empty-password.conf \
+${WORKDIR}/60-allow-root-login.conf \
 ${WORKDIR}/80-oe.conf \
 "$d"/
 
diff --git 
a/meta/recipes-connectivity/openssh/openssh-config/60-allow-root-login.conf 
b/meta/recipes-connectivity/openssh/openssh-config/60-allow-root-login.conf
new file mode 100644
index ..1073982f77c1
--- /dev/null
+++ b/meta/recipes-connectivity/openssh/openssh-config/60-allow-root-login.conf
@@ -0,0 +1 @@
+PermitRootLogin yes
-- 
2.44.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196958): 
https://lists.openembedded.org/g/openembedded-core/message/196958
Mute This Topic: https://lists.openembedded.org/mt/104868007/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 5/7] openssh: replace 'allow-empty-password' rootfs scipt by configuration

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Install 'openssh-config-allow-empty-password' when corresponding
IMAGE_FEATURES are active.

Signed-off-by: Enrico Scholz 
---
 meta/classes-recipe/core-image.bbclass  | 1 +
 meta/classes-recipe/rootfs-postcommands.bbclass | 6 --
 meta/recipes-connectivity/openssh/openssh-config.bb | 2 ++
 .../openssh/openssh-config/60-allow-empty-password.conf | 1 +
 4 files changed, 4 insertions(+), 6 deletions(-)
 create mode 100644 
meta/recipes-connectivity/openssh/openssh-config/60-allow-empty-password.conf

diff --git a/meta/classes-recipe/core-image.bbclass 
b/meta/classes-recipe/core-image.bbclass
index adf236e0693e..63e0e99b2a56 100644
--- a/meta/classes-recipe/core-image.bbclass
+++ b/meta/classes-recipe/core-image.bbclass
@@ -84,6 +84,7 @@ CORE_IMAGE_EXTRA_INSTALL ?= ""
 IMAGE_INSTALL ?= "${CORE_IMAGE_BASE_INSTALL}"
 
 OPENSSH_FEATURE_CONFIGURATION = "\
+${@bb.utils.contains_any('IMAGE_FEATURES', [ 'debug-tweaks', 
'allow-empty-password' ], 'openssh-config-allow-empty-password', '',d)} \
 "
 
 inherit image
diff --git a/meta/classes-recipe/rootfs-postcommands.bbclass 
b/meta/classes-recipe/rootfs-postcommands.bbclass
index e81b69a239b5..88f88505b5ed 100644
--- a/meta/classes-recipe/rootfs-postcommands.bbclass
+++ b/meta/classes-recipe/rootfs-postcommands.bbclass
@@ -246,12 +246,6 @@ zap_empty_root_password () {
 # allow dropbear/openssh to accept logins from accounts with an empty password 
string
 #
 ssh_allow_empty_password () {
-   for config in sshd_config sshd_config_readonly; do
-   if [ -e ${IMAGE_ROOTFS}${sysconfdir}/ssh/$config ]; then
-   sed -i 
's/^[#[:space:]]*PermitEmptyPasswords.*/PermitEmptyPasswords yes/' 
${IMAGE_ROOTFS}${sysconfdir}/ssh/$config
-   fi
-   done
-
if [ -e ${IMAGE_ROOTFS}${sbindir}/dropbear ] ; then
if grep -q DROPBEAR_EXTRA_ARGS 
${IMAGE_ROOTFS}${sysconfdir}/default/dropbear 2>/dev/null ; then
if ! grep -q "DROPBEAR_EXTRA_ARGS=.*-B" 
${IMAGE_ROOTFS}${sysconfdir}/default/dropbear ; then
diff --git a/meta/recipes-connectivity/openssh/openssh-config.bb 
b/meta/recipes-connectivity/openssh/openssh-config.bb
index 312a1c903f63..20dfe086f8ab 100644
--- a/meta/recipes-connectivity/openssh/openssh-config.bb
+++ b/meta/recipes-connectivity/openssh/openssh-config.bb
@@ -4,6 +4,7 @@ LICENSE = "MIT"
 LIC_FILES_CHKSUM = 
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
 
 SRC_URI = "\
+file://60-allow-empty-password.conf \
 file://80-oe.conf \
 "
 
@@ -11,6 +12,7 @@ do_install() {
 d=${D}${sysconfdir}/ssh/sshd_config.d
 install -d "$d"
 install -p -m 0644 \
+${WORKDIR}/60-allow-empty-password.conf \
 ${WORKDIR}/80-oe.conf \
 "$d"/
 
diff --git 
a/meta/recipes-connectivity/openssh/openssh-config/60-allow-empty-password.conf 
b/meta/recipes-connectivity/openssh/openssh-config/60-allow-empty-password.conf
new file mode 100644
index ..04e75ab6cefa
--- /dev/null
+++ 
b/meta/recipes-connectivity/openssh/openssh-config/60-allow-empty-password.conf
@@ -0,0 +1 @@
+PermitEmptyPasswords yes
-- 
2.44.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196956): 
https://lists.openembedded.org/g/openembedded-core/message/196956
Mute This Topic: https://lists.openembedded.org/mt/104868004/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/7] openssh-config: initial checkin

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Place OE specific openssh setup (which was removed in a previous
patch) in a configuration snippet.

Signed-off-by: Enrico Scholz 
---
 .../openssh/openssh-config.bb | 30 +++
 .../openssh/openssh-config/80-oe.conf |  5 
 .../openssh/openssh_9.6p1.bb  |  2 ++
 3 files changed, 37 insertions(+)
 create mode 100644 meta/recipes-connectivity/openssh/openssh-config.bb
 create mode 100644 meta/recipes-connectivity/openssh/openssh-config/80-oe.conf

diff --git a/meta/recipes-connectivity/openssh/openssh-config.bb 
b/meta/recipes-connectivity/openssh/openssh-config.bb
new file mode 100644
index ..2ef6a770fcfe
--- /dev/null
+++ b/meta/recipes-connectivity/openssh/openssh-config.bb
@@ -0,0 +1,30 @@
+SUMMARY = "OE openssh configuration"
+DESCRIPTION = "Provides openssh configuration snippets which correspond to 
IMAGE_FEATURE"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = 
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+SRC_URI = "\
+file://80-oe.conf \
+"
+
+do_install() {
+d=${D}${sysconfdir}/ssh/sshd_config.d
+install -d "$d"
+install -p -m 0644 \
+${WORKDIR}/80-oe.conf \
+"$d"/
+}
+
+python populate_packages:prepend() {
+dir = d.expand('${sysconfdir}/ssh/sshd_config.d')
+
+do_split_packages(d, dir, '^60-(.*)\.conf',
+  output_pattern='${PN}-%s',
+  description="openssh config for '%s' feature",
+  extra_depends='',
+  prepend=True)
+}
+
+PACKAGES_DYNAMIC = "${PN}-*"
+
+FILES:${PN} = "${sysconfdir}/ssh/sshd_config.d/80-oe.conf"
diff --git a/meta/recipes-connectivity/openssh/openssh-config/80-oe.conf 
b/meta/recipes-connectivity/openssh/openssh-config/80-oe.conf
new file mode 100644
index ..9a92d3c8d197
--- /dev/null
+++ b/meta/recipes-connectivity/openssh/openssh-config/80-oe.conf
@@ -0,0 +1,5 @@
+KbdInteractiveAuthentication no
+
+Compression no
+ClientAliveInterval 15
+ClientAliveCountMax 4
diff --git a/meta/recipes-connectivity/openssh/openssh_9.6p1.bb 
b/meta/recipes-connectivity/openssh/openssh_9.6p1.bb
index d500ca6019fe..406b349f8283 100644
--- a/meta/recipes-connectivity/openssh/openssh_9.6p1.bb
+++ b/meta/recipes-connectivity/openssh/openssh_9.6p1.bb
@@ -191,6 +191,8 @@ RCONFLICTS:${PN}-sshd = "dropbear"
 CONFFILES:${PN}-sshd = "${sysconfdir}/ssh/sshd_config"
 CONFFILES:${PN}-ssh = "${sysconfdir}/ssh/ssh_config"
 
+RRECOMMENDS:${PN} += "openssh-config"
+
 ALTERNATIVE_PRIORITY = "90"
 ALTERNATIVE:${PN}-scp = "scp"
 ALTERNATIVE:${PN}-ssh = "ssh"
-- 
2.44.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196953): 
https://lists.openembedded.org/g/openembedded-core/message/196953
Mute This Topic: https://lists.openembedded.org/mt/104868001/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 3/7] openssh: move configuration tweaking in configuration recipe

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Instead of applying DISTRO_FEATURE based setup directly to
sshd_config, add it to our configuration snippet.

Signed-off-by: Enrico Scholz 
---
 meta/recipes-connectivity/openssh/openssh-config.bb | 10 ++
 meta/recipes-connectivity/openssh/openssh_9.6p1.bb  |  8 
 2 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/meta/recipes-connectivity/openssh/openssh-config.bb 
b/meta/recipes-connectivity/openssh/openssh-config.bb
index 2ef6a770fcfe..312a1c903f63 100644
--- a/meta/recipes-connectivity/openssh/openssh-config.bb
+++ b/meta/recipes-connectivity/openssh/openssh-config.bb
@@ -13,6 +13,16 @@ do_install() {
 install -p -m 0644 \
 ${WORKDIR}/80-oe.conf \
 "$d"/
+
+f=$d/80-oe.conf
+
+if [ "${@bb.utils.filter('DISTRO_FEATURES', 'pam', d)}" ]; then
+echo "UsePAM yes" >> "$f"
+fi
+
+if [ "${@bb.utils.filter('DISTRO_FEATURES', 'x11', d)}" ]; then
+echo "X11Forwarding yes" >> "$f"
+fi
 }
 
 python populate_packages:prepend() {
diff --git a/meta/recipes-connectivity/openssh/openssh_9.6p1.bb 
b/meta/recipes-connectivity/openssh/openssh_9.6p1.bb
index 406b349f8283..e792b459d838 100644
--- a/meta/recipes-connectivity/openssh/openssh_9.6p1.bb
+++ b/meta/recipes-connectivity/openssh/openssh_9.6p1.bb
@@ -109,11 +109,6 @@ do_compile_ptest() {
 do_install:append () {
if [ "${@bb.utils.filter('DISTRO_FEATURES', 'pam', d)}" ]; then
install -D -m 0644 ${WORKDIR}/sshd ${D}${sysconfdir}/pam.d/sshd
-   sed -i -e 's:#UsePAM no:UsePAM yes:' 
${D}${sysconfdir}/ssh/sshd_config
-   fi
-
-   if [ "${@bb.utils.filter('DISTRO_FEATURES', 'x11', d)}" ]; then
-   sed -i -e 's:#X11Forwarding no:X11Forwarding yes:' 
${D}${sysconfdir}/ssh/sshd_config
fi
 
install -d ${D}${sysconfdir}/init.d
@@ -188,9 +183,6 @@ RPROVIDES:${PN}-sshd = "sshd"
 RCONFLICTS:${PN} = "dropbear"
 RCONFLICTS:${PN}-sshd = "dropbear"
 
-CONFFILES:${PN}-sshd = "${sysconfdir}/ssh/sshd_config"
-CONFFILES:${PN}-ssh = "${sysconfdir}/ssh/ssh_config"
-
 RRECOMMENDS:${PN} += "openssh-config"
 
 ALTERNATIVE_PRIORITY = "90"
-- 
2.44.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196954): 
https://lists.openembedded.org/g/openembedded-core/message/196954
Mute This Topic: https://lists.openembedded.org/mt/104868002/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] tcp-wrappers: drop libnsl2 build dependency

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

The only libnsl2 function which is used by tcp-wrappers is
'yp_get_default_domain()'.  When USE_GETDOMAIN is set, this is
implemented as a simple wrapper around getdomainname() so that libnsl2
is not used at all.

We added a patch which does '#include ' to avoid
implicit forward declaration.  By conditionalizing this, the libnsl2
dependency can be dropped completely.

Signed-off-by: Enrico Scholz 
---
 ...0001-Fix-implicit-function-declaration-warnings.patch | 9 +++--
 meta/recipes-extended/tcp-wrappers/tcp-wrappers_7.6.bb   | 2 --
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git 
a/meta/recipes-extended/tcp-wrappers/tcp-wrappers-7.6/0001-Fix-implicit-function-declaration-warnings.patch
 
b/meta/recipes-extended/tcp-wrappers/tcp-wrappers-7.6/0001-Fix-implicit-function-declaration-warnings.patch
index ec793ac8ffed..474703885d9b 100644
--- 
a/meta/recipes-extended/tcp-wrappers/tcp-wrappers-7.6/0001-Fix-implicit-function-declaration-warnings.patch
+++ 
b/meta/recipes-extended/tcp-wrappers/tcp-wrappers-7.6/0001-Fix-implicit-function-declaration-warnings.patch
@@ -20,11 +20,16 @@ diff --git a/hosts_access.c b/hosts_access.c
 index 0133e5e..58697ea 100644
 --- a/hosts_access.c
 +++ b/hosts_access.c
-@@ -33,6 +33,7 @@ static char sccsid[] = "@(#) hosts_access.c 1.21 97/02/12 
02:13:22";
+@@ -33,6 +33,12 @@ static char sccsid[] = "@(#) hosts_access.c 1.21 97/02/12 
02:13:22";
  #endif
  #include 
  #include 
-+#include 
++#ifdef USE_GETDOMAIN
++/* defined in workarounds.c */
++extern int yp_get_default_domain(char **ptr);
++#else
++#  include 
++#endif/* USE_GETDOMAIN */
  #include 
  #include 
  #include 
diff --git a/meta/recipes-extended/tcp-wrappers/tcp-wrappers_7.6.bb 
b/meta/recipes-extended/tcp-wrappers/tcp-wrappers_7.6.bb
index c655da1199e7..bcd1d6f7929e 100644
--- a/meta/recipes-extended/tcp-wrappers/tcp-wrappers_7.6.bb
+++ b/meta/recipes-extended/tcp-wrappers/tcp-wrappers_7.6.bb
@@ -7,8 +7,6 @@ SECTION = "console/network"
 LICENSE = "BSD-1-Clause"
 LIC_FILES_CHKSUM = "file://DISCLAIMER;md5=071bd69cb78b1ea5e3da5c3127fa"
 
-DEPENDS += "libnsl2"
-
 PACKAGES = "${PN}-dbg libwrap libwrap-doc libwrap-dev libwrap-staticdev ${PN} 
${PN}-doc"
 FILES:libwrap = "${base_libdir}/lib*${SOLIBS}"
 FILES:libwrap-doc = "${mandir}/man3 ${mandir}/man5"
-- 
2.43.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193285): 
https://lists.openembedded.org/g/openembedded-core/message/193285
Mute This Topic: https://lists.openembedded.org/mt/103500355/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] rust: why is it built in do_install()

[Enrico Scholz via lists.openembedded.org]

Alexander Kanavin  writes:

> Any changes would have to fulfil this:
>
> 1. Rust's installation procedure should run under pseudo, e.g. in
> do_install.

The current recipe does *not* run installation under pseudo

| rust_do_install:class-target() {
| export PSEUDO_UNLOAD=1
| rust_runx install

Only class-native runs under pseudo (or is pseudo automatically disabled
for this class?).


Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#187025): 
https://lists.openembedded.org/g/openembedded-core/message/187025
Mute This Topic: https://lists.openembedded.org/mt/101098826/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] rust: why is it built in do_install()

[Enrico Scholz via lists.openembedded.org]

Hello,

rust recipe does

| do_compile () {
| }
|  
| rust_do_install() {
| rust_runx install
| }
| 
| rust_do_install:class-nativesdk() {
| export PSEUDO_UNLOAD=1
| rust_runx install
| rust_runx install clippy
| rust_runx install rustfmt


What is the reason to run the expensive "rust_runx" in do_install() and
not in do_compile().

This:

- is unexpected... do_install is usually expected to run fast

- might interfere with different ${PARALLEL_MAKE} and ${PARALLEL_MAKEINST}
  setup

- might slow down the build process because the 'class-native' variant
  seems to run with active 'pseudo'


I guess it is done in this way because of

|config.set("install", "prefix",  e(d.getVar("D") + d.getVar("prefix")))

where '${D}' is cleared by do_install.


Wouldn't it be better to replace this by

|config.set("install", "prefix",  e(d.getVar("B") + '/_install' + 
d.getVar("prefix")))


| do_compile () {
| rust_runx install
| rust_runx install clippy
| rust_runx install rustfmt
| }
|
| rust_do_install() {
| cp -a ${B}/_install ${D}

?


[adding 'rustfmt' to 'class-native' would be nice because e.g. 'bindgen'
 users might expect it]



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#187012): 
https://lists.openembedded.org/g/openembedded-core/message/187012
Mute This Topic: https://lists.openembedded.org/mt/101098826/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] oe.data: allow to mask out secret variables

[Enrico Scholz via lists.openembedded.org]

Alexander Kanavin  writes:

>> Else, there are sometimes not many ways to work without them.
>> E.g. SSTATE_MIRRORS has contain the secret token because it is
>> used directly by bitbake; perhaps I could use a wget wrapper and
>> write a custom curl python class...
>
> Yes, the secret needs to be in a file (or other access-controlled
> facility), and read from it by the process that needs it, and only
> directly prior to using it. Having it in a bitbake variable which gets
> passed through a million tasks and components

Where is the problem?  I known only one component
(rootfs-postcommands.bbclass) which dumps the whole environment and
leaks it.

Else, when there is a malicious component that wants to steal secrets
from a bitbake variable, what would stop it from reading the secret from
a file?

Your suggestion (write secrets in files instead of bitbake variables)
does not improve security but causes only extra work.



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#184881): 
https://lists.openembedded.org/g/openembedded-core/message/184881
Mute This Topic: https://lists.openembedded.org/mt/100368202/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] oe.data: allow to mask out secret variables

[Enrico Scholz via lists.openembedded.org]

Alexander Kanavin  writes:

> Please no. These things can leak out in a million other ways

no; that is very unlikely.  The parts which are dealing with secrets
usually take care about not leaking them.

All major CI systems have the same problem (need secret variables) and
at least gitlab solves it in the same way (mark it as to be masked and
replace it in logs).


> (e.g. if you publish logs),

Secrets do not appear in the usual 'bitbake ...' output only in the deep
.../temp/log.do_* files.

I do not think that people are really publishing these files.


> it's better to just scrub them prior to publishing with a post-script.

Sounds unergonomic; you have to know which variables are secret.  You
have to read and interpret the testdata.json file, substitute values and
write it back.

It is much better to do it in the first place.  The classes which are
dealing with secrets can mark them as such.


> Having secrets in bitbake variables is a bad idea to begin with.

Yes; because they are exported in testdata.json ;)

Else, there are sometimes not many ways to work without them.
E.g. SSTATE_MIRRORS has contain the secret token because it is used
directly by bitbake; perhaps I could use a wget wrapper and write a
custom curl python class...



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#184878): 
https://lists.openembedded.org/g/openembedded-core/message/184878
Mute This Topic: https://lists.openembedded.org/mt/100368202/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] oe.data: allow to mask out secret variables

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Some integrations require that passwords or secret tokens are
assigned to bitbake variables.  E.g. the meta-dependencytrack
layer has a 'DEPENDENCYTRACK_API_KEY' or my sstate-server requires
a 'SSTATE_SERVER_SESSION' token.

These secrets will appear in testdata.json which can easily leak them
when the deploy directory is published publicly.

Patch adds a special 'secrets' flag for variables.  When a variable is
marked with it, its content will be replaced by '**masked**'.

E.g. formerly

|"SSTATE_SERVER_PATH": "HKBOZ8C279S4iwBA",
|"SSTATE_MIRRORS": "... 
https://sstate/api/v1/download/HKBOZ8C279S4iwBA/sstate/...

and now

|"SSTATE_SERVER_PATH": "**masked**",
|"SSTATE_MIRRORS": "... 
https://sstate/api/v1/download/**masked**/sstate

Corresponding bbclass contains

| SSTATE_SERVER_PATH ??= "-"
| SSTATE_SERVER_PATH[secret] = "true"

Signed-off-by: Enrico Scholz 
---
 meta/lib/oe/data.py | 29 +++--
 1 file changed, 27 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/data.py b/meta/lib/oe/data.py
index 37121cfad2b7..57a8e5b5e049 100644
--- a/meta/lib/oe/data.py
+++ b/meta/lib/oe/data.py
@@ -5,7 +5,9 @@
 #
 
 import json
+import re
 import oe.maketype
+import oe.types
 
 def typed_value(key, d):
 """Construct a value for the specified metadata variable, using its flags
@@ -23,9 +25,10 @@ def typed_value(key, d):
 except (TypeError, ValueError) as exc:
 bb.msg.fatal("Data", "%s: %s" % (key, str(exc)))
 
-def export2json(d, json_file, expand=True, searchString="",replaceString=""):
+def export2json(d, json_file, expand=True, searchString="",replaceString="", 
mask_secrets=True):
 data2export = {}
 keys2export = []
+secrets = []
 
 for key in d.keys():
 if key.startswith("_"):
@@ -38,12 +41,34 @@ def export2json(d, json_file, expand=True, 
searchString="",replaceString=""):
 continue
 elif d.getVarFlag(key, "func"):
 continue
+elif mask_secrets and oe.types.boolean(d.getVarFlag(key, "secret") or 
"false"):
+var = d.getVar(key)
+
+## When secret variable contains a placeholder (is empty
+## or single character), show it.  When it is too short
+## emit a warning and exclude it completely from output
+## but do not mask out its value in other places.
+if var is None or len(var) < 2:
+bb.debug(1, "variable '%s' is marked as secret but seems to 
contain some placeholder; showing it" % key)
+elif len(var) < 8:
+bb.warn("variable '%s' is marked as secret but content is too 
short; skipping it" % key)
+continue
+else:
+secrets.append(re.escape(var))
 
 keys2export.append(key)
 
+if len(secrets) == 0:
+secrets = None
+else:
+secrets = re.compile('|'.join(secrets))
+
 for key in keys2export:
 try:
-data2export[key] = d.getVar(key, 
expand).replace(searchString,replaceString)
+var = d.getVar(key, expand).replace(searchString,replaceString)
+if secrets:
+var = secrets.sub("**masked**", var)
+data2export[key] = var
 except bb.data_smart.ExpansionError:
 data2export[key] = ''
 except AttributeError:
-- 
2.41.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#184876): 
https://lists.openembedded.org/g/openembedded-core/message/184876
Mute This Topic: https://lists.openembedded.org/mt/100368202/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2] shadow-sysroot: add license information

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Recipe references 'login.defs' in LIC_FILES_CHKSUM.  This causes some
problems:

- file does not contain a single word which is related with its license

- changing this file (here: increasing SYS_UID_MIN) invalidates
  LIC_FILES_CHKSUM

Add 'SPDX-License-Identifier' to the file and limit the checksum to
this part.

Signed-off-by: Enrico Scholz 
---
 meta/recipes-extended/shadow/files/login.defs_shadow-sysroot | 1 +
 meta/recipes-extended/shadow/shadow-sysroot_4.6.bb   | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-extended/shadow/files/login.defs_shadow-sysroot 
b/meta/recipes-extended/shadow/files/login.defs_shadow-sysroot
index 8a68dd341a61..09df77d2e751 100644
--- a/meta/recipes-extended/shadow/files/login.defs_shadow-sysroot
+++ b/meta/recipes-extended/shadow/files/login.defs_shadow-sysroot
@@ -1,3 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause OR Artistic-1.0
 #
 # /etc/login.defs - Configuration control definitions for the shadow package.
 #
diff --git a/meta/recipes-extended/shadow/shadow-sysroot_4.6.bb 
b/meta/recipes-extended/shadow/shadow-sysroot_4.6.bb
index e05fa237a240..6580bd916644 100644
--- a/meta/recipes-extended/shadow/shadow-sysroot_4.6.bb
+++ b/meta/recipes-extended/shadow/shadow-sysroot_4.6.bb
@@ -3,7 +3,7 @@ HOMEPAGE = "http://github.com/shadow-maint/shadow;
 BUGTRACKER = "http://github.com/shadow-maint/shadow/issues;
 SECTION = "base utils"
 LICENSE = "BSD-3-Clause | Artistic-1.0"
-LIC_FILES_CHKSUM = 
"file://login.defs_shadow-sysroot;md5=25e2f2de4dfc8f966ac5cdfce45cd7d5"
+LIC_FILES_CHKSUM = 
"file://login.defs_shadow-sysroot;endline=1;md5=ceddfb61608e4db87012499555184aed"
 
 DEPENDS = "base-passwd"
 
-- 
2.41.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#184258): 
https://lists.openembedded.org/g/openembedded-core/message/184258
Mute This Topic: https://lists.openembedded.org/mt/100138690/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] shadow-sysroot: add license information

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Recipe references 'login.defs' in LIC_FILES_CHKSUM.  This causes some
problems:

- file does not contain a single word which is related with its license

- changing this file (here: increasing SYS_UID_MIN) invalidates
  LIC_FILES_CHKSUM

Add 'SPDX-License-Identifier' to the file and limit the checksum to
this part.

Signed-off-by: Enrico Scholz 
---
 meta/recipes-extended/shadow/files/login.defs_shadow-sysroot | 1 +
 meta/recipes-extended/shadow/shadow-sysroot_4.6.bb   | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-extended/shadow/files/login.defs_shadow-sysroot 
b/meta/recipes-extended/shadow/files/login.defs_shadow-sysroot
index 8a68dd341a61..a82f7969b19e 100644
--- a/meta/recipes-extended/shadow/files/login.defs_shadow-sysroot
+++ b/meta/recipes-extended/shadow/files/login.defs_shadow-sysroot
@@ -1,3 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause | Artistic-1.0
 #
 # /etc/login.defs - Configuration control definitions for the shadow package.
 #
diff --git a/meta/recipes-extended/shadow/shadow-sysroot_4.6.bb 
b/meta/recipes-extended/shadow/shadow-sysroot_4.6.bb
index e05fa237a240..5f3c7e121f91 100644
--- a/meta/recipes-extended/shadow/shadow-sysroot_4.6.bb
+++ b/meta/recipes-extended/shadow/shadow-sysroot_4.6.bb
@@ -3,7 +3,7 @@ HOMEPAGE = "http://github.com/shadow-maint/shadow;
 BUGTRACKER = "http://github.com/shadow-maint/shadow/issues;
 SECTION = "base utils"
 LICENSE = "BSD-3-Clause | Artistic-1.0"
-LIC_FILES_CHKSUM = 
"file://login.defs_shadow-sysroot;md5=25e2f2de4dfc8f966ac5cdfce45cd7d5"
+LIC_FILES_CHKSUM = 
"file://login.defs_shadow-sysroot;endline=1;md5=ce8f87ba229dc6de3285a0aac9199544"
 
 DEPENDS = "base-passwd"
 
-- 
2.41.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#184257): 
https://lists.openembedded.org/g/openembedded-core/message/184257
Mute This Topic: https://lists.openembedded.org/mt/100138576/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] RFE: add 'LAYERDIR_' variable

[Enrico Scholz via lists.openembedded.org]

Hi,

currently, it is difficult to reuse recipes from other layers because
their path is required but not known.  E.g. when one tries to add a
backported imx-gpu-viv recipe [1] to a local layer by

| require recipes-graphics/imx-gpu-viv/imx-gpu-viv-6.inc

do_fetch() will fail because some SRC_URI can not be resolved.  To
support this, FILESEXTRAPATHS must contain the path of the original
recipe.

Determining this path is difficult at the moment, because it is unknown
and depends on the layout of BBLAYERS.


I suggest to add a new standard variable 'LAYERDIR_' which
points to '${LAYERDIR}' of the layer.  It is probably difficult to set
it automatically (value of  is not known); instead of, it must
be defined manually in 'layer.conf' like related variables.

E.g.

| BBFILE_COLLECTIONS += "freescale-layer"
| BBFILE_PATTERN_freescale-layer := "^${LAYERDIR}/"
| 
| LAYERDIR_freescale-layer := "${LAYERDIR}"<<

Diagnostic should be easy (iterate over BBFILE_COLLECTIONS and emit
warning when corresponding 'LAYERDIR_' is undefined).


What do you think?


Enrico

Footnotes:
[1]  
https://github.com/Freescale/meta-freescale/blob/master/recipes-graphics/imx-gpu-viv/imx-gpu-viv_6.4.11.p1.0-aarch64.bb

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#184185): 
https://lists.openembedded.org/g/openembedded-core/message/184185
Mute This Topic: https://lists.openembedded.org/mt/100097753/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2] sstate: show progress bar again

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Transition to ThreadPoolExecutor (eb6a6820928472ef194b963b606454e731f9486f)
broke the

| Checking sstate mirror object availability: ...

progress bar because the removed 'thread_worker' was still referenced
in an asynchronous function.  As the result of the future is never
read, the resulting backtrace is silently discarded.

Replace the information given to 'ProcessProgress' by a counter.

Signed-off-by: Enrico Scholz 
---
 meta/classes-global/sstate.bbclass | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/classes-global/sstate.bbclass 
b/meta/classes-global/sstate.bbclass
index 2c8e7b8cc232..f5e0e623e28d 100644
--- a/meta/classes-global/sstate.bbclass
+++ b/meta/classes-global/sstate.bbclass
@@ -925,6 +925,8 @@ sstate_unpack_package () {
 BB_HASHCHECK_FUNCTION = "sstate_checkhashes"
 
 def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, 
summary=True, **kwargs):
+import itertools
+
 found = set()
 missed = set()
 
@@ -1019,7 +1021,7 @@ def sstate_checkhashes(sq_data, d, siginfo=False, 
currentcount=0, summary=True,
 connection_cache_pool.put(connection_cache)
 
 if progress:
-bb.event.fire(bb.event.ProcessProgress(msg, len(tasklist) - 
thread_worker.tasks.qsize()), d)
+bb.event.fire(bb.event.ProcessProgress(msg, 
next(cnt_tasks_done)), d)
 
 tasklist = []
 for tid in missed:
@@ -1029,6 +1031,8 @@ def sstate_checkhashes(sq_data, d, siginfo=False, 
currentcount=0, summary=True,
 if tasklist:
 nproc = min(int(d.getVar("BB_NUMBER_THREADS")), len(tasklist))
 
+## thread-safe counter
+cnt_tasks_done = itertools.count(start = 1)
 progress = len(tasklist) >= 100
 if progress:
 msg = "Checking sstate mirror object availability"
-- 
2.38.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173515): 
https://lists.openembedded.org/g/openembedded-core/message/173515
Mute This Topic: https://lists.openembedded.org/mt/95130758/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] sstate: show progress bar again

[Enrico Scholz via lists.openembedded.org]

Jose Quaresma  writes:

>> Transition to ThreadPoolExecutor (eb6a6820928472ef194b963b606454e731f9486f)
>> broke he
>>
>> | Checking sstate mirror object availability: ...
>>
>> progress bar because the removed 'thread_worker' was still referenced
>> in an asynchronous function.  As the result of the future is never
>> read, the resulting backtrace is silently discarded.
>
> I have tested the patch at the time and the progress bar works as expected,
> maybe something breaks after that.

Probably, you saw only the full 100% bar produced by
'bb.event.ProcessFinished'

But the patch does

| -def checkstatus(thread_worker, arg):
| +def checkstatus(arg):
|  ...
|  bb.event.fire(bb.event.ProcessProgress(msg, len(tasklist) - 
thread_worker.tasks.qsize()), d)

This will always raise an exception and 'bb.event.ProcessProgress' will
never be fired.


> Can the num_tasks_done be moved to there and call the first
> next(num_tasks_done) so we can remove the +1 on the ProcessProgress

I will send a v2



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173418): 
https://lists.openembedded.org/g/openembedded-core/message/173418
Mute This Topic: https://lists.openembedded.org/mt/95064126/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] sstate: show progress bar again

[Enrico Scholz via lists.openembedded.org]

From: Enrico Scholz 

Transition to ThreadPoolExecutor (eb6a6820928472ef194b963b606454e731f9486f)
broke the

| Checking sstate mirror object availability: ...

progress bar because the removed 'thread_worker' was still referenced
in an asynchronous function.  As the result of the future is never
read, the resulting backtrace is silently discarded.

Replace the information given to 'ProcessProgress' by a counter.

Signed-off-by: Enrico Scholz 
---
 meta/classes-global/sstate.bbclass | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/classes-global/sstate.bbclass 
b/meta/classes-global/sstate.bbclass
index 2c8e7b8cc232..3fabca4a3160 100644
--- a/meta/classes-global/sstate.bbclass
+++ b/meta/classes-global/sstate.bbclass
@@ -925,6 +925,8 @@ sstate_unpack_package () {
 BB_HASHCHECK_FUNCTION = "sstate_checkhashes"
 
 def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, 
summary=True, **kwargs):
+import itertools
+
 found = set()
 missed = set()
 
@@ -1019,7 +1021,7 @@ def sstate_checkhashes(sq_data, d, siginfo=False, 
currentcount=0, summary=True,
 connection_cache_pool.put(connection_cache)
 
 if progress:
-bb.event.fire(bb.event.ProcessProgress(msg, len(tasklist) - 
thread_worker.tasks.qsize()), d)
+bb.event.fire(bb.event.ProcessProgress(msg, 
next(num_tasks_done) + 1), d)
 
 tasklist = []
 for tid in missed:
@@ -1029,6 +1031,8 @@ def sstate_checkhashes(sq_data, d, siginfo=False, 
currentcount=0, summary=True,
 if tasklist:
 nproc = min(int(d.getVar("BB_NUMBER_THREADS")), len(tasklist))
 
+## thread-safe counter
+num_tasks_done = itertools.count()
 progress = len(tasklist) >= 100
 if progress:
 msg = "Checking sstate mirror object availability"
-- 
2.38.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173383): 
https://lists.openembedded.org/g/openembedded-core/message/173383
Mute This Topic: https://lists.openembedded.org/mt/95064126/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

DKIM compatible maillist setup (was: [OE-core] [PATCH 1/2] npm.bbclass: fix typo in 'fund' config option)

[Enrico Scholz via lists.openembedded.org]

Luca Ceresoli  writes:

> As you can see above, your sender address is getting mangled. This is
> not your fault, it is done by DMARC for anti-phishing, but it makes
> applying your patches annoying.
>
> Can you please try to work around that by setting the sendemail.from
> parameter in your git config?

Thanks for pointing me to this git option.

Would it be possible to configure the maillist so that it does not
mangle the message (no "unsubscribe" footer or "[OE-core]" subject
prefix)?  Or is this not supported with groups.io?



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169714): 
https://lists.openembedded.org/g/openembedded-core/message/169714
Mute This Topic: https://lists.openembedded.org/mt/93206041/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/2] npm.bbclass: fix architecture mapping

[Enrico Scholz via lists.openembedded.org]

Use the same code as the 'nodejs_16.4.bb recipe' for mapping the OE
arch to the NPM arch.

A noticeable change (and fix for exiting problems) is the move from
'arm' to 'arm64' for 'aarch64'.

Signed-off-by: Enrico Scholz 
---
 meta/classes-recipe/npm.bbclass | 24 +++-
 1 file changed, 11 insertions(+), 13 deletions(-)

diff --git a/meta/classes-recipe/npm.bbclass b/meta/classes-recipe/npm.bbclass
index 8e73400678db..20350cea255f 100644
--- a/meta/classes-recipe/npm.bbclass
+++ b/meta/classes-recipe/npm.bbclass
@@ -28,20 +28,18 @@ NPM_INSTALL_DEV ?= "0"
 
 NPM_NODEDIR ?= "${RECIPE_SYSROOT_NATIVE}${prefix_native}"
 
-def npm_target_arch_map(target_arch):
-"""Maps arch names to npm arch names"""
+## must match mapping in nodejs.bb (openembedded-meta)
+def map_nodejs_arch(a, d):
 import re
-if re.match("p(pc|owerpc)(|64)", target_arch):
-return "ppc"
-elif re.match("i.86$", target_arch):
-return "ia32"
-elif re.match("x86_64$", target_arch):
-return "x64"
-elif re.match("arm64$", target_arch):
-return "arm"
-return target_arch
-
-NPM_ARCH ?= "${@npm_target_arch_map(d.getVar("TARGET_ARCH"))}"
+
+if   re.match('i.86$', a): return 'ia32'
+elif re.match('x86_64$', a): return 'x64'
+elif re.match('aarch64$', a): return 'arm64'
+elif re.match('(powerpc64|powerpc64le|ppc64le)$', a): return 'ppc64'
+elif re.match('powerpc$', a): return 'ppc'
+return a
+
+NPM_ARCH ?= "${@map_nodejs_arch(d.getVar("TARGET_ARCH"), d)}"
 
 NPM_PACKAGE = "${WORKDIR}/npm-package"
 NPM_CACHE = "${WORKDIR}/npm-cache"
-- 
2.37.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169668): 
https://lists.openembedded.org/g/openembedded-core/message/169668
Mute This Topic: https://lists.openembedded.org/mt/93178725/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/2] npm.bbclass: fix typo in 'fund' config option

[Enrico Scholz via lists.openembedded.org]

Configuration option is named 'fund', not 'funds'.

Signed-off-by: Enrico Scholz 
---
 meta/classes-recipe/npm.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-recipe/npm.bbclass b/meta/classes-recipe/npm.bbclass
index deea53c9ec65..8e73400678db 100644
--- a/meta/classes-recipe/npm.bbclass
+++ b/meta/classes-recipe/npm.bbclass
@@ -54,7 +54,7 @@ def npm_global_configs(d):
 # Ensure no network access is done
 configs.append(("offline", "true"))
 configs.append(("proxy", "http://invalid;))
-configs.append(("funds", False))
+configs.append(("fund", False))
 configs.append(("audit", False))
 # Configure the cache directory
 configs.append(("cache", d.getVar("NPM_CACHE")))
-- 
2.37.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169669): 
https://lists.openembedded.org/g/openembedded-core/message/169669
Mute This Topic: https://lists.openembedded.org/mt/93178726/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] Error building an npm package: npmignore: not found

[Enrico Scholz via lists.openembedded.org]

"Michael Opdenacker via lists.openembedded.org"
 writes:

> I'm testing the Yocto Project Documentation for creating NPM packages

npm is broken since kirkstone; see
https://patchwork.yoctoproject.org/project/oe-core/list/?series=4303 for
patches.



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167327): 
https://lists.openembedded.org/g/openembedded-core/message/167327
Mute This Topic: https://lists.openembedded.org/mt/91903472/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 4/6, v2] npm: disable 'audit' + 'fund'

[Enrico Scholz via lists.openembedded.org]

'audit' can cause extra network traffic; 'fund' is not needed.

Signed-off-by: Enrico Scholz 
---
 meta/classes/npm.bbclass | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 11c80a738e..b613c8e8f5 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -53,6 +53,8 @@ def npm_global_configs(d):
 # Ensure no network access is done
 configs.append(("offline", "true"))
 configs.append(("proxy", "http://invalid;))
+configs.append(("fund", False))
+configs.append(("audit", False))
 # Configure the cache directory
 configs.append(("cache", d.getVar("NPM_CACHE")))
 return configs
-- 
2.36.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165889): 
https://lists.openembedded.org/g/openembedded-core/message/165889
Mute This Topic: https://lists.openembedded.org/mt/91209209/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 0/6] npm.bbclass: work with nodejs 16

[Enrico Scholz via lists.openembedded.org]

"Enrico Scholz via lists.openembedded.org"
 writes:

> This patchset requires an additional one in the oe-meta layer.

for reference: https://lists.openembedded.org/g/openembedded-devel/message/97186

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165880): 
https://lists.openembedded.org/g/openembedded-core/message/165880
Mute This Topic: https://lists.openembedded.org/mt/91205437/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/6] npm: replace 'npm pack' call by 'tar czf'

[Enrico Scholz via lists.openembedded.org]

'npm pack' is a maintainer tool which tries to execute 'prepare'
and similar scripts.  This fails usually in OE because it requires
completely installed 'node_modules'.

Earlier nodejs versions supported an undocumented 'ignore-scripts'
option.  This has been removed in nodejs 16.

We could patch 'package.json' and remove the unwanted scripts.  But
this might complicate local workflows (applying patches) and installed
packages will contain the modified 'package.json'.

Instead of, package it manually by 'tar czf'.  As a sideeffect,
'do_configure' is running much faster now.

Signed-off-by: Enrico Scholz 
---
 meta/classes/npm.bbclass | 35 +--
 1 file changed, 29 insertions(+), 6 deletions(-)

diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index ba50fcac20..91f8f36b0d 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -57,13 +57,36 @@ def npm_global_configs(d):
 configs.append(("cache", d.getVar("NPM_CACHE")))
 return configs
 
+## 'npm pack' runs 'prepare' and 'prepack' scripts. Support for
+## 'ignore-scripts' which prevents this behavior has been removed
+## from nodejs 16.  Use simple 'tar' instead of.
 def npm_pack(env, srcdir, workdir):
-"""Run 'npm pack' on a specified directory"""
-import shlex
-cmd = "npm pack %s" % shlex.quote(srcdir)
-args = [("ignore-scripts", "true")]
-tarball = env.run(cmd, args=args, workdir=workdir).strip("\n")
-return os.path.join(workdir, tarball)
+"""Emulate 'npm pack' on a specified directory"""
+import subprocess
+import os
+import json
+
+src = os.path.join(srcdir, 'package.json')
+with open(src) as f:
+j = json.load(f)
+
+# base does not really matter and is for documentation purposes
+# only.  But the 'version' part must exist because other parts of
+# the bbclass rely on it.
+base = j['name'].split('/')[-1]
+tarball = os.path.join(workdir, "%s-%s.tgz" % (base, j['version']));
+
+# TODO: real 'npm pack' does not include directories while 'tar'
+# does.  But this does not seem to matter...
+subprocess.run(['tar', 'czf', tarball,
+'--exclude', './node-modules',
+'--exclude-vcs',
+'--transform', 's,^\./,package/,',
+'--mtime', '1985-10-26T08:15:00.000Z',
+'.'],
+   check = True, cwd = srcdir)
+
+return tarball
 
 python npm_do_configure() {
 """
-- 
2.36.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165877): 
https://lists.openembedded.org/g/openembedded-core/message/165877
Mute This Topic: https://lists.openembedded.org/mt/91205436/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 6/6] npm: use npm_registry to cache package

[Enrico Scholz via lists.openembedded.org]

With nodejs 16, the simple 'npm cache add' approach does not work
anymore because its fetcher implementation downloads also meta
information from the registry.

We have to generate these information and add them to the cache.
There is no direct support in 'npm' for task so we have to implement
it manually.

This implementation consists of a openembedded python module (in
oe-core) and a nodejs version specific helper (in oe-meta).

Signed-off-by: Enrico Scholz 
---
 meta/classes/npm.bbclass | 15 +--
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 1e41072116..f3c3e2f5b7 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -19,7 +19,7 @@
 
 inherit python3native
 
-DEPENDS:prepend = "nodejs-native "
+DEPENDS:prepend = "nodejs-native nodejs-oe-cache-native "
 RDEPENDS:${PN}:append:class-target = " nodejs"
 
 EXTRA_OENPM = ""
@@ -46,6 +46,7 @@ NPM_ARCH ?= "${@npm_target_arch_map(d.getVar("TARGET_ARCH"))}"
 NPM_PACKAGE = "${WORKDIR}/npm-package"
 NPM_CACHE = "${WORKDIR}/npm-cache"
 NPM_BUILD = "${WORKDIR}/npm-build"
+NPM_REGISTRY = "${WORKDIR}/npm-registry"
 
 def npm_global_configs(d):
 """Get the npm global configuration"""
@@ -111,16 +112,18 @@ python npm_do_configure() {
 from bb.fetch2.npm import npm_unpack
 from bb.fetch2.npmsw import foreach_dependencies
 from bb.progress import OutOfProgressHandler
+from oe.npm_registry import NpmRegistry
 
 bb.utils.remove(d.getVar("NPM_CACHE"), recurse=True)
 bb.utils.remove(d.getVar("NPM_PACKAGE"), recurse=True)
 
 env = NpmEnvironment(d, configs=npm_global_configs(d))
+registry = NpmRegistry(d.getVar('NPM_REGISTRY'), d.getVar('NPM_CACHE'))
 
-def _npm_cache_add(tarball):
-"""Run 'npm cache add' for a specified tarball"""
-cmd = "npm cache add %s" % shlex.quote(tarball)
-env.run(cmd)
+def _npm_cache_add(tarball, pkg):
+"""Add tarball to local registry and register it in the
+   cache"""
+registry.add_pkg(tarball, pkg)
 
 def _npm_integrity(tarball):
 """Return the npm integrity of a specified tarball"""
@@ -184,7 +187,7 @@ python npm_do_configure() {
 # Add the dependency to the npm cache
 destdir = os.path.join(d.getVar("S"), destsuffix)
 (tarball, pkg) = npm_pack(env, destdir, tmpdir)
-_npm_cache_add(tarball)
+_npm_cache_add(tarball, pkg)
 # Add its signature to the cached shrinkwrap
 dep = _npmsw_dependency_dict(cached_shrinkwrap, deptree)
 dep["version"] = pkg['version']
-- 
2.36.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165876): 
https://lists.openembedded.org/g/openembedded-core/message/165876
Mute This Topic: https://lists.openembedded.org/mt/91205435/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 0/6] npm.bbclass: work with nodejs 16

[Enrico Scholz via lists.openembedded.org]

nodejs 16 changed internal caching significantly which breaks the
existing npm.bblcass.

Simulate parts of the npm registry and cache data in the way as
expected.

This patchset requires an additional one in the oe-meta layer.

Enrico Scholz (6):
  npm: replace 'npm pack' call by 'tar czf'
  npm: return content of 'package.json' in 'npm_pack'
  npm: take 'version' directly from 'package.json'
  npm: disable 'audit' + 'fund'
  lib:npm_registry: initial checkin
  npm: use npm_registry to cache package

 meta/classes/npm.bbclass|  65 +-
 meta/lib/oe/npm_registry.py | 169 
 2 files changed, 213 insertions(+), 21 deletions(-)
 create mode 100644 meta/lib/oe/npm_registry.py

-- 
2.36.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165878): 
https://lists.openembedded.org/g/openembedded-core/message/165878
Mute This Topic: https://lists.openembedded.org/mt/91205437/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 5/6] lib:npm_registry: initial checkin

[Enrico Scholz via lists.openembedded.org]

Helper module to:

- generate meta information from package.json content.  This data has
  a format as provided by https://registry.npmjs.org

- put this meta information and the corresponding tarball in the
  nodejs cache.  This uses an external, nodejs version specific helper
  script (oe-npm-cache) shipped in oe-meta

To avoid further nodejs version dependencies, future versions of this
module might omit the caching completely and serve meta information
and tarball by an http server.

Signed-off-by: Enrico Scholz 
---
 meta/lib/oe/npm_registry.py | 169 
 1 file changed, 169 insertions(+)
 create mode 100644 meta/lib/oe/npm_registry.py

diff --git a/meta/lib/oe/npm_registry.py b/meta/lib/oe/npm_registry.py
new file mode 100644
index 00..96c0affb45
--- /dev/null
+++ b/meta/lib/oe/npm_registry.py
@@ -0,0 +1,169 @@
+import bb
+import json
+import subprocess
+
+_ALWAYS_SAFE = frozenset('ABCDEFGHIJKLMNOPQRSTUVWXYZ'
+ 'abcdefghijklmnopqrstuvwxyz'
+ '0123456789'
+ '_.-~')
+
+MISSING_OK = object()
+
+REGISTRY = "https://registry.npmjs.org;
+
+# we can not use urllib.parse here because npm expects lowercase
+# hex-chars but urllib generates uppercase ones
+def uri_quote(s, safe = '/'):
+res = ""
+safe_set = set(safe)
+for c in s:
+if c in _ALWAYS_SAFE or c in safe_set:
+res += c
+else:
+res += '%%%02x' % ord(c)
+return res
+
+class PackageJson:
+def __init__(self, spec):
+self.__spec = spec
+
+@property
+def name(self):
+return self.__spec['name']
+
+@property
+def version(self):
+return self.__spec['version']
+
+@property
+def empty_manifest(self):
+return {
+'name': self.name,
+'description': self.__spec.get('description', ''),
+'versions': {},
+}
+
+def base_filename(self):
+return uri_quote(self.name, safe = '@')
+
+def as_manifest_entry(self, tarball_uri):
+res = {}
+
+## NOTE: 'npm install' requires more than basic meta information;
+## e.g. it takes 'bin' from this manifest entry but not the actual
+## 'package.json'
+for (idx,dflt) in [('name', None),
+   ('description', ""),
+   ('version', None),
+   ('bin', MISSING_OK),
+   ('man', MISSING_OK),
+   ('scripts', MISSING_OK),
+   ('directories', MISSING_OK),
+   ('dependencies', MISSING_OK),
+   ('devDependencies', MISSING_OK),
+   ('optionalDependencies', MISSING_OK),
+   ('license', "unknown")]:
+if idx in self.__spec:
+res[idx] = self.__spec[idx]
+elif dflt == MISSING_OK:
+pass
+elif dflt != None:
+res[idx] = dflt
+else:
+raise Exception("%s-%s: missing key %s" % (self.name,
+   self.version,
+   idx))
+
+res['dist'] = {
+'tarball': tarball_uri,
+}
+
+return res
+
+class ManifestImpl:
+def __init__(self, base_fname, spec):
+self.__base = base_fname
+self.__spec = spec
+
+def load(self):
+try:
+with open(self.filename, "r") as f:
+res = json.load(f)
+except IOError:
+res = self.__spec.empty_manifest
+
+return res
+
+def save(self, meta):
+with open(self.filename, "w") as f:
+json.dump(meta, f, indent = 2)
+
+@property
+def filename(self):
+return self.__base + ".meta"
+
+class Manifest:
+def __init__(self, base_fname, spec):
+self.__base = base_fname
+self.__spec = spec
+self.__lockf = None
+self.__impl = None
+
+def __enter__(self):
+self.__lockf = bb.utils.lockfile(self.__base + ".lock")
+self.__impl  = ManifestImpl(self.__base, self.__spec)
+return self.__impl
+
+def __exit__(self, exc_type, exc_val, exc_tb):
+bb.utils.unlockfile(self.__lockf)
+
+class NpmCache:
+def __init__(self, cache):
+self.__cache = cache
+
+@property
+def path(self):
+return self.__cache
+
+def run(self, type, key, fname):
+subprocess.run(['oe-npm-cache', self.__cache, type, key, fname],
+   check = True)
+
+class NpmRegistry:
+def __init__(self, path, cache):
+self.__path = path
+self.__cache = NpmCache(cache + '/_cacache')
+bb.utils.mkdirhier(self.__path)
+bb.utils.mkdirhier(sel

[OE-core] [PATCH 3/6] npm: take 'version' directly from 'package.json'

[Enrico Scholz via lists.openembedded.org]

We know the content of 'package.json' from earlier patches; there is
no need to parse the tarball name to extract the version.

Signed-off-by: Enrico Scholz 
---
 meta/classes/npm.bbclass | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 014f793450..11c80a738e 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -125,11 +125,6 @@ python npm_do_configure() {
 sha512 = bb.utils.sha512_file(tarball)
 return "sha512-" + base64.b64encode(bytes.fromhex(sha512)).decode()
 
-def _npm_version(tarball):
-"""Return the version of a specified tarball"""
-regex = r"-(\d+\.\d+\.\d+(-.*)?(\+.*)?)\.tgz"
-return re.search(regex, tarball).group(1)
-
 def _npmsw_dependency_dict(orig, deptree):
 """
 Return the sub dictionary in the 'orig' dictionary corresponding to the
@@ -190,7 +185,7 @@ python npm_do_configure() {
 _npm_cache_add(tarball)
 # Add its signature to the cached shrinkwrap
 dep = _npmsw_dependency_dict(cached_shrinkwrap, deptree)
-dep["version"] = _npm_version(tarball)
+dep["version"] = pkg['version']
 dep["integrity"] = _npm_integrity(tarball)
 if params.get("dev", False):
 dep["dev"] = True
-- 
2.36.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165875): 
https://lists.openembedded.org/g/openembedded-core/message/165875
Mute This Topic: https://lists.openembedded.org/mt/91205434/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/6] npm: return content of 'package.json' in 'npm_pack'

[Enrico Scholz via lists.openembedded.org]

We have to read 'package.json' to calculate the name of the tarball.
This content is interesting for later patches.

Signed-off-by: Enrico Scholz 
---
 meta/classes/npm.bbclass | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 91f8f36b0d..014f793450 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -86,7 +86,7 @@ def npm_pack(env, srcdir, workdir):
 '.'],
check = True, cwd = srcdir)
 
-return tarball
+return (tarball, j)
 
 python npm_do_configure() {
 """
@@ -186,7 +186,7 @@ python npm_do_configure() {
 with tempfile.TemporaryDirectory() as tmpdir:
 # Add the dependency to the npm cache
 destdir = os.path.join(d.getVar("S"), destsuffix)
-tarball = npm_pack(env, destdir, tmpdir)
+(tarball, pkg) = npm_pack(env, destdir, tmpdir)
 _npm_cache_add(tarball)
 # Add its signature to the cached shrinkwrap
 dep = _npmsw_dependency_dict(cached_shrinkwrap, deptree)
@@ -207,7 +207,7 @@ python npm_do_configure() {
 
 # Configure the main package
 with tempfile.TemporaryDirectory() as tmpdir:
-tarball = npm_pack(env, d.getVar("S"), tmpdir)
+(tarball, _) = npm_pack(env, d.getVar("S"), tmpdir)
 npm_unpack(tarball, d.getVar("NPM_PACKAGE"), d)
 
 # Configure the cached manifest file and cached shrinkwrap file
@@ -280,7 +280,7 @@ python npm_do_compile() {
 args.append(("build-from-source", "true"))
 
 # Pack and install the main package
-tarball = npm_pack(env, d.getVar("NPM_PACKAGE"), tmpdir)
+(tarball, _) = npm_pack(env, d.getVar("NPM_PACKAGE"), tmpdir)
 cmd = "npm install %s %s" % (shlex.quote(tarball), 
d.getVar("EXTRA_OENPM"))
 env.run(cmd, args=args)
 }
-- 
2.36.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165873): 
https://lists.openembedded.org/g/openembedded-core/message/165873
Mute This Topic: https://lists.openembedded.org/mt/91205432/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 4/6] npm: disable 'audit' + 'fund'

[Enrico Scholz via lists.openembedded.org]

'audit' can cause extra network traffic; 'fund' is not needed.

Signed-off-by: Enrico Scholz 
---
 meta/classes/npm.bbclass | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 11c80a738e..1e41072116 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -53,6 +53,8 @@ def npm_global_configs(d):
 # Ensure no network access is done
 configs.append(("offline", "true"))
 configs.append(("proxy", "http://invalid;))
+configs.append(("funds", False))
+configs.append(("audit", False))
 # Configure the cache directory
 configs.append(("cache", d.getVar("NPM_CACHE")))
 return configs
-- 
2.36.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165872): 
https://lists.openembedded.org/g/openembedded-core/message/165872
Mute This Topic: https://lists.openembedded.org/mt/91205431/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] npm.bblcass dysfunctional in kirkstone

[Enrico Scholz via lists.openembedded.org]

richard.pur...@linuxfoundation.org writes:

>>Is there any solution or other ways to use npm in kirkstone?
>
> We're lacking people working on the npm/node support. If people send
> patches to fix it and add test cases we can look to improve it but this
> kind of thing doesn't happen without developers working on it and we
> simply don't have those at the moment :(.

ok; I am on it and I got it to work.  But I have to cleanup things first
before I can submit the changes.


Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165862): 
https://lists.openembedded.org/g/openembedded-core/message/165862
Mute This Topic: https://lists.openembedded.org/mt/91183010/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] npm.bblcass dysfunctional in kirkstone

[Enrico Scholz via lists.openembedded.org]

Hello,

with nodejs 16, 'npm.bbclass' stops to work and fails (at least) because
of:

1. 'ignore-scripts' has no effect with 'npm pack' anymore.  This command
   will now execute 'prepare' scripts which usually fail.

   Patching out 'scripts/prepare' from package.json should solve it (can
   be done by npm.bbclass).

   This problem happens in do_configure()


2. In do_compile(), 'npm install ' is executed.  This tries to
   contact the registry to fetch the manifest of dependencies.  Because
   offline mode is enforced, it will fail with ENOTCACHED.

   E.g.

   | npm install md5-2.3.0.tgz
   | npm sill fetch manifest charenc@0.0.2
   | npm verb stack Error: request to https://registry.npmjs.org/charenc 
failed: cache mode is 'only-if-cached' but no cached response is available
   | npm ERR! code ENOTCACHED


   This url (https://registry.npmjs.org/charenc) was not added to the
   npm cache.

   Is there any solution or other ways to use npm in kirkstone?


Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165795): 
https://lists.openembedded.org/g/openembedded-core/message/165795
Mute This Topic: https://lists.openembedded.org/mt/91183010/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2] convert-variables: Script for Inclusive Language variable renames

[Enrico Scholz via lists.openembedded.org]

Richard Purdie  writes:

>> > > > This script searches for a list of variable that have been renamed
>> > > > and converts them to their more descriptive names.
>> 
>> s/descriptive/politically correct/
>
> We did try and make some of the names better describe what the variables
> do and make the variables more consistent.
>
> For example, a reference to HASHBASE was changed to BASEHASH which
> better matches every other reference to that thing in the code.

When you really feel that this Yoda style naming must be fixed; then ok.
But for me, the gain would not be large enough to break the api.


> The BB_ENV variables were also traditionally very confusing. Once you
> know, you know but for new users they weren't great.

The new naming is much more confusing.  While this kind of operation
(exclude or include something) was previously named by "whitelist" +
"blacklist" more or less consistently, it is scattered around across
multiple variants now.

Especially for the hash related lists, searching for "whitelist" or
"blacklist" in the sources revealed often the right way how to do
things.


> Also see the discussion about the license variable issues. I'm very
> consciously blocking the "simple" change as I want to see things
> improve.
>
>> Variable names were perfectly readable.  A change should make things
>> better, not worse.
>
> See above, there are at least some changes which do. I'm sad you don't
> agree.

ok; some changes are ok.  E.g.

| "PNBLACKLIST":"SKIP_RECIPE",

because former name is misleading (it is not really a list/set but has
some specially addressing by varflags).


>> Why is the rename done at all?  It makes the product just worse due to a
>> less usable api and because it causes a lot of work for the users of the
>> api (I shudder when I think about updating my CI to work with new and
>> old branches).
>
> For better or worse some of the terminology we have is offensive to
> some people.

Is there any real-world evidence that this is really the case?  Especially
the embedded sector is filled with blacklisted terms (i2c master/slave,
spi mosi/miso) so I do not think that somebody is really offended.

In the opposite, I feel offended when such changes are labeled as
(technical) improvements ("more descriptive names").


> Obviously (and sadly) there will probably always be some element of
> something which someone could be offended by but these issues have
> come under a particular spotlight. People with far more experience of
> this than us have produced guidelines on the key issues,

In my experience, such people have too much spare time and invented a
problem only to come up with a solution later.


> We do have people wanting to try and improve things. Projects do
> need to be open to and able to change. If we don't try and take this
> input and help those people make such changes where appropriate,
> we'll just alienate and frustrate more users even if those users
> were not personally offended by the language, a kind of "rot" can
> set in to our community. I do not want to see that

I see a much higher risk when significant changes are done due to
political reasons rather than technical ones.


> I have advised caution all the way through this. Should the project
> get this wrong, the potential for negative social media feedback
> against the project is huge.

Does such negative social media feedback really exist resp. does it
exist in channels that are relevant to us?

Linux kernel has not "cleaned" its api either and just said that future
development should avoid certain terms. I can not remember any negative
feedback.

Just do the same here: avoid some terms in future development and when
there are really technical reasons to touch existing variables, then
rename them.


> For me personally, there is also a huge risk should I "misstep" in
> handling this. The potential to break things for users is also really
> high, I realise that. I have done a lot of work to try and make sure
> issues are at least clearly reported to users with some idea of how
> to resolve them so the transition is less painful. I do have some
> experience of making changes to the project and am trying my best to
> use that to our benefit.

You are doing a great job, but the project evolved beautiful over the
last >15 years with the old identifier and I can not remember any
complaints about them.



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161950): 
https://lists.openembedded.org/g/openembedded-core/message/161950
Mute This Topic: https://lists.openembedded.org/mt/89199536/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2] convert-variables: Script for Inclusive Language variable renames

[Enrico Scholz via lists.openembedded.org]

Richard Purdie  writes:

>> > This script searches for a list of variable that have been renamed
>> > and converts them to their more descriptive names.

s/descriptive/politically correct/


>> again: most of these renamings make identifiers much less descriptive
>> because they now sound like boolean flags instead of lists
>
> Whilst you could say they sound like booleans, they clearly can't be
> as that wouldn't make sense, e.g. "Ignoring variables in basehash"
> makes no sense.

True; because it does not make sense, these variables should not be
named in this way.


> Secondly, most metadata variables are lists of things.

The renamed ones; yes.  But when I look in bitbake.conf, I can not say
that most variables are lists.


> Adding "LIST" to every variable that is a list of items would just make
> the variable names much longer with little end resulting readability
> gain.

True; explicit type annotations like in the hungarian notation are
nonsense.  But "whitelist" and "blacklist" are words with a meaning;
the "list" is not used as a type annotation there.

Variable names were perfectly readable.  A change should make things
better, not worse.


> Unfortunately there likely isn't a perfect rename and these were the
> best we were able to come up with.

Why is the rename done at all?  It makes the product just worse due to a
less usable api and because it causes a lot of work for the users of the
api (I shudder when I think about updating my CI to work with new and
old branches).



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161941): 
https://lists.openembedded.org/g/openembedded-core/message/161941
Mute This Topic: https://lists.openembedded.org/mt/89199536/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2] convert-variables: Script for Inclusive Language variable renames

[Enrico Scholz via lists.openembedded.org]

"Saul Wold"  writes:

> From: Saul Wold 
>
> This script searches for a list of variable that have been renamed
> and converts them to their more descriptive names.

again: most of these renamings make identifiers much less descriptive
because they now sound like boolean flags instead of lists


> +"BB_SETSCENE_ENFORCE_WHITELIST":"BB_SETSCENE_ENFORCE_IGNORE_TASKS",
> +"BB_HASHBASE_WHITELIST":"BB_BASEHASH_IGNORE_VARS",
> +"BB_HASHTASK_WHITELIST":"BB_TASKHASH_IGNORE_TASKS",
> +"CVE_CHECK_PN_WHITELIST":"CVE_CHECK_SKIP_RECIPE",
> +"CVE_CHECK_WHITELIST":"CVE_CHECK_IGNORE",
> +"MULTI_PROVIDER_WHITELIST":"BB_MULTI_PROVIDER_ALLOWED",
> +"PNBLACKLIST":"SKIP_RECIPE",
> +"SDK_LOCAL_CONF_BLACKLIST":"ESDK_LOCAL_CONF_ALLOW",
> +"SDK_LOCAL_CONF_WHITELIST":"ESDK_LOCAL_CONF_REMOVE",
> +"SDK_INHERIT_BLACKLIST":"ESDK_CLASS_INHERIT_DISABLE",
> +"SSTATE_DUPWHITELIST":"SSTATE_ALLOW_OVERLAP_FILES",
> +"SYSROOT_DIRS_BLACKLIST":"SYSROOT_DIRS_IGNORE",
> +"UNKNOWN_CONFIGURE_WHITELIST":"UNKNOWN_CONFIGURE_OPT_IGNORE",



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161937): 
https://lists.openembedded.org/g/openembedded-core/message/161937
Mute This Topic: https://lists.openembedded.org/mt/89199536/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] runqemu: preload uninative libraries when host gl drivers are in use

[Enrico Scholz via lists.openembedded.org]

"Alexander Kanavin"  writes:

> --- a/meta/classes/qemuboot.bbclass
> +++ b/meta/classes/qemuboot.bbclass
> @@ -109,7 +109,7 @@ def qemuboot_vars(d):
>  build_vars = ['MACHINE', 'TUNE_ARCH', 'DEPLOY_DIR_IMAGE',
>  'KERNEL_IMAGETYPE', 'IMAGE_NAME', 'IMAGE_LINK_NAME',
>  'STAGING_DIR_NATIVE', 'STAGING_BINDIR_NATIVE',
> -'STAGING_DIR_HOST', 'SERIAL_CONSOLES']
> +'STAGING_DIR_HOST', 'SERIAL_CONSOLES', 'UNINATIVE_LOADER']
>  return build_vars + [k for k in d.keys() if k.startswith('QB_')]

Build without uninative fails now with

| File: '.../meta/classes/qemuboot.bbclass', lineno: 141, function: 
do_write_qemuboot_conf
|  0137:else:
|  0138:val = d.getVar(k)
|  0139:# we only want to write out relative paths so that we can 
relocate images
|  0140:# and still run them
|  *** 0141:if val.startswith(topdir):
|  0142:val = os.path.relpath(val, finalpath)
|  0143:cf.set('config_bsp', k, '%s' % val)
|  0144:
|  0145:# QB_DEFAULT_KERNEL's value of KERNEL_IMAGETYPE is the name of 
a symlink
| Exception: AttributeError: 'NoneType' object has no attribute 'startswith'



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161670): 
https://lists.openembedded.org/g/openembedded-core/message/161670
Mute This Topic: https://lists.openembedded.org/mt/88916306/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [oe] [OE-core] Inclusive Language Proposal for YP/OE

[Enrico Scholz via lists.openembedded.org]

Alexander Kanavin  writes:

>> > Would CANCEL be clearer to you than HALT?
>>
>> mmmh for me as a developer (and non-native english speaker), "cancel"
>> means some ordered ending of an operation.
>>
>> But the condition above causes an emergency abort.
>>
>
> Cancel is the same as abort: a request to immediately stop the operation
> (or not even start it) without reaching the originally requested
> outcome.

https://english.stackexchange.com/questions/535153/what-is-the-difference-between-cancel-and-abort

| Cancel implies the action is rescinded before it implements...
|
| Abort is an emergency procedure to interrupt...


> Halt implies the operation is not going to continue

Really?

https://translate.google.com/?sl=en=de=halt=translate says

| a suspension of movement or activity, typically a temporary one.

Examples in

  https://dictionary.cambridge.org/de/worterbuch/englisch/halt

sound like there is a chance to continue after the "halt" (show the
permit, resolve the pay dispute).


When we want to continue this inclusion BS, then "halt" seems to be
offending to some people because it can mean

| ... having a physical disability.



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161381): 
https://lists.openembedded.org/g/openembedded-core/message/161381
Mute This Topic: https://lists.openembedded.org/mt/88906624/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] gcc: enable zstd compression of LTO bytecode

[Enrico Scholz via lists.openembedded.org]

Richard Purdie  writes:

>>   | lto1: internal compiler error: original not compressed with zstd
>>
>>  DEPENDS =+ "mpfr gmp libmpc zlib flex-native"
>> -NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native flex-native"
>> +NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native flex-native 
>> zstd-native"
>>  
>>  LICENSE = "GPL-3.0-with-GCC-exception & GPLv3"
>>  
>
> Should we be passing some flag to gcc explicitly to configure this
> deterministicly?

Both '--with-zstd' and '--without-zstd' make it deterministic (require
zstd.h and abort when non-existing resp. ignore it when existing).

Perhaps '--without-zstd' should be added in the hardknott and honister
branches to prevent pollution by host environment.



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161379): 
https://lists.openembedded.org/g/openembedded-core/message/161379
Mute This Topic: https://lists.openembedded.org/mt/8010/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [oe] [OE-core] Inclusive Language Proposal for YP/OE

[Enrico Scholz via lists.openembedded.org]

Bryan Evenson  writes:

>> >> > For BB_DISKMON_DIRS, the actions "ABORT, STOPTASKS and WARN"
>> >> > would become "HALT, NO_NEW_TASKS and "WARN".
>> >>
>> >> I am not an native english speaker, but for "HALT" I will have to
>> >> think twice whether it will pause the operation or abort it.  I would
>> >> stay at "ABORT" because it makes much more clear what happens.
>
> Would CANCEL be clearer to you than HALT?

mmmh for me as a developer (and non-native english speaker), "cancel"
means some ordered ending of an operation.

But the condition above causes an emergency abort.

I do not know how this can be described without using "abort" or some
extensively long terms.


>> >> > BB_HASHCONFIG_WHITELIST -> BB_HASHCONFIG_IGNORE_VARS
>> >> > BB_SETSCENE_ENFORCE_WHITELIST -> BB_SETSCENE_ENFORCE_IGNORE_TASKS
>> >> > BB_HASHBASE_WHITELIST -> BB_BASEHASH_IGNORE_VARS
>> >> > MULTI_PROVIDER_WHITELIST -> BB_MULTI_PROVIDER_ALLOWED
>> >>
>> >> The new variable names sound like boolean flags, not like lists.
>
> Would BB_HASHCONFIG_IGNORELIST or BB_HASHCONFIG_ALLOWEDLIST make more
> sense to you?

yes; it is much better.  But should it be an "IGNORELIST" or an
"IGNORE*D*LIST"?



Enrico
[who is irritated how people and especially developer can waste their
(and other developers) time in trying to change something which was
completely fine before]

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161377): 
https://lists.openembedded.org/g/openembedded-core/message/161377
Mute This Topic: https://lists.openembedded.org/mt/88906624/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [oe] [OE-core] Inclusive Language Proposal for YP/OE

[Enrico Scholz via lists.openembedded.org]

Alexander Kanavin  writes:

>> > For BB_DISKMON_DIRS, the actions "ABORT, STOPTASKS and WARN" would
>> > become "HALT, NO_NEW_TASKS and "WARN".
>>
>> I am not an native english speaker, but for "HALT" I will have to
>> think twice whether it will pause the operation or abort it.  I would
>> stay at "ABORT" because it makes much more clear what happens.
>
> I'm not taking a stand here, but just providing the context for where
> the push for avoidance of 'abort' comes from:
> https://inclusivenaming.org/word-lists/tier-1/
>
> Keep in mind that for non-native english speakers none of these words
> are emotionally charged; they're just terms.

Yes; these are terms.  But it should be possible to understand the
meaning of these terms without using a special "inclusivenaming"
dictionary.  Else, we could just use "A", "B" and "C" for the actions
above and explain these "terms" in some documentation later.


>> > BB_HASHCONFIG_WHITELIST -> BB_HASHCONFIG_IGNORE_VARS
>> > BB_SETSCENE_ENFORCE_WHITELIST -> BB_SETSCENE_ENFORCE_IGNORE_TASKS
>> > BB_HASHBASE_WHITELIST -> BB_BASEHASH_IGNORE_VARS
>> > MULTI_PROVIDER_WHITELIST -> BB_MULTI_PROVIDER_ALLOWED
>>
>> The new variable names sound like boolean flags, not like lists.
>
> I'd say the context should make it clear that they are lists

It is bad when a context is required to understand the meaning of a
variable.

And where do you see a context when local.conf contains

| BB_HASHCONFIG_IGNORE_VARS = "true"



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161365): 
https://lists.openembedded.org/g/openembedded-core/message/161365
Mute This Topic: https://lists.openembedded.org/mt/88906624/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] Inclusive Language Proposal for YP/OE

[Enrico Scholz via lists.openembedded.org]

"Jon Mason"  writes:

> For BB_DISKMON_DIRS, the actions "ABORT, STOPTASKS and WARN" would
> become "HALT, NO_NEW_TASKS and "WARN".

I am not an native english speaker, but for "HALT" I will have to think
twice whether it will pause the operation or abort it.  I would stay at
"ABORT" because it makes much more clear what happens.


> BB_HASHCONFIG_WHITELIST -> BB_HASHCONFIG_IGNORE_VARS
> BB_SETSCENE_ENFORCE_WHITELIST -> BB_SETSCENE_ENFORCE_IGNORE_TASKS
> BB_HASHBASE_WHITELIST -> BB_BASEHASH_IGNORE_VARS
> MULTI_PROVIDER_WHITELIST -> BB_MULTI_PROVIDER_ALLOWED

The new variable names sound like boolean flags, not like lists.


> SSTATE_DUPWHITELIST -> SSTATE_ALLOW_OVERLAP_FILES
> CVE_CHECK_PN_WHITELIST -> CVE_CHECK_SKIPRECIPE
> CVE_CHECK_WHITELIST -> CVE_CHECK_IGNORECVE
> SYSROOT_DIRS_BLACKLIST -> SYSROOT_DIRS_IGNORE
> LICENSE_FLAGS_WHITELIST -> LICENSE_FLAGS_ACCEPTED
> UNKNOWN_CONFIGURE_WHITELIST -> UNKNOWN_CONFIGURE_OPT_IGNORE
> SDK_LOCAL_CONF_BLACKLIST -> ESDK_LOCALCONF_REMOVE
> SDK_LOCAL_CONF_WHITELIST -> ESDK_LOCALCONF_ALLOW
> SDK_INHERIT_BLACKLIST -> ESDK_CLASS_INHERIT_DISABLE

ditto; sounds like flags but not like lists


Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161344): 
https://lists.openembedded.org/g/openembedded-core/message/161344
Mute This Topic: https://lists.openembedded.org/mt/88650128/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] gcc: enable zstd compression of LTO bytecode

[Enrico Scholz via lists.openembedded.org]

Khem Raj  writes:

>> gcc-10+ supports zstd compression of LTO bytecode.  Install the
>> corresponding package to enable this feature in a deterministic way.
>>
>> NOTE: previously built LTO object files (without this compression)
>> must be regenerated; gcc will fail else with
>>
>>   | lto1: internal compiler error: original not compressed with zstd
>>
>> in this case.
>>
>> NOTE: it seems to be possible that zstd support is enabled non
>> deterministically (perhaps by host system pollution).
>>
>> I had the problem that the SDK gcc was built with zstd while the
>> cross gcc was built without it.  Libraries (built by cross gcc and
>> uncompressed hence) could not be used with the SDK gcc.
>
> this means regenerating entire shared state isnt it ?

yes (at least when lto.inc is used (which is *not* by default)).  When a
bad sstate really matters for 'master', perhaps there can be wait for a
change in glibc or so before applying this patch.

Or do you think that adding a DISTRO_FEATURE like 'lto-zstd' would make
sense?  When this flag is missing, '--without-zstd' must be added to
EXTRA_OECONF.  Also some early [vardeps] must be set (where?) to avoid
the sstate problem.

Globally enabling 'zstd' appears easier because drawbacks (costs for
compression + decompression) should be negligible for zstd.



Enrico

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161320): 
https://lists.openembedded.org/g/openembedded-core/message/161320
Mute This Topic: https://lists.openembedded.org/mt/8010/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] gcc: enable zstd compression of LTO bytecode

[Enrico Scholz via lists.openembedded.org]

gcc-10+ supports zstd compression of LTO bytecode.  Install the
corresponding package to enable this feature in a deterministic way.

NOTE: previously built LTO object files (without this compression)
must be regenerated; gcc will fail else with

  | lto1: internal compiler error: original not compressed with zstd

in this case.

NOTE: it seems to be possible that zstd support is enabled non
deterministically (perhaps by host system pollution).

I had the problem that the SDK gcc was built with zstd while the
cross gcc was built without it.  Libraries (built by cross gcc and
uncompressed hence) could not be used with the SDK gcc.

Signed-off-by: Enrico Scholz 
---
 meta/recipes-devtools/gcc/gcc-11.2.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/gcc/gcc-11.2.inc 
b/meta/recipes-devtools/gcc/gcc-11.2.inc
index f07a5be507b3..62db035760aa 100644
--- a/meta/recipes-devtools/gcc/gcc-11.2.inc
+++ b/meta/recipes-devtools/gcc/gcc-11.2.inc
@@ -11,7 +11,7 @@ BINV = "11.2.0"
 FILESEXTRAPATHS =. "${FILE_DIRNAME}/gcc:${FILE_DIRNAME}/gcc/backport:"
 
 DEPENDS =+ "mpfr gmp libmpc zlib flex-native"
-NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native flex-native"
+NATIVEDEPS = "mpfr-native gmp-native libmpc-native zlib-native flex-native 
zstd-native"
 
 LICENSE = "GPL-3.0-with-GCC-exception & GPLv3"
 
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161292): 
https://lists.openembedded.org/g/openembedded-core/message/161292
Mute This Topic: https://lists.openembedded.org/mt/8010/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] $PWD in terminal.bbclass

[Enrico Scholz via Openembedded-core]

Hi,

terminal.bbclass exports the original environment before running the
terminal command.  This environment includes the original $PWD too so
that the generated script (run.do_terminal) looks like

| #!/usr/bin/env /bin/bash
| ...
| export PWD="/srv/projects/.../build"   << the directory with conf/local.conf 
e.g.
| ...
| do_terminal() {
| exec /bin/bash -c ...
| }


Assigning PWD works like 'cd' and changes the actual path so that
do_terminal() will be executed from the directory where bitbake has
been started from.  But usually, the [dirs] directory (e.g. '${B}') is
expected.

Is this really intented?  Here, this breaks kernel's '-c menuconfig' (at
least).

A trivial fix would be something like

|envdata.delVar("PS1")
|+   envdata.delVar("PWD")

in terminal.bbclass.  But is there any reason to have PWD in ${BB_ORIGENV}
at all?  imo, there is no reason to keep this variable and it should be
removed in bitbake already.


Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] systemd: change default locale from C.UTF-8 to C

[Enrico Scholz via Openembedded-core]

Richard Purdie

writes:

>> current default locale is set to C.UTF-8, but glibc not support
>> locale C.UTF-8. so set to the default locale C.
>
> Doesn't your second patch mean we don't need this one?

C.UTF-8 instead of plain C has runtime costs because additional files
need to be read by services.

imo, plain C should be a PACKAGE_CONFIG option which is enabled by
default.


Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] dhcp: fix isc_boolean_t patch

[Enrico Scholz via Openembedded-core]

There is a copy & paste error in 5775e9ef2fce1d0b9860602f1065bb1e09bc1b61
which breaks dhcpd.

Cc: Armin Kuster 
Signed-off-by: Enrico Scholz 
---
 ...1-dhcpd-fix-Replace-custom-isc_boolean_t-with-C-standa.patch | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/meta/recipes-connectivity/dhcp/dhcp/0001-dhcpd-fix-Replace-custom-isc_boolean_t-with-C-standa.patch
 
b/meta/recipes-connectivity/dhcp/dhcp/0001-dhcpd-fix-Replace-custom-isc_boolean_t-with-C-standa.patch
index d2e57714cd..fa3d9c4e1f 100644
--- 
a/meta/recipes-connectivity/dhcp/dhcp/0001-dhcpd-fix-Replace-custom-isc_boolean_t-with-C-standa.patch
+++ 
b/meta/recipes-connectivity/dhcp/dhcp/0001-dhcpd-fix-Replace-custom-isc_boolean_t-with-C-standa.patch
@@ -1816,7 +1816,7 @@ Index: dhcp-4.4.1/common/options.c
  {
if (universe->save_func)
 -  (*universe->save_func)(universe, options, oc, ISC_FALSE);
-+  (*universe->save_func)(universe, options, oc, true);
++  (*universe->save_func)(universe, options, oc, false);
else
log_error("can't store options in %s space.", universe->name);
  }
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] utils.py: added sh_quote() function

[Enrico Scholz via Openembedded-core]

This function is a wrapper around "shlex.quote()" and can be used in
"${@...}" context where shlex (or pipes, which provides similar
functionality) is unavailable.

Signed-off-by: Enrico Scholz 
---
 meta/lib/oe/utils.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index 3a496090f3..cedd053d36 100644
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -490,3 +490,6 @@ class ImageQAFailed(bb.build.FuncFailed):
 
 return msg
 
+def sh_quote(string):
+import shlex
+return shlex.quote(string)
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] pseudo: build with recent attr

[Enrico Scholz via Openembedded-core]

"Burton, Ross"  writes:

> | In file included from port_wrappers.c:8:0,
> |  from pseudo_wrappers.c:307:
> | ports/linux/xattr/pseudo_wrappers.c: In function ‘shared_getxattr’:
> | ports/linux/xattr/pseudo_wrappers.c:134:11: error: ‘ENOATTR’ undeclared 
> (first use in this function)
> |errno = ENOATTR;
>
> Breaks when building with old xattr.

Yes; defining ENOATTR in  was a mistake, violated POSIX
and causes such problems now...  I am tending to rewrite the patch with
something like

-#include 
+#include 
+#include 
+#ifndef ENOATTR
+# define ENOATTR ENODATA
+#endif

and I am disussing this with Seebs atm...


Enrico
-- 
SIGMA Chemnitz GmbH   Registergericht:   Amtsgericht Chemnitz HRB 1750
Am Erlenwald 13   Geschaeftsfuehrer: Grit Freitag, Frank Pyritz
09128 Chemnitz
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] attr: updated acl + attr

[Enrico Scholz via Openembedded-core]

"Burton, Ross"  writes:

>> Recent acl and attr package versions changed their buildsystem to
>> standard automake so that previous buildsystem related patches are not
>> needed anymore and recipes become very simple.
>
> pseudo-native doesn't build anymore:

strange... Requires an explicit '-c cleansstate pseudo-native' to show
the error but it built fine with a removed TMPDIR


> | In file included from pseudo_ports.h:15:0,
> |  from pseudo.h:171,
> |  from pseudo.c:37:
> | ports/linux/xattr/portdefs.h:1:24: fatal error: attr/xattr.h: No such file 
> or directory
> |  #include 

I thnk, fixing it in pseudo is the correct way because 
has been removed by intention by upstream.  Most software seems to use
this "new" header already, but there can be a problem with ENOATTR which
requires either an #include of , a manual #define or
replacement by ENODATA.


Alternatively, we can ship -- like Fedora -- a compatibility wrapper
like

-- 
/* legacy wrapper; see
 * 
http://git.savannah.nongnu.org/cgit/attr.git/commit/include?id=7921157890d07858d092f4003ca4c6bae9fd2c38
 */

#warning Using legacy ; please update program to include 


#include 
#include "attributes.h"
-- 



Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] pseudo: build with recent attr

[Enrico Scholz via Openembedded-core]

Recent attr package removed  which is shipped by libc
for a very long time already.

Signed-off-by: Enrico Scholz 
---
 ...001-use-sys-xattr.h-not-attr-xattr.h.patch | 54 +++
 meta/recipes-devtools/pseudo/pseudo_git.bb|  1 +
 2 files changed, 55 insertions(+)
 create mode 100644 
meta/recipes-devtools/pseudo/files/0001-use-sys-xattr.h-not-attr-xattr.h.patch

diff --git 
a/meta/recipes-devtools/pseudo/files/0001-use-sys-xattr.h-not-attr-xattr.h.patch
 
b/meta/recipes-devtools/pseudo/files/0001-use-sys-xattr.h-not-attr-xattr.h.patch
new file mode 100644
index 00..5eddea5ba1
--- /dev/null
+++ 
b/meta/recipes-devtools/pseudo/files/0001-use-sys-xattr.h-not-attr-xattr.h.patch
@@ -0,0 +1,54 @@
+From 7132668ce2409e0919a79d6f236827665f068427 Mon Sep 17 00:00:00 2001
+From: Enrico Scholz 
+Date: Fri, 14 Dec 2018 18:41:04 +0100
+Subject: [PATCH, pseudo] use , not 
+
+ has been removed from recent attr package:
+
+   The xattr syscalls are provided by glibc since ages, so there is
+   no need to use the indirect system call "syscall" anymore. This
+   removes the need for the  header; use 
+   instead.
+
+   
http://git.savannah.nongnu.org/cgit/attr.git/commit/include?id=7921157890d07858d092f4003ca4c6bae9fd2c38
+
+For Linux, we have to include  to define ENODATA.
+
+Signed-off-by: Enrico Scholz 
+---
+Upstream-Status: pending
+
+ ports/linux/subports | 4 ++--
+ ports/linux/xattr/portdefs.h | 3 ++-
+ 2 files changed, 4 insertions(+), 3 deletions(-)
+
+diff --git a/ports/linux/subports b/ports/linux/subports
+index a29044a..44e0e9e 100755
+--- a/ports/linux/subports
 b/ports/linux/subports
+@@ -29,11 +29,11 @@ fi
+ if$port_xattr; then
+   cat > dummy.c <
+-#include 
++#include 
+ int i;
+ EOF
+   if ! ${CC} -c -o dummy.o dummy.c >/dev/null 2>&1; then
+-  echo >&2 "Warning: Can't compile trivial program using 
".
++  echo >&2 "Warning: Can't compile trivial program using 
".
+   echo >&2 " xattr support will require that header."
+   fi
+   echo "linux/xattr"
+diff --git a/ports/linux/xattr/portdefs.h b/ports/linux/xattr/portdefs.h
+index 367ca60..bb577f1 100644
+--- a/ports/linux/xattr/portdefs.h
 b/ports/linux/xattr/portdefs.h
+@@ -1,2 +1,3 @@
+-#include 
++#include 
++#include 
+ #include 
+-- 
+2.19.2
+
diff --git a/meta/recipes-devtools/pseudo/pseudo_git.bb 
b/meta/recipes-devtools/pseudo/pseudo_git.bb
index 79e0b6bd21..a0147d2d19 100644
--- a/meta/recipes-devtools/pseudo/pseudo_git.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_git.bb
@@ -6,6 +6,7 @@ SRC_URI = "git://git.yoctoproject.org/pseudo \
file://fallback-group \
file://moreretries.patch \
file://toomanyfiles.patch \
+   file://0001-use-sys-xattr.h-not-attr-xattr.h.patch \
"
 
 SRCREV = "6294b344e5140f5467e6860f45a174440015304e"
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] attr: updated acl + attr

[Enrico Scholz via Openembedded-core]

Recent acl and attr package versions changed their buildsystem to
standard automake so that previous buildsystem related patches are not
needed anymore and recipes become very simple.

Other, functionality related patches are integrated in upstream and
were removed too.

Because of the buildsystem changes, the 'ptest' parts of the recipes do
not work anymore and would not a complete rewrite.  For now, 'ptest'
has been removed.

Signed-off-by: Enrico Scholz 
---
 ...file-libacl-should-depend-on-include.patch |  51 --
 ...-order-of-expected-output-of-getfacl.patch | 198 --
 meta/recipes-support/attr/acl/configure.ac|  49 --
 meta/recipes-support/attr/acl/run-ptest   |  64 --
 .../acl/test-fix-directory-permissions.patch  |  37 -
 .../test-fix-insufficient-quoting-of.patch| 100 ---
 ...-SELinux-machines-for-root-testcases.patch |  84 ---
 meta/recipes-support/attr/acl_2.2.52.bb   |  52 --
 meta/recipes-support/attr/acl_2.2.53.bb   |  23 +
 meta/recipes-support/attr/attr.inc|  46 --
 .../0001-Use-stdint-types-consistently.patch  |  69 --
 .../attr/attr/attr-Missing-configure.ac.patch |  63 --
 .../attr/attr/dont-use-decl-macros.patch  |  56 --
 meta/recipes-support/attr/attr_2.4.47.bb  |  15 -
 meta/recipes-support/attr/attr_2.4.48.bb  |  27 +
 meta/recipes-support/attr/ea-acl.inc  |  52 --
 ...option-to-enable-disable-static-libr.patch |  70 --
 ...e-attr.5-man-page-moved-to-man-pages.patch | 240 ---
 .../Remove-the-section-2-man-pages.patch  | 666 --
 .../attr/files/relative-libdir.patch  |  25 -
 meta/recipes-support/attr/files/run-ptest |   5 -
 21 files changed, 50 insertions(+), 1942 deletions(-)
 delete mode 100644 
meta/recipes-support/attr/acl/Makefile-libacl-should-depend-on-include.patch
 delete mode 100644 
meta/recipes-support/attr/acl/acl-fix-the-order-of-expected-output-of-getfacl.patch
 delete mode 100644 meta/recipes-support/attr/acl/configure.ac
 delete mode 100644 meta/recipes-support/attr/acl/run-ptest
 delete mode 100644 
meta/recipes-support/attr/acl/test-fix-directory-permissions.patch
 delete mode 100644 
meta/recipes-support/attr/acl/test-fix-insufficient-quoting-of.patch
 delete mode 100644 
meta/recipes-support/attr/acl/test-fixups-on-SELinux-machines-for-root-testcases.patch
 delete mode 100644 meta/recipes-support/attr/acl_2.2.52.bb
 create mode 100644 meta/recipes-support/attr/acl_2.2.53.bb
 delete mode 100644 meta/recipes-support/attr/attr.inc
 delete mode 100644 
meta/recipes-support/attr/attr/0001-Use-stdint-types-consistently.patch
 delete mode 100644 
meta/recipes-support/attr/attr/attr-Missing-configure.ac.patch
 delete mode 100644 meta/recipes-support/attr/attr/dont-use-decl-macros.patch
 delete mode 100644 meta/recipes-support/attr/attr_2.4.47.bb
 create mode 100644 meta/recipes-support/attr/attr_2.4.48.bb
 delete mode 100644 meta/recipes-support/attr/ea-acl.inc
 delete mode 100644 
meta/recipes-support/attr/files/0001-Added-configure-option-to-enable-disable-static-libr.patch
 delete mode 100644 
meta/recipes-support/attr/files/Remove-the-attr.5-man-page-moved-to-man-pages.patch
 delete mode 100644 
meta/recipes-support/attr/files/Remove-the-section-2-man-pages.patch
 delete mode 100644 meta/recipes-support/attr/files/relative-libdir.patch
 delete mode 100644 meta/recipes-support/attr/files/run-ptest

diff --git 
a/meta/recipes-support/attr/acl/Makefile-libacl-should-depend-on-include.patch 
b/meta/recipes-support/attr/acl/Makefile-libacl-should-depend-on-include.patch
deleted file mode 100644
index 4c7cba3b88..00
--- 
a/meta/recipes-support/attr/acl/Makefile-libacl-should-depend-on-include.patch
+++ /dev/null
@@ -1,51 +0,0 @@
-From d82457ce5ca7455e336da5e244d95f90e52aaef8 Mon Sep 17 00:00:00 2001
-From: Robert Yang 
-Date: Tue, 18 Apr 2017 01:17:26 -0700
-Subject: [PATCH] Makefile: libacl should depend on include
-
-Fixed race issue:
- In file included from acl_copy_entry.c:22:0:
- libacl.h:19:21: fatal error: sys/acl.h: No such file or directory
-  #include 
-
-[snip]
-
- compilation terminated.
- acl_get_file.c:27:24: fatal error: acl/libacl.h: No such file or directory
-  #include 
- ^
-
-The acl.h is in "include" directory, and include/Makefile creates
-symlink "sys" and "acl" poinst to current dirctory:
-$ ls include/ -l
-acl -> .
-sys -> .
-
-So if "libacl" target runs before "include", the error would happen
-since no "acl" or "sys" directory.
-
-Let libacl depend on include can fix the problem.
-
-Upstream-Status: Pending
-
-Signed-off-by: Robert Yang 

- Makefile | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/Makefile b/Makefile
-index dce32d3..8a79379 100644
 a/Makefile
-+++ b/Makefile
-@@ -48,7 +48,7 @@ else
- endif
- 
- # tool/lib dependencies
--libacl: libmisc
-+libacl: include libmisc
- getfacl setfacl chacl: libacl
- 
- ife

Re: [OE-core] Submission of 'emit-buildvars' class?

[Enrico Scholz via Openembedded-core]

Alexander Kanavin  writes:

>> I do not know, how this can be done with plain OE.  As said, SDKs are too
>> heavy weighted, 'devshell' too slow and can not be scripted, 'externalsrc'
>> is broken, slow and can not be scripted.
>
> For better or worse, Yocto Project has standardized on devtool (not
> devshell!)

'devtool' is for writing recipes, not for writing software.


> and externalsrc for these kind of local development workflows. You are
> very welcome to improve and add to them, so they are not broken

I tried 'externalsrc' after the per-package sysroot changes but just run
into bugs (class tried to write into read-only mounted source dir, every
2nd 'do_fetch' failed because sysroot was already populated).


Doing a 'make -f ../Makefile.common install' which writes data directly
into the NFS root is much faster than doing all the steps of the dev*
tools.


> or slow,

They *are* slow.

| $ cd workspace/kernel
| 
| $ git commit -m 'empty' --allow-empty
| 
| $ time bitbake linux-mainline
| real4m7.417s
| user0m0.763s
| sys 0m0.110s

vs.

| time make -f ../Makefile.kernel  -j12
| real0m13.204s
| user0m54.888s
| sys 0m17.970s



Even a plain 'bitbake'  is takes 0.5 sec:

| $ time bitbake
| Nothing to do.  Use 'bitbake world' to build everything, or run 'bitbake 
--help' for usage information.
| 
| real0m0.588s
| user0m0.255s
| sys 0m0.023s

vs.

| $ time make
| real0m0.007s
| user0m0.002s
| sys 0m0.005s



And some things are impossible because they are not supported by the
design.  E.g. how would would you implement the flymake 'check-syntax'
target[1]

| check-syntax:
| gcc -o /dev/null -S ${CHK_SOURCES} || true

with devtool/devshell/externalsrc?


Deciding which task to execute (compile, do_compile_kernelmodules) and
installation into NFS root is not supported by dev* either.



Enrico


Footnotes: 
[1]  
https://www.gnu.org/software/emacs/manual/html_mono/flymake.html#Example_002d_002d_002dConfiguring-a-tool-called-via-make

-- 
SIGMA Chemnitz GmbH   Registergericht:   Amtsgericht Chemnitz HRB 1750
Am Erlenwald 13   Geschaeftsfuehrer: Grit Freitag, Frank Pyritz
09128 Chemnitz
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] Submission of 'emit-buildvars' class?

[Enrico Scholz via Openembedded-core]

Alexander Kanavin  writes:

>> we are using for some time an 'emit-buildvars' class which exports
>> bitbake variables into makefile fragments.  This class is used for for
>> developing of kernel, u-boot and other low level software and for
>> generic packaging.
>
> I don't quite understand how this is useful? Can you provide specific
> usage scenarios please?

I ma doing perhaps 70% of my development for kernel, u-boot and barebox.
Building them with the actual toolchain requires only a

| make -f ../Makefile.kernel tftp

Result is available after a few seconds on the TFTP server; when using a
specialized variant ('kbuild-file=...'), even faster so that it can be
used in Emacs flymake mode.


Ditto for other programs (testsuites or so); you write your recipe with
all the 'DEPENDS', execute the 'emit_buildvars' task and build your
application with

| make -f ../Makefile.common R=my-app

When you see that something is missing in DEPENDS, add it, reexecute
'emit_buildvars' and it will be in the per-package sysroot.  Excluding
the build time of the dependency, this won't take more than 30 seconds.


Or, it is trivial to run unfsd in the enviornment of the image recipe
and to build kernel modules with

| make -f ../Makefile.kernel modules modules-install

These modules will be installed there and can be used immediately on the
target system:

| # rmmod foo-module
| # modprobe foo-module


I do not know, how this can be done with plain OE.  As said, SDKs are too
heavy weighted, 'devshell' too slow and can not be scripted, 'externalsrc'
is broken, slow and can not be scripted.


Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] Submission of 'emit-buildvars' class?

[Enrico Scholz via Openembedded-core]

Hello,

we are using for some time an 'emit-buildvars' class which exports
bitbake variables into makefile fragments.  This class is used for for
developing of kernel, u-boot and other low level software and for
generic packaging.

The class supports exporting images over userspace NFS and is available
at

  
https://github.com/sigma-embedded/meta-de.sigma-chemnitz/blob/sumo/classes/elito-emit-buildvars.bbclass
  
https://github.com/sigma-embedded/meta-de.sigma-chemnitz/tree/sumo/scripts/buildvars
  
https://github.com/sigma-embedded/meta-de.sigma-chemnitz/blob/sumo/classes/elito-nfs-export.bbclass

It is much faster than 'devshell' and can be used in a scripted
environment.  A complete SDK is far too heavy weight for our tasks
and not flexible enough.

Is there common interest in such a functionality? If so, I would split
it out from our layer and submit it for core.



A more detailed description:

The generated makefile fragment contains information about the per-package
sysroot and can be used to set e.g. $PATH.  The system requires a small
Makefile to instantiate these variables; e.g. for common cases something
like

  workspace/Makefile.common ---
|  # --*- makefile -*--
| WORKSPACE_DIR := $(abspath $(dir $(lastword $(MAKEFILE_LIST
| DEPLOY_DIR ?= ${WORKSPACE_DIR}/../build/tmp/deploy
| MACHINE ?= imx8m-var-dart
| 
| _BUILDVAR_STYLE ?= _generic_
| include ${DEPLOY_DIR}/buildvars/${MACHINE}/$R.mk
| 
| %:
|   ${MAKE} $@

is enough.  For kernel development, the makefile contains some more
information about the image which is going to be exported over NFS.

  workspace/Makefile.kernel ---
| # --*- makefile -*--
| WORKSPACE_DIR := $(abspath $(dir $(lastword $(MAKEFILE_LIST
| DEPLOY_DIR ?= ${WORKSPACE_DIR}/../build/tmp/deploy
| MACHINE ?= imx8m-var-dart
| 
| _BUILDVAR_STYLE := kernel
| IMAGE_RECIPE = camtest-image
| 
| HAVE_NFSROOT=1
| include ${WORKSPACE_DIR}/../sources/de.sigma-chemnitz.core/mk/nfs-opt.mk
| include ${DEPLOY_DIR}/buildvars/${MACHINE}/linux-mainline.mk


Now, generate the '.mk' fragments either manually:

| $ bitbake busybox -c emit_buildvars

or enable it within the recipe (recommended when working on it)

| BUILDVARS_EMIT = "true"

and build the package.


When you try to fix something in 'busybox', extract the sources
somewhere (e.g. in workspace/) and call

| make -f ../Makefile.common R=busybox all

to build it.


Else, see the class for more details.



Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] pseudo: host user contamination

[Enrico Scholz]

Seebs  writes:

> There are weird calling conventions out there. For instance, "pass
> floating point values in registers, but integers on stack", or "pass
> first N arguments in registers", and so on. I don't know if any of
> them are active in stuff Linux supports, but I'm aware that this is an
> area where you can get really strange behaviors.

__builtin_apply() should deal with it.  If you are really paranoid, assume
a huge stack size (e.g. 1024).  But accordingly syscall(2) man-page, there
are to be expected not more than 7 arguments for syscalls.  So, define the
size for known architectures and a fallback of '7 * sizeof(uintmax_t)' or
so.


Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] pseudo: host user contamination

[Enrico Scholz]

Seebs <se...@seebs.net> writes:

> On Tue, 27 Mar 2018 21:20:24 +0200
> Enrico Scholz <enrico.sch...@sigma-chemnitz.de> wrote:
>
>> SYS_readahead is one of a few syscalls which pass 64 bit arguments on
>> 32 bit architectures.  Without the manual splitting, the ABI will
>> cause the compiler to insert a dummy argument so that registers are
>> aligned for 64 bit values.
>
> I'm now even more confused. This sounds like the compiler *would*
> insert the argument without being told to, because the ABI "will cause"
> that, in which case the manual splitting wouldn't be necessary?

For example, by ARM EABI function arguments are transmitted in r0, r1,
r2, r3.  But 64 bit values must be aligned to even registers.

So, assuming code like

| void foo(int a, unsigned long long b)
| {
| }
| 
| void bar(void)
| {
|   foo(1, 2)
| }

The compiler generates

mov r2, #2
mov r3, #0
mov r0, #1
bl  foo

e.g. it skips 'r1'.


When you use the variadic syscall(2) function, you pass an extra argument
at front (the syscall number)

  syscall(__NR_readahead, fd, offset_64bit, count);

  --> when doing the 'svc', fd goes into 'r0' and offset_64bit
  into 'r1' + 'r2'

In-kernel function does not have have the syscall-number and is

  sys_readahead(int fd, loff_t, count)

  --> 'fd' is expected in 'r0', offset in 'r2' + 'r3' due to the ABI


Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] pseudo: host user contamination

[Enrico Scholz]

Seebs  writes:

>> There are no 64 bit arguments (on 32 bit platforms) which might
>> require a special treatment as described in [1]
>> "Architecture-specific requirements".
>
> Okay, ignore the pointer case, and pretend it's the 64-bit value case,
> since we have specific-ish documentation for that.
>
> Look at the example for SYS_readahead, stating that the caller must
> pass an extra value.

SYS_readahead is one of a few syscalls which pass 64 bit arguments on 32
bit architectures.  Without the manual splitting, the ABI will cause the
compiler to insert a dummy argument so that registers are aligned for 64
bit values.

The caller of syscall(2) has to split such arguments. When the wrapper
does not handle the syscall itself, it just need to pass the arguments
as-is.


> At that point, if you have a series of va_arg calls corresponding to
> the values that would have been arguments had they not passed the extra
> value, I don't think you get the expected arguments. So far as I can
> tell, if the caller actually wrote
>   varargsfunc(SYS_readahead, 0, uint64_t_value, ...)
> and the function did
>   va_arg(ap, uint64_t);

That's true and must be checked when writing a wrapper for a syscall
which takes a 64 bit argument on a 32 bit architecture.

But for renameat2() it does not matter; every of its arguments fits into
a single register.



Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] pseudo: host user contamination

[Enrico Scholz]

Seebs <seebs-59mtl4g6zbfeowh0uzb...@public.gmane.org> writes:

> On Tue, 27 Mar 2018 16:42:03 +0200
> Enrico Scholz 
> <enrico.scholz-wttK6gPy29v+Hn7q9Vec/7nah6klm...@public.gmane.org> wrote:
>
>> will work to wrap syscall(2).  Params for _renameat2_syscall() can be
>> extracted by va_args.
>
> Does anyone have access to an actual 64-bit EABI ARM system to verify
> the argument passing for renameat2 there?

Does this really matter here?  Because the caller has to set them
accordingly the ABI, you can extract the arguments by

int olddirfd= va_arg(ap, int);
char const *oldpath = va_arg(ap, char consr *);
int newdirfd= va_arg(ap, int);
char const *newpath = va_arg(ap, char consr *);
unsigned int flags  = va_arg(ap, unsigned int);

There are no 64 bit arguments (on 32 bit platforms) which might require a
special treatment as described in [1] "Architecture-specific requirements".



Enrico

Footnotes: 
[1]  http://man7.org/linux/man-pages/man2/syscall.2.html
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] pseudo: host user contamination

[Enrico Scholz]

Seebs  writes:

>> >> Since the man page gave the ia64 example, I went and checked, and
>> >> it is indeed the case that calls other than syscall(2) will
>> >> clobber r10 after system calls,
>> 
>> I think you are misinterpreting the man-page.  In "Architecture
>> calling conventions" it documents the calling convention into the
>> kernel.  syscall(2) itself is an ordinary function which has to
>> follow the userspace ABI; after jumping into the kernel and setting
>> 'errno' in error case, it restores registers as needed.
>
> I don't think this is what it's talking about.

Perhaps we have different man pages but e.g. [1] mentions only registers
in the context of the kernel interface but not when entering/leaving
syscall(2) itself.


>> Some ABIs allow functions to clobber registers (they are not restored
>> after leaving the function and do not carry a return value); e.g. on
>> ARM, these are r0-r3 and r12.  That's probably the case for r10 in
>> ia64 too.
>
> Maybe you missed the previous message where I pointed out that this
> behavior is, at least on MIPS, an explicit step taken by glibc's
> syscall implementation (and many other system calls).

When, then this is completely undocumented and a glibc-only thing.
Other implementations[2] follow the behavior described in the man page
and do not set some magic registers on return.

I did not found the glibc syscall implementation for MIPS atm.



Enrico

Footnotes: 
[1]  http://man7.org/linux/man-pages/man2/syscall.2.html

[2]  
https://android.googlesource.com/platform/bionic/+/ae5c3dd73844e6a9e1a14dbf893eab5142902f18/libc/arch-mips/syscalls/syscall.S
 
https://github.com/ops-class/os161/blob/master/userland/lib/libc/arch/mips/syscalls-mips.S
 
https://github.com/m-labs/uclibc-lm32/blob/master/libc/sysdeps/linux/mips/syscall.S
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] pseudo: host user contamination

[Enrico Scholz]

Seebs  writes:

> And so far as I know, it's not actually *possible* to in the general
> case. I really don't think it's safe to try to catch syscall().

I think, something like


static void (*orig_syscall)();
long syscall(long number, ...)
{
switch (number) {
case __NR_renameat2: return _renameat2_syscall(...);
}

void*res =__builtin_apply(orig_syscall, __builtin_apply_args(),
  sizeof(uintmax_t) * 7);

__builtin_return(res);
}


will work to wrap syscall(2).  Params for _renameat2_syscall() can be
extracted by va_args.


Code generated above is very ineffective; perhaps you can create
specialized assembly instructions which just jump into orig_syscall.


Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] pseudo: host user contamination

[Enrico Scholz]

Andre McCurdy 
writes:

>> Since the man page gave the ia64 example, I went and checked, and it
>> is indeed the case that calls other than syscall(2) will clobber r10
>> after system calls,

I think you are misinterpreting the man-page.  In "Architecture
calling conventions" it documents the calling convention into the
kernel.  syscall(2) itself is an ordinary function which has to
follow the userspace ABI; after jumping into the kernel and setting
'errno' in error case, it restores registers as needed.

Some ABIs allow functions to clobber registers (they are not restored
after leaving the function and do not carry a return value); e.g. on
ARM, these are r0-r3 and r12.  That's probably the case for r10 in ia64
too.



Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] pseudo: host user contamination

[Enrico Scholz]

Seebs  writes:

>> > I... am really unsure whether it's possible to catch that, because
>> > I really, really, don't want to try to intercept raw syscall()
>> > calls. I don't think that ends well.
>
> Potentially. We rely pretty heavily on the assumption that an *actual*
> syscall can go through.

I think, this would end in implementing architecture dependening
assembly code.  E.g. for ARM you can write


syscall:
cmp r0, #__NR_renameat2
beq renameat2
ldr r12, _orig_syscall_addr
mov pc, [r9, r12]

_orig_syscall_addr: .word   orig_syscall_addr


(Untested; the last three lines are probably wrong and try to get the
address of the variable where the original syscall() address has been
stored into).


> Although... Actually, I don't even know if this is an actual syscall.
> This could be an actual glibc wrapper around the syscall interface,
> just like all the others, which is not the *actual* raw syscall or
> whatever, and... I have no idea how often that is or isn't hit.

'ltrace' catches it.



Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] pseudo: host user contamination

[Enrico Scholz]

"Burton, Ross"  writes:

>> do_install() {
>> install -D -p -m 0644 bar ${D}/bin/bar
>> install -d -m 0755 ${D}/usr/bin
>> mv ${D}/bin/bar ${D}/usr/bin/bar
>> ln -s /usr/bin/bar ${D}/bin/bar
>> }
>>
>> FILES_${PN} = "/bin/* /usr/bin/*"
>> ---
>>
>> WARNING: foo-1.0-r0 do_package_qa: QA Issue: foo: /foo/usr/bin/bar
>> is owned by uid 505, which is the same as the user running
>> bitbake. This may be due to host contamination
>> [host-user-contaminated]
>
> Works for me: install correctly changes owner to root:root as it
> should.

I think, 'mv' is the culprit.  It calls 'renameat2()' directly over
'syscall()':

| $ ltrace mv foo bar
| ...
| syscall(316, 0xff9c, 0x7fff1564a341, 0xff9c)  
  = 0


Perhaps, 'pseudo' does not catch this?


Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] pseudo: host user contamination

[Enrico Scholz]

Enrico Scholz
<enrico.scholz-wttK6gPy29v+Hn7q9Vec/7nah6klm...@public.gmane.org>
writes:

> in some packages (e.g. ncurses or glibc) I get package-qa warnings due
> to host user contamination; e.g.

can be reproduced by

---
LICENSE = "closed"

do_compile() {
echo foo > bar
}

do_install() {
install -D -p -m 0644 bar ${D}/bin/bar
install -d -m 0755 ${D}/usr/bin
mv ${D}/bin/bar ${D}/usr/bin/bar
ln -s /usr/bin/bar ${D}/bin/bar
}

FILES_${PN} = "/bin/* /usr/bin/*"
---

WARNING: foo-1.0-r0 do_package_qa: QA Issue: foo: /foo/usr/bin/bar is owned by 
uid 505, which is the same as the user running bitbake. This may be due to host 
contamination [host-user-contaminated]



Enrico
-- 
SIGMA Chemnitz GmbH   Registergericht:   Amtsgericht Chemnitz HRB 1750
Am Erlenwald 13   Geschaeftsfuehrer: Grit Freitag, Frank Pyritz
09128 Chemnitz
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] pseudo: host user contamination

[Enrico Scholz]

Hello,

in some packages (e.g. ncurses or glibc) I get package-qa warnings due
to host user contamination; e.g.

| WARNING: ncurses-6.0+20170715-r0 do_package_qa: QA Issue: ncurses: 
/ncurses-terminfo-base/etc/terminfo/a/ansi is owned by uid 505, which is the 
same as the user running bitbake. This may be due to host contamination
| ncurses: /ncurses-terminfo-base/etc/terminfo/d/dumb is owned by uid 505, 
which is the same as the user running bitbake. This may be due to host 
contamination
| ...
| ncurses: /ncurses-terminfo-base/etc/terminfo/x/xterm-256color is owned by uid 
505, which is the same as the user running bitbake. This may be due to host 
contamination [host-user-contaminated]


This is 100% reproducible by 'bitbake ncurses -c cleansstate && bitbake
ncurses'.


Pseudo log contains

| debug_logfile: fd 2
| pid 16096 [parent 16095], doing new pid setup and server start
| Setup complete, sending SIGUSR1 to pid 16095.
| path mismatch [2 links]: ino 382716451 db '/dev/shm/sem.mp-6eoy3v9n' req 
'/dev/shm/8JXYzj'.
| inode mismatch: 
'/ncurses/6.0+20170715-r0/image/usr/share/terminfo/a/ansi' ino 39261519 in 
db, 39262517 in request.
| symlink mismatch: 
'/ncurses/6.0+20170715-r0/image/usr/share/terminfo/a/ansi' [39261519] db 
mode 0100644, header mode 0120777 (unlinking db)

for all the reported files.and a lot of

| path mismatch [91 links]: ino 1605070 db ...

like messages for unrelated files (e.g  sysroot components).


Seen on recent rocko and master (1-2 weeks before).

System is Fedora 27 in a docker container; rootfs is on an overlaysfs,
recipes on a bind-mounted btrfs fs and build (TMPDIR) happens in an ext4
fs.  SELinux is active.



Is this issue known resp. does a solution exist?



Enrico
-- 
SIGMA Chemnitz GmbH   Registergericht:   Amtsgericht Chemnitz HRB 1750
Am Erlenwald 13   Geschaeftsfuehrer: Grit Freitag, Frank Pyritz
09128 Chemnitz

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oe-init-env: search 'bitbake' outside of oe-core

[Enrico Scholz]

atm, 'oe-init-build-env' expects 'bitbake' to be within the OE core git
repository.  This complicates the project setup because you have to
manage the 'bitbake' directory or symlink manually (or specify the
bitbake location explicitly).

Looking for 'bitbake' outside the main git repository will ease project
management significantly.  Now, you can put everything into git submodules,
clone the project with

| git clone --recursive ...

and continue immediately with

| /oe-init-build-env

E.g. when you had previously

| .
| |-- build/
| `-- sources/
| `-- org.openembedded.core/
| `-- bitbake -> ../bitbake

(where 'bitbake' must be created manually after cloning the project),
you can have now

| .
| |-- build/
| `-- sources/
| |-- bitbake/
| `-- org.openembedded.core/

which is completely managed by 'git'.

Patch adds $OEROOT/.. to the search path for 'bitbake'

Signed-off-by: Enrico Scholz <enrico.sch...@sigma-chemnitz.de>
---
 scripts/oe-buildenv-internal | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/oe-buildenv-internal b/scripts/oe-buildenv-internal
index 77f98a32b9..6773872326 100755
--- a/scripts/oe-buildenv-internal
+++ b/scripts/oe-buildenv-internal
@@ -101,6 +101,7 @@ unset BDIR
 
 if [ -z "$BITBAKEDIR" ]; then
 BITBAKEDIR="$OEROOT/bitbake$BBEXTRA"
+test -d "$BITBAKEDIR" || BITBAKEDIR="$OEROOT/../bitbake$BBEXTRA"
 fi
 
 BITBAKEDIR=$(readlink -f "$BITBAKEDIR")
-- 
2.14.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] image_types: support lz4 compressed squashfs

[Enrico Scholz]

Signed-off-by: Enrico Scholz <enrico.sch...@sigma-chemnitz.de>
---
 meta/classes/image_types.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 9b646e9..61dca62 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -107,6 +107,7 @@ IMAGE_CMD_btrfs () {
 IMAGE_CMD_squashfs = "mksquashfs ${IMAGE_ROOTFS} 
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.squashfs ${EXTRA_IMAGECMD} 
-noappend"
 IMAGE_CMD_squashfs-xz = "mksquashfs ${IMAGE_ROOTFS} 
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.squashfs-xz ${EXTRA_IMAGECMD} 
-noappend -comp xz"
 IMAGE_CMD_squashfs-lzo = "mksquashfs ${IMAGE_ROOTFS} 
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.squashfs-lzo 
${EXTRA_IMAGECMD} -noappend -comp lzo"
+IMAGE_CMD_squashfs-lz4 = "mksquashfs ${IMAGE_ROOTFS} 
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.squashfs-lz4 
${EXTRA_IMAGECMD} -noappend -comp lz4"
 
 # By default, tar from the host is used, which can be quite old. If
 # you need special parameters (like --xattrs) which are only supported
@@ -236,6 +237,7 @@ do_image_btrfs[depends] += 
"btrfs-tools-native:do_populate_sysroot"
 do_image_squashfs[depends] += "squashfs-tools-native:do_populate_sysroot"
 do_image_squashfs_xz[depends] += "squashfs-tools-native:do_populate_sysroot"
 do_image_squashfs_lzo[depends] += "squashfs-tools-native:do_populate_sysroot"
+do_image_squashfs_lz4[depends] += "squashfs-tools-native:do_populate_sysroot"
 do_image_elf[depends] += "virtual/kernel:do_populate_sysroot 
mkelfimage-native:do_populate_sysroot"
 do_image_ubi[depends] += "mtd-utils-native:do_populate_sysroot"
 do_image_ubifs[depends] += "mtd-utils-native:do_populate_sysroot"
@@ -251,7 +253,7 @@ IMAGE_TYPES = " \
 btrfs \
 iso \
 hddimg \
-squashfs squashfs-xz squashfs-lzo \
+squashfs squashfs-xz squashfs-lzo squashfs-lz4 \
 ubi ubifs multiubi \
 tar tar.gz tar.bz2 tar.xz tar.lz4 \
 cpio cpio.gz cpio.xz cpio.lzma cpio.lz4 \
-- 
2.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/3] wic: apply --extra-space + --overhead to squashfs

[Enrico Scholz]

Ed Bartosh  writes:

> I agree. --size is less suitable for your needs than extra space and
> overhead factor. I still don't like the idea of using them to reserve
> non-formatted space.

Btw, exactly this is done for extX already.


> Any other ideas how to do it?

Perhaps, Logic can be moved into the disk_size() method:

|def disk_size(self):
|method = getattr(self, "_get_disk_size_" + prefix, None)
|if method:
|return method(self.size)
|elif self.fixed_size:
|return self.fixed_size
|else:
|return self.size
|
|def _get_disk_size_squashfs(size):
|return self.get_rootfs_size(size)

But I did not checked whether this really works.


Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/3] wic: apply --extra-space + --overhead to squashfs

[Enrico Scholz]

Ed Bartosh  writes:

>> >> >> The --extra-space and --overhead option did not had an effect to 
>> >> >> squashfs
>> >> >> partitions.  Although squashfs is read-only, it can be useful to 
>> >> >> allocate
>> >> >> more space for the on-disk partition to avoid repartitioning of the 
>> >> >> whole
>> >> >> disk when a new (and larger) squashfs image is written on later 
>> >> >> updates.
>> >> >
>> >> > Is it possible to just use --size or --fixed-size in .wks to allocate
>> >> > partition of certain size?
>> >> 
>> >> --fixed-size works with this patch (tested); --size should work.
>> >
>> > Sorry for not being clear. Why do we need this if we can use --size to
>> > explicity specify partition size?
>> 
>> --size and --fixed-size did not had an effect for squashfs with the old
>> code.
>>
>
> I'd propose to fix this instead of applying extra space and overhead
> factor to the filesystems that don't support this by design.
>
> The idea is to make size and fixed-size parameters to set partition size
> and use extra-space and overhead-factor to extend filesystem size as
> they're currently used.

For what is this overhead to be used?  In most cases, to allow future
updates.  And this is required for squashfs too.

Only difference is, that updates for squashfs are to be applied on
partition level and these for other file systems on file system level.
But the need for extra space exists in both cases.


>> I want/need it to allow updates of the running system without complete
>> repartitioning.  E.g. at wic creation time, the squashfs has a certain
>> size.  Sometime in the future, there are needed e.g. 5 MiB more because
>> a new package was added or so.
>>
>
> Yep, I understood what you want. I think it's better achieved by setting
> partition size with --size option than artificially apply extra-space and
> overhead factor to the filesystem that doesn't support this.

It really does not matter for me whether the filesystem itself can be
extended or whether I need a larger partition to apply future updates.

I just need a partition which provides an absolute or relative amount of
additional space.


>> I need to reserve space in the partition so that I can 'dd' the new
>> image without a complete repartitioning.
>> 
>> --extra-space/--overhead has the meaning which I want (e.g. to say
>> "reserve +20% for changes in the future").
>
> So far extra-space and overhead factor were used to allocate additional
> space on the file system. This is different from the way you want to
> use them. You're not allocating space on squashfs, you're allocating
> unformatted space on the partition. It's better to use --size for this.

No; --size is not what I want. --size (or --fixed-size) assume that I
know the absolute size.

Of course, I can get this size empirically.  But it is highly inflexible
and requires different wks files for different image types.

I want to reserve an extra percentage for future updates.  And these
percentages can be in the range of 1 - 5 MB (for small images with
only test tools) or several hundred MB (for large image types with
e.g. desktop environments).

--size or --fixed-size for these image types would be in completely
different ranges while --extra-space/--overhead fits to all.


> I mean that method is called for all possible filesystems, but makes
> sense only for squashfs. It makes code less understandable for no
> reason. When code is used in the only place it's much more easy to
> understand from my point of view.

Not in my one.



Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/3] wic: apply --extra-space + --overhead to squashfs

[Enrico Scholz]

Ed Bartosh  writes:

>> >> The --extra-space and --overhead option did not had an effect to squashfs
>> >> partitions.  Although squashfs is read-only, it can be useful to allocate
>> >> more space for the on-disk partition to avoid repartitioning of the whole
>> >> disk when a new (and larger) squashfs image is written on later updates.
>> >
>> > Is it possible to just use --size or --fixed-size in .wks to allocate
>> > partition of certain size?
>> 
>> --fixed-size works with this patch (tested); --size should work.
>
> Sorry for not being clear. Why do we need this if we can use --size to
> explicity specify partition size?

--size and --fixed-size did not had an effect for squashfs with the old
code.


> --extra-space and --overhead have the same meaning as
> IMAGE_ROOTFS_EXTRA_SPACE and IMAGE_OVERHEAD_SIZE variables. From my
> point of view their purpose is to allocate additional space on the
> file system. This doesn't make sense for squashfs. That's the reason
> those options are not used for squashfs. Using them to implicitly make
> bigger partition is a questionable and possible confusing approach.

I want/need it to allow updates of the running system without complete
repartitioning.  E.g. at wic creation time, the squashfs has a certain
size.  Sometime in the future, there are needed e.g. 5 MiB more because
a new package was added or so.

I need to reserve space in the partition so that I can 'dd' the new
image without a complete repartitioning.

--extra-space/--overhead has the meaning which I want (e.g. to say
"reserve +20% for changes in the future").


>> I would keep it as is.  It is a non trivial function which is reusable.
>> Perhaps, somebody adds CRAMFS or another file system which does not
>> support extending its size.
>> 
>
> I personally find this much more understandable and logical:

me not ;) As I said, functionality is perfect for putting into an extra
method and it makes code much more readable.  When somebody implements
CRAMFS he can use this method instead of copy the code.


> Then introducing a method that will be called for all supported
> filesystems except squashfs without any effect.

You mean, function should be called at the end of prepare_rootfs()
(after "method(...)")?  Yes, it would reduce code duplication but other
methods needs to be touch too (to give information whether rootfs size
has been adjusted already).


Enrico
-- 
SIGMA Chemnitz GmbH   Registergericht:   Amtsgericht Chemnitz HRB 1750
Am Erlenwald 13   Geschaeftsfuehrer: Grit Freitag, Frank Pyritz
09128 Chemnitz
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 3/3] wic: apply --extra-space + --overhead to squashfs

[Enrico Scholz]

The --extra-space and --overhead option did not had an effect to squashfs
partitions.  Although squashfs is read-only, it can be useful to allocate
more space for the on-disk partition to avoid repartitioning of the whole
disk when a new (and larger) squashfs image is written on later updates.

Patch calls get_rootfs_size() *after* creating the image and truncates
it then.

Signed-off-by: Enrico Scholz <enrico.sch...@sigma-chemnitz.de>
---
 scripts/lib/wic/partition.py | 13 +
 1 file changed, 13 insertions(+)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index b623bb9..bf9bbc3 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -195,6 +195,17 @@ class Partition():
"larger (%d kB) than its allowed size %d kB" %
(self.mountpoint, self.size, self.fixed_size))
 
+def _extend_rootfs_image(self, rootfs):
+"""Enlarges the rootfs so that it fulfills size/overhead-factor
+constraints"""
+
+sz = (os.stat(rootfs).st_size + 1023) // 1024
+pad_sz = self.get_rootfs_size(sz)
+
+if pad_sz > sz:
+with open(rootfs, 'a') as f:
+os.ftruncate(f.fileno(), pad_sz * 1024)
+
 def prepare_rootfs(self, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot):
 """
@@ -338,6 +349,8 @@ class Partition():
(rootfs_dir, rootfs, extraopts)
 exec_native_cmd(squashfs_cmd, native_sysroot, pseudo=pseudo)
 
+self._extend_rootfs_image(rootfs)
+
 def prepare_empty_partition_ext(self, rootfs, oe_builddir,
 native_sysroot):
 """
-- 
2.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/3] wic: apply --extra-space + --overhead to squashfs

[Enrico Scholz]

Ed Bartosh  writes:

>> The --extra-space and --overhead option did not had an effect to squashfs
>> partitions.  Although squashfs is read-only, it can be useful to allocate
>> more space for the on-disk partition to avoid repartitioning of the whole
>> disk when a new (and larger) squashfs image is written on later updates.
>
> Is it possible to just use --size or --fixed-size in .wks to allocate
> partition of certain size?

--fixed-size works with this patch (tested); --size should work.


>> +def __extend_rootfs_image(self, rootfs):
>
> Do we really need to mangle name of this method?

ok; I will make it _extend_rootfs_image


> As this function is not going to be used anywhere I'd just use this
> code in prepare_rootfs_squashfs. It would be less generic, but much
> more readable and understandable.

I would keep it as is.  It is a non trivial function which is reusable.
Perhaps, somebody adds CRAMFS or another file system which does not
support extending its size.

partition.py is full of redundant code (look for all the "du" calls or
generally at the beginning of the prepare_rootfs_*() methods) where
probably somebody thought that it is not going to be used anywhere else
;)


Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] wic: apply --extra-space + --overhead to squashfs

[Enrico Scholz]

From: Enrico Scholz <enrico.sch...@ensc.de>

The --extra-space and --overhead option did not had an effect to squashfs
partitions.  Although squashfs is read-only, it can be useful to allocate
more space for the on-disk partition to avoid repartitioning of the whole
disk when a new (and larger) squashfs image is written on later updates.

Patch calls get_rootfs_size() *after* creating the image and truncates
it then.

Signed-off-by: Enrico Scholz <enrico.sch...@ensc.de>
---
 scripts/lib/wic/partition.py | 13 +
 1 file changed, 13 insertions(+)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index b623bb9..da651f8 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -195,6 +195,17 @@ class Partition():
"larger (%d kB) than its allowed size %d kB" %
(self.mountpoint, self.size, self.fixed_size))
 
+def __extend_rootfs_image(self, rootfs):
+"""Enlarges the rootfs so that it fulfills size/overhead-factor
+constraints"""
+
+sz = (os.stat(rootfs).st_size + 1023) // 1024
+pad_sz = self.get_rootfs_size(sz)
+
+if pad_sz > sz:
+with open(rootfs, 'a') as f:
+os.ftruncate(f.fileno(), pad_sz * 1024)
+
 def prepare_rootfs(self, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot):
 """
@@ -338,6 +349,8 @@ class Partition():
(rootfs_dir, rootfs, extraopts)
 exec_native_cmd(squashfs_cmd, native_sysroot, pseudo=pseudo)
 
+self.__extend_rootfs_image(rootfs)
+
 def prepare_empty_partition_ext(self, rootfs, oe_builddir,
 native_sysroot):
 """
-- 
2.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] wic: allow multiple /boot partitions with different content

[Enrico Scholz]

From: Enrico Scholz <enrico.sch...@ensc.de>

It can be useful to have multiple partitions with '--source bootimg-partition'
but different content.

E.g. for TI AM335x, one boot partition can contain an first stage
bootloader ("MLO"), while the real bootloader and kernel plus devicetree
are in another one.

Patch allows to specify multiple IMAGE_BOOT_FILES with optional "_label-XXX"
or "_uuid-XXX" overrides.

E.g. with this patch, a .wks file with

| part --source bootimg-partition ... --label=mlo --active
| part --source bootimg-partition ... --label=boot0
| part --source bootimg-partition ... --label=boot1

and a recipe with

| IMAGE_BOOT_FILES_label-mlo = "\
|   MLO-${MACHINE}.img;MLO \
| "
|
| IMAGE_BOOT_FILES_label-boot0 = "\
|   u-boot-${MACHINE}.img;u-boot.img \
|   zImage \
| "
|
| IMAGE_BOOT_FILES_label-boot1 = "${IMAGE_BOOT_FILES_label-boot0}"
|
| WICVARS += " \
|   IMAGE_BOOT_FILES_label-mlo \
|   IMAGE_BOOT_FILES_label-boot0 \
|   IMAGE_BOOT_FILES_label-boot1 \
| "

is possible.  It will create one partition with the MLO and two redundant
ones with the uboot + kernel.

Signed-off-by: Enrico Scholz <enrico.sch...@ensc.de>
---
 scripts/lib/wic/plugins/source/bootimg-partition.py | 17 +
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/bootimg-partition.py 
b/scripts/lib/wic/plugins/source/bootimg-partition.py
index ca074a3..67e5498 100644
--- a/scripts/lib/wic/plugins/source/bootimg-partition.py
+++ b/scripts/lib/wic/plugins/source/bootimg-partition.py
@@ -54,7 +54,7 @@ class BootimgPartitionPlugin(SourcePlugin):
 - sets up a vfat partition
 - copies all files listed in IMAGE_BOOT_FILES variable
 """
-hdddir = "%s/boot" % cr_workdir
+hdddir = "%s/boot.%d" % (cr_workdir, part.lineno)
 install_cmd = "install -d %s" % hdddir
 exec_cmd(install_cmd)
 
@@ -65,10 +65,19 @@ class BootimgPartitionPlugin(SourcePlugin):
 
 logger.debug('Kernel dir: %s', bootimg_dir)
 
-boot_files = get_bitbake_var("IMAGE_BOOT_FILES")
+boot_files = None
+for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s", part.label), 
(None, None)):
+if fmt:
+var = fmt % id
+else:
+var = ""
+
+boot_files = get_bitbake_var("IMAGE_BOOT_FILES" + var)
+if boot_files is not None:
+break
 
-if not boot_files:
-raise WicError('No boot files defined, IMAGE_BOOT_FILES unset')
+if boot_files is None:
+raise WicError('No boot files defined, IMAGE_BOOT_FILES unset for 
entry #%d' % part.lineno)
 
 logger.debug('Boot files: %s', boot_files)
 
-- 
2.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] wic: accept '-' in bitbake variables

[Enrico Scholz]

From: Enrico Scholz <enrico.sch...@ensc.de>

'-' is valid and common in bitbake variables (e.g. 'FOO_pn-bar'). Accept
it and other characters when reading the .env file.

Also, allow variables to be empty.

Signed-off-by: Enrico Scholz <enrico.sch...@ensc.de>
---
 scripts/lib/wic/misc.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
index 4e07cd6..0792480 100644
--- a/scripts/lib/wic/misc.py
+++ b/scripts/lib/wic/misc.py
@@ -183,7 +183,7 @@ class BitbakeVars(defaultdict):
 self.default_image = None
 self.vars_dir = None
 
-def _parse_line(self, line, image, matcher=re.compile(r"^(\w+)=(.+)")):
+def _parse_line(self, line, image, 
matcher=re.compile(r"^([a-zA-Z0-9\-_+./~]+)=(.*)")):
 """
 Parse one line from bitbake -e output or from .env file.
 Put result key-value pair into the storage.
-- 
2.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] wic: enhanced bootimage + squashfs support

[Enrico Scholz]

Patchset allows to use different bootimage sources and to reserve more
space for squashfs partitions.

Enrico Scholz (3):
  wic: accept '-' in bitbake variables
  wic: allow multiple /boot partitions with different content
  wic: apply --extra-space + --overhead to squashfs

 scripts/lib/wic/misc.py |  2 +-
 scripts/lib/wic/partition.py| 13 +
 scripts/lib/wic/plugins/source/bootimg-partition.py | 17 +
 3 files changed, 27 insertions(+), 5 deletions(-)

-- 
2.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] image_types: support lz4 compressed squashfs

[Enrico Scholz]

Signed-off-by: Enrico Scholz <enrico.sch...@sigma-chemnitz.de>
---
 meta/classes/image_types.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 9b646e9..d086061 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -107,6 +107,7 @@ IMAGE_CMD_btrfs () {
 IMAGE_CMD_squashfs = "mksquashfs ${IMAGE_ROOTFS} 
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.squashfs ${EXTRA_IMAGECMD} 
-noappend"
 IMAGE_CMD_squashfs-xz = "mksquashfs ${IMAGE_ROOTFS} 
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.squashfs-xz ${EXTRA_IMAGECMD} 
-noappend -comp xz"
 IMAGE_CMD_squashfs-lzo = "mksquashfs ${IMAGE_ROOTFS} 
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.squashfs-lzo 
${EXTRA_IMAGECMD} -noappend -comp lzo"
+IMAGE_CMD_squashfs-lz4 = "mksquashfs ${IMAGE_ROOTFS} 
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.squashfs-lzo 
${EXTRA_IMAGECMD} -noappend -comp lz4"
 
 # By default, tar from the host is used, which can be quite old. If
 # you need special parameters (like --xattrs) which are only supported
@@ -236,6 +237,7 @@ do_image_btrfs[depends] += 
"btrfs-tools-native:do_populate_sysroot"
 do_image_squashfs[depends] += "squashfs-tools-native:do_populate_sysroot"
 do_image_squashfs_xz[depends] += "squashfs-tools-native:do_populate_sysroot"
 do_image_squashfs_lzo[depends] += "squashfs-tools-native:do_populate_sysroot"
+do_image_squashfs_lz4[depends] += "squashfs-tools-native:do_populate_sysroot"
 do_image_elf[depends] += "virtual/kernel:do_populate_sysroot 
mkelfimage-native:do_populate_sysroot"
 do_image_ubi[depends] += "mtd-utils-native:do_populate_sysroot"
 do_image_ubifs[depends] += "mtd-utils-native:do_populate_sysroot"
@@ -251,7 +253,7 @@ IMAGE_TYPES = " \
 btrfs \
 iso \
 hddimg \
-squashfs squashfs-xz squashfs-lzo \
+squashfs squashfs-xz squashfs-lzo squashfs-lz4 \
 ubi ubifs multiubi \
 tar tar.gz tar.bz2 tar.xz tar.lz4 \
 cpio cpio.gz cpio.xz cpio.lzma cpio.lz4 \
-- 
2.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] externalsrc: place copy of git index into /tmp and do not use copyfile2

[Enrico Scholz]

Using shutil.copy2() to copy .git/index to a temporary file tries to
copy SELinux attributes which might fail for confined users in SELinux
environments.

E.g. our builders are running in docker containers and modification of
sources (inclusive updated of .git/index) is done outside.  Trying to
copy .git/index fails with

| $ python3 -c 'import shutil; shutil.copy2("index", "a")'
| ...
| PermissionError: [Errno 13] Permission denied: 'a'

and an AVC like

| denied  { relabelto } for  pid=18043 comm="python3" name="a" dev="dm-29" 
ino=1067553 scontext=system_u:system_r:container_t:s0:c39,c558 
tcontext=unconfined_u:object_r:build_file_t:s0 tclass=file permissive=0

is created.  This can not be solved by adapting the SELinux policy because
this is a very deep constraint violation:

| constrain file { create relabelfrom relabelto } ((u1 == u2 -Fail-)  or (t1 == 
can_change_object_identity -Fail-) ); Constraint DENIED
|
| Possible cause is the source user (system_u) and target user (unconfined_u) 
are different.


I do not see much sense in using 'shutil.copy2()' here; 'shutil.copyfile()'
seems to be a better choice (target file is created in a secure way by
tempfile.NamedTemporaryFile()).

By placing the tempfile into /tmp we avoid potential problems related to
git's 'core.sharedRepository'.  As a (positive) side effect, the source
tree will not be modified anymore (at least by this part of code) which
prevented to mount it read-only from somewhere else.

Signed-off-by: Enrico Scholz <enrico.sch...@sigma-chemnitz.de>
---
 meta/classes/externalsrc.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
index 529be49..9aabb42 100644
--- a/meta/classes/externalsrc.bbclass
+++ b/meta/classes/externalsrc.bbclass
@@ -189,9 +189,9 @@ def srctree_hash_files(d, srcdir=None):
 
 ret = " "
 if os.path.exists(git_dir):
-with tempfile.NamedTemporaryFile(dir=git_dir, 
prefix='oe-devtool-index') as tmp_index:
+with tempfile.NamedTemporaryFile(prefix='oe-devtool-index') as 
tmp_index:
 # Clone index
-shutil.copy2(os.path.join(git_dir, 'index'), tmp_index.name)
+shutil.copyfile(os.path.join(git_dir, 'index'), tmp_index.name)
 # Update our custom index
 env = os.environ.copy()
 env['GIT_INDEX_FILE'] = tmp_index.name
-- 
2.9.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] tzcode-native: quote ${CC}

[Enrico Scholz]

build fails else with

| + make -j 8 -l 6 cc=ccache gcc
| make: *** No rule to make target 'gcc'.  Stop.

Signed-off-by: Enrico Scholz <enrico.sch...@sigma-chemnitz.de>
---
 meta/recipes-extended/tzcode/tzcode-native_2017b.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-extended/tzcode/tzcode-native_2017b.bb 
b/meta/recipes-extended/tzcode/tzcode-native_2017b.bb
index 2084f79..165d2c6 100644
--- a/meta/recipes-extended/tzcode/tzcode-native_2017b.bb
+++ b/meta/recipes-extended/tzcode/tzcode-native_2017b.bb
@@ -18,7 +18,7 @@ S = "${WORKDIR}"
 
 inherit native
 
-EXTRA_OEMAKE += "cc=${CC}"
+EXTRA_OEMAKE += "cc='${CC}'"
 
 do_install () {
 install -d ${D}${bindir}/
-- 
2.9.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] Per-recipe sysroot + BBCLASSEXTEND=cross

[Enrico Scholz]

Hello,

when having

--- test-a.bb ---
LICENSE = "GPLv3"
LIC_FILES_CHKSUM = 
"file://${COMMON_LICENSE_DIR}/GPL-3.0;md5=c79ff39f19dfec6d293b95dea7b07891"

BBCLASSEXTEND = "cross"

do_install_forcevariable() {
install -d ${D}
touch ${D}/foo
}


and

--- test-b.bb ---
LICENSE = "GPLv3"
LIC_FILES_CHKSUM = 
"file://${COMMON_LICENSE_DIR}/GPL-3.0;md5=c79ff39f19dfec6d293b95dea7b07891"

DEPENDS = "test-a-cross"


the sysroot of 'test-b' will not contain the files from 'test-a-cross'
because the manifest could not be found:

| $ ./bitbake test-b
| WARNING: test-b-1.0-r0 do_prepare_recipe_sysroot: Manifest ... 
manifest-allarch-test-a-cross.populate_sysroot not found?
| ...

The sysroot code expects to find '-cross-' in the PN, so I changed
test-a.bb to contain

---
python () {
override = d.getVar("CLASSOVERRIDE", True) or ""
if override == "class-cross":
d.appendVar("PN", "-${TARGET_ARCH}")
d.setVar("SPECIAL_PKGSUFFIX", "-cross-${TARGET_ARCH}")
}
---


Is there a better way to use BBCLASSEXTEND to build cross recipes?


Enrico
-- 
SIGMA Chemnitz GmbH   Registergericht:   Amtsgericht Chemnitz HRB 1750
Am Erlenwald 13   Geschaeftsfuehrer: Grit Freitag, Frank Pyritz
09128 Chemnitz

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] tzcode-native: quote ${CC}

[Enrico Scholz]

Denys Dmytriyenko  writes:

>> -EXTRA_OEMAKE += "cc=${CC}"
>> +EXTRA_OEMAKE += "cc='${CC}'"
>
> Should these be double-quotes around ${CC}?

I wrote it in this way:

- to 60%, because '' is more correct than "" in this context (see below)

- to 40%, because "" would clutter code too much and I can not say
  whether it must be quoted as \", \\" or whether perhaps plain " works
  too


A yet more correct solution would be

| export cc = "${CC}"

in the recipe (outside of do_*()) and perhaps adding '-e' to EXTRA_OEMAKE.


This handles additionally the case when 'CC' contains a single quote
(which causes misbehavior in my patch).

Double quotes are much more worse because CC must not contain '$', '`'
or '\\' (which is not uncommon in flags like '-DFOO()=\"bar\"').  In OE,
it is not expected that these characters are interpreted directly by the
shell (${CC} is passed properly quoted in the environment).

But all this does not matter... correct quoting is neglactted in OE and
bitbake; when you are lucky, commands are written as "foo '%s'" % path.
But nobody uses functions like 'pipes.quote()' or a non-shell variants
like subprocess.call(['foo', path]).

So, you can assume that '${FOO}' expands to the same value like "${FOO}".


Enrico
-- 
SIGMA Chemnitz GmbH   Registergericht:   Amtsgericht Chemnitz HRB 1750
Am Erlenwald 13   Geschaeftsfuehrer: Grit Freitag, Frank Pyritz
09128 Chemnitz
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] tzcode-native: quote ${CC}

[Enrico Scholz]

build fails else with

| + make -j 8 -l 6 cc=ccache gcc
| make: *** No rule to make target 'gcc'.  Stop.

Signed-off-by: Enrico Scholz <enrico.sch...@sigma-chemnitz.de>
---
 meta/recipes-extended/tzcode/tzcode-native_2017b.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-extended/tzcode/tzcode-native_2017b.bb 
b/meta/recipes-extended/tzcode/tzcode-native_2017b.bb
index 2084f79..165d2c6 100644
--- a/meta/recipes-extended/tzcode/tzcode-native_2017b.bb
+++ b/meta/recipes-extended/tzcode/tzcode-native_2017b.bb
@@ -18,7 +18,7 @@ S = "${WORKDIR}"
 
 inherit native
 
-EXTRA_OEMAKE += "cc=${CC}"
+EXTRA_OEMAKE += "cc='${CC}'"
 
 do_install () {
 install -d ${D}${bindir}/
-- 
2.9.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] how to *securely* do a remote install of an OE image?

[Enrico Scholz]

"Robert P. J. Day" 
writes:

>   here's a puzzler someone just presented me with ... given an
> established OE build, and a way to download and install it to a
> remote system, how would one set up root and/or non-root accounts
> and passwords to then allow an *authorized* someone to log in to the
> newly-installed system and configure it?
>
>   as in, if the installed image contains a "root" account and default
> password, there is the potential of someone sneaking in there in the
> interval between installation and when the "authorized" user connects
> to do the configuration, even if the first act of the authorized user
> is to change the root password.

I do not think that this can be prevented.  When an attacker gains root
access, he can do everything with the system.  E.g. he can replace your
intrusion detection system with a custom one which does not report his
login.

You could use hardware features like burning fuses, or erasing keys in
TPM chips or increasing login counters there.  But again, an attacker
can alter the underlying login mechanism to report an untouched system.

So, you should try to prevent that an attacker can login to the systsem.


>   my immediate reaction was to use SSH keys, where the
> newly-installed system would require SSH logins, and would have to
> match the corresponding private key.

Yes; public key authentication is the way to go... You will never get
the system secure with a static default root password.

For SSH keys we are using [1] for ages.

OpenSSH supports CA keys[2]; e.g. you do not need to add every allowed
pubkey to 'authorized_keys' but only a CA pubkey.  This CA signs user
keys which are allowed to login.

Alternatively, you can implement something which sets the default
root password based on some serial number of the device.  But this is
difficultly to implement and brute force attacks might be very simple
(serial numbers are usually predictable).

Or, you can implement an OTP mechanism based on DSS or so.  E.g. device
contains a public key and presents a random number at login.  You have
to sign it with the corresponding secret key and enter it as a password.

 


Enrico

Footnotes: 
[1]  
https://github.com/sigma-embedded/elito-de.sigma-chemnitz/blob/master/meta/classes/elito-image.bbclass#L56

[2]  
https://www.digitalocean.com/community/tutorials/how-to-create-an-ssh-ca-to-validate-hosts-and-clients-with-ubuntu

-- 
SIGMA Chemnitz GmbH   Registergericht:   Amtsgericht Chemnitz HRB 1750
Am Erlenwald 13   Geschaeftsfuehrer: Grit Freitag, Frank Pyritz
09128 Chemnitz
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 4/4] ncurses: fix beaglebone do_configure failure

[Enrico Scholz]

"Burton, Ross"  writes:

>> c) fix the function that detects the "broken" use of CC as it should be
>> extracting the flags and moving them to CFLAGS.  Clearly this isn't working
>> somewhere.

Upstream tries to do it correctly for nearly a year and fails everytime
at some cornercases.


> I've a local working fix that basically does this:
>
> - cf_flags=`echo "$CC" | sed -e 's/^.*[ ]\(-[^ ]\)/\1/'`
> + cf_flags=`echo "$CC" | cut -f2- -d' '`

This will break for

  CC='ccache gcc -m32'

Perhaps there are cases (not for OE), where gcc contains a space... How
to deal with this?



Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 4/4] ncurses: fix beaglebone do_configure failure

[Enrico Scholz]

Hongxu Jia  writes:

> Append ${HOST_CC_ARCH} to ${CFLAGS} could fix the issue.

I think, the correct fix would be

a) bother the ncurses author to remove the stupid CC sanity check, or

b) create a wrapper around CC like in

   
https://github.com/sigma-embedded/elito-de.sigma-chemnitz/blob/master/meta-fixup/recipes/ncurses/ncurses_6.0%2B20160625.bbappend


Enrico
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] host-user-contaminated QA check

[Enrico Scholz]

Patrick Ohly 
writes:

> Recently the host-user-contaminated QA check triggered for the trousers
> recipe in meta-security:
>
> WARNING: trousers-0.3.14+gitAUTOINC+4b9a70d578-r0 do_package_qa: QA Issue: 
> trousers: /trousers/etc/tcsd.conf is owned by uid 1000, which is the same as 
> the user running bitbake. This may be due to host contamination 
> [host-user-contaminated]
>
> However, that's a false positive in this case. UID 1000 got assigned to
> the "tss" user in the target sysroot during the build, and tcsd.conf is
> correctly and intentionally owned by that user because tcsd checks
> ownership and refuses to start when owned by someone else (including
> root). It just happened that the UID was the same.
>
> This is likely to affect all recipes with files owned by dynamically
> created users, in particular when the host system assigns UIDs from the
> same range as the target system (quick poll: who else has 1000 as his
> UID on his main Linux box? ;-)

Usually, this can not happen.  There is reserved a range for dynamically
created users (standard says 100-499, some distributions use 100-999).

In this case, there is probably some '--system' flag missing when the
'tss' user is created (--> packaging bug).



Enrico
-- 
SIGMA Chemnitz GmbH   Registergericht:   Amtsgericht Chemnitz HRB 1750
Am Erlenwald 13   Geschaeftsfuehrer: Grit Freitag, Frank Pyritz
09128 Chemnitz
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core