Re: [OE-core] [PATCH] shadow: 'useradd' copies root's extended attributes
On Mon, 2018-01-15 at 15:33 +0100, José Bollo wrote: > A possibility would be to filter the copied extended attributes. For > SELinux we can just tell to not copy "security" attributes. See > manual of the command "tar" (recent version) that has options > --xattrs-exclude and --xattr-include. > > Is there a need to copy extended attributes except for Smack? In theory file-based capabilities. In practice those probably don't occur in a home directory template. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] shadow: 'useradd' copies root's extended attributes
On Tue, 2018-01-09 at 11:51 -0600, Mark Hatle wrote: > On 1/4/18 4:41 AM, Patrick Ohly wrote: > > On Thu, 2018-01-04 at 11:18 +0100, José Bollo wrote: > > > > Do you agree to move the patch to Smack specific layer? Such > > > > as > > > > meta-security? > > > > > > I agree. > > > > Layers like meta-security should not modify recipes from other > > layers, > > at least not by default. That would violate the "Yocto Compatible > > 2.0" > > rules. > > You can modify (bbappend) to an existing recipe. You can't change > the behavior > (specifically the md5sum) of the function though, unless that new > functionality > is enabled.) That's what I meant with "by default". > 'smack' should be able to do the same thing, with a similar distro > feature. I'm not convinced that building core components differently depending on such distro features is desirable, because it makes "smack" and "selinux" mutually exclusive. I'd prefer a solution where support for both can be enabled and then on the image itself the tools decide what to do. Whether that's always possible of course is a different question. In this case I think it is, by adding the exception for security.selinux. But I'll leave that up to you and Jose to decide. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] shadow: 'useradd' copies root's extended attributes
On Fri, 2018-01-05 at 01:07 +, Fan, Wenzong wrote: > It works and will override the labels of home dir that SELinux > applied, that's the issue. > > For SELinux enabled system, the user's home dir should have lavel > 'user_home_dir_t' instead of 'etc_t', it prevents users from creating > files in their home dir. Sounds like the "copy xattr" function needs to become a bit smarter: it needs to understand some of the semantic involved and skip those SELinux xattrs that are always meant to be set dynamically by the running kernel. Wenzong, which xattrs are those? Do you agree with the proposed solution? Jose, can you look into updating your patch accordingly? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] shadow: 'useradd' copies root's extended attributes
On Thu, 2018-01-04 at 19:39 +0800, wenzong fan wrote: > If so, I think we should wrapper the logic with: > > +#if defined(WITH_ATTR) && !defined(WITH_SELINUX) > + attr_copy_file (def_template, user_home, NULL, NULL); > +#endif Does attr_copy_file fail when SELinux is active? In other words, why should it be disabled when using SELinux? File capabilities are also stored in xattrs. It might be relevant to copy those when using SELinux. Or do I miss something? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] shadow: 'useradd' copies root's extended attributes
On Thu, 2018-01-04 at 11:18 +0100, José Bollo wrote: > > Do you agree to move the patch to Smack specific layer? Such as > > meta-security? > > I agree. Layers like meta-security should not modify recipes from other layers, at least not by default. That would violate the "Yocto Compatible 2.0" rules. Besides, it would be harder to maintain in a separate layer - for the maintainer of that layer. I still think this patch belong into OE-core, even though it is then more work for the OE-core maintainer. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] qemu: use upstream swtpm support
This aligns TPM support in OE with the approach accepted and merged
upstream.
An update for swtpm in meta-security was already sent
("[meta-security][PATCH 1/1] swtpm/libtpm: update to latest master").
Changes:
-v2: rebased
Patrick Ohly (1):
qemu: use upstream swtpm support
meta/recipes-devtools/qemu/qemu/0001-Provide-support-for-the-CUSE-TPM.patch
| 870
+---
meta/recipes-devtools/qemu/qemu/0001-tpm-Clean-up-driver-registration-lookup.patch
| 154 ++-
meta/recipes-devtools/qemu/qemu/0002-Introduce-condition-to-notify-waiters-of-completed-c.patch
| 86 +--
meta/recipes-devtools/qemu/qemu/0002-tpm-Clean-up-model-registration-lookup.patch
| 121 -
meta/recipes-devtools/qemu/qemu/0003-Introduce-condition-in-TPM-backend-for-notification.patch
| 79 +-
meta/recipes-devtools/qemu/qemu/0003-tpm-backend-Remove-unneeded-member-variable-from-bac.patch
| 75 +-
meta/recipes-devtools/qemu/qemu/0004-Add-support-for-VM-suspend-resume-for-TPM-TIS-v2.9.patch
| 719 +-
meta/recipes-devtools/qemu/qemu/0004-tpm-backend-Move-thread-handling-inside-TPMBackend.patch
| 417 -
meta/recipes-devtools/qemu/qemu/0005-tpm-backend-Initialize-and-free-data-members-in-it-s.patch
| 185 +-
meta/recipes-devtools/qemu/qemu/0006-tpm-backend-Made-few-interface-methods-optional.patch
| 284 +++-
meta/recipes-devtools/qemu/qemu/0007-tpm-backend-Add-new-api-to-read-backend-TpmInfo.patch
| 293 -
meta/recipes-devtools/qemu/qemu/0008-tpm-backend-Move-realloc_buffer-implementation-to-tp.patch
| 140 ++-
meta/recipes-devtools/qemu/qemu/0009-tpm-passthrough-move-reusable-code-to-utils.patch
| 182 -
meta/recipes-devtools/qemu/qemu/0010-tpm-Added-support-for-TPM-emulator.patch
| 1059
-
meta/recipes-devtools/qemu/qemu/0011-tpm-Move-tpm_cleanup-to-right-place.patch
| 43 +++-
meta/recipes-devtools/qemu/qemu/0012-tpm-Use-EMSGSIZE-instead-of-EBADMSG-to-compile-on-Op.patch
| 67 +-
meta/recipes-devtools/qemu/qemu/chardev-connect-socket-to-a-spawned-command.patch
| 227 +++-
meta/recipes-devtools/qemu/qemu_2.10.1.bb
| 17 +-
18 files changed, 3260 insertions(+), 1758 deletions(-)
delete mode 100644
meta/recipes-devtools/qemu/qemu/0001-Provide-support-for-the-CUSE-TPM.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0001-tpm-Clean-up-driver-registration-lookup.patch
delete mode 100644
meta/recipes-devtools/qemu/qemu/0002-Introduce-condition-to-notify-waiters-of-completed-c.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0002-tpm-Clean-up-model-registration-lookup.patch
delete mode 100644
meta/recipes-devtools/qemu/qemu/0003-Introduce-condition-in-TPM-backend-for-notification.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0003-tpm-backend-Remove-unneeded-member-variable-from-bac.patch
delete mode 100644
meta/recipes-devtools/qemu/qemu/0004-Add-support-for-VM-suspend-resume-for-TPM-TIS-v2.9.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0004-tpm-backend-Move-thread-handling-inside-TPMBackend.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0005-tpm-backend-Initialize-and-free-data-members-in-it-s.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0006-tpm-backend-Made-few-interface-methods-optional.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0007-tpm-backend-Add-new-api-to-read-backend-TpmInfo.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0008-tpm-backend-Move-realloc_buffer-implementation-to-tp.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0009-tpm-passthrough-move-reusable-code-to-utils.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0010-tpm-Added-support-for-TPM-emulator.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0011-tpm-Move-tpm_cleanup-to-right-place.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0012-tpm-Use-EMSGSIZE-instead-of-EBADMSG-to-compile-on-Op.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/chardev-connect-socket-to-a-spawned-command.patch
base-commit: a7cd9d1183be603777fc9c8c448281fe01224f7b
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] qemu: use upstream swtpm support
This aligns TPM support in OE with the approach accepted and merged
upstream.
An update for swtpm in meta-security was already sent
("[meta-security][PATCH 1/1] swtpm/libtpm: update to latest master").
Patrick Ohly (1):
qemu: use upstream swtpm support
meta/recipes-devtools/qemu/qemu/0001-Provide-support-for-the-CUSE-TPM.patch
| 870
+---
meta/recipes-devtools/qemu/qemu/0001-tpm-Clean-up-driver-registration-lookup.patch
| 154 ++-
meta/recipes-devtools/qemu/qemu/0002-Introduce-condition-to-notify-waiters-of-completed-c.patch
| 86 +--
meta/recipes-devtools/qemu/qemu/0002-tpm-Clean-up-model-registration-lookup.patch
| 121 -
meta/recipes-devtools/qemu/qemu/0003-Introduce-condition-in-TPM-backend-for-notification.patch
| 79 +-
meta/recipes-devtools/qemu/qemu/0003-tpm-backend-Remove-unneeded-member-variable-from-bac.patch
| 75 +-
meta/recipes-devtools/qemu/qemu/0004-Add-support-for-VM-suspend-resume-for-TPM-TIS-v2.9.patch
| 719 +-
meta/recipes-devtools/qemu/qemu/0004-tpm-backend-Move-thread-handling-inside-TPMBackend.patch
| 417 -
meta/recipes-devtools/qemu/qemu/0005-tpm-backend-Initialize-and-free-data-members-in-it-s.patch
| 185 +-
meta/recipes-devtools/qemu/qemu/0006-tpm-backend-Made-few-interface-methods-optional.patch
| 284 +++-
meta/recipes-devtools/qemu/qemu/0007-tpm-backend-Add-new-api-to-read-backend-TpmInfo.patch
| 293 -
meta/recipes-devtools/qemu/qemu/0008-tpm-backend-Move-realloc_buffer-implementation-to-tp.patch
| 140 ++-
meta/recipes-devtools/qemu/qemu/0009-tpm-passthrough-move-reusable-code-to-utils.patch
| 182 -
meta/recipes-devtools/qemu/qemu/0010-tpm-Added-support-for-TPM-emulator.patch
| 1059
-
meta/recipes-devtools/qemu/qemu/0011-tpm-Move-tpm_cleanup-to-right-place.patch
| 43 +++-
meta/recipes-devtools/qemu/qemu/0012-tpm-Use-EMSGSIZE-instead-of-EBADMSG-to-compile-on-Op.patch
| 67 +-
meta/recipes-devtools/qemu/qemu/chardev-connect-socket-to-a-spawned-command.patch
| 227 +++-
meta/recipes-devtools/qemu/qemu_2.10.0.bb
| 17 +-
18 files changed, 3260 insertions(+), 1758 deletions(-)
delete mode 100644
meta/recipes-devtools/qemu/qemu/0001-Provide-support-for-the-CUSE-TPM.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0001-tpm-Clean-up-driver-registration-lookup.patch
delete mode 100644
meta/recipes-devtools/qemu/qemu/0002-Introduce-condition-to-notify-waiters-of-completed-c.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0002-tpm-Clean-up-model-registration-lookup.patch
delete mode 100644
meta/recipes-devtools/qemu/qemu/0003-Introduce-condition-in-TPM-backend-for-notification.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0003-tpm-backend-Remove-unneeded-member-variable-from-bac.patch
delete mode 100644
meta/recipes-devtools/qemu/qemu/0004-Add-support-for-VM-suspend-resume-for-TPM-TIS-v2.9.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0004-tpm-backend-Move-thread-handling-inside-TPMBackend.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0005-tpm-backend-Initialize-and-free-data-members-in-it-s.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0006-tpm-backend-Made-few-interface-methods-optional.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0007-tpm-backend-Add-new-api-to-read-backend-TpmInfo.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0008-tpm-backend-Move-realloc_buffer-implementation-to-tp.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0009-tpm-passthrough-move-reusable-code-to-utils.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0010-tpm-Added-support-for-TPM-emulator.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0011-tpm-Move-tpm_cleanup-to-right-place.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/0012-tpm-Use-EMSGSIZE-instead-of-EBADMSG-to-compile-on-Op.patch
create mode 100644
meta/recipes-devtools/qemu/qemu/chardev-connect-socket-to-a-spawned-command.patch
base-commit: 3b413a80578caacd9a7f405f3c51a3921d78a60d
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] rootfs-postcommands.bbclass: ensure that rootfs gets mounted ro
When read-only-rootfs is active, we need to ensure that the rootfs
does not get mounted read/write by the kernel or initramfs. Adding
"ro" to the boot parameters achieves that.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/classes/rootfs-postcommands.bbclass | 8
1 file changed, 8 insertions(+)
diff --git a/meta/classes/rootfs-postcommands.bbclass
b/meta/classes/rootfs-postcommands.bbclass
index 5391e7a..a4e627f 100644
--- a/meta/classes/rootfs-postcommands.bbclass
+++ b/meta/classes/rootfs-postcommands.bbclass
@@ -14,6 +14,14 @@ ROOTFS_POSTPROCESS_COMMAND += "rootfs_update_timestamp ; "
# Tweak the mount options for rootfs in /etc/fstab if read-only-rootfs is
enabled
ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("IMAGE_FEATURES",
"read-only-rootfs", "read_only_rootfs_hook; ", "",d)}'
+# We also need to do the same for the kernel boot parameters,
+# otherwise kernel or initramfs end up mounting the rootfs read/write
+# (the default) if supported by the underlying storage.
+#
+# We do this with _append because the default value might get set later with ?=
+# and we don't want to disable such a default that by setting a value here.
+APPEND_append = '${@bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs", "
ro", "", d)}'
+
# Generates test data file with data store variables expanded in json format
ROOTFS_POSTPROCESS_COMMAND += "write_image_test_data ; "
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] connman.inc: do not check IMAGE_FEATURES
Recipes can't rely on IMAGE_FEATURES to determine whether the
resulting packages will be used in an image with read/write or
read-only rootfs because IMAGE_FEATURES is a per-image recipe
variable.
The connman.inc code checked IMAGE_FEATURES to determine whether
/var/run/connman needs to be created via tmpfiles.d when booting a
read-only rootfs. In my tests that is not necessary (anymore?),
something (connman itself?) creates the missing directory.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/recipes-connectivity/connman/connman.inc | 3 ---
1 file changed, 3 deletions(-)
diff --git a/meta/recipes-connectivity/connman/connman.inc
b/meta/recipes-connectivity/connman/connman.inc
index 6c8f405..2b03f9c 100644
--- a/meta/recipes-connectivity/connman/connman.inc
+++ b/meta/recipes-connectivity/connman/connman.inc
@@ -97,9 +97,6 @@ do_install_append() {
# For read-only filesystem, do not create links during bootup
if ${@bb.utils.contains('DISTRO_FEATURES','systemd','true','false',d)};
then
-if
${@bb.utils.contains('IMAGE_FEATURES','read-only-rootfs','true','false',d)};
then
-echo "d/var/run/connman- - - -" >
${D}${sysconfdir}/tmpfiles.d/connman_resolvconf.conf
-fi
ln -sf ../run/connman/resolv.conf ${D}${sysconfdir}/resolv-conf.connman
fi
}
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/2] fixes for read-only rootfs
When investigating in refkit why /etc/machine-id was getting permanently modified in a rootfs despite read-only-rootfs in IMAGE_FEATURES, I noticed two things: - the boot parameters did not include "ro", the initramfs thus mounted the rootfs read/write, and systemd then wrote /etc/machine-id before mounting read-only due to the options in /etc/fstab; this explains the issue I was seeing - one bogus check for read-only-rootfs crept into connman.inc Let's make sure that "ro" gets added to the boot parameters to avoid the issue. I prefer to keep related functionality together, so I decided to put that into rootfs-postcommands.bbclass next to the fstab change. It could also go into image.bbclass itself. The special case in connman.inc doesn't seem to be needed, therefore I propose to remove it without trying to find a different solution. Patrick Ohly (2): connman.inc: do not check IMAGE_FEATURES rootfs-postcommands.bbclass: ensure that rootfs gets mounted ro meta/classes/rootfs-postcommands.bbclass | 8 meta/recipes-connectivity/connman/connman.inc | 3 --- 2 files changed, 8 insertions(+), 3 deletions(-) base-commit: 3b413a80578caacd9a7f405f3c51a3921d78a60d -- git-series 0.9.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] read-only rootfs + boot parameters
Hello!
I noticed that APPEND is left unchanged when adding "read-only-rootfs"
to IMAGE_FEATURES. The effect is (at least in refkit, which uses an
initramfs based on initramfs-framework) that the rootfs initially gets
mounted read/write and then gets remounted by systemd as read-only -
but only after writing to /etc/machine-id.
Is that the desired behavior or an oversight?
To me, read-only means read-only, with no exceptions. The obvious fix
is:
APPEND_append = "${@ bb.utils.contains('IMAGE_FEATURES', 'read-only-rootfs', '
ro', '', d)} "
I'm just not sure whether that should go into image.bbclass or
elsewhere.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/2] connman: correct the systemd boot in read only rootfs
On Mon, 2017-06-12 at 18:10 +0300, Maxin B. John wrote:
> connman fails to start in systemd based read-only images while
> creating links:
>
> Jun 08 12:53:56 qemux86-64 systemd[1]: Starting Create Volatile Files
> and Directories...
> Jun 08 12:53:56 qemux86-64 systemd-tmpfiles[366]:
> [[0;1;31msymlink(/var/run/connman/resolv.conf, /etc/resolv.conf)
> failed:
> Read-only file system[[0m
>
> Fix this failure and make connman co-exist with systemd-resolved.
>
> Signed-off-by: Maxin B. John <maxin.j...@intel.com>
> ---
> meta/recipes-connectivity/connman/connman.inc | 15 ++-
> ...vice-stop-systemd-resolved-when-we-use-co.patch | 29
> ++
> meta/recipes-connectivity/connman/connman_1.34.bb | 1 +
> 3 files changed, 44 insertions(+), 1 deletion(-)
> create mode 100644 meta/recipes-connectivity/connman/connman/0001-
> connman.service-stop-systemd-resolved-when-we-use-co.patch
>
> diff --git a/meta/recipes-connectivity/connman/connman.inc
> b/meta/recipes-connectivity/connman/connman.inc
> index cc2d469..ab18f2f 100644
> --- a/meta/recipes-connectivity/connman/connman.inc
> +++ b/meta/recipes-connectivity/connman/connman.inc
> @@ -13,7 +13,7 @@ LICENSE = "GPLv2"
> LIC_FILES_CHKSUM =
> "file://COPYING;md5=12f884d2ae1ff87c09e5b7ccc2c4ca7e \
> file://src/main.c;beginline=1;endline=20;md5=486
> a279a6ab0c8d152bcda3a5b5edc36"
>
> -inherit autotools pkgconfig systemd update-rc.d bluetooth
> +inherit autotools pkgconfig systemd update-rc.d bluetooth update-
> alternatives
>
> DEPENDS = "dbus glib-2.0 ppp readline"
>
> @@ -69,6 +69,11 @@ SYSTEMD_SERVICE_${PN} = "connman.service"
> SYSTEMD_SERVICE_${PN}-vpn = "connman-vpn.service"
> SYSTEMD_SERVICE_${PN}-wait-online = "connman-wait-online.service"
>
> +ALTERNATIVE_PRIORITY = "100"
> +ALTERNATIVE_${PN} ="resolv-conf"
> +ALTERNATIVE_TARGET[resolv-conf] = "${sysconfdir}/resolv-
> conf.connman"
> +ALTERNATIVE_LINK_NAME[resolv-conf] = "${sysconfdir}/resolv.conf"
> +
> do_install_append() {
> if ${@bb.utils.contains('DISTRO_FEATURES','sysvinit','true',
> 'false',d)}; then
> install -d ${D}${sysconfdir}/init.d
> @@ -89,6 +94,14 @@ do_install_append() {
> # Automake 1.12 won't install empty directories, but we need
> the
> # plugins directory to be present for ownership
> mkdir -p ${D}${libdir}/connman/plugins
> +
> +# For read-only filesystem, do not create links during bootup
> +if ${@bb.utils.contains('DISTRO_FEATURES','systemd','true','fals
> e',d)}; then
> +if ${@bb.utils.contains('IMAGE_FEATURES','re
> ad-only-rootfs','true','false',d)}; then
> +echo "d/var/run/connman- - - -" >
> ${D}${sysconfdir}/tmpfiles.d/connman_resolvconf.conf
> +fi
> +ln -sf ../run/connman/resolv.conf ${D}${sysconfdir}/resolv-
> conf.connman
> +fi
> }
This check for 'IMAGE_FEATURES' is bogus: that's a per-image recipe
variable, which can't be assumed to be set consistently for all images
in the base configuration and therefore the connman recipe can't depend
on it. The effect is that the tmpfiles.d entry doesn't get created
when setting IMAGE_FEATURES only for some images.
It still works for me (refkit, based on OE-core Rocko at the moment).
Something has created /var/run/connman (perhaps connman itself?) and
the resolv.conf inside it, so /etc/resolv.conf -> /etc/resolv-
conf.connman -> ../run/connman/resolv.conf = /run/connman/resolv.conf
exists.
But the bogus lines should be removed nonetheless, because it causes
the connman recipe to depend on IMAGE_FEATURES.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] native CA cert bundles (was: Re: [PATCH 3/3] cve-check-tool: Use CA cert bundle in correct sysroot)
On Tue, 2017-11-21 at 10:06 -0200, Otavio Salvador wrote: > On Tue, Nov 21, 2017 at 6:04 AM, Patrick Ohly <patrick.o...@intel.com > > wrote: > > On Thu, 2017-02-09 at 21:38 +0200, Jussi Kukkonen wrote: > > There is https://bugzilla.yoctoproject.org/show_bug.cgi?id=9883 > > open > > about some aspect of this, but it doesn't actually address the > > underlying question about what the right behavior should be. It's > > based > > on the assumption that libcurl-native should always use ca- > > certificates-native. > > > > Thoughts anyone? > > I agree it should use ca-certificates-native for all native; it > allows for self-signed internal certificates to be added for internal > development. But that's not what bitbake itself uses. Are you saying that bitbake fetchers etc. should also use whatever certificates are configured for ca-certificates-native? That leads to a chicken-and-egg problem. A solution where custom certificates need to be configured in two different places (system for bitbake, ca-certificates-native for some other tools) sounds sub-optimal to me. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] native CA cert bundles (was: Re: [PATCH 3/3] cve-check-tool: Use CA cert bundle in correct sysroot)
On Thu, 2017-02-09 at 21:38 +0200, Jussi Kukkonen wrote:
> Native libcurl looks for CA certs in the wrong place by
> default.
> * Add patch that allows overriding the default CA certificate
> location. Patch is originally from meta-security-isafw.
> * Use the new --cacert to set the correct CA bundle path
>
> Signed-off-by: Jussi Kukkonen <jussi.kukko...@intel.com>
> ---
> .../cve-check-tool/cve-check-tool_5.6.4.bb | 4 +-
> ...ow-overriding-default-CA-certificate-file.patch | 215
> +
> 2 files changed, 218 insertions(+), 1 deletion(-)
> create mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-
> curl-allow-overriding-default-CA-certificate-file.patch
>
> diff --git a/meta/recipes-devtools/cve-check-tool/cve-check-
> tool_5.6.4.bb b/meta/recipes-devtools/cve-check-tool/cve-check-
> tool_5.6.4.bb
> index c78af67..fcd3182 100644
> --- a/meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb
> +++ b/meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb
> @@ -9,6 +9,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=e8c1458438ead3
> c34974bc0be3a03ed6"
> SRC_URI = "https://github.com/ikeydoherty/${BPN}/releases/download/v
> ${PV}/${BP}.tar.xz \
> file://check-for-malloc_trim-before-using-it.patch \
> file://0001-print-progress-in-percent-when-downloading-
> CVE-db.patch \
> + file://0001-curl-allow-overriding-default-CA-certificate-
> file.patch \
> "
>
> SRC_URI[md5sum] = "c5f4247140fc9be3bf41491d31a34155"
> @@ -39,7 +40,8 @@ do_populate_cve_db() {
> [ -z "${cve_file}" ] && cve_file="${TMPDIR}/cve_check"
>
> bbdebug 2 "Updating cve-check-tool database located in $cve_dir"
> -if cve-check-update -d "$cve_dir" ; then
> +# --cacert works around curl-native not finding the CA bundle
> +if cve-check-update --cacert ${sysconfdir}/ssl/certs/ca-
> certificates.crt -d "$cve_dir" ; then
I went back to this patch to see how the problem was solved, because I
am facing it again elsewhere. Now that I think about it again, I'm
starting to wonder which SSL certificates the native tools really
should trust. Tools like Python or wget are taken from the host, which
means they use the host defaults for SSL. That native tools built with
bitbake then try to use ca-certificates-native looks inconsistent to
me.
There is https://bugzilla.yoctoproject.org/show_bug.cgi?id=9883 open
about some aspect of this, but it doesn't actually address the
underlying question about what the right behavior should be. It's based
on the assumption that libcurl-native should always use ca-
certificates-native.
Thoughts anyone?
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2 1/2] scripts/oe-selftest: oe-selftest-internal wrapper scripts that isolates execution
On Tue, 2017-11-14 at 11:42 -0600, Leonardo Sandoval wrote: > On Tue, 14 Nov 2017 17:09:28 +0100 > Patrick Ohly <patrick.o...@intel.com> wrote: > > Is oe-selftest-internal even going to be in the default PATH? It > > probably shouldn't be, once it truly becomes an implementation > > detail. > > where can we placed it besides the scripts folder? scripts/lib/ perhaps, then call it from oe-selftest with an absolute path? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2 1/2] scripts/oe-selftest: oe-selftest-internal wrapper scripts that isolates execution
On Tue, 2017-11-14 at 10:09 -0600, Leonardo Sandoval wrote: > On Tue, 14 Nov 2017 09:24:30 +0100 > Patrick Ohly <patrick.o...@intel.com> wrote: > > > On Mon, 2017-11-13 at 10:17 -0800, > > leonardo.sandoval.gonza...@linux.intel.com wrote: > > > From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.c > > > om> > > > > > > The main idea is to isolate the oe-selftest execution so neither > > > the > > > current build dir nor the configuration data is touch/polluted. > > > This > > > approach uses a wrapper script (which is the one presented on > > > this > > > commit) which creates a unique directory and inside it copies all > > > scripts and metadata, re-initializes the enviroment (re-sources > > > oe- > > > init-build-env) and finally launches the oe-selftest-internal > > > (which > > > used to be oe-selftest) command, passing command arguments to the > > > latter. > > > > This mode of operation may or may not be desirable. Can it be made > > optional? > > good idea. However, you can call the oe-selftest-internal for the > this purpose. While doable, it feels wrong to rename something as "internal" and then still expect people to call it directly. A command line parameter for the new oe-selftest which controls this behavior and just calls oe-selftest-internal directly when requested feels cleaner and more discoverable. Is oe-selftest-internal even going to be in the default PATH? It probably shouldn't be, once it truly becomes an implementation detail. > > In refkit CI testing, several selftests run tests on build > > artifacts > > (primarily the images) produced during the previous build and only > > build them if needed, i.e. they run "bitbake some-image" and that > > is > > usually fast because the image already exists. Reusing sstate and > > download dir also is important for speed. > > oe-selftest creates a new build/conf folder, with the same conf files > as the ones present in your current build/conf, so this means that > you can set there DL_DIR and SSTATE_DIR (for example) on your main > local.conf and these will be used by oe-selftest, effectively reusing > these folders. Instead of leaving it to the developer to figure this out, should the oe-selftest have --[no-]reuse-download and --[no]-reuse-sstate options? > > -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] oeqa/selftest/runtime_test: fix postinst_rootfs_and_boot
On Mon, 2017-11-06 at 17:55 +, Ross Burton wrote:
> testcommand = 'ls /etc/' + fileboot_name
Can't this be removed?
> with runqemu('core-image-minimal') as qemu:
> - ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog)
> - status, output = ssh.run(testcommand)
> + status, output = qemu.run_serial("-f /etc/" +
> fileboot_name)
Did you mean "test -f"?
> self.assertEqual(status, 0, 'File %s was not created
> at first boot (%s)' % (fileboot_name, output))
run_serial has the quirk that status == 1 on *success*. Yes, weird.
The test probably passed because it was testing for failure, and the
missing "test" ensured that the command failed.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] systemd: create wheel sysuser group offline
On Mon, 2017-11-13 at 12:18 -0800, Andre McCurdy wrote: > On Mon, Nov 13, 2017 at 6:48 AM, Patrick Ohly <patrick.o...@intel.com > > wrote: > > On Thu, 2017-11-09 at 21:54 -0800, Andre McCurdy wrote: > > > The default systemd-tmpfiles config file expects to be able to > > > create > > > files etc belonging to the wheel system group. Currently the > > > wheel > > > group is created at run time by systemd-sysusers, but that > > > doesn't > > > happen if systemd-sysusers is disabled (as it currently is by > > > default > > > when building with musl libc). > > > > Isn't this something that the systemd_create_users rootfs > > postprocess > > command in rootfs-postcommands.bbclass already takes care of? > > systemd_create_users() does a build time pass over the > systemd-sysusers config files, but those files are not installed if > systemd is configured without sysusers support. I didn't know that this is optional. To me it sounds like an invalid (or let's say, unexpected) configuration to install tmpfiles config files but not the sysusers files, because as you said, the tmpfiles may depend on the sysusers. Anyway, I just wanted to know because I was wondering whether it is really necessary to duplicate the user creation information in the systemd recipe. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] systemd: create wheel sysuser group offline
On Thu, 2017-11-09 at 21:54 -0800, Andre McCurdy wrote: > The default systemd-tmpfiles config file expects to be able to create > files etc belonging to the wheel system group. Currently the wheel > group is created at run time by systemd-sysusers, but that doesn't > happen if systemd-sysusers is disabled (as it currently is by default > when building with musl libc). Isn't this something that the systemd_create_users rootfs postprocess command in rootfs-postcommands.bbclass already takes care of? I know that it is has issues (https://bugzilla.yoctoproject.org/show_bu g.cgi?id=9789), but it should at least create the wheel group. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/7] oeqa/runqemu: Only show stdout/stderr upon test failure
On Thu, 2017-11-09 at 11:55 +, Richard Purdie wrote: > + # We only want to print runqemu stdout/stderr if there is a test > case failure > + buffer = True Does the value matter? The other code only seems to check for the presence of the "buffer" attribute. Changing this to "buffer = False" and still get buffering would be surprising. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] qemu: upgrade to 2.10.1
On Thu, 2017-10-26 at 09:37 -0500, Leonardo Sandoval wrote: > I used devtool upgrade and the only hunk I need to remove was > thisone, the rest were applied cleanly. The rest was applied *silently*, but not *cleanly*. The lesson here is basically that developers shouldn't blindly trust the tools. If something is odd, investigate. What was odd in this case is that it looked like a patch was partially merged. Why would anyone do that? And indeed, it was merged completely. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] qemu: upgrade to 2.10.1
On Thu, 2017-10-19 at 13:10 -0700, leonardo.sandoval.gonza...@linux.intel.com wrote: > From: Leonardo Sandoval <leonardo.sandoval.gonza...@linux.intel.com> > > All CVE patches removed because these are already integrated in > 2.10.1. ... > meta/recipes-devtools/qemu/qemu/glibc-2.25.patch | 14 - diff --git a/meta/recipes-devtools/qemu/qemu/glibc-2.25.patch > b/meta/recipes-devtools/qemu/qemu/glibc-2.25.patch > index a6908bdbf9..25569449e4 100644 > --- a/meta/recipes-devtools/qemu/qemu/glibc- > 2.25.patch > +++ b/meta/recipes-devtools/qemu/qemu/glibc-2.25.patch > @@ -72,17 +72,3 @@ diff -uNr qemu-2.8.0.orig/configure qemu- > 2.8.0/configure > # Hold two types of flag: > # CONFIG_THREAD_SETNAME_BYTHREAD - we've got a way of setting > the name on > # a thread we have a handle to > -diff -uNr qemu-2.8.0.orig/include/sysemu/os-posix.h qemu- > 2.8.0/include/sysemu/os-posix.h > qemu-2.8.0.orig/include/sysemu/os-posix.h2016-12-20 > 21:16:48.0 +0100 > -+++ qemu-2.8.0/include/sysemu/os-posix.h 2017-02-21 > 19:07:18.009090381 +0100 > -@@ -34,6 +34,10 @@ > - #include > - #include > - > -+#ifdef CONFIG_SYSMACROS > -+#include > -+#endif > -+ > - void os_set_line_buffering(void); > - void os_set_proc_name(const char *s); > - void os_setup_signal_handling(void); Instead of removing just this hunk from the glibc-2.25.patch, please remove the entire patch. It is already in 2.10.0. I was about to send a patch doing just that when I saw your version update. Here's the commit message for my patch: The patch is already present in the upstream 2.10.0. Patching during a build succeeds by adding the same hunks again to configure (which seems to cause no problems during build) and skipping the one which it detects as already applied, but "devtool modify qemu-native" is more picky: ERROR: Applying 'glibc-2.25.patch' failed: checking file configure Hunk #1 succeeded at 4986 with fuzz 2 (offset 259 lines). checking file configure Hunk #1 succeeded at 6047 with fuzz 1 (offset 352 lines). checking file include/sysemu/os-posix.h Reversed (or previously applied) patch detected! Assume -R? [n] Apply anyway? [n] Skipping patch. 1 out of 1 hunk ignored ERROR: Function failed: patch_do_patch ERROR: Logfile of failure stored in: .../devtooltmp-6_22hcm3/temp/log.do_patch.31897 NOTE: Tasks Summary: Attempted 3 tasks of which 0 didn't need to be rerun and 1 failed. ERROR: Extracting source for qemu-native failed -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] useradd-staticids: explain how to fix the the problem
When a distro uses useradd-staticids.bbclass and some developer
unfamiliar with the static ID mechanism tries to add a recipe which
needs new IDs, the resulting error or warning is typically not
something that the developer will understand.
Even experienced developers do not get enough information. They first
must find out whether the missing ID is for a system user or group,
then locate the file(s) in which the ID could be added. Both of this
is now part of the message:
ERROR: .../meta/recipes-extended/cronie/cronie_1.5.1.bb: cronie -
cronie: system groupname crontab does not have a static ID defined.
Add crontab to one of these files: /.../conf/distro/include/my-distro-group
The case that no file was found is also handled:
ERROR: .../meta/recipes-extended/cronie/cronie_1.5.1.bb: cronie -
cronie: system groupname crontab does not have a static ID defined.
USERADD_GID_TABLES file(s) not found in BBPATH: files/group
It would be nice if the error message could also list the range in
which a new ID needs to be allocated, but /etc/login.defs isn't
available at the time of creating the message, so that part is still
something that a developer needs to know.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/classes/useradd-staticids.bbclass | 62 ---
1 file changed, 29 insertions(+), 33 deletions(-)
diff --git a/meta/classes/useradd-staticids.bbclass
b/meta/classes/useradd-staticids.bbclass
index 3d0bc09..589a99f 100644
--- a/meta/classes/useradd-staticids.bbclass
+++ b/meta/classes/useradd-staticids.bbclass
@@ -38,10 +38,14 @@ def update_useradd_static_config(d):
return id_table
-def handle_missing_id(id, type, pkg):
+def handle_missing_id(id, type, pkg, files, var, value):
# For backwards compatibility we accept "1" in addition to "error"
error_dynamic = d.getVar('USERADD_ERROR_DYNAMIC')
msg = "%s - %s: %sname %s does not have a static ID defined." %
(d.getVar('PN'), pkg, type, id)
+if files:
+msg += " Add %s to one of these files: %s" % (id, files)
+else:
+msg += " %s file(s) not found in BBPATH: %s" % (var, value)
if error_dynamic == 'error' or error_dynamic == '1':
raise NotImplementedError(msg)
elif error_dynamic == 'warn':
@@ -49,23 +53,24 @@ def update_useradd_static_config(d):
elif error_dynamic == 'skip':
raise bb.parse.SkipRecipe(msg)
+# Return a list of configuration files based on either the default
+# files/group or the contents of USERADD_GID_TABLES, resp.
+# files/passwd for USERADD_UID_TABLES.
+# Paths are resolved via BBPATH.
+def get_table_list(d, var, default):
+files = []
+bbpath = d.getVar('BBPATH', True)
+tables = d.getVar(var, True)
+if not tables:
+tables = default
+for conf_file in tables.split():
+files.append(bb.utils.which(bbpath, conf_file))
+return (' '.join(files), var, default)
+
# We parse and rewrite the useradd components
def rewrite_useradd(params, is_pkg):
parser = oe.useradd.build_useradd_parser()
-# Return a list of configuration files based on either the default
-# files/passwd or the contents of USERADD_UID_TABLES
-# paths are resolved via BBPATH
-def get_passwd_list(d):
-str = ""
-bbpath = d.getVar('BBPATH')
-passwd_tables = d.getVar('USERADD_UID_TABLES')
-if not passwd_tables:
-passwd_tables = 'files/passwd'
-for conf_file in passwd_tables.split():
-str += " %s" % bb.utils.which(bbpath, conf_file)
-return str
-
newparams = []
users = None
for param in oe.useradd.split_commands(params):
@@ -86,10 +91,12 @@ def update_useradd_static_config(d):
# all new users get the default ('*' which prevents login) until
the user is
# specifically configured by the system admin.
if not users:
-users = merge_files(get_passwd_list(d), 7)
+files, table_var, table_value = get_table_list(d,
'USERADD_UID_TABLES', 'files/passwd')
+users = merge_files(files, 7)
+type = 'system user' if uaargs.system else 'normal user'
if uaargs.LOGIN not in users:
-handle_missing_id(uaargs.LOGIN, 'user', pkg)
+handle_missing_id(uaargs.LOGIN, type, pkg, files, table_var,
table_value)
newparams.append(param)
continue
@@ -147,7 +154,7 @@ def update_useradd_static_config(d):
# Should be an error if a specific option is set...
if not uaargs.uid or not uaargs.uid.isdigit() or not uaargs.gid:
- handle_missing_id(uaargs.LOGIN, 'user', pkg)
+
[OE-core] [PATCH 0/2] enhance useradd-staticids
I recently tried to enable useradd-staticids.bbclass in a distro which includes a large portion of meta-oe. This was a rather arduous task because plenty of IDs need to be defined for recipes which are part of the world build but not actually used in the distro. The error or warning messages also aren't particularly helpful. These two patches address both problems. However, long term I think it would be worthwhile to rethink the entire current approach. At the moment, useradd-staticids.bbclass rewrites the USER/GROUPADD parameters at parse time for all future usages of the parameters, including setting up local sysroots. This has the downside that changing IDs later on leads to warnings or errors about sysroots containing the old IDs. I think static IDs are only needed for the preinst scripts in packages for rootfs and target installation, and nowhere else. Therefore IMHO it would be better to inject the static IDs only into those scripts. Warnings and errors then can be triggered as part of installing the package, if desired. Then it becomes possible to build recipes where some packages create users or groups without actually defining static IDs, as long as those packages then don't get used. Triggering a stricter error already during parsing or build time would still be possible. I'm just not sure how useful that is - perhaps to catch potential problems without actually having to install the packages. Patrick Ohly (2): useradd-staticids: skip recipes without static IDs useradd-staticids: explain how to fix the the problem meta/classes/useradd-staticids.bbclass | 74 +-- 1 file changed, 37 insertions(+), 37 deletions(-) base-commit: 39ffa0f3779305c5e8ef86fe4572e961c5912021 -- git-series 0.9.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] useradd-staticids: skip recipes without static IDs
When enabling useradd-staticids.bbclass, one has to define static IDs
for all recipes in a world build, otherwise those without static IDs
generate parse errors or warnings, depending on USERADD_ERROR_DYNAMIC.
Defining unused IDs is a lot of work and clutters the passwd/group
file of a distro.
Distros which want to avoid this can now set USERADD_ERROR_DYNAMIC =
"skip" and recipes which would have triggered a message then silently
get disabled. Trying to build them then still shows the error message:
$ bitbake apt
...
ERROR: Nothing PROVIDES 'apt'
ERROR: apt was skipped: apt - apt: username _apt does not have a
static ID defined.
However, some recipes only get selected indirectly through
dependencies. If the recipe providing something that is needed for the
build gets skipped (for example, the nfs-utils recipe providing the
nfs-utils-client package), then the error message is a bit more
obscure:
ERROR: Nothing RPROVIDES 'nfs-utils-client' (but
.../meta/recipes-core/packagegroups/packagegroup-core-nfs.bb RDEPENDS
on or otherwise requires it)
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/classes/useradd-staticids.bbclass | 12
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/meta/classes/useradd-staticids.bbclass
b/meta/classes/useradd-staticids.bbclass
index eb8e59e..3d0bc09 100644
--- a/meta/classes/useradd-staticids.bbclass
+++ b/meta/classes/useradd-staticids.bbclass
@@ -40,10 +40,14 @@ def update_useradd_static_config(d):
def handle_missing_id(id, type, pkg):
# For backwards compatibility we accept "1" in addition to "error"
-if d.getVar('USERADD_ERROR_DYNAMIC') == 'error' or
d.getVar('USERADD_ERROR_DYNAMIC') == '1':
-raise NotImplementedError("%s - %s: %sname %s does not have a
static ID defined. Skipping it." % (d.getVar('PN'), pkg, type, id))
-elif d.getVar('USERADD_ERROR_DYNAMIC') == 'warn':
-bb.warn("%s - %s: %sname %s does not have a static ID defined." %
(d.getVar('PN'), pkg, type, id))
+error_dynamic = d.getVar('USERADD_ERROR_DYNAMIC')
+msg = "%s - %s: %sname %s does not have a static ID defined." %
(d.getVar('PN'), pkg, type, id)
+if error_dynamic == 'error' or error_dynamic == '1':
+raise NotImplementedError(msg)
+elif error_dynamic == 'warn':
+bb.warn(msg)
+elif error_dynamic == 'skip':
+raise bb.parse.SkipRecipe(msg)
# We parse and rewrite the useradd components
def rewrite_useradd(params, is_pkg):
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] libarchive: 3.2.2 -> 3.3.1
On Mon, 2017-04-24 at 09:16 +0800, Huang Qiyu wrote:
> 3) Delete three patches, since they are integrated upstream.
...
> non-recursive-extract-and-list.patch
...
> -Upstream-Status: Submitted
> [https://github.com/libarchive/libarchive/pull/812]
What made you think that this patch had been integrated upstream? It
was submitted, but unfortunately has not been merged yet despite a
favorable review.
Please be a bit more careful in future updates. Removing the patch
causes meta-swupd to break.
I've sent a patch adding back the patch ("libarchive: re-add non-
recursive extract and list support") but it is not clear whether that
can be merged in time for 2.4.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] libarchive: re-add non-recursive extract and list support
This patch is needed for meta-swupd. Without it, some bsdtar
invocations fail with:
bsdtar: Option -n is not permitted in mode -x
The patch was removed in the update to 3.3.1 with the claim that it
had been merged upstream, but that is not the case.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/recipes-extended/libarchive/libarchive/non-recursive-extract-and-list.patch
| 153 -
meta/recipes-extended/libarchive/libarchive_3.3.2.bb
| 1 +-
2 files changed, 154 insertions(+)
create mode 100644
meta/recipes-extended/libarchive/libarchive/non-recursive-extract-and-list.patch
diff --git
a/meta/recipes-extended/libarchive/libarchive/non-recursive-extract-and-list.patch
b/meta/recipes-extended/libarchive/libarchive/non-recursive-extract-and-list.patch
new file mode 100644
index 000..cd7be51
--- /dev/null
+++
b/meta/recipes-extended/libarchive/libarchive/non-recursive-extract-and-list.patch
@@ -0,0 +1,153 @@
+From 47f7566f6829c2b14e21a699916de4998c72 Mon Sep 17 00:00:00 2001
+From: Patrick Ohly <patrick.o...@intel.com>
+Date: Mon, 24 Oct 2016 12:54:48 +0200
+Subject: [PATCH 1/1] non-recursive extract and list
+
+Sometimes it makes sense to extract or list a directory contained in
+an archive without also doing the same for the content of the
+directory, i.e. allowing -n (= --no-recursion) in combination with the
+x and t modes.
+
+bsdtar uses the match functionality in libarchive to track include
+matches. A new libarchive API call
+archive_match_include_directories_recursively() gets introduced to
+influence the matching behavior, with the default behavior as before.
+
+Non-recursive matching can be achieved by anchoring the path match at
+both start and end. Asking for a directory which itself isn't in the
+archive when in non-recursive mode is an error and handled by the
+existing mechanism for tracking unused inclusion entries.
+
+Upstream-Status: Submitted [https://github.com/libarchive/libarchive/pull/812]
+
+Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
+
+---
+ libarchive/archive.h | 2 ++
+ libarchive/archive_match.c | 30 +-
+ tar/bsdtar.1 | 3 +--
+ tar/bsdtar.c | 12 ++--
+ 4 files changed, 42 insertions(+), 5 deletions(-)
+
+diff --git a/libarchive/archive.h b/libarchive/archive.h
+index 32710201..59fb4aa6 100644
+--- a/libarchive/archive.h
b/libarchive/archive.h
+@@ -1093,6 +1093,8 @@ __LA_DECL intarchive_match_excluded(struct archive *,
+ */
+ __LA_DECL int archive_match_path_excluded(struct archive *,
+ struct archive_entry *);
++/* Control recursive inclusion of directory content when directory is
included. Default on. */
++__LA_DECL int archive_match_include_directories_recursively(struct archive *,
int _enabled);
+ /* Add exclusion pathname pattern. */
+ __LA_DECL int archive_match_exclude_pattern(struct archive *, const char *);
+ __LA_DECL int archive_match_exclude_pattern_w(struct archive *,
+diff --git a/libarchive/archive_match.c b/libarchive/archive_match.c
+index be72066e..bb6a3407 100644
+--- a/libarchive/archive_match.c
b/libarchive/archive_match.c
+@@ -93,6 +93,9 @@ struct archive_match {
+ /* exclusion/inclusion set flag. */
+ int setflag;
+
++ /* Recursively include directory content? */
++ int recursive_include;
++
+ /*
+* Matching filename patterns.
+*/
+@@ -223,6 +226,7 @@ archive_match_new(void)
+ return (NULL);
+ a->archive.magic = ARCHIVE_MATCH_MAGIC;
+ a->archive.state = ARCHIVE_STATE_NEW;
++ a->recursive_include = 1;
+ match_list_init(&(a->inclusions));
+ match_list_init(&(a->exclusions));
+ __archive_rb_tree_init(&(a->exclusion_tree), _ops_mbs);
+@@ -471,6 +475,28 @@ archive_match_path_excluded(struct archive *_a,
+ }
+
+ /*
++ * When recursive inclusion of directory content is enabled,
++ * an inclusion pattern that matches a directory will also
++ * include everything beneath that directory. Enabled by default.
++ *
++ * For compatibility with GNU tar, exclusion patterns always
++ * match if a subset of the full patch matches (i.e., they are
++ * are not rooted at the beginning of the path) and thus there
++ * is no corresponding non-recursive exclusion mode.
++ */
++int
++archive_match_include_directories_recursively(struct archive *_a, int
_enabled)
++{
++ struct archive_match *a;
++
++ archive_check_magic(_a, ARCHIVE_MATCH_MAGIC,
++ ARCHIVE_STATE_NEW, "archive_match_include_directories_recursively");
++ a = (struct archive_match *)_a;
++ a->recursive_include = _enabled;
++ return (ARCHIVE_OK);
++}
++
++/*
+ * Utility functions to get statistic information for inclusion patterns.
+ */
+ int
Re: [OE-core] [PATCH V2 1/1] ovmf: fix do_compile error when len(tmp)=410
On Mon, 2017-09-11 at 21:20 -0400, Dengke Du wrote: > diff --git a/meta/recipes-core/ovmf/ovmf/0004-ovmf-enable-long-path- > file.patch b/meta/recipes-core/ovmf/ovmf/0004-ovmf-enable-long-path- > file.patch > new file mode 100644 > index 000..9cc85ad > --- /dev/null > +++ b/meta/recipes-core/ovmf/ovmf/0004-ovmf-enable-long-path- > file.patch > @@ -0,0 +1,28 @@ > +From 032fc6b1f7691bd537fd2a6bd13821fcf3c45e64 Mon Sep 17 00:00:00 > 2001 > +From: Dengke Du <dengke...@windriver.com> > +Date: Mon, 11 Sep 2017 02:21:55 -0400 > +Subject: [PATCH] ovmf: enable long path file > + > +Upstream-Status: Pending Pending which action? I.e. what's the next step? To me this patch looks more like local customization to fit into path lengths used by the OE build system, so "Inappropriate" might be more reasonable. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] hidraw support in the kernel(s)
Hello! I am currently trying to get fwupd [1] working. For updating a Lenovo Unifying Dongle (the USB thingie which connects to wireless mice and keyboards), the kernel has to support hidraw, aka CONFIG_HIDRAW, otherwise fwupd does not find the dongle. http://www.signal11.us/oss/udev/ is an article that seems to explain the underlying API used by fwupd. Other apps might also need it. The recommendation is to enable it unless there are reasons not to do so: http://cateee.net/lkddb/web-lkddb/HIDRAW.html So is there a reason why CONFIG_HIDRAW is disabled in linux-intel and (presumably, I haven't checked) linux-yocto for intel-corei7-64? What would be the right way of enabling it? On by default as part of some existing kernel feature perhaps? Or a new feature? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] initramfs-framework: split setup-live and install-efi into separate recipes
On Mon, 2017-09-04 at 09:46 -0300, Otavio Salvador wrote: > On Mon, Sep 4, 2017 at 3:35 AM, Patrick Ohly <patrick.o...@intel.com> > wrote: > > On Sat, 2017-09-02 at 15:17 -0300, Otavio Salvador wrote: > > > On Fri, Sep 1, 2017 at 9:04 PM, California Sullivan > > > <california.l.sulli...@intel.com> wrote: > > > > Having these the initramfs-framework recipe > > > > forced initramfs-framework > > > > users to build several tools they didn't need, and made it more > > > > difficult to declare the recipe as allarch. > > > > > > > > Fixes [YOCTO #12024]. > > > > > > > > Signed-off-by: California Sullivan <california.l.sullivan@intel > > > > .com > > > > > > > > > > > The patch should be based on the allarch patch I sent as it adds > > > the > > > dependencies on the whitelist. So you can also set them as > > > allarch. > > > > I would prefer to keep initramfs-framework-setup-live/install-efi > > as > > arch-specific and only make initramfs-framework itself allarch. > > It's > > just a gut feeling about the (minor) advantage of not having to > > rebuild > > these two small recipes vs. the additional complexity of the > > layer.conf > > and having to keep that up-to-date. > > The recipe is a shell script and there is no reason to not make it > allarch. The need to keep layer.conf up to date is minor as it is > going to be need only when new dependencies are added. Not ofthen... The need for that was already missed once, in the initial patch which introduced "inherit allarch". I find it likely that whoever changes the scripts in the future will also miss it, causing additional work for those who then have to deal with the auto builder errors. At least add a comment to the DEPENDS which explains that layer.conf needs to be edited together with the recipe(s). But that's just my opinion. If Richard and/or Ross aren't worried about a too complex OE-core layer.conf, then that's fine with me too. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] initramfs-framework: split setup-live and install-efi into separate recipes
On Sat, 2017-09-02 at 15:17 -0300, Otavio Salvador wrote: > On Fri, Sep 1, 2017 at 9:04 PM, California Sullivan > <california.l.sulli...@intel.com> wrote: > > Having these the initramfs-framework recipe > > forced initramfs-framework > > users to build several tools they didn't need, and made it more > > difficult to declare the recipe as allarch. > > > > Fixes [YOCTO #12024]. > > > > Signed-off-by: California Sullivan <california.l.sulli...@intel.com > > > > > The patch should be based on the allarch patch I sent as it adds the > dependencies on the whitelist. So you can also set them as allarch. I would prefer to keep initramfs-framework-setup-live/install-efi as arch-specific and only make initramfs-framework itself allarch. It's just a gut feeling about the (minor) advantage of not having to rebuild these two small recipes vs. the additional complexity of the layer.conf and having to keep that up-to-date. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v4] initramfs-framework: Change recipe to be allarch
On Wed, 2017-08-30 at 11:39 +0200, Patrick Ohly wrote:
> As I said before ("Re: [OE-core] [PATCH v6 1/1] initramfs-framework:
> module to support boot live image" from July 31st), I consider it a
> mistake that support for live boot with all its dependencies was
> added directly to initramfs-framework.bb.
>
> I was in a hurry at the time and therefore didn't file a bug, but I
> can also do that.
See https://bugzilla.yoctoproject.org/show_bug.cgi?id=12024
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v4] initramfs-framework: Change recipe to be allarch
On Wed, 2017-08-30 at 09:46 -0300, Otavio Salvador wrote: > On Wed, Aug 30, 2017 at 6:39 AM, Patrick Ohly <patrick.o...@intel.co > wrote: > > Let's not dig us deeper into this hole and instead split out the > > live > > boot module into its own, arch-specific recipe. > > > > Then initramfs-framework can become allarch without having to make > > layer.conf more complicated. > > I think this can be in two steps. Splitting it out makes sense as it > avoids the build of useless packages if someone is not using the live > modules but it can be another patch. I'm not sure in which order you want to get it done: first split out, then update (i.e. remove the layer.conf change) your patch and merge that, or apply the v4 revision of your patch and later clean up (including removal of the layer.conf change)? Personally I'm in favor of first splitting out the modules. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v4] initramfs-framework: Change recipe to be allarch
On Tue, 2017-08-29 at 17:43 -0300, Otavio Salvador wrote:
> There is no COMPATIBLE_HOST in the recipe neither it makes sense for
> this to be machine specific.
>
> Possibly, initramfs-framework's based modules may be machine specific
> but if there is the case they can just RDEPENDS on
> initramfs-framework-base and provide the specific module as another
> recipe.
>
> Signed-off-by: Otavio Salvador <ota...@ossystems.com.br>
> ---
>
> Changes in v4:
> - Drop INHIBIT_DEFAULT_DEPS as allarch defines it
>
> Changes in v3:
> - Add the new dependencies to the hash whitelist
As I said before ("Re: [OE-core] [PATCH v6 1/1] initramfs-framework:
module to support boot live image" from July 31st), I consider it a
mistake that support for live boot with all its dependencies was added
directly to initramfs-framework.bb.
I was in a hurry at the time and therefore didn't file a bug, but I can
also do that.
> diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
> index 38bec33197..04aa730160 100644
> --- a/meta/conf/layer.conf
> +++ b/meta/conf/layer.conf
> @@ -50,8 +50,12 @@ SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS += " \
> docbook-xsl-stylesheets->perl \
> ca-certificates->openssl \
> initramfs-framework->${VIRTUAL-RUNTIME_base-utils} \
> - initramfs-framework->systemd \
> + initramfs-framework->dosfstools \
> + initramfs-framework->e2fsprogs \
> initramfs-framework->eudev \
> + initramfs-framework->parted \
> + initramfs-framework->systemd \
> + initramfs-framework->util-linux \
Let's not dig us deeper into this hole and instead split out the live
boot module into its own, arch-specific recipe.
Then initramfs-framework can become allarch without having to make
layer.conf more complicated.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] bitbake.conf: Add run-parts to HOSTTOOLS
On Thu, 2017-08-24 at 10:19 +0100, André Draszik wrote: > From: André Draszik <adras...@tycoint.com> > > ca-certificates runs a postinst task, update-ca- > certificates, > which ultimately wants to execute run-parts. > > Signed-off-by: André Draszik <adras...@tycoint.com> > --- > meta/conf/bitbake.conf | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > index 334ba2361f..8011689118 100644 > --- a/meta/conf/bitbake.conf > +++ b/meta/conf/bitbake.conf > @@ -469,7 +469,7 @@ HOSTTOOLS += " \ > fgrep file find flock g++ gawk gcc getconf getopt git grep > gunzip gzip \ > head hostname install ld ldd ln ls make makeinfo md5sum mkdir > mknod \ > mktemp mv nm objcopy objdump od patch perl pod2man pr printf pwd > python python2 \ > -python2.7 python3 ranlib readelf readlink rm rmdir rpcgen sed sh > sha256sum \ > +python2.7 python3 ranlib readelf readlink rm rmdir rpcgen run- > parts sed sh sha256sum \ > sleep sort split stat strings strip tail tar tee test touch tr > true uname \ > uniq wc wget which xargs \ > " Is run-parts guaranteed to be available on the host? IMHO it would be better to ensure that run-parts is in recipe-sysroot- native when installing ca-certificates. PACKAGE_WRITE_DEPS can be set in ca-certificates.bb for that. I'm just not sure about what provides run-parts in OE. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] bitbake.conf: fix ineffective include conf/target/${TARGET_SYS}.conf
On Wed, 2017-02-22 at 02:21 -0800, Andre McCurdy wrote:
> TARGET_SYS is defined in terms of TARGET_ARCH, so it's not valid
> until after TUNE_ARCH has been set by the machine config. The
> original order of includes resulted in an attempt to include
> non-existent files such as:
>
> conf/target/INVALID-oe-linux.conf
>
> Signed-off-by: Andre McCurdy <armccu...@gmail.com>
> ---
> meta/conf/bitbake.conf | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index e421650..18d1cfb 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -705,9 +705,9 @@ include conf/auto.conf
> include conf/local.conf
> require conf/multiconfig/${BB_CURRENT_MC}.conf
> include conf/build/${BUILD_SYS}.conf
> -include conf/target/${TARGET_SYS}.conf
> include conf/machine/${MACHINE}.conf
> include conf/machine-sdk/${SDKMACHINE}.conf
> +include conf/target/${TARGET_SYS}.conf
> include conf/distro/${DISTRO}.conf
I think conf/target/${TARGET_SYS}.conf must be included after
${DISTRO}.conf, because TARGET_SYS contains
${TARGET_ARCH}${TARGET_VENDOR} and TARGET_VENDOR gets changed by a
${DISTRO}.conf like poky.conf.
I also found this issue when writing an automated test that detects
when include file names change while parsing, and I agree that it
should be either fixed or removed.
See the "Yocto Compatible 2.0 support code" mail thread for details. At
that time I had missed that there was already a pending patch for it.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 0/2] Yocto Compatible 2.0 support code
On Fri, 2017-06-09 at 10:12 +0200, Patrick Ohly wrote:
> I also get for all recipes (i.e. the error is in the base
> configuration):
>
> meta/conf/bitbake.conf:752: include/require/inherit
> "conf/target/${TARGET_SYS}.conf" resulted in including
> "conf/target/x86_64-oe-linux.conf" while parsing. Variables effecting the
> parameter changed later such that "conf/target/x86_64-refkit-linux.conf"
> would have been included at the end of the recipe.
>
> None of these two files exist, so it doesn't make a difference. But is
> it really intended that a conf/target/${TARGET_SYS}.conf gets included
> that isn't the one for the final TARGET_SYS? That looks like a genuine
> bug to me.
And Andre McCurdy also came to the same conclusion - see his
"[OE-core] [PATCH] bitbake.conf: fix ineffective include
conf/target/${TARGET_SYS}.conf"
patch. However, that patch IMHO is incomplete, because it doesn't account for
${DISTRO}.conf changing the TARGET_VENDOR part of TARGET_SYS. I'll comment
there, too.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] autoconf-archive: move from meta-oe to OE-core
On Thu, 2017-08-10 at 20:53 -0500, Mark Hatle wrote: > On 8/10/17 6:56 PM, Martin Jansa wrote: > > On Thu, Aug 10, 2017 at 03:34:48PM -0500, Mark Hatle wrote: > > > On 8/10/17 3:18 PM, Martin Jansa wrote: > > > > -2 > > > > > > I agree that autoconf-archive should be in oe-core. But... > > > > No argument about that, it was already merged to oe-core, I was > > only commenting about the gnome-common change included in this > > commit. The meta-oe patch is here: https://patchwork.openembedded.org/series/7995/# > Sorry, I missed the context.. ya if the gnome change is arch or board > specific that is wrong. There should be no reason for that. The patch itself doesn't change anything around that, autoconf-archive already was arch specific. Martin is right, because of that the gnome-common->autoconf-archive dependency can't be done as in the patch above. But what is the right fix? Is autoconf-archive really arch specific or can "inherit allarch" be added to it? If not, can we add it to SIGGEN_EXCLUDERECIPES_ABISAFE to allow the gnome-common->autoconf-archive dependency? That would also prevent rebuilding software when updating autoconf-archive, which may or may not be the right thing to do - I'm undecided myself. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v6 1/1] initramfs-framework: module to support boot live image
On Wed, 2017-07-12 at 12:29 -0700, wei.tee...@intel.com wrote:
> +SUMMARY_initramfs-module-setup-live = "initramfs support for setup
> live"
> +RDEPENDS_initramfs-module-setup-live = "${PN}-base udev-extraconf"
> +FILES_initramfs-module-setup-live = "/init.d/80-setup-live"
Same problem as with install-efi: this RDEPENDS in initramfs-framework
makes building the recipe more complicated than strictly necessary.
Please move setup-live to its own recipe.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/3] initramfs-framework: include install-efi module in recipe for installation
On Sun, 2017-07-23 at 16:51 -0700, wei.tee...@intel.com wrote:
> +
> +SUMMARY_initramfs-module-install-efi = "initramfs support for installation
> option"
> +RDEPENDS_initramfs-module-install-efi = "${PN}-base parted e2fsprogs-mke2fs
> dosfstools util-linux-blkid"
> +FILES_initramfs-module-install-efi = "/init.d/install-efi.sh"
By making the install-efi module a part of initramfs-module that is
enabled unconditionally, parted now needs to be built for the target
even when only the rest of initramfs-module is used.
Can the module please be split out into its own recipe?
Alternatively one could add a PACKAGECONFIG for it, but that's less
flexible.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] autoconf-archive: GPLv3 + autoconf exception
The COPYING file specifies pure GPLv3, not GPLv2 & GPLv3, with the
autoconf exception in COPYING.EXCEPTION. OE-core currently has
GPL-3.0-with-GCC-exception for this in meta/conf/licenses.conf, so
this is used here despite the deprecation note for that license
identifier in https://spdx.org/licenses/GPL-3.0-with-autoconf-exception.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git
a/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
index d77c37d..104dc38 100644
--- a/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
+++ b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
@@ -1,8 +1,9 @@
SUMMARY = "a collection of freely re-usable Autoconf macros"
HOMEPAGE = "http://www.gnu.org/software/autoconf-archive/;
SECTION = "devel"
-LICENSE = "GPLv3"
-LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
+LICENSE = "GPL-3.0-with-autoconf-exception"
+LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
+file://COPYING.EXCEPTION;md5=fdef168ebff3bc2f13664c365a5fb515"
SRC_URI = "${GNU_MIRROR}/${BPN}/${BPN}-${PV}.tar.xz"
SRC_URI[md5sum] = "bf19d4cddce260b3c3e1d51d42509071"
base-commit: da29bc17e4dd748f50b054c5e3afaf8d41bf4077
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 1/2] autoconf-archive: move from meta-oe to OE-core
Having common macros in OE-core that are needed by autotools based
projects makes sense. For example, tpm2.0-tools in meta-measured
depended on meta-oe only because of autoconf-archive.
This is a verbatim copy of the autoconf-archive recipe in
meta-openembedded rev 1cbd1bc1, with just one change: the patch which
disabled the installation of ax_code_coverage.m4 and
ax_check_enable_debug.m4 and the dependency on gnome-common were
removed.
So now autoconf-archive in OE-core provides them. gnome-common in
meta-oe will be changed to not install them and instead depend on
autoconf-archive.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/recipes-devtools/autoconf-archive/autoconf-archive.inc | 13
+
meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb | 13
+
2 files changed, 26 insertions(+)
create mode 100644 meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
create mode 100644
meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
diff --git a/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
b/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
new file mode 100644
index 000..4f63e0f
--- /dev/null
+++ b/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
@@ -0,0 +1,13 @@
+LICENSE = "GPLv3"
+HOMEPAGE = "http://www.gnu.org/software/autoconf-archive/;
+SECTION = "devel"
+
+DEPENDS += "m4-native"
+DEPENDS_class-native = "m4-native gnu-config-native"
+DEPENDS_class-nativesdk = "m4-nativesdk gnu-config-nativesdk"
+
+RDEPENDS_${PN} = "m4 gnu-config"
+
+SRC_URI = "${GNU_MIRROR}/${BPN}/${BPN}-${PV}.tar.xz"
+
+inherit autotools
diff --git
a/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
new file mode 100644
index 000..0a1a771
--- /dev/null
+++ b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
@@ -0,0 +1,13 @@
+require autoconf-archive.inc
+
+
+PARALLEL_MAKE = ""
+
+LICENSE = "GPLv2 & GPLv3"
+LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
+
+SRC_URI[md5sum] = "bf19d4cddce260b3c3e1d51d42509071"
+SRC_URI[sha256sum] =
"e8f2efd235f842bad2f6938bf4a72240a5e5fcd248e8444335e63beb60fabd82"
+
+EXTRA_OECONF += "ac_cv_path_M4=m4"
+BBCLASSEXTEND = "native nativesdk"
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 2/2] autoconf-archive: simplify and fix recipe
The COPYING file specifies pure GPLv3, not GPLv2 & GPLv3, with the
autoconf exception in COPYING.EXCEPTION. OE-core currently has
GPL-3.0-with-GCC-exception for this in meta/conf/licenses.conf, so
this is used here despite the deprecation note for that license
identifier in https://spdx.org/licenses/GPL-3.0-with-autoconf-exception.
All of the explicit dependencies seem unnecessary, and RDEPENDS_${PN}
doesn't do anything for native recipes either, so all of that gets
removed.
It also built fine without the m4 and parallel build workarounds.
There's no need to have a separate .inc file.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/recipes-devtools/autoconf-archive/autoconf-archive.inc | 13
-
meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb | 19
++-
2 files changed, 10 insertions(+), 22 deletions(-)
delete mode 100644 meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
diff --git a/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
b/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
deleted file mode 100644
index 4f63e0f..000
--- a/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
+++ /dev/null
@@ -1,13 +0,0 @@
-LICENSE = "GPLv3"
-HOMEPAGE = "http://www.gnu.org/software/autoconf-archive/;
-SECTION = "devel"
-
-DEPENDS += "m4-native"
-DEPENDS_class-native = "m4-native gnu-config-native"
-DEPENDS_class-nativesdk = "m4-nativesdk gnu-config-nativesdk"
-
-RDEPENDS_${PN} = "m4 gnu-config"
-
-SRC_URI = "${GNU_MIRROR}/${BPN}/${BPN}-${PV}.tar.xz"
-
-inherit autotools
diff --git
a/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
index 0a1a771..104dc38 100644
--- a/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
+++ b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
@@ -1,13 +1,14 @@
-require autoconf-archive.inc
-
-
-PARALLEL_MAKE = ""
-
-LICENSE = "GPLv2 & GPLv3"
-LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
-
+SUMMARY = "a collection of freely re-usable Autoconf macros"
+HOMEPAGE = "http://www.gnu.org/software/autoconf-archive/;
+SECTION = "devel"
+LICENSE = "GPL-3.0-with-autoconf-exception"
+LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
+file://COPYING.EXCEPTION;md5=fdef168ebff3bc2f13664c365a5fb515"
+
+SRC_URI = "${GNU_MIRROR}/${BPN}/${BPN}-${PV}.tar.xz"
SRC_URI[md5sum] = "bf19d4cddce260b3c3e1d51d42509071"
SRC_URI[sha256sum] =
"e8f2efd235f842bad2f6938bf4a72240a5e5fcd248e8444335e63beb60fabd82"
-EXTRA_OECONF += "ac_cv_path_M4=m4"
+inherit autotools
+
BBCLASSEXTEND = "native nativesdk"
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 0/2] move autoconf-archive into OE-core
V2: - removed the gnome-common dependency that was still in V1 - merged the .inc file into the .bb file and cleaned up the recipe V3: - LICENSE = "GPL-3.0-with-autoconf-exception" Patrick Ohly (2): autoconf-archive: move from meta-oe to OE-core autoconf-archive: simplify and fix recipe meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb | 14 ++ 1 file changed, 14 insertions(+) create mode 100644 meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb base-commit: d3a41fbd94462efc8c6f1b55f6fb54001b447c45 -- git-series 0.9.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2 2/2] autoconf-archive: simplify and fix recipe
On Fri, 2017-07-28 at 09:44 -0700, Khem Raj wrote: > On Fri, Jul 28, 2017 at 7:49 AM, Patrick Ohly <patrick.o...@intel.com > > +LICENSE = "GPLv3" > > Perhaps GPL-3.0-with-autoconf-exception would be more accurate here. I wasn't sure whether such a thing existed already, but yes, as it does it could be used here, with COPYING.EXCEPTION added as second LIC_FILES_CHKSUM entry. I didn't want to be that fancy, but I can also do a v3 with that change. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 0/2] move autoconf-archive into OE-core
V2: - removed the gnome-common dependency that was still in V1 - merged the .inc file into the .bb file and cleaned up the recipe Patrick Ohly (2): autoconf-archive: move from meta-oe to OE-core autoconf-archive: simplify and fix recipe meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb | 13 + 1 file changed, 13 insertions(+) create mode 100644 meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb base-commit: d3a41fbd94462efc8c6f1b55f6fb54001b447c45 -- git-series 0.9.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 2/2] autoconf-archive: simplify and fix recipe
The COPYING file specifies pure GPLv3, not GPLv2 & GPLv3. There is
also the COPYING.EXCEPTION file with the autotools exception, which
gets ignored here in the recipe to keep it simpler.
All of the explicit dependencies seem unnecessary, and RDEPENDS_${PN}
doesn't do anything for native recipes either, so all of that gets
removed.
It also built fine without the m4 and parallel build workarounds.
There's no need to have a separate .inc file.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/recipes-devtools/autoconf-archive/autoconf-archive.inc | 13
-
meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb | 14
+++---
2 files changed, 7 insertions(+), 20 deletions(-)
delete mode 100644 meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
diff --git a/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
b/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
deleted file mode 100644
index 4f63e0f..000
--- a/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
+++ /dev/null
@@ -1,13 +0,0 @@
-LICENSE = "GPLv3"
-HOMEPAGE = "http://www.gnu.org/software/autoconf-archive/;
-SECTION = "devel"
-
-DEPENDS += "m4-native"
-DEPENDS_class-native = "m4-native gnu-config-native"
-DEPENDS_class-nativesdk = "m4-nativesdk gnu-config-nativesdk"
-
-RDEPENDS_${PN} = "m4 gnu-config"
-
-SRC_URI = "${GNU_MIRROR}/${BPN}/${BPN}-${PV}.tar.xz"
-
-inherit autotools
diff --git
a/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
index 0a1a771..d77c37d 100644
--- a/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
+++ b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
@@ -1,13 +1,13 @@
-require autoconf-archive.inc
-
-
-PARALLEL_MAKE = ""
-
-LICENSE = "GPLv2 & GPLv3"
+SUMMARY = "a collection of freely re-usable Autoconf macros"
+HOMEPAGE = "http://www.gnu.org/software/autoconf-archive/;
+SECTION = "devel"
+LICENSE = "GPLv3"
LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
+SRC_URI = "${GNU_MIRROR}/${BPN}/${BPN}-${PV}.tar.xz"
SRC_URI[md5sum] = "bf19d4cddce260b3c3e1d51d42509071"
SRC_URI[sha256sum] =
"e8f2efd235f842bad2f6938bf4a72240a5e5fcd248e8444335e63beb60fabd82"
-EXTRA_OECONF += "ac_cv_path_M4=m4"
+inherit autotools
+
BBCLASSEXTEND = "native nativesdk"
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 1/2] autoconf-archive: move from meta-oe to OE-core
Having common macros in OE-core that are needed by autotools based
projects makes sense. For example, tpm2.0-tools in meta-measured
depended on meta-oe only because of autoconf-archive.
This is a verbatim copy of the autoconf-archive recipe in
meta-openembedded rev 1cbd1bc1, with just one change: the patch which
disabled the installation of ax_code_coverage.m4 and
ax_check_enable_debug.m4 and the dependency on gnome-common were
removed.
So now autoconf-archive in OE-core provides them. gnome-common in
meta-oe will be changed to not install them and instead depend on
autoconf-archive.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/recipes-devtools/autoconf-archive/autoconf-archive.inc | 13
+
meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb | 13
+
2 files changed, 26 insertions(+)
create mode 100644 meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
create mode 100644
meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
diff --git a/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
b/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
new file mode 100644
index 000..4f63e0f
--- /dev/null
+++ b/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
@@ -0,0 +1,13 @@
+LICENSE = "GPLv3"
+HOMEPAGE = "http://www.gnu.org/software/autoconf-archive/;
+SECTION = "devel"
+
+DEPENDS += "m4-native"
+DEPENDS_class-native = "m4-native gnu-config-native"
+DEPENDS_class-nativesdk = "m4-nativesdk gnu-config-nativesdk"
+
+RDEPENDS_${PN} = "m4 gnu-config"
+
+SRC_URI = "${GNU_MIRROR}/${BPN}/${BPN}-${PV}.tar.xz"
+
+inherit autotools
diff --git
a/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
new file mode 100644
index 000..0a1a771
--- /dev/null
+++ b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
@@ -0,0 +1,13 @@
+require autoconf-archive.inc
+
+
+PARALLEL_MAKE = ""
+
+LICENSE = "GPLv2 & GPLv3"
+LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
+
+SRC_URI[md5sum] = "bf19d4cddce260b3c3e1d51d42509071"
+SRC_URI[sha256sum] =
"e8f2efd235f842bad2f6938bf4a72240a5e5fcd248e8444335e63beb60fabd82"
+
+EXTRA_OECONF += "ac_cv_path_M4=m4"
+BBCLASSEXTEND = "native nativesdk"
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] autoconf-archive: move from meta-oe to OE-core
Having common macros in OE-core that are needed by autotools based
projects makes sense. For example, tpm2.0-tools in meta-measured
depended on meta-oe only because of autoconf-archive.
This is a verbatim copy of the autoconf-archive recipe in
meta-openembedded rev 1cbd1bc1, with just one change: the patch which
disabled the installation of ax_code_coverage.m4 and
ax_check_enable_debug.m4 was removed.
So now autoconf-archive in OE-core provides them. gnome-common in
meta-oe will be changed to not install them and instead depend on
autoconf-archive.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/recipes-devtools/autoconf-archive/autoconf-archive.inc | 15
+++
meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb | 13
+
2 files changed, 28 insertions(+)
create mode 100644 meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
create mode 100644
meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
diff --git a/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
b/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
new file mode 100644
index 000..9684d1f
--- /dev/null
+++ b/meta/recipes-devtools/autoconf-archive/autoconf-archive.inc
@@ -0,0 +1,15 @@
+LICENSE = "GPLv3"
+HOMEPAGE = "http://www.gnu.org/software/autoconf-archive/;
+SECTION = "devel"
+
+DEPENDS += "m4-native"
+DEPENDS_class-native = "m4-native gnu-config-native"
+DEPENDS_class-nativesdk = "m4-nativesdk gnu-config-nativesdk"
+
+RDEPENDS_${PN} = "m4 gnu-config gnome-common"
+RDEPENDS_${PN}_class-native = "m4-native gnu-config-native"
+RDEPENDS_${PN}_class-nativesdk = "m4-nativesdk gnu-config-nativesdk"
+
+SRC_URI = "${GNU_MIRROR}/${BPN}/${BPN}-${PV}.tar.xz"
+
+inherit autotools
diff --git
a/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
new file mode 100644
index 000..0a1a771
--- /dev/null
+++ b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
@@ -0,0 +1,13 @@
+require autoconf-archive.inc
+
+
+PARALLEL_MAKE = ""
+
+LICENSE = "GPLv2 & GPLv3"
+LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
+
+SRC_URI[md5sum] = "bf19d4cddce260b3c3e1d51d42509071"
+SRC_URI[sha256sum] =
"e8f2efd235f842bad2f6938bf4a72240a5e5fcd248e8444335e63beb60fabd82"
+
+EXTRA_OECONF += "ac_cv_path_M4=m4"
+BBCLASSEXTEND = "native nativesdk"
base-commit: d3a41fbd94462efc8c6f1b55f6fb54001b447c45
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] image_types_wic: add dependency to e2fsprogs-native
On Wed, 2017-07-26 at 19:21 +0300, Ed Bartosh wrote: > Added e2fsprogs-native to the list of default dependencies for > wic (WKS_FILE_DEPENDS_DEFAULT) as all fs-related utilities > have to be in this list. > > Thanks to Patrick Ohly for noticing this. > > [YOCTO #11817] > > Signed-off-by: Ed Bartosh <ed.bart...@linux.intel.com> Acked-by: Patrick Ohly <patrick.o...@intel.com> -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] image: Fix "metadata is not deterministic" when chaining 2+ CONVERSION_CMDs
On Tue, 2017-07-25 at 15:58 -0400, Tom Rini wrote: > When we have more than one CONVERSION_CMD being used, for example > ext4.gz.sha256sum we will see errors about "metadata is not > deterministic". This is because we do not have a stable order of > intermediate files that will be removed in the generated shell > command. > We fix this by calling sorted() on the set of rm_tmp_images so that > we > will have a stable hash again. > > Cc: Patrick Ohly <patrick.o...@intel.com> > Signed-off-by: Tom Rini <tr...@konsulko.com> Acked-by: Patrick Ohly <patrick.o...@intel.com> -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] gettext.bbclass: also search for files in target sysroot
fwupd contains polkit policy files that it translates using polkit.its
and polkit.loc files that the next polkit release is going to
install (see https://github.com/hughsie/fwupd/issues/107).
In order to make that work with OE-core, the gettext tools must be
told to look also for files in the recipe-sysroot. Otherwise it only
uses the GETTEXTDATADIR set by the gettext-native tool wrappers, and
that only points to the files provided by gettext-native itself.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/classes/gettext.bbclass | 5 +
1 file changed, 5 insertions(+)
diff --git a/meta/classes/gettext.bbclass b/meta/classes/gettext.bbclass
index d60a0c1..689ef55 100644
--- a/meta/classes/gettext.bbclass
+++ b/meta/classes/gettext.bbclass
@@ -17,3 +17,8 @@ DEPENDS_GETTEXT ??= "virtual/gettext gettext-native"
BASEDEPENDS_append = " ${@gettext_dependencies(d)}"
EXTRA_OECONF_append = " ${@gettext_oeconf(d)}"
+
+# Without this, msgfmt from gettext-native will not find ITS files
+# provided by target recipes (for example, polkit.its).
+GETTEXTDATADIRS_append_class-target = ":${STAGING_DATADIR}/gettext"
+export GETTEXTDATADIRS
base-commit: d3a41fbd94462efc8c6f1b55f6fb54001b447c45
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] image.bbclass: Correct chaining compression support
On Mon, 2017-07-24 at 07:19 -0400, Tom Rini wrote:
> On Mon, Jul 24, 2017 at 10:35:37AM +0200, Patrick Ohly wrote:
> > On Fri, 2017-07-21 at 18:06 -0400, Tom Rini wrote:
> > > The fix for this inadvertently broke chaining
> > > compression/conversion. First, correct the u-boot conversion
> > > code.
> > >
> > > Fixes: 46bc438374de ("image.bbclass: do exact
> > > match for rootfs type")
> > > Cc: Zhenhua Luo <zhenhua....@nxp.com>
> > > Cc: Richard Purdie <richard.pur...@linuxfoundation.org>
> > > Cc: Patrick Ohly <patrick.o...@intel.com>
> > > Signed-off-by: Tom Rini <tr...@konsulko.com>
> > > ---
> > > This change is fairly important and, imho, innocuous and should
> > > be
> > > populated to pyro/etc, once merged to master. The next part of
> > > the
> > > series is clean-up and while with my U-Boot hat on, I would say
> > > should
> > > be pushed more widely, I am biased.
> > > ---
> > > meta/classes/image.bbclass | 2 +-
> > > meta/classes/image_types_uboot.bbclass | 13 +
> > > 2 files changed, 6 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/meta/classes/image.bbclass
> > > b/meta/classes/image.bbclass
> > > index de535ce6fcff..bd6a5b7b810a 100644
> > > --- a/meta/classes/image.bbclass
> > > +++ b/meta/classes/image.bbclass
> > > @@ -453,7 +453,7 @@ python () {
> > > rm_tmp_images = set()
> > > def gen_conversion_cmds(bt):
> > > for ctype in ctypes:
> > > - if bt[bt.find('.') + 1:] == ctype:
> > > + if bt.endswith("." + ctype)
> >
> > This reverts 46bc438374de and thus restores the code as I had
> > originally written it.
> >
> > Looking at 46bc438374de, it's not clear to me how the commit
> > message
> > matches the code, i.e. I don't understand the commit. So it was an
> > incorrect fix for the problem described in that commit message, and
> > the
> > right one are your changes to the u-boot conversion command?
>
> Ah, so the important bit is the other half of this patch, which
> addresses the problem 46bc438374de was intended to deal with,
> correctly.
> Prior to the chaining compression/conversion support, the "u-boot"
> targets would clean up their intermediate files. With your patch
> those
> files get cleaned up automatically and that the mkimage calling
> function
> was also doing it lead to build failures. But since we no longer
> need
> to have a manual cleaning step, we can just drop it.
Makes sense.
Acked-by: Patrick Ohly <patrick.o...@intel.com>
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] image.bbclass: Correct chaining compression support
On Fri, 2017-07-21 at 18:06 -0400, Tom Rini wrote:
> The fix for this inadvertently broke chaining
> compression/conversion. First, correct the u-boot conversion code.
>
> Fixes: 46bc438374de ("image.bbclass: do exact
> match for rootfs type")
> Cc: Zhenhua Luo <zhenhua@nxp.com>
> Cc: Richard Purdie <richard.pur...@linuxfoundation.org>
> Cc: Patrick Ohly <patrick.o...@intel.com>
> Signed-off-by: Tom Rini <tr...@konsulko.com>
> ---
> This change is fairly important and, imho, innocuous and should be
> populated to pyro/etc, once merged to master. The next part of the
> series is clean-up and while with my U-Boot hat on, I would say
> should
> be pushed more widely, I am biased.
> ---
> meta/classes/image.bbclass | 2 +-
> meta/classes/image_types_uboot.bbclass | 13 +
> 2 files changed, 6 insertions(+), 9 deletions(-)
>
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index de535ce6fcff..bd6a5b7b810a 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -453,7 +453,7 @@ python () {
> rm_tmp_images = set()
> def gen_conversion_cmds(bt):
> for ctype in ctypes:
> - if bt[bt.find('.') + 1:] == ctype:
> + if bt.endswith("." + ctype)
This reverts 46bc438374de and thus restores the code as I had
originally written it.
Looking at 46bc438374de, it's not clear to me how the commit message
matches the code, i.e. I don't understand the commit. So it was an
incorrect fix for the problem described in that commit message, and the
right one are your changes to the u-boot conversion command?
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] image-vm: Convert vmdk/vdi/qcow2/hdddirect images to IMAGE_CMD
On Fri, 2017-07-21 at 16:52 -0400, Tom Rini wrote:
> On Fri, Jul 21, 2017 at 03:01:33PM +0200, Patrick Ohly wrote:
> > On Fri, 2017-07-21 at 07:21 -0400, Tom Rini wrote:
> > > On Fri, Jul 21, 2017 at 09:47:21AM +0200, Patrick Ohly wrote:
> > > > On Thu, 2017-07-20 at 18:43 -0400, Tom Rini wrote:
> > > > > The support for writing vmdk/vdi/qcow2 images has not been converted
> > > > > to make
> > > > > use of the IMAGE_CMD infrastructure and instead relies on custom
> > > > > logic for
> > > > > adding tasks in the right place. Convert these images to making use
> > > > > of
> > > > > IMAGE_CMD.
> > > >
> > > > Thanks for working on this. I still have
> > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=10204 open for
> > > > enhancing vmdk/vdi/qcow2.
> > > >
> > > > However, your patch doesn't go as far as described in that bug, does it?
> > > > Instead of allowing, for example, IMAGE_FSTYPES = "wic.vmdk", it merely
> > > > changes how IMAGE_FSTYPES = "vmdk" is implemented.
> > > >
> > > > The current patch has its merits as it simplifies the implementation,
> > > > but I think it would be worthwhile to go all the way, even if it changes
> > > > how images are going to be named.
> > >
> > > Ah, interesting. I guess the first thing I would argue against is
> > > saying that hdddirect should be masked. There are reasons to want to
> > > have a raw image created.
> >
> > I'm not sure I understand what you mean with "hdddirect should be
> > masked" - it's been a while that I looked at the code.
> >
> > > I will take a look into using CONVERSIONCMD for vmdk/qcow2/vdi to allow
> > > for wic.vmdk to work. But we also must have vmdk.xz work (and fwiw with
> > > my series you can get arbitrary compression that we support done, I did
> > > vmdk.xz, qcow2.lzo and vdi.bz2 just for 'fun').
> >
> > The same is true with CONVERSIONCMD. They can be chained multiple times.
> >
> > > But I'm not 100% sure that I like changing the normal case from "vmdk"
> > > to "wic.vmdk" or "hdddirect.vmdk" just to get a disk image to throw at
> > > ${VM_TECHNOLOGY}.
> >
> > Yes, that's the biggest user-visible change. Basically the internal
> > implementation (multiple conversions chained together) becomes visible
> > in the end-result.
> >
> > That is a good and a bad thing. The good thing is that it is visible how
> > the .vmdk image was created. The bad thing is that some users might not
> > care and get confused.
>
> So, Patrick, Ed and I talked on IRC a bit about this, and I think
> without putting too many words in their mouths, I think it can be
> summarized like this.
>
> We all agree that vmdk/qcow2/vdi are a type of conversion. It would be
> quite handy to use these conversions on other things, such as wic.
>
> And at least from my point of view, the contents of "hdddirect" doesn't
> matter so much as the overall functionality.
>
> So that here's where I'm at now. I've found out when Patrick's chaining
> compression code (so that you could do silly things like ext3.gz.bz2 or
> _useful_ things like wic.vmdk.xz) was broken, and I see the best way to
> fix that too. So I'm going to v2 this series shortly which addresses
> the problem of chaining compression types, and then brings the VM stuff
> over to using that too.
Thanks for the good summary and working on this functionality.
I'd like to add that you pointed out that a "generic" vmdk type is what
some developers ask for when they don't care about the exact way how the
resulting image is created (hdddirect vs. wic, for example). We agreed
that this is useful.
I didn't get around to mentioning it on IRC, but what I would find it
useful is if someone (BSP and/or distro?) can define which kind of raw
image is used as input for the conversion to vmdk. Hard-coding that to
just hdddirect always felt wrong to me.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] image-vm: Convert vmdk/vdi/qcow2/hdddirect images to IMAGE_CMD
On Fri, 2017-07-21 at 07:21 -0400, Tom Rini wrote:
> On Fri, Jul 21, 2017 at 09:47:21AM +0200, Patrick Ohly wrote:
> > On Thu, 2017-07-20 at 18:43 -0400, Tom Rini wrote:
> > > The support for writing vmdk/vdi/qcow2 images has not been converted to
> > > make
> > > use of the IMAGE_CMD infrastructure and instead relies on custom logic for
> > > adding tasks in the right place. Convert these images to making use of
> > > IMAGE_CMD.
> >
> > Thanks for working on this. I still have
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=10204 open for
> > enhancing vmdk/vdi/qcow2.
> >
> > However, your patch doesn't go as far as described in that bug, does it?
> > Instead of allowing, for example, IMAGE_FSTYPES = "wic.vmdk", it merely
> > changes how IMAGE_FSTYPES = "vmdk" is implemented.
> >
> > The current patch has its merits as it simplifies the implementation,
> > but I think it would be worthwhile to go all the way, even if it changes
> > how images are going to be named.
>
> Ah, interesting. I guess the first thing I would argue against is
> saying that hdddirect should be masked. There are reasons to want to
> have a raw image created.
I'm not sure I understand what you mean with "hdddirect should be
masked" - it's been a while that I looked at the code.
> I will take a look into using CONVERSIONCMD for vmdk/qcow2/vdi to allow
> for wic.vmdk to work. But we also must have vmdk.xz work (and fwiw with
> my series you can get arbitrary compression that we support done, I did
> vmdk.xz, qcow2.lzo and vdi.bz2 just for 'fun').
The same is true with CONVERSIONCMD. They can be chained multiple times.
> But I'm not 100% sure that I like changing the normal case from "vmdk"
> to "wic.vmdk" or "hdddirect.vmdk" just to get a disk image to throw at
> ${VM_TECHNOLOGY}.
Yes, that's the biggest user-visible change. Basically the internal
implementation (multiple conversions chained together) becomes visible
in the end-result.
That is a good and a bad thing. The good thing is that it is visible how
the .vmdk image was created. The bad thing is that some users might not
care and get confused.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] image-vm: Convert vmdk/vdi/qcow2/hdddirect images to IMAGE_CMD
On Thu, 2017-07-20 at 18:43 -0400, Tom Rini wrote: > The support for writing vmdk/vdi/qcow2 images has not been converted to make > use of the IMAGE_CMD infrastructure and instead relies on custom logic for > adding tasks in the right place. Convert these images to making use of > IMAGE_CMD. Thanks for working on this. I still have https://bugzilla.yoctoproject.org/show_bug.cgi?id=10204 open for enhancing vmdk/vdi/qcow2. However, your patch doesn't go as far as described in that bug, does it? Instead of allowing, for example, IMAGE_FSTYPES = "wic.vmdk", it merely changes how IMAGE_FSTYPES = "vmdk" is implemented. The current patch has its merits as it simplifies the implementation, but I think it would be worthwhile to go all the way, even if it changes how images are going to be named. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] ovmf-shell-image.bb: simplify dependencies
The image consists only of the EFI system partition, therefore we can avoid depending on the default wic tools. Signed-off-by: Patrick Ohly <patrick.o...@intel.com> --- meta/recipes-core/ovmf/ovmf-shell-image.bb | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/meta/recipes-core/ovmf/ovmf-shell-image.bb b/meta/recipes-core/ovmf/ovmf-shell-image.bb index 029547b..0d2b8bf 100644 --- a/meta/recipes-core/ovmf/ovmf-shell-image.bb +++ b/meta/recipes-core/ovmf/ovmf-shell-image.bb @@ -1,10 +1,13 @@ DESCRIPTION = "boot image with UEFI shell and tools" # For this image recipe, only the wic format with a -# single vfat partition makes sense. +# single vfat partition makes sense. Because we have no +# boot loader and no rootfs partition, not additional +# tools are needed for this .wks file. IMAGE_FSTYPES_forcevariable = 'wic' - WKS_FILE = "ovmf/ovmf-shell-image.wks" +WKS_FILE_DEPENDS = "" + inherit image # We want a minimal image with just ovmf-shell-efi unpacked in it. We base-commit: c4c2fb3732dbb290b7f0ca43af2e8662f99e4582 -- git-series 0.9.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 01/30] oeqa/core/loader: Switch method definition for _make_failed_test
On Mon, 2017-07-17 at 14:41 -0500, Aníbal Limón wrote: > > On 07/17/2017 02:14 PM, Patrick Ohly wrote: > > On Fri, 2017-07-14 at 10:27 -0500, Aníbal Limón wrote: > >> > >> On 07/14/2017 04:52 AM, Patrick Ohly wrote: > >>> On Tue, 2017-07-11 at 15:23 -0500, Aníbal Limón wrote: > >>>> This was a mistake of me to define wrong what methods needs > >>>> to be defined by certain python version. > >>>> > >>>> See rev d8380d098a290510b442a7abd2dd5a50cabf5844. > >>> > >>> This will fix this error that we see in Refkit with current OE-core > >>> master, right? > >>> > >>> 00:07:10.313 Traceback (most recent call last): > >>> 00:07:10.313 File > >>> "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", > >>> line 70, in > >>> 00:07:10.313 ret = main() > >>> 00:07:10.313 File > >>> "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", > >>> line 57, in main > >>> 00:07:10.313 results = args.func(logger, args) > >>> 00:07:10.313 File > >>> "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", > >>> line 215, in run > >>> 00:07:10.313 rc = self._internal_run(logger, args) > >>> 00:07:10.313 File > >>> "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", > >>> line 176, in _internal_run > >>> 00:07:10.313 self.tc.loadTests(self.module_paths, > >>> **self.tc_kwargs['load']) > >>> 00:07:10.313 File > >>> "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/context.py", > >>> line 51, in loadTests > >>> 00:07:10.313 self.suites = self.loader.discover() > >>> 00:07:10.313 File > >>> "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/loader.py", > >>> line 286, in discover > >>> 00:07:10.313 pattern='*.py', top_level_dir=path) > >>> 00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 275, > >>> in discover > >>> 00:07:10.313 tests = list(self._find_tests(start_dir, pattern)) > >>> 00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 327, > >>> in _find_tests > >>> 00:07:10.313 yield _make_failed_import_test(name, self.suiteClass) > >>> 00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 39, > >>> in _make_failed_import_test > >>> 00:07:10.313 return _make_failed_test(name, ImportError(message), > >>> suiteClass) > >>> 00:07:10.313 TypeError: _make_failed_test() missing 1 required positional > >>> argument: 'suiteClass' > >>> > >>> Can this particular patch please be merged into OE-core master > >>> independently from the patch series? It's not really related to it > >>> anyway. > >> > >> Yes this will fix that error, showing the error into the case. > > > > The fix doesn't work for me: > > > > File "/usr/lib/python3.5/unittest/loader.py", line 41, in > > _make_failed_import_test > > return _make_failed_test(name, ImportError(message), suiteClass, > > message) > > TypeError: _make_failed_test() takes 3 positional arguments but 4 were given > > > > You need to mimic the exact same signature as in Python itself, and > > Python 3.5 seems to be different again: > > > > /usr/lib/python3.5/unittest/loader.py:def _make_failed_test(methodname, > > exception, suiteClass, message): > > This is strange, i tested on a python 3.5.x and python 3.4.3 versions. > I don't know why isn't working on your machine. Have you tested with a test case that fails? python3.5 = 3.5.3 definitely has four parameters, as quoted above. I don't know which version has "def _make_failed_test(classname, exception, suiteClass)". The actual error in refkit is: ImportError: Failed to import test module: refkit_ostree Traceback (most recent call last): File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name __import_
Re: [OE-core] [PATCH 01/30] oeqa/core/loader: Switch method definition for _make_failed_test
On Fri, 2017-07-14 at 10:27 -0500, Aníbal Limón wrote: > > On 07/14/2017 04:52 AM, Patrick Ohly wrote: > > On Tue, 2017-07-11 at 15:23 -0500, Aníbal Limón wrote: > >> This was a mistake of me to define wrong what methods needs > >> to be defined by certain python version. > >> > >> See rev d8380d098a290510b442a7abd2dd5a50cabf5844. > > > > This will fix this error that we see in Refkit with current OE-core > > master, right? > > > > 00:07:10.313 Traceback (most recent call last): > > 00:07:10.313 File > > "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", > > line 70, in > > 00:07:10.313 ret = main() > > 00:07:10.313 File > > "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", > > line 57, in main > > 00:07:10.313 results = args.func(logger, args) > > 00:07:10.313 File > > "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", > > line 215, in run > > 00:07:10.313 rc = self._internal_run(logger, args) > > 00:07:10.313 File > > "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", > > line 176, in _internal_run > > 00:07:10.313 self.tc.loadTests(self.module_paths, > > **self.tc_kwargs['load']) > > 00:07:10.313 File > > "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/context.py", > > line 51, in loadTests > > 00:07:10.313 self.suites = self.loader.discover() > > 00:07:10.313 File > > "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/loader.py", > > line 286, in discover > > 00:07:10.313 pattern='*.py', top_level_dir=path) > > 00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 275, in > > discover > > 00:07:10.313 tests = list(self._find_tests(start_dir, pattern)) > > 00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 327, in > > _find_tests > > 00:07:10.313 yield _make_failed_import_test(name, self.suiteClass) > > 00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 39, in > > _make_failed_import_test > > 00:07:10.313 return _make_failed_test(name, ImportError(message), > > suiteClass) > > 00:07:10.313 TypeError: _make_failed_test() missing 1 required positional > > argument: 'suiteClass' > > > > Can this particular patch please be merged into OE-core master > > independently from the patch series? It's not really related to it > > anyway. > > Yes this will fix that error, showing the error into the case. The fix doesn't work for me: File "/usr/lib/python3.5/unittest/loader.py", line 41, in _make_failed_import_test return _make_failed_test(name, ImportError(message), suiteClass, message) TypeError: _make_failed_test() takes 3 positional arguments but 4 were given You need to mimic the exact same signature as in Python itself, and Python 3.5 seems to be different again: /usr/lib/python3.5/unittest/loader.py:def _make_failed_test(methodname, exception, suiteClass, message): You can probably get away with if sys.version_info >= (3,4,4): def _make_failed_test(classname, exception, suiteClass, message=None): The actual problem in refkit is that we are hitting this _make_failed_test() at all. After fixing meta/lib/oeqa/core/loader.py, I am getting a more useful exception and know what to do about the actual problem. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 01/30] oeqa/core/loader: Switch method definition for _make_failed_test
On Tue, 2017-07-11 at 15:23 -0500, Aníbal Limón wrote: > This was a mistake of me to define wrong what methods needs > to be defined by certain python version. > > See rev d8380d098a290510b442a7abd2dd5a50cabf5844. This will fix this error that we see in Refkit with current OE-core master, right? 00:07:10.313 Traceback (most recent call last): 00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 70, in 00:07:10.313 ret = main() 00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 57, in main 00:07:10.313 results = args.func(logger, args) 00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 215, in run 00:07:10.313 rc = self._internal_run(logger, args) 00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 176, in _internal_run 00:07:10.313 self.tc.loadTests(self.module_paths, **self.tc_kwargs['load']) 00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/context.py", line 51, in loadTests 00:07:10.313 self.suites = self.loader.discover() 00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/loader.py", line 286, in discover 00:07:10.313 pattern='*.py', top_level_dir=path) 00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 275, in discover 00:07:10.313 tests = list(self._find_tests(start_dir, pattern)) 00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 327, in _find_tests 00:07:10.313 yield _make_failed_import_test(name, self.suiteClass) 00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 39, in _make_failed_import_test 00:07:10.313 return _make_failed_test(name, ImportError(message), suiteClass) 00:07:10.313 TypeError: _make_failed_test() missing 1 required positional argument: 'suiteClass' Can this particular patch please be merged into OE-core master independently from the patch series? It's not really related to it anyway. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 20/30] oeqa/selftest/cases: runqemu enable thraded runs
On Wed, 2017-07-12 at 10:01 -0500, Aníbal Limón wrote: > > Add also a wrapper for runqemu and we can get rid of "from > > oeqa.utils.commands import" completely. > > Yes we can after the refkit and other selftest are adapted. My proposal wasn't to remove oeqa.utils.commands, only the "import" statements for it in selftest test cases. There might be legitimate uses for oeqa.utils.commands outside of selftests. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 03/30] selftest/cases/package: Call parent setUpClass method
On Wed, 2017-07-12 at 10:44 -0500, Aníbal Limón wrote: > > On 07/12/2017 01:51 AM, Patrick Ohly wrote: > > On Tue, 2017-07-11 at 15:23 -0500, Aníbal Limón wrote: > >> def setUpClass(cls): > >> +super(VersionOrdering, cls).setUpClass() > >> + > > > > In Python3 one can simply use super().setUpClass(). > > > > Not sure whether that's worth a V2 by itself. There's also plenty of > > other code outside of this patch which could be simplified like this. > > Yes may be is better to send a patch changing all the cases to the new > py3 form. Definitely not something for this patch series. However, I think new code should avoid the more complex form whenever possible. So if you do a V2, then I'd prefer to have super() in your patch. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] distrooverrides.bbclass: fix default configuration
When using distrooverrides.bbclass without setting
DISTRO_FEATURES_OVERRIDES, the code failed because of a spelling error
in the default.
[YOCTO #11759]
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/classes/distrooverrides.bbclass | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/classes/distrooverrides.bbclass
b/meta/classes/distrooverrides.bbclass
index 9a42712..9f4db0d 100644
--- a/meta/classes/distrooverrides.bbclass
+++ b/meta/classes/distrooverrides.bbclass
@@ -18,8 +18,8 @@
# of these overrides should be limited to .bb and .bbappend files,
# because then DISTRO_FEATURES is final.
-DISTRO_FEATURE_OVERRIDES ?= ""
-DISTRO_FEATURE_OVERRIDES[doc] = "A space-separated list of entries. \
+DISTRO_FEATURES_OVERRIDES ?= ""
+DISTRO_FEATURES_OVERRIDES[doc] = "A space-separated list of entries.
\
Each entry is added to OVERRIDES as df- if is in
DISTRO_FEATURES."
DISTRO_FEATURES_FILTER_NATIVE_append = " ${DISTRO_FEATURES_OVERRIDES}"
base-commit: f441186bafe2b8a80d3229f13aacf92138f63cd8
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/2] some "su -" fixes
Ran into this while experimenting with stateless logins Patrick Ohly (2): util-linux: fix "su -" and package su separately base-files: ignore "mesg n" error messages meta/recipes-core/base-files/base-files/share/dot.profile | 3 +- meta/recipes-core/util-linux/util-linux.inc | 13 ++-- 2 files changed, 13 insertions(+), 3 deletions(-) base-commit: 7dd5dfc4d56f1201110d947ce1ca3c6d64fbc7da -- git-series 0.9.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] util-linux: fix "su -" and package su separately
"su -" == "su --login" was broken because it uses /etc/pam.d/su-l and
lacking that, falls back to /etc/pam.d/other which denies the
operation. The fix is to symlink "su-l" to the normal "su" pam config
file.
Because "su" usually comes from "shadow" and has been broken like this
without anyone noticing, it probably is not used much and thus should
be packaged separately so that it can be installed only when really
needed. For backwards compatibility, "util-linux" still pulls it in.
It is a bit strange that DISTRO_FEATURES are getting checked when
deciding whether the packages should be defined. It is not wrong, the
packages will be simply empty and thus probably not created when the
distro feature is on and the package config is off. Perhaps there is a
reason, so this is kept unchanged. The symlink however only gets
created when su.util-linux really gets built.
[YOCTO #11126]
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/recipes-core/util-linux/util-linux.inc | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/meta/recipes-core/util-linux/util-linux.inc
b/meta/recipes-core/util-linux/util-linux.inc
index 1656e92..47c2839 100644
--- a/meta/recipes-core/util-linux/util-linux.inc
+++ b/meta/recipes-core/util-linux/util-linux.inc
@@ -35,7 +35,7 @@ PACKAGES =+ "util-linux-agetty util-linux-fdisk
util-linux-cfdisk util-linux-sfd
util-linux-partx util-linux-hwclock util-linux-mountpoint \
util-linux-findfs util-linux-getopt util-linux-sulogin
util-linux-prlimit"
PACKAGES += "${@bb.utils.contains('PACKAGECONFIG', 'pylibmount',
'util-linux-pylibmount', '', d)}"
-PACKAGES =+ "${@bb.utils.contains('DISTRO_FEATURES', 'pam',
'util-linux-runuser', '', d)}"
+PACKAGES =+ "${@bb.utils.contains('DISTRO_FEATURES', 'pam',
'util-linux-runuser util-linux-su', '', d)}"
PACKAGES_DYNAMIC = "^util-linux-lib.*"
@@ -91,6 +91,8 @@ FILES_util-linux-findfs = "${sbindir}/findfs"
FILES_util-linux-getopt = "${base_bindir}/getopt.${BPN}"
FILES_util-linux-runuser = "${sbindir}/runuser"
FILES_util-linux-prlimit = "${bindir}/prlimit"
+FILES_util-linux-su = "${bindir}/su.util-linux ${sysconfdir}/pam.d/su-l"
+CONFFILES_util-linux-su = "${sysconfdir}/pam.d/su-l"
FILES_util-linux-pylibmount =
"${PYTHON_SITEPACKAGES_DIR}/libmount/pylibmount.so \
${PYTHON_SITEPACKAGES_DIR}/libmount/__init__.* \
@@ -116,9 +118,10 @@ RREPLACES_util-linux-blkid = "e2fsprogs-blkid"
RDEPENDS_util-linux-reset += "ncurses"
RDEPENDS_util-linux-runuser += "libpam"
+RDEPENDS_util-linux-su += "libpam"
RDEPENDS_${PN} = "util-linux-umount util-linux-swaponoff util-linux-losetup
util-linux-sulogin util-linux-lsblk"
-RDEPENDS_${PN} += "${@bb.utils.contains('DISTRO_FEATURES', 'pam',
'util-linux-runuser', '', d)}"
+RDEPENDS_${PN} += "${@bb.utils.contains('DISTRO_FEATURES', 'pam',
'util-linux-runuser util-linux-su', '', d)}"
RRECOMMENDS_${PN} = "util-linux-fdisk util-linux-cfdisk util-linux-sfdisk
util-linux-mount util-linux-readprofile util-linux-mkfs util-linux-mountpoint
util-linux-prlimit"
@@ -182,6 +185,12 @@ do_install () {
install -m 0644 ${WORKDIR}/runuser.pamd
${D}${sysconfdir}/pam.d/runuser
install -m 0644 ${WORKDIR}/runuser-l.pamd
${D}${sysconfdir}/pam.d/runuser-l
fi
+ if [ "${@bb.utils.filter('PACKAGECONFIG', 'pam', d)}" ]; then
+ # Required for "su -" aka "su --login" because
+ # otherwise it uses "other", which has "auth pam_deny.so"
+ # and thus prevents the operation.
+ ln -s su ${D}${sysconfdir}/pam.d/su-l
+ fi
}
# reset and nologin causes a conflict with ncurses-native and shadow-native
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] base-files: ignore "mesg n" error messages
When using "su - myuser" to change from root to a non-privileged user, "mesg n" from the default .profile fails with "mesg: error: tty device is not owned by group `tty' or "mesg: cannot open /dev/ttyS0: Permission denied", depending on whether mesg comes from busybox or util-linux. This does not happen during a normal login because permissions on /dev/tty* get changed while doing that, something that isn't possible with plain "su -". As the error can't be avoided and failures of mesg probably aren't particularly important, now error messages get dumped to /dev/null. [YOCTO #11127] Signed-off-by: Patrick Ohly <patrick.o...@intel.com> --- meta/recipes-core/base-files/base-files/share/dot.profile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/meta/recipes-core/base-files/base-files/share/dot.profile b/meta/recipes-core/base-files/base-files/share/dot.profile index 979793e..a873160 100644 --- a/meta/recipes-core/base-files/base-files/share/dot.profile +++ b/meta/recipes-core/base-files/base-files/share/dot.profile @@ -7,4 +7,5 @@ fi # path set by /etc/profile # export PATH -mesg n +# Might fail after "su - myuser" when /dev/tty* is not writable by "myuser". +mesg n 2>/dev/null -- git-series 0.9.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 20/30] oeqa/selftest/cases: runqemu enable thraded runs
s/thraded/threaded/ in the subject. There are more spelling mistakes
elsewhere ("wrapper methos"). I don't know how important that is.
On Tue, 2017-07-11 at 15:23 -0500, Aníbal Limón wrote:
> - Update to use wrappers {bitbake,get_bb_var} from OESelfTestCase class.
s/OESelfTestCase/OESelftestTestCase/
Sorry to be pedantic, but I wanted to look up what these wrappers do and
couldn't even find the class ;-}
I've found them in the "oeqa/selftest/case: Add wrappers to
utils.commands modules" patch. I'm a bit worried that this entire patch
series is introducing concepts and methods without any documentation or
explanations why things are done this way. I suspect it will make it
very hard to write selftests correctly.
For example, this patch and others like it seem fairly arbitrary. It
doesn't explain why self.bitbake() is better than bitbake(). If in some
future patch or test someone were to use bitbake() when they should have
used self.bitbake() it's not going to be obvious either whether that is
correct.
Perhaps all OE tests should have these wrappers and only
OESelftestTestCase does something special with them? Then we can
gradually replace the direct calls to oeqa.utils.commands completely.
Add also a wrapper for runqemu and we can get rid of "from
oeqa.utils.commands import" completely.
> - Run into the main thread because it needs tinfoil to run.
>
> Signed-off-by: Aníbal Limón <anibal.li...@linux.intel.com>
> ---
> meta/lib/oeqa/selftest/cases/runqemu.py | 7 +++
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/meta/lib/oeqa/selftest/cases/runqemu.py
> b/meta/lib/oeqa/selftest/cases/runqemu.py
> index 4050a4123ba..e30cb24046f 100644
> --- a/meta/lib/oeqa/selftest/cases/runqemu.py
> +++ b/meta/lib/oeqa/selftest/cases/runqemu.py
> @@ -6,12 +6,11 @@ import re
> import logging
>
> from oeqa.selftest.case import OESelftestTestCase
> -from oeqa.utils.commands import bitbake, runqemu, get_bb_var
> +from oeqa.utils.commands import runqemu
> from oeqa.core.decorator.oeid import OETestID
>
> class RunqemuTests(OESelftestTestCase):
> """Runqemu test class"""
> -
> image_is_ready = False
> deploy_dir_image = ''
>
> @@ -37,8 +36,8 @@ SYSLINUX_TIMEOUT = "10"
> )
>
> if not RunqemuTests.image_is_ready:
> -RunqemuTests.deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE')
> -bitbake(self.recipe)
> +RunqemuTests.deploy_dir_image =
> self.get_bb_var('DEPLOY_DIR_IMAGE')
> +self.bitbake(self.recipe)
> RunqemuTests.image_is_ready = True
>
> @OETestID(2001)
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 18/30] oeqa/selftest/cases: tinfoil to run in the main thread
On Tue, 2017-07-11 at 15:23 -0500, Aníbal Limón wrote: > The tinfoil tests needs to initialize bitbake internals and wasn't > designed to be in a thread environment causing problems when try > for example set signal handlers in a thread different than the > main one. > > To workaround this execute tinfoil tests in the main thread and > don't use custom builddir because isn't make sense too. > > Signed-off-by: Aníbal Limón <anibal.li...@linux.intel.com> > --- > meta/lib/oeqa/selftest/cases/tinfoil.py | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/meta/lib/oeqa/selftest/cases/tinfoil.py > b/meta/lib/oeqa/selftest/cases/tinfoil.py > index aa1af7e0423..7248b755812 100644 > --- a/meta/lib/oeqa/selftest/cases/tinfoil.py > +++ b/meta/lib/oeqa/selftest/cases/tinfoil.py > @@ -9,7 +9,6 @@ from oeqa.core.decorator.oeid import OETestID > > class TinfoilTests(OESelftestTestCase): > """ Basic tests for the tinfoil API """ > - > @OETestID(1568) > def test_getvar(self): > with bb.tinfoil.Tinfoil() as tinfoil: There's just a line removal in this patch. That's probably not what was intended? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 03/30] selftest/cases/package: Call parent setUpClass method
On Tue, 2017-07-11 at 15:23 -0500, Aníbal Limón wrote: > def setUpClass(cls): > +super(VersionOrdering, cls).setUpClass() > + In Python3 one can simply use super().setUpClass(). Not sure whether that's worth a V2 by itself. There's also plenty of other code outside of this patch which could be simplified like this. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 6/8] image_types_wic: set default WKS_FILE_DEPENDS
On Thu, 2017-07-06 at 10:56 +0300, Ed Bartosh wrote:
> diff --git a/meta/classes/image_types_wic.bbclass
> b/meta/classes/image_types_wic.bbclass
> index 05ee68d..e9750b5 100644
> --- a/meta/classes/image_types_wic.bbclass
> +++ b/meta/classes/image_types_wic.bbclass
> @@ -40,7 +40,10 @@ USING_WIC =
> "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s
> WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> do_image_wic[depends] += "${@' '.join('%s-native:do_populate_sysroot'
> % r for r in ('parted', 'gptfdisk', 'dosfstools', 'mtools'))}"
> -WKS_FILE_DEPENDS ??= ''
> +WKS_FILE_DEPENDS ??= 'syslinux-native bmap-tools-native
> cdrtools-native btrfs-tools-native squashfs-tools-native'
> +WKS_FILE_DEPENDS_append_x86 = " syslinux grub-efi systemd-boot"
> +WKS_FILE_DEPENDS_append_x86-64 = " syslinux grub-efi systemd-boot"
> +
Using _append here adds these additional dependencies even when
WKS_FILE_DEPENDS has been set explicitly. How about this:
WKS_FILE_DEPENDS_DEFAULT = "syslinux-native bmap-tools-native
cdrtools-native btrfs-tools-native squashfs-tools-native"
WKS_FILE_DEPENDS_BOOTLOADERS = ""
WKS_FILE_DEPENDS_BOOTLOADERS_x86 = "syslinux grub-efi systemd-boot"
WKS_FILE_DEPENDS_BOOTLOADERS_x86-64 = "syslinux grub-efi systemd-boot"
WKS_FILE_DEPENDS ??= "${WKS_FILE_DEPENDS_DEFAULT}
${WKS_FILE_DEPENDS_BOOTLOADERS}"
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] systemd: enable resolved and networkd
On Mon, 2017-06-12 at 18:10 +0300, Maxin B. John wrote: > Enable systemd-resolved and systemd-networkd by default. > Make it co-exist with connman and Fix associated problems > in read-only rootfs. > > Fixes [YOCTO #11331] Let me come back to this, because I think it is not quite working as intended yet. The goal is that the "right" resolver is chosen via alternative priorities, right? So during build time, we set /etc/resolv.conf to what is the desired resolver. However, there's still a L+ entry for /etc/resolv.conf in /usr/lib/tmpfiles.d/connman_resolvconf.conf: L+ /etc/resolv.conf- - - - /var/run/connman/resolv.conf As a result, when systemd is used and the rootfs is read/write, then systemd overwrites /etc/resolv.conf, leading to: # ls -l /etc/resolv.conf lrwxrwxrwx1 root root28 Jul 6 14:44 /etc/resolv.conf -> /var/run/connman/resolv.conf That happens even if systemd-resolved has a higher priority and should be used. Maxin, do you agree? Can you finish this work and patch the ConnMan recipe so that it behaves as expected? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] bootimg-efi.py: Use IMGDEPLOYDIR instead of DEPLOY_DIR_IMAGE for initrd
On Mon, 2017-07-03 at 15:44 +0300, Ed Bartosh wrote: > On Mon, Jul 03, 2017 at 11:27:53AM +0200, Patrick Ohly wrote: > > On Mon, 2017-07-03 at 11:36 +0300, Ed Bartosh wrote: > > > On Fri, Jun 30, 2017 at 10:53:30AM -0700, Alejandro Hernandez wrote: > > > > When using wic to create an image from a certain build, wic is expecting > > > > to find initrd at the final destination of our images > > > > (DEPLOY_DIR_IMAGE), > > > > which is wrong, since the initrd file has not been copied to the final > > > > directory yet, > > > > > > Is it possible to ensure that initrd is deployed before wic is run by > > > making do_image_wic depend on initrd deploy task? > > > > Not in this case, because both do_image_wic and do_image_cpio share the > > same deploy task. > > Now I understand what's going on here. Basically Alejandro is trying to > use one format of the same image in another. It's like trying to put > .wic image inside ext2 image or something similar. > > I believe it doesn't fit current design of oe image creation system. > I'd suggest to use different image recipes to achieve this. Yes, that is also my preferred solution (see other email from this morning). -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] sanity.bbclass: fix AttributeError in mirror format checks
On Fri, 2017-06-30 at 14:24 +0300, Mikko Ylinen wrote:
> mirrors is a list after split() and results in:
>
> AttributeError: 'list' object has no attribute 'strip'
>
> when the 'mirror values are pairs' check fails.
>
> Signed-off-by: Mikko Ylinen <mikko.yli...@linux.intel.com>
> ---
> meta/classes/sanity.bbclass | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
> index e8064ac483..1f74026e13 100644
> --- a/meta/classes/sanity.bbclass
> +++ b/meta/classes/sanity.bbclass
> @@ -839,7 +839,7 @@ def check_sanity_everybuild(status, d):
>
> # Split into pairs
> if len(mirrors) % 2 != 0:
> -bb.warn('Invalid mirror variable value for %s: %s, should
> contain paired members.' % (mirror_var, mirrors.strip()))
> +bb.warn('Invalid mirror variable value for %s: %s, should
> contain paired members.' % (mirror_var, str(mirrors).strip()))
.strip() is redundant here, because str(mirrors) will produce [...],
i.e. the string will never have leading or trailing spaces.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] bootimg-efi.py: Use IMGDEPLOYDIR instead of DEPLOY_DIR_IMAGE for initrd
On Mon, 2017-07-03 at 11:36 +0300, Ed Bartosh wrote: > On Fri, Jun 30, 2017 at 10:53:30AM -0700, Alejandro Hernandez wrote: > > When using wic to create an image from a certain build, wic is expecting > > to find initrd at the final destination of our images (DEPLOY_DIR_IMAGE), > > which is wrong, since the initrd file has not been copied to the final > > directory yet, > > Is it possible to ensure that initrd is deployed before wic is run by > making do_image_wic depend on initrd deploy task? Not in this case, because both do_image_wic and do_image_cpio share the same deploy task. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2 0/5] #11662 - wic should mount /boot
On Mon, 2017-07-03 at 10:31 +0300, Ed Bartosh wrote: > On Fri, Jun 30, 2017 at 08:34:30PM +0200, Patrick Ohly wrote: > > then I don't see a need for any additional flags. Just > > don't use the features which result in a rootfs modification. > > I also didn't see it till last message from Otavio. Now I do - they > don't want to change .wks files. They're using standard wks from > scripts/lib/wic/canned-wks or from standard layers and they don't want > to duplicate them when they don't want rootfs modifications. > > It could be a valid reason to have --no-fstab-update option I think. > However, I'm still not 100% convinced I'm ok with this if nobody else > objects. Okay, now I see what the purpose is. I prefer a --no-fstab-update over a general --no-rootfs-update because for each case where wic would normally modify the rootfs, some other mechanism must be in place which makes that modification redundant (like using PARTUUID). Having separate parameters forces the developers to think about it. Just my 2 cents... -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] bootimg-efi.py: Use IMGDEPLOYDIR instead of DEPLOY_DIR_IMAGE for initrd
On Sat, 2017-07-01 at 16:48 -0500, Alejandro Hernandez wrote: > > On 07/01/2017 10:09 AM, Patrick Ohly wrote: > > Is it really intended that do_image_wic runs for > > core-image-tiny-initramfs? How and where is the output of > > core-image-tiny-initramfs used? > Yes, do_image_wic is intended to run automatically, hence why wic was > added to IMAGE_FSTYPES > I'm not sure I understand what you mean by the second question, but the > output is just as in any other image My point was that core-image-tiny-initramfs isn't like other *-initramfs image recipes at all, and having an outdated DESCRIPTION in it doesn't help much either. > > From the DESCRIPTION: > > > > core-image-tiny-initramfs doesn't actually generate an image but > > rather generates boot and rootfs artifacts into a common > > location that can subsequently be picked up by external image > > generation tools such as wic. > This is the old description, when it was though that we would have a common > artifacts directory, which never happened due to some changes in wic > last year > after which it was decided that core-image-tiny-initramfs would generate > the artifacts > AND a wic image. If that remains the solution (see below), will the DESCRIPTION and perhaps even the file name be changed? > > I still think your patch breaks images which use the initrd parameter to > > reference an initrd produced by some other recipe. > I see your point, if an initrd is produced by some other recipe it'd be > somewhere else, but at the same time, what if an initrd is produced by > the image recipe itself?, like in this case I see two solutions: 1) keep using the recipe as currently intended and add another sourceparam to bootimg-efi.py which changes the default source for initrd from DEPLOY_DIR_IMAGE (the current behavior) to IMGDEPLOYDIR 2) use two recipes, one for the initramfs (the current core-image-tiny-initramfs.bb, but without wic in IMAGE_FSTYPES) and a separate core-image-tiny.bb where do_rootfs gets replaced with an empty stub I personally find the second solution cleaner. It's the standard way of doing image + initramfs and thus will be easier to understand and have less unexpected pitfalls. One such pitfall in solution one exists even with your current patch: do_image_wic does not depend on do_image_cpio (run "bitbake -g core-image-tiny-initramfs:do_image_wic", "grep '^.core-image-tiny-initramfs.do_image_wic' task-depends.dot") and thus there's a race condition between the two tasks of the same recipe. That can be fixed, of course, but it is making the solution even more unusual. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] bootimg-efi.py: Use IMGDEPLOYDIR instead of DEPLOY_DIR_IMAGE for initrd
On Fri, 2017-06-30 at 15:38 -0500, Alejandro Hernandez wrote:
> Hey Patrick,
>
>
> On 06/30/2017 01:43 PM, Patrick Ohly wrote:
> > On Fri, 2017-06-30 at 10:53 -0700, Alejandro Hernandez wrote:
> >> When using wic to create an image from a certain build, wic is expecting
> >> to find initrd at the final destination of our images (DEPLOY_DIR_IMAGE),
> >> which is wrong, since the initrd file has not been copied to the final
> >> directory yet, so instead of trying to use an initrd file from
> >> DEPLOY_DIR_IMAGE we get it from IMGDEPLOYDIR, which is the directory
> >> where the resulting images are placed before their final destination,
> >> and its where we can find the correct initrd file for our image.
> > Can you give an example with some real image and initramfs recipe where
> > this works? In other words, provide an example where the old behavior
> > fails and the new one works?
> Sure, so the problem comes when you add "wic" to IMAGE_FSTYPES (which is
> not default for core-image-minimal nor core-image-minimal-initramfs).
> If you do this, do_image_wic will be executed automatically, and if it
> tries to use foo.cpio.gz file as initrd, it will fail to find it on
> DEPLOY_DIR_IMAGE (unless a previous build was completed without
> executing do_image_wic) since this file is still on IMGDEPLOYDIR and hasnt
> been copied to DEPLOY_DIR_IMAGE yet.
>
> So a real example on real hardware:
>
> if you build:
> DISTRO="poky-tiny"
> MACHINE="intel-corei7-64"
> bitbake core-image-tiny-initramfs
>
> You'll get an error stating that the foo.cpio.gz file cant be found.
Just for completeness, foo.cpio.gz =
core-image-tiny-initramfs-intel-corei7-64.cpio.gz, which is the
initramfs image produced by the recipe itself.
Is it really intended that do_image_wic runs for
core-image-tiny-initramfs? How and where is the output of
core-image-tiny-initramfs used?
>From the DESCRIPTION:
core-image-tiny-initramfs doesn't actually generate an image but
rather generates boot and rootfs artifacts into a common
location that can subsequently be picked up by external image
generation tools such as wic.
If core-image-tiny-initramfs is meant to produce an initramfs, similar
to core-image-minimal-initramfs, then it shouldn't use "wic" in its
IMAGE_FSTYPES. Currently "wic" is listed:
# don't actually generate an image, just the artifacts needed for one
IMAGE_FSTYPES = "${INITRAMFS_FSTYPES} wic"
Does the recipe still work as intended without your patch when you
remove wic here?
> I did a build for both core-image-minimal and
> core-image-minimal-initramfs and I dont see how theyre affected, the
> necessary files are
> on IMGDEPLOYDIR in both cases as well.
Are you sure that core-image-minimal copies an initrd in bootimg-efi.py?
It checks source_params, but systemd-bootdisk.wks does not specify an
initrd in its sourceparams:
part /boot --source bootimg-efi --sourceparams="loader=systemd-boot"
--ondisk sda --label msdos --active --align 1024
That's different for the systemd-bootdisk-tiny64.wks that is used for
core-image-tiny-initramfs, which has:
part /boot --source bootimg-efi
--sourceparams="loader=systemd-boot,initrd=core-image-tiny-initramfs-intel-corei7-64.cpio.gz"
--ondisk sda --label msdos --active --align 1024
I still think your patch breaks images which use the initrd parameter to
reference an initrd produced by some other recipe.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] bootimg-efi.py: Use IMGDEPLOYDIR instead of DEPLOY_DIR_IMAGE for initrd
On Fri, 2017-06-30 at 10:53 -0700, Alejandro Hernandez wrote: > When using wic to create an image from a certain build, wic is expecting > to find initrd at the final destination of our images (DEPLOY_DIR_IMAGE), > which is wrong, since the initrd file has not been copied to the final > directory yet, so instead of trying to use an initrd file from > DEPLOY_DIR_IMAGE we get it from IMGDEPLOYDIR, which is the directory > where the resulting images are placed before their final destination, > and its where we can find the correct initrd file for our image. Can you give an example with some real image and initramfs recipe where this works? In other words, provide an example where the old behavior fails and the new one works? When using core-image-minimal and core-image-minimal-initramfs, then both have different IMGDEPLOYDIRs, therefore code in core-image-minimal can only find the .cpio.gz in the DEPLOY_DIR_IMAGE. So to me, the current approach seems correct. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2 0/5] #11662 - wic should mount /boot
On Fri, 2017-06-30 at 14:33 -0300, Otavio Salvador wrote: > Other possible rootfs changes also should be possible to be disabled > but IMO it should be per-feature (one for fstab, one for exclude, > ...). I also think it should be per-feature, it necessary at all. I still do not fully understand under which circumstances wic modifies the rootfs. If that happens only when explicitly requested in the wks file as Ed said, then I don't see a need for any additional flags. Just don't use the features which result in a rootfs modification. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2 0/5] #11662 - wic should mount /boot
On Fri, 2017-06-30 at 15:23 +0300, Ed Bartosh wrote: > On Fri, Jun 30, 2017 at 11:02:13AM +0200, Patrick Ohly wrote: > > On Fri, 2017-06-30 at 11:37 +0300, Ed Bartosh wrote: > > > > I'm not sure I understand this. If you don't want fstab to be > > > changed > > > > you should not specify mount points in .wks > > > > There is only one reason to have mount points in .wks: to make wic > > > to > > > > change /etc/fstab, which you apparently don't want. So, don't > > > specify > > > > mount points and you'll have what you want. > > > > > > > > Having additional option for this looks redundand to me. > > > > > > After thinking a bit more about it I'd propose to have global wic > > > option > > > to avoid rootfs content changes. Not just fstab updates, but any > > > changes. For now this option (--no-rootfs-update ?) should prevent > > > creating > > > images if either mount points are specified or --exclude-path is used > > > in .wks > > > > Why does --exclude-path conflict with --no-rootfs-update? Is that a > > conceptual problem or an implementation problem? > > > > I thought that removing directories from original rootfs is a > modification. But it's not actually removed from the original roofs directory, right? For me, "not modified" refers to that and the files in it. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2 0/5] #11662 - wic should mount /boot
On Fri, 2017-06-30 at 11:37 +0300, Ed Bartosh wrote: > > I'm not sure I understand this. If you don't want fstab to be > changed > > you should not specify mount points in .wks > > There is only one reason to have mount points in .wks: to make wic > to > > change /etc/fstab, which you apparently don't want. So, don't > specify > > mount points and you'll have what you want. > > > > Having additional option for this looks redundand to me. > > After thinking a bit more about it I'd propose to have global wic > option > to avoid rootfs content changes. Not just fstab updates, but any > changes. For now this option (--no-rootfs-update ?) should prevent > creating > images if either mount points are specified or --exclude-path is used > in .wks Why does --exclude-path conflict with --no-rootfs-update? Is that a conceptual problem or an implementation problem? If I'm not mistaken, --exclude-path merely means "take this rootfs, but exclude certain parts". That's in line with --no-rootfs-update == "do not modify the content of the rootfs", as it just helps with choosing where content goes (the "single rootfs" -> "different partitions" approach). -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2 3/6] yocto-compat-layer.py: apply test_signatures to all layers
On Wed, 2017-06-28 at 11:08 +0200, Mark Hatle wrote: > On 6/27/17 5:33 PM, Patrick Ohly wrote: > > Software layers were previously allowed to change signatures, but > > that's not desired for those layers either. The rule that a layer > > which is "Yocto Compatible 2.0" must not change signatures unless > > explicitly requested holds for all kinds of layers. > > > > However, as this is something that software layers might not be able > > to do right away, testing for signature changes in software layers can > > be disabled. It's on by default, as that was Richard's > > recommendation. Whether that should change needs further discussion as > > part of finalizing "Yocto Compatible 2.0". > > > > As it might still change, the tool now has both a with/without > > parameter so that users of the tool can choose the desired behavior > > without being affected by future changes to the default. > > How would you regulate the behavior of a software layer that is doing > bbappends > or similar to a system provided component. By adding a PACKAGECONFIG that is off by default? But I haven't tried this and whether it influences task signatures. Do you have a specific example? Regarding these patches, is it okay to merge them as they are now? Without them, we cannot test software layers for signature changes, so won't know how much of a problem it would be. The tool and "Yocto Compatible 2.0" are work in progress, so there's still time to refine it after merging. My motivation for getting them merged already now is a) to make the change available to others and b) to use the strict version of the check in refkit (where we currently satisfy the criteria). -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2 1/6] yocto-compat-layer.py: avoid adding layers more than once
On Tue, 2017-06-27 at 15:46 -0700, Christopher Larson wrote: > > On Tue, Jun 27, 2017 at 8:33 AM, Patrick Ohly <patrick.o...@intel.com> > wrote: > add_layer_dependencies() might get called more than once, or > one of > the layer dependencies might already be present. The function > should > not add layers again because doing so can cause warnings like: > > WARNING: Duplicate inclusion > for > .../meta-openembedded/meta-oe/conf/distro/include/meta_oe_security_flags.inc > in .../meta-openembedded/meta-oe/conf/layer.conf > > Signed-off-by: Patrick Ohly <patrick.o...@intel.com> > --- > > I’m curious about why this isn’t either just calling out to > `bitbake-layers add-layer` or using one of the existing functions we > have for modifying bblayers, rather than doing its own +=. I don't know exactly, Mark implemented it like that. My guess is that more control was needed over which layers get added. For example, "bitbake-layers add-layer" only adds one layer, but does not recursively add layers it depends on. This could be improved of course. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCHv3 1/2] oeqa/selftest/{context, case}: Handle KeyboardInterrupt/SIGINT and SIGTERM
On Tue, 2017-06-27 at 15:25 -0500, Aníbal Limón wrote: > In order to avoid corrupt local.conf and bblayers.conf adds > signal handler for SIGTERM and use try/finally (KeyboardIntrrupt) block > to restore previously backuped configuration. Looks good to me now, with one minor nitpick which I should have noticed earlier: the second patch fixes a problem introduced by the first one (right?), so when applied in this order, bisecting gets broken. I think they should be merged the other way around to avoid that (calling the parent's setUpClass was fine before, it just wasn't required). -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCHv2 1/2] oeqa/selftest/{context, case}: Handle KeyboardInterrupt/SIGINT and SIGTERM
On Tue, 2017-06-27 at 11:30 -0500, Aníbal Limón wrote: > In order to avoid corrupt local.conf and bblayers.conf adds > signal handler for SIGTERM and use try/finally (KeyboardIntrrupt) block > to restore previously backuped configuration. Thanks for pointing out that SIGTERM isn't going through the "finally" block - I thought it would, hence my previous question. However, in this incarnation of your patch there's now cleanup code in various places. Would it be perhaps cleaner to just raise a suitable exception in the signal handler and leave all the cleanup code in the "finally" block? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2] bitbake.conf: Add sdl-config to HOSTTOOLS if using host SDL
On Tue, 2017-06-27 at 21:11 +1000, Jonathan Liu wrote:
> Hi Patrick,
>
> On 27 June 2017 at 20:38, Patrick Ohly <patrick.o...@intel.com> wrote:
> > On Tue, 2017-06-27 at 20:24 +1000, Jonathan Liu wrote:
> >> Hi Patrick,
> >>
> >> The original problem was that bitbake would print out the error:
> >> "libsdl-native is set to be ASSUME_PROVIDED but sdl-config can't be
> >> found in PATH. Please either install it, or configure qemu not to
> >> require sdl.", if "libsdl-native" was in ASSUME_PROVIDED even if the
> >> host has sdl-config in its PATH.
> >>
> >> This occurred really early for a clean build and bitbake would bail
> >> out. The sanity check is in meta/classes/sanity.bbclass.
> >
> > I've not hit that problem, probably because the sanity check was not run
> > again when I changed ASSUME_PROVIDED. I can reproduce it in a clean
> > build directory without conf/sanity_info.
> >
> > I think extending HOSTTOOLS merely to satisfy sanity.bbclass is the
> > wrong solution to the problem. It makes sdl-config available to all
> > recipes, which is unnecessary and potentially introduces back host
> > contamination.
> >
> > It is unnecessary because the qemu recipe has special code that enables
> > the use of the host SDL when told to do so via ASSUME_PROVIDED.
> >
> > Can you come up with a better solution, probably by patching
> > sanity.bbclass?
>
> I can't think of any at this stage.
Here's what qemu.inc does:
do_configure_prepend_class-native() {
# Append build host pkg-config paths for native target since the host
may provide sdl
BHOST_PKGCONFIG_PATH=$(PATH=/usr/bin:/bin pkg-config --variable pc_path
pkg-config || echo "")
if [ ! -z "$BHOST_PKGCONFIG_PATH" ]; then
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$BHOST_PKGCONFIG_PATH
fi
insanity.bbclass could use the host pkg-config to ensure that sdl.pc is
installed.
> Feel free to post a patch if you come up with something better.
Sorry, I don't have time for that. I've filed
https://bugzilla.yoctoproject.org/show_bug.cgi?id=11725 so that we don't
forget about it.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 6/6] yocto-compat-layer.py: make signature check code reusable
This moves the main content of test_signature into a helper
function. It can be reused by arbitrary tests that need to do
a before/after signature comparison. Long-term this might even
be useful in oeqa itself.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
scripts/lib/compatlayer/__init__.py | 66 ++-
scripts/lib/compatlayer/cases/common.py | 62 +---
2 files changed, 70 insertions(+), 58 deletions(-)
diff --git a/scripts/lib/compatlayer/__init__.py
b/scripts/lib/compatlayer/__init__.py
index 451e1de..7197e85 100644
--- a/scripts/lib/compatlayer/__init__.py
+++ b/scripts/lib/compatlayer/__init__.py
@@ -324,3 +324,69 @@ def get_depgraph(targets=['world'], failsafe=False):
if depgraph is None:
raise RuntimeError('Could not retrieve the depgraph.')
return depgraph
+
+def compare_signatures(old_sigs, curr_sigs):
+'''
+Compares the result of two get_signatures() calls. Returns None if no
+problems found, otherwise a string that can be used as additional
+explanation in self.fail().
+'''
+# task -> (old signature, new signature)
+sig_diff = {}
+for task in old_sigs:
+if task in curr_sigs and \
+ old_sigs[task] != curr_sigs[task]:
+sig_diff[task] = (old_sigs[task], curr_sigs[task])
+
+if not sig_diff:
+return None
+
+# Beware, depgraph uses task=. whereas get_signatures()
+# uses :. Need to convert sometimes. The output follows
+# the convention from get_signatures() because that seems closer to
+# normal bitbake output.
+def sig2graph(task):
+pn, taskname = task.rsplit(':', 1)
+return pn + '.' + taskname
+def graph2sig(task):
+pn, taskname = task.rsplit('.', 1)
+return pn + ':' + taskname
+depgraph = get_depgraph(failsafe=True)
+depends = depgraph['tdepends']
+
+# If a task A has a changed signature, but none of its
+# dependencies, then we need to report it because it is
+# the one which introduces a change. Any task depending on
+# A (directly or indirectly) will also have a changed
+# signature, but we don't need to report it. It might have
+# its own changes, which will become apparent once the
+# issues that we do report are fixed and the test gets run
+# again.
+sig_diff_filtered = []
+for task, (old_sig, new_sig) in sig_diff.items():
+deps_tainted = False
+for dep in depends.get(sig2graph(task), ()):
+if graph2sig(dep) in sig_diff:
+deps_tainted = True
+break
+if not deps_tainted:
+sig_diff_filtered.append((task, old_sig, new_sig))
+
+msg = []
+msg.append('%d signatures changed, initial differences (first hash before,
second after):' %
+ len(sig_diff))
+for diff in sorted(sig_diff_filtered):
+recipe, taskname = diff[0].rsplit(':', 1)
+cmd = 'bitbake-diffsigs --task %s %s --signature %s %s' % \
+ (recipe, taskname, diff[1], diff[2])
+msg.append(' %s: %s -> %s' % diff)
+msg.append(' %s' % cmd)
+try:
+output = check_command('Determining signature difference failed.',
+ cmd).decode('utf-8')
+except RuntimeError as error:
+output = str(error)
+if output:
+msg.extend([' ' + line for line in output.splitlines()])
+msg.append('')
+return '\n'.join(msg)
diff --git a/scripts/lib/compatlayer/cases/common.py
b/scripts/lib/compatlayer/cases/common.py
index 4c8a543..55e8ba4 100644
--- a/scripts/lib/compatlayer/cases/common.py
+++ b/scripts/lib/compatlayer/cases/common.py
@@ -4,7 +4,7 @@
import glob
import os
import unittest
-from compatlayer import get_signatures, LayerType, check_command, get_depgraph
+from compatlayer import get_signatures, LayerType, check_command,
get_depgraph, compare_signatures
from compatlayer.case import OECompatLayerTestCase
class CommonCompatLayer(OECompatLayerTestCase):
@@ -47,61 +47,7 @@ class CommonCompatLayer(OECompatLayerTestCase):
raise unittest.SkipTest("Not testing for signature changes in a
software layer %s." \
% self.tc.layer['name'])
-# task -> (old signature, new signature)
-sig_diff = {}
curr_sigs, _ = get_signatures(self.td['builddir'], failsafe=True)
-for task in self.td['sigs']:
-if task in curr_sigs and \
- self.td['sigs'][task] != curr_sigs[task]:
-sig_diff[task] = (self.td['sigs'][task], curr_sigs[task])
-
-if sig_diff:
-# Beware, depgraph uses task=. whereas
get_signatures()
-# uses :. Need to convert sometimes. The output
follows
-# the convention from get_signatures() because that seems closer to
-# normal bitbake output.
-
[OE-core] [PATCH v2 5/6] yocto-compat-layer.py: allow README with suffix
It may be useful to append a suffix denoting the file format. For
example, README.rst is rendered differently when viewed on Github, and
also helps editors to switch to a mode more suitable for the format.
The tests uses a file pattern to find the README file(s) and treats
the one with the shortest name as the main one which must not be
empty.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
scripts/lib/compatlayer/cases/common.py | 14 ++
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/scripts/lib/compatlayer/cases/common.py
b/scripts/lib/compatlayer/cases/common.py
index ede002d..4c8a543 100644
--- a/scripts/lib/compatlayer/cases/common.py
+++ b/scripts/lib/compatlayer/cases/common.py
@@ -1,6 +1,7 @@
# Copyright (C) 2017 Intel Corporation
# Released under the MIT license (see COPYING.MIT)
+import glob
import os
import unittest
from compatlayer import get_signatures, LayerType, check_command, get_depgraph
@@ -8,15 +9,20 @@ from compatlayer.case import OECompatLayerTestCase
class CommonCompatLayer(OECompatLayerTestCase):
def test_readme(self):
-readme_file = os.path.join(self.tc.layer['path'], 'README')
-self.assertTrue(os.path.isfile(readme_file),
-msg="Layer doesn't contains README file.")
+# The top-level README file may have a suffix (like README.rst or
README.txt).
+readme_files = glob.glob(os.path.join(self.tc.layer['path'],
'README*'))
+self.assertTrue(len(readme_files) > 0,
+msg="Layer doesn't contains README file.")
+# There might be more than one file matching the file pattern above
+# (for example, README.rst and README-COPYING.rst). The one with the
shortest
+# name is considered the "main" one.
+readme_file = sorted(readme_files)[0]
data = ''
with open(readme_file, 'r') as f:
data = f.read()
self.assertTrue(data,
-msg="Layer contains README file but is empty.")
+msg="Layer contains a README file but it is empty.")
def test_parse(self):
check_command('Layer %s failed to parse.' % self.tc.layer['name'],
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 4/6] yocto-compat-layer.py: add test_world
"test_signatures" ignores wold build breakage for the sake of
reporting differences also when a world build is broken. Therefore we
need a dedicated test that a world build at least theoretically can
proceed without obvious parse time problems (dependencies, parse
errors, dangling .bbappends, etc.).
This is similar to the BSP test_machine_world. The difference is
that test_world doesn't change the MACHINE.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
scripts/lib/compatlayer/cases/common.py | 9 +
1 file changed, 9 insertions(+)
diff --git a/scripts/lib/compatlayer/cases/common.py
b/scripts/lib/compatlayer/cases/common.py
index a1cdbab..ede002d 100644
--- a/scripts/lib/compatlayer/cases/common.py
+++ b/scripts/lib/compatlayer/cases/common.py
@@ -26,6 +26,15 @@ class CommonCompatLayer(OECompatLayerTestCase):
check_command('Layer %s failed to show environment.' %
self.tc.layer['name'],
'bitbake -e')
+def test_world(self):
+'''
+"bitbake world" is expected to work. test_signatures does not cover
that
+because it is more lenient and ignores recipes in a world build that
+are not actually buildable, so here we fail when "bitbake -S none
world"
+fails.
+'''
+get_signatures(self.td['builddir'], failsafe=False)
+
def test_signatures(self):
if self.tc.layer['type'] == LayerType.SOFTWARE and \
not self.tc.test_software_layer_signatures:
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 2/6] yocto-compat-layer.py: tolerate broken world builds during signature diff
The "test_signatures" test ignored a broken world build when getting
signatures, but the code which then tried to analyze a difference
found by the test didn't, which prevented printing the difference.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
scripts/lib/compatlayer/__init__.py | 7 ++-
scripts/lib/compatlayer/cases/common.py | 2 +-
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/scripts/lib/compatlayer/__init__.py
b/scripts/lib/compatlayer/__init__.py
index eaae4e5..451e1de 100644
--- a/scripts/lib/compatlayer/__init__.py
+++ b/scripts/lib/compatlayer/__init__.py
@@ -290,7 +290,7 @@ def get_signatures(builddir, failsafe=False, machine=None):
return (sigs, tune2tasks)
-def get_depgraph(targets=['world']):
+def get_depgraph(targets=['world'], failsafe=False):
'''
Returns the dependency graph for the given target(s).
The dependency graph is taken directly from DepTreeEvent.
@@ -309,6 +309,11 @@ def get_depgraph(targets=['world']):
elif isinstance(event, bb.command.CommandCompleted):
break
elif isinstance(event, bb.event.NoProvider):
+if failsafe:
+# The event is informational, we will get information
about the
+# remaining dependencies eventually and thus can
ignore this
+# here like we do in get_signatures(), if desired.
+continue
if event._reasons:
raise RuntimeError('Nothing provides %s: %s' %
(event._item, event._reasons))
else:
diff --git a/scripts/lib/compatlayer/cases/common.py
b/scripts/lib/compatlayer/cases/common.py
index 8eeada9..2dfcbb1 100644
--- a/scripts/lib/compatlayer/cases/common.py
+++ b/scripts/lib/compatlayer/cases/common.py
@@ -50,7 +50,7 @@ class CommonCompatLayer(OECompatLayerTestCase):
def graph2sig(task):
pn, taskname = task.rsplit('.', 1)
return pn + ':' + taskname
-depgraph = get_depgraph()
+depgraph = get_depgraph(failsafe=True)
depends = depgraph['tdepends']
# If a task A has a changed signature, but none of its
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 3/6] yocto-compat-layer.py: apply test_signatures to all layers
Software layers were previously allowed to change signatures, but
that's not desired for those layers either. The rule that a layer
which is "Yocto Compatible 2.0" must not change signatures unless
explicitly requested holds for all kinds of layers.
However, as this is something that software layers might not be able
to do right away, testing for signature changes in software layers can
be disabled. It's on by default, as that was Richard's
recommendation. Whether that should change needs further discussion as
part of finalizing "Yocto Compatible 2.0".
As it might still change, the tool now has both a with/without
parameter so that users of the tool can choose the desired behavior
without being affected by future changes to the default.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
scripts/lib/compatlayer/cases/common.py | 5 +++--
scripts/lib/compatlayer/context.py | 3 ++-
scripts/yocto-compat-layer.py | 12 +---
3 files changed, 14 insertions(+), 6 deletions(-)
diff --git a/scripts/lib/compatlayer/cases/common.py
b/scripts/lib/compatlayer/cases/common.py
index 2dfcbb1..a1cdbab 100644
--- a/scripts/lib/compatlayer/cases/common.py
+++ b/scripts/lib/compatlayer/cases/common.py
@@ -27,8 +27,9 @@ class CommonCompatLayer(OECompatLayerTestCase):
'bitbake -e')
def test_signatures(self):
-if self.tc.layer['type'] == LayerType.SOFTWARE:
-raise unittest.SkipTest("Layer %s isn't BSP or DISTRO one." \
+if self.tc.layer['type'] == LayerType.SOFTWARE and \
+ not self.tc.test_software_layer_signatures:
+raise unittest.SkipTest("Not testing for signature changes in a
software layer %s." \
% self.tc.layer['name'])
# task -> (old signature, new signature)
diff --git a/scripts/lib/compatlayer/context.py
b/scripts/lib/compatlayer/context.py
index 4932238..7811d4a 100644
--- a/scripts/lib/compatlayer/context.py
+++ b/scripts/lib/compatlayer/context.py
@@ -9,6 +9,7 @@ import re
from oeqa.core.context import OETestContext
class CompatLayerTestContext(OETestContext):
-def __init__(self, td=None, logger=None, layer=None):
+def __init__(self, td=None, logger=None, layer=None,
test_software_layer_signatures=True):
super(CompatLayerTestContext, self).__init__(td, logger)
self.layer = layer
+self.test_software_layer_signatures = test_software_layer_signatures
diff --git a/scripts/yocto-compat-layer.py b/scripts/yocto-compat-layer.py
index 30c55a9..a16974f 100755
--- a/scripts/yocto-compat-layer.py
+++ b/scripts/yocto-compat-layer.py
@@ -30,12 +30,12 @@ CASES_PATHS =
[os.path.join(os.path.abspath(os.path.dirname(__file__)),
'lib', 'compatlayer', 'cases')]
logger = scriptutils.logger_create(PROGNAME, stream=sys.stdout)
-def test_layer_compatibility(td, layer):
+def test_layer_compatibility(td, layer, test_software_layer_signatures):
from compatlayer.context import CompatLayerTestContext
logger.info("Starting to analyze: %s" % layer['name'])
logger.info("--")
-tc = CompatLayerTestContext(td=td, logger=logger, layer=layer)
+tc = CompatLayerTestContext(td=td, logger=logger, layer=layer,
test_software_layer_signatures=test_software_layer_signatures)
tc.loadTests(CASES_PATHS)
return tc.runTests()
@@ -53,6 +53,12 @@ def main():
help='List of MACHINEs to be used during testing', action='store')
parser.add_argument('--additional-layers', nargs="+",
help='List of additional layers to add during testing',
action='store')
+group = parser.add_mutually_exclusive_group()
+group.add_argument('--with-software-layer-signature-check',
action='store_true', dest='test_software_layer_signatures',
+ default=True,
+ help='check that software layers do not change
signatures (on by default)')
+group.add_argument('--without-software-layer-signature-check',
action='store_false', dest='test_software_layer_signatures',
+ help='disable signature checking for software layers')
parser.add_argument('-n', '--no-auto', help='Disable auto layer discovery',
action='store_true')
parser.add_argument('-d', '--debug', help='Enable debug output',
@@ -173,7 +179,7 @@ def main():
layers_tested = layers_tested + 1
continue
-result = test_layer_compatibility(td, layer)
+result = test_layer_compatibility(td, layer,
args.test_software_layer_signatures)
results[layer['name']] = result
results_status[layer['name']] = 'PASS' if
results[layer['name']].wasSuccessful() else 'FAIL'
layers_tested = layers_tested + 1
--
git-series 0.9.1
--
_
[OE-core] [PATCH v2 1/6] yocto-compat-layer.py: avoid adding layers more than once
add_layer_dependencies() might get called more than once, or one of
the layer dependencies might already be present. The function should
not add layers again because doing so can cause warnings like:
WARNING: Duplicate inclusion for
.../meta-openembedded/meta-oe/conf/distro/include/meta_oe_security_flags.inc in
.../meta-openembedded/meta-oe/conf/layer.conf
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
scripts/lib/compatlayer/__init__.py | 17 +++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/scripts/lib/compatlayer/__init__.py
b/scripts/lib/compatlayer/__init__.py
index e35f8c0..eaae4e5 100644
--- a/scripts/lib/compatlayer/__init__.py
+++ b/scripts/lib/compatlayer/__init__.py
@@ -4,6 +4,7 @@
# Released under the MIT license (see COPYING.MIT)
import os
+import re
import subprocess
from enum import Enum
@@ -189,10 +190,22 @@ def add_layer_dependencies(bblayersconf, layer, layers,
logger):
if layer_depends is None:
return False
else:
+# Don't add a layer that is already present.
+added = set()
+output = check_command('Getting existing layers failed.',
'bitbake-layers show-layers').decode('utf-8')
+for layer, path, pri in re.findall(r'^(\S+) +([^\n]*?) +(\d+)$',
output, re.MULTILINE):
+added.add(path)
+
for layer_depend in layer_depends:
-logger.info('Adding layer dependency %s' % layer_depend['name'])
+name = layer_depend['name']
+path = layer_depend['path']
+if path in added:
+continue
+else:
+added.add(path)
+logger.info('Adding layer dependency %s' % name)
with open(bblayersconf, 'a+') as f:
-f.write("\nBBLAYERS += \"%s\"\n" % layer_depend['path'])
+f.write("\nBBLAYERS += \"%s\"\n" % path)
return True
def add_layer(bblayersconf, layer, layers, logger):
--
git-series 0.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 0/6] yocto-compat-layer.py: various enhancements
While enhancing the layer structure in intel-iot-refkit I ran into various cases where enhancements to the tool were necessary. intel-iot-refkit now has oe-selftests that check all layers using this tool. In addition, the signature checking utility code is imported into a custom test that also covers a refkit specific case (including refkit-config.inc must not change signatures). Changes in V2: - turn signature checking in software layers on/off via command line flags Patrick Ohly (6): yocto-compat-layer.py: avoid adding layers more than once yocto-compat-layer.py: tolerate broken world builds during signature diff yocto-compat-layer.py: apply test_signatures to all layers yocto-compat-layer.py: add test_world yocto-compat-layer.py: allow README with suffix yocto-compat-layer.py: make signature check code reusable scripts/lib/compatlayer/__init__.py | 90 - scripts/lib/compatlayer/cases/common.py | 92 +++--- scripts/lib/compatlayer/context.py | 3 +- scripts/yocto-compat-layer.py | 12 ++- 4 files changed, 125 insertions(+), 72 deletions(-) base-commit: 20b3574749420a1fef2cb2e0579584453dd4c5c5 -- git-series 0.9.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] commands.py: live output logging + result.error encoding fix
Tests that use bitbake("my-test-image") can run for a long time
without any indication to the user of oe-selftest about what's going
on. The test author has to log the bitbake output explicitly,
otherwise it is lost in case of test failures.
Now it is possible to use bitbake("my-test-image",
output_log=self.logger) to get more output both on the console and in
the XML output (when xmlrunner is installed). Example output:
2017-06-23 12:23:14,144 - oe-selftest - INFO - Running tests...
2017-06-23 12:23:14,145 - oe-selftest - INFO -
--
2017-06-23 12:23:14,151 - oe-selftest - INFO - Running: bitbake my-test-image
2017-06-23 12:23:16,363 - oe-selftest - INFO - Loading cache...done.
2017-06-23 12:23:17,575 - oe-selftest - INFO - Loaded 3529 entries from
dependency cache.
2017-06-23 12:23:18,811 - oe-selftest - INFO - Parsing recipes...done.
2017-06-23 12:23:19,659 - oe-selftest - INFO - Parsing of 2617 .bb files
complete (2612 cached, 5 parsed). 3533 targets, 460 skipped, 0 masked, 0 errors.
2017-06-23 12:23:19,659 - oe-selftest - INFO - NOTE: Resolving any missing task
queue dependencies
Because the implementation was already using threading, the same is
done to decouple reading and writing the different pipes instead of
trying to multiplex IO in a single thread. Previously the helper
thread waited for command completion, now that is done in the main
thread.
The most common case (no input data, joined stdout/stderr) still uses
one extra thread and a single read(), so performance should be roughly
the same as before.
Probably unintentionally, result.error was left as byte string when
migrating to Python3. OE-core doesn't seem to use runCmd() with split
output at the moment, so changing result.error to be treated the same
as result.output (i.e. decoded to a normal strings) seems like a
relatively safe API change (or rather, implementation fix).
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
merge: wait()
---
meta/lib/oeqa/utils/commands.py | 107 ++---
1 file changed, 85 insertions(+), 22 deletions(-)
diff --git a/meta/lib/oeqa/utils/commands.py b/meta/lib/oeqa/utils/commands.py
index 57286fc..5e53454 100644
--- a/meta/lib/oeqa/utils/commands.py
+++ b/meta/lib/oeqa/utils/commands.py
@@ -13,6 +13,7 @@ import sys
import signal
import subprocess
import threading
+import time
import logging
from oeqa.utils import CommandError
from oeqa.utils import ftools
@@ -25,7 +26,7 @@ except ImportError:
pass
class Command(object):
-def __init__(self, command, bg=False, timeout=None, data=None, **options):
+def __init__(self, command, bg=False, timeout=None, data=None,
output_log=None, **options):
self.defaultopts = {
"stdout": subprocess.PIPE,
@@ -48,41 +49,103 @@ class Command(object):
self.options.update(options)
self.status = None
+# We collect chunks of output before joining them at the end.
+self._output_chunks = []
+self._error_chunks = []
self.output = None
self.error = None
-self.thread = None
+self.threads = []
+self.output_log = output_log
self.log = logging.getLogger("utils.commands")
def run(self):
self.process = subprocess.Popen(self.cmd, **self.options)
-def commThread():
-self.output, self.error = self.process.communicate(self.data)
-
-self.thread = threading.Thread(target=commThread)
-self.thread.start()
+def readThread(output, stream, logfunc):
+if logfunc:
+for line in stream:
+output.append(line)
+logfunc(line.decode("utf-8", errors='replace').rstrip())
+else:
+output.append(stream.read())
+
+def readStderrThread():
+readThread(self._error_chunks, self.process.stderr,
self.output_log.error if self.output_log else None)
+
+def readStdoutThread():
+readThread(self._output_chunks, self.process.stdout,
self.output_log.info if self.output_log else None)
+
+def writeThread():
+try:
+self.process.stdin.write(self.data)
+self.process.stdin.close()
+except OSError as ex:
+# It's not an error when the command does not consume all
+# of our data. subprocess.communicate() also ignores that.
+if ex.errno != EPIPE:
+raise
+
+# We write in a separate thread because then we can read
+# without worrying about deadlocks. The additional thread is
+# expected to terminate by itself and we mark it as a daemon,
+# so even it should happen to not terminate for whatever
+# reason, the main process will still exit, which will then
[OE-core] [PATCH 2/2] runcmd.py: unit testing for runCmd()
This covers the traditional API as well as the new output_log feature.
While testing, it was noticed that killing hanging commands does not
work when a shell is used to run the command(s). This might be worth
fixing.
Signed-off-by: Patrick Ohly <patrick.o...@intel.com>
---
meta/lib/oeqa/selftest/cases/runcmd.py | 117 ++-
1 file changed, 117 insertions(+)
create mode 100644 meta/lib/oeqa/selftest/cases/runcmd.py
diff --git a/meta/lib/oeqa/selftest/cases/runcmd.py
b/meta/lib/oeqa/selftest/cases/runcmd.py
new file mode 100644
index 000..6c785ca
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/runcmd.py
@@ -0,0 +1,117 @@
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import runCmd
+from oeqa.utils import CommandError
+
+import subprocess
+import threading
+import time
+import signal
+
+class MemLogger(object):
+def __init__(self):
+self.info_msgs = []
+self.error_msgs = []
+
+def info(self, msg):
+self.info_msgs.append(msg)
+
+def error(self, msg):
+self.error_msgs.append(msg)
+
+class RunCmdTests(OESelftestTestCase):
+""" Basic tests for runCmd() utility function """
+
+# The delta is intentionally smaller than the timeout, to detect cases
where
+# we incorrectly apply the timeout more than once.
+TIMEOUT = 2
+DELTA = 1
+
+def test_result_okay(self):
+result = runCmd("true")
+self.assertEqual(result.status, 0)
+
+def test_result_false(self):
+result = runCmd("false", ignore_status=True)
+self.assertEqual(result.status, 1)
+
+def test_shell(self):
+# A shell is used for all string commands.
+result = runCmd("false; true", ignore_status=True)
+self.assertEqual(result.status, 0)
+
+def test_no_shell(self):
+self.assertRaises(FileNotFoundError,
+ runCmd, "false; true", shell=False)
+
+def test_list_not_found(self):
+self.assertRaises(FileNotFoundError,
+ runCmd, ["false; true"])
+
+def test_list_okay(self):
+result = runCmd(["true"])
+self.assertEqual(result.status, 0)
+
+def test_result_assertion(self):
+self.assertRaisesRegexp(AssertionError, "Command 'echo .* false'
returned non-zero exit status 1:\nfoobar",
+runCmd, "echo foobar >&2; false", shell=True)
+
+def test_result_exception(self):
+self.assertRaisesRegexp(CommandError, "Command 'echo .* false'
returned non-zero exit status 1 with output: foobar",
+runCmd, "echo foobar >&2; false", shell=True,
assert_error=False)
+
+def test_output(self):
+result = runCmd("echo stdout; echo stderr >&2", shell=True)
+self.assertEqual("stdout\nstderr", result.output)
+self.assertEqual("", result.error)
+
+def test_output_split(self):
+result = runCmd("echo stdout; echo stderr >&2", shell=True,
stderr=subprocess.PIPE)
+self.assertEqual("stdout", result.output)
+self.assertEqual("stderr", result.error)
+
+def test_timeout(self):
+numthreads = threading.active_count()
+start = time.time()
+# Killing a hanging process only works when not using a shell?!
+result = runCmd(['sleep', '60'], timeout=self.TIMEOUT,
ignore_status=True)
+self.assertEqual(result.status, -signal.SIGTERM)
+end = time.time()
+self.assertLess(end - start, self.TIMEOUT + self.DELTA)
+self.assertEqual(numthreads, threading.active_count())
+
+def test_timeout_split(self):
+numthreads = threading.active_count()
+start = time.time()
+# Killing a hanging process only works when not using a shell?!
+result = runCmd(['sleep', '60'], timeout=self.TIMEOUT,
ignore_status=True, stderr=subprocess.PIPE)
+self.assertEqual(result.status, -signal.SIGTERM)
+end = time.time()
+self.assertLess(end - start, self.TIMEOUT + self.DELTA)
+self.assertEqual(numthreads, threading.active_count())
+
+def test_stdin(self):
+numthreads = threading.active_count()
+result = runCmd("cat", data=b"hello world", timeout=self.TIMEOUT)
+self.assertEqual("hello world", result.output)
+self.assertEqual(numthreads, threading.active_count())
+
+def test_stdin_timeout(self):
+numthreads = threading.active_count()
+start = time.time()
+result = runCmd(['sleep', '60'], data=b"hello world",
timeout=self.TIMEOUT, ignore_status=True)
+self.assertEqual(result.status, -signal.SIGTERM)
+end = time.time()
+self.assertLess(end - start, self.TIM
[OE-core] [PATCH 0/2] enhance and test runCmd()
This is mostly motivated by some long-running build tests triggered in refkit where better output handling would be very desirable. I had this idea already before Leonardo proposed to change the runCmd() defaults, but as that's now a hot topic, it seemed like a good time to actually implement it. The result.error encoding fix is something that Leonardo also had in his "[OE-core] [PATCH 2/2] selftest/cases: use stderr data when querying for errors" patch. Beware that this patch series conflicts with those patches because that same code is changed and because the current default semantic (joined stdout/stderr) gets explicitly tested. Change in V2: - fix race condition between "process has closed stdout/stderr" and "process has terminated" by calling wait() instead of poll() Patrick Ohly (2): commands.py: live output logging + result.error encoding fix runcmd.py: unit testing for runCmd() meta/lib/oeqa/selftest/cases/runcmd.py | 117 ++- meta/lib/oeqa/utils/commands.py| 107 +++- 2 files changed, 202 insertions(+), 22 deletions(-) create mode 100644 meta/lib/oeqa/selftest/cases/runcmd.py base-commit: 563cab8e823c3fde8ae4785ceaf4d68a5d3e25df -- git-series 0.9.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2] bitbake.conf: Add sdl-config to HOSTTOOLS if using host SDL
On Tue, 2017-06-27 at 20:24 +1000, Jonathan Liu wrote: > Hi Patrick, > > The original problem was that bitbake would print out the error: > "libsdl-native is set to be ASSUME_PROVIDED but sdl-config can't be > found in PATH. Please either install it, or configure qemu not to > require sdl.", if "libsdl-native" was in ASSUME_PROVIDED even if the > host has sdl-config in its PATH. > > This occurred really early for a clean build and bitbake would bail > out. The sanity check is in meta/classes/sanity.bbclass. I've not hit that problem, probably because the sanity check was not run again when I changed ASSUME_PROVIDED. I can reproduce it in a clean build directory without conf/sanity_info. I think extending HOSTTOOLS merely to satisfy sanity.bbclass is the wrong solution to the problem. It makes sdl-config available to all recipes, which is unnecessary and potentially introduces back host contamination. It is unnecessary because the qemu recipe has special code that enables the use of the host SDL when told to do so via ASSUME_PROVIDED. Can you come up with a better solution, probably by patching sanity.bbclass? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2] bitbake.conf: Add sdl-config to HOSTTOOLS if using host SDL
On Tue, 2017-06-27 at 11:05 +0200, Patrick Ohly wrote: > Even if you had checked the right variable, is that really necessary? > I'm building qemu with ASSUME_PROVIDED += "libsdl-native" just fine on > Debian Jessie, without sdl-config in HOSTTOOLS. I'm still interested in learning what problem it fixes. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2] bitbake.conf: Add sdl-config to HOSTTOOLS if using host SDL
On Tue, 2017-06-27 at 19:53 +1000, Jonathan Liu wrote: > Hi Patrick, > > Something is really strange. HOSTTOOLS doesn't appear to be working > anymore in pyro branch > If I do "bitbake -c devshell qemu-native", I can run the following > commands successfully which should have been filtered out by > HOSTTOOLS: > > $ which sdl-config > /usr/bin/sdl-config > $ which ncdu > /usr/bin/ncdu > $ which whoami > /usr/bin/whoami > > For some reason /usr/bin is contained in PATH. Previously it was filtered out. I think that's done intentionally for devshell (and only for devshell) because developers expect to have access to all host tools. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
