Re: [OE-core] [pyro][rocko][PATCH] base.bbclass: fix do_unpack[cleandirs] varflag handling

[Enrico Joerns]

On 03/27/2018 04:27 PM, akuster808 wrote:

On 03/27/2018 03:40 AM, Enrico Joerns wrote:

Ping!

If there should be severe changes in the recent base / kernel class
code that might make this useless, this is at least a candidate for
backporting!


This has to be in master first before  I can back port. Since you have
the setup that shows the problem, maybe you could double check on master
to see if it is also affected.


The patch was created against master and is still valid for current 
master. Thus, of course master is also affected!


I makes every line containing a `do_unpack[cleandirs] += ...` in a 
recipe being silently ignored (such as in kernel.bbclass).


Regards, Enrico


On 01/31/2018 10:49 AM, Enrico Joerns wrote:

Hey,

any comments on that? Not using this patch currently disturbs our
companies Yocto kernel workflow. Thus I wonder if no one else have
had issues when building kernels from fully custom 'defconfig' files,
too.

Regards, Enrico

On 01/21/2018 12:44 AM, Enrico Jorns wrote:

As introduced by a56fb90dc3805494eeaf04c60538425e8d52efc5
('base.bbclass
wipe ${S} before unpacking source') the base.bbclass uses a python
anonymous function to set the 'do_unpack' varflag 'cleandirs' to either
'${S}' or '${S}/patches' depending on equality of '${S}' and
'${WORKDIR}'.

Not that this only differs from the way almost all other recipes set or
modify a tasks 'cleandirs' flag, it also has a significant impact on
the
kernel.bbclass (and possibly further ones) and causes incorrect
behavior for rebuilds triggered by source modification, e.g. by a
change
of the defconfig file for a kernel build.

The kernel.bbclass tries to extend do_unpack[cleandirs]:

| do_unpack[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B}
${STAGING_KERNEL_BUILDDIR}"

As python anonymous functions are evaluated at the very end of recipe
parsing, the d.setVarFlag('do_unpack', 'cleandirs', '${S}')
statement in
base.bbclass will overwrite every modification to cleandirs that is
done
as shown for the kernel class above.

As a result of this, a change to a kernels 'defconfig' will lead to an
updated defconfig file in ${WORKDIR}, but as ${B} never gets cleaned
and
${B}/.config still exists, it will not be copied to ${B}/.config and
thus not find its way in the build kernel.

This is a severe issue for the kernel development and build process!

This patch changes setting of the cleandirs varflag in base.bbclass to
a simple variable assignment as almost all other recipes do it. This
now
again allows overwriting or appending the varflag with common methods
such as done in kernel.bbclass.

This issue affects morty, pyro, rocko and master.

Signed-off-by: Enrico Jorns <e...@pengutronix.de>
---
   meta/classes/base.bbclass | 8 ++--
   1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 912e81e002..2949b074d8 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -152,12 +152,8 @@ python base_do_fetch() {
   addtask unpack after do_fetch
   do_unpack[dirs] = "${WORKDIR}"
-python () {
-    if d.getVar('S') != d.getVar('WORKDIR'):
-    d.setVarFlag('do_unpack', 'cleandirs', '${S}')
-    else:
-    d.setVarFlag('do_unpack', 'cleandirs', os.path.join('${S}',
'patches'))
-}
+do_unpack[cleandirs] = "${@d.getVar('S') if d.getVar('S') !=
d.getVar('WORKDIR') else os.path.join('${S}', 'patches')}"
+
   python base_do_unpack() {
   src_uri = (d.getVar('SRC_URI') or "").split()
   if len(src_uri) == 0:










--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [pyro][rocko][PATCH] base.bbclass: fix do_unpack[cleandirs] varflag handling

[Enrico Joerns]

Ping!

If there should be severe changes in the recent base / kernel class code 
that might make this useless, this is at least a candidate for backporting!


/Enrico

On 01/31/2018 10:49 AM, Enrico Joerns wrote:

Hey,

any comments on that? Not using this patch currently disturbs our 
companies Yocto kernel workflow. Thus I wonder if no one else have had 
issues when building kernels from fully custom 'defconfig' files, too.


Regards, Enrico

On 01/21/2018 12:44 AM, Enrico Jorns wrote:

As introduced by a56fb90dc3805494eeaf04c60538425e8d52efc5 ('base.bbclass
wipe ${S} before unpacking source') the base.bbclass uses a python
anonymous function to set the 'do_unpack' varflag 'cleandirs' to either
'${S}' or '${S}/patches' depending on equality of '${S}' and 
'${WORKDIR}'.


Not that this only differs from the way almost all other recipes set or
modify a tasks 'cleandirs' flag, it also has a significant impact on the
kernel.bbclass (and possibly further ones) and causes incorrect
behavior for rebuilds triggered by source modification, e.g. by a change
of the defconfig file for a kernel build.

The kernel.bbclass tries to extend do_unpack[cleandirs]:

| do_unpack[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} 
${STAGING_KERNEL_BUILDDIR}"


As python anonymous functions are evaluated at the very end of recipe
parsing, the d.setVarFlag('do_unpack', 'cleandirs', '${S}') statement in
base.bbclass will overwrite every modification to cleandirs that is done
as shown for the kernel class above.

As a result of this, a change to a kernels 'defconfig' will lead to an
updated defconfig file in ${WORKDIR}, but as ${B} never gets cleaned and
${B}/.config still exists, it will not be copied to ${B}/.config and
thus not find its way in the build kernel.

This is a severe issue for the kernel development and build process!

This patch changes setting of the cleandirs varflag in base.bbclass to
a simple variable assignment as almost all other recipes do it. This now
again allows overwriting or appending the varflag with common methods
such as done in kernel.bbclass.

This issue affects morty, pyro, rocko and master.

Signed-off-by: Enrico Jorns <e...@pengutronix.de>
---
  meta/classes/base.bbclass | 8 ++--
  1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 912e81e002..2949b074d8 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -152,12 +152,8 @@ python base_do_fetch() {
  addtask unpack after do_fetch
  do_unpack[dirs] = "${WORKDIR}"
-python () {
-    if d.getVar('S') != d.getVar('WORKDIR'):
-    d.setVarFlag('do_unpack', 'cleandirs', '${S}')
-    else:
-    d.setVarFlag('do_unpack', 'cleandirs', os.path.join('${S}', 
'patches'))

-}
+do_unpack[cleandirs] = "${@d.getVar('S') if d.getVar('S') != 
d.getVar('WORKDIR') else os.path.join('${S}', 'patches')}"

+
  python base_do_unpack() {
  src_uri = (d.getVar('SRC_URI') or "").split()
  if len(src_uri) == 0:





--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] base.bbclass: fix do_unpack[cleandirs] varflag handling

[Enrico Joerns]

Hey,

any comments on that? Not using this patch currently disturbs our 
companies Yocto kernel workflow. Thus I wonder if no one else have had 
issues when building kernels from fully custom 'defconfig' files, too.


Regards, Enrico

On 01/21/2018 12:44 AM, Enrico Jorns wrote:

As introduced by a56fb90dc3805494eeaf04c60538425e8d52efc5 ('base.bbclass
wipe ${S} before unpacking source') the base.bbclass uses a python
anonymous function to set the 'do_unpack' varflag 'cleandirs' to either
'${S}' or '${S}/patches' depending on equality of '${S}' and '${WORKDIR}'.

Not that this only differs from the way almost all other recipes set or
modify a tasks 'cleandirs' flag, it also has a significant impact on the
kernel.bbclass (and possibly further ones) and causes incorrect
behavior for rebuilds triggered by source modification, e.g. by a change
of the defconfig file for a kernel build.

The kernel.bbclass tries to extend do_unpack[cleandirs]:

| do_unpack[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} 
${STAGING_KERNEL_BUILDDIR}"

As python anonymous functions are evaluated at the very end of recipe
parsing, the d.setVarFlag('do_unpack', 'cleandirs', '${S}') statement in
base.bbclass will overwrite every modification to cleandirs that is done
as shown for the kernel class above.

As a result of this, a change to a kernels 'defconfig' will lead to an
updated defconfig file in ${WORKDIR}, but as ${B} never gets cleaned and
${B}/.config still exists, it will not be copied to ${B}/.config and
thus not find its way in the build kernel.

This is a severe issue for the kernel development and build process!

This patch changes setting of the cleandirs varflag in base.bbclass to
a simple variable assignment as almost all other recipes do it. This now
again allows overwriting or appending the varflag with common methods
such as done in kernel.bbclass.

This issue affects morty, pyro, rocko and master.

Signed-off-by: Enrico Jorns 
---
  meta/classes/base.bbclass | 8 ++--
  1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 912e81e002..2949b074d8 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -152,12 +152,8 @@ python base_do_fetch() {
  addtask unpack after do_fetch
  do_unpack[dirs] = "${WORKDIR}"
  
-python () {

-if d.getVar('S') != d.getVar('WORKDIR'):
-d.setVarFlag('do_unpack', 'cleandirs', '${S}')
-else:
-d.setVarFlag('do_unpack', 'cleandirs', os.path.join('${S}', 'patches'))
-}
+do_unpack[cleandirs] = "${@d.getVar('S') if d.getVar('S') != d.getVar('WORKDIR') 
else os.path.join('${S}', 'patches')}"
+
  python base_do_unpack() {
  src_uri = (d.getVar('SRC_URI') or "").split()
  if len(src_uri) == 0:



--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH morty] wic: partition: Add fsck to avoid corrupt EXT file systems

[Enrico Joerns]

Hi Daniel,

On 03/30/2017 02:30 PM, Daniel Schultz wrote:

This patch avoids the creation of a corrupt EXT file system.

Since there are no checks if a EXT file system was successfully created,
this should add to prevent possible system failures.

Signed-off-by: Daniel Schultz 
---
 scripts/lib/wic/partition.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 3b3bd2d..152eb7b 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -239,6 +239,9 @@ class Partition():
 (self.fstype, extra_imagecmd, rootfs, label_str, rootfs_dir)
 exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)

+mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
+exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
+
 def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir,
  native_sysroot, pseudo):
 """



maybe it's worth noting this check should not primary target fixing a 
corrupt ext4 (bad if mkfs created on!) but performs some directory 
optimization on the build host that otherwise would be performed on the 
target the next time something triggers an fsck.
Performing those optimizations on the target will also make fsck trigger 
a reboot!


As the options as provided will not fix a real corrupted EXT it might 
also be useful to change the patch description to not confuse about the 
impact of the change.


Regards, Enrico

--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] systemd: make vconsole a PACKAGECONFIG option

[Enrico Joerns]

On 03/29/2017 11:07 AM, Enrico Jorns wrote:

Allowing to remove the systemd-vconsole-setup package without specifying
the --disable-vconsole configure option for systemd will make the system
boot with the failure prompt

| systemd-udevd[142]: failed to execute '/lib/systemd/systemd-vconsole-setup' 
'/lib/systemd/systemd-vconsole-setup': No such file or directory
| systemd-udevd[96]: Process '/lib/systemd/systemd-vconsole-setup' failed with 
exit code 2.

as the 90-vconsole.rules will still be installed with having a
RUN+="/lib/systemd/systemd-vconsole-setup" in it that attempts to
execute a non-existing binary.

Signed-off-by: Enrico Jorns 
---
 meta/recipes-core/systemd/systemd_232.bb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/systemd/systemd_232.bb 
b/meta/recipes-core/systemd/systemd_232.bb
index fa6a6a817f..398cb46f0d 100644
--- a/meta/recipes-core/systemd/systemd_232.bb
+++ b/meta/recipes-core/systemd/systemd_232.bb
@@ -47,6 +47,7 @@ PACKAGECONFIG ??= "xz \
randomseed \
machined \
backlight \
+   vconsole \
quotacheck \
hostnamed \
${@bb.utils.contains('TCLIBC', 'glibc', 'myhostname 
sysusers', '', d)} \
@@ -79,6 +80,7 @@ PACKAGECONFIG[resolved] = 
"--enable-resolved,--disable-resolved"
 PACKAGECONFIG[networkd] = "--enable-networkd,--disable-networkd"
 PACKAGECONFIG[machined] = "--enable-machined,--disable-machined"
 PACKAGECONFIG[backlight] = "--enable-backlight,--disable-backlight"
+PACKAGECONFIG[vconsole] = 
"--enable-vconsole,--disable-vconsole,,${PN}-vconsole-setup"
 PACKAGECONFIG[quotacheck] = "--enable-quotacheck,--disable-quotacheck"
 PACKAGECONFIG[hostnamed] = "--enable-hostnamed,--disable-hostnamed"
 PACKAGECONFIG[myhostname] = "--enable-myhostname,--disable-myhostname"
@@ -475,7 +477,6 @@ RDEPENDS_${PN} += "kmod dbus util-linux-mount udev (= 
${EXTENDPKGV})"
 RDEPENDS_${PN} += "volatile-binds update-rc.d"

 RRECOMMENDS_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 
'serial-getty-generator', '', 'systemd-serialgetty', d)} \
-  systemd-vconsole-setup \
   systemd-extra-utils \
   systemd-compat-units udev-hwdb \
   util-linux-agetty  util-linux-fsck e2fsprogs-e2fsck \



Ooops, forgot the maintainer CC ;)

Enrico

--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] image_types: perform fsck on created ext image

[Enrico Joerns]

Hi Daniel,

On 03/27/2017 09:37 AM, Daniel Schultz wrote:

I have sent fixes for the same problem on Monday.
These include a fix of the wrong return code after optimizations (now it
returns 0) and an fsck execution after the creation of EXT partitions.

The patch which is identical to yours wasn't applied so far, the other
ones were.


ouh, we should have checked that before, but we did not assume that
someone would have posted a fix for this relatively old behavior during
the last days. Reality disabused us... Anyway, good to see that we are
not the only one that stumbled across this issue ;)

But, there is another difference between our patches. We use `-pvfD`,
while you use `-fy` as an fsck.extN option.

I would expect `-p` to be sufficient for letting fsck perform minor
optimizations. If you use `-y` instead, fsck will fix also real issues.
This is not what it should do here, even if it will return 1 afterwards.
If the file system was created with errors I guess we should let the
image creation fail without further touching the ext4.

-D will let fsck perform further directory optimizations
-v might be helpful for debugging purpose


Yes, that's better. I will them also apply to the WIC tools.

Do you think the test expression is still needed?
My opinion is, that it will obfuscate the return code of fsck to either
true or false. This could confuse other people. What do you think?


well, as far as I see it, the test expression will only be needed if 
fsck still returns an error code for those uncritical optimizations it 
performs. With the backported patch applied you had in your series i 
guess there should be no need for the test expression, but I did not 
test it yet in this setup.


Best regards, Enrico

--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] image_types: perform fsck on created ext image

[Enrico Joerns]

Hi Daniel,

On 03/24/2017 02:34 PM, Daniel Schultz wrote:

Hi Enrico,

I have sent fixes for the same problem on Monday.
These include a fix of the wrong return code after optimizations (now it
returns 0) and an fsck execution after the creation of EXT partitions.

The patch which is identical to yours wasn't applied so far, the other
ones were.


ouh, we should have checked that before, but we did not assume that 
someone would have posted a fix for this relatively old behavior during 
the last days. Reality disabused us... Anyway, good to see that we are 
not the only one that stumbled across this issue ;)


But, there is another difference between our patches. We use `-pvfD`, 
while you use `-fy` as an fsck.extN option.


I would expect `-p` to be sufficient for letting fsck perform minor 
optimizations. If you use `-y` instead, fsck will fix also real issues. 
This is not what it should do here, even if it will return 1 afterwards. 
If the file system was created with errors I guess we should let the 
image creation fail without further touching the ext4.


-D will let fsck perform further directory optimizations
-v might be helpful for debugging purpose


Best regards

Enrico

--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] Contents of non-rootfs partitions

[Enrico Joerns]

On 11/23/2016 10:24 AM, Maciej Borzęcki wrote:

On Wed, Nov 23, 2016 at 9:40 AM, Enrico Joerns <e...@pengutronix.de> wrote:

On 11/22/2016 12:54 PM, Kristian Amlie wrote:


On 22/11/16 12:10, Patrick Ohly wrote:


...



All of these introduce some special mechanism. Let me propose something
that might integrate better with the existing tooling:

The "rootfs" directory gets redefined as representing the entire virtual
file system. When creating a disk image, it gets split up into different
partitions based on the image configuration.

For example, the /home or /data directories in the rootfs could hold the
content that in some image configurations goes into separate partitions.

The advantage of this approach is that the tooling for staging content
for image creation does not need to be changed. The same staged content
then can be used to create different images, potentially even using
different partition layouts.



That's a very good idea. I think it beats all of my suggestions!



I totally agree with this solution, because it is the same approach we are
already using for several years.

Instead of wic we are using the genimage tool
(http://public.pengutronix.de/software/genimage/) together with a Yocto
class for easing image creation, but its pretty much the same concerning its
base purpose.

We called the option to split up 'mountpoint' while assuming that the
splitted partition will be mounted to the same location in the file system
it was taken from.

An example configuration would look like this (stripped down to the
essential parts):

  image my-disk-image {
  [...]
  partition root {
  image = "rootfs.ext4"
  [...]
  }

  partition home {
  image = "home.ext4"
  [...]
  }
  }

  image rootfs.ext4 {
  [...]
  mountpoint = "/"
  }

  image home.ext4 {
  [...]
  mountpoint = "/home"
  }

Maybe a similar approach could be used in wic, too.


I went through the README from genimage repo
https://git.pengutronix.de/cgit/genimage/tree/README. Definitely looks
interesting and covers storage medias that are not supported by wic
right now. The split option does not seem to be documented though. If I
understood you correctly, the partition is seeded with data from
location specified by `mountpoint` inside IMAGE_ROOTFS. Is that correct?


Yes, what the genimage class (see below) does, is unpacking the final 
rootfs.tar.xz inside a fakeroot environment and call the genimage tool 
with the --rootpath argument set to this path.


The mountpoint="/home" tells genimage to pack all content in directory 
/home into a separate data ext4 image (temporary named 'home.ext4') and 
all other content in a rootfs ext4 image (temporary named 'rootfs.ext4').


The `image` section then specifies how to construct the final disk image 
using the referenced generated rootfs and datafs images (and optionally 
some bootloader, non-partition-data, etc.).



Note that this is only one way to use genimage. Another common way to 
use it is to simply construct complex disk images out of different 
filesystem images, e.g. with


  partition barebox {
  image = "barebox-any-platform.img"
  size = 1M
  in-partition-table = false
  }


I see there's a large overlap in functionality with wic (at least for
block devices), so perhaps the best way would be introduce genimage as
another IMAGE_CMD.

I've quickly checked at https://layers.openembedded.org and there are no
recipes for genimage/libconfuse listed there. Do happen to have done
integration with OE in some private repo already? Would it be possible
for you to post the relevant patches?



We have a layer for this, which is potentially publicly available but 
not advertised as the general plan was to move some of its content to 
upstream (oe-core, meta-oe) and split up other content (e.g. the rauc 
update tool stuff) to separate layers that will be made publicly 
available. But, as always, there are more plans than time left ;)


Well, to make a long story short, the path to the git repository is:

  https://git.pengutronix.de/cgit/meta-ptx

The genimage class I mentioned above you can find in it, too:

  https://git.pengutronix.de/cgit/meta-ptx/tree/classes/genimage.bbclass


When starting with Yocto I thought about changing from genimage to wic 
for disk image creation as wic already seemed to become the standard 
solution for building disk images. But back in these days it was a very 
separate tool and there was no integration with the BSP itself (which is 
crucial when having to reproduce builds). But the main reason why I 
decided to stay with genimage was that wic simply did not cover most of 
the use cases I required for building complex disk images for different 
embedded platforms. I'm pretty sure that this has changed in some 
points, but ha

Re: [OE-core] Contents of non-rootfs partitions

[Enrico Joerns]

On 11/22/2016 12:54 PM, Kristian Amlie wrote:

On 22/11/16 12:10, Patrick Ohly wrote:

...


All of these introduce some special mechanism. Let me propose something
that might integrate better with the existing tooling:

The "rootfs" directory gets redefined as representing the entire virtual
file system. When creating a disk image, it gets split up into different
partitions based on the image configuration.

For example, the /home or /data directories in the rootfs could hold the
content that in some image configurations goes into separate partitions.

The advantage of this approach is that the tooling for staging content
for image creation does not need to be changed. The same staged content
then can be used to create different images, potentially even using
different partition layouts.


That's a very good idea. I think it beats all of my suggestions!


I totally agree with this solution, because it is the same approach we 
are already using for several years.


Instead of wic we are using the genimage tool 
(http://public.pengutronix.de/software/genimage/) together with a Yocto 
class for easing image creation, but its pretty much the same concerning 
its base purpose.


We called the option to split up 'mountpoint' while assuming that the 
splitted partition will be mounted to the same location in the file 
system it was taken from.


An example configuration would look like this (stripped down to the 
essential parts):


  image my-disk-image {
  [...]
  partition root {
  image = "rootfs.ext4"
  [...]
  }

  partition home {
  image = "home.ext4"
  [...]
  }
  }

  image rootfs.ext4 {
  [...]
  mountpoint = "/"
  }

  image home.ext4 {
  [...]
  mountpoint = "/home"
  }

Maybe a similar approach could be used in wic, too.


Best regards, Enrico

--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] systemd: fix useradd parameter generation

[Enrico Joerns]

On 08/09/2016 10:24 AM, Enrico Joerns wrote:

On 08/09/2016 07:17 AM, Anders Darander wrote:

* Enrico Jorns <e...@pengutronix.de> [160808 15:55]:

Constructing the USERADD_PARAM_${PN} value with += (as done before)
resulted in having a whitespace added for each entry not having its
PACKAGECONFIG parameter activated.



This is not only a cosmetical issue. The useradd class concatenates all
useradd commands from all subpackages of a package by doing a
";".join(). When having a useradd command that ends with ";" and some
extra spaces, concatenation will result in an empty useradd argument
("commandA;;commandB"). The useradd command will be called with this
and make the Yocto build break by returning an error code.



Fortunately, there is no problem when having a "commandA;;commandB" in
the useradd string, this will be ignored. Thus using "_append"
instead of
"+=" is sufficient here.


NAK

Fix useradd.bbclass instead. Otherwise, the same issue will happen with
other recipes. Having a single variable, that you're not allowed to use
the normal way to add to, would be more confusing. Personally, I would
find it hard to rember that I need to use .= for USERADD_PARAM...


Yes, I did not have .= in mind when creating this solution, but this
would be the better approach.
Nevertheless, I also prefer a more general solution, but was not sure
about how it could look like. Maybe I've found one now.

So, forget about this patch, I will post a useradd class patch in a
separate thread instead.


Just saw yocto patch 553fa35147adb996d515c3d27f62555d9c969cd8 (oe-core: 
e8d4356c38e3c2aacd6dc49231c73bcb7d597308) seems to address and fix this 
problem. Did not find that before... :-|


--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] systemd: fix useradd parameter generation

[Enrico Joerns]

On 08/09/2016 07:17 AM, Anders Darander wrote:

* Enrico Jorns  [160808 15:55]:

Constructing the USERADD_PARAM_${PN} value with += (as done before)
resulted in having a whitespace added for each entry not having its
PACKAGECONFIG parameter activated.



This is not only a cosmetical issue. The useradd class concatenates all
useradd commands from all subpackages of a package by doing a
";".join(). When having a useradd command that ends with ";" and some
extra spaces, concatenation will result in an empty useradd argument
("commandA;;commandB"). The useradd command will be called with this
and make the Yocto build break by returning an error code.



Fortunately, there is no problem when having a "commandA;;commandB" in
the useradd string, this will be ignored. Thus using "_append" instead of
"+=" is sufficient here.


NAK

Fix useradd.bbclass instead. Otherwise, the same issue will happen with
other recipes. Having a single variable, that you're not allowed to use
the normal way to add to, would be more confusing. Personally, I would
find it hard to rember that I need to use .= for USERADD_PARAM...


Yes, I did not have .= in mind when creating this solution, but this 
would be the better approach.
Nevertheless, I also prefer a more general solution, but was not sure 
about how it could look like. Maybe I've found one now.


So, forget about this patch, I will post a useradd class patch in a 
separate thread instead.


Regards, Enrico

--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [krogoth] systemd: Create missing sysusers offline

[Enrico Joerns]

Hi,

this patch is required, for example, to successfully build systemd with 
networkd enabled in krogoth release, too. Thus it should be backported.


It is in oe-core master, id d18957925c6c073b7194e3a233efea24e436f74e.

Best regards, Enrico

On 04/24/2016 06:52 PM, Khem Raj wrote:

Some system users which are needed by systemd components were missing
create these users knobbed with relevant packageconfig

Signed-off-by: Khem Raj 
---
Changes from v1->v2:
 -Create systemd-journal-remote
 -Create systemd-journal-upload

 meta/recipes-core/systemd/systemd_229.bb | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/systemd/systemd_229.bb 
b/meta/recipes-core/systemd/systemd_229.bb
index c23c749..cf34921 100644
--- a/meta/recipes-core/systemd/systemd_229.bb
+++ b/meta/recipes-core/systemd/systemd_229.bb
@@ -319,10 +319,16 @@ PACKAGES =+ "\
 SYSTEMD_PACKAGES = "${@bb.utils.contains('PACKAGECONFIG', 'binfmt', '${PN}-binfmt', 
'', d)}"
 SYSTEMD_SERVICE_${PN}-binfmt = "systemd-binfmt.service"

-USERADD_PACKAGES = "${PN}"
+USERADD_PACKAGES = "${PN} ${PN}-extra-utils"
 USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'microhttpd', 
'--system -d / -M --shell /bin/nologin systemd-journal-gateway;', '', d)}"
+USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'microhttpd', 
'--system -d / -M --shell /bin/nologin systemd-journal-remote;', '', d)}"
+USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'journal-upload', 
'--system -d / -M --shell /bin/nologin systemd-journal-upload;', '', d)}"
 USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'timesyncd', 
'--system -d / -M --shell /bin/nologin systemd-timesync;', '', d)}"
+USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'networkd', '--system 
-d / -M --shell /bin/nologin systemd-network;', '', d)}"
+USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'coredump', '--system 
-d / -M --shell /bin/nologin systemd-coredump;', '', d)}"
+USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'resolved', '--system 
-d / -M --shell /bin/nologin systemd-resolve;', '', d)}"
 GROUPADD_PARAM_${PN} = "-r lock; -r systemd-journal"
+USERADD_PARAM_${PN}-extra-utils += "--system -d / -M --shell /bin/nologin 
systemd-bus-proxy"

 FILES_${PN}-analyze = "${bindir}/systemd-analyze"




--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] systemd: Create missing sysusers offline

[Enrico Joerns]

Hi,

On 05/19/2016 10:02 AM, Richard Purdie wrote:

On Thu, 2016-05-19 at 08:36 +0100, Richard Purdie wrote:

On Tue, 2016-05-17 at 21:40 -0700, Khem Raj wrote:

Some system users which are needed by systemd components were
missing
create these users knobbed with relevant packageconfig

Signed-off-by: Khem Raj 
---
  meta/recipes-core/systemd/systemd_229.bb | 8 +++-
  1 file changed, 7 insertions(+), 1 deletion(-)


This looks better but there do appear to still be problems:

https://autobuilder.yoctoproject.org/main/builders/nightly-qa-systemd
/b
uilds/776/steps/Running%20Sanity%20Tests/logs/stdio

and

https://autobuilder.yoctoproject.org/main/builders/nightly-qa-systemd
/b
uilds/776/steps/Running%20Sanity%20Tests_2/logs/stdio




I just stumbled over this issue when building systemd with networkd 
enabled in krogoth release.

Shouldn't this patch be backported to krogoth as well?

Regards, Enrico

--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] perl-ptest.inc: fix tar call to prevent objcopy failure

[Enrico Joerns]

Hi Renato,

thanks for the comments on my changes.

On 05/31/2016 01:35 PM, Renato Caldas wrote:

My suggestion is that you submit a new patch with those improvements
on top of the quick fix I made. I suggest you also add quotes to the
--exclude options per tar's man page.


I did not find any hints about quotes, where is it written? It also 
seems to me that the man page in current tar (1.29) is a bit incomplete 
as it does not mention any 'exclude' option, anymore. At least on my 
distro (debian).



You might also want to simplify the commit message a bit. I'm fairly
new to yocto (and my view may be wrong), but this is how I would do
it:
- change the component name from "perl-ptest.inc:" to "perl:"


Maybe "perl-ptest" would be ok, too, as it matches the resulting 
packages name.



- use the commit title to describe the change you made, not exactly
what bug it fixed. Example: "fix tar call according to its man page"
(or something like that)
- describe the change in simpler terms. Taking what you wrote, I would
rewrite it like this:

"The existing tar call on do_install_ptest() did not match the man
page, but worked with older tar versions. The new 1.29 version of tar
has stricter argument handling, and future versions may be even
stricter. Failure to use it according to its manual may result in
arguments being silently ignored and breaking the build."



Yes, sounds good to me. And my original message does not match the 
remaining changes anymore. Is it ok if I take your text, and add you as 
Signed-off for the patch then?




So while changing the position of the "*" fixed it for tar 1.29, your
proposed changes are important to future-proof the perl recipe for
newer tar versions. As such, please do submit a new patch.


I will do so, then.


Best regard, Enrico

--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH V2 1/4] Upgrade systemd to 229

[Enrico Joerns]

On 02/22/2016 10:36 PM, Khem Raj wrote:

diff --git a/meta/recipes-core/systemd/systemd_228.bb 
b/meta/recipes-core/systemd/systemd_229.bb
similarity index 93%
rename from meta/recipes-core/systemd/systemd_228.bb
rename to meta/recipes-core/systemd/systemd_229.bb
index b160816..16af0ac 100644
--- a/meta/recipes-core/systemd/systemd_228.bb
+++ b/meta/recipes-core/systemd/systemd_229.bb
@@ -24,35 +24,40 @@ SECTION = "base/shell"

  inherit useradd pkgconfig autotools perlnative update-rc.d 
update-alternatives qemu systemd ptest gettext bash-completion

-SRCREV = "dd050decb6ad131ebdeabb71c4f9ecb4733269c0"
+SRCREV = "714c62b46379abb7558c544665522aca91691e10"

-PV = "228+git${SRCPV}"
+PV = "229+git${SRCPV}"


While checking systemd version for trying to update to 230 here, I 
discovered that your SRCREV does not match the v229 tag! It is a bunch 
of commits past v229 (unlike, for example the 228 SRCREV which did 
exactly match the v228 tag.).


Was that done on purpose?


Regards, Enrico

--
Pengutronix e.K.   | Enrico Jörns|
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core