[OE-core] [PATCH] runqemu: If using a vmtype image do not add the -no-reboot flag

[Jason Wessel]

There is no way to use runqemu and turn off the no-reboot flag with a
command line argument.  If someone really wants it back it can be
added with the qemuparams="" argument.

Also if you use the "halt -p" from user space, the qemu will exit when
it is complete.  It is impossible to test self deploying image with
runqemu if you cannot reboot the device.  One might argue that you can
run runqemu in a loop, but that defeats the purpose of having a
wrapper around a tool that already does what is needed in the first
place.

For the vmtype images, the -no-reboot flag should not be added.

Signed-off-by: Jason Wessel 
---
 scripts/runqemu | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/runqemu b/scripts/runqemu
index 21680b4..5ae3e5e 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -1241,7 +1241,9 @@ class BaseConfig(object):
 vm_drive = '-drive if=virtio,file=%s,format=%s' % 
(self.rootfs, rootfs_format)
 
 # All branches above set vm_drive.
-self.rootfs_options = '%s -no-reboot' % vm_drive
+self.rootfs_options = vm_drive
+if not self.fstype in self.vmtypes:
+self.rootfs_options += ' -no-reboot'
 self.kernel_cmdline = 'root=%s rw' % (self.get('QB_KERNEL_ROOT'))
 
 if self.fstype == 'nfs':
-- 
2.17.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#139962): 
https://lists.openembedded.org/g/openembedded-core/message/139962
Mute This Topic: https://lists.openembedded.org/mt/75107612/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] module.bbclass: swap AR and LD order

[Jason Wessel]

On 1/24/20 3:19 PM, Khem Raj wrote:
> On Fri, Jan 24, 2020 at 1:05 PM Jason Wessel  
> wrote:
>>
>> On 1/24/20 1:13 PM, Khem Raj wrote:
>>> On 1/24/20 10:19 AM, Christopher Larson wrote:
>>>> What makefile change caused this? That behavior doesn't make much sense
>>>> given how make processes its command-line arguments.
>>>>
>>>
>>> I agree with you here, it could be a rare check where one want to define
>>> what collect progam should be used ( ld or ar )
>>>
>>> besides, recently we moved AR to be gcc-ar/llvm-ar by default in config
>>> metadata, which is not going to work out of box for compiling kernel and
>>> modules, so overriding it with KERNEL_AR in
>>> module_do_compile/install/configure tasks is good change.
>>
>> I am not actually sure which part of kbuild is parsing the LD
>> arguments.  My best guess is that it is actually all the nested
>> gnumake filter calls causing the issue.
>>
>> With some further experimentation I determined that a space at the end
>> of the LD= is what causes it to think the next argument is part of the
>> prior one.  The poky distro is defining KERNEL_LD as:
>>
>> % bitbake vboxguestdrivers -e |grep ^KERNEL_LD
>> KERNEL_LD="x86_64-poky-linux-ld.bfd "
>>
>> The kbuild really doesn't like that space at the end.  You can trash out
>> a host native kernel compile module build by adding the arguments:
>> LD="ld " JUNK_VAR="junk"
>>
>> Attempting to debug/fix kbuild doesn't seem like the right answer.
>>
>> The next version of this patch looks like what is below.  Would that be ok?
>>
>>
>> diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
>> index c0dfa35061..7b0a85a06d 100644
>> --- a/meta/classes/module.bbclass
>> +++ b/meta/classes/module.bbclass
>> @@ -12,6 +12,9 @@ python __anonymous () {
>>  if dep.startswith("kernel-module-"):
>>  extra_symbols.append("${STAGING_INCDIR}/" + dep + 
>> "/Module.symvers")
>>  d.setVar('KBUILD_EXTRA_SYMBOLS', " ".join(extra_symbols))
>> +d.setVar('KERNEL_CC', d.getVar('KERNEL_CC').rstrip())
>> +d.setVar('KERNEL_LD', d.getVar('KERNEL_LD').rstrip())
>> +d.setVar('KERNEL_AR', d.getVar('KERNEL_AR').rstrip())
>>  }
>>
> 
> no one else uses KERNEL_* vars, so if modules cant use it then we have
> an issue, perhaps just delete it in base class where KERNEL_* vars are set
> 


I am not sure I follow what you want to see changed.  Do you want the to try
and fix up the problem where the variables are initially set?  Example:

classes/kernel-arch.bbclass:KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd 
${HOST_LD_KERNEL_ARCH}"

I see there are some BSPs that append to KERNEL_LD, so this problem could happen
at a later point. 

The other possibility is adding an echo to force the trimming without using the 
python.

@@ -41,8 +38,8 @@ module_do_compile() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
oe_runmake KERNEL_PATH=${STAGING_KERNEL_DIR}   \
   KERNEL_VERSION=${KERNEL_VERSION}\
-  CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
-  AR="${KERNEL_AR}" \
+  CC="$(echo ${KERNEL_CC})" LD="$(echo ${KERNEL_LD})" \
+  AR="$(echo ${KERNEL_AR})" \
   O=${STAGING_KERNEL_BUILDDIR} \
   KBUILD_EXTRA_SYMBOLS="${KBUILD_EXTRA_SYMBOLS}" \
   ${MAKE_TARGETS}
@@ -52,7 +49,7 @@ module_do_install() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
oe_runmake DEPMOD=echo 
MODLIB="${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}" \
   INSTALL_FW_PATH="${D}${nonarch_base_libdir}/firmware" \
-  CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
+  CC="$(echo ${KERNEL_CC})" LD="$(echo ${KERNEL_LD})" \
   O=${STAGING_KERNEL_BUILDDIR} \
   ${MODULES_INSTALL_TARGET}
 

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] module.bbclass: swap AR and LD order

[Jason Wessel]

On 1/24/20 1:13 PM, Khem Raj wrote:
> On 1/24/20 10:19 AM, Christopher Larson wrote:
>> What makefile change caused this? That behavior doesn't make much sense 
>> given how make processes its command-line arguments.
>>
> 
> I agree with you here, it could be a rare check where one want to define 
> what collect progam should be used ( ld or ar )
> 
> besides, recently we moved AR to be gcc-ar/llvm-ar by default in config 
> metadata, which is not going to work out of box for compiling kernel and 
> modules, so overriding it with KERNEL_AR in 
> module_do_compile/install/configure tasks is good change.

I am not actually sure which part of kbuild is parsing the LD
arguments.  My best guess is that it is actually all the nested
gnumake filter calls causing the issue.  

With some further experimentation I determined that a space at the end
of the LD= is what causes it to think the next argument is part of the
prior one.  The poky distro is defining KERNEL_LD as:

% bitbake vboxguestdrivers -e |grep ^KERNEL_LD
KERNEL_LD="x86_64-poky-linux-ld.bfd "

The kbuild really doesn't like that space at the end.  You can trash out
a host native kernel compile module build by adding the arguments:
LD="ld " JUNK_VAR="junk"

Attempting to debug/fix kbuild doesn't seem like the right answer. 

The next version of this patch looks like what is below.  Would that be ok? 


diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
index c0dfa35061..7b0a85a06d 100644
--- a/meta/classes/module.bbclass
+++ b/meta/classes/module.bbclass
@@ -12,6 +12,9 @@ python __anonymous () {
 if dep.startswith("kernel-module-"):
 extra_symbols.append("${STAGING_INCDIR}/" + dep + 
"/Module.symvers")
 d.setVar('KBUILD_EXTRA_SYMBOLS', " ".join(extra_symbols))
+d.setVar('KERNEL_CC', d.getVar('KERNEL_CC').rstrip())
+d.setVar('KERNEL_LD', d.getVar('KERNEL_LD').rstrip())
+d.setVar('KERNEL_AR', d.getVar('KERNEL_AR').rstrip())
 }

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] module.bbclass: swap AR and LD order

[Jason Wessel]

The 5.x kernels seem to have made a change to the linker command line 
processing.

When trying to build out of tree kernel modules, such as the
virtualbox guest additions, the following error is printed:

| make[1]: Entering directory 
'/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo'
| make[3]: Entering directory 
'/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
| make V= CONFIG_MODULE_SIG= CONFIG_MODULE_SIG_ALL= -C 
/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/5.2.28-yocto-standard/build
 
M=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest
 
SRCROOT=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest
 modules
| make[3]: Entering directory 
'/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
|   LD [M]  
/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo/vboxvideo.o
| x86_64-poky-linux-ld.bfd: cannot find AR=x86_64-poky-linux-ar: No such file 
or directory

The AR arguments passed to the make command are being processed as
command line arguments to the linker due to some changes in the
kernel's Makefile.  For now the easiest fix is to just swap the
arguments in the module.bbclass.

Signed-off-by: Jason Wessel 
---
 meta/classes/module.bbclass | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
index c0dfa35061..e1e5fdaa72 100644
--- a/meta/classes/module.bbclass
+++ b/meta/classes/module.bbclass
@@ -38,8 +38,7 @@ module_do_compile() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
oe_runmake KERNEL_PATH=${STAGING_KERNEL_DIR}   \
   KERNEL_VERSION=${KERNEL_VERSION}\
-  CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
-  AR="${KERNEL_AR}" \
+  CC="${KERNEL_CC}" AR="${KERNEL_AR}" LD="${KERNEL_LD}" \
   O=${STAGING_KERNEL_BUILDDIR} \
   KBUILD_EXTRA_SYMBOLS="${KBUILD_EXTRA_SYMBOLS}" \
   ${MAKE_TARGETS}
-- 
2.23.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] wic: Using the right rootfs size during prepare_rootfs

[Jason Wessel]

This looks correct now with the two patches merged.

Thanks,
Jason.

On 9/26/19 4:35 AM, Alessio Igor Bogani wrote:

The commit 8e48b4d6c4 makes wic ignores IMAGE_ROOTFS_SIZE for rootfs
size and makes it uses the computed one only. Re-add support for
IMAGE_ROOTFS_SIZE variable and compute roots size only if the former
is not defined. Moreover the size of a provided directory with
--rootfs-dir="" in the .wks file should always be computed on the fly,
else every partition will be constrained to be the same size as what
ever value was in ROOTFS_SIZE.

Signed-off-by: Alessio Igor Bogani 
Signed-off-by: Jason Wessel 
---
  scripts/lib/wic/partition.py | 23 +--
  1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 2a71d7b1d6..d809408e1a 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -212,13 +212,24 @@ class Partition():
  if os.path.isfile(rootfs):
  os.remove(rootfs)
  
-# If size is not specified compute it from the rootfs_dir size

  if not self.size and real_rootfs:
-# Use the same logic found in get_rootfs_size()
-# from meta/classes/image.bbclass
-du_cmd = "du -ks %s" % rootfs_dir
-out = exec_cmd(du_cmd)
-self.size = int(out.split()[0])
+# The rootfs size is not set in .ks file so try to get it
+# from bitbake variable
+rsize_bb = get_bitbake_var('ROOTFS_SIZE')
+rdir = get_bitbake_var('IMAGE_ROOTFS')
+if rsize_bb and rdir == rootfs_dir:
+# Bitbake variable ROOTFS_SIZE is calculated in
+# Image._get_rootfs_size method from meta/lib/oe/image.py
+# using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT,
+# IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE
+self.size = int(round(float(rsize_bb)))
+else:
+# Bitbake variable ROOTFS_SIZE is not defined so compute it
+# from the rootfs_dir size using the same logic found in
+# get_rootfs_size() from meta/classes/image.bbclass
+du_cmd = "du -ks %s" % rootfs_dir
+out = exec_cmd(du_cmd)
+self.size = int(out.split()[0])
  
  prefix = "ext" if self.fstype.startswith("ext") else self.fstype

  method = getattr(self, "prepare_rootfs_" + prefix)



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] wic: Using the right rootfs size during prepare_rootfs

[Jason Wessel]

SIZE is not defined so compute it
+# from the rootfs_dir size using the same logic found in
+# get_rootfs_size() from meta/classes/image.bbclass
+du_cmd = "du -ks %s" % rootfs_dir
+out = exec_cmd(du_cmd)
+self.size = int(out.split()[0])
  
  prefix = "ext" if self.fstype.startswith("ext") else self.fstype

  method = getattr(self, "prepare_rootfs_" + prefix)



>From 04ba59686d6e70d7ae347a8e92667edbb882cb9d Mon Sep 17 00:00:00 2001
From: Jason Wessel 
Date: Wed, 25 Sep 2019 12:31:57 -0700
Subject: [PATCH] wic: Only use ROOTFS_SIZE for the IMAGE_ROOTFS

The size of a provided directory with --rootfs-dir="" in the .wks file
should always be computed on the fly, else every partition will be constrained to be the same size as what ever value was in ROOTFS_SIZE.

Bitbake will provide the ROOTFS_SIZE along with the IMAGE_ROOTFS from
the image.bbclass.

Signed-off-by: Jason Wessel 
---
 scripts/lib/wic/partition.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 81df15f71b..d809408e1a 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -216,7 +216,8 @@ class Partition():
 # The rootfs size is not set in .ks file so try to get it
 # from bitbake variable
 rsize_bb = get_bitbake_var('ROOTFS_SIZE')
-if rsize_bb:
+rdir = get_bitbake_var('IMAGE_ROOTFS')
+if rsize_bb and rdir == rootfs_dir:
 # Bitbake variable ROOTFS_SIZE is calculated in
 # Image._get_rootfs_size method from meta/lib/oe/image.py
 # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT,
-- 
2.23.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] gnupg: Extend -native wrapper to fix gpgme-native's gpgconf problems

[Jason Wessel]

The gpg commit signing in ostree-native doesn't work properly when
running from sstate.  The ostree-native is linked with gpgme-native's
libraries, which have calls into gpg.

Ultimately it turned out the problem was that gpgme calls gpgconf and
some of the other gnupg-native binaries directly.  Not all the
binaries have a wrapper which sets the environment variable GNUPG_BIN.
Without this wrapper these binaries it gets the path assignment from
the original compilation which causes a fault when running from sstate
in a new tmp directory because these paths will not exist.

Signed-off-by: Jason Wessel 
---
 meta/recipes-support/gnupg/gnupg_2.2.17.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-support/gnupg/gnupg_2.2.17.bb 
b/meta/recipes-support/gnupg/gnupg_2.2.17.bb
index e5456dd9b9..689cf8a75e 100644
--- a/meta/recipes-support/gnupg/gnupg_2.2.17.bb
+++ b/meta/recipes-support/gnupg/gnupg_2.2.17.bb
@@ -46,6 +46,9 @@ do_install_append() {
 
 do_install_append_class-native() {
create_wrapper ${D}${bindir}/gpg2 GNUPG_BINDIR=${STAGING_BINDIR_NATIVE}
+   create_wrapper ${D}${bindir}/gpgconf 
GNUPG_BINDIR=${STAGING_BINDIR_NATIVE}
+   create_wrapper ${D}${bindir}/gpg-agent 
GNUPG_BINDIR=${STAGING_BINDIR_NATIVE}
+   create_wrapper ${D}${bindir}/gpg-connect-agent 
GNUPG_BINDIR=${STAGING_BINDIR_NATIVE}
 }
 
 PACKAGECONFIG ??= "gnutls"
-- 
2.23.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] serial-getty@.service: Allow device to fast fail if it does not exist

[Jason Wessel]

Some BSPs use a USB serial port which may or may not actually be
plugged all the time.  It is quite useful to have a USB serial port
have a getty running but it does not make sense to wait for it for 90
seconds before completing the system startup if it might never get
plugged in.  The typical example is that a USB serial device might
only need to be plugged in when debugging, upgrading, or initially
configuring a device.

This change is somewhat subtle.  Systemd uses the "BindsTo" directive
to ensure existence of the device in order to start the service as
well as to terminate the service if the device goes away.  The "After"
directive makes that same relationship stronger.  When used together
this has the undesired side effect that systemd will wait until its
internal time out value of 90 seconds for the device to come on line
before executing a fail operation or letting other tasks and groups
continue.  This is certainly the kind of behavior we want for a disk,
but not for serial ports in general.

The "BindsTo" directive is replaced by the combination of the "PartOf"
and the "ConditionPathExists" directives.  The "After" directive is
unchanged because that will wait for the udev rules to process.  The
"PartOf" directive will issue a stop to the getty service if the
device goes away, similar to the "BindsTo" directive.  The
"ConditionPathExists" is what allows the service to fail fast vs
waiting for the timeout.  When a USB device is not plugged in at boot
you would find a message in the system journal like:

systemd[1]: Condition check resulted in Serial Getty on \
 ttyUSB0 being skipped.

If you want to observe the problem with qemu, it is easy to replicate.
Simply add the following line to your local.conf for a x86-64 qemu
build.

SERIAL_CONSOLES="115200;ttyS0 115200;ttyUSB0"

Login right after the system boots and observe:

   root@qemux86-64:~# systemctl list-jobs |cat
   JOB UNIT TYPE  STATE
 1 multi-user.targetstart waiting
69 serial-getty@ttyUSB0.service start waiting
64 getty.target start waiting
71 dev-ttyUSB0.device   start running
62 systemd-update-utmp-runlevel.service start waiting

   5 jobs listed.

You can see above that the dev-ttyUSB0.device will block for 1min 30
seconds.  While that might not be a problem for this reference build.
It is certainly a problem for images that have software watchdogs that
verify the system booted up all the way to systemd completion in less
than 90 seconds.

This other nice effect of this change is that the fast fail device
extend to additional serial ports that may not exist on ARM BSPs or
that might be configured in or out by the dtb files on different
boards.

Signed-off-by: Jason Wessel 
---
 .../systemd/systemd-serialgetty/serial-getty@.service  | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git 
a/meta/recipes-core/systemd/systemd-serialgetty/serial-getty@.service 
b/meta/recipes-core/systemd/systemd-serialgetty/serial-getty@.service
index e8b027e97d..15af16a9f8 100644
--- a/meta/recipes-core/systemd/systemd-serialgetty/serial-getty@.service
+++ b/meta/recipes-core/systemd/systemd-serialgetty/serial-getty@.service
@@ -9,7 +9,8 @@
 Description=Serial Getty on %I
 Documentation=man:agetty(8) man:systemd-getty-generator(8)
 Documentation=http://0pointer.de/blog/projects/serial-console.html
-BindsTo=dev-%i.device
+PartOf=dev-%i.device
+ConditionPathExists=/dev/%i
 After=dev-%i.device systemd-user-sessions.service plymouth-quit-wait.service
 After=rc-local.service
 
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] serial-getty@.service: Allow device to fast fail if it does not exist

[Jason Wessel]

On 8/28/19 5:41 PM, Richard Purdie wrote:

On Wed, 2019-08-28 at 15:24 -0500, Jason Wessel wrote:

On 8/27/19 7:15 PM, richard.pur...@linuxfoundation.org wrote:

On Tue, 2019-08-27 at 19:03 -0500, Jason Wessel wrote:

On 8/27/19 5:58 PM, Richard Purdie wrote:

Hi Jason,
Somehow this change is responsible for this build failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/976

(steps 5c and 7c so failure during testimage).

I have bisected it to this change, I haven't looked into why.

Thanks for tracking it down.   I am sure how to try an duplicate
this.  I clicked around to try and find out a bit about what it
is
running for these phases of the build but it is not very obvious.

Is there a local.conf I can try along with what ever commands it
ran?

The configuration its using is shown in
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/978/steps/8/logs/stdio
for each step.


I tried the configuration on my system but it seems to work fine, see
below.   Is there a way to get complete log file of the boot on the
broken host?  I am also curious if it fails every time or not.

/home/pokybuild/yocto-worker/qa-
extras2/build/build/tmp/work/qemux86_64-poky-linux/core-image-
sato/1.0-r0/testimage/qemu_boot_log.2019082722

I am not sure it will tell us anything further or not, but it
certainly looks like the system didn't boot correctly in the first
place.

I've rerun it as
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/983




I'll take a further look at it tomorrow.   I have no idea why systemd emitted 
this message:

[    4.750194] systemd[1]: Unnecessary job for /dev/ttyS1 was removed.
[    4.751666] systemd[1]: Unnecessary job for /dev/ttyS0 was removed.

[  OK  ] Started Serial Getty on ttyS0.
 Stopping Serial Getty on ttyS0...

I have an image now with the problem so, I'll make a copy of it in case the problem 
"goes away" again.  It will be interesting to see what the deal is with this 
corner case.

Cheers,

Jason.




--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] serial-getty@.service: Allow device to fast fail if it does not exist

[Jason Wessel]

On 8/27/19 7:15 PM, richard.pur...@linuxfoundation.org wrote:

On Tue, 2019-08-27 at 19:03 -0500, Jason Wessel wrote:


On 8/27/19 5:58 PM, Richard Purdie wrote:

Hi Jason,
Somehow this change is responsible for this build failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/976

(steps 5c and 7c so failure during testimage).

I have bisected it to this change, I haven't looked into why.


Thanks for tracking it down.   I am sure how to try an duplicate
this.  I clicked around to try and find out a bit about what it is
running for these phases of the build but it is not very obvious.

Is there a local.conf I can try along with what ever commands it ran?


The configuration its using is shown in
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/978/steps/8/logs/stdio
for each step.




I tried the configuration on my system but it seems to work fine, see below.   
Is there a way to get complete log file of the boot on the broken host?  I am 
also curious if it fails every time or not.

/home/pokybuild/yocto-worker/qa-extras2/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/testimage/qemu_boot_log.2019082722

I am not sure it will tell us anything further or not, but it certainly looks 
like the system didn't boot correctly in the first place.


Jason.





Initialising tasks: 100% 
|###
Sstate summary: Wanted 0 Found 0 Missed 0 Current 90 (0% match, 100% complete)
NOTE: Executing Tasks
NOTE: Setscene tasks completed
Started HTTPService on 128.224.149.46:38317
Test requires apt to be installed
Stopped HTTPService on 128.224.149.46:38317
Test requires autoconf to be installed
Test requires gtk+3 to be installed
Test requires autoconf to be installed
Started HTTPService on 128.224.149.46:32961
Stopped HTTPService on 128.224.149.46:32961
Test requires gcc to be installed
Test requires g++ to be installed
Test requires g++ to be installed
Test requires make to be installed
Test requires python3-pygobject to be installed
Test requires gcc to be installed
Test requires ldd to be installed
Test requires logrotate to be installed
Test case logrotate.LogrotateTest.test_2_logrotate depends on 
logrotate.LogrotateTest.test_1_logrotate_setup but it didn'
Not appropiate for systemd image
Test requires opkg to be installed
Test requires pam to be in DISTRO_FEATURES
Test requires ptest-runner to be installed
Test requires systemtap to be installed
Startup finished in 3.840s (kernel) + 2.157s (userspace) = 5.998s.
RESULTS:
RESULTS - connman.ConnmanTest.test_connmand_help: PASSED (0.11s)
RESULTS - connman.ConnmanTest.test_connmand_running: PASSED (0.11s)
RESULTS - date.DateTest.test_date: PASSED (0.63s)
RESULTS - df.DfTest.test_df: PASSED (0.11s)
RESULTS - dnf.DnfBasicTest.test_dnf_help: PASSED (0.78s)
RESULTS - dnf.DnfBasicTest.test_dnf_history: PASSED (0.44s)
RESULTS - dnf.DnfBasicTest.test_dnf_info: PASSED (0.37s)
RESULTS - dnf.DnfBasicTest.test_dnf_search: PASSED (0.35s)
RESULTS - dnf.DnfBasicTest.test_dnf_version: PASSED (0.32s)
RESULTS - dnf.DnfRepoTest.test_dnf_exclude: PASSED (5.03s)
RESULTS - dnf.DnfRepoTest.test_dnf_install: PASSED (0.78s)
RESULTS - dnf.DnfRepoTest.test_dnf_install_dependency: PASSED (1.77s)
RESULTS - dnf.DnfRepoTest.test_dnf_install_from_disk: PASSED (1.97s)
RESULTS - dnf.DnfRepoTest.test_dnf_install_from_http: PASSED (1.52s)
RESULTS - dnf.DnfRepoTest.test_dnf_installroot: PASSED (6.91s)
RESULTS - dnf.DnfRepoTest.test_dnf_makecache: PASSED (0.42s)
RESULTS - dnf.DnfRepoTest.test_dnf_reinstall: PASSED (0.61s)
RESULTS - dnf.DnfRepoTest.test_dnf_repoinfo: PASSED (0.35s)
RESULTS - oe_syslog.SyslogTest.test_syslog_running: PASSED (0.12s)
RESULTS - oe_syslog.SyslogTestConfig.test_syslog_logger: PASSED (1.21s)
RESULTS - oe_syslog.SyslogTestConfig.test_syslog_restart: PASSED (0.21s)
RESULTS - parselogs.ParseLogsTest.test_parselogs: PASSED (1.91s)
RESULTS - perl.PerlTest.test_perl_works: PASSED (0.11s)
RESULTS - ping.PingTest.test_ping: PASSED (0.06s)
RESULTS - python.PythonTest.test_python3: PASSED (0.12s)
RESULTS - rpm.RpmBasicTest.test_rpm_help: PASSED (0.11s)
RESULTS - rpm.RpmBasicTest.test_rpm_query: PASSED (0.18s)
RESULTS - rpm.RpmBasicTest.test_rpm_query_nonroot: PASSED (0.89s)
RESULTS - 
rpm.RpmInstallRemoveTest.test_check_rpm_install_removal_log_file_size: PASSED 
(3.54s)
RESULTS - rpm.RpmInstallRemoveTest.test_rpm_install: PASSED (0.37s)
RESULTS - rpm.RpmInstallRemoveTest.test_rpm_remove: PASSED (0.15s)
RESULTS - scp.ScpTest.test_scp_file: PASSED (0.41s)
RESULTS - ssh.SSHTest.test_ssh: PASSED (0.51s)
RESULTS - systemd.SystemdBasicTests.test_systemd_basic: PASSED (0.12s)
RESULTS - systemd.SystemdBasicTests.test_systemd_failed: PASSED (0.21s)
RESULTS - systemd.SystemdBasicTests.test_systemd_list: PASSED (0.44s)
RESULTS - systemd.SystemdJournalTests.test_systemd_boot_time: PASSED (0.11s)
RESULTS - systemd.SystemdJournalTests.test_systemd_journal

Re: [OE-core] [PATCH] serial-getty@.service: Allow device to fast fail if it does not exist

[Jason Wessel]

On 8/27/19 5:58 PM, Richard Purdie wrote:

Hi Jason,
Somehow this change is responsible for this build failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/976

(steps 5c and 7c so failure during testimage).

I have bisected it to this change, I haven't looked into why.



Thanks for tracking it down.   I am sure how to try an duplicate this.  I 
clicked around to try and find out a bit about what it is running for these 
phases of the build but it is not very obvious.

Is there a local.conf I can try along with what ever commands it ran?


Jason.

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] serial-getty@.service: Allow device to fast fail if it does not exist

[Jason Wessel]

Some BSPs use a USB serial port which may or may not actually be
plugged all the time.  It is quite useful to have a USB serial port
have a getty running but it does not make sense to wait for it for 90
seconds before completing the system startup if it might never get
plugged in.  The typical example is that a USB serial device might
only need to be plugged in when debugging, upgrading, or initially
configuring a device.

This change is somewhat subtle.  Systemd uses the "BindsTo" directive
to ensure existence of the device in order to start the service as
well as to terminate the service if the device goes away.  The "After"
directive makes that same relationship stronger, and has the undesired
side effect that systemd will wait until its internal time out value
for the device to come on line before executing a fail operation or
letting other tasks and groups continue.  This is certainly the kind
of behavior we want for a disk, but not for serial ports in general.

The kernel module loader and device detection will have run a long
time before the getty startup.  By the time the getty startup occurs
the system has all the serial devices its going to get.

If you want to observe the problem with qemu, it is easy to replicate.
Simply add the following line to your local.conf for a x86-64 qemu
build.

SERIAL_CONSOLES="115200;ttyS0 115200;ttyUSB0"

Login right after the system boots and observe:

   root@qemux86-64:~# systemctl list-jobs |cat
   JOB UNIT TYPE  STATE
 1 multi-user.targetstart waiting
69 serial-getty@ttyUSB0.service start waiting
64 getty.target start waiting
71 dev-ttyUSB0.device   start running
62 systemd-update-utmp-runlevel.service start waiting

   5 jobs listed.

You can see above that the dev-ttyUSB0.device will block for 1min 30
seconds.  While that might not be a problem for this reference build.
It is certainly a problem for images that have software watchdogs that
verify the system booted up all the way to systemd completion in less
than 90 seconds.

This other nice effect of this change is that the fast fail device
extend to additional serial ports that may not exist on ARM BSPs or
that might be configured in or out by the dtb files on different
boards.

Signed-off-by: Jason Wessel 
---
 .../systemd/systemd-serialgetty/serial-getty@.service   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/meta/recipes-core/systemd/systemd-serialgetty/serial-getty@.service 
b/meta/recipes-core/systemd/systemd-serialgetty/serial-getty@.service
index e8b027e97d..a20092a173 100644
--- a/meta/recipes-core/systemd/systemd-serialgetty/serial-getty@.service
+++ b/meta/recipes-core/systemd/systemd-serialgetty/serial-getty@.service
@@ -10,7 +10,7 @@ Description=Serial Getty on %I
 Documentation=man:agetty(8) man:systemd-getty-generator(8)
 Documentation=http://0pointer.de/blog/projects/serial-console.html
 BindsTo=dev-%i.device
-After=dev-%i.device systemd-user-sessions.service plymouth-quit-wait.service
+After=systemd-user-sessions.service plymouth-quit-wait.service
 After=rc-local.service
 
 # If additional gettys are spawned during boot then we should make
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] Revert "kernel.bbclass: adjust a condition checking"

[Jason Wessel]

This reverts commit 6676411fccff2d331878e4ca1f9411aafb056a80.

This revert restores the original code and adds a comment.  The commit
that was reverted broke a number of wic templates and tools which rely
on the initramfs creation dependency and the case where the
INITRAMFS_IMAGE_BUNDLE is not set.

If an end user does not want the INITRAMFS_IMAGE generated, it should
be set to "".

[ Issue: LIN1019-1791 ]

Signed-off-by: Jason Wessel 
---
 meta/classes/kernel.bbclass | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index bf3674238f..ebcb79a528 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -96,8 +96,10 @@ python __anonymous () {
 d.setVar('ALLOW_EMPTY_%s-image-%s' % (kname, typelower), '1')
 
 image = d.getVar('INITRAMFS_IMAGE')
-image_bundle = d.getVar('INITRAMFS_IMAGE_BUNDLE')
-if image and bb.utils.to_boolean(image_bundle, False):
+# If the INTIRAMFS_IMAGE is set but the INITRAMFS_IMAGE_BUNDLE is set to 0,
+# the do_bundle_initramfs does nothing, but the INITRAMFS_IMAGE is built
+# standalone for use by wic and other tools.
+if image:
 d.appendVarFlag('do_bundle_initramfs', 'depends', ' 
${INITRAMFS_IMAGE}:do_image_complete')
 
 # NOTE: setting INITRAMFS_TASK is for backward compatibility
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC PATCH 1/2] cross-localedef-native: Add hardlink resolver from util-linux

[Jason Wessel]

On 8/16/19 4:50 PM, richard.pur...@linuxfoundation.org wrote:

On Fri, 2019-08-16 at 14:29 -0700, Khem Raj wrote:

On Fri, Aug 16, 2019 at 2:25 PM Richard Purdie
 wrote:

This was discussed on IRC. We really really want to avoid that as a
dependency if we care about build performance. Its really easy to
just
add that in but the implications are significant :(


A captured version of this utility would also mean that we have to
keep this in sync for bug fixes and security fixes etc. which could
also be considered as expense, so how much do we lose if util-linux-
native is brought early into dependency chain?

The tool is being used for a specific purpose at build time. Its not
exposed on target and its use is extremely constrained to a specific
set of files where its behaviour should be deterministic.

As such, yes there are questions about bug fixes and about security but
I don't think its going to be a serious burden. We'd need to carry this
until such times as the host systems we support have hardlink which is
probably a few years out but will arrive.

The impact of having util-linux-native in the dependency graph at this
point is significant. I say that as someone who has done a lot of work
in trimming down the times of our bootstrap process. This is exactly
the kind of dependency I've removed and avoided.

glibc is special in that its the worst choke point on the dependency
graph. Nothing to do with target binaries happens until it is ready and
then gcc-runtime can build. Whilst the locales are split from glibc,
they are still connected in the task graph for packaging purposes so
this is important.

Its been pointed out there are other places where util-linux-native has
crept in, in particular python3-native. My preference would be to
minimise these dependencies rather than use them as a way to justify
adding it here too.



So what is the consensus here?   Can we require the host system to have 
hardlink?   It is available as a stand alone binary on Ubuntu 14.04 and up and 
on Fedora systems, but isn't installed by default, and we'd have to check the 
version because the arguments sadly are slightly different.   We could 
certainly make util-linux a dependency, but that is why this patch existed, in 
order to avoid the dependency.

I did find a python version at 
https://raw.githubusercontent.com/akaihola/hardlinkpy/master/hardlink.py. I 
made a one line modification to it and it works, but it isn't nearly as 
efficient.  The proof is below, using the same locale-tree each time.  It is 
important to look at both the real and user time.  The difference gets more 
significant when you run it under pseudo, where we expect it to run 100% of the 
time.  It is roughly 10x slower to do this in python.


time recipe-sysroot-native/usr/bin/cross-localedef-hardlink -c -v locale-tree
Directories: 982
Objects:    7441
Regular files:  6441
Comparisons:    3331
Linked: 3109
Saved: 754675712

real    0m0.641s
user    0m0.120s
sys 0m0.518s

time python3 hardlink.py -v 0 -c locale-tree


Hard linking Statistics:
Directories   : 982
Regular files : 6441
Comparisons   : 31884
Hardlinked this run   : 3109
Total hardlinks   : 3109
Bytes saved this run  : 744089843 (709.619 mebibytes)
Total bytes saved : 744089843 (709.619 mebibytes)
Total run time    : 2.124422073364258 seconds

real    0m2.180s
user    0m1.220s
sys 0m0.953s


Now lets run it under pseudo 

time recipe-sysroot-native/usr/bin/cross-localedef-hardlink -c -v locale-tree
Directories: 982
Objects:    7441
Regular files:  6441
Comparisons:    3331
Linked: 3109
Saved: 754675712

real    0m2.505s
user    0m0.373s
sys 0m1.286s

time python3 hardlink.py -v 0 -c locale-tree


Hard linking Statistics:
Directories   : 982
Regular files : 6441
Comparisons   : 31884
Hardlinked this run   : 3109
Total hardlinks   : 3109
Bytes saved this run  : 744089843 (709.619 mebibytes)
Total bytes saved : 744089843 (709.619 mebibytes)
Total run time    : 12.971824884414673 seconds

real    0m13.046s
user    0m3.633s
sys 0m5.192s


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 2/2] libc-package.bbclass: Split locale hard link processing into two parts

[Jason Wessel]

The locale-processing in cross-localedef was proven to be unsafe to
run in parallel due to the way it tried to make hard links to files
that could disappear before the link operation was completed.

To avoid corruption of the pseudo database, and create a
deterministically generated link tree, the operation of the locale
generation and the hard link resolution have been split into two
separate parts.  A side effect of this change is that the do_package()
rule for glibc-locale will be slightly smaller because some of the
hard links were missed in the past, particularly if you had a lot of
cpus free to perform the work.

Before the patch:
% du -sk locale-tree
312524  locale-tree

With the patch:
% du -sk locale-tree
290772 locale-tree

A number of comparisons were performed such as diffing the output,
cross checking the link references using tar's listing output, and
comparing against the results with out a parallel build to ensure the
locale-tree is correct in its final form.

[YOCTO #11299]
[YOCTO #12434]

Signed-off-by: Jason Wessel 
---
 meta/classes/libc-package.bbclass | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/classes/libc-package.bbclass 
b/meta/classes/libc-package.bbclass
index 8859dad566..a66e540884 100644
--- a/meta/classes/libc-package.bbclass
+++ b/meta/classes/libc-package.bbclass
@@ -82,6 +82,9 @@ do_collect_bins_from_locale_tree() {
parent=$(dirname ${localedir})
mkdir -p ${PKGD}/$parent
tar -cf - -C $treedir/$parent -p $(basename ${localedir}) | tar -xf - 
-C ${PKGD}$parent
+
+   # Finalize tree by chaning all duplicate files into hard links
+   cross-localedef-hardlink -c -v ${WORKDIR}/locale-tree
 }
 
 inherit qemu
@@ -265,7 +268,7 @@ python package_do_split_gconvs () {
 bb.error("locale_arch_options not found for target_arch=" + 
target_arch)
 bb.fatal("unknown arch:" + target_arch + " for 
locale_arch_options")
 
-localedef_opts += " --force  --no-archive --prefix=%s \
+localedef_opts += " --force --no-hard-links --no-archive 
--prefix=%s \
 --inputfile=%s/%s/i18n/locales/%s --charmap=%s %s/%s" \
 % (treedir, treedir, datadir, locale, encoding, outputpath, 
name)
 
@@ -273,7 +276,7 @@ python package_do_split_gconvs () {
 (path, i18npath, gconvpath, localedef_opts)
 else: # earlier slower qemu way 
 qemu = qemu_target_binary(d) 
-localedef_opts = "--force --no-archive --prefix=%s \
+localedef_opts = "--force --no-hard-links --no-archive --prefix=%s 
\
 --inputfile=%s/i18n/locales/%s --charmap=%s %s" \
 % (treedir, datadir, locale, encoding, name)
 
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 1/2] cross-localedef-native: Add hardlink resolver from util-linux

[Jason Wessel]

The hard link resolver that is built into localedef cannot be run in
parallel.  It will search sibling directories (which are be processed
in parallel) and perform a creation of a .tmp file and remove the
original and move the .tmp file in.  The problem is that if a probe
occurs a hard link can be requested to the file that is being removed.
This will lead to a stray copy or potentially, on a loaded system
cause race condition which pseudo cannot deal with, where it is left
with a hard link request to a file that no longer exists.  In this
situation psuedo will inherit the permissions of what ever the target
inode had to offer.

In short, there are two problems:

  1) You will be left with stray copies when using the hard link
 resolution that is built in while running in parallel with
 localedef.

  2) When running under pseudo the possibility exists for uid/gid
 leakage when the source file is removed before the hard link can
 be completed.

The solution is to call localedef with --no-hard-links and separately
process the hardlinks at a later point.  To do this requires the
inclusion of the hardlink utility found in modern versions of
util-linux.  Most host systems do not have this, so it will be
included with the cross-localedef binary.

There are two patches here.  The first imports the raw version of
hardlink.c and a couple of header files directly from util-linux. The
second patch applies the fix-ups to make it compile, along with a
change to recipe to package the new binary.

[YOCTO #11299]
[YOCTO #12434]

Signed-off-by: Jason Wessel 
---
 .../glibc/cross-localedef-native_2.30.bb  |3 +
 .../glibc/add-cross-localedef-hardlink.patch  | 1115 +
 ...le-separate-from-util-linux-hardlink.patch |  226 
 3 files changed, 1344 insertions(+)
 create mode 100644 
meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch
 create mode 100644 
meta/recipes-core/glibc/glibc/allow-compile-separate-from-util-linux-hardlink.patch

diff --git a/meta/recipes-core/glibc/cross-localedef-native_2.30.bb 
b/meta/recipes-core/glibc/cross-localedef-native_2.30.bb
index e4923c73d5..29cd43e89a 100644
--- a/meta/recipes-core/glibc/cross-localedef-native_2.30.bb
+++ b/meta/recipes-core/glibc/cross-localedef-native_2.30.bb
@@ -30,6 +30,8 @@ SRC_URI = "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \

file://0022-eglibc-Forward-port-cross-locale-generation-support.patch \
file://0023-Define-DUMMY_LOCALE_T-if-not-defined.patch \

file://0024-localedef-add-to-archive-uses-a-hard-coded-locale-pa.patch \
+   file://add-cross-localedef-hardlink.patch \
+   file://allow-compile-separate-from-util-linux-hardlink.patch \
 "
 # Makes for a rather long rev (22 characters), but...
 #
@@ -43,4 +45,5 @@ CFLAGS += "-fgnu89-inline -std=gnu99 -DIS_IN\(x\)='0'"
 do_install() {
install -d ${D}${bindir}
install -m 0755 ${B}/localedef ${D}${bindir}/cross-localedef
+   install -m 0755 ${B}/cross-localedef-hardlink 
${D}${bindir}/cross-localedef-hardlink
 }
diff --git a/meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch 
b/meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch
new file mode 100644
index 00..8471121949
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch
@@ -0,0 +1,1115 @@
+---
+ locale/programs/c.h|  407 ++
+ locale/programs/cross-localedef-hardlink.c |  528 
+
+ locale/programs/xalloc.h   |  129 +++
+ localedef/Makefile.in  |8 
+ 4 files changed, 1071 insertions(+), 1 deletion(-)
+
+--- /dev/null
 b/locale/programs/cross-localedef-hardlink.c
+@@ -0,0 +1,528 @@
++/*
++ * hardlink - consolidate duplicate files via hardlinks
++ *
++ * Copyright (C) 2018 Red Hat, Inc. All rights reserved.
++ * Written by Jakub Jelinek 
++ *
++ * Copyright (C) 2019 Karel Zak 
++ *
++ * This program is free software; you can redistribute it and/or modify
++ * it under the terms of the GNU General Public License as published by
++ * the Free Software Foundation; either version 2 of the License, or
++ * (at your option) any later version.
++ *
++ * This program is distributed in the hope that it would be useful,
++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
++ * GNU General Public License for more details.
++ *
++ * You should have received a copy of the GNU General Public License along
++ * with this program; if not, write to the Free Software Foundation, Inc.,
++ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
++ */
++#include 
++#include 
++#include 
++#include 
++#include 
++#include 
++#include 
++#include 
++#include 
++#include 
++#include 
++#ifdef HAVE_PCRE
++# define PCRE2_CODE_UNIT_WIDTH 8
++# include 
++#endif
++
++#incl

[OE-core] [PATCH v2 4/4] dpkg: Provide update-alternative for start-stop-daemon

[Jason Wessel]

When using DISTRO_FEATURE usrmerge, busybox and something that
requires dpkg-start-stop there is a conflict which prints the
following error.

===

ERROR: core-image-minimal-1.0-r1 do_rootfs: Postinstall scriptlets of 
['busybox'] have failed. If the intention is to defer them to first boot,
then please place them into pkg_postinst_ontarget_${PN} ().
Deferring to first boot via 'exit 1' is no longer supported.
Details of the failure are in 
/build/tmp-glibc/work/intel_x86_64-linux/core-image-minimal/1.0-r1/temp/log.do_rootfs.
ERROR: core-image-minimal-1.0-r1 do_rootfs: Function failed: do_rootfs

===

Looking at the log file, it shows the problem:

update-alternatives: Error: not linking 
/build/tmp-glibc/work/intel_x86_64-linux/core-image-minimal/1.0-r1/rootfs/usr/sbin/start-stop-daemon
 to /usr/bin/busybox.nosuid since 
/build/tmp-glibc/work/intel_x86_64-linux/core-image-minimal/1.0-r1/rootfs/usr/sbin/start-stop-daemon
 exists and is not a link

===

The solution is to add an update-alternative to start-stop-daemon for dpkg.

Signed-off-by: Jason Wessel 
---
 meta/recipes-devtools/dpkg/dpkg.inc | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/dpkg/dpkg.inc 
b/meta/recipes-devtools/dpkg/dpkg.inc
index fee7a75814..1c3c585d79 100644
--- a/meta/recipes-devtools/dpkg/dpkg.inc
+++ b/meta/recipes-devtools/dpkg/dpkg.inc
@@ -9,7 +9,7 @@ RDEPENDS_${PN}_class-native = ""
 
 UPSTREAM_CHECK_URI = "${DEBIAN_MIRROR}/main/d/dpkg/"
 
-inherit autotools gettext perlnative pkgconfig perl-version
+inherit autotools gettext perlnative pkgconfig perl-version update-alternatives
 
 PERL_class-native = "${STAGING_BINDIR_NATIVE}/perl-native/perl"
 
@@ -76,7 +76,10 @@ RDEPENDS_${PN}-perl += "perl-module-carp 
perl-module-constant \
 # the bitbake version.
 #
 PACKAGES =+ "${PN}-start-stop"
-FILES_${PN}-start-stop = "${sbindir}/start-stop-daemon"
+FILES_${PN}-start-stop = "${sbindir}/start-stop-daemon.${BPN}"
+ALTERNATIVE_${PN}-start-stop = "start-stop-daemon"
+ALTERNATIVE_LINK_NAME[start-stop-daemon] = "${sbindir}/start-stop-daemon"
+ALTERNATIVE_PRIORITY = "100"
 
 RDEPENDS_${PN} += "${PN}-start-stop"
 
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 3/4] wic: Make disk partition size consistently computed

[Jason Wessel]

When using different root directories with a wks file wic is using the
value from the original ROOTFS_SIZE which is not correct.  Example:

Number  Start   End SizeFile system  Name   Flags
 1  20.5kB  318MB   318MB   fat16otaefi legacy_boot, msftdata
 2  318MB   636MB   318MB   ext4 otaboot
 3  636MB   1709MB  1074MB  ext4 otaroot
 4  1709MB  2027MB  318MB   ext4 otaboot_b
 5  2027MB  3101MB  1074MB  ext4 otaroot_b
 6  3101MB  5249MB  2147MB  ext4 fluxdata

The partitions 1, 2, and 3 incorrectly inherit the size, instead of using
a computed size.  With the patch applied it is working properly:

Number  Start   End SizeFile system  Name   Flags
 1  20.5kB  14.5MB  14.5MB  fat16otaefi legacy_boot, msftdata
 2  14.5MB  65.3MB  50.8MB  ext4 otaboot
 3  65.3MB  1139MB  1074MB  ext4 otaroot
 4  1139MB  1190MB  50.8MB  ext4 otaboot_b
 5  1190MB  2264MB  1074MB  ext4 otaroot_b
 6  2264MB  4411MB  2147MB  ext4 fluxdata

As for the removal of the bb.warn, if the size is not specified, it is
not something to warn the end user about.  Some of my default images
make use of the head room + a computed directory size or while generating
images.

Signed-off-by: Jason Wessel 
---
 scripts/lib/wic/partition.py | 18 ++
 1 file changed, 6 insertions(+), 12 deletions(-)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index f427c8101b..2a71d7b1d6 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -212,19 +212,13 @@ class Partition():
 if os.path.isfile(rootfs):
 os.remove(rootfs)
 
-# Get rootfs size from bitbake variable if it's not set in .ks file
+# If size is not specified compute it from the rootfs_dir size
 if not self.size and real_rootfs:
-# Bitbake variable ROOTFS_SIZE is calculated in
-# Image._get_rootfs_size method from meta/lib/oe/image.py
-# using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT,
-# IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE
-rsize_bb = get_bitbake_var('ROOTFS_SIZE')
-if rsize_bb:
-logger.warning('overhead-factor was specified, but size was 
not,'
-   ' so bitbake variables will be used for the 
size.'
-   ' In this case both IMAGE_OVERHEAD_FACTOR and '
-   '--overhead-factor will be applied')
-self.size = int(round(float(rsize_bb)))
+# Use the same logic found in get_rootfs_size()
+# from meta/classes/image.bbclass
+du_cmd = "du -ks %s" % rootfs_dir
+out = exec_cmd(du_cmd)
+self.size = int(out.split()[0])
 
 prefix = "ext" if self.fstype.startswith("ext") else self.fstype
 method = getattr(self, "prepare_rootfs_" + prefix)
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 2/4] wic: Add partition type for msdos partition tables

[Jason Wessel]

In order to create a msdos partition table disk image that can auto
expand after the image is copied to an SD card, wic needs the ability
to have a primary partition as the last entry.  The desired use case
is to be able to create an A/B update partition image scheme with a
/var volume that can be auto expanded to the remainder of the SD card
at run time.

The typical .wks file will look similar to the following:

bootloader --ptable msdos
part / --source rawcopy --sourceparams="file=u-boot.imx" \
   --ondisk mmcblk --no-table --align 1 --size 1
part /boot --source bootimg-partition \
   --ondisk mmcblk --fstype=vfat --label boot --active --align 4 --size 16
part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaboot --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaroot --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaboot_b --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaroot_b --align 4 --type logical
part /var --source rawcopy --sourceparams="file=imx6_var.otaimg" \
--ondisk mmcblk --fstype=ext4 --label fluxdata --align 4

Without the patch applied, wic will generate an SD card image that looks like:

Disk boot.img: 890940s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  StartEnd  Size Type  File system  Flags
 1  2056s48001s   45946s   primary   fat16lba
 2  48008s   132467s  84460s   primary   ext4
 3  132472s  454467s  321996s  primary   ext4
 4  454471s  890939s  436469s  extended   lba
 5  454472s  538931s  84460s   logical   ext4
 6  538936s  860931s  321996s  logical   ext4
 7  860936s  890939s  30004s   logical   ext4 boot

With the patch applied a primary partition can be created at the end
of the image which can be expanded to fill the free space on the media
where the image has been copied, which looks like:

Disk boot.img: 890940s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  StartEnd  Size Type  File system  Flags
 1  2056s48001s   45946s   primary   fat16lba
 2  48007s   860931s  812925s  extended   lba
 5  48008s   132467s  84460s   logical   ext4
 6  132472s  454467s  321996s  logical   ext4
 7  454472s  538931s  84460s   logical   ext4
 8  538936s  860931s  321996s  logical   ext4
 3  860936s  890939s  30004s   primary   ext4 boot

Signed-off-by: Jason Wessel 
---
 scripts/lib/wic/ksparser.py  |  2 ++
 scripts/lib/wic/partition.py |  1 +
 scripts/lib/wic/plugins/imager/direct.py | 37 +---
 3 files changed, 29 insertions(+), 11 deletions(-)

diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index 3e670033ba..6a643ba3af 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -151,6 +151,8 @@ class KickStart():
 part.add_argument('--part-name')
 part.add_argument('--part-type')
 part.add_argument('--rootfs-dir')
+part.add_argument('--type', default='primary',
+choices = ('primary', 'logical'))
 
 # --size and --fixed-size cannot be specified together; options
 # extra-space and --overhead-factor should also raise a parser
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 01466b258d..f427c8101b 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -50,6 +50,7 @@ class Partition():
 self.use_uuid = args.use_uuid
 self.uuid = args.uuid
 self.fsuuid = args.fsuuid
+self.type = args.type
 
 self.lineno = lineno
 self.source_file = ""
diff --git a/scripts/lib/wic/plugins/imager/direct.py 
b/scripts/lib/wic/plugins/imager/direct.py
index 91fc5e70f6..3ce6ad55b8 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -300,6 +300,10 @@ class PartitionedImage():
 self.path = path  # Path to the image file
 self.numpart = 0  # Number of allocated partitions
 self.realpart = 0 # Number of partitions in the partition table
+self.primary_part_num = 0  # Number of primary partitions (msdos)
+self.extendedpart = 0  # Create extended partition before this 
logical partition (msdos)
+self.extended_size_sec = 0 # Size of exteded partition (msdos)
+self.logical_part_cnt = 0  # Number of total logical paritions (msdos)
 self.offset = 0   # Offset of next partition (in sectors)
 self.min_size = 0 # Minimum required disk size to fit

[OE-core] [PATCH v2 1/4] image_types_wic.bbclass: Copy the .wks and .env files to deploy image dir

[Jason Wessel]

When using a .wks.in file, the only place that the generated .wks file
exists in the tmp/work area.  A copy should be left behind in the
deploy directory so that you can easily run the wic tool to
re-generate or modify a new image without running bitbake.  Custom
.wks.in files can reference any number of bitbake variables, so it is
important to save the result.

below is an example of using the generated .wks file in the deploy
area.  The full name of my generated .wks file was
core-image-minimal-ostree-uboot-ab.wks, but since you usually only
have a single .wks file per image you can use a wild card like:

  cd tmp*/deploy/images/*
  wic create --vars . -e core-image-minimal -s -m core-image-minimal-*.wks

Signed-off-by: Jason Wessel 
---
 meta/classes/image_types_wic.bbclass | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image_types_wic.bbclass 
b/meta/classes/image_types_wic.bbclass
index 519aeb1b04..f350dc2723 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -77,6 +77,11 @@ python do_write_wks_template () {
 wks_file = d.getVar('WKS_FULL_PATH')
 with open(wks_file, 'w') as f:
 f.write(template_body)
+f.close()
+# Copy the finalized wks file to the deploy directory for later use
+depdir = d.getVar('IMGDEPLOYDIR')
+basename = d.getVar('IMAGE_BASENAME')
+bb.utils.copyfile(wks_file, "%s/%s" % (depdir, basename + '-' + 
os.path.basename(wks_file)))
 }
 
 python () {
@@ -105,7 +110,7 @@ python () {
 # file in process_wks_template as well, so just put it in
 # a variable and let the metadata deal with the deps.
 d.setVar('_WKS_TEMPLATE', body)
-bb.build.addtask('do_write_wks_template', 'do_image_wic', 
None, d)
+bb.build.addtask('do_write_wks_template', 'do_image_wic', 
'do_image', d)
 bb.build.addtask('do_image_wic', 'do_image_complete', None, d)
 }
 
@@ -127,6 +132,10 @@ python do_rootfs_wicenv () {
 value = d.getVar(var)
 if value:
 envf.write('%s="%s"\n' % (var, value.strip()))
+envf.close()
+# Copy .env file to deploy directory for later use with stand alone wic
+depdir = d.getVar('IMGDEPLOYDIR')
+bb.utils.copyfile(os.path.join(outdir, basename) + '.env', 
os.path.join(depdir, basename) + '.env')
 }
 addtask do_rootfs_wicenv after do_image before do_image_wic
 do_rootfs_wicenv[vardeps] += "${WICVARS}"
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/4] image_types_wic.bbclass: Copy the .wks and .env files to deploy image dir

[Jason Wessel]

On 8/8/19 1:42 PM, Richard Purdie wrote:

On Thu, 2019-08-08 at 05:55 -0700, Jason Wessel wrote:

When using a .wks.in file, the only place that the generated .wks file
exists in the tmp/work area.  A copy should be left behind in the
deploy directory so that you can easily run the wic tool to
re-generate or modify a new image without running bitbake.  Custom
.wks.in files can reference any number of bitbake variables, so it is
important to save the result.

below is an example of using the generated .wks file in the deploy
area.  The full name of my generated .wks file was
core-image-minimal-ostree-uboot-ab.wks, but since you usually only
have a single .wks file per image you can use a wild card like:

   cd tmp*/deploy/images/*
   wic create --vars . -e core-image-minimal -s -m core-image-minimal-*.wks

Signed-off-by: Jason Wessel 
---
  meta/classes/image_types_wic.bbclass | 9 +
  1 file changed, 9 insertions(+)

diff --git a/meta/classes/image_types_wic.bbclass 
b/meta/classes/image_types_wic.bbclass
index 519aeb1b04..ce94138b3c 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -77,6 +77,11 @@ python do_write_wks_template () {
  wks_file = d.getVar('WKS_FULL_PATH')
  with open(wks_file, 'w') as f:
  f.write(template_body)
+f.close()
+# Copy the finalized wks file to the deploy directory for later use
+depdir = d.getVar('DEPLOY_DIR_IMAGE')
+basename = d.getVar('IMAGE_BASENAME')
+bb.utils.copyfile(wks_file, "%s/%s" % (depdir, basename + '-' + 
os.path.basename(wks_file)))
  }


Nothing should be writing to DEPLOY_DIR_IMAGE directly as it means its
not being managed by sstate and won't restore in a build from sstate.
There is an intermediate directory it should be written to and the
sstate code will put it into DEPLOY_DIR_IMAGE. For image classes it
might be IMGDEPLOYDIR but you'd have to check this task/code runs
between do_image and do_image_complete.




Thank you for the pointer.  It works exactly as intended so long as the 
dependency
is changed properly, and the correct staging directory is used.
There is no directory creation required because the do_image() takes care of all
that.

I'll send v2 of the series, but this this the only patch that changed.

Jason.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/4] image_types_wic.bbclass: Copy the .wks and .env files to deploy image dir

[Jason Wessel]

On 8/8/19 7:55 AM, Jason Wessel wrote:

When using a .wks.in file, the only place that the generated .wks file
exists in the tmp/work area.  A copy should be left behind in the
deploy directory so that you can easily run the wic tool to
re-generate or modify a new image without running bitbake.  Custom
.wks.in files can reference any number of bitbake variables, so it is
important to save the result.

below is an example of using the generated .wks file in the deploy
area.  The full name of my generated .wks file was
core-image-minimal-ostree-uboot-ab.wks, but since you usually only
have a single .wks file per image you can use a wild card like:

   cd tmp*/deploy/images/*
   wic create --vars . -e core-image-minimal -s -m core-image-minimal-*.wks

Signed-off-by: Jason Wessel 
---
  meta/classes/image_types_wic.bbclass | 9 +
  1 file changed, 9 insertions(+)

diff --git a/meta/classes/image_types_wic.bbclass 
b/meta/classes/image_types_wic.bbclass
index 519aeb1b04..ce94138b3c 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -77,6 +77,11 @@ python do_write_wks_template () {
  wks_file = d.getVar('WKS_FULL_PATH')
  with open(wks_file, 'w') as f:
  f.write(template_body)
+f.close()
+# Copy the finalized wks file to the deploy directory for later use
+depdir = d.getVar('DEPLOY_DIR_IMAGE')


This needs a call to "bb.utils.mkdirhier(depdir)" in order to work properly.

I didn't find the problem until using a higher degree of parallelism where the 
deploy directory might not actually be there to write out the file.  I'll send 
a v2 after more tests with the rest of the series.  It was all well tested 
originally on a single server, extended testing found the problem.


Cheers,
Jason.


+basename = d.getVar('IMAGE_BASENAME')
+bb.utils.copyfile(wks_file, "%s/%s" % (depdir, basename + '-' + 
os.path.basename(wks_file)))
  }
  
  python () {

@@ -127,6 +132,10 @@ python do_rootfs_wicenv () {
  value = d.getVar(var)
  if value:
  envf.write('%s="%s"\n' % (var, value.strip()))
+envf.close()
+# Copy to deploy directory for later use with stand alone wic
+depdir = d.getVar('DEPLOY_DIR_IMAGE')
+bb.utils.copyfile(os.path.join(outdir, basename) + '.env', 
os.path.join(depdir, basename) + '.env')
  }
  addtask do_rootfs_wicenv after do_image before do_image_wic
  do_rootfs_wicenv[vardeps] += "${WICVARS}"



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/4] wic: Make disk partition size consistently computed

[Jason Wessel]

When using different root directories with a wks file wic is using the
value from the original ROOTFS_SIZE which is not correct.  Example:

Number  Start   End SizeFile system  Name   Flags
 1  20.5kB  318MB   318MB   fat16otaefi legacy_boot, msftdata
 2  318MB   636MB   318MB   ext4 otaboot
 3  636MB   1709MB  1074MB  ext4 otaroot
 4  1709MB  2027MB  318MB   ext4 otaboot_b
 5  2027MB  3101MB  1074MB  ext4 otaroot_b
 6  3101MB  5249MB  2147MB  ext4 fluxdata

The partitions 1, 2, and 3 incorrectly inherit the size, instead of using
a computed size.  With the patch applied it is working properly:

Number  Start   End SizeFile system  Name   Flags
 1  20.5kB  14.5MB  14.5MB  fat16otaefi legacy_boot, msftdata
 2  14.5MB  65.3MB  50.8MB  ext4 otaboot
 3  65.3MB  1139MB  1074MB  ext4 otaroot
 4  1139MB  1190MB  50.8MB  ext4 otaboot_b
 5  1190MB  2264MB  1074MB  ext4 otaroot_b
 6  2264MB  4411MB  2147MB  ext4 fluxdata

As for the removal of the bb.warn, if the size is not specified, it is
not something to warn the end user about.  Some of my default images
make use of the head room + a computed directory size or while generating
images.

Signed-off-by: Jason Wessel 
---
 scripts/lib/wic/partition.py | 18 ++
 1 file changed, 6 insertions(+), 12 deletions(-)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index f427c8101b..2a71d7b1d6 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -212,19 +212,13 @@ class Partition():
 if os.path.isfile(rootfs):
 os.remove(rootfs)
 
-# Get rootfs size from bitbake variable if it's not set in .ks file
+# If size is not specified compute it from the rootfs_dir size
 if not self.size and real_rootfs:
-# Bitbake variable ROOTFS_SIZE is calculated in
-# Image._get_rootfs_size method from meta/lib/oe/image.py
-# using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT,
-# IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE
-rsize_bb = get_bitbake_var('ROOTFS_SIZE')
-if rsize_bb:
-logger.warning('overhead-factor was specified, but size was 
not,'
-   ' so bitbake variables will be used for the 
size.'
-   ' In this case both IMAGE_OVERHEAD_FACTOR and '
-   '--overhead-factor will be applied')
-self.size = int(round(float(rsize_bb)))
+# Use the same logic found in get_rootfs_size()
+# from meta/classes/image.bbclass
+du_cmd = "du -ks %s" % rootfs_dir
+out = exec_cmd(du_cmd)
+self.size = int(out.split()[0])
 
 prefix = "ext" if self.fstype.startswith("ext") else self.fstype
 method = getattr(self, "prepare_rootfs_" + prefix)
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/4] wic: Add partition type for msdos partition tables

[Jason Wessel]

In order to create a msdos partition table disk image that can auto
expand after the image is copied to an SD card, wic needs the ability
to have a primary partition as the last entry.  The desired use case
is to be able to create an A/B update partition image scheme with a
/var volume that can be auto expanded to the remainder of the SD card
at run time.

The typical .wks file will look similar to the following:

bootloader --ptable msdos
part / --source rawcopy --sourceparams="file=u-boot.imx" \
   --ondisk mmcblk --no-table --align 1 --size 1
part /boot --source bootimg-partition \
   --ondisk mmcblk --fstype=vfat --label boot --active --align 4 --size 16
part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaboot --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaroot --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaboot_b --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaroot_b --align 4 --type logical
part /var --source rawcopy --sourceparams="file=imx6_var.otaimg" \
--ondisk mmcblk --fstype=ext4 --label fluxdata --align 4

Without the patch applied, wic will generate an SD card image that looks like:

Disk boot.img: 890940s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  StartEnd  Size Type  File system  Flags
 1  2056s48001s   45946s   primary   fat16lba
 2  48008s   132467s  84460s   primary   ext4
 3  132472s  454467s  321996s  primary   ext4
 4  454471s  890939s  436469s  extended   lba
 5  454472s  538931s  84460s   logical   ext4
 6  538936s  860931s  321996s  logical   ext4
 7  860936s  890939s  30004s   logical   ext4 boot

With the patch applied a primary partition can be created at the end
of the image which can be expanded to fill the free space on the media
where the image has been copied, which looks like:

Disk boot.img: 890940s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  StartEnd  Size Type  File system  Flags
 1  2056s48001s   45946s   primary   fat16lba
 2  48007s   860931s  812925s  extended   lba
 5  48008s   132467s  84460s   logical   ext4
 6  132472s  454467s  321996s  logical   ext4
 7  454472s  538931s  84460s   logical   ext4
 8  538936s  860931s  321996s  logical   ext4
 3  860936s  890939s  30004s   primary   ext4 boot

Signed-off-by: Jason Wessel 
---
 scripts/lib/wic/ksparser.py  |  2 ++
 scripts/lib/wic/partition.py |  1 +
 scripts/lib/wic/plugins/imager/direct.py | 37 +---
 3 files changed, 29 insertions(+), 11 deletions(-)

diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index 3e670033ba..6a643ba3af 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -151,6 +151,8 @@ class KickStart():
 part.add_argument('--part-name')
 part.add_argument('--part-type')
 part.add_argument('--rootfs-dir')
+part.add_argument('--type', default='primary',
+choices = ('primary', 'logical'))
 
 # --size and --fixed-size cannot be specified together; options
 # extra-space and --overhead-factor should also raise a parser
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 01466b258d..f427c8101b 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -50,6 +50,7 @@ class Partition():
 self.use_uuid = args.use_uuid
 self.uuid = args.uuid
 self.fsuuid = args.fsuuid
+self.type = args.type
 
 self.lineno = lineno
 self.source_file = ""
diff --git a/scripts/lib/wic/plugins/imager/direct.py 
b/scripts/lib/wic/plugins/imager/direct.py
index 91fc5e70f6..3ce6ad55b8 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -300,6 +300,10 @@ class PartitionedImage():
 self.path = path  # Path to the image file
 self.numpart = 0  # Number of allocated partitions
 self.realpart = 0 # Number of partitions in the partition table
+self.primary_part_num = 0  # Number of primary partitions (msdos)
+self.extendedpart = 0  # Create extended partition before this 
logical partition (msdos)
+self.extended_size_sec = 0 # Size of exteded partition (msdos)
+self.logical_part_cnt = 0  # Number of total logical paritions (msdos)
 self.offset = 0   # Offset of next partition (in sectors)
 self.min_size = 0 # Minimum required disk size to fit

[OE-core] [PATCH 1/4] image_types_wic.bbclass: Copy the .wks and .env files to deploy image dir

[Jason Wessel]

When using a .wks.in file, the only place that the generated .wks file
exists in the tmp/work area.  A copy should be left behind in the
deploy directory so that you can easily run the wic tool to
re-generate or modify a new image without running bitbake.  Custom
.wks.in files can reference any number of bitbake variables, so it is
important to save the result.

below is an example of using the generated .wks file in the deploy
area.  The full name of my generated .wks file was
core-image-minimal-ostree-uboot-ab.wks, but since you usually only
have a single .wks file per image you can use a wild card like:

  cd tmp*/deploy/images/*
  wic create --vars . -e core-image-minimal -s -m core-image-minimal-*.wks

Signed-off-by: Jason Wessel 
---
 meta/classes/image_types_wic.bbclass | 9 +
 1 file changed, 9 insertions(+)

diff --git a/meta/classes/image_types_wic.bbclass 
b/meta/classes/image_types_wic.bbclass
index 519aeb1b04..ce94138b3c 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -77,6 +77,11 @@ python do_write_wks_template () {
 wks_file = d.getVar('WKS_FULL_PATH')
 with open(wks_file, 'w') as f:
 f.write(template_body)
+f.close()
+# Copy the finalized wks file to the deploy directory for later use
+depdir = d.getVar('DEPLOY_DIR_IMAGE')
+basename = d.getVar('IMAGE_BASENAME')
+bb.utils.copyfile(wks_file, "%s/%s" % (depdir, basename + '-' + 
os.path.basename(wks_file)))
 }
 
 python () {
@@ -127,6 +132,10 @@ python do_rootfs_wicenv () {
 value = d.getVar(var)
 if value:
 envf.write('%s="%s"\n' % (var, value.strip()))
+envf.close()
+# Copy to deploy directory for later use with stand alone wic
+depdir = d.getVar('DEPLOY_DIR_IMAGE')
+bb.utils.copyfile(os.path.join(outdir, basename) + '.env', 
os.path.join(depdir, basename) + '.env')
 }
 addtask do_rootfs_wicenv after do_image before do_image_wic
 do_rootfs_wicenv[vardeps] += "${WICVARS}"
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/4] dpkg: Provide update-alternative for start-stop-daemon

[Jason Wessel]

When using DISTRO_FEATURE usrmerge, busybox and something that
requires dpkg-start-stop there is a conflict which prints the
following error.

===

ERROR: core-image-minimal-1.0-r1 do_rootfs: Postinstall scriptlets of 
['busybox'] have failed. If the intention is to defer them to first boot,
then please place them into pkg_postinst_ontarget_${PN} ().
Deferring to first boot via 'exit 1' is no longer supported.
Details of the failure are in 
/build/tmp-glibc/work/intel_x86_64-linux/core-image-minimal/1.0-r1/temp/log.do_rootfs.
ERROR: core-image-minimal-1.0-r1 do_rootfs: Function failed: do_rootfs

===

Looking at the log file, it shows the problem:

update-alternatives: Error: not linking 
/build/tmp-glibc/work/intel_x86_64-linux/core-image-minimal/1.0-r1/rootfs/usr/sbin/start-stop-daemon
 to /usr/bin/busybox.nosuid since 
/build/tmp-glibc/work/intel_x86_64-linux/core-image-minimal/1.0-r1/rootfs/usr/sbin/start-stop-daemon
 exists and is not a link

===

The solution is to add an update-alternative to start-stop-daemon for dpkg.

Signed-off-by: Jason Wessel 
---
 meta/recipes-devtools/dpkg/dpkg.inc | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/dpkg/dpkg.inc 
b/meta/recipes-devtools/dpkg/dpkg.inc
index fee7a75814..1c3c585d79 100644
--- a/meta/recipes-devtools/dpkg/dpkg.inc
+++ b/meta/recipes-devtools/dpkg/dpkg.inc
@@ -9,7 +9,7 @@ RDEPENDS_${PN}_class-native = ""
 
 UPSTREAM_CHECK_URI = "${DEBIAN_MIRROR}/main/d/dpkg/"
 
-inherit autotools gettext perlnative pkgconfig perl-version
+inherit autotools gettext perlnative pkgconfig perl-version update-alternatives
 
 PERL_class-native = "${STAGING_BINDIR_NATIVE}/perl-native/perl"
 
@@ -76,7 +76,10 @@ RDEPENDS_${PN}-perl += "perl-module-carp 
perl-module-constant \
 # the bitbake version.
 #
 PACKAGES =+ "${PN}-start-stop"
-FILES_${PN}-start-stop = "${sbindir}/start-stop-daemon"
+FILES_${PN}-start-stop = "${sbindir}/start-stop-daemon.${BPN}"
+ALTERNATIVE_${PN}-start-stop = "start-stop-daemon"
+ALTERNATIVE_LINK_NAME[start-stop-daemon] = "${sbindir}/start-stop-daemon"
+ALTERNATIVE_PRIORITY = "100"
 
 RDEPENDS_${PN} += "${PN}-start-stop"
 
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3] sqlite3: Fix zlib determinism problem

[Jason Wessel]

sqlite3-native in particular was finding zlib from the host if zlib-devel
was installed. This could lead to races where pseudo-native may or may not
fail to build.

We don't need/use compressed page support with sqlite so disable the dependency
(it doesn't have a configure option so use a autoconf cache variable).

The target binaries were not previously building with zlib, so we will
leave the default being zlib turned off, while the host binaries were
building with it "occasionally", but not for anything at runtime.

Signed-off-by: Jason Wessel 
---
 meta/recipes-support/sqlite/sqlite3.inc | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-support/sqlite/sqlite3.inc 
b/meta/recipes-support/sqlite/sqlite3.inc
index daf3db5e9f..7cf23d0447 100644
--- a/meta/recipes-support/sqlite/sqlite3.inc
+++ b/meta/recipes-support/sqlite/sqlite3.inc
@@ -34,6 +34,9 @@ PACKAGECONFIG[json1] = "--enable-json1,--disable-json1"
 PACKAGECONFIG[rtree] = "--enable-rtree,--disable-rtree"
 PACKAGECONFIG[session] = "--enable-session,--disable-session"
 PACKAGECONFIG[dyn_ext] = 
"--enable-dynamic-extensions,--disable-dynamic-extensions"
+PACKAGECONFIG[zlib] = ",,zlib"
+
+CACHED_CONFIGUREVARS += "${@bb.utils.contains('PACKAGECONFIG', 'zlib', '', 
'ac_cv_search_deflate=no',d)}"
 
 EXTRA_OECONF = " \
 --enable-shared \
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] sqlite3: Fix zlib determinism problem

[Jason Wessel]

sqlite3-native in particular was finding zlib from the host if zlib-devel
was installed. This could lead to races where pseudo-native may or may not
fail to build.

We don't need/use compressed page support with sqlite so disable the dependency
(it doesn't have a configure option so use a autoconf cache variable).

The target binaries were not previously building with zlib, so we will
leave the default being zlib turned off, while the host binaries were
building with it "occasionally", but not for anything at runtime.

Signed-off-by: Jason Wessel 
---
 meta/recipes-support/sqlite/sqlite3.inc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-support/sqlite/sqlite3.inc 
b/meta/recipes-support/sqlite/sqlite3.inc
index daf3db5e9f..2e48f303ee 100644
--- a/meta/recipes-support/sqlite/sqlite3.inc
+++ b/meta/recipes-support/sqlite/sqlite3.inc
@@ -2,6 +2,8 @@ SUMMARY = "Embeddable SQL database engine"
 HOMEPAGE = "http://www.sqlite.org;
 SECTION = "libs"
 
+DEPENDS = "zlib-native"
+
 PE = "3"
 
 def sqlite_download_version(d):
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] sqlite3: Always build sqlite3-native with zlib

[Jason Wessel]

On 8/6/19 2:17 PM, Alexander Kanavin wrote:

On Tue, 6 Aug 2019 at 20:52, Jason Wessel mailto:jason.wes...@windriver.com>> wrote:


Just to follow up to be very specific about how I deleted files, you can 
see the identical failure with:

sudo yum install -y zlib-devel

rm -rf tmp

bitbake -c cleansstate sqlite3-native

bitbake sqlite3-native

sudo yum remove -y zlib-devel

bitbake -c cleansstate pseudo-native

bitbake pseudo-native


I seriously doubt this is exactly what is going on on the host builder 
which failed.  More analysis would be needed as I explained earlier to draw a 
final conclusion.  Either way sqlite3-native should be configured to build 
consistently, which would prevent this kind of contamination.


This is not quite what I meant. I meant looking at the source code of sqlite, 
and finding out how exactly it is looking for zlib, and where things might go 
wrong (e.g. host contamination, or other possible causes of non-determinism). 
There might be a configure switch to control that behavior, or something 
similar that we can use to achieve deterministic builds.



Sqlite3 doesn't have a an explicit disable.  It has the typical autoconf check 
for a header/library.  From configure.ac:


AC_CHECK_HEADERS(zlib.h,[
  AC_SEARCH_LIBS(deflate,z,[BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_HAVE_ZLIB"])
])

This means it would need to be patched if you want to make it a pkgconfig 
option, or play games with the configure cache values...  I chose zlib as a 
default dependency because it was the easiest to implement.  If we really want 
to make it not build with zlib or build it conditionally, it is certainly 
possible.  Is that what is needed to get this patch accepted?

Jason.


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] sqlite3: Always build sqlite3-native with zlib

[Jason Wessel]

On 8/6/19 1:36 PM, Jason Wessel wrote:

On 8/6/19 12:55 PM, Alexander Kanavin wrote:



On Tue, 6 Aug 2019 at 19:31, Jason Wessel mailto:jason.wes...@windriver.com>> wrote:

The build of sqlite3-native is not fully deterministic.  There is some
corner case as shown with the recent pseudo commit where the sqlite3
will sometimes link against zlib.  It is possible that in a build with
sstate that since the zlib is not listed, that pseudo-native will fail
to build because it calls the pkg-config to get the link arguments for
sqlite3 and will ultimately try to link against zlib.


It would be good to dig a little deeper and get to the root cause of why sqlite 
*sometimes* does it. The fix provided below seems like fixing the symptom, not 
the problem.

Alex



Without getting access to the build machine where it failed to perform more in 
depth forensics it is a bit hard to determine the exact cause.  I performed 
some local testing using the Fedora 28 and could not replicate the exact 
circumstance unless I deleted a file in the sysroot.  From what I can tell 
there are two ways of seeing the problem.

1) Host contamination

% pkg-config sqlite3 --libs --static
-lsqlite3 -lm -ldl -lz -lpthread

This was not the way the sqlite3 was built.  However there is no way to know if 
this was the circumstance without poking around in the log files and re-running 
the context of the run.do_compile script in the packages temp directory.

2) Dependency race

This is the one where I deleted a file to simulate the problem, but I couldn't 
reproduce it.  For this, we'd really need to look at the failed machine's log 
of the build of sqlite3, which may be long gone from the sstate.  But it should 
have the pkg-config files laying around.  If it shows that it did build against 
the zlib, but zlib was not listed dependency it was never going to build 
because the recipe-sysroot-native is not going to have the required library.

There of course might be other ways to end up with the same result but these 
are the most likely candidates, each of which is fixed by listing the 
dependency.   I too would like to know the actual root cause, but without 
access to the build it is hard to track it down.




Just to follow up to be very specific about how I deleted files, you can see 
the identical failure with:

sudo yum install -y zlib-devel

rm -rf tmp

bitbake -c cleansstate sqlite3-native

bitbake sqlite3-native

sudo yum remove -y zlib-devel

bitbake -c cleansstate pseudo-native

bitbake pseudo-native


I seriously doubt this is exactly what is going on on the host builder which 
failed.  More analysis would be needed as I explained earlier to draw a final 
conclusion.  Either way sqlite3-native should be configured to build 
consistently, which would prevent this kind of contamination.

Cheers,

Jason.

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] sqlite3: Always build sqlite3-native with zlib

[Jason Wessel]

On 8/6/19 12:55 PM, Alexander Kanavin wrote:



On Tue, 6 Aug 2019 at 19:31, Jason Wessel mailto:jason.wes...@windriver.com>> wrote:

The build of sqlite3-native is not fully deterministic.  There is some
corner case as shown with the recent pseudo commit where the sqlite3
will sometimes link against zlib.  It is possible that in a build with
sstate that since the zlib is not listed, that pseudo-native will fail
to build because it calls the pkg-config to get the link arguments for
sqlite3 and will ultimately try to link against zlib.


It would be good to dig a little deeper and get to the root cause of why sqlite 
*sometimes* does it. The fix provided below seems like fixing the symptom, not 
the problem.

Alex



Without getting access to the build machine where it failed to perform more in 
depth forensics it is a bit hard to determine the exact cause.  I performed 
some local testing using the Fedora 28 and could not replicate the exact 
circumstance unless I deleted a file in the sysroot.  From what I can tell 
there are two ways of seeing the problem.

1) Host contamination

% pkg-config sqlite3 --libs --static
-lsqlite3 -lm -ldl -lz -lpthread

This was not the way the sqlite3 was built.  However there is no way to know if 
this was the circumstance without poking around in the log files and re-running 
the context of the run.do_compile script in the packages temp directory.

2) Dependency race

This is the one where I deleted a file to simulate the problem, but I couldn't 
reproduce it.  For this, we'd really need to look at the failed machine's log 
of the build of sqlite3, which may be long gone from the sstate.  But it should 
have the pkg-config files laying around.  If it shows that it did build against 
the zlib, but zlib was not listed dependency it was never going to build 
because the recipe-sysroot-native is not going to have the required library.

There of course might be other ways to end up with the same result but these 
are the most likely candidates, each of which is fixed by listing the 
dependency.   I too would like to know the actual root cause, but without 
access to the build it is hard to track it down.

Cheers,

Jason

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] sqlite3: Always build sqlite3-native with zlib

[Jason Wessel]

The build of sqlite3-native is not fully deterministic.  There is some
corner case as shown with the recent pseudo commit where the sqlite3
will sometimes link against zlib.  It is possible that in a build with
sstate that since the zlib is not listed, that pseudo-native will fail
to build because it calls the pkg-config to get the link arguments for
sqlite3 and will ultimately try to link against zlib.

===  As observed on the YP Autobuilder ===
/pseudo-native/1.9.0+gitAUTOINC+060058bb29-r0/recipe-sysroot-native/lib 
-Wl,-O1 -Wl,--allow-shlib-undefined 
-Wl,--dynamic-linker=TOPDIR/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
 
TOPDIR/tmp/work/x86_64-linux/pseudo-native/1.9.0+gitAUTOINC+060058bb29-r0/recipe-sysroot-native/usr/lib/libsqlite3.a
 -lz -lm -ldl -lpthread -lpthread -ldl -lpthread
TOPDIR//tmp/home/hosttoolspokybuild//yoctold-:worker/ qemuarmcannot/build 
/buildfind/ tmp/-hosttoolslz/ld
: cannot find -lz
==

The simple fix is to depend to the zlib-native which will only impact
the sqlite3-native build, and then deterministically complete all the
dependencies for pseudo-native.  For a target build the host native
library is ignored as usual.

Signed-off-by: Jason Wessel 
---
 meta/recipes-support/sqlite/sqlite3.inc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-support/sqlite/sqlite3.inc 
b/meta/recipes-support/sqlite/sqlite3.inc
index daf3db5e9f..2e48f303ee 100644
--- a/meta/recipes-support/sqlite/sqlite3.inc
+++ b/meta/recipes-support/sqlite/sqlite3.inc
@@ -2,6 +2,8 @@ SUMMARY = "Embeddable SQL database engine"
 HOMEPAGE = "http://www.sqlite.org;
 SECTION = "libs"
 
+DEPENDS = "zlib-native"
+
 PE = "3"
 
 def sqlite_download_version(d):
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3] pseudo: Fix openat() with a symlink pointing to a directory

[Jason Wessel]

On 8/6/19 10:39 AM, Jason Wessel wrote:

On 8/6/19 9:59 AM, Jason Wessel wrote:

On 8/6/19 9:20 AM, Richard Purdie wrote:

On Mon, 2019-08-05 at 09:32 -0700, Jason Wessel wrote:

While working with ostree disk generation in conjunction with wic, I
found a problem with pseudo where it tried to resolve a symlink when
it shouldn't, based on openat() flags. A C program has been
constructed to test pseudo to show that it is working properly with
the correct behavior around openat().


[...]


Many thanks to Peter Seebach for fixing the problem in the pseudo
code
to use the same logic which was already there for the
AT_SYMLINK_NOFOLLOW.

Also updated is the license MD5 checksum since the master branch of
pseudo has had the SPDX data updated.

Signed-off-by: Jason Wessel 
---
meta/recipes-devtools/pseudo/pseudo.inc| 2 +-
meta/recipes-devtools/pseudo/pseudo_git.bb | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)


pseudo-native didn't build on fedora28 or centos7:

http://errors.yoctoproject.org/Errors/Latest/?filter=9382629d4bbcd26196c36d31ffdf097b30023150=commit=150

(see the pseudo-native failures).

It looks like an issue on hosts which don't have zlib-devel installed,



at a very approximate guess -  I've not been able to look in detail.



I'll try and setup a test, since I can easily run fedora28.   I wonder if we 
can just use zlib-native to solve it, or if it is a a chicken-egg problem with 
the early boot strap.



I cannot duplicate the problem.  It must have to do with the way your 
particular system is setup.   What is interesting is that the -lz is not 
actually called on my system, where I most certainly don't have the libz devel 
package installed.  I am guessing there is a configure failure of some sort?

Do you think on the fedora28 system you could send me the "rpm -q -a" output?



I found out where the link flags come from in the pseudo recipe.


for sqlite_link_opt in $(pkg-config sqlite3 --libs --static)
do
case "$sqlite_link_opt" in
-lsqlite3)
;;
-l*)
SQLITE_LDADD="${SQLITE_LDADD} ${sqlite_link_opt}"


This implies that your local sqlite3 is perhaps different in some manner.  I am 
not sure exactly what kind of edge case you have going on with your system.  :-)

Cheers,
Jason.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3] pseudo: Fix openat() with a symlink pointing to a directory

[Jason Wessel]

On 8/6/19 9:59 AM, Jason Wessel wrote:

On 8/6/19 9:20 AM, Richard Purdie wrote:

On Mon, 2019-08-05 at 09:32 -0700, Jason Wessel wrote:

While working with ostree disk generation in conjunction with wic, I
found a problem with pseudo where it tried to resolve a symlink when
it shouldn't, based on openat() flags. A C program has been
constructed to test pseudo to show that it is working properly with
the correct behavior around openat().


[...]


Many thanks to Peter Seebach for fixing the problem in the pseudo
code
to use the same logic which was already there for the
AT_SYMLINK_NOFOLLOW.

Also updated is the license MD5 checksum since the master branch of
pseudo has had the SPDX data updated.

Signed-off-by: Jason Wessel 
---
   meta/recipes-devtools/pseudo/pseudo.inc| 2 +-
   meta/recipes-devtools/pseudo/pseudo_git.bb | 2 +-
   2 files changed, 2 insertions(+), 2 deletions(-)


pseudo-native didn't build on fedora28 or centos7:

http://errors.yoctoproject.org/Errors/Latest/?filter=9382629d4bbcd26196c36d31ffdf097b30023150=commit=150

(see the pseudo-native failures).

It looks like an issue on hosts which don't have zlib-devel installed,



at a very approximate guess -  I've not been able to look in detail.



I'll try and setup a test, since I can easily run fedora28.   I wonder if we 
can just use zlib-native to solve it, or if it is a a chicken-egg problem with 
the early boot strap.



I cannot duplicate the problem.  It must have to do with the way your 
particular system is setup.   What is interesting is that the -lz is not 
actually called on my system, where I most certainly don't have the libz devel 
package installed.  I am guessing there is a configure failure of some sort?

Do you think on the fedora28 system you could send me the "rpm -q -a" output?

Jason.

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3] pseudo: Fix openat() with a symlink pointing to a directory

[Jason Wessel]

On 8/6/19 9:20 AM, Richard Purdie wrote:

On Mon, 2019-08-05 at 09:32 -0700, Jason Wessel wrote:

While working with ostree disk generation in conjunction with wic, I
found a problem with pseudo where it tried to resolve a symlink when
it shouldn't, based on openat() flags. A C program has been
constructed to test pseudo to show that it is working properly with
the correct behavior around openat().


[...]


Many thanks to Peter Seebach for fixing the problem in the pseudo
code
to use the same logic which was already there for the
AT_SYMLINK_NOFOLLOW.

Also updated is the license MD5 checksum since the master branch of
pseudo has had the SPDX data updated.

Signed-off-by: Jason Wessel 
---
  meta/recipes-devtools/pseudo/pseudo.inc| 2 +-
  meta/recipes-devtools/pseudo/pseudo_git.bb | 2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


pseudo-native didn't build on fedora28 or centos7:

http://errors.yoctoproject.org/Errors/Latest/?filter=9382629d4bbcd26196c36d31ffdf097b30023150=commit=150

(see the pseudo-native failures).

It looks like an issue on hosts which don't have zlib-devel installed,



at a very approximate guess -  I've not been able to look in detail.



I'll try and setup a test, since I can easily run fedora28.   I wonder if we 
can just use zlib-native to solve it, or if it is a a chicken-egg problem with 
the early boot strap.

Jason.


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3] pseudo: Fix openat() with a symlink pointing to a directory

[Jason Wessel]

While working with ostree disk generation in conjunction with wic, I
found a problem with pseudo where it tried to resolve a symlink when
it shouldn't, based on openat() flags. A C program has been
constructed to test pseudo to show that it is working properly with
the correct behavior around openat().

 #include 
 #include 
 #include 
 #include 
 #include 
 #include 
 #include 

int main()
{
/*
 * Tested with: gcc -Wall -o app app.c ; echo "no pseudo" ;
 * ./app ; echo "pseudo"; pseudo ./app
 */
system("rm -rf tdir tlink");
system("mkdir tdir");
system("ln -s tdir tlink");
DIR *dir = opendir(".");
int dfd = dirfd(dir);

int target_dfd = openat (dfd, "tlink", O_RDONLY | O_NONBLOCK |
 O_DIRECTORY | O_CLOEXEC | O_NOFOLLOW);
if (target_dfd == -1) {
printf("Test 1 good\n");
} else {
printf("Test 1 failed\n");
close(target_dfd);
}
target_dfd = openat (dfd, "tlink", O_RDONLY | O_NONBLOCK |
 O_DIRECTORY | O_CLOEXEC);
if (target_dfd == -1) {
printf("Test 2 failed\n");
} else {
printf("Test 2 good\n");
close(target_dfd);
}
/* Test 3 make sure the owner of the link is root  */
struct stat sbuf;
if (!lstat("tlink", ) && sbuf.st_uid == 0) {
printf("Test 3 good\n");
} else {
printf("Test 3 failed\n");
}
/* Test 4 tests open with the "rb" flag, owner should not change */
int ofd = openat(dfd,"./tlink", O_RDONLY|O_CLOEXEC);
if (ofd >= 0) {
if (fstat(ofd, ) != 0)
printf("ERROR in fstat test 4\n");
else if (sbuf.st_uid == 0)
printf("Test 4 good\n");
close(ofd);
} else {
printf("Test 4 failed with openat()\n");
}
/* Test pseudo db to see the fstat() above did not delete the DB entry */
if (!lstat("tlink", ) && sbuf.st_uid == 0)
printf("Test 5 good\n");
else
printf("Test 5 failed... tlink is owned by %i and not 0\n", 
sbuf.st_uid);
return 0;
}

int main()
{
/* Tested with: gcc -Wall -o app app.c ; echo "no pseudo" ; ./app ; echo 
"pseudo"; pseudo ./app */
system("rm -rf tdir tlink");
system("mkdir tdir");
system("ln -s tdir tlink");
DIR *dir = opendir(".");
int dfd = dirfd(dir);

int target_dfd = openat (dfd, "tlink", O_RDONLY | O_NONBLOCK | O_DIRECTORY 
| O_CLOEXEC | O_NOFOLLOW);
if (target_dfd == -1) {
printf("This is right\n");
} else {
    printf("This is broken\n");
}
return 0;
}

Many thanks to Peter Seebach for fixing the problem in the pseudo code
to use the same logic which was already there for the
AT_SYMLINK_NOFOLLOW.

Also updated is the license MD5 checksum since the master branch of
pseudo has had the SPDX data updated.

Signed-off-by: Jason Wessel 
---
 meta/recipes-devtools/pseudo/pseudo.inc| 2 +-
 meta/recipes-devtools/pseudo/pseudo_git.bb | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/pseudo/pseudo.inc 
b/meta/recipes-devtools/pseudo/pseudo.inc
index 8b05735bb1..8b34909726 100644
--- a/meta/recipes-devtools/pseudo/pseudo.inc
+++ b/meta/recipes-devtools/pseudo/pseudo.inc
@@ -4,7 +4,7 @@
 
 SUMMARY = "Pseudo gives fake root capabilities to a normal user"
 HOMEPAGE = "http://git.yoctoproject.org/cgit/cgit.cgi/pseudo;
-LIC_FILES_CHKSUM = "file://COPYING;md5=243b725d71bb5df4a1e5920b344b86ad"
+LIC_FILES_CHKSUM = "file://COPYING;md5=a1d8023a6f953ac6ea4af765ff62d574"
 SECTION = "base"
 LICENSE = "LGPL2.1"
 DEPENDS = "sqlite3 attr"
diff --git a/meta/recipes-devtools/pseudo/pseudo_git.bb 
b/meta/recipes-devtools/pseudo/pseudo_git.bb
index 51db84c4d4..6cf711e4c2 100644
--- a/meta/recipes-devtools/pseudo/pseudo_git.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_git.bb
@@ -8,7 +8,7 @@ SRC_URI = "git://git.yoctoproject.org/pseudo \
file://toomanyfiles.patch \
"
 
-SRCREV = "3fa7c853e0bcd6fe23f7524c2a3c9e3af90901c3"
+SRCREV = "060058bb29f70b244e685b3c704eb0641b736f73"
 S = "${WORKDIR}/git"
 PV = "1.9.0+git${SRCPV}"
 
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] pseudo: Upgrade to latest to fix openat() with a directory symlink [NAK]

[Jason Wessel]

If we understood more about the nature of the race condition a test case could 
probably be constructed. For now, it is worth a try to see if it is any better. 
 I am certain the timing will change ever so slightly, so we could hit the 
glibc-locale issue more or less...

All of the regression tests completed over the weekend and the new code which 
found the problem in the first place is also working well. I'll send a new 
patch out to move pseudo forward.

Cheers,
Jason.


On 8/3/19 9:57 AM, Khem Raj wrote:

I see the locale issue atleast 5-7 times a week on world builds so I
will be able to see if that frequency stays same after this fix.

On Sat, Aug 3, 2019 at 7:23 AM Seebs  wrote:

On Sat, 3 Aug 2019 05:33:46 -0700
Khem Raj  wrote:


Will this fix the file ownership issue that we see with Glibc-locale
packages from time to time?

I have no idea. Since I haven't got a reliable reproducer for it, I
can't test it in a sane way.

-s



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] pseudo: Upgrade to latest to fix openat() with a directory symlink [NAK]

[Jason Wessel]

It took a while to narrow this down to a concise test case, and I am not 
exactly sure what is going on in pseudo.  The C app is created based on 
mimicking exactly the python code that causes the failure, so that bitbake can 
be entirely removed from the picture.

If you use the master branch of pseudo with the C app below, it will something 
like the following but with a different owner uid if yours is not 5002.

===
Test 1 good
Test 2 good
Test 3 good
Test 4 good
Test 5 failed... tlink is owned by 5002 and not 0
===


The sequence of openat() followed by an fstat() on the opened file handle, will 
erase the pseudo uid entry for the symlink, as shown by the following lstat() 
in test 5. The culprit appears to be the fstat(), but it could be something 
much more complex than that...  The next step is to figure out why the recent 
change to openat() to address test case 1, caused this new problem.


 test case app.c 

#include 
#include 
#include 
#include 
#include 
#include 
#include 

int main()
{
/* Tested with: gcc -Wall -o app app.c ; pseudo ./app */
system("rm -rf tdir tlink");
system("mkdir tdir");
system("ln -s tdir tlink");
DIR *dir = opendir(".");
int dfd = dirfd(dir);

int target_dfd = openat (dfd, "tlink", O_RDONLY | O_NONBLOCK | O_DIRECTORY 
| O_CLOEXEC | O_NOFOLLOW);
if (target_dfd == -1) {
printf("Test 1 good\n");
} else {
printf("Test 1 failed\n");
close(target_dfd);
}
target_dfd = openat (dfd, "tlink", O_RDONLY | O_NONBLOCK | O_DIRECTORY 
| O_CLOEXEC);
if (target_dfd == -1) {
printf("Test 2 failed\n");
} else {
printf("Test 2 good\n");
close(target_dfd);
}
/* Test 3 make sure the owner of the link is root  */
struct stat sbuf;
if (!lstat("tlink", ) && sbuf.st_uid == 0) {
printf("Test 3 good\n");
} else {
printf("Test 3 failed\n");
}
/* Test 4 tests open with the "rb" flag, owner should not change */
int ofd = openat(dfd,"./tlink", O_RDONLY|O_CLOEXEC);
if (ofd >= 0) {
if (fstat(ofd, ) != 0)
printf("ERROR in fstat test 4\n");
else if (sbuf.st_uid == 0)
printf("Test 4 good\n");
close(ofd);
} else {
printf("Test 4 failed with openat()\n");
}
/* In pseudo, after the fstat above, it seems the db is corrupted */
if (!lstat("tlink", ) && sbuf.st_uid == 0)
    printf("Test 5 good\n");
else
printf("Test 5 failed... tlink is owned by %i and not 0\n", 
sbuf.st_uid);

return 0;
}



On 8/1/19 6:57 PM, Seebs wrote:

On Thu, 1 Aug 2019 16:37:26 -0500
Jason Wessel  wrote:


It seems to have caused really odd problems with the oe link
management that were not there previously, such as:


WARNING: pinentry-1.1.0-r0 do_package_qa: QA Issue:
pinentry: /usr/bin/pinentry is owned by uid 5002, which is the same
as the user running bitbake. This may be due to host contamination
[host-user-contaminated]

I'll continue to look into the problem.


There's a possibility that the right flag is something like
(flags_NOFOLLOW)&&!(flags_PATH)

or something like that. There's a handful of references to this in
wrapfuncs.in in ports/unix and ports/linux.

-s



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] pseudo: Upgrade to latest to fix openat() with a directory symlink [NAK]

[Jason Wessel]

While this is a real problem.  We need to put this patch on hold.

It seems to have caused really odd problems with the oe link management that 
were not there previously, such as:


WARNING: pinentry-1.1.0-r0 do_package_qa: QA Issue: pinentry: /usr/bin/pinentry 
is owned by uid 5002, which is the same as the user running bitbake. This may 
be due to host contamination [host-user-contaminated]

I'll continue to look into the problem.

Cheers,
Jason.

On 8/1/19 3:03 PM, Jason Wessel wrote:

While working with ostree disk generation in conjunction with wic, I
found a problem with pseudo where it tried to resolve a symlink when
it shouldn't, based on openat() flags.  I narrowed down the problem to
a simple c program to reproduce the issue:

int main()
{
 /* Tested with: gcc -Wall -o app app.c ; echo "no pseudo" ; ./app ; echo 
"pseudo"; pseudo ./app */
 system("rm -rf tdir tlink");
 system("mkdir tdir");
 system("ln -s tdir tlink");
 DIR *dir = opendir(".");
 int dfd = dirfd(dir);

 int target_dfd = openat (dfd, "tlink", O_RDONLY | O_NONBLOCK | O_DIRECTORY 
| O_CLOEXEC | O_NOFOLLOW);
 if (target_dfd == -1) {
 printf("This is right\n");
 } else {
 printf("This is broken\n");
 }
 return 0;
}

Many thanks to Peter Seebach for fixing the problem in the pseudo code
to use the same logic which was already there for the
AT_SYMLINK_NOFOLLOW.

Also updated is the license MD5 checksum since the master branch of
pseudo has had teh SPDX data updated.

Signed-off-by: Jason Wessel 
---
  meta/recipes-devtools/pseudo/pseudo.inc| 2 +-
  meta/recipes-devtools/pseudo/pseudo_git.bb | 2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/pseudo/pseudo.inc 
b/meta/recipes-devtools/pseudo/pseudo.inc
index 8b05735bb1..8b34909726 100644
--- a/meta/recipes-devtools/pseudo/pseudo.inc
+++ b/meta/recipes-devtools/pseudo/pseudo.inc
@@ -4,7 +4,7 @@
  
  SUMMARY = "Pseudo gives fake root capabilities to a normal user"

  HOMEPAGE = "http://git.yoctoproject.org/cgit/cgit.cgi/pseudo;
-LIC_FILES_CHKSUM = "file://COPYING;md5=243b725d71bb5df4a1e5920b344b86ad"
+LIC_FILES_CHKSUM = "file://COPYING;md5=a1d8023a6f953ac6ea4af765ff62d574"
  SECTION = "base"
  LICENSE = "LGPL2.1"
  DEPENDS = "sqlite3 attr"
diff --git a/meta/recipes-devtools/pseudo/pseudo_git.bb 
b/meta/recipes-devtools/pseudo/pseudo_git.bb
index 51db84c4d4..3350c3fabd 100644
--- a/meta/recipes-devtools/pseudo/pseudo_git.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_git.bb
@@ -8,7 +8,7 @@ SRC_URI = "git://git.yoctoproject.org/pseudo \
 file://toomanyfiles.patch \
 "
  
-SRCREV = "3fa7c853e0bcd6fe23f7524c2a3c9e3af90901c3"

+SRCREV = "097ca3e245200c4a4333964af59a106c42ff3bca"
  S = "${WORKDIR}/git"
  PV = "1.9.0+git${SRCPV}"
  



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] pseudo: Upgrade to latest to fix openat() with a directory symlink

[Jason Wessel]

While working with ostree disk generation in conjunction with wic, I
found a problem with pseudo where it tried to resolve a symlink when
it shouldn't, based on openat() flags.  I narrowed down the problem to
a simple c program to reproduce the issue:

int main()
{
/* Tested with: gcc -Wall -o app app.c ; echo "no pseudo" ; ./app ; echo 
"pseudo"; pseudo ./app */
system("rm -rf tdir tlink");
system("mkdir tdir");
system("ln -s tdir tlink");
DIR *dir = opendir(".");
int dfd = dirfd(dir);

int target_dfd = openat (dfd, "tlink", O_RDONLY | O_NONBLOCK | O_DIRECTORY 
| O_CLOEXEC | O_NOFOLLOW);
if (target_dfd == -1) {
printf("This is right\n");
} else {
printf("This is broken\n");
}
return 0;
}

Many thanks to Peter Seebach for fixing the problem in the pseudo code
to use the same logic which was already there for the
AT_SYMLINK_NOFOLLOW.

Also updated is the license MD5 checksum since the master branch of
pseudo has had teh SPDX data updated.

Signed-off-by: Jason Wessel 
---
 meta/recipes-devtools/pseudo/pseudo.inc| 2 +-
 meta/recipes-devtools/pseudo/pseudo_git.bb | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/pseudo/pseudo.inc 
b/meta/recipes-devtools/pseudo/pseudo.inc
index 8b05735bb1..8b34909726 100644
--- a/meta/recipes-devtools/pseudo/pseudo.inc
+++ b/meta/recipes-devtools/pseudo/pseudo.inc
@@ -4,7 +4,7 @@
 
 SUMMARY = "Pseudo gives fake root capabilities to a normal user"
 HOMEPAGE = "http://git.yoctoproject.org/cgit/cgit.cgi/pseudo;
-LIC_FILES_CHKSUM = "file://COPYING;md5=243b725d71bb5df4a1e5920b344b86ad"
+LIC_FILES_CHKSUM = "file://COPYING;md5=a1d8023a6f953ac6ea4af765ff62d574"
 SECTION = "base"
 LICENSE = "LGPL2.1"
 DEPENDS = "sqlite3 attr"
diff --git a/meta/recipes-devtools/pseudo/pseudo_git.bb 
b/meta/recipes-devtools/pseudo/pseudo_git.bb
index 51db84c4d4..3350c3fabd 100644
--- a/meta/recipes-devtools/pseudo/pseudo_git.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_git.bb
@@ -8,7 +8,7 @@ SRC_URI = "git://git.yoctoproject.org/pseudo \
file://toomanyfiles.patch \
"
 
-SRCREV = "3fa7c853e0bcd6fe23f7524c2a3c9e3af90901c3"
+SRCREV = "097ca3e245200c4a4333964af59a106c42ff3bca"
 S = "${WORKDIR}/git"
 PV = "1.9.0+git${SRCPV}"
 
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] pseudo: Upgrade to latest to fix openat() with a directory symlink

[Jason Wessel]

Looks like the checksum license changed between the prior commit and now.  So 
I'll send a v2, because we don't want to break the master branch of oe-core.

Cheers,
Jason.

On 8/1/19 12:55 PM, Jason Wessel wrote:

While working with ostree disk generation in conjunction with wic, I
found a problem with pseudo where it tried to resolve a symlink when
it shouldn't, based on openat() flags.  I narrowed down the problem to
a simple c program to reproduce the issue:

int main()
{
 /* Tested with: gcc -Wall -o app app.c ; echo "no pseudo" ; ./app ; echo 
"pseudo"; pseudo ./app */
 system("rm -rf tdir tlink");
 system("mkdir tdir");
 system("ln -s tdir tlink");
 DIR *dir = opendir(".");
 int dfd = dirfd(dir);

 int target_dfd = openat (dfd, "tlink", O_RDONLY | O_NONBLOCK | O_DIRECTORY 
| O_CLOEXEC | O_NOFOLLOW);
 if (target_dfd == -1) {
 printf("This is right\n");
 } else {
 printf("This is broken\n");
 }
 return 0;
}

Many thanks to Peter Seebach for fixing the problem in the pseudo code
to use the same logic which was already there for the
AT_SYMLINK_NOFOLLOW.

Signed-off-by: Jason Wessel 
---
  meta/recipes-devtools/pseudo/pseudo_git.bb | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/pseudo/pseudo_git.bb 
b/meta/recipes-devtools/pseudo/pseudo_git.bb
index 51db84c4d4..3350c3fabd 100644
--- a/meta/recipes-devtools/pseudo/pseudo_git.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_git.bb
@@ -8,7 +8,7 @@ SRC_URI = "git://git.yoctoproject.org/pseudo \
 file://toomanyfiles.patch \
 "
  
-SRCREV = "3fa7c853e0bcd6fe23f7524c2a3c9e3af90901c3"

+SRCREV = "097ca3e245200c4a4333964af59a106c42ff3bca"
  S = "${WORKDIR}/git"
  PV = "1.9.0+git${SRCPV}"
  



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] pseudo: Upgrade to latest to fix openat() with a directory symlink

[Jason Wessel]

While working with ostree disk generation in conjunction with wic, I
found a problem with pseudo where it tried to resolve a symlink when
it shouldn't, based on openat() flags.  I narrowed down the problem to
a simple c program to reproduce the issue:

int main()
{
/* Tested with: gcc -Wall -o app app.c ; echo "no pseudo" ; ./app ; echo 
"pseudo"; pseudo ./app */
system("rm -rf tdir tlink");
system("mkdir tdir");
system("ln -s tdir tlink");
DIR *dir = opendir(".");
int dfd = dirfd(dir);

int target_dfd = openat (dfd, "tlink", O_RDONLY | O_NONBLOCK | O_DIRECTORY 
| O_CLOEXEC | O_NOFOLLOW);
if (target_dfd == -1) {
printf("This is right\n");
} else {
printf("This is broken\n");
}
return 0;
}

Many thanks to Peter Seebach for fixing the problem in the pseudo code
to use the same logic which was already there for the
AT_SYMLINK_NOFOLLOW.

Signed-off-by: Jason Wessel 
---
 meta/recipes-devtools/pseudo/pseudo_git.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/pseudo/pseudo_git.bb 
b/meta/recipes-devtools/pseudo/pseudo_git.bb
index 51db84c4d4..3350c3fabd 100644
--- a/meta/recipes-devtools/pseudo/pseudo_git.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_git.bb
@@ -8,7 +8,7 @@ SRC_URI = "git://git.yoctoproject.org/pseudo \
file://toomanyfiles.patch \
"
 
-SRCREV = "3fa7c853e0bcd6fe23f7524c2a3c9e3af90901c3"
+SRCREV = "097ca3e245200c4a4333964af59a106c42ff3bca"
 S = "${WORKDIR}/git"
 PV = "1.9.0+git${SRCPV}"
 
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/3] glibc/glibc-locale: Fix do_stash_locale to work with usrmerge and multilibs

[Jason Wessel]

On 7/6/19 2:31 PM, Adrian Bunk wrote:

On Thu, Jul 04, 2019 at 06:57:50AM -0500, Jason Wessel wrote:

On 7/4/19 5:57 AM, Adrian Bunk wrote:

On Thu, Jul 04, 2019 at 11:13:18AM +0200, Martin Jansa wrote:

I don't see how this change (or the previous glibc-locale one) would cause
that, but since the oe-core upgrade yesterday I'm seeing following
glibc-locale.do_package failure:
http://errors.yoctoproject.org/Errors/Details/250557/

DEBUG: Executing shell function do_prep_locale_tree
tar: i18n: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
gzip:
TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz:
No such file or directory
WARNING:
TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.23303:1
exit 1 from 'gunzip $i'

anyone else seeing this?


Yes.



I don't know that it was specifically this change that causes the problem but 
rather any change to this function.   If that is not the case can either of you 
send me your local.conf file, and I'll try to reproduce it.  If the following 
doesn't work around the problem, there is clearly another corner case that is 
not well understood.

bitbake -c cleansstate glibc glibc-locale
...


bitbake -c cleansstate glibc glibc-locale nativesdk-glibc nativesdk-glibc-locale

This seems to workaround the problem in the cases I have seen so far,
so what you said about the cause looks plausible to me.




I chatted with Richard to figure out how to fix the original implementation, so 
this will not continue to happen.  I am no fan of incremental or sstate based 
builds failing.  The pending fix is below.

https://patchwork.openembedded.org/patch/162886/

Cheers,
Jason.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] glibc / glibc-locale: Fix stash_locale determinism problems

[Jason Wessel]

When using sstate, or performing an incremental build any change to
the do_stash_locale() will cause a build failure because
do_stash_locale() was destroying the results obtained from the
do_install() with several mv operations.  A recent change to
do_stash_locale() for a different problem illustrated a number of
build failures for users in the community.

To fix the problem, do_stash_locale() must use copy operations instead
of the mv operations.  Because this is changed to a copy, the sysroot
and package stage need to remove the files that would have been
previously removed.  The correct "fixup" code to deal with the removal
already existed in the previous do_poststash_install_cleanup().  All
that needed change was the path to where to remove the files
from the sysroot and package stages.

In order to force a re-compilation of glibc some unused white space
was removed from do_compile() for glibc.  I could not find any other
way around this and we don't want to have all the community folks to
have another iteration where they have to remove their tmp directories
or purge some portion of the sstate.  It also makes this change
bisectable. If the change to the glibc is not included, it will fail
with the following message:

=
| DEBUG: Executing shell function do_prep_locale_tree
| tar: i18n: Cannot stat: No such file or directory
| tar: Exiting with failure status due to previous errors
| gzip: 
/poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz:
 No such file or directory
=

After this one time change I tested changing only the
do_stash_locale() function and it now works well because it is
deterministically operating off the sstate data or a local build.

Signed-off-by: Jason Wessel 
---
 meta/recipes-core/glibc/glibc-package.inc | 40 ++-
 meta/recipes-core/glibc/glibc_2.29.bb |  1 -
 2 files changed, 24 insertions(+), 17 deletions(-)

diff --git a/meta/recipes-core/glibc/glibc-package.inc 
b/meta/recipes-core/glibc/glibc-package.inc
index ff17a193c3..5cfb1b6ab9 100644
--- a/meta/recipes-core/glibc/glibc-package.inc
+++ b/meta/recipes-core/glibc/glibc-package.inc
@@ -164,13 +164,13 @@ do_stash_locale () {
install -d ${dest}${base_libdir} ${dest}${bindir} ${dest}${libdir} 
${dest}${datadir}
# Hide away the locale data from the deployment
if [ -e ${D}${bindir}/localedef ]; then
-   mv -f ${D}${bindir}/localedef ${dest}${bindir}
+   cp -a ${D}${bindir}/localedef ${dest}${bindir}
fi
if [ -e ${D}${libdir}/gconv ]; then
-   mv -f ${D}${libdir}/gconv ${dest}${libdir}
+   cp -a ${D}${libdir}/gconv ${dest}${libdir}
fi
if [ -e ${D}${datadir}/i18n ]; then
-   mv ${D}${datadir}/i18n ${dest}${datadir}
+   cp -a  ${D}${datadir}/i18n ${dest}${datadir}
fi
 
# Make a copy of all the libraries into the locale stash
@@ -210,30 +210,38 @@ python do_stash_locale_setscene () {
 }
 addtask do_stash_locale_setscene
 
-do_poststash_install_cleanup () {
-   # Remove all files which do_stash_locale would remove (mv)
-   # since that task could have come from sstate and not get run.
+PACKAGE_PREPROCESS_FUNCS += "stash_locale_package_cleanup"
+SYSROOT_PREPROCESS_FUNCS += "stash_locale_sysroot_cleanup"
+stash_locale_cleanup () {
+   cleanupdir=$1
+   # Remove all files which do_stash_locale() copies
for i in ${bashscripts}; do
-   rm -f ${D}${bindir}/$i
+   rm -f ${cleanupdir}${bindir}/$i
done
-   rm -f ${D}${bindir}/localedef
-   rm -rf ${D}${datadir}/i18n
-   rm -rf ${D}${libdir}/gconv
-   rm -rf ${D}/${localedir}
-   rm -rf ${D}${datadir}/locale
+   rm -f ${cleanupdir}${bindir}/localedef
+   rm -rf ${cleanupdir}${datadir}/i18n
+   rm -rf ${cleanupdir}${libdir}/gconv
+   rm -rf ${cleanupdir}/${localedir}
+   rm -rf ${cleanupdir}${datadir}/locale
if [ "${libdir}" != "${exec_prefix}/lib" ] && [ "${root_prefix}/lib" != 
"${exec_prefix}/lib" ]; then
-   if [ -d "${D}${exec_prefix}/lib" ]; then
+   if [ -d "${cleanupdir}${exec_prefix}/lib" ]; then
if [ -z "${ARCH_DYNAMIC_LOADER}" -o \
-! -e "${D}${exec_prefix}/lib/${ARCH_DYNAMIC_LOADER}" ]; 
then
+! -e 
"${cleanupdir}${exec_prefix}/lib/${ARCH_DYNAMIC_LOADER}" ]; then
# error out if directory isn't empty
# this dir should only contain locale dir
# which has been deleted in the previous step
-   rmdir ${D}${exec_prefix}/lib
+   rmdir ${cleanupdir}${exec_prefix}/lib
fi
fi
fi
 }
-addtask do_poststash_install_cleanup after do

Re: [OE-core] [PATCH 3/3] glibc/glibc-locale: Fix do_stash_locale to work with usrmerge and multilibs

[Jason Wessel]

On 7/4/19 5:57 AM, Adrian Bunk wrote:

On Thu, Jul 04, 2019 at 11:13:18AM +0200, Martin Jansa wrote:

I don't see how this change (or the previous glibc-locale one) would cause
that, but since the oe-core upgrade yesterday I'm seeing following
glibc-locale.do_package failure:
http://errors.yoctoproject.org/Errors/Details/250557/

DEBUG: Executing shell function do_prep_locale_tree
tar: i18n: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
gzip:
TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz:
No such file or directory
WARNING:
TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.23303:1
exit 1 from 'gunzip $i'

anyone else seeing this?


Yes.




I don't know that it was specifically this change that causes the problem but 
rather any change to this function.   If that is not the case can either of you 
send me your local.conf file, and I'll try to reproduce it.  If the following 
doesn't work around the problem, there is clearly another corner case that is 
not well understood.

bitbake -c cleansstate glibc glibc-locale

I believe that any change to the .inc file which has locale stash function will cause the 
problem due to the fact it performs and "mv" operation to move the files.  When 
it runs a second time, there are no more files to move and you will not have them for use 
later on.  When I was experimenting earlier, I tried running the glibc 
temp/run.do_install script to put the files back, but then hit a problem with localdef 
being missing, because it comes from the temp/run.do_compile functions.

My conclusion thus far has been is that the ties between the glibc and 
glibc-locale are very subtle and break easily.  This is a separate problem than 
what I was attempting to fix, where the usrmerge + multilibs didn't even get 
the correct files in the first place.

Cheers,

Jason.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] glibc/glibc-locale: Fix do_stash_locale to work with usrmerge and multilibs

[Jason Wessel]

The do_stash_locale was not working consistently across the 4 build
configurations and the multilib, usrmerge configuration would fail
entirely with the obscure message:

| DEBUG: Executing shell function do_prep_locale_tree
| tar: i18n: Cannot stat: No such file or directory
| tar: Exiting with failure status due to previous errors
| gzip: 
/poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz:
 No such file or directory
| WARNING: 
/poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.124690:1
 exit 1 from 'gunzip $i'

Here is the 4 build configurations without the patch applied:

A) x86-64 no multilibs, no usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service 
|wc -l
909
B) x86-64 no multilibs, usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service 
|wc -l
909
C) x86-64 multilibs, no usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service 
|wc -l
885
D) x86-64 multilibs, usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service 
|wc -l
864

The issue here is that all the moves should be processed first, then a
copy should be made of the lib directories, but only in the case they
are different when using the usrmerge feature.  Even though the build
worked for the multilib configuration without usrmerge, the content
was not the same.

After applying the patch the same number of files are in all the
configurations.  The list of files was also diffed, after normalizing
the directory names to ensure all the correct files were copied.

Ultimately there are probably additional files that should be pruned
from what is copied to the stated_locale, but the purpose of this
patch is make it 100% consistent between the build types and fix the
builds.

Signed-off-by: Jason Wessel 
---
 meta/recipes-core/glibc/glibc-package.inc | 19 +--
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/meta/recipes-core/glibc/glibc-package.inc 
b/meta/recipes-core/glibc/glibc-package.inc
index a1d79b3075..ff17a193c3 100644
--- a/meta/recipes-core/glibc/glibc-package.inc
+++ b/meta/recipes-core/glibc/glibc-package.inc
@@ -162,21 +162,28 @@ bashscripts = "mtrace sotruss xtrace"
 do_stash_locale () {
dest=${LOCALESTASH}
install -d ${dest}${base_libdir} ${dest}${bindir} ${dest}${libdir} 
${dest}${datadir}
-   if [ "${base_libdir}" != "${libdir}" ]; then
-   cp -fpPR ${D}${base_libdir}/* ${dest}${base_libdir}
-   fi
+   # Hide away the locale data from the deployment
if [ -e ${D}${bindir}/localedef ]; then
mv -f ${D}${bindir}/localedef ${dest}${bindir}
fi
if [ -e ${D}${libdir}/gconv ]; then
mv -f ${D}${libdir}/gconv ${dest}${libdir}
fi
-   if [ -e ${D}${exec_prefix}/lib ]; then
-   cp -fpPR ${D}${exec_prefix}/lib ${dest}${exec_prefix}
-   fi
if [ -e ${D}${datadir}/i18n ]; then
mv ${D}${datadir}/i18n ${dest}${datadir}
fi
+
+   # Make a copy of all the libraries into the locale stash
+   cp -fpPR ${D}${libdir}/* ${dest}${libdir}
+   if [ "${base_libdir}" != "${libdir}" ]; then
+   cp -fpPR ${D}${base_libdir}/* ${dest}${base_libdir}
+   fi
+   if [ -e ${D}${exec_prefix}/lib ]; then
+   if [ ${exec_prefix}/lib != ${base_libdir} ] && [ 
${exec_prefix}/lib != ${libdir} ]; then
+   cp -fpPR ${D}${exec_prefix}/lib ${dest}${exec_prefix}
+   fi
+   fi
+
cp -fpPR ${D}${datadir}/* ${dest}${datadir}
rm -rf ${D}${datadir}/locale/
cp -fpPR ${WORKDIR}/SUPPORTED ${dest}
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] glibc-locale: Fix build error with PACKAGE_NO_GCONV = "1"

[Jason Wessel]

When the PACKAGE_NO_GCONV is set to 1 an empty directory is left behind from 
the do_install rule:

=
ERROR: glibc-locale-2.29-r0 do_package: QA Issue: glibc-locale: 
Files/directories were installed but not shipped in any package:
  /usr/lib
  /usr/lib/locale
Please set FILES such that these items are packaged. Alternatively if they are 
unneeded, avoid installing them or delete them within do_install.
glibc-locale: 2 installed and not shipped files. [installed-vs-shipped]
ERROR: glibc-locale-2.29-r0 do_package: Fatal QA errors found, failing task.
=

The simple fix is to prune the empty directory.

Signed-off-by: Jason Wessel 
---
 meta/recipes-core/glibc/glibc-locale.inc | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
b/meta/recipes-core/glibc/glibc-locale.inc
index f033635bd0..bf5eaee938 100644
--- a/meta/recipes-core/glibc/glibc-locale.inc
+++ b/meta/recipes-core/glibc/glibc-locale.inc
@@ -87,6 +87,9 @@ do_install() {
if [ ${PACKAGE_NO_GCONV} -eq 0 ]; then
copy_locale_files ${libdir}/gconv 0755
copy_locale_files ${datadir}/i18n 0644
+   else
+   # Remove the libdir if it is empty when gconv is not copied
+   find ${D}${libdir} -type d -empty -delete
fi
copy_locale_files ${datadir}/locale 0644
install -m 0644 ${LOCALETREESRC}/SUPPORTED ${WORKDIR}/SUPPORTED
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] Fixes for build failures for multilibs + usrmerge, NLS and GCONV

[Jason Wessel]

Here are a few more fixes for usrmerge, multilibs, NLS, and GCONV.

Thanks,
Jason.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] psmisc: Fix dependency for USE_NLS=no

[Jason Wessel]

When using USE_NLS="no" in the local.conf psmisc will fail to
compile as follows:

| autoreconf: Entering directory `.'
| autoreconf: running: autopoint --force
| autoreconf: failed to run autopoint: No such file or directory
| autoreconf: autopoint is needed because this package uses Gettext
| ERROR: autoreconf execution failed.

This is because the gettext.bbclass returns gettext-minimal-native for
the host dependency which does not include autopoint.  The autopoint
utility is required to build psmisc, so it needs to list
gettext-native as a dependency.

Signed-off-by: Jason Wessel 
---
 meta/recipes-extended/psmisc/psmisc.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-extended/psmisc/psmisc.inc 
b/meta/recipes-extended/psmisc/psmisc.inc
index 82ef947709..594a10cf22 100644
--- a/meta/recipes-extended/psmisc/psmisc.inc
+++ b/meta/recipes-extended/psmisc/psmisc.inc
@@ -7,7 +7,7 @@ command sends a specified signal (SIGTERM if nothing is 
specified) to \
 processes identified by name.  The fuser command identifies the PIDs \
 of processes that are using specified files or filesystems."
 SECTION = "base"
-DEPENDS = "ncurses virtual/libintl"
+DEPENDS = "ncurses virtual/libintl gettext-native"
 LICENSE = "GPLv2"
 
 SRC_URI = "${SOURCEFORGE_MIRROR}/psmisc/psmisc-${PV}.tar.gz"
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] glibc: Fix multilibs + usrmerge builds

[Jason Wessel]

The build of glibc fails when you have multilibs enabled + the distro
feature usrmerge.  Here is an example configuration:

===
MACHINE = "qemux86-64"
VIRTUAL-RUNTIME_init_manager = "systemd"
DISTRO_FEATURES_append = " systemd "
DISTRO_FEATURES_append += " usrmerge"

require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
===

This will fail with the following error:

NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
ERROR: glibc-2.28-r0 do_poststash_install_cleanup: Function failed: 
do_poststash_install_cleanup (log file is located at 
/poky/build/tmp/work/core2-64-poky-linux/glibc/2.28-r0/temp/log.do_poststash_install_cleanup.107893)
ERROR: Logfile of failure stored in: 
/poky/build/tmp/work/core2-64-poky-linux/glibc/2.28-r0/temp/log.do_poststash_install_cleanup.107893

The fix is to not perform the rmdir check when using the multilib + usr/merge, 
namely:

if [ "${libdir}" != "${exec_prefix}/lib" ] && [ "${root_prefix}/lib" != 
"${exec_prefix}/lib" ]; then

This will evaluate as follows (collecting the output from bitbake -e glibc)

* no multilibs no usrmerge
if [ "/usr/lib" != "/usr/lib" ] && [ "/lib" != "/usr/lib" ]; then
* no multilibs yes usrmerge
if [ "/usr/lib" != "/usr/lib" ] && [ "/usr/lib" != "/usr/lib" ]; then
* yes multilibs no usrmerge
    if [ "/usr/lib64" != "/usr/lib" ] && [ "/lib" != "/usr/lib" ]; then
* yes multilibs yes user merge
if [ "/usr/lib64" != "/usr/lib" ] && [ "/usr/lib" != "/usr/lib" ]; then

Signed-off-by: Jason Wessel 
---
 meta/recipes-core/glibc/glibc-package.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/glibc/glibc-package.inc 
b/meta/recipes-core/glibc/glibc-package.inc
index b7c64a0b2e..a1d79b3075 100644
--- a/meta/recipes-core/glibc/glibc-package.inc
+++ b/meta/recipes-core/glibc/glibc-package.inc
@@ -214,7 +214,7 @@ do_poststash_install_cleanup () {
rm -rf ${D}${libdir}/gconv
rm -rf ${D}/${localedir}
rm -rf ${D}${datadir}/locale
-   if [ "${libdir}" != "${exec_prefix}/lib" ]; then
+   if [ "${libdir}" != "${exec_prefix}/lib" ] && [ "${root_prefix}/lib" != 
"${exec_prefix}/lib" ]; then
if [ -d "${D}${exec_prefix}/lib" ]; then
if [ -z "${ARCH_DYNAMIC_LOADER}" -o \
 ! -e "${D}${exec_prefix}/lib/${ARCH_DYNAMIC_LOADER}" ]; 
then
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] glibc: Fix multilibs + usrmerge builds

[Jason Wessel]

This version was against the thud branch.  I'll send a v2 for the master branch 
since there was some fuzz that the auto patch checker couldn't deal with.  The 
same problem exists in thud and the master branch.

Cheers,
Jason.


On 6/29/19 7:41 PM, Jason Wessel wrote:

The build of glibc fails when you have multilibs enabled + the distro
feature usrmerge.  Here is an example configuration:

===
MACHINE = "qemux86-64"
VIRTUAL-RUNTIME_init_manager = "systemd"
DISTRO_FEATURES_append = " systemd "
DISTRO_FEATURES_append += " usrmerge"

require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
===

This will fail with the following error:

NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
ERROR: glibc-2.28-r0 do_poststash_install_cleanup: Function failed: 
do_poststash_install_cleanup (log file is located at 
/poky/build/tmp/work/core2-64-poky-linux/glibc/2.28-r0/temp/log.do_poststash_install_cleanup.107893)
ERROR: Logfile of failure stored in: 
/poky/build/tmp/work/core2-64-poky-linux/glibc/2.28-r0/temp/log.do_poststash_install_cleanup.107893

The fix is to not perform the rmdir check when using the multilib + usr/merge, 
namely:

if [ "${libdir}" != "${exec_prefix}/lib" ] && [ "${root_prefix}/lib" != 
"${exec_prefix}/lib" ]; then

This will evaluate as follows (collecting the output from bitbake -e glibc)

* no multilibs no usrmerge
 if [ "/usr/lib" != "/usr/lib" ] && [ "/lib" != "/usr/lib" ]; then
* no multilibs yes usrmerge
 if [ "/usr/lib" != "/usr/lib" ] && [ "/usr/lib" != "/usr/lib" ]; then
* yes multilibs no usrmerge
     if [ "/usr/lib64" != "/usr/lib" ] && [ "/lib" != "/usr/lib" ]; then
* yes multilibs yes user merge
 if [ "/usr/lib64" != "/usr/lib" ] && [ "/usr/lib" != "/usr/lib" ]; then

Signed-off-by: Jason Wessel 
---
  meta/recipes-core/glibc/glibc-package.inc | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/glibc/glibc-package.inc 
b/meta/recipes-core/glibc/glibc-package.inc
index a98ae1a29c..3648612f76 100644
--- a/meta/recipes-core/glibc/glibc-package.inc
+++ b/meta/recipes-core/glibc/glibc-package.inc
@@ -206,7 +206,7 @@ do_poststash_install_cleanup () {
rm -rf ${D}${libdir}/gconv
rm -rf ${D}/${localedir}
rm -rf ${D}${datadir}/locale
-   if [ "${libdir}" != "${exec_prefix}/lib" ]; then
+   if [ "${libdir}" != "${exec_prefix}/lib" ] && [ "${root_prefix}/lib" != 
"${exec_prefix}/lib" ]; then
if [ -d ${D}${exec_prefix}/lib ]; then
# error out if directory isn't empty
# this dir should only contain locale dir



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] glibc: Fix multilibs + usrmerge builds

[Jason Wessel]

The build of glibc fails when you have multilibs enabled + the distro
feature usrmerge.  Here is an example configuration:

===
MACHINE = "qemux86-64"
VIRTUAL-RUNTIME_init_manager = "systemd"
DISTRO_FEATURES_append = " systemd "
DISTRO_FEATURES_append += " usrmerge"

require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
===

This will fail with the following error:

NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
ERROR: glibc-2.28-r0 do_poststash_install_cleanup: Function failed: 
do_poststash_install_cleanup (log file is located at 
/poky/build/tmp/work/core2-64-poky-linux/glibc/2.28-r0/temp/log.do_poststash_install_cleanup.107893)
ERROR: Logfile of failure stored in: 
/poky/build/tmp/work/core2-64-poky-linux/glibc/2.28-r0/temp/log.do_poststash_install_cleanup.107893

The fix is to not perform the rmdir check when using the multilib + usr/merge, 
namely:

if [ "${libdir}" != "${exec_prefix}/lib" ] && [ "${root_prefix}/lib" != 
"${exec_prefix}/lib" ]; then

This will evaluate as follows (collecting the output from bitbake -e glibc)

* no multilibs no usrmerge
if [ "/usr/lib" != "/usr/lib" ] && [ "/lib" != "/usr/lib" ]; then
* no multilibs yes usrmerge
if [ "/usr/lib" != "/usr/lib" ] && [ "/usr/lib" != "/usr/lib" ]; then
* yes multilibs no usrmerge
    if [ "/usr/lib64" != "/usr/lib" ] && [ "/lib" != "/usr/lib" ]; then
* yes multilibs yes user merge
if [ "/usr/lib64" != "/usr/lib" ] && [ "/usr/lib" != "/usr/lib" ]; then

Signed-off-by: Jason Wessel 
---
 meta/recipes-core/glibc/glibc-package.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/glibc/glibc-package.inc 
b/meta/recipes-core/glibc/glibc-package.inc
index a98ae1a29c..3648612f76 100644
--- a/meta/recipes-core/glibc/glibc-package.inc
+++ b/meta/recipes-core/glibc/glibc-package.inc
@@ -206,7 +206,7 @@ do_poststash_install_cleanup () {
rm -rf ${D}${libdir}/gconv
rm -rf ${D}/${localedir}
rm -rf ${D}${datadir}/locale
-   if [ "${libdir}" != "${exec_prefix}/lib" ]; then
+   if [ "${libdir}" != "${exec_prefix}/lib" ] && [ "${root_prefix}/lib" != 
"${exec_prefix}/lib" ]; then
if [ -d ${D}${exec_prefix}/lib ]; then
# error out if directory isn't empty
# this dir should only contain locale dir
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] gobject-introspection.bbclass: Disable writing to $HOME/.cache/g-ir-scanner

[Jason Wessel]

On 4/11/19 3:01 PM, Alexander Kanavin wrote:
> Yes, I agree.
> 
> Alex
> 
>> On 11 Apr 2019, at 21.59, Richard Purdie 
>>  wrote:
>>
>>> On Thu, 2019-04-11 at 11:36 -0500, Jason Wessel wrote:
>>>
>>> It all appears to work fine without the need to patch anything in
>>> master-next.  
>>>
>>> Thank you for the explanation.
>>
>> This is starting to look like something we probably want to pull into
>> the warrior release given the problems with meson converted recipes
>> otherwise?
>>


I concur, with Alex.  I talked to Mark Hatle about this earlier today.  If it 
is not fixed prior to the release of the warrior branch we were going to submit 
it as a back port, due to the fact this causes build stability problems. 

Jason. 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] gobject-introspection.bbclass: Disable writing to $HOME/.cache/g-ir-scanner

[Jason Wessel]

On 4/11/19 10:35 AM, Alexander Kanavin wrote:
> On Thu, 11 Apr 2019 at 17:25, Jason Wessel  wrote:
>>> Generally because generating introspection data is disabled for native
>>> packages, as a matter of policy (do not build something which is
>>> unused and untested).
>>>
>>> If you could add
>>> EXTRA_OEMESON_append_class-native = " ${GI_DISABLE_FLAG}"
>>>
>>> to the gdk-pixbuf recipe, and re-test (without your fix, to be extra
>>> sure) that g-ir-scanner is not invoked and nothing shows up in its
>>> cache, I would appreciate.
>>
>>
>> Thank you for the explanation.  I think the problem is more widespread than 
>> just gdk-pixbuf, now that I have a slightly better idea about what we are 
>> looking for.  My build is not even complete and I can see there are three 
>> additional places the g-ir-scanner is creating .cache entries.
>>
>> /poky/build% grep -r /usr/bin/g-ir-scanner tmp/work/*/*/*/temp |awk -F/ 
>> '{print $4}' |sort -u
>> atk-native
>> at-spi2-core-native
>> pango-native
>>
>>
>> Is it the case the bbclass file should be altered to set this variable for 
>> the -native, or does each package need to be modified individually?
> 
> I think Andreas's patchset that was recently posted (and is currently
> in master-next) takes care of this in a general way, where you don't
> have to go and fix every recipe. The unfortunate bit is that meson
> does not have a standard flag for enabling/disabling introspection,
> but we found a way to deal with this that is as generic as possible.
> 
> If you try with master-next, would be good to know.
> 

It all appears to work fine without the need to patch anything in master-next.  

Thank you for the explanation.

Jason.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] gobject-introspection.bbclass: Disable writing to $HOME/.cache/g-ir-scanner

[Jason Wessel]

On 4/11/19 9:44 AM, Alexander Kanavin wrote:
> Generally because generating introspection data is disabled for native
> packages, as a matter of policy (do not build something which is
> unused and untested).
> 
> If you could add
> EXTRA_OEMESON_append_class-native = " ${GI_DISABLE_FLAG}"
> 
> to the gdk-pixbuf recipe, and re-test (without your fix, to be extra
> sure) that g-ir-scanner is not invoked and nothing shows up in its
> cache, I would appreciate.


Thank you for the explanation.  I think the problem is more widespread than 
just gdk-pixbuf, now that I have a slightly better idea about what we are 
looking for.  My build is not even complete and I can see there are three 
additional places the g-ir-scanner is creating .cache entries.  

/poky/build% grep -r /usr/bin/g-ir-scanner tmp/work/*/*/*/temp |awk -F/ '{print 
$4}' |sort -u 
atk-native
at-spi2-core-native
pango-native


Is it the case the bbclass file should be altered to set this variable for the 
-native, or does each package need to be modified individually? 

Jason. 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] gobject-introspection.bbclass: Disable writing to $HOME/.cache/g-ir-scanner

[Jason Wessel]

Please understand I have no idea how gdk-pixbuf-native is supposed to compile.  
Perhaps you can elaborate further as to why g-ir-scanner cannot be called in 
this case.  I did not observe any obvious way that the g-ir-scanner was 
explicitly disabled for the build of the -native package with in the 
do_configure stage. 

Looking at the log files from the compilation phase it appears to use meson as 
the build tool.  The compile log file indicates that it is using the pkg-config 
tool to find g-ir-scanner and subsequently use it. 


Log file: 
x86_64-linux/gdk-pixbuf-native/2.38.0-r0/build/meson-logs/meson-log.txt
===
Called 
`/poky/build/tmp/work/x86_64-linux/gdk-pixbuf-native/2.38.0-r0/recipe-sysroot-native/usr/bin/pkg-config-native
 --variable=g_ir_scanner gobject-introspection-1.0` -> 0
/poky/build/tmp/work/x86_64-linux/gdk-pixbuf-native/2.38.0-r0/recipe-sysroot-native/usr/lib/pkgconfig/../../../usr/bin/g-ir-scanner
Got pkgconfig variable g_ir_scanner : 
/poky/build/tmp/work/x86_64-linux/gdk-pixbuf-native/2.38.0-r0/recipe-sysroot-native/usr/lib/pkgconfig/../../../usr/bin/g-ir-scanner

===


Jason.

On 4/11/19 6:14 AM, Alexander Kanavin wrote:
> gdk-pixbuf-native should not be running g-ir-scanner at all, as we
> explicitly disable this for native builds. Can you please look into
> why it is happening, because the patch is really treating the symptom
> rather than the problem.
> 
> Alex
> 
> On Wed, 10 Apr 2019 at 23:54, Jason Wessel  wrote:
>>
>> Investigation of an end user going over their quota in the user's home
>> directory showed that the gobject-instrospection is writing files into
>> the ~/.cache/g-ir-scanner when building -native packages.  The test
>> package in this case was gdk-pixbuf-native.
>>
>> Setting the GI_SCANNER_DISABLE_CACHE in the bbclass should be all that
>> is needed.  The qemu wrapper was already setting this properly, but we
>> also need it for the -native builds.
>>
>> meta/recipes-gnome/gobject-introspection/gobject-introspection_1.58.3.bb:131:
>> export GI_SCANNER_DISABLE_CACHE=1
>>
>> Signed-off-by: Jason Wessel 
>> ---
>>  meta/classes/gobject-introspection.bbclass | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/meta/classes/gobject-introspection.bbclass 
>> b/meta/classes/gobject-introspection.bbclass
>> index 4ceb0c68b1..359620fba7 100644
>> --- a/meta/classes/gobject-introspection.bbclass
>> +++ b/meta/classes/gobject-introspection.bbclass
>> @@ -27,6 +27,9 @@ DEPENDS_append_class-nativesdk = " 
>> gobject-introspection-native"
>>  # This is used by introspection tools to find .gir includes
>>  export XDG_DATA_DIRS = "${STAGING_DATADIR}:${STAGING_LIBDIR}"
>>
>> +# Disable writing to $HOME/.cache/g-ir-scanner
>> +export GI_SCANNER_DISABLE_CACHE = "1"
>> +
>>  do_configure_prepend_class-target () {
>>  # introspection.m4 pre-packaged with upstream tarballs does not yet
>>  # have our fixes
>> --
>> 2.17.1
>>
>> --
>> ___
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] gobject-introspection.bbclass: Disable writing to $HOME/.cache/g-ir-scanner

[Jason Wessel]

Investigation of an end user going over their quota in the user's home
directory showed that the gobject-instrospection is writing files into
the ~/.cache/g-ir-scanner when building -native packages.  The test
package in this case was gdk-pixbuf-native.

Setting the GI_SCANNER_DISABLE_CACHE in the bbclass should be all that
is needed.  The qemu wrapper was already setting this properly, but we
also need it for the -native builds.

meta/recipes-gnome/gobject-introspection/gobject-introspection_1.58.3.bb:131:
export GI_SCANNER_DISABLE_CACHE=1

Signed-off-by: Jason Wessel 
---
 meta/classes/gobject-introspection.bbclass | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/classes/gobject-introspection.bbclass 
b/meta/classes/gobject-introspection.bbclass
index 4ceb0c68b1..359620fba7 100644
--- a/meta/classes/gobject-introspection.bbclass
+++ b/meta/classes/gobject-introspection.bbclass
@@ -27,6 +27,9 @@ DEPENDS_append_class-nativesdk = " 
gobject-introspection-native"
 # This is used by introspection tools to find .gir includes
 export XDG_DATA_DIRS = "${STAGING_DATADIR}:${STAGING_LIBDIR}"
 
+# Disable writing to $HOME/.cache/g-ir-scanner
+export GI_SCANNER_DISABLE_CACHE = "1"
+
 do_configure_prepend_class-target () {
 # introspection.m4 pre-packaged with upstream tarballs does not yet
 # have our fixes
-- 
2.17.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/3] boost: Fix link problems with caffe and Movidius SDK

[Jason Wessel]

I can't seem to find where cmake's python checker magically decides the name of 
the shared object, but I did find an override, so you can drop the patch.   I 
had even gone so far as to simply make a zero length file instead of the 
symlink, and everything still built and run.  It is just some cmake detection 
which is broken.  The ubuntu/debian systems where this code is heavily tested 
all provide a link for the libboost_python.so which explains why this never 
turned up previously.  They don't get the option to pick python2 or python3 for 
boost, they always get both for now.

The override to build with python boost for cmake and caffe is as follows:

cmake -DBoost_PYTHON_LIBRARY_DEBUG=/usr/lib64/libboost_python3.so \
   -DBoost_PYTHON_LIBRARY_RELEASE=/usr/lib64/libboost_python3.so \
   -DBLAS=Open \
   ..


Cheers,
Jason.


On 03/14/2018 09:55 AM, Jason Wessel wrote:

Perhaps a bug in the cmake detect which is part of boost, because I am not 
using python2 for caffe.

I used the path of least resistance because as far as I could tell boost only 
let you build one or the other and in my case it was the python3 boost api. In 
the end caffe was working and all the python3 tests with the Movidius SDK pass 
(which use python3).

Jason.

On 03/14/2018 09:50 AM, Burton, Ross wrote:

I thought that boost_python was the py2 binary, and boost_python3 was the py3 
one?  Is this not a bug in caffe?

On 13 March 2018 at 03:08, Jason Wessel <jason.wes...@windriver.com 
<mailto:jason.wes...@windriver.com>> wrote:

When using an image built with the Yocto Project which has a compiler
and all the required libraries, projects such as caffe for the
Movidius SDK which use python3 and boost fail to build because they
look for libboost_python.so.  The error that cmake returns doesn't
even point to the fact that this is why the configuration fails.

Example showing the problem with the missing symlink:

  git clone https://github.com/weiliu89/caffe.git 
<https://github.com/weiliu89/caffe.git>
  cd caffe
  mkdir build
  cd build
  cmake -DBLAS=Open ..

-- Configuring done
CMake Error at CMakeLists.txt:85 (add_dependencies):
  The dependency target "pycaffe" of target "pytest" does not exist.


Conditionally creating the link when building python3 support into
boost is all that is needed.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com 
<mailto:jason.wes...@windriver.com>>
---
 meta/recipes-support/boost/boost.inc | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-support/boost/boost.inc 
b/meta/recipes-support/boost/boost.inc
index 0461ec6..c53edf7 100644
--- a/meta/recipes-support/boost/boost.inc
+++ b/meta/recipes-support/boost/boost.inc
@@ -195,6 +195,9 @@ do_install() {
                fi
        done

+       if [ -e ${D}${libdir}/libboost_python3.so ]; then
+               ln -s libboost_python3.so ${D}${libdir}/libboost_python.so
+       fi
 }

 BBCLASSEXTEND = "native nativesdk"
--
2.7.4

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org 
<mailto:Openembedded-core@lists.openembedded.org>
http://lists.openembedded.org/mailman/listinfo/openembedded-core 
<http://lists.openembedded.org/mailman/listinfo/openembedded-core>






-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/3] alsa-lib: Add an RPROVIDES for alsa-lib

[Jason Wessel]

# dnf install python-pyalsaaudio
Last metadata expiration check: 0:00:44 ago on Wed Mar 14 11:15:23 2018.
Error:
 Problem: conflicting requests
  - nothing provides alsa-lib needed by python-pyalsaaudio-0.8.4-r0.1.core2_64

---
The python-pyalsaaudio recipe in question did the following:

DEPENDS += "alsa-lib"

RDEPENDS_${PN} += "\
    alsa-lib \
"

Obviously since nothing provides that, it isn't going to work. It is probably 
better in this case to change the python recipe to RDEPENDS on libasound which 
is what the alsa-lib recipe generates.  I can understand how the author of the 
python recipe might have been confused, but it begs the question of how it 
worked in the first place.  :-)

I can reach out to the owner of the layer that provided the recipe and you can 
drop this patch.

Jason.

On 03/14/2018 09:51 AM, Burton, Ross wrote:

This sounds very odd.  Can you give an example of this happening?

Ross

On 13 March 2018 at 03:08, Jason Wessel <jason.wes...@windriver.com 
<mailto:jason.wes...@windriver.com>> wrote:

There are a number of other recipes outside oe-core that depend on
alsa-lib and the rpm4 dependencies resolve sometimes to include
alsa-lib and the packages in the other layers fail to install because
the dependency cannot be resolved, despite the fact that libasound is
installed.

The simple is to add an RPROVIDES for alsa-lib to libasound which
provides all the shared objects for alsa.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com 
<mailto:jason.wes...@windriver.com>>
---
 meta/recipes-multimedia/alsa/alsa-lib_1.1.5.bb <http://alsa-lib_1.1.5.bb> 
| 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-multimedia/alsa/alsa-lib_1.1.5.bb 
<http://alsa-lib_1.1.5.bb> b/meta/recipes-multimedia/alsa/alsa-lib_1.1.5.bb 
<http://alsa-lib_1.1.5.bb>
index c5bf107..f7f01e8 100644
--- a/meta/recipes-multimedia/alsa/alsa-lib_1.1.5.bb 
<http://alsa-lib_1.1.5.bb>
+++ b/meta/recipes-multimedia/alsa/alsa-lib_1.1.5.bb 
<http://alsa-lib_1.1.5.bb>
@@ -25,6 +25,7 @@ FILES_alsa-server = "${bindir}/*"
 FILES_alsa-conf = "${datadir}/alsa/"

 RDEPENDS_libasound = "alsa-conf"
+RPROVIDES_libasound = "alsa-lib"

 # alsa-lib gets automatically added to alsa-lib-dev dependencies, but the
 # alsa-lib package doesn't exist. libasound is the real library package.
--
2.7.4

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org 
<mailto:Openembedded-core@lists.openembedded.org>
http://lists.openembedded.org/mailman/listinfo/openembedded-core 
<http://lists.openembedded.org/mailman/listinfo/openembedded-core>




-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/3] boost: Fix link problems with caffe and Movidius SDK

[Jason Wessel]

Perhaps a bug in the cmake detect which is part of boost, because I am not 
using python2 for caffe.

I used the path of least resistance because as far as I could tell boost only 
let you build one or the other and in my case it was the python3 boost api. In 
the end caffe was working and all the python3 tests with the Movidius SDK pass 
(which use python3).

Jason.

On 03/14/2018 09:50 AM, Burton, Ross wrote:

I thought that boost_python was the py2 binary, and boost_python3 was the py3 
one?  Is this not a bug in caffe?

On 13 March 2018 at 03:08, Jason Wessel <jason.wes...@windriver.com 
<mailto:jason.wes...@windriver.com>> wrote:

When using an image built with the Yocto Project which has a compiler
and all the required libraries, projects such as caffe for the
Movidius SDK which use python3 and boost fail to build because they
look for libboost_python.so.  The error that cmake returns doesn't
even point to the fact that this is why the configuration fails.

Example showing the problem with the missing symlink:

  git clone https://github.com/weiliu89/caffe.git 
<https://github.com/weiliu89/caffe.git>
  cd caffe
  mkdir build
  cd build
  cmake -DBLAS=Open ..

-- Configuring done
CMake Error at CMakeLists.txt:85 (add_dependencies):
  The dependency target "pycaffe" of target "pytest" does not exist.


Conditionally creating the link when building python3 support into
boost is all that is needed.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com 
<mailto:jason.wes...@windriver.com>>
---
 meta/recipes-support/boost/boost.inc | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-support/boost/boost.inc 
b/meta/recipes-support/boost/boost.inc
index 0461ec6..c53edf7 100644
--- a/meta/recipes-support/boost/boost.inc
+++ b/meta/recipes-support/boost/boost.inc
@@ -195,6 +195,9 @@ do_install() {
                fi
        done

+       if [ -e ${D}${libdir}/libboost_python3.so ]; then
+               ln -s libboost_python3.so ${D}${libdir}/libboost_python.so
+       fi
 }

 BBCLASSEXTEND = "native nativesdk"
--
2.7.4

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org 
<mailto:Openembedded-core@lists.openembedded.org>
http://lists.openembedded.org/mailman/listinfo/openembedded-core 
<http://lists.openembedded.org/mailman/listinfo/openembedded-core>




-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] python3: Fix purelib install and runtime paths

[Jason Wessel]

oe-core commit: 45afadf0b6 fixed the pip problem with purelib for
python2, even though the the patch stated it was for python3.  This
patch addresses the purelib problem for python3.

If you install the package python3-pip you will have a pip3 binary
where you can see the problem on the device easily where the modules
install into the incorrect area and are not able to be referenced by
python3 at all.

Example error:
   pip3 install imutils
   pip3 list |grep imutils || echo ERROR no imutils
  ERROR no imutils
   python3 -c 'import imutils'
 Traceback (most recent call last):
   File "", line 1, in 
 ImportError: No module named 'imutils'

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 .../python/python3/python-3.3-multilib.patch| 21 +
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/python-3.3-multilib.patch 
b/meta/recipes-devtools/python/python3/python-3.3-multilib.patch
index 77da615..cc35dc1 100644
--- a/meta/recipes-devtools/python/python3/python-3.3-multilib.patch
+++ b/meta/recipes-devtools/python/python3/python-3.3-multilib.patch
@@ -52,11 +52,13 @@ Index: Python-3.5.4/Lib/distutils/command/install.py
  WINDOWS_SCHEME = {
  'purelib': '$base/Lib/site-packages',
  'platlib': '$base/Lib/site-packages',
-@@ -30,7 +32,7 @@ WINDOWS_SCHEME = {
+@@ -29,8 +31,8 @@ WINDOWS_SCHEME = {
+ 
  INSTALL_SCHEMES = {
  'unix_prefix': {
- 'purelib': '$base/lib/python$py_version_short/site-packages',
+-'purelib': '$base/lib/python$py_version_short/site-packages',
 -'platlib': '$platbase/lib/python$py_version_short/site-packages',
++'purelib': 
'$platbase/'+libname+'/python$py_version_short/site-packages',
 +'platlib': 
'$platbase/'+libname+'/python$py_version_short/site-packages',
  'headers': 
'$base/include/python$py_version_short$abiflags/$dist_name',
  'scripts': '$base/bin',
@@ -103,10 +105,11 @@ Index: Python-3.5.4/Lib/sysconfig.py
  'posix_prefix': {
 -'stdlib': '{installed_base}/lib/python{py_version_short}',
 -'platstdlib': '{platbase}/lib/python{py_version_short}',
+-'purelib': '{base}/lib/python{py_version_short}/site-packages',
+-'platlib': '{platbase}/lib/python{py_version_short}/site-packages',
 +'stdlib': '{installed_base}/'+sys.lib+'/python{py_version_short}',
 +'platstdlib': '{platbase}/'+sys.lib+'/python{py_version_short}',
- 'purelib': '{base}/lib/python{py_version_short}/site-packages',
--'platlib': '{platbase}/lib/python{py_version_short}/site-packages',
++'purelib': 
'{platbase}/'+sys.lib+'/python{py_version_short}/site-packages',
 +'platlib': 
'{platbase}/'+sys.lib+'/python{py_version_short}/site-packages',
  'include':
  '{installed_base}/include/python{py_version_short}{abiflags}',
@@ -117,10 +120,11 @@ Index: Python-3.5.4/Lib/sysconfig.py
  'posix_home': {
 -'stdlib': '{installed_base}/lib/python',
 -'platstdlib': '{base}/lib/python',
+-'purelib': '{base}/lib/python',
+-'platlib': '{base}/lib/python',
 +'stdlib': '{installed_base}/'+sys.lib+'/python',
 +'platstdlib': '{base}/'+sys.lib+'/python',
- 'purelib': '{base}/lib/python',
--'platlib': '{base}/lib/python',
++'purelib': '{base}/'+sys.lib+'/python',
 +'platlib': '{base}/'+sys.lib+'/python',
  'include': '{installed_base}/include/python',
  'platinclude': '{installed_base}/include/python',
@@ -131,10 +135,11 @@ Index: Python-3.5.4/Lib/sysconfig.py
  'posix_user': {
 -'stdlib': '{userbase}/lib/python{py_version_short}',
 -'platstdlib': '{userbase}/lib/python{py_version_short}',
+-'purelib': '{userbase}/lib/python{py_version_short}/site-packages',
+-'platlib': '{userbase}/lib/python{py_version_short}/site-packages',
 +'stdlib': '{userbase}/'+sys.lib+'/python{py_version_short}',
 +'platstdlib': '{userbase}/'+sys.lib+'/python{py_version_short}',
- 'purelib': '{userbase}/lib/python{py_version_short}/site-packages',
--'platlib': '{userbase}/lib/python{py_version_short}/site-packages',
++'purelib': 
'{userbase}/'+sys.lib+'/python{py_version_short}/site-packages',
 +'platlib': 
'{userbase}/'+sys.lib+'/python{py_version_short}/site-packages',
  'include': '{userbase}/include/python{py_version_short}',
  'scripts': '{userbase}/bin',
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] boost: Fix link problems with caffe and Movidius SDK

[Jason Wessel]

When using an image built with the Yocto Project which has a compiler
and all the required libraries, projects such as caffe for the
Movidius SDK which use python3 and boost fail to build because they
look for libboost_python.so.  The error that cmake returns doesn't
even point to the fact that this is why the configuration fails.

Example showing the problem with the missing symlink:

  git clone https://github.com/weiliu89/caffe.git
  cd caffe
  mkdir build
  cd build
  cmake -DBLAS=Open ..

-- Configuring done
CMake Error at CMakeLists.txt:85 (add_dependencies):
  The dependency target "pycaffe" of target "pytest" does not exist.


Conditionally creating the link when building python3 support into
boost is all that is needed.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/recipes-support/boost/boost.inc | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-support/boost/boost.inc 
b/meta/recipes-support/boost/boost.inc
index 0461ec6..c53edf7 100644
--- a/meta/recipes-support/boost/boost.inc
+++ b/meta/recipes-support/boost/boost.inc
@@ -195,6 +195,9 @@ do_install() {
fi
done
 
+   if [ -e ${D}${libdir}/libboost_python3.so ]; then
+   ln -s libboost_python3.so ${D}${libdir}/libboost_python.so
+   fi
 }
 
 BBCLASSEXTEND = "native nativesdk"
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] alsa-lib: Add an RPROVIDES for alsa-lib

[Jason Wessel]

There are a number of other recipes outside oe-core that depend on
alsa-lib and the rpm4 dependencies resolve sometimes to include
alsa-lib and the packages in the other layers fail to install because
the dependency cannot be resolved, despite the fact that libasound is
installed.

The simple is to add an RPROVIDES for alsa-lib to libasound which
provides all the shared objects for alsa.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/recipes-multimedia/alsa/alsa-lib_1.1.5.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-multimedia/alsa/alsa-lib_1.1.5.bb 
b/meta/recipes-multimedia/alsa/alsa-lib_1.1.5.bb
index c5bf107..f7f01e8 100644
--- a/meta/recipes-multimedia/alsa/alsa-lib_1.1.5.bb
+++ b/meta/recipes-multimedia/alsa/alsa-lib_1.1.5.bb
@@ -25,6 +25,7 @@ FILES_alsa-server = "${bindir}/*"
 FILES_alsa-conf = "${datadir}/alsa/"
 
 RDEPENDS_libasound = "alsa-conf"
+RPROVIDES_libasound = "alsa-lib"
 
 # alsa-lib gets automatically added to alsa-lib-dev dependencies, but the
 # alsa-lib package doesn't exist. libasound is the real library package.
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] python3: Fix install purelib to make pip3-python work properly

[Jason Wessel]

The oe-core version of python3 patches the purelib use directory to
the system libdir so as to make it work with multilibs properly inside
the patch fix_for_using_different_libdir.patch with:

-'purelib': '{base}/lib/python{py_version_short}/site-packages',
+'purelib': '{base}/'+sys.lib+'/python{py_version_short}/site-packages',

The problem is that this broke the pip3-python package because the
install directory is out of sync when using a multilib version of
python.  When ever a module is installed with pip3 install that is a
purelib it will get installed to a location that python3 will never
reference and cause random failures.

This patch fixes the purelib install directory to match the purelib
use directory for externally managed python modules when using
multilibs.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/recipes-devtools/python/python/multilib.patch | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/python/python/multilib.patch 
b/meta/recipes-devtools/python/python/multilib.patch
index f5568d2d2b..94fdfeb630 100644
--- a/meta/recipes-devtools/python/python/multilib.patch
+++ b/meta/recipes-devtools/python/python/multilib.patch
@@ -41,11 +41,13 @@ Index: Python-2.7.13/Lib/distutils/command/install.py
  if sys.version < "2.2":
  WINDOWS_SCHEME = {
  'purelib': '$base',
-@@ -42,7 +44,7 @@ else:
+@@ -41,8 +43,8 @@ else:
+ 
  INSTALL_SCHEMES = {
  'unix_prefix': {
- 'purelib': '$base/lib/python$py_version_short/site-packages',
+-'purelib': '$base/lib/python$py_version_short/site-packages',
 -'platlib': '$platbase/lib/python$py_version_short/site-packages',
++'purelib': 
'$platbase/'+libname+'/python$py_version_short/site-packages',
 +'platlib': 
'$platbase/'+libname+'/python$py_version_short/site-packages',
  'headers': '$base/include/python$py_version_short/$dist_name',
  'scripts': '$base/bin',
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] package_manager.py: Print offending package instead of non-sense trace

[Jason Wessel]

If you have a package that does not generate a manifest due to using a
noexec rule, the package name should be printed so the problem can be
tracked down.  With out the patch you get an error that makes it look
more like the package_manager is broken as shown below.

oe-core/meta/lib/oe/package_manager.py', lineno: 534, function: 
create_packages_dir
 0530:
 0531:for dep in rpmdeps:
 0532:c = taskdepdata[dep][0]
 0533:manifest, d2 = oe.sstatesig.find_sstate_manifest(c, 
taskdepdata[dep][2], taskname, d, multilibs)
 *** 0534:if not os.path.exists(manifest):
 0535:continue
 0536:with open(manifest, "r") as f:
 0537:for l in f:
 0538:l = l.strip()
File: '/usr/lib/python3.5/genericpath.py', lineno: 19, function: exists
 0015:# This is false for dangling symbolic links on systems that support 
them.
 0016:def exists(path):
 0017:"""Test whether a path exists.  Returns False for broken symbolic 
links"""
 0018:try:
 *** 0019:os.stat(path)
 0020:except OSError:
 0021:return False
 0022:return True
 0023:
Exception: TypeError: stat: can't specify None for path argument

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/lib/oe/package_manager.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 287ca9a..980649f 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -531,6 +531,8 @@ def create_packages_dir(d, rpm_repo_dir, deploydir, 
taskname, filterbydependenci
 for dep in rpmdeps:
 c = taskdepdata[dep][0]
 manifest, d2 = oe.sstatesig.find_sstate_manifest(c, 
taskdepdata[dep][2], taskname, d, multilibs)
+if not manifest:
+bb.fatal("No manifest generated from: %s in %s" % (c, 
taskdepdata[dep][2]))
 if not os.path.exists(manifest):
 continue
 with open(manifest, "r") as f:
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] image.bbclass: Use deltask instead of noexec for package_write_rpm

[Jason Wessel]

On 03/05/2018 07:52 AM, Jason Wessel wrote:

On 03/03/2018 09:50 AM, Richard Purdie wrote:

On Sat, 2018-03-03 at 12:46 -0300, Otavio Salvador wrote:

On Fri, Mar 2, 2018 at 5:27 PM, Jason Wessel <jason.wessel@windriver.
com> wrote:


If you install an image into another image (this is the case for
custom initrd for example), it will fail with a non obvious python
backtrace.  This patch modifies the package_manager.py print which
package causes the failure for the future, if it is missing a
manifest.

The error you get looks like this:

oe-core/meta/lib/oe/package_manager.py', lineno: 534, function:
create_packages_dir
   0530:
   0531:for dep in rpmdeps:
   0532:c = taskdepdata[dep][0]
   0533:manifest, d2 =
oe.sstatesig.find_sstate_manifest(c, taskdepdata[dep][2], taskname,
d, multilibs)
   *** 0534:if not os.path.exists(manifest):
   0535:continue
   0536:with open(manifest, "r") as f:
   0537:for l in f:
   0538:l = l.strip()
File: '/usr/lib/python3.5/genericpath.py', lineno: 19, function:
exists
   0015:# This is false for dangling symbolic links on systems
that support them.
   0016:def exists(path):
   0017:"""Test whether a path exists.  Returns False for
broken symbolic links"""
   0018:try:
   *** 0019:os.stat(path)
   0020:except OSError:
   0021:return False
   0022:return True
   0023:
Exception: TypeError: stat: can't specify None for path argument

The root cause of the problem was determined to be the noexec
clause
in the image.bbclass.  When the package_write_rpm task runs it
expects
that a manifest will be generated.  Not running the task eliminates
the problem.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>

What about the other image types?


FWIW I'm testing:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=master-next=d77755d02f23ba410a2c55596c4538b8c5c2f63a

(but haven't posted it yet, I will shortly, just wanted to check it was
roughly going to work first).

I'd propose taking my patch above along with Jason's patch rewritten to
add the bb.fatal().




I agree.

Reviewed-by: Jason Wessel <jason.wes...@windriver.com>

Attached is the revised bbfatal patch.



The attachment didn't have the old patch deleted so don't use that.  I sent it 
to the list separately.

Cheers,
Jason.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] image.bbclass: Use deltask instead of noexec for package_write_rpm

[Jason Wessel]

On 03/03/2018 09:50 AM, Richard Purdie wrote:

On Sat, 2018-03-03 at 12:46 -0300, Otavio Salvador wrote:

On Fri, Mar 2, 2018 at 5:27 PM, Jason Wessel <jason.wessel@windriver.
com> wrote:


If you install an image into another image (this is the case for
custom initrd for example), it will fail with a non obvious python
backtrace.  This patch modifies the package_manager.py print which
package causes the failure for the future, if it is missing a
manifest.

The error you get looks like this:

oe-core/meta/lib/oe/package_manager.py', lineno: 534, function:
create_packages_dir
  0530:
  0531:for dep in rpmdeps:
  0532:c = taskdepdata[dep][0]
  0533:manifest, d2 =
oe.sstatesig.find_sstate_manifest(c, taskdepdata[dep][2], taskname,
d, multilibs)
  *** 0534:if not os.path.exists(manifest):
  0535:continue
  0536:with open(manifest, "r") as f:
  0537:for l in f:
  0538:l = l.strip()
File: '/usr/lib/python3.5/genericpath.py', lineno: 19, function:
exists
  0015:# This is false for dangling symbolic links on systems
that support them.
  0016:def exists(path):
  0017:"""Test whether a path exists.  Returns False for
broken symbolic links"""
  0018:try:
  *** 0019:os.stat(path)
  0020:except OSError:
  0021:return False
  0022:return True
  0023:
Exception: TypeError: stat: can't specify None for path argument

The root cause of the problem was determined to be the noexec
clause
in the image.bbclass.  When the package_write_rpm task runs it
expects
that a manifest will be generated.  Not running the task eliminates
the problem.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>

What about the other image types?


FWIW I'm testing:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=master-next=d77755d02f23ba410a2c55596c4538b8c5c2f63a

(but haven't posted it yet, I will shortly, just wanted to check it was
roughly going to work first).

I'd propose taking my patch above along with Jason's patch rewritten to
add the bb.fatal().




I agree.

Reviewed-by: Jason Wessel <jason.wes...@windriver.com>

Attached is the revised bbfatal patch.

Jason.
>From 71201901b917d31c694cde00640c461edb10b4e7 Mon Sep 17 00:00:00 2001
From: Jason Wessel <jason.wes...@windriver.com>
Date: Fri, 2 Mar 2018 13:49:16 -0600
Subject: [PATCH] package_manager.py: Print offending package instead of
 non-sense trace

If you have a package that does not generate a manifest due to using a
noexec rule, the package name should be printed so the problem can be
tracked down.  With out the patch you get an error that makes it look
more like the package_manager is broken as shown below.

oe-core/meta/lib/oe/package_manager.py', lineno: 534, function: create_packages_dir
 0530:
 0531:for dep in rpmdeps:
 0532:c = taskdepdata[dep][0]
 0533:manifest, d2 = oe.sstatesig.find_sstate_manifest(c, taskdepdata[dep][2], taskname, d, multilibs)
 *** 0534:if not os.path.exists(manifest):
 0535:continue
 0536:with open(manifest, "r") as f:
 0537:for l in f:
 0538:l = l.strip()
File: '/usr/lib/python3.5/genericpath.py', lineno: 19, function: exists
 0015:# This is false for dangling symbolic links on systems that support them.
 0016:def exists(path):
 0017:"""Test whether a path exists.  Returns False for broken symbolic links"""
 0018:try:
 *** 0019:os.stat(path)
 0020:except OSError:
 0021:return False
 0022:    return True
 0023:
Exception: TypeError: stat: can't specify None for path argument

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/classes/image.bbclass | 2 +-
 meta/lib/oe/package_manager.py | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 23ed53d..ad44f11 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -629,7 +629,7 @@ deltask do_package_qa
 do_packagedata[noexec] = "1"
 do_package_write_ipk[noexec] = "1"
 do_package_write_deb[noexec] = "1"
-do_package_write_rpm[noexec] = "1"
+deltask do_package_write_rpm
 
 # Prepare the root links to point to the /usr counterparts.
 create_merged_usr_symlinks() {
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 287ca9a..980649f 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -531,6 +531,8 @@ def create_packages_dir(d, rpm_repo_dir, deploydir, taskname, filterbydependenci
 for dep in rpmdeps:
 c = taskdepdata[dep][0]
 manifest, d2 = oe.sstatesig.find_sstate_manifest(c, taskdepdata[dep][2], taskname, d, mul

[OE-core] [PATCH] image.bbclass: Use deltask instead of noexec for package_write_rpm

[Jason Wessel]

If you install an image into another image (this is the case for
custom initrd for example), it will fail with a non obvious python
backtrace.  This patch modifies the package_manager.py print which
package causes the failure for the future, if it is missing a
manifest.

The error you get looks like this:

oe-core/meta/lib/oe/package_manager.py', lineno: 534, function: 
create_packages_dir
 0530:
 0531:for dep in rpmdeps:
 0532:c = taskdepdata[dep][0]
 0533:manifest, d2 = oe.sstatesig.find_sstate_manifest(c, 
taskdepdata[dep][2], taskname, d, multilibs)
 *** 0534:if not os.path.exists(manifest):
 0535:continue
 0536:with open(manifest, "r") as f:
 0537:for l in f:
 0538:l = l.strip()
File: '/usr/lib/python3.5/genericpath.py', lineno: 19, function: exists
 0015:# This is false for dangling symbolic links on systems that support 
them.
 0016:def exists(path):
 0017:"""Test whether a path exists.  Returns False for broken symbolic 
links"""
 0018:try:
 *** 0019:os.stat(path)
 0020:except OSError:
 0021:return False
 0022:return True
 0023:
Exception: TypeError: stat: can't specify None for path argument

The root cause of the problem was determined to be the noexec clause
in the image.bbclass.  When the package_write_rpm task runs it expects
that a manifest will be generated.  Not running the task eliminates
the problem.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/classes/image.bbclass | 2 +-
 meta/lib/oe/package_manager.py | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 23ed53d..ad44f11 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -629,7 +629,7 @@ deltask do_package_qa
 do_packagedata[noexec] = "1"
 do_package_write_ipk[noexec] = "1"
 do_package_write_deb[noexec] = "1"
-do_package_write_rpm[noexec] = "1"
+deltask do_package_write_rpm
 
 # Prepare the root links to point to the /usr counterparts.
 create_merged_usr_symlinks() {
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 287ca9a..980649f 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -531,6 +531,8 @@ def create_packages_dir(d, rpm_repo_dir, deploydir, 
taskname, filterbydependenci
 for dep in rpmdeps:
 c = taskdepdata[dep][0]
 manifest, d2 = oe.sstatesig.find_sstate_manifest(c, 
taskdepdata[dep][2], taskname, d, multilibs)
+if not manifest:
+bb.fatal("No manifest generated from: %s in %s" % (c, 
taskdepdata[dep][2]))
 if not os.path.exists(manifest):
 continue
 with open(manifest, "r") as f:
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] logging.bbclass: Enclose the tr string in quotes

[Jason Wessel]

On some linux hosts image recipes will fail to build as follows:

ERROR: build-essential-0.3-r0 do_image_ext3: Usage: bbdebug [123] "message"
ERROR: build-essential-0.3-r0 do_image_ext3: Function failed: do_image_ext3 
(log file is located at 
/opt/build/tmp/work/intel_corei7_64-wrs-linux/build-essential/0.3-r0/temp/log.do_image_ext3.43744)
ERROR: Logfile of failure stored in: 
/opt/build/tmp/work/intel_corei7_64-wrs-linux/build-essential/0.3-r0/temp/log.do_image_ext3.43744
ERROR: Task 
(/opt/layers/meta-overc/meta-build/recipes-core/images/build-essential_0.3.bb:do_image_ext3)
 failed with exit code '1'

Running with bitbake -v -v -v -D we get in the log file:
+ bbdebug 1 Executing 'dd if=/dev/zero 
of=/opt/build/tmp/work/intel_corei7_64-wrs-linux/build-essential/0.3-r0/deploy-build-essential-image-complete/build-essential-intel-corei7-64-20180220190510.rootfs.ext3
 seek=484486 count=0 bs=1024'
+ USAGE='Usage: bbdebug [123] "message"'
+ '[' 3 -lt 2 ']'
+ DBGLVL=1
+ shift
++ echo 1

++ echo 1
++ tr -d t

++ tr -d t
+ NONDIGITS=1
+ '[' 1 ']'
+ bbfatal 'Usage: bbdebug [123] "message"'

The debug output tells us that the NONDIGITS check failed to remove
the digits using the tr expression.  Enclosing the expression in
quotes causes it to work properly.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/classes/logging.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/logging.bbclass b/meta/classes/logging.bbclass
index 06c7c31c3e..a0c94e98c7 100644
--- a/meta/classes/logging.bbclass
+++ b/meta/classes/logging.bbclass
@@ -86,7 +86,7 @@ bbdebug() {

# Strip off the debug level and ensure it is an integer
DBGLVL=$1; shift
-   NONDIGITS=$(echo "$DBGLVL" | tr -d [:digit:])
+   NONDIGITS=$(echo "$DBGLVL" | tr -d "[:digit:]")
if [ "$NONDIGITS" ]; then
bbfatal "$USAGE"
fi
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] libgfortran: Work around libbacktrace build failures

[Jason Wessel]

While it is not clear how to depend on the libbacktrace in such a way
that it will be in the sysroot for a build, it is possible to generate
the library and header on the fly which will have the same c linkage
to the shared object as the one provided internally with gcc by asking
it to build at the time libgfortran is building.

The build of libgfortran fails as follows when fortran is enabled in
the local.conf with:

   FORTRAN_forcevariable = ",fortran"

Error text:

  libtool: compile: x86_64-wrs-linux-gcc -m64 -march=core2 -mtune=core2
  -msse3 -mfpmath=sse
  --sysroot=/opt/build/libgfortran/7.2.0-r0/recipe-sysroot
  -DHAVE_CONFIG_H
  -I. -I../../../../../../../../work-shared/gcc-7.2.0-r0/gcc-7.2.0/libgfortran
  
-iquote../../../../../../../../work-shared/gcc-7.2.0-r0/gcc-7.2.0/libgfortran/io
  
-I../../../../../../../../work-shared/gcc-7.2.0-r0/gcc-7.2.0/libgfortran/../gcc
  
-I../../../../../../../../work-shared/gcc-7.2.0-r0/gcc-7.2.0/libgfortran/../gcc/config
  
-I/opt/build/libgfortran/7.2.0-r0/gcc-7.2.0/build.x86_64-wrs-linux.x86_64-wrs-linux/x86_64-wrs-linux/libgfortran/../.././gcc
  
-I../../../../../../../../work-shared/gcc-7.2.0-r0/gcc-7.2.0/libgfortran/../libgcc
  
-I/opt/build/libgfortran/7.2.0-r0/gcc-7.2.0/build.x86_64-wrs-linux.x86_64-wrs-linux/x86_64-wrs-linux/libgfortran/../libgcc
  
-I../../../../../../../../work-shared/gcc-7.2.0-r0/gcc-7.2.0/libgfortran/../libbacktrace
  
-I/opt/build/libgfortran/7.2.0-r0/gcc-7.2.0/build.x86_64-wrs-linux.x86_64-wrs-linux/x86_64-wrs-linux/libgfortran/../libbacktrace
  -I../libbacktrace -std=gnu11 -Wall -Wstrict-prototypes
  -Wmissing-prototypes -Wold-style-definition -Wextra -Wwrite-strings
  -Werror=implicit-function-declaration -Werror=vla -fcx-fortran-rules
  -ffunction-sections -fdata-sections -std=gnu11 -O2 -pipe -g
  -Wunknown-pragmas -c
  
../../../../../../../../work-shared/gcc-7.2.0-r0/gcc-7.2.0/libgfortran/runtime/backtrace.c
  -fPIC -DPIC -o .libs/backtrace.o
  
../../../../../../../../work-shared/gcc-7.2.0-r0/gcc-7.2.0/libgfortran/runtime/backtrace.c:36:10:
  fatal error: backtrace-supported.h: No such file or directory #include
  "backtrace-supported.h"

This problem was also reported here over a year ago:
https://lists.yoctoproject.org/pipermail/yocto/2016-November/033228.html

Using this patch I was able to build fortan and use it to build a
working version of Atlas on the target system.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/recipes-devtools/gcc/libgfortran.inc | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/meta/recipes-devtools/gcc/libgfortran.inc 
b/meta/recipes-devtools/gcc/libgfortran.inc
index 5f5d4af758..2f806e12e0 100644
--- a/meta/recipes-devtools/gcc/libgfortran.inc
+++ b/meta/recipes-devtools/gcc/libgfortran.inc
@@ -15,6 +15,13 @@ do_configure () {
# Easiest way to stop bad RPATHs getting into the library since we have 
a
# broken libtool here
sed -i -e 's/hardcode_into_libs=yes/hardcode_into_libs=no/' 
${B}/${TARGET_SYS}/libgfortran/libtool
+   # Setup libbacktrace and build it because it is needed for the compile 
phase
+   # This is a work around for the proper libbacktrace integration for 
fortran
+   mkdir -p ${B}/${TARGET_SYS}/libbacktrace
+   cd ${B}/${TARGET_SYS}/libbacktrace
+   btrelpath=${@os.path.relpath("${S}/libbacktrace", 
"${B}/${TARGET_SYS}/libbacktrace")}
+   $btrelpath/configure ${CONFIGUREOPTS} ${EXTRA_OECONF}
+   oe_runmake MULTIBUILDTOP=${B}/${TARGET_SYS}/libbacktrace/
 }
 EXTRACONFFUNCS += "extract_stashed_builddir"
 do_configure[depends] += "${COMPILERDEP}"
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] ovmf: Fix build failure for PARALLEL_MAKE with no space

[Jason Wessel]

The ovmf package fails to build if you use a PARALLEL_MAKE flag
without a space in it.  If you put the following in local.conf for
example:

 PARALLEL_MAKE = "-j4"

The build will fail with as follows:

| build.exe: error: Option -n only allows one instance in command line!
| WARNING: /opt/build/ovmf/git-r0/temp/run.do_compile.26080:1 exit 2 from 
'/opt/build/ovmf/git-r0/git/OvmfPkg/build.sh $PARALLEL_JOBS -a $OVMF_ARCH -b 
RELEASE -t ${FIXED_GCCVER}'
| ERROR: Function failed: do_compile (log file is located at 
/opt/build/ovmf/git-r0/temp/log.do_compile.26080)

The failure is because the build.sh expects a space.  The simple work
around is to fix the recipe to add some white space.  The build.sh
only cares that it gets at least one space between the arguments, the
extra space if someone uses "-j 4" does not cause problems.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/recipes-core/ovmf/ovmf_git.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/ovmf/ovmf_git.bb 
b/meta/recipes-core/ovmf/ovmf_git.bb
index a98826210e..fa0d66291d 100644
--- a/meta/recipes-core/ovmf/ovmf_git.bb
+++ b/meta/recipes-core/ovmf/ovmf_git.bb
@@ -151,7 +151,7 @@ do_compile_class-native() {
 
 do_compile_class-target() {
 export LFLAGS="${LDFLAGS}"
-PARALLEL_JOBS="${@ '${PARALLEL_MAKE}'.replace('-j', '-n')}"
+PARALLEL_JOBS="${@ '${PARALLEL_MAKE}'.replace('-j', '-n ')}"
 OVMF_ARCH="X64"
 if [ "${TARGET_ARCH}" != "x86_64" ] ; then
 OVMF_ARCH="IA32"
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [ PATCH 0/1 ] ovmf: Fix build failure for PARALLEL_MAKE with no space

[Jason Wessel]

Please consider for master and the rocko branch.  It is a simple build
fix reported by someone using PARALLEL_MAKE = "-j4" instead of PARALLEL_MAKE = 
"-j 4".

It is the only package that we have hit so far that exhibits this kind of 
failure
so it seems it would be best to just fix it.  :-)

Thanks,
Jason.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] systemd: Fix rootfs transaction error when PACKAGECONFIG has polkit

[Jason Wessel]

It should work because it has nothing to do with that particular passwd file.   
This is builds successfully on a ubuntu 16.04 host which has no polkitd user.

Also with the user add stanza in the patch it should be adding the user that is 
required when PACKAGECONFIG has the polkitd added to the psuedo passwd file.   
We have quite a few configurations building properly.  Where I have seen 
transient failures before is when the -native sysroot does not get rebuilt but 
that is typically with other recipes.

Is this still a problem?


Jason.


On 09/06/2017 01:35 PM, Martin Jansa wrote:

Does this work on hosts without polkitd user?

Here it failed with:

| chown: invalid user: ‘polkitd:root’






On Tue, Aug 15, 2017 at 8:55 PM, Jason Wessel <jason.wes...@windriver.com 
<mailto:jason.wes...@windriver.com>> wrote:

The systemd 234 added some files to the polkit directory and the
directory the files live in must be owned by the polkitd user, else
you will receive the following error when the rootfs is being
assembled:

Error: Transaction check error:
  file /usr/share/polkit-1/rules.d conflicts between attempted installs of 
polkit-0.113-r0.15.core2_64 and systemd-1:234-r0.0.core2_64

The fix similar to other packages such as libvirt where the user must
exist and the directory must be created with the proper attributes.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com 
<mailto:jason.wes...@windriver.com>>
---
 meta/recipes-core/systemd/systemd_234.bb <http://systemd_234.bb> | 9 
+
 1 file changed, 9 insertions(+)

diff --git a/meta/recipes-core/systemd/systemd_234.bb <http://systemd_234.bb> 
b/meta/recipes-core/systemd/systemd_234.bb <http://systemd_234.bb>
index ad7fc99b90..4560cf4175 100644
--- a/meta/recipes-core/systemd/systemd_234.bb <http://systemd_234.bb>
+++ b/meta/recipes-core/systemd/systemd_234.bb <http://systemd_234.bb>
@@ -245,6 +245,14 @@ do_install() {
ln -s ../run/systemd/resolve/resolv.conf 
${D}${sysconfdir}/resolv-conf.systemd
fi
install -Dm 0755 ${S}/src/systemctl/systemd-sysv-install.SKELETON 
${D}${systemd_unitdir}/systemd-sysv-install
+
+   # If polkit is setup fixup permissions and ownership
+   if [ "${@bb.utils.contains('PACKAGECONFIG', 'polkit', 'polkit', '', d)}" = 
"polkit" ] ; then
+   if [ -d ${D}${datadir}/polkit-1/rules.d ] ; then
+   chmod 700 ${D}${datadir}/polkit-1/rules.d
+   chown polkitd:root ${D}${datadir}/polkit-1/rules.d
+   fi
+   fi
 }

 do_install_ptest () {
@@ -308,6 +316,7 @@ USERADD_PARAM_${PN} += 
"${@bb.utils.contains('PACKAGECONFIG', 'timesyncd', '--sy
 USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'networkd', 
'--system -d / -M --shell /bin/nologin systemd-network;', '', d)}"
 USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'coredump', 
'--system -d / -M --shell /bin/nologin systemd-coredump;', '', d)}"
 USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'resolved', 
'--system -d / -M --shell /bin/nologin systemd-resolve;', '', d)}"
+USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'polkit', 
'--system --no-create-home --user-group --home-dir ${sysconfdir}/polkit-1 polkitd;', '', 
d)}"
 GROUPADD_PARAM_${PN} = "-r lock; -r systemd-journal"
 USERADD_PARAM_${PN}-extra-utils += "--system -d / -M --shell /bin/nologin 
systemd-bus-proxy;"

--
2.11.0

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org 
<mailto:Openembedded-core@lists.openembedded.org>
http://lists.openembedded.org/mailman/listinfo/openembedded-core 
<http://lists.openembedded.org/mailman/listinfo/openembedded-core>




-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] systemd: Fix rootfs transaction error when PACKAGECONFIG has polkit

[Jason Wessel]

The systemd 234 added some files to the polkit directory and the
directory the files live in must be owned by the polkitd user, else
you will receive the following error when the rootfs is being
assembled:

Error: Transaction check error:
  file /usr/share/polkit-1/rules.d conflicts between attempted installs of 
polkit-0.113-r0.15.core2_64 and systemd-1:234-r0.0.core2_64

The fix similar to other packages such as libvirt where the user must
exist and the directory must be created with the proper attributes.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/recipes-core/systemd/systemd_234.bb | 9 +
 1 file changed, 9 insertions(+)

diff --git a/meta/recipes-core/systemd/systemd_234.bb 
b/meta/recipes-core/systemd/systemd_234.bb
index ad7fc99b90..4560cf4175 100644
--- a/meta/recipes-core/systemd/systemd_234.bb
+++ b/meta/recipes-core/systemd/systemd_234.bb
@@ -245,6 +245,14 @@ do_install() {
ln -s ../run/systemd/resolve/resolv.conf 
${D}${sysconfdir}/resolv-conf.systemd
fi
install -Dm 0755 ${S}/src/systemctl/systemd-sysv-install.SKELETON 
${D}${systemd_unitdir}/systemd-sysv-install
+
+   # If polkit is setup fixup permissions and ownership
+   if [ "${@bb.utils.contains('PACKAGECONFIG', 'polkit', 'polkit', '', 
d)}" = "polkit" ] ; then
+   if [ -d ${D}${datadir}/polkit-1/rules.d ] ; then
+   chmod 700 ${D}${datadir}/polkit-1/rules.d
+   chown polkitd:root ${D}${datadir}/polkit-1/rules.d
+   fi
+   fi
 }
 
 do_install_ptest () {
@@ -308,6 +316,7 @@ USERADD_PARAM_${PN} += 
"${@bb.utils.contains('PACKAGECONFIG', 'timesyncd', '--sy
 USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'networkd', 
'--system -d / -M --shell /bin/nologin systemd-network;', '', d)}"
 USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'coredump', 
'--system -d / -M --shell /bin/nologin systemd-coredump;', '', d)}"
 USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'resolved', 
'--system -d / -M --shell /bin/nologin systemd-resolve;', '', d)}"
+USERADD_PARAM_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'polkit', 
'--system --no-create-home --user-group --home-dir ${sysconfdir}/polkit-1 
polkitd;', '', d)}"
 GROUPADD_PARAM_${PN} = "-r lock; -r systemd-journal"
 USERADD_PARAM_${PN}-extra-utils += "--system -d / -M --shell /bin/nologin 
systemd-bus-proxy;"
 
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] openssl10: Fix conflict between openssl and openssl10 man pages.

[Jason Wessel]

The package resolver failes to assemble images because some of the man
pages in openssl10 conflict with the openssl package.  In the case
where you want openssl, openssh and the documentation installed in the
same system you will see the failure.

The work around is to rename all the openssl10 man pages and symlinks
to have a prefix of openssl10-.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/recipes-connectivity/openssl/openssl10.inc | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/meta/recipes-connectivity/openssl/openssl10.inc 
b/meta/recipes-connectivity/openssl/openssl10.inc
index c93d5d8738..54cf4598e1 100644
--- a/meta/recipes-connectivity/openssl/openssl10.inc
+++ b/meta/recipes-connectivity/openssl/openssl10.inc
@@ -199,6 +199,16 @@ do_install () {
ln -sf ${sysconfdir}/ssl/certs ${D}${libdir}/ssl/certs
ln -sf ${sysconfdir}/ssl/private ${D}${libdir}/ssl/private
ln -sf ${sysconfdir}/ssl/openssl.cnf ${D}${libdir}/ssl/openssl.cnf
+
+   # Rename man pages to prefix openssl10-*
+   for f in `find ${D}${mandir} -type f`; do
+   mv $f $(dirname $f)/openssl10-$(basename $f)
+   done
+   for f in `find ${D}${mandir} -type l`; do
+   ln_f=`readlink $f`
+   rm -f $f
+   ln -s openssl10-$ln_f $(dirname $f)/openssl10-$(basename $f)
+   done
 }
 
 do_install_ptest () {
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] Revert "kernel: Modify kernel modules installation path."

[Jason Wessel]

On 01/20/2017 07:52 AM, Burton, Ross wrote:


On 20 January 2017 at 13:48, Jason Wessel <jason.wes...@windriver.com 
<mailto:jason.wes...@windriver.com>> wrote:

WARNING: linux-yocto-4.8.12+gitAUTOINC+3edb4de355_9bcb4ea3fa-r0 do_package: 
QA Issue: linux-yocto: Files/directories were installed but not shipped in any 
package:
  /lib64
  /lib/modules/4.8.17-yocto-standard/modules.builtin
  /lib64/firmware
  /lib64/firmware/cpia2
  /lib64/firmware/cpia2/stv0672_vp4.bin
Please set FILES such that these items are packaged. Alternatively if they 
are unneeded, avoid installing them or delete them within do_install.
linux-yocto: 5 installed and not shipped files. [installed-vs-shipped]

I suspect this is trivial with fresh eyes.  What multilib configuration are you using?  
My usual "test multilib" stanza is:

#require conf/multilib.conf
#MULTILIBS_x86-64 = "multilib:lib32"
#DEFAULTTUNE_virtclass-multilib-lib32 = "x86"

I'm guessing this isn't going to produce the same paths that you're seeing 
problems with.

Ross



It looks like it probably should.

MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"


If for some reason you wanted to perform the exact same build I have the 
instructions to do so:

https://github.com/WindRiver-OpenSourceLabs/wr-core/blob/master/docs/README-intel-x86.TXT

That basically amounts to:

git clone -b master --recurse-submodules 
https://github.com/WindRiver-OpenSourceLabs/wr-core

cd wr-core

. init-intel-x86-env


Jason.

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] Revert "kernel: Modify kernel modules installation path."

[Jason Wessel]

On 01/20/2017 07:30 AM, Burton, Ross wrote:


On 20 January 2017 at 13:29, Maxin B. John > wrote:

> The issue here is that in a multilib build the ${base_libdir} is 
resolving to /lib64 which completely breaks some tools that fully expect 
/lib/modules to be a fixed location regardless if the target is 32/64 or something 
else.
>

I would expect the kernel to be ok with this change if we provide the 
kernel command
line parameter: "firmware_class.path=${base_libdir}" or similar.




You are correct that this could work, but that is not a parameter I would 
expect to set in the default case.  It would only be something I used to 
execute a system rollback where I need firmware to get the device running again.



I believe that using nonarch_base_libdir is the proper fix here, we need to 
understand why this didn't just work.

Ross



When I had tried the search and replace I still ended up with the following 
problem, which is why I elected to just revert to what worked previously.


WARNING: linux-yocto-4.8.12+gitAUTOINC+3edb4de355_9bcb4ea3fa-r0 do_package: QA 
Issue: linux-yocto: Files/directories were installed but not shipped in any 
package:
  /lib64
  /lib/modules/4.8.17-yocto-standard/modules.builtin
  /lib64/firmware
  /lib64/firmware/cpia2
  /lib64/firmware/cpia2/stv0672_vp4.bin
Please set FILES such that these items are packaged. Alternatively if they are 
unneeded, avoid installing them or delete them within do_install.
linux-yocto: 5 installed and not shipped files. [installed-vs-shipped]

I am sure this problem can be corrected and debugged.


Cheers,

Jason.

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] Revert "kernel: Modify kernel modules installation path."

[Jason Wessel]

The issue here is that in a multilib build the ${base_libdir} is resolving to 
/lib64 which completely breaks some tools that fully expect /lib/modules to be 
a fixed location regardless if the target is 32/64 or something else.

It may be fixable by using, ${nonarch_base_libdir}, but it wasn't quite that 
simple because it didn't fully work for the firmware loader so for now it would 
be best to just revert the two patches and keep oe-core working.

Cheers,
Jason.


On 01/19/2017 03:14 PM, Jason Wessel wrote:

This reverts commit 0ccb2efe9837e2915c093341a662dffc1df00866.

The OVS fails to function and the kernel modules cannot be found by
any of the kernel tools such as depmod because they are installed
into the wrong directory in multilib 64bit/32bit bulids.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
  meta/classes/kernel-module-split.bbclass |  2 +-
  meta/classes/kernel.bbclass  | 12 ++--
  2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index 0e4e110f0c..efe1b42656 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -127,7 +127,7 @@ python split_kernel_module_packages () {
  postinst = d.getVar('pkg_postinst_modules')
  postrm = d.getVar('pkg_postrm_modules')
  
-modules = do_split_packages(d, root='${base_libdir}/modules', file_regex=module_regex, output_pattern=module_pattern, description='%s kernel module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, extra_depends='kernel-%s' % (d.getVar("KERNEL_VERSION")))

+modules = do_split_packages(d, root='/lib/modules', file_regex=module_regex, 
output_pattern=module_pattern, description='%s kernel module', postinst=postinst, 
postrm=postrm, recursive=True, hook=frob_metadata, extra_depends='kernel-%s' % 
(d.getVar("KERNEL_VERSION")))
  if modules:
  metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE')
  d.appendVar('RDEPENDS_' + metapkg, ' '+' '.join(modules))
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 74517e8d92..3630042dbd 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -296,11 +296,11 @@ kernel_do_install() {
#
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
if (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
-   oe_runmake DEPMOD=echo 
MODLIB=${D}${base_libdir}/modules/${KERNEL_VERSION} 
INSTALL_FW_PATH=${D}${base_libdir}/firmware modules_install
-   rm "${D}${base_libdir}/modules/${KERNEL_VERSION}/build"
-   rm "${D}${base_libdir}/modules/${KERNEL_VERSION}/source"
+   oe_runmake DEPMOD=echo INSTALL_MOD_PATH="${D}" modules_install
+   rm "${D}/lib/modules/${KERNEL_VERSION}/build"
+   rm "${D}/lib/modules/${KERNEL_VERSION}/source"
# If the kernel/ directory is empty remove it to prevent QA 
issues
-   rmdir --ignore-fail-on-non-empty 
"${D}${base_libdir}/modules/${KERNEL_VERSION}/kernel"
+   rmdir --ignore-fail-on-non-empty 
"${D}/lib/modules/${KERNEL_VERSION}/kernel"
else
bbnote "no modules to install"
fi
@@ -478,7 +478,7 @@ EXPORT_FUNCTIONS do_compile do_install do_configure
  # kernel-image becomes kernel-image-${KERNEL_VERSION}
  PACKAGES = "kernel kernel-base kernel-vmlinux kernel-image kernel-dev 
kernel-modules"
  FILES_${PN} = ""
-FILES_kernel-base = "${base_libdir}/modules/${KERNEL_VERSION}/modules.order 
${base_libdir}/modules/${KERNEL_VERSION}/modules.builtin"
+FILES_kernel-base = "/lib/modules/${KERNEL_VERSION}/modules.order 
/lib/modules/${KERNEL_VERSION}/modules.builtin"
  FILES_kernel-image = ""
  FILES_kernel-dev = "/boot/System.map* /boot/Module.symvers* /boot/config* 
${KERNEL_SRC_PATH} /lib/modules/${KERNEL_VERSION}/build"
  FILES_kernel-vmlinux = "/boot/vmlinux-${KERNEL_VERSION_NAME}"
@@ -511,7 +511,7 @@ pkg_postinst_kernel-base () {
  PACKAGESPLITFUNCS_prepend = "split_kernel_packages "
  
  python split_kernel_packages () {

-do_split_packages(d, root='${base_libdir}/firmware', 
file_regex='^(.*)\.(bin|fw|cis|csp|dsp)$', output_pattern='kernel-firmware-%s', 
description='Firmware for %s', recursive=True, extra_depends='')
+do_split_packages(d, root='/lib/firmware', 
file_regex='^(.*)\.(bin|fw|cis|csp|dsp)$', output_pattern='kernel-firmware-%s', 
description='Firmware for %s', recursive=True, extra_depends='')
  }
  
  # Many scripts want to look in arch/$arch/boot for the bootable



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] Revert "kernel: Modify kernel modules installation path."

[Jason Wessel]

This reverts commit 0ccb2efe9837e2915c093341a662dffc1df00866.

The OVS fails to function and the kernel modules cannot be found by
any of the kernel tools such as depmod because they are installed
into the wrong directory in multilib 64bit/32bit bulids.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 meta/classes/kernel-module-split.bbclass |  2 +-
 meta/classes/kernel.bbclass  | 12 ++--
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index 0e4e110f0c..efe1b42656 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -127,7 +127,7 @@ python split_kernel_module_packages () {
 postinst = d.getVar('pkg_postinst_modules')
 postrm = d.getVar('pkg_postrm_modules')
 
-modules = do_split_packages(d, root='${base_libdir}/modules', 
file_regex=module_regex, output_pattern=module_pattern, description='%s kernel 
module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, 
extra_depends='kernel-%s' % (d.getVar("KERNEL_VERSION")))
+modules = do_split_packages(d, root='/lib/modules', 
file_regex=module_regex, output_pattern=module_pattern, description='%s kernel 
module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, 
extra_depends='kernel-%s' % (d.getVar("KERNEL_VERSION")))
 if modules:
 metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE')
 d.appendVar('RDEPENDS_' + metapkg, ' '+' '.join(modules))
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 74517e8d92..3630042dbd 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -296,11 +296,11 @@ kernel_do_install() {
#
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
if (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
-   oe_runmake DEPMOD=echo 
MODLIB=${D}${base_libdir}/modules/${KERNEL_VERSION} 
INSTALL_FW_PATH=${D}${base_libdir}/firmware modules_install
-   rm "${D}${base_libdir}/modules/${KERNEL_VERSION}/build"
-   rm "${D}${base_libdir}/modules/${KERNEL_VERSION}/source"
+   oe_runmake DEPMOD=echo INSTALL_MOD_PATH="${D}" modules_install
+   rm "${D}/lib/modules/${KERNEL_VERSION}/build"
+   rm "${D}/lib/modules/${KERNEL_VERSION}/source"
# If the kernel/ directory is empty remove it to prevent QA 
issues
-   rmdir --ignore-fail-on-non-empty 
"${D}${base_libdir}/modules/${KERNEL_VERSION}/kernel"
+   rmdir --ignore-fail-on-non-empty 
"${D}/lib/modules/${KERNEL_VERSION}/kernel"
else
bbnote "no modules to install"
fi
@@ -478,7 +478,7 @@ EXPORT_FUNCTIONS do_compile do_install do_configure
 # kernel-image becomes kernel-image-${KERNEL_VERSION}
 PACKAGES = "kernel kernel-base kernel-vmlinux kernel-image kernel-dev 
kernel-modules"
 FILES_${PN} = ""
-FILES_kernel-base = "${base_libdir}/modules/${KERNEL_VERSION}/modules.order 
${base_libdir}/modules/${KERNEL_VERSION}/modules.builtin"
+FILES_kernel-base = "/lib/modules/${KERNEL_VERSION}/modules.order 
/lib/modules/${KERNEL_VERSION}/modules.builtin"
 FILES_kernel-image = ""
 FILES_kernel-dev = "/boot/System.map* /boot/Module.symvers* /boot/config* 
${KERNEL_SRC_PATH} /lib/modules/${KERNEL_VERSION}/build"
 FILES_kernel-vmlinux = "/boot/vmlinux-${KERNEL_VERSION_NAME}"
@@ -511,7 +511,7 @@ pkg_postinst_kernel-base () {
 PACKAGESPLITFUNCS_prepend = "split_kernel_packages "
 
 python split_kernel_packages () {
-do_split_packages(d, root='${base_libdir}/firmware', 
file_regex='^(.*)\.(bin|fw|cis|csp|dsp)$', output_pattern='kernel-firmware-%s', 
description='Firmware for %s', recursive=True, extra_depends='')
+do_split_packages(d, root='/lib/firmware', 
file_regex='^(.*)\.(bin|fw|cis|csp|dsp)$', output_pattern='kernel-firmware-%s', 
description='Firmware for %s', recursive=True, extra_depends='')
 }
 
 # Many scripts want to look in arch/$arch/boot for the bootable
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] Revert "linux-firmware: Modify firmware installation path"

[Jason Wessel]

This reverts commit a46a23fed9acd097ecc8ab3985d18688abe7b065.

The 64bit/32bit multilib builds break because the kernel's
firmware loader cannot find the firmware because it
gets placed int the wrong directory.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 .../linux-firmware/linux-firmware_git.bb   | 168 ++---
 1 file changed, 82 insertions(+), 86 deletions(-)

diff --git a/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb 
b/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
index 5942997fda..4fedcfbc67 100644
--- a/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
+++ b/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
@@ -195,27 +195,27 @@ do_compile() {
 }
 
 do_install() {
-   install -d  ${D}${base_libdir}/firmware/
-   cp -r * ${D}${base_libdir}/firmware/
+   install -d  ${D}/lib/firmware/
+   cp -r * ${D}/lib/firmware/
 
# Avoid Makefile to be deployed
-   rm ${D}${base_libdir}/firmware/Makefile
+   rm ${D}/lib/firmware/Makefile
 
# Remove unbuild firmware which needs cmake and bash
-   rm ${D}${base_libdir}/firmware/carl9170fw -rf
+   rm ${D}/lib/firmware/carl9170fw -rf
 
# Remove pointless bash script
-   rm ${D}${base_libdir}/firmware/configure
+   rm ${D}/lib/firmware/configure
 
# Libertas sd8686
-   ln -sf libertas/sd8686_v9.bin ${D}${base_libdir}/firmware/sd8686.bin
-   ln -sf libertas/sd8686_v9_helper.bin 
${D}${base_libdir}/firmware/sd8686_helper.bin
+   ln -sf libertas/sd8686_v9.bin ${D}/lib/firmware/sd8686.bin
+   ln -sf libertas/sd8686_v9_helper.bin ${D}/lib/firmware/sd8686_helper.bin
 
# fixup wl12xx location, after 2.6.37 the kernel searches a different 
location for it
-   ( cd ${D}${base_libdir}/firmware ; ln -sf ti-connectivity/* . )
+   ( cd ${D}/lib/firmware ; ln -sf ti-connectivity/* . )
 
 # Copy the iwlwifi ucode
-cp ${WORKDIR}/iwlwifi-8000C-19.ucode ${D}${base_libdir}/firmware/
+cp ${WORKDIR}/iwlwifi-8000C-19.ucode ${D}/lib/firmware/
 }
 
 
@@ -252,21 +252,21 @@ LICENSE_${PN}-ath6k = "Firmware-atheros_firmware"
 LICENSE_${PN}-ath9k = "Firmware-atheros_firmware"
 LICENSE_${PN}-atheros-license = "Firmware-atheros_firmware"
 
-FILES_${PN}-atheros-license = 
"${base_libdir}/firmware/LICENCE.atheros_firmware"
+FILES_${PN}-atheros-license = "/lib/firmware/LICENCE.atheros_firmware"
 FILES_${PN}-ar9170 = " \
-  ${base_libdir}/firmware/ar9170*.fw \
+  /lib/firmware/ar9170*.fw \
 "
 FILES_${PN}-carl9170 = " \
-  ${base_libdir}/firmware/carl9170*.fw \
+  /lib/firmware/carl9170*.fw \
 "
 FILES_${PN}-ath6k = " \
-  ${base_libdir}/firmware/ath6k \
+  /lib/firmware/ath6k \
 "
 FILES_${PN}-ath9k = " \
-  ${base_libdir}/firmware/ar9271.fw \
-  ${base_libdir}/firmware/ar7010*.fw \
-  ${base_libdir}/firmware/htc_9271.fw \
-  ${base_libdir}/firmware/htc_7010.fw \
+  /lib/firmware/ar9271.fw \
+  /lib/firmware/ar7010*.fw \
+  /lib/firmware/htc_9271.fw \
+  /lib/firmware/htc_7010.fw \
 "
 
 RDEPENDS_${PN}-ar9170 += "${PN}-atheros-license"
@@ -280,14 +280,14 @@ LICENSE_${PN}-ar3k-license = "Firmware-qualcommAthos_ar3k"
 LICENSE_${PN}-ath10k = "Firmware-qualcommAthos_ath10k"
 LICENSE_${PN}-ath10k-license = "Firmware-qualcommAthos_ath10k"
 
-FILES_${PN}-ar3k-license = 
"${base_libdir}/firmware/LICENSE.QualcommAtheros_ar3k"
+FILES_${PN}-ar3k-license = "/lib/firmware/LICENSE.QualcommAtheros_ar3k"
 FILES_${PN}-ar3k = " \
-  ${base_libdir}/firmware/ar3k \
+  /lib/firmware/ar3k \
 "
 
-FILES_${PN}-ath10k-license = 
"${base_libdir}/firmware/LICENSE.QualcommAtheros_ath10k"
+FILES_${PN}-ath10k-license = "/lib/firmware/LICENSE.QualcommAtheros_ath10k"
 FILES_${PN}-ath10k = " \
-  ${base_libdir}/firmware/ath10k \
+  /lib/firmware/ath10k \
 "
 
 RDEPENDS_${PN}-ar3k += "${PN}-ar3k-license"
@@ -297,9 +297,9 @@ RDEPENDS_${PN}-ath10k += "${PN}-ath10k-license"
 LICENSE_${PN}-ralink = "Firmware-ralink-firmware"
 LICENSE_${PN}-ralink-license = "Firmware-ralink-firmware"
 
-FILES_${PN}-ralink-license = 
"${base_libdir}/firmware/LICENCE.ralink-firmware.txt"
+FILES_${PN}-ralink-license = "/lib/firmware/LICENCE.ralink-firmware.txt"
 FILES_${PN}-ralink = " \
-  ${base_libdir}/firmware/rt*.bin \
+  /lib/firmware/rt*.bin \
 "
 
 RDEPENDS_${PN}-ralink += "${PN}-ralink-license"
@@ -308,9 +308,9 @@ RDEPENDS_${PN}-ralink += "${PN}-ralink-license"
 LICENSE_${PN}-radeon = "Firmware-radeon"
 LICENSE_${PN}-radeon-license = "Firmware-radeon"
 
-FILES_${PN}-radeon-license = "${base_libdir}/firmware/LICENSE.radeon"
+FILES_${PN}-radeon-license = "/lib/firmware/LICENSE.radeon"
 FILES_${PN}-radeon = " \
-  ${base_libdir}

[OE-core] [PATCH] systemd: Backport cgroup fix from 233 to 232

[Jason Wessel]

There is a critical regression in the default behavior with systemd
232 which prevents lxc, docker, and opencontainers from working
properly out of the box.  The change was already committed to the
systemd 233 code stream.

The failure looks like what is shown below.

% lxc-start -n container -F
lxc-start: cgfsng.c: parse_hierarchies: 825 Failed to find current cgroup for 
controller 'name=systemd'
lxc-start: cgfsng.c: all_controllers_found: 431 no systemd controller 
mountpoint found
lxc-start: start.c: lxc_spawn: 1082 failed initializing cgroup support
lxc-start: start.c: __lxc_start: 1332 failed to spawn 'container'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by 
setting the --logfile and --logpriority options.

Commit 843d5baf6aad6c53fc00ea8d95d83209a4f92de1 from the systemd git
has been backported and can be dropped in a future uprev.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 ...use-the-unified-hierarchy-for-the-systemd.patch | 51 ++
 meta/recipes-core/systemd/systemd_232.bb   |  1 +
 2 files changed, 52 insertions(+)
 create mode 100644 
meta/recipes-core/systemd/systemd/0020-back-port-233-don-t-use-the-unified-hierarchy-for-the-systemd.patch

diff --git 
a/meta/recipes-core/systemd/systemd/0020-back-port-233-don-t-use-the-unified-hierarchy-for-the-systemd.patch
 
b/meta/recipes-core/systemd/systemd/0020-back-port-233-don-t-use-the-unified-hierarchy-for-the-systemd.patch
new file mode 100644
index 000..ef2d868
--- /dev/null
+++ 
b/meta/recipes-core/systemd/systemd/0020-back-port-233-don-t-use-the-unified-hierarchy-for-the-systemd.patch
@@ -0,0 +1,51 @@
+This is a direct backport from systemd-233's stream to fix lxc/docker.
+
+% lxc-start -n container -F
+lxc-start: cgfsng.c: parse_hierarchies: 825 Failed to find current cgroup for 
controller 'name=systemd'
+lxc-start: cgfsng.c: all_controllers_found: 431 no systemd controller 
mountpoint found
+lxc-start: start.c: lxc_spawn: 1082 failed initializing cgroup support
+lxc-start: start.c: __lxc_start: 1332 failed to spawn 'container'
+lxc-start: lxc_start.c: main: 344 The container failed to start.
+lxc-start: lxc_start.c: main: 348 Additional information can be obtained by 
setting the --logfile and --logpriority options.
+
+## begin backport ##
+
+From 843d5baf6aad6c53fc00ea8d95d83209a4f92de1 Mon Sep 17 00:00:00 2001
+From: Martin Pitt <martin.p...@ubuntu.com>
+Date: Thu, 10 Nov 2016 05:33:13 +0100
+Subject: [PATCH] core: don't use the unified hierarchy for the systemd cgroup
+ yet (#4628)
+
+Too many things don't get along with the unified hierarchy yet:
+
+ * https://github.com/opencontainers/runc/issues/1175
+ * https://github.com/docker/docker/issues/28109
+ * https://github.com/lxc/lxc/issues/1280
+
+So revert the default to the legacy hierarchy for now. Developers of the above
+software can opt into the unified hierarchy with
+"systemd.legacy_systemd_cgroup_controller=0".
+---
+ src/basic/cgroup-util.c | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/src/basic/cgroup-util.c b/src/basic/cgroup-util.c
+index 5e73753..dc13025 100644
+--- a/src/basic/cgroup-util.c
 b/src/basic/cgroup-util.c
+@@ -2423,10 +2423,10 @@ bool cg_is_unified_systemd_controller_wanted(void) {
+ 
+ r = 
get_proc_cmdline_key("systemd.legacy_systemd_cgroup_controller=", );
+ if (r < 0)
+-return true;
++return false;
+ 
+ if (r == 0)
+-wanted = true;
++wanted = false;
+ else
+ wanted = parse_boolean(value) <= 0;
+ }
+-- 
+2.10.1
+
diff --git a/meta/recipes-core/systemd/systemd_232.bb 
b/meta/recipes-core/systemd/systemd_232.bb
index baee02e..a978bec 100644
--- a/meta/recipes-core/systemd/systemd_232.bb
+++ b/meta/recipes-core/systemd/systemd_232.bb
@@ -32,6 +32,7 @@ SRC_URI += " \
file://0017-remove-duplicate-include-uchar.h.patch \
file://0018-check-for-uchar.h-in-configure.patch \

file://0019-socket-util-don-t-fail-if-libc-doesn-t-support-IDN.patch \
+   
file://0020-back-port-233-don-t-use-the-unified-hierarchy-for-the-systemd.patch 
\
 "
 SRC_URI_append_libc-uclibc = "\

file://0002-units-Prefer-getty-to-agetty-in-console-setup-system.patch \
-- 
2.10.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] liburi-perl: Uprev from 1.60 to 1.71 to pickup bugfixes and compatibilty

[Jason Wessel]

Looks like someone beat me to it, but it had not been merged yet, so this can 
be ignored.

Cheers,
Jason.

On 12/07/2016 12:31 PM, Jason Wessel wrote:

The exo recipe from meta-oe no longer compiles and some parts of
URI::Escape no longer work due to changes in perl 5.  The main reason
for the uprev is to pickup the fix for this problem:

ERROR: \C is in regex; marked by <-- HERE in m/(\C <-- HERE )/ at 
/perl5/URI/Escape.pm line 205.

The perl 5.24 transitioned the \C regex from deprecated into a hard
error.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
  .../perl/{liburi-perl_1.60.bb => liburi-perl_1.71.bb} | 8 
  1 file changed, 4 insertions(+), 4 deletions(-)
  rename meta/recipes-devtools/perl/{liburi-perl_1.60.bb => 
liburi-perl_1.71.bb} (66%)

diff --git a/meta/recipes-devtools/perl/liburi-perl_1.60.bb 
b/meta/recipes-devtools/perl/liburi-perl_1.71.bb
similarity index 66%
rename from meta/recipes-devtools/perl/liburi-perl_1.60.bb
rename to meta/recipes-devtools/perl/liburi-perl_1.71.bb
index 8809a44..1806d93 100644
--- a/meta/recipes-devtools/perl/liburi-perl_1.60.bb
+++ b/meta/recipes-devtools/perl/liburi-perl_1.71.bb
@@ -6,14 +6,14 @@ and manipulate the various components that make up these 
strings."
  SECTION = "libs"
  LICENSE = "Artistic-1.0 | GPL-1.0+"
  
-LIC_FILES_CHKSUM = "file://README;beginline=26;endline=30;md5=6c33ae5c87fd1c4897714e122dd9c23d"

+LIC_FILES_CHKSUM = "file://LICENSE;md5=c453e94fae672800f83bc1bd7a38b53f"
  
  DEPENDS += "perl"
  
-SRC_URI = "http://www.cpan.org/authors/id/G/GA/GAAS/URI-${PV}.tar.gz;

+SRC_URI = "http://www.cpan.org/authors/id/E/ET/ETHER/URI-${PV}.tar.gz;
  
-SRC_URI[md5sum] = "70f739be8ce28b8baba7c5920ffee4dc"

-SRC_URI[sha256sum] = 
"1f92d3dc64acb8845e9917c945e22b9a5275aeb9ff924eb7873c3b7a5c0d2377"
+SRC_URI[md5sum] = "247c3da29a794f72730e01aa5a715daf"
+SRC_URI[sha256sum] = 
"9c8eca0d7f39e74bbc14706293e653b699238eeb1a7690cc9c136fb8c2644115"
  
  S = "${WORKDIR}/URI-${PV}"
  



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] liburi-perl: Uprev from 1.60 to 1.71 to pickup bugfixes and compatibilty

[Jason Wessel]

The exo recipe from meta-oe no longer compiles and some parts of
URI::Escape no longer work due to changes in perl 5.  The main reason
for the uprev is to pickup the fix for this problem:

ERROR: \C is in regex; marked by <-- HERE in m/(\C <-- HERE )/ at 
/perl5/URI/Escape.pm line 205.

The perl 5.24 transitioned the \C regex from deprecated into a hard
error.

Signed-off-by: Jason Wessel <jason.wes...@windriver.com>
---
 .../perl/{liburi-perl_1.60.bb => liburi-perl_1.71.bb} | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)
 rename meta/recipes-devtools/perl/{liburi-perl_1.60.bb => liburi-perl_1.71.bb} 
(66%)

diff --git a/meta/recipes-devtools/perl/liburi-perl_1.60.bb 
b/meta/recipes-devtools/perl/liburi-perl_1.71.bb
similarity index 66%
rename from meta/recipes-devtools/perl/liburi-perl_1.60.bb
rename to meta/recipes-devtools/perl/liburi-perl_1.71.bb
index 8809a44..1806d93 100644
--- a/meta/recipes-devtools/perl/liburi-perl_1.60.bb
+++ b/meta/recipes-devtools/perl/liburi-perl_1.71.bb
@@ -6,14 +6,14 @@ and manipulate the various components that make up these 
strings."
 SECTION = "libs"
 LICENSE = "Artistic-1.0 | GPL-1.0+"
 
-LIC_FILES_CHKSUM = 
"file://README;beginline=26;endline=30;md5=6c33ae5c87fd1c4897714e122dd9c23d"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=c453e94fae672800f83bc1bd7a38b53f"
 
 DEPENDS += "perl"
 
-SRC_URI = "http://www.cpan.org/authors/id/G/GA/GAAS/URI-${PV}.tar.gz;
+SRC_URI = "http://www.cpan.org/authors/id/E/ET/ETHER/URI-${PV}.tar.gz;
 
-SRC_URI[md5sum] = "70f739be8ce28b8baba7c5920ffee4dc"
-SRC_URI[sha256sum] = 
"1f92d3dc64acb8845e9917c945e22b9a5275aeb9ff924eb7873c3b7a5c0d2377"
+SRC_URI[md5sum] = "247c3da29a794f72730e01aa5a715daf"
+SRC_URI[sha256sum] = 
"9c8eca0d7f39e74bbc14706293e653b699238eeb1a7690cc9c136fb8c2644115"
 
 S = "${WORKDIR}/URI-${PV}"
 
-- 
2.10.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] initscripts: Fix regression for requiring /bin/bash

[Jason Wessel]

It is not possible to create a rootfs with only busybox + initscipts.
This is a result of a regression from commit
a4b53872a8a9a2743299acbff015f7f2750a69d6 (initscripts: add
/sbin/sushell for systemd service debug-shell).

The /sbin/sushell should just use /bin/sh else you end up with a
problem when creating the end image with a failed smart transaction
shown below:

ERROR: Unable to install packages. Command 
'/proj/bitbake_build/tmp/sysroots/x86_64-linux/usr/bin/smart --quiet 
--data-dir=/proj/bitbake_build/tmp/work/qemuarma9-wrs-linux-gnueabi/wrlinux-image-glibc-small/1.0-r1/rootfs/var/lib/smart
 install -y dropbear@armv7at2_vfp hac@armv7at2_vfp run-postinsts@all 
kernel-modules@qemuarma9 packagegroup-core-boot@qemuarma9' returned 1:
error: Can't install initscripts-1.0-r155.0@armv7at2_vfp: no package provides 
/bin/bash

Signed-off-by: Jason Wessel jason.wes...@windriver.com
---
 .../initscripts/initscripts-1.0/sushell|4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/initscripts/initscripts-1.0/sushell 
b/meta/recipes-core/initscripts/initscripts-1.0/sushell
index 7d96941..c319d95 100644
--- a/meta/recipes-core/initscripts/initscripts-1.0/sushell
+++ b/meta/recipes-core/initscripts/initscripts-1.0/sushell
@@ -1,5 +1,5 @@
-#!/bin/bash
+#!/bin/sh
 
-[ -z $SUSHELL ]  SUSHELL=/bin/bash
+[ -z $SUSHELL ]  SUSHELL=/bin/sh
 
 exec $SUSHELL
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] relocate_sdk.py: Possible bug, /lib64/ld-linux-x86-64.so.2 not relocated

[Jason Wessel]

On 03/01/2014 08:26 AM, Stefan Agner wrote:
 Hi,

 Using top of dylan branch, I generated a SDK using bitbake
 meta-toolchain. I'm running Arch Linux, but I also see similar issues on
 Ubuntu 12.04.4 LTS:

 Some binaries segfault when running them. I discovered, that the dynamic
 linker of the host is used:
 $ ldd
 /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc
 linux-vdso.so.1 (0x7fffc8f11000)
 libc.so.6 =
 /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/../../../lib/libc.so.6
 (0x7f4d85eb1000)
 /lib64/ld-linux-x86-64.so.2 (0x7f4d8625f000)

 When forcing the dynamic linker of the SDK, the binary works:
 $
 /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/lib/ld-linux-x86-64.so.2
 /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc
 arm-angstrom-linux-gnueabi-gcc: fatal error: no input files
 compilation terminated.

 Digging around I found this commit by Jason:
 3752a9c6d772b39bbe04d62ef4d3527b4c7198c1
 relocate_sdk.py: Fix corruption of sdk binaries

 It changes the behavior and don't relocates stuff which start with
 /lib64. When I install the SDK with -S (copy the relocate scripts), and
 remove the condition around line 95, the binaries work as expected. Also
 LDD shows that ld-linux-x86-64.so.2 of the SDK will be used.

 Btw, I think this bug is related to that:
 https://github.com/Angstrom-distribution/setup-scripts/issues/25

 Since I don't understand the original intention of that commit, I'm not
 sure how to fix this properly...

My guess is that the compiler recipe is not using the right linker.  The change 
to ignore binaries that are linked against the host libc is there 
intentionally.  We don't want to relocate binaries that are for a specific host 
or set of hosts.  The main reason it is there is to allow for the inclusion of 
binaries that were generated by the SDK as well as binaries that came from 
outside the SDK.

When you use the host's ld to link something and end up linking against the 
host's linux-vdso.so.1 there is not enough space for the path relocations.  The 
proper fix should be to ensure the recipe that builds your compiler has the 
same patches and linking of binaries as the oe-core cross compiler.

Cheers,
Jason.
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 1/6] unfs3: Add a NFSv3 user mode server for use with runqemu

[Jason Wessel]

On 02/01/2014 03:22 AM, Richard Purdie wrote:
 Hi Jason,

 We're seeing a parallel make race on the autobuilder and it seems its
 from this patch:

You actually still need the patch or you get a different parallel build error 
with the lib file.  I tracked down the regression from the original.

The original patch looks like:

--- a/Config/Makefile.in
+++ b/Config/Makefile.in
@@ -12,7 +12,9 @@ all: lib.a
 lib.a: $(OBJS)
$(AR) crs lib.a $(OBJS)
 
-y.tab.h y.tab.c: exports.y
+y.tab.h: y.tab.c
+
+y.tab.c: exports.y
$(YACC) -d exports.y


So I am thinking something was lost in porting/translation.  I'll send you a 
patch shortly.  Thanks for the pointer to the build error.

Cheers,
Jason.

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] unfs3: Fix dependencies and allow target builds

[Jason Wessel]

Fixed in this patch:
  * All patches marked as submitted to the upstream
  * Remove the pseudo dependency because unfs3 can fully stand alone
or be used with pseudo and it does not link against pseudo
  * Dependencies to flex for nativesdk and target builds are fixed
such that unfs3 can be deployed into an image
  * Delete commented out unfs3 references in separatebuilddir.inc
because unfs3 works correctly with autotools.

Signed-off-by: Jason Wessel jason.wes...@windriver.com
---
 meta/conf/distro/include/seperatebuilddir.inc  |2 --
 .../unfs3/unfs3/alternate_rpc_ports.patch  |2 +-
 .../unfs3/unfs3/fix_compile_warning.patch  |2 +-
 .../fix_pid_race_parent_writes_child_pid.patch |2 +-
 .../unfs3/unfs3/fix_warnings.patch |2 +-
 .../unfs3/unfs3/relative_max_socket_path_len.patch |2 +-
 .../unfs3/unfs3/rename_fh_cache.patch  |2 +-
 .../unfs3/unfs3/tcp_no_delay.patch |2 +-
 .../unfs3/unfs3/unfs3_parallel_build.patch |2 +-
 meta/recipes-devtools/unfs3/unfs3_0.9.22.r490.bb   |   17 ++---
 10 files changed, 10 insertions(+), 25 deletions(-)

diff --git a/meta/conf/distro/include/seperatebuilddir.inc 
b/meta/conf/distro/include/seperatebuilddir.inc
index 64e8f6d..a4869c5 100644
--- a/meta/conf/distro/include/seperatebuilddir.inc
+++ b/meta/conf/distro/include/seperatebuilddir.inc
@@ -528,7 +528,6 @@ B_pn-nativesdk-renderproto = ${SEPB}
 B_pn-nativesdk-sqlite3 = ${SEPB}
 B_pn-nativesdk-systemtap = ${SEPB}
 # Not automake, no out of tree build support
-#B_pn-nativesdk-unfs3 = ${SEPB}
 B_pn-nativesdk-util-macros = ${SEPB}
 B_pn-nativesdk-xcb-proto = ${SEPB}
 B_pn-nativesdk-xcmiscproto = ${SEPB}
@@ -676,7 +675,6 @@ B_pn-tremor = ${SEPB}
 B_pn-tslib = ${SEPB}
 B_pn-udev = ${SEPB}
 # Not automake, no out of tree build support
-#B_pn-unfs3-native = ${SEPB}
 B_pn-usbutils = ${SEPB}
 B_pn-util-linux = ${SEPB}
 B_pn-util-linux-native = ${SEPB}
diff --git a/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch 
b/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
index 029582e..ff745d4 100644
--- a/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
+++ b/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
@@ -5,7 +5,7 @@ be able to specify alternate rpc port numbers.
 
 Jason Wessel jason.wes...@windriver.com
 
-Upstream-Status: Pending
+Upstream-Status: Submitted http://sourceforge.net/p/unfs3/bugs/5/
 
 ---
  daemon.c |   44 +++-
diff --git a/meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch 
b/meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch
index 604824a..aada014 100644
--- a/meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch
+++ b/meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch
@@ -5,7 +5,7 @@ If this one does fail something is very, very wrong.
 
 Signed-off-by: Jason Wessel jason.wes...@windriver.com
 
-Upstream-Status: Pending
+Upstream-Status: Submitted http://sourceforge.net/p/unfs3/bugs/5/
 
 ---
  daemon.c |3 ++-
diff --git 
a/meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch 
b/meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch
index 76d7555..46b187e 100644
--- 
a/meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch
+++ 
b/meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch
@@ -10,7 +10,7 @@ on a busy system.
 
 Signed-off-by: Jason Wessel jason.wes...@windriver.com
 
-Upstream-Status: Pending
+Upstream-Status: Submitted http://sourceforge.net/p/unfs3/bugs/5/
 
 ---
  daemon.c |   12 +---
diff --git a/meta/recipes-devtools/unfs3/unfs3/fix_warnings.patch 
b/meta/recipes-devtools/unfs3/unfs3/fix_warnings.patch
index ed61ea1..cda8655 100644
--- a/meta/recipes-devtools/unfs3/unfs3/fix_warnings.patch
+++ b/meta/recipes-devtools/unfs3/unfs3/fix_warnings.patch
@@ -10,7 +10,7 @@ exports.y:350: warning: large integer implicitly truncated to 
unsigned type
 
 Signed-off-by: Jason Wessel jason.wes...@windriver.com
 
-Upstream-Status: Pending
+Upstream-Status: Submitted http://sourceforge.net/p/unfs3/bugs/5/
 
 ---
  Config/exports.l |3 +++
diff --git 
a/meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch 
b/meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch
index 17ad7b6..219dd35 100644
--- a/meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch
+++ b/meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch
@@ -12,7 +12,7 @@ heavy use of named sockets with long names.
 
 Signed-off-by: Jason Wessel jason.wes...@windriver.com
 
-Upstream-Status: Pending
+Upstream-Status: Submitted http://sourceforge.net/p/unfs3/bugs/5/
 
 ---
  nfs.c |   29 +++--
diff --git a/meta/recipes-devtools/unfs3/unfs3/rename_fh_cache.patch 
b/meta/recipes-devtools/unfs3/unfs3/rename_fh_cache.patch
index

[OE-core] [PATCH 0/1] unfs3: Fix dependencies and allow target builds

[Jason Wessel]

The prior series which introduced unfs3 was going to have a v3, but
the v2 series already got merged to the master branch.

The patch that follows was originally going to be squashed into
different patches in the series.  The aim is to address the remaining
comments from Phil.

Thanks,
Jason.


___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] unfs3: Fix dependencies and allow target builds

[Jason Wessel]

Fixed in this patch:
  * All patches marked as submitted to the upstream
  * Remove the pseudo dependency because unfs3 can fully stand alone
or be used with pseudo and it does not link against pseudo
  * Dependencies to flex for nativesdk and target builds are fixed
such that unfs3 can be deployed into an image
  * Add unfs3 references in separatebuilddir.inc because unfs3
works correctly with autotools.

Signed-off-by: Jason Wessel jason.wes...@windriver.com
---
 meta/conf/distro/include/seperatebuilddir.inc  |7 +++
 .../unfs3/unfs3/alternate_rpc_ports.patch  |2 +-
 .../unfs3/unfs3/fix_compile_warning.patch  |2 +-
 .../fix_pid_race_parent_writes_child_pid.patch |2 +-
 .../unfs3/unfs3/fix_warnings.patch |2 +-
 .../unfs3/unfs3/relative_max_socket_path_len.patch |2 +-
 .../unfs3/unfs3/rename_fh_cache.patch  |2 +-
 .../unfs3/unfs3/tcp_no_delay.patch |2 +-
 .../unfs3/unfs3/unfs3_parallel_build.patch |2 +-
 meta/recipes-devtools/unfs3/unfs3_0.9.22.r490.bb   |   17 ++---
 10 files changed, 13 insertions(+), 27 deletions(-)

diff --git a/meta/conf/distro/include/seperatebuilddir.inc 
b/meta/conf/distro/include/seperatebuilddir.inc
index 64e8f6d..c6897a5 100644
--- a/meta/conf/distro/include/seperatebuilddir.inc
+++ b/meta/conf/distro/include/seperatebuilddir.inc
@@ -527,8 +527,7 @@ B_pn-nativesdk-readline = ${SEPB}
 B_pn-nativesdk-renderproto = ${SEPB}
 B_pn-nativesdk-sqlite3 = ${SEPB}
 B_pn-nativesdk-systemtap = ${SEPB}
-# Not automake, no out of tree build support
-#B_pn-nativesdk-unfs3 = ${SEPB}
+B_pn-nativesdk-unfs3 = ${SEPB}
 B_pn-nativesdk-util-macros = ${SEPB}
 B_pn-nativesdk-xcb-proto = ${SEPB}
 B_pn-nativesdk-xcmiscproto = ${SEPB}
@@ -675,8 +674,8 @@ B_pn-time = ${SEPB}
 B_pn-tremor = ${SEPB}
 B_pn-tslib = ${SEPB}
 B_pn-udev = ${SEPB}
-# Not automake, no out of tree build support
-#B_pn-unfs3-native = ${SEPB}
+B_pn-unfs3 = ${SEPB}
+B_pn-unfs3-native = ${SEPB}
 B_pn-usbutils = ${SEPB}
 B_pn-util-linux = ${SEPB}
 B_pn-util-linux-native = ${SEPB}
diff --git a/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch 
b/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
index 029582e..ff745d4 100644
--- a/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
+++ b/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
@@ -5,7 +5,7 @@ be able to specify alternate rpc port numbers.
 
 Jason Wessel jason.wes...@windriver.com
 
-Upstream-Status: Pending
+Upstream-Status: Submitted http://sourceforge.net/p/unfs3/bugs/5/
 
 ---
  daemon.c |   44 +++-
diff --git a/meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch 
b/meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch
index 604824a..aada014 100644
--- a/meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch
+++ b/meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch
@@ -5,7 +5,7 @@ If this one does fail something is very, very wrong.
 
 Signed-off-by: Jason Wessel jason.wes...@windriver.com
 
-Upstream-Status: Pending
+Upstream-Status: Submitted http://sourceforge.net/p/unfs3/bugs/5/
 
 ---
  daemon.c |3 ++-
diff --git 
a/meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch 
b/meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch
index 76d7555..46b187e 100644
--- 
a/meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch
+++ 
b/meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch
@@ -10,7 +10,7 @@ on a busy system.
 
 Signed-off-by: Jason Wessel jason.wes...@windriver.com
 
-Upstream-Status: Pending
+Upstream-Status: Submitted http://sourceforge.net/p/unfs3/bugs/5/
 
 ---
  daemon.c |   12 +---
diff --git a/meta/recipes-devtools/unfs3/unfs3/fix_warnings.patch 
b/meta/recipes-devtools/unfs3/unfs3/fix_warnings.patch
index ed61ea1..cda8655 100644
--- a/meta/recipes-devtools/unfs3/unfs3/fix_warnings.patch
+++ b/meta/recipes-devtools/unfs3/unfs3/fix_warnings.patch
@@ -10,7 +10,7 @@ exports.y:350: warning: large integer implicitly truncated to 
unsigned type
 
 Signed-off-by: Jason Wessel jason.wes...@windriver.com
 
-Upstream-Status: Pending
+Upstream-Status: Submitted http://sourceforge.net/p/unfs3/bugs/5/
 
 ---
  Config/exports.l |3 +++
diff --git 
a/meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch 
b/meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch
index 17ad7b6..219dd35 100644
--- a/meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch
+++ b/meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch
@@ -12,7 +12,7 @@ heavy use of named sockets with long names.
 
 Signed-off-by: Jason Wessel jason.wes...@windriver.com
 
-Upstream-Status: Pending
+Upstream-Status: Submitted http://sourceforge.net/p/unfs3/bugs/5/
 
 ---
  nfs.c |   29 +++--
diff

Re: [OE-core] [PATCH 1/1] unfs3: Fix dependencies and allow target builds

[Jason Wessel]

On 01/29/2014 08:19 AM, Richard Purdie wrote:
 On Wed, 2014-01-29 at 08:12 -0600, Jason Wessel wrote:
  Fixed in this patch:
* All patches marked as submitted to the upstream
* Remove the pseudo dependency because unfs3 can fully stand alone
  or be used with pseudo and it does not link against pseudo
* Dependencies to flex for nativesdk and target builds are fixed
  such that unfs3 can be deployed into an image
* Delete commented out unfs3 references in separatebuilddir.inc
  because unfs3 works correctly with autotools.
  
  Signed-off-by: Jason Wessel jason.wes...@windriver.com
  ---
   meta/conf/distro/include/seperatebuilddir.inc  |2 --
   .../unfs3/unfs3/alternate_rpc_ports.patch  |2 +-
   .../unfs3/unfs3/fix_compile_warning.patch  |2 +-
   .../fix_pid_race_parent_writes_child_pid.patch |2 +-
   .../unfs3/unfs3/fix_warnings.patch |2 +-
   .../unfs3/unfs3/relative_max_socket_path_len.patch |2 +-
   .../unfs3/unfs3/rename_fh_cache.patch  |2 +-
   .../unfs3/unfs3/tcp_no_delay.patch |2 +-
   .../unfs3/unfs3/unfs3_parallel_build.patch |2 +-
   meta/recipes-devtools/unfs3/unfs3_0.9.22.r490.bb   |   17 
  ++---
   10 files changed, 10 insertions(+), 25 deletions(-)
  
  diff --git a/meta/conf/distro/include/seperatebuilddir.inc 
  b/meta/conf/distro/include/seperatebuilddir.inc
  index 64e8f6d..a4869c5 100644
  --- a/meta/conf/distro/include/seperatebuilddir.inc
  +++ b/meta/conf/distro/include/seperatebuilddir.inc
  @@ -528,7 +528,6 @@ B_pn-nativesdk-renderproto = ${SEPB}
   B_pn-nativesdk-sqlite3 = ${SEPB}
   B_pn-nativesdk-systemtap = ${SEPB}
   # Not automake, no out of tree build support
  -#B_pn-nativesdk-unfs3 = ${SEPB}
   B_pn-nativesdk-util-macros = ${SEPB}
   B_pn-nativesdk-xcb-proto = ${SEPB}
   B_pn-nativesdk-xcmiscproto = ${SEPB}
  @@ -676,7 +675,6 @@ B_pn-tremor = ${SEPB}
   B_pn-tslib = ${SEPB}
   B_pn-udev = ${SEPB}
   # Not automake, no out of tree build support
  -#B_pn-unfs3-native = ${SEPB}
   B_pn-usbutils = ${SEPB}
   B_pn-util-linux = ${SEPB}
   B_pn-util-linux-native = ${SEPB}
 If unfs is now autotools based, can we enable out of tree builds here?

I did not realize this was an opt in list.   I'll send a v2.  Testing shows 
that it works properly.

Cheers,
Jason.
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 5/6] runqemu-export-rootfs: update for unfs3

[Jason Wessel]

The unfs3 no longer has an rpc.mountd component. There is just a
single server for mountd and nfsd requests.  This means changing
the name of the server in the scripts that check for it.

[YOCTO #5639]

Signed-off-by: Jason Wessel jason.wes...@windriver.com
---
 scripts/runqemu-export-rootfs |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/runqemu-export-rootfs b/scripts/runqemu-export-rootfs
index 2d87996..79403ab 100755
--- a/scripts/runqemu-export-rootfs
+++ b/scripts/runqemu-export-rootfs
@@ -46,8 +46,8 @@ if [ -z $SYSROOT_SETUP_SCRIPT ]; then
 fi
 . $SYSROOT_SETUP_SCRIPT
 
-if [ ! -e $OECORE_NATIVE_SYSROOT/usr/sbin/rpc.mountd ]; then
-   echo Error: Unable to find rpc.mountd binary in 
$OECORE_NATIVE_SYSROOT/usr/sbin/
+if [ ! -e $OECORE_NATIVE_SYSROOT/usr/sbin/unfsd ]; then
+   echo Error: Unable to find unfsd binary in 
$OECORE_NATIVE_SYSROOT/usr/sbin/
 
if [ x$OECORE_DISTRO_VERSION = x ]; then
echo Have you run 'bitbake meta-ide-support'?
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 6/6] runqemu, runqemu-internal: Allow slirp for NFS and KVM use

[Jason Wessel]

The default slirp address for the NFS server is 10.0.2.2.  If not
using a tap interface this address must be used or the target system
cannot connect properly.  Also the ip=... kernel arguments need to be
set to dhcp when using slirp or the root NFS will not get setup
properly.

The call to cleanup() results in a routine which is not defined when
setting up the NFS because it is called before acquire() for the
locking of the tap interfaces, the solution being to simply not call
cleanup() that early.

When using slirp, kvm should not execute the vhost net checks because
the vhost net will not be configure or used.

Signed-off-by: Jason Wessel jason.wes...@windriver.com
---
 scripts/runqemu  |   10 ++
 scripts/runqemu-internal |   19 +++
 2 files changed, 17 insertions(+), 12 deletions(-)

diff --git a/scripts/runqemu b/scripts/runqemu
index dcb2931..9272b6f 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -279,10 +279,12 @@ if [ x$KVM_ENABLED = xyes ]; then
 exit 1;
 fi
 if [ ! -w /dev/vhost-net -o ! -r /dev/vhost-net ]; then
-echo You have no rights on /dev/vhost-net.
-echo Please change the ownership of this file as described at:
-echo $YOCTO_PARAVIRT_KVM_WIKI;
-exit 1;
+   if [ $SLIRP_ENABLED != yes ] ; then
+echo You have no rights on /dev/vhost-net.
+echo Please change the ownership of this file as described at:
+echo $YOCTO_PARAVIRT_KVM_WIKI;
+exit 1;
+   fi
 fi
 fi
 
diff --git a/scripts/runqemu-internal b/scripts/runqemu-internal
index cecb527..e3b0729 100755
--- a/scripts/runqemu-internal
+++ b/scripts/runqemu-internal
@@ -114,7 +114,7 @@ NFSRUNNING=false
 ORIG_STTY=$(stty -g)
 
 if [ $SLIRP_ENABLED = yes ]; then
-KERNEL_NETWORK_CMD=
+KERNEL_NETWORK_CMD=ip=dhcp
 QEMU_TAP_CMD=
 QEMU_UI_OPTIONS=-show-cursor -usb -usbdevice wacom-tablet
 if [ $KVM_ACTIVE = yes ]; then
@@ -301,9 +301,18 @@ if [ $FSTYPE != nfs -a $FSTYPE != vmdk -a ! -f 
$ROOTFS ]; then
 return 1
 fi
 
-if [ $FSTYPE = nfs ]; then
+if [ $NFS_SERVER =  ]; then
 NFS_SERVER=192.168.7.1
+if [ $SLIRP_ENABLED = yes ]; then
+   NFS_SERVER=10.0.2.2
+fi
+fi
+
+if [ $FSTYPE = nfs ]; then
 NFS_DIR=`echo $ROOTFS | sed 's/^[^:]*:\(.*\)/\1/'`
+if [ $NFS_INSTANCE =  ] ; then
+   NFS_INSTANCE=0
+fi
 MOUNTD_RPCPORT=$[ 2 + $NFS_INSTANCE ]
 NFSD_RPCPORT=$[ 1 + $NFS_INSTANCE ]
 NFSD_PORT=$[ 3049 + 2 * $NFS_INSTANCE ]
@@ -317,17 +326,11 @@ if [ $FSTYPE = nfs ]; then
 echo runqemu-export-rootfs restart $ROOTFS
 runqemu-export-rootfs restart $ROOTFS
 if [ $? != 0 ]; then
-cleanup
 return 1
 fi
 NFSRUNNING=true
 fi
 
-if [ $NFS_SERVER =  ]; then
-NFS_SERVER=192.168.7.1
-NFS_DIR=$ROOTFS
-fi
-
 if [ $MACHINE = qemuarm -o $MACHINE = qemuarmv6 -o $MACHINE = 
qemuarmv7 ]; then
 QEMU=qemu-system-arm
 MACHINE_SUBTYPE=versatilepb
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 3/6] runqemu: Use the newer unfs3 for serving user space nfs

[Jason Wessel]

From: Saul Wold s...@linux.intel.com

This new version correctly handles the 64bit ext3 / ext4 issues we
were seeing with the older unfs-server which did not handle 64bit file
systems correctly, producing the duplicate cookies.

[YOCTO #5639]

Signed-off-by: Saul Wold s...@linux.intel.com
---
 scripts/runqemu-export-rootfs |   53 -
 scripts/runqemu-internal  |2 +-
 2 files changed, 6 insertions(+), 49 deletions(-)

diff --git a/scripts/runqemu-export-rootfs b/scripts/runqemu-export-rootfs
index bbdaf5b..2d87996 100755
--- a/scripts/runqemu-export-rootfs
+++ b/scripts/runqemu-export-rootfs
@@ -88,8 +88,7 @@ MOUNT_PORT=$[ 3048 + 2 * $NFS_INSTANCE ]
 
 ## For debugging you would additionally add
 ## --debug all
-MOUNTD_OPTS=--allow-non-root --mount-pid $MOUNTPID -f $EXPORTS --rmtab $RMTAB 
--prog $NFS_MOUNTPROG -r -P $MOUNT_PORT
-NFSD_OPTS=--allow-non-root --nfs-pid $NFSPID -f $EXPORTS --prog $NFS_NFSPROG 
-P $NFS_PORT -r
+UNFSD_OPTS=-p -N -i $NFSPID -e $EXPORTS -x $NFS_NFSPROG -n $NFS_PORT -y 
$NFS_MOUNTPROG -m $MOUNT_PORT
 
 # Setup the exports file
 if [ $1 = start ]; then
@@ -115,53 +114,18 @@ case $1 in
exit 1
fi
 
-   echo Starting User Mode rpc.mountd
-   echo   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/rpc.mountd 
$MOUNTD_OPTS
-   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/rpc.mountd 
$MOUNTD_OPTS
-   if [ ! $? = 0 ]; then
-   echo =
-   echo Error starting MOUNTD
-   echo =
-   if [ ! x$RPCBIND_RUNNING = x ] ; then
-   echo   If you see an error above that says:
-   echo RPC: Authentication error; why = Client 
credential too weak
-   echo   You need to add the -i option when running 
rpcbind
-   echo ===
-   echo For recent Fedora/RedHat hosts:
-   echo Add RPCBIND_ARGS=-i to /etc/sysconfig/rpcbind
-   echo   or
-   echo Add RPCBIND_OPTIONS=-i to /etc/sysconfig/rpcbind
-   echo Then run as root: /etc/init.d/rpcbind restart
-   echo ===
-   echo For recent Debian/Ubuntu hosts:
-   echo Add OPTIONS=\-i -w\  to /etc/default/rpcbind
-   echo sudo service portmap restart
-   fi
-
-   exit 1
-   fi
-
echo Starting User Mode nfsd
-   echo   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/rpc.nfsd 
$NFSD_OPTS
-   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/rpc.nfsd $NFSD_OPTS
+   echo   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/unfsd 
$UNFSD_OPTS
+   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/unfsd $UNFSD_OPTS
if [ ! $? = 0 ]; then
echo Error starting nfsd
exit 1
fi
-   # Check to make sure everything started ok.
-   if [ ! -f $MOUNTPID ]; then
-   echo rpc.mountd did not start correctly
-   exit 1
-   fi
+Check to make sure everything started ok.
if [ ! -f $NFSPID ]; then
echo rpc.nfsd did not start correctly
exit 1
fi
-   ps -fp `cat $MOUNTPID`  /dev/null 2 /dev/null
-   if [ ! $? = 0 ]; then
-   echo rpc.mountd did not start correctly
-   exit 1
-   fi
ps -fp `cat $NFSPID`  /dev/null 2 /dev/null
if [ ! $? = 0 ]; then
echo rpc.nfsd did not start correctly
@@ -169,16 +133,9 @@ case $1 in
fi
echo  
echo On your target please remember to add the following options for 
NFS
-   echo 
nfsroot=IP_ADDRESS:$NFS_EXPORT_DIR,nfsvers=2,mountprog=$NFS_MOUNTPROG,nfsprog=$NFS_NFSPROG,udp
+   echo 
nfsroot=IP_ADDRESS:$NFS_EXPORT_DIR,nfsvers=3,port=$NFSD_PORT,mountprog=$MOUNTD_RPCPORT,nfsprog=$NFSD_RPCPORT,udp,mountport=$MOUNTD_PORT
;;
   stop)
-   if [ -f $MOUNTPID ]; then
-   echo Stopping rpc.mountd
-   kill `cat $MOUNTPID`
-   rm -f $MOUNTPID
-   else
-   echo No PID file, not stopping rpc.mountd
-   fi
if [ -f $NFSPID ]; then
echo Stopping rpc.nfsd
kill `cat $NFSPID`
diff --git a/scripts/runqemu-internal b/scripts/runqemu-internal
index 284b88c..cecb527 100755
--- a/scripts/runqemu-internal
+++ b/scripts/runqemu-internal
@@ -308,7 +308,7 @@ if [ $FSTYPE = nfs ]; then
 NFSD_RPCPORT=$[ 1 + $NFS_INSTANCE ]
 NFSD_PORT=$[ 3049 + 2 * $NFS_INSTANCE ]
 MOUNTD_PORT=$[ 3048 + 2 * $NFS_INSTANCE ]
-
UNFS_OPTS=nfsvers=2,mountprog=$MOUNTD_RPCPORT,nfsprog=$NFSD_RPCPORT,udp,port=$NFSD_PORT,mountport=$MOUNTD_PORT
+

[OE-core] [PATCH v2 2/6] unfs3, unfs-server: Replace all instances of unfs-server with unfs3

[Jason Wessel]

Other recipes dependencies and even some comments need to be updated
for the removal of unfs-server and the replacement with unfs3.  The
unfs3 is a complete drop in replacement providing all the prior
functionality of NFSv2 but also adding NFSv3.

[YOCTO #5639]

Signed-off-by: Jason Wessel jason.wes...@windriver.com
---
 meta/conf/distro/include/seperatebuilddir.inc  |4 ++--
 meta/recipes-core/meta/meta-ide-support.bb |2 +-
 .../nativesdk-packagegroup-sdk-host.bb |2 +-
 .../adt-installer/scripts/adt_installer_internal   |2 +-
 .../packagegroups/packagegroup-toolset-native.bb   |2 +-
 meta/site/ix86-common  |2 +-
 meta/site/x86_64-linux |2 +-
 7 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/meta/conf/distro/include/seperatebuilddir.inc 
b/meta/conf/distro/include/seperatebuilddir.inc
index 3c36c22..64e8f6d 100644
--- a/meta/conf/distro/include/seperatebuilddir.inc
+++ b/meta/conf/distro/include/seperatebuilddir.inc
@@ -528,7 +528,7 @@ B_pn-nativesdk-renderproto = ${SEPB}
 B_pn-nativesdk-sqlite3 = ${SEPB}
 B_pn-nativesdk-systemtap = ${SEPB}
 # Not automake, no out of tree build support
-#B_pn-nativesdk-unfs-server = ${SEPB}
+#B_pn-nativesdk-unfs3 = ${SEPB}
 B_pn-nativesdk-util-macros = ${SEPB}
 B_pn-nativesdk-xcb-proto = ${SEPB}
 B_pn-nativesdk-xcmiscproto = ${SEPB}
@@ -676,7 +676,7 @@ B_pn-tremor = ${SEPB}
 B_pn-tslib = ${SEPB}
 B_pn-udev = ${SEPB}
 # Not automake, no out of tree build support
-#B_pn-unfs-server-native = ${SEPB}
+#B_pn-unfs3-native = ${SEPB}
 B_pn-usbutils = ${SEPB}
 B_pn-util-linux = ${SEPB}
 B_pn-util-linux-native = ${SEPB}
diff --git a/meta/recipes-core/meta/meta-ide-support.bb 
b/meta/recipes-core/meta/meta-ide-support.bb
index d4c410e..40bda28 100644
--- a/meta/recipes-core/meta/meta-ide-support.bb
+++ b/meta/recipes-core/meta/meta-ide-support.bb
@@ -4,7 +4,7 @@ LICENSE = MIT
 LIC_FILES_CHKSUM = 
file://${COREBASE}/LICENSE;md5=4d92cd373abda3937c2bc47fbc49d690 \
 
file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420
 
-DEPENDS = virtual/libc gdb-cross qemu-native qemu-helper-native 
unfs-server-native
+DEPENDS = virtual/libc gdb-cross qemu-native qemu-helper-native unfs3-native
 PR = r3
 
 inherit meta toolchain-scripts
diff --git a/meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb 
b/meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb
index 1065f30..855fe4e 100644
--- a/meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb
+++ b/meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb
@@ -15,7 +15,7 @@ RDEPENDS_${PN} = \
 nativesdk-qemu \
 nativesdk-qemu-helper \
 nativesdk-pseudo \
-nativesdk-unfs-server \
+nativesdk-unfs3 \
 nativesdk-opkg \
 nativesdk-libtool \
 nativesdk-autoconf \
diff --git 
a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal 
b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
index 2fef7b0..e3422f5 100755
--- 
a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
+++ 
b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
@@ -146,7 +146,7 @@ fi
 
 if [ $YOCTOADT_NFS_UTIL == Y ] || [ $YOCTOADT_NFS_UTIL == y ]; then
   echo_info \nInstalling unfs ...
-  $OPKG_INSTALL_NATIVE_CMD nativesdk-unfs-server  $YOCTOADT_INSTALL_LOG_FILE
+  $OPKG_INSTALL_NATIVE_CMD nativesdk-unfs3  $YOCTOADT_INSTALL_LOG_FILE
   check_result
 fi
 
diff --git a/meta/recipes-gnome/packagegroups/packagegroup-toolset-native.bb 
b/meta/recipes-gnome/packagegroups/packagegroup-toolset-native.bb
index c88f82a..07d4f1d 100644
--- a/meta/recipes-gnome/packagegroups/packagegroup-toolset-native.bb
+++ b/meta/recipes-gnome/packagegroups/packagegroup-toolset-native.bb
@@ -213,7 +213,7 @@ DEPENDS = \
 tiff-native \
 tzcode-native \
 u-boot-mkimage-native \
-unfs-server-native \
+unfs3-native \
 unifdef-native \
 unzip-native \
 update-rc.d-native \
diff --git a/meta/site/ix86-common b/meta/site/ix86-common
index 21fcf38..5cd2ce2 100644
--- a/meta/site/ix86-common
+++ b/meta/site/ix86-common
@@ -216,7 +216,7 @@ moz_cv_size_of_JS_BYTES_PER_DOUBLE=8
 #ofono
 ac_cv_lib_c_signalfd=${ac_cv_lib_c_signalfd=yes}
 
-#unfs-server
+#unfs3
 nfsd_cv_broken_setfsuid=${nfsd_cv_broken_setfsuid=0}
 nfsd_cv_func_statfs=${nfsd_cv_func_statfs=statfs2_bsize}
 nfsd_cv_bsd_signals=${nfsd_cv_bsd_signals=yes}
diff --git a/meta/site/x86_64-linux b/meta/site/x86_64-linux
index b4434fe..573a907 100644
--- a/meta/site/x86_64-linux
+++ b/meta/site/x86_64-linux
@@ -98,7 +98,7 @@ ac_cv_alignof_CORBA_wchar=2
 lf_cv_sane_realloc=yes
 as_cv_unaligned_access=${as_cv_unaligned_access=yes}
 
-#unfs-server
+#unfs3
 nfsd_cv_broken_setfsuid=${nfsd_cv_broken_setfsuid=0}
 nfsd_cv_func_statfs=${nfsd_cv_func_statfs=statfs2_bsize}
 nfsd_cv_bsd_signals=${nfsd_cv_bsd_signals

[OE-core] [PATCH v2 1/6] unfs3: Add a NFSv3 user mode server for use with runqemu

[Jason Wessel]

The user mode nfs server allows the use of runqemu without any root
privileges and may even be accelerated with kvm.

Example:

runqemu-extract-sdk 
tmp-eglibc/deploy/images/qemux86-64/core-image-minimal-qemux86-64.tar.bz2 rootfs

runqemu qemux86-64 `pwd`/rootfs nographic slirp kvm

[YOCTO #5639]

Signed-off-by: Jason Wessel jason.wes...@windriver.com
---
 meta/files/common-licenses/unfs3   |   24 +++
 .../unfs3/unfs3/alternate_rpc_ports.patch  |  158 
 .../unfs3/unfs3/fix_compile_warning.patch  |   25 
 .../fix_pid_race_parent_writes_child_pid.patch |   61 
 .../unfs3/unfs3/fix_warnings.patch |   53 +++
 .../unfs3/unfs3/relative_max_socket_path_len.patch |   74 +
 .../unfs3/unfs3/rename_fh_cache.patch  |   64 
 .../unfs3/unfs3/tcp_no_delay.patch |   56 +++
 .../unfs3/unfs3/unfs3_parallel_build.patch |   36 +
 meta/recipes-devtools/unfs3/unfs3_0.9.22.r490.bb   |   44 ++
 10 files changed, 595 insertions(+)
 create mode 100644 meta/files/common-licenses/unfs3
 create mode 100644 meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
 create mode 100644 meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch
 create mode 100644 
meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch
 create mode 100644 meta/recipes-devtools/unfs3/unfs3/fix_warnings.patch
 create mode 100644 
meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch
 create mode 100644 meta/recipes-devtools/unfs3/unfs3/rename_fh_cache.patch
 create mode 100644 meta/recipes-devtools/unfs3/unfs3/tcp_no_delay.patch
 create mode 100644 meta/recipes-devtools/unfs3/unfs3/unfs3_parallel_build.patch
 create mode 100644 meta/recipes-devtools/unfs3/unfs3_0.9.22.r490.bb

diff --git a/meta/files/common-licenses/unfs3 b/meta/files/common-licenses/unfs3
new file mode 100644
index 000..7f2b53f
--- /dev/null
+++ b/meta/files/common-licenses/unfs3
@@ -0,0 +1,24 @@
+UNFS3 user-space NFSv3 server
+(C) 2003, Pascal Schmidt unfs3-ser...@ewetel.net
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice,
+   this list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+3. The name of the author may not be used to endorse or promote products
+   derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED
+WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
+EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
+OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch 
b/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
new file mode 100644
index 000..029582e
--- /dev/null
+++ b/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
@@ -0,0 +1,158 @@
+Add ability to specify rcp port numbers
+
+In order to run more than one unfs server on a host system, you must
+be able to specify alternate rpc port numbers.
+
+Jason Wessel jason.wes...@windriver.com
+
+Upstream-Status: Pending
+
+---
+ daemon.c |   44 +++-
+ mount.c  |4 ++--
+ 2 files changed, 33 insertions(+), 15 deletions(-)
+
+--- a/daemon.c
 b/daemon.c
+@@ -78,6 +78,8 @@ int opt_testconfig = FALSE;
+ struct in_addr opt_bind_addr;
+ int opt_readable_executables = FALSE;
+ char *opt_pid_file = NULL;
++int nfs_prog = NFS3_PROGRAM;
++int mount_prog = MOUNTPROG;
+ 
+ /* Register with portmapper? */
+ int opt_portmapper = TRUE;
+@@ -206,7 +208,7 @@ static void parse_options(int argc, char
+ {
+ 
+ int opt = 0;
+-char *optstring = bcC:de:hl:m:n:prstTuwi:;
++char *optstring = bcC:de:hl:m:n:prstTuwi:x:y:;
+ 
+ while (opt != -1) {
+   opt = getopt(argc, argv, optstring);
+@@ -261,8 +263,24 @@ static void parse_options(int argc, char
+   printf
+   (\t-r  report unreadable executables as 
readable\n);
+   printf(\t-T  test exports file and exit\n);
++  printf(\t-x port   alternate NFS

Re: [OE-core] [PATCH v2 1/6] unfs3: Add a NFSv3 user mode server for use with runqemu

[Jason Wessel]

On 01/23/2014 09:01 AM, Phil Blundell wrote:
 On Thu, 2014-01-23 at 08:32 -0600, Jason Wessel wrote:
 +++ b/meta/files/common-licenses/unfs3
 @@ -0,0 +1,24 @@
 +UNFS3 user-space NFSv3 server
 +(C) 2003, Pascal Schmidt unfs3-ser...@ewetel.net
 +
 +Redistribution and use in source and binary forms, with or without
 +modification, are permitted provided that the following conditions are met:
 +
 +1. Redistributions of source code must retain the above copyright notice,
 +   this list of conditions and the following disclaimer.
 +2. Redistributions in binary form must reproduce the above copyright notice,
 +   this list of conditions and the following disclaimer in the documentation
 +   and/or other materials provided with the distribution.
 +3. The name of the author may not be used to endorse or promote products
 +   derived from this software without specific prior written permission.
 Isn't this just the 3-clause BSD licence?

Because it did not diff between the two (the words are different) I included 
the license.

It appears Pascal modified it ever so slightly.  Perhaps legally they mean the 
same thing, but given I am not a lawyer I opted to include it.

-- The one from above
3. The name of the author may not be used to endorse or promote products
   derived from this software without specific prior written permission.

--- vs BSD listed in YP
3. Neither the name of the University nor the names of its contributors
   may be used to endorse or promote products derived from this software
   without specific prior written permission.



There were other subtle changes too.



 --- /dev/null
 +++ b/meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
 @@ -0,0 +1,158 @@
 +Add ability to specify rcp port numbers
 +
 +In order to run more than one unfs server on a host system, you must
 +be able to specify alternate rpc port numbers.
 +
 +Jason Wessel jason.wes...@windriver.com
 +
 +Upstream-Status: Pending
 I think you said in the cover letter that the patches had been sent
 upstream.  If that's the case then it should be Upstream-Status:
 Submitted.




Thanks,  I'll make a v3, these are all submitted and pending upstream review.

 +RDEPENDS_${PN} = pseudo
 +RDEPENDS_${PN}_class-native = pseudo-native
 +RDEPENDS_${PN}_class-nativesdk = pseudo-nativesdk
 That looks a bit odd.  Are the latter two lines doing anything very
 useful?


The usnfs3 is useless without pseudo.   The 2nd of the two might not be needed, 
but it had been there since the YP 1.3 days.   For building the nativesdk and 
having a working unfs3 the 3rd one was required.   I'll leave this in there for 
the v3.



 +# This recipe is intended for -native and -nativesdk builds only,
 +# not target installs:
 Why?  If this is really the case, shouldn't it be named unfs3-native in
 the first place?

Now that the autoconf pieces are fixed in the v2, I can remove this restriction 
in v3.   It does work on the target system now.

Jason.
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/6] unfs3, unfs-server: Replace all instances of unfs-server with unfs3

[Jason Wessel]

Other recipes dependencies and even some comments need to be updated
for the removal of unfs-server and the replacement with unfs3.  The
unfs3 is a complete drop in replacement providing all the prior
functionality of NFSv2 but also adding NFSv3.

[YOCTO #5639]

Signed-off-by: Jason Wessel jason.wes...@windriver.com
---
 meta/conf/distro/include/seperatebuilddir.inc  |4 ++--
 meta/recipes-core/meta/meta-ide-support.bb |2 +-
 .../nativesdk-packagegroup-sdk-host.bb |2 +-
 .../adt-installer/scripts/adt_installer_internal   |2 +-
 .../packagegroups/packagegroup-toolset-native.bb   |2 +-
 meta/site/ix86-common  |2 +-
 meta/site/x86_64-linux |2 +-
 7 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/meta/conf/distro/include/seperatebuilddir.inc 
b/meta/conf/distro/include/seperatebuilddir.inc
index 3c36c22..64e8f6d 100644
--- a/meta/conf/distro/include/seperatebuilddir.inc
+++ b/meta/conf/distro/include/seperatebuilddir.inc
@@ -528,7 +528,7 @@ B_pn-nativesdk-renderproto = ${SEPB}
 B_pn-nativesdk-sqlite3 = ${SEPB}
 B_pn-nativesdk-systemtap = ${SEPB}
 # Not automake, no out of tree build support
-#B_pn-nativesdk-unfs-server = ${SEPB}
+#B_pn-nativesdk-unfs3 = ${SEPB}
 B_pn-nativesdk-util-macros = ${SEPB}
 B_pn-nativesdk-xcb-proto = ${SEPB}
 B_pn-nativesdk-xcmiscproto = ${SEPB}
@@ -676,7 +676,7 @@ B_pn-tremor = ${SEPB}
 B_pn-tslib = ${SEPB}
 B_pn-udev = ${SEPB}
 # Not automake, no out of tree build support
-#B_pn-unfs-server-native = ${SEPB}
+#B_pn-unfs3-native = ${SEPB}
 B_pn-usbutils = ${SEPB}
 B_pn-util-linux = ${SEPB}
 B_pn-util-linux-native = ${SEPB}
diff --git a/meta/recipes-core/meta/meta-ide-support.bb 
b/meta/recipes-core/meta/meta-ide-support.bb
index d4c410e..40bda28 100644
--- a/meta/recipes-core/meta/meta-ide-support.bb
+++ b/meta/recipes-core/meta/meta-ide-support.bb
@@ -4,7 +4,7 @@ LICENSE = MIT
 LIC_FILES_CHKSUM = 
file://${COREBASE}/LICENSE;md5=4d92cd373abda3937c2bc47fbc49d690 \
 
file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420
 
-DEPENDS = virtual/libc gdb-cross qemu-native qemu-helper-native 
unfs-server-native
+DEPENDS = virtual/libc gdb-cross qemu-native qemu-helper-native unfs3-native
 PR = r3
 
 inherit meta toolchain-scripts
diff --git a/meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb 
b/meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb
index 1065f30..855fe4e 100644
--- a/meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb
+++ b/meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb
@@ -15,7 +15,7 @@ RDEPENDS_${PN} = \
 nativesdk-qemu \
 nativesdk-qemu-helper \
 nativesdk-pseudo \
-nativesdk-unfs-server \
+nativesdk-unfs3 \
 nativesdk-opkg \
 nativesdk-libtool \
 nativesdk-autoconf \
diff --git 
a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal 
b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
index 2eb13a9..9f4cb39 100755
--- 
a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
+++ 
b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
@@ -146,7 +146,7 @@ fi
 
 if [ $YOCTOADT_NFS_UTIL == Y ] || [ $YOCTOADT_NFS_UTIL == y ]; then
   echo_info \nInstalling unfs ...
-  $OPKG_INSTALL_NATIVE_CMD nativesdk-unfs-server  $YOCTOADT_INSTALL_LOG_FILE
+  $OPKG_INSTALL_NATIVE_CMD nativesdk-unfs3  $YOCTOADT_INSTALL_LOG_FILE
   check_result
 fi
 
diff --git a/meta/recipes-gnome/packagegroups/packagegroup-toolset-native.bb 
b/meta/recipes-gnome/packagegroups/packagegroup-toolset-native.bb
index c88f82a..07d4f1d 100644
--- a/meta/recipes-gnome/packagegroups/packagegroup-toolset-native.bb
+++ b/meta/recipes-gnome/packagegroups/packagegroup-toolset-native.bb
@@ -213,7 +213,7 @@ DEPENDS = \
 tiff-native \
 tzcode-native \
 u-boot-mkimage-native \
-unfs-server-native \
+unfs3-native \
 unifdef-native \
 unzip-native \
 update-rc.d-native \
diff --git a/meta/site/ix86-common b/meta/site/ix86-common
index 21fcf38..5cd2ce2 100644
--- a/meta/site/ix86-common
+++ b/meta/site/ix86-common
@@ -216,7 +216,7 @@ moz_cv_size_of_JS_BYTES_PER_DOUBLE=8
 #ofono
 ac_cv_lib_c_signalfd=${ac_cv_lib_c_signalfd=yes}
 
-#unfs-server
+#unfs3
 nfsd_cv_broken_setfsuid=${nfsd_cv_broken_setfsuid=0}
 nfsd_cv_func_statfs=${nfsd_cv_func_statfs=statfs2_bsize}
 nfsd_cv_bsd_signals=${nfsd_cv_bsd_signals=yes}
diff --git a/meta/site/x86_64-linux b/meta/site/x86_64-linux
index b4434fe..573a907 100644
--- a/meta/site/x86_64-linux
+++ b/meta/site/x86_64-linux
@@ -98,7 +98,7 @@ ac_cv_alignof_CORBA_wchar=2
 lf_cv_sane_realloc=yes
 as_cv_unaligned_access=${as_cv_unaligned_access=yes}
 
-#unfs-server
+#unfs3
 nfsd_cv_broken_setfsuid=${nfsd_cv_broken_setfsuid=0}
 nfsd_cv_func_statfs=${nfsd_cv_func_statfs=statfs2_bsize}
 nfsd_cv_bsd_signals=${nfsd_cv_bsd_signals

[OE-core] [PATCH 0/6] Replace unfs-server with unfs3

[Jason Wessel]

[YOCTO #5639]

The goal of this patchset it to upgrade the user mode NFS server from
NFSv2 to NFSv3.  To do so requires the use of a new server.  The key
reason this update needs to occurs is that the old server uses the
inode from the file system for the NFS hash value.  The former is now
64 bits on modern kernels while the later is only 32 bits due to the
protocol and implementation with in the the unfs-server.  This causes
all sorts of problem with directory loops and duplicate file system
references that had been reported in the bugzilla.

That should also make it possible to extract a rootfs and run it as a
non-root user.

Example:

runqemu-extract-sdk 
tmp-eglibc/deploy/images/qemux86-64/core-image-minimal-qemux86-64.tar.bz2 rootfs

runqemu qemux86-64 `pwd`/rootfs nographic slirp


Cheers,
Jason.


Jason Wessel (5):
  unfs3: Add a NFSv3 user mode server for use with runqemu
  unfs3,unfs-server: Replace all instances of unfs-server with unfs3
  unfs-server: Remove unfs-server recipe and patches
  runqemu-export-rootfs: update for unfs3
  runqemu,runqemu-internal: Allow slirp for NFS and KVM use

Saul Wold (1):
  runqemu: Use the newer unfs3 for serving user space nfs

 meta/conf/distro/include/seperatebuilddir.inc  |4 +-
 meta/files/common-licenses/unfs3   |   24 +
 meta/recipes-core/meta/meta-ide-support.bb |2 +-
 .../nativesdk-packagegroup-sdk-host.bb |2 +-
 .../adt-installer/scripts/adt_installer_internal   |2 +-
 .../001-2.2b47-2.2b51.patch| 2349 
 .../unfs-server-2.1+2.2beta47/002-destdir.patch|   73 -
 .../unfs-server-2.1+2.2beta47/003-manpages.patch   |   33 -
 .../unfs-server-2.1+2.2beta47/004-strsignal.patch  |   53 -
 .../unfs-server-2.1+2.2beta47/005-sys-time.patch   |   34 -
 .../unfs-server-2.1+2.2beta47/006-reiserfs.patch   | 1277 ---
 .../unfs-server-2.1+2.2beta47/007-map.patch|   83 -
 .../unfs-server-2.1+2.2beta47/008-configure.patch  |   18 -
 .../unfs-server-2.1+2.2beta47/009-multirw.patch|   20 -
 .../unfs-server-2.1+2.2beta47/010-realpath.patch   |   35 -
 .../011-fno-strict-aliasing.patch  |   18 -
 .../unfs-server-2.1+2.2beta47/012-nostrip.patch|   18 -
 .../unfs-server-2.1+2.2beta47/013-mntpathlen.patch |   37 -
 .../014-uninitialized.patch|   17 -
 .../unfs-server-2.1+2.2beta47/015-setattr.patch|   31 -
 .../016-makefile.in.patch  |   19 -
 .../017-wrs-dynamic-rpc.patch  |  263 ---
 .../018-remove-tcp-wrappers.patch  |   25 -
 .../019-pid-before-fork.patch  |  130 --
 .../020-undefined-chmod-fix.patch  |   23 -
 .../unfs-server-2.1+2.2beta47/021-nolibwrap.patch  |   25 -
 .../022-add-close-on-exec-descriptors.patch|   66 -
 .../023-no-rpc-register.patch  |   34 -
 .../unfs-server/unfs-server_2.1+2.2beta47.bb   |   78 -
 .../unfs3/unfs3/alternate_rpc_ports.patch  |  156 ++
 .../unfs3/unfs3/fix_compile_warning.patch  |   16 +
 .../fix_pid_race_parent_writes_child_pid.patch |   47 +
 .../unfs3/unfs3/fix_warnings.patch |   51 +
 .../unfs3/force_4_byte_long_on_64_bit_host.patch   |   39 +
 .../unfs3/unfs3/relative_max_socket_path_len.patch |   70 +
 .../unfs3/unfs3/rename_fh_cache.patch  |   62 +
 .../unfs3/unfs3/tcp_no_delay.patch |   50 +
 .../unfs3/unfs3/unfs3_parallel_build.patch |   36 +
 meta/recipes-devtools/unfs3/unfs3_0.9.22.bb|   48 +
 .../packagegroups/packagegroup-toolset-native.bb   |2 +-
 meta/site/ix86-common  |2 +-
 meta/site/x86_64-linux |2 +-
 scripts/runqemu|   10 +-
 scripts/runqemu-export-rootfs  |   57 +-
 scripts/runqemu-internal   |   21 +-
 45 files changed, 632 insertions(+), 4830 deletions(-)
 create mode 100644 meta/files/common-licenses/unfs3
 delete mode 100644 
meta/recipes-devtools/unfs-server/unfs-server-2.1+2.2beta47/001-2.2b47-2.2b51.patch
 delete mode 100644 
meta/recipes-devtools/unfs-server/unfs-server-2.1+2.2beta47/002-destdir.patch
 delete mode 100644 
meta/recipes-devtools/unfs-server/unfs-server-2.1+2.2beta47/003-manpages.patch
 delete mode 100644 
meta/recipes-devtools/unfs-server/unfs-server-2.1+2.2beta47/004-strsignal.patch
 delete mode 100644 
meta/recipes-devtools/unfs-server/unfs-server-2.1+2.2beta47/005-sys-time.patch
 delete mode 100644 
meta/recipes-devtools/unfs-server/unfs-server-2.1+2.2beta47/006-reiserfs.patch
 delete mode 100644 
meta/recipes-devtools/unfs-server/unfs-server-2.1+2.2beta47/007-map.patch
 delete mode 100644 
meta/recipes-devtools/unfs-server/unfs-server-2.1+2.2beta47/008-configure.patch
 delete mode 100644

[OE-core] [PATCH 6/6] runqemu, runqemu-internal: Allow slirp for NFS and KVM use

[Jason Wessel]

The default slirp address for the NFS server is 10.0.2.2.  If not
using a tap interface this address must be used or the target system
cannot connect properly.  Also the ip=... kernel arguments need to be
set to dhcp when using slirp or the root NFS will not get setup
properly.

The call to cleanup() results in a routine which is not defined when
setting up the NFS because it is called before acquire() for the
locking of the tap interfaces, the solution being to simply not call
cleanup() that early.

When using slirp, kvm should not execute the vhost net checks because
the vhost net will not be configure or used.

Signed-off-by: Jason Wessel jason.wes...@windriver.com
---
 scripts/runqemu  |   10 ++
 scripts/runqemu-internal |   19 +++
 2 files changed, 17 insertions(+), 12 deletions(-)

diff --git a/scripts/runqemu b/scripts/runqemu
index dcb2931..9272b6f 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -279,10 +279,12 @@ if [ x$KVM_ENABLED = xyes ]; then
 exit 1;
 fi
 if [ ! -w /dev/vhost-net -o ! -r /dev/vhost-net ]; then
-echo You have no rights on /dev/vhost-net.
-echo Please change the ownership of this file as described at:
-echo $YOCTO_PARAVIRT_KVM_WIKI;
-exit 1;
+   if [ $SLIRP_ENABLED != yes ] ; then
+echo You have no rights on /dev/vhost-net.
+echo Please change the ownership of this file as described at:
+echo $YOCTO_PARAVIRT_KVM_WIKI;
+exit 1;
+   fi
 fi
 fi
 
diff --git a/scripts/runqemu-internal b/scripts/runqemu-internal
index cecb527..e3b0729 100755
--- a/scripts/runqemu-internal
+++ b/scripts/runqemu-internal
@@ -114,7 +114,7 @@ NFSRUNNING=false
 ORIG_STTY=$(stty -g)
 
 if [ $SLIRP_ENABLED = yes ]; then
-KERNEL_NETWORK_CMD=
+KERNEL_NETWORK_CMD=ip=dhcp
 QEMU_TAP_CMD=
 QEMU_UI_OPTIONS=-show-cursor -usb -usbdevice wacom-tablet
 if [ $KVM_ACTIVE = yes ]; then
@@ -301,9 +301,18 @@ if [ $FSTYPE != nfs -a $FSTYPE != vmdk -a ! -f 
$ROOTFS ]; then
 return 1
 fi
 
-if [ $FSTYPE = nfs ]; then
+if [ $NFS_SERVER =  ]; then
 NFS_SERVER=192.168.7.1
+if [ $SLIRP_ENABLED = yes ]; then
+   NFS_SERVER=10.0.2.2
+fi
+fi
+
+if [ $FSTYPE = nfs ]; then
 NFS_DIR=`echo $ROOTFS | sed 's/^[^:]*:\(.*\)/\1/'`
+if [ $NFS_INSTANCE =  ] ; then
+   NFS_INSTANCE=0
+fi
 MOUNTD_RPCPORT=$[ 2 + $NFS_INSTANCE ]
 NFSD_RPCPORT=$[ 1 + $NFS_INSTANCE ]
 NFSD_PORT=$[ 3049 + 2 * $NFS_INSTANCE ]
@@ -317,17 +326,11 @@ if [ $FSTYPE = nfs ]; then
 echo runqemu-export-rootfs restart $ROOTFS
 runqemu-export-rootfs restart $ROOTFS
 if [ $? != 0 ]; then
-cleanup
 return 1
 fi
 NFSRUNNING=true
 fi
 
-if [ $NFS_SERVER =  ]; then
-NFS_SERVER=192.168.7.1
-NFS_DIR=$ROOTFS
-fi
-
 if [ $MACHINE = qemuarm -o $MACHINE = qemuarmv6 -o $MACHINE = 
qemuarmv7 ]; then
 QEMU=qemu-system-arm
 MACHINE_SUBTYPE=versatilepb
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/6] runqemu: Use the newer unfs3 for serving user space nfs

[Jason Wessel]

From: Saul Wold s...@linux.intel.com

This new version correctly handles the 64bit ext3 / ext4 issues we
were seeing with the older unfs-server which did not handle 64bit file
systems correctly, producing the duplicate cookies.

[YOCTO #5639]

Signed-off-by: Saul Wold s...@linux.intel.com
---
 scripts/runqemu-export-rootfs |   53 -
 scripts/runqemu-internal  |2 +-
 2 files changed, 6 insertions(+), 49 deletions(-)

diff --git a/scripts/runqemu-export-rootfs b/scripts/runqemu-export-rootfs
index bbdaf5b..2d87996 100755
--- a/scripts/runqemu-export-rootfs
+++ b/scripts/runqemu-export-rootfs
@@ -88,8 +88,7 @@ MOUNT_PORT=$[ 3048 + 2 * $NFS_INSTANCE ]
 
 ## For debugging you would additionally add
 ## --debug all
-MOUNTD_OPTS=--allow-non-root --mount-pid $MOUNTPID -f $EXPORTS --rmtab $RMTAB 
--prog $NFS_MOUNTPROG -r -P $MOUNT_PORT
-NFSD_OPTS=--allow-non-root --nfs-pid $NFSPID -f $EXPORTS --prog $NFS_NFSPROG 
-P $NFS_PORT -r
+UNFSD_OPTS=-p -N -i $NFSPID -e $EXPORTS -x $NFS_NFSPROG -n $NFS_PORT -y 
$NFS_MOUNTPROG -m $MOUNT_PORT
 
 # Setup the exports file
 if [ $1 = start ]; then
@@ -115,53 +114,18 @@ case $1 in
exit 1
fi
 
-   echo Starting User Mode rpc.mountd
-   echo   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/rpc.mountd 
$MOUNTD_OPTS
-   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/rpc.mountd 
$MOUNTD_OPTS
-   if [ ! $? = 0 ]; then
-   echo =
-   echo Error starting MOUNTD
-   echo =
-   if [ ! x$RPCBIND_RUNNING = x ] ; then
-   echo   If you see an error above that says:
-   echo RPC: Authentication error; why = Client 
credential too weak
-   echo   You need to add the -i option when running 
rpcbind
-   echo ===
-   echo For recent Fedora/RedHat hosts:
-   echo Add RPCBIND_ARGS=-i to /etc/sysconfig/rpcbind
-   echo   or
-   echo Add RPCBIND_OPTIONS=-i to /etc/sysconfig/rpcbind
-   echo Then run as root: /etc/init.d/rpcbind restart
-   echo ===
-   echo For recent Debian/Ubuntu hosts:
-   echo Add OPTIONS=\-i -w\  to /etc/default/rpcbind
-   echo sudo service portmap restart
-   fi
-
-   exit 1
-   fi
-
echo Starting User Mode nfsd
-   echo   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/rpc.nfsd 
$NFSD_OPTS
-   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/rpc.nfsd $NFSD_OPTS
+   echo   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/unfsd 
$UNFSD_OPTS
+   $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/sbin/unfsd $UNFSD_OPTS
if [ ! $? = 0 ]; then
echo Error starting nfsd
exit 1
fi
-   # Check to make sure everything started ok.
-   if [ ! -f $MOUNTPID ]; then
-   echo rpc.mountd did not start correctly
-   exit 1
-   fi
+Check to make sure everything started ok.
if [ ! -f $NFSPID ]; then
echo rpc.nfsd did not start correctly
exit 1
fi
-   ps -fp `cat $MOUNTPID`  /dev/null 2 /dev/null
-   if [ ! $? = 0 ]; then
-   echo rpc.mountd did not start correctly
-   exit 1
-   fi
ps -fp `cat $NFSPID`  /dev/null 2 /dev/null
if [ ! $? = 0 ]; then
echo rpc.nfsd did not start correctly
@@ -169,16 +133,9 @@ case $1 in
fi
echo  
echo On your target please remember to add the following options for 
NFS
-   echo 
nfsroot=IP_ADDRESS:$NFS_EXPORT_DIR,nfsvers=2,mountprog=$NFS_MOUNTPROG,nfsprog=$NFS_NFSPROG,udp
+   echo 
nfsroot=IP_ADDRESS:$NFS_EXPORT_DIR,nfsvers=3,port=$NFSD_PORT,mountprog=$MOUNTD_RPCPORT,nfsprog=$NFSD_RPCPORT,udp,mountport=$MOUNTD_PORT
;;
   stop)
-   if [ -f $MOUNTPID ]; then
-   echo Stopping rpc.mountd
-   kill `cat $MOUNTPID`
-   rm -f $MOUNTPID
-   else
-   echo No PID file, not stopping rpc.mountd
-   fi
if [ -f $NFSPID ]; then
echo Stopping rpc.nfsd
kill `cat $NFSPID`
diff --git a/scripts/runqemu-internal b/scripts/runqemu-internal
index 284b88c..cecb527 100755
--- a/scripts/runqemu-internal
+++ b/scripts/runqemu-internal
@@ -308,7 +308,7 @@ if [ $FSTYPE = nfs ]; then
 NFSD_RPCPORT=$[ 1 + $NFS_INSTANCE ]
 NFSD_PORT=$[ 3049 + 2 * $NFS_INSTANCE ]
 MOUNTD_PORT=$[ 3048 + 2 * $NFS_INSTANCE ]
-
UNFS_OPTS=nfsvers=2,mountprog=$MOUNTD_RPCPORT,nfsprog=$NFSD_RPCPORT,udp,port=$NFSD_PORT,mountport=$MOUNTD_PORT
+