Re: [OE-core] [PATCH] wic: Add include-dir option

[Martin Hundebøll]

Hi Armin,

On 27/02/2020 05.26, Armin Kuster wrote:

This option allows for the inclusion of a single directory
for a partition.


I am unsure how this is used in a .wks file.

From the code it looks similar to the include-path option, but relative 
to the rootfs source dir instead of the wic working dir?


But the include-dir value is also used in the destination path?

Thanks,
Martin


Signed-off-by: Armin Kuster 
---
  scripts/lib/wic/help.py  |  3 +++
  scripts/lib/wic/ksparser.py  |  1 +
  scripts/lib/wic/partition.py |  1 +
  scripts/lib/wic/plugins/source/rootfs.py | 10 --
  4 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index 4d342fcf05..517f68e11e 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -979,6 +979,9 @@ DESCRIPTION
   copies. This option only has an effect with the 
rootfs
   source plugin.
  
+ --include-dir: This option is specific to wic. It adds the contents

+of the given directory to the resulting partition.
+
   --extra-space: This option is specific to wic. It adds extra
  space after the space filled by the content
  of the partition. The final size can go
diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index 650b976223..b8abc33c87 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -138,6 +138,7 @@ class KickStart():
  part.add_argument('--align', type=int)
  part.add_argument('--exclude-path', nargs='+')
  part.add_argument('--include-path', nargs='+')
+part.add_argument('--include-dir')
  part.add_argument("--extra-space", type=sizetype)
  part.add_argument('--fsoptions', dest='fsopts')
  part.add_argument('--fstype', default='vfat',
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 2d95f78439..0b735fffd9 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -31,6 +31,7 @@ class Partition():
  self.extra_space = args.extra_space
  self.exclude_path = args.exclude_path
  self.include_path = args.include_path
+self.include_dir = args.include_dir
  self.fsopts = args.fsopts
  self.fstype = args.fstype
  self.label = args.label
diff --git a/scripts/lib/wic/plugins/source/rootfs.py 
b/scripts/lib/wic/plugins/source/rootfs.py
index 705aeb5563..d1c59cab8a 100644
--- a/scripts/lib/wic/plugins/source/rootfs.py
+++ b/scripts/lib/wic/plugins/source/rootfs.py
@@ -71,7 +71,7 @@ class RootfsPlugin(SourcePlugin):
  
  new_rootfs = None

  # Handle excluded paths.
-if part.exclude_path or part.include_path:
+if part.exclude_path or part.include_path or part.include_dir:
  # We need a new rootfs directory we can delete files from. Copy to
  # workdir.
  new_rootfs = os.path.realpath(os.path.join(cr_workdir, "rootfs%d" 
% part.lineno))
@@ -79,7 +79,13 @@ class RootfsPlugin(SourcePlugin):
  if os.path.lexists(new_rootfs):
  shutil.rmtree(os.path.join(new_rootfs))
  
-copyhardlinktree(part.rootfs_dir, new_rootfs)

+if part.include_dir:
+src = os.path.realpath(os.path.join(part.rootfs_dir, 
part.include_dir))
+dst = os.path.realpath(os.path.join(new_rootfs, 
part.include_dir))
+copyhardlinktree(src, dst)
+
+else:
+copyhardlinktree(part.rootfs_dir, new_rootfs)
  
  for path in part.include_path or []:

  copyhardlinktree(path, new_rootfs)


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] gstreamer: Fix reproducibility issue around libcap

[Martin Hundebøll]

Hi,

On 19/02/2020 16.23, Richard Purdie wrote:

Add an option to avoid builds depending on the presence of setcap
from the host system.

Signed-off-by: Richard Purdie 
---
  .../gstreamer/gstreamer1.0/capfix.patch   | 37 +++
  .../gstreamer/gstreamer1.0_1.16.1.bb  |  2 +
  2 files changed, 39 insertions(+)
  create mode 100644 meta/recipes-multimedia/gstreamer/gstreamer1.0/capfix.patch

diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0/capfix.patch 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0/capfix.patch
new file mode 100644
index 000..7ca3d5ad4a6
--- /dev/null
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0/capfix.patch
@@ -0,0 +1,37 @@
+Currently gstreamer configuration depends on whether setcap is found on the 
host
+system. Turn this into a configure option to make builds deterinistic.
+
+RP 2020/2/19
+Signed-off-by: Richard Purdie 
+Upstream-Status: Pending
+
+Index: gstreamer-1.16.1/libs/gst/helpers/meson.build
+===
+--- gstreamer-1.16.1.orig/libs/gst/helpers/meson.build
 gstreamer-1.16.1/libs/gst/helpers/meson.build
+@@ -73,7 +73,12 @@ if have_ptp
+ endif
+   endif
+
+-  setcap = find_program('setcap', '/usr/sbin/setcap', '/sbin/setcap', 
required : false)
++  setcap_feature = get_option('setcap')
++  if setcap_feature.disabled()
++setcap = find_program('dontexist', required : false)
++  else
++setcap = find_program('setcap', '/usr/sbin/setcap', '/sbin/setcap', 
required : false)
++  endif


I think this can be simplified by using get_option() directly for the 
"required" argument:


setcap = find_program('setcap', '/usr/sbin/setcap', '/sbin/setcap', 
required : get_option('setcap'))


// Martin

+
+   # user/group to change to in gst-ptp-helper
+   ptp_helper_setuid_user = get_option('ptp-helper-setuid-user')
+Index: gstreamer-1.16.1/meson_options.txt
+===
+--- gstreamer-1.16.1.orig/meson_options.txt
 gstreamer-1.16.1/meson_options.txt
+@@ -26,6 +26,7 @@ option('libunwind', type : 'feature', va
+ option('libdw', type : 'feature', value : 'auto', description : 'Use libdw to 
generate better backtraces from libunwind')
+ option('dbghelp', type : 'feature', value : 'auto', description : 'Use 
dbghelp to generate backtraces')
+ option('bash-completion', type : 'feature', value : 'auto', description : 
'Install bash completion files')
++option('setcap', type : 'feature', value : 'auto', description : 'Use setcap')
+
+ # Common feature options
+ option('examples', type : 'feature', value : 'auto', yield : true)
diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0_1.16.1.bb 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0_1.16.1.bb
index 6b8a5a0eb01..68f5ca649fe 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0_1.16.1.bb
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0_1.16.1.bb
@@ -21,6 +21,7 @@ SRC_URI = " \
  file://0002-meson-build-gir-even-when-cross-compiling-if-introsp.patch \
  file://0003-meson-Add-valgrind-feature.patch \
  file://0004-meson-Add-option-for-installed-tests.patch \
+file://capfix.patch \
  "
  SRC_URI[md5sum] = "c505fb818b36988daaa846e9e63eabe8"
  SRC_URI[sha256sum] = 
"02211c3447c4daa55919c5c0f43a82a6fbb51740d57fc3af0639d46f1cf4377d"
@@ -39,6 +40,7 @@ PACKAGECONFIG[unwind] = 
"-Dlibunwind=enabled,-Dlibunwind=disabled,libunwind"
  PACKAGECONFIG[dw] = "-Dlibdw=enabled,-Dlibdw=disabled,elfutils"
  PACKAGECONFIG[bash-completion] = 
"-Dbash-completion=enabled,-Dbash-completion=disabled,bash-completion"
  PACKAGECONFIG[tools] = "-Dtools=enabled,-Dtools=disabled"
+PACKAGECONFIG[setcap] = "-Dsetcap=enabled,-Dsetcap=disabled,libcap"
  
  # TODO: put this in a gettext.bbclass patch

  def gettext_oemeson(d):


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] sstate.bbclass: fix issue while handling long sstate filenames

[Martin Hundebøll]

On 06/02/2020 01.57, Jaewon Lee wrote:

When moving to python3, divison using '/' now returns float instead of
an integer. In upstream commit b8025e972081b70960ffcbcbe43a7118041556a1
sstate filenames longer than the limit are changed to just include
necessary info + 3 fields just for information. The space left over
after the necessary info is divided into 3 for each of the fields.
Casting the outcome of that division to int to solve the following error
message:

avail = (254 - len(hash + "_" + taskname + extension) -
len(components[0]) - len(components[1]) - len(components[5]) -
len(components[6]) - 7) / 3
 >components[2] = components[2][:avail]
  components[3] = components[3][:avail]
TypeError: slice indices must be integers or None or have an __index__
method

Signed-off-by: Jaewon Lee 
Signed-off-by: Mark Hatle 
---
  meta/classes/sstate.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index faa6470..53a6d06 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -20,7 +20,7 @@ def generate_sstatefn(spec, hash, taskname, siginfo, d):
  components = spec.split(":")
  # Fields 0,5,6 are mandatory, 1 is most useful, 2,3,4 are just for 
information
  # 7 is for the separators
-avail = (254 - len(hash + "_" + taskname + extension) - 
len(components[0]) - len(components[1]) - len(components[5]) - len(components[6]) - 7) / 3
+avail = int((254 - len(hash + "_" + taskname + extension) - 
len(components[0]) - len(components[1]) - len(components[5]) - len(components[6]) - 7) / 
3)


Python has a nifty // operator that resembles integer division from C:

>>> 3 / 2

1.5

>>> 3 // 2

1


// Martin


  components[2] = components[2][:avail]
  components[3] = components[3][:avail]
  components[4] = components[4][:avail]


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] package: disable libseccomp for find

[Martin Hundebøll]

Distros where libseccomp is enabled for the find util segfaults in
do_package:

ERROR: base-files-3.0.14-r89 do_package: Fatal errors occurred in subprocesses:
Command '['file', '-b', 
'build/tmp-glibc/work/iwg26-oe-linux-gnueabi/base-files/3.0.14-r89/package/etc/skel/.bashrc']'
 died with .: Traceback (most recent call last):
  File "poky/meta/lib/oe/utils.py", line 272, in run
ret = self._target(*self._args, **self._kwargs)
  File "poky/meta/lib/oe/package.py", line 70, in is_elf
result = subprocess.check_output(["file", "-b", path], 
stderr=subprocess.STDOUT).decode("utf-8")
  File "/usr/lib/python3.8/subprocess.py", line 411, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['file', '-b', 
'build/tmp-glibc/work/iwg26-oe-linux-gnueabi/base-files/3.0.14-r89/package/etc/skel/.bashrc']'
 died with .

Signed-off-by: Martin Hundebøll 
---
 meta/lib/oe/package.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index b8585d4253..0c22307c46 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -71,7 +71,7 @@ def is_kernel_module_signed(path):
 # 16 - kernel module
 def is_elf(path):
 exec_type = 0
-result = subprocess.check_output(["file", "-b", path], 
stderr=subprocess.STDOUT).decode("utf-8")
+result = subprocess.check_output(["file", "-S", "-b", path], 
stderr=subprocess.STDOUT).decode("utf-8")
 
 if "ELF" in result:
 exec_type |= 1
-- 
2.25.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] file: explicitly disable seccomp

[Martin Hundebøll]

On 18/10/2019 23.28, Richard Purdie wrote:

On Fri, 2019-10-18 at 18:03 +0530, Khem Raj wrote:



On Fri, Oct 18, 2019 at 4:58 PM Ross Burton 
wrote:

file will automatically enable seccomp if the seccomp headers are
available, but
the build will fail on Opensuse Tumbleweed because the include
paths are wrong.

Enabling seccomp is a bad idea because it interacts badly with
pseudo (causing
build failures), so explicitly and globally disable seccomp.

Signed-off-by: Ross Burton 
---
  meta/recipes-devtools/file/file_5.37.bb | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/meta/recipes-devtools/file/file_5.37.bb
b/meta/recipes-devtools/file/file_5.37.bb
index a840dbc012b..c53a120b840 100644
--- a/meta/recipes-devtools/file/file_5.37.bb
+++ b/meta/recipes-devtools/file/file_5.37.bb
@@ -21,6 +21,8 @@ S = "${WORKDIR}/git"

  inherit autotools update-alternatives

+EXTRA_OECONF += "--disable-libseccomp"
+


On host it makes sense but how about for target recipe ? We do have
libseccomp support
Perhaps it’s best to disable it for native and native sdk alone ?


Given the problem reports in the media and the decisions made by most
other distros I think disabling this is probably safest until its been
make more usable (no other distro is shipping this enabled now).


Arch Linux enabled libseccomp as of today:

https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/file=a52fa0a37bad1f4ab67a934a4e0025e70aa1fd0e

This breaks the use of `file` in do_package():

ERROR: base-files-3.0.14-r89 do_package: Fatal errors occurred in 
subprocesses:


Command '['file', '-b', 
'/home/martin/clients/connectedcars/yocto/build/tmp-glibc/work/iwg26-oe-linux-gnueabi/base-files/3.0.14-r89/package/etc/skel/.profile']' 
died with .: Traceback (most recent call last):


  File 
"/home/martin/clients/connectedcars/yocto/poky/meta/lib/oe/utils.py", 
line 272, in run


ret = self._target(*self._args, **self._kwargs)

  File 
"/home/martin/clients/connectedcars/yocto/poky/meta/lib/oe/package.py", 
line 70, in is_elf


result = subprocess.check_output(["file", "-b", path], 
stderr=subprocess.STDOUT).decode("utf-8")


  File "/usr/lib/python3.8/subprocess.py", line 411, in check_output

return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,

  File "/usr/lib/python3.8/subprocess.py", line 512, in run 



raise CalledProcessError(retcode, process.args,

subprocess.CalledProcessError: Command '['file', '-b', 
'/home/martin/clients/connectedcars/yocto/build/tmp-glibc/work/iwg26-oe-linux-gnueabi/base-files/3.0.14-r89/package/etc/skel/.profile']' 
died with .


Disabling libseccomp sandboxing at runtime fixes it for me:

diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py

index efd36b3758..bc46d7c27c 100644

--- a/meta/lib/oe/package.py

+++ b/meta/lib/oe/package.py

@@ -67,7 +67,7 @@ def is_kernel_module_signed(path):

 # 16 - kernel module

 def is_elf(path):

 exec_type = 0

-result = subprocess.check_output(["file", "-b", path], 
stderr=subprocess.STDOUT).decode("utf-8")


+result = subprocess.check_output(["file", "-S", "-b", path], 
stderr=subprocess.STDOUT).decode("utf-8")




 if "ELF" in result:

 exec_type |= 1


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 6/9] linux-yocto-dev: bump to v5.2-rc

[Martin Hundebøll]

On 10/07/2019 16.28, Richard Purdie wrote:

On Wed, 2019-07-10 at 15:27 +0100, richard.pur...@linuxfoundation.org
wrote:

On Wed, 2019-07-10 at 10:21 -0400, Bruce Ashfield wrote:

On Wed, Jul 10, 2019 at 10:15 AM Martin Hundebøll <
mar...@geanix.com>


Nope, I haven't seen that. But then again, all my machines have
realpath available.

As you hinted, building all of coreutils as a dependency for the
kernel is a bit heavy. We could make
realpath a host requirement,  or otherwise, split out realpath (but
you'd still need to build coreutils, so
that really doesn't help).

I'd lean towards just making it a host requirement .. but I'm sure
Richard can advise better than me
on the pro's and con's of the options.


realpath is in the standard HOSTTOOLS in bitbake.conf so its already
required?


http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/conf/bitbake.conf?id=12e1a0de18fc05e68be8e6f1688c1499e37b076b


Now I see; I am building on thud...

Armin, can you backport commit 12e1a0de18fc05e68be8e6f1688c1499e37b076b 
to active branches?


Thanks,
Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 6/9] linux-yocto-dev: bump to v5.2-rc

[Martin Hundebøll]

Hi Bruce,

On 30/05/2019 14.44, bruce.ashfi...@gmail.com wrote:

From: Bruce Ashfield 

Signed-off-by: Bruce Ashfield 
---
  meta/recipes-kernel/linux/linux-yocto-dev.bb | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb 
b/meta/recipes-kernel/linux/linux-yocto-dev.bb
index ae8c343008..3a055c12ad 100644
--- a/meta/recipes-kernel/linux/linux-yocto-dev.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb
@@ -30,7 +30,7 @@ SRC_URI = 
"git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name
  SRCREV_machine ?= '${@oe.utils.conditional("PREFERRED_PROVIDER_virtual/kernel", 
"linux-yocto-dev", "${AUTOREV}", "29594404d7fe73cd80eaa4ee8c43dcc53970c60e", d)}'
  SRCREV_meta ?= '${@oe.utils.conditional("PREFERRED_PROVIDER_virtual/kernel", "linux-yocto-dev", 
"${AUTOREV}", "29594404d7fe73cd80eaa4ee8c43dcc53970c60e", d)}'
  
-LINUX_VERSION ?= "5.0-rc+"

+LINUX_VERSION ?= "5.2-rc+"
  LINUX_VERSION_EXTENSION ?= "-yoctodev-${LINUX_KERNEL_TYPE}"
  PV = "${LINUX_VERSION}+git${SRCPV}"


My own kernel recipe (more or less mainline) failed to build when I 
bumped to v5.2. It fails to generate the out-of-tree Makefile due to a 
missing dependency on `realpath`[1].


The quick-fix is to add coreutils-native to DEPENDS, but I am not sure 
if this is what we want.


Have you seen the same?

// Martin

[1] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=48b5ffd1268788afb01525e71e864e901e9aa070

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] The state of reproducible Builds

[Martin Hundebøll]

Hi,

On 02/07/2019 16.13, Joshua Watt wrote:

For detecting malicous binaries not built from the claimed sources 1. is
sufficient. For distributions like Debian that build natively this is
even the only option available since the host compiler is used.

Doing 2. would of course be more desirable, but it can also be done in
a second step after all issues related to building on exactly the same
host have been sorted out.


I think there are also other use cases for #2 besides detecting 
malicious binaries/source code, such as hash equivalence, or even being 
able use sstate when making a reproducible build. You are correct that 
this can be done in a second step, but I think that everyone needs to be 
aware of the limitations that will present when #2 is not present (the 
main one being that you probably can't make a reproducible build if you 
use sstate).


Our use case for reproducible builds is to limit delta update sizes. 
I.e. updating one package shouldn;t change the binary output from other 
independent packages.


// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [V3][Oe-core][PATCH] sysstat: Add PACKAGECONFIG[cron] for '--enable-install-cron' option

[Martin Hundebøll]

Hi,

On 15/05/2019 08.42, Haiqing Bai wrote:

If 'cron' is added into PACKAGECONFIG, for sysvinit, the related
cron scripts will be installed; for systemd, the services of
data collect and summary will be installed.

Signed-off-by: Haiqing Bai 
---
  meta/recipes-extended/sysstat/sysstat.inc | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-extended/sysstat/sysstat.inc 
b/meta/recipes-extended/sysstat/sysstat.inc
index 0bc7e14d36..79555e6cf3 100644
--- a/meta/recipes-extended/sysstat/sysstat.inc
+++ b/meta/recipes-extended/sysstat/sysstat.inc
@@ -19,6 +19,7 @@ inherit autotools-brokensep gettext systemd
  
  PACKAGECONFIG ??= ""

  PACKAGECONFIG[lm-sensors] = 
"--enable-sensors,--disable-sensors,lmsensors,lmsensors-libsensors"
+PACKAGECONFIG[cron] = "--enable-install-cron 
--enable-copy-only,--disable-install-cron --disable-copy-only"
  
  EXTRA_OECONF += "--disable-stripping"
  
@@ -59,7 +60,6 @@ pkg_postinst_${PN} () {

  fi
  }
  
-

-FILES_${PN} += "${libdir}/sa"
+FILES_${PN} += "${libdir}/sa ${systemd_unitdir}/system/*"


You can use "${systemd_system_unitdir}" instead of 
"${systemd_unitdir}/system/". And also no need to append the asterisk.


// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] distutils: Tidy and simplify for readability

[Martin Hundebøll]

On 14/05/2019 08.16, Douglas Royds via Openembedded-core wrote:
Can anyone explain why I appear on patchwork as "Andrii Bordunov via 
Openembedded-core"?


Yes.

Because the DMARC policy for your domain is "quarantine":

  % dig _dmarc.taitradio.com TXT | grep -oE 'p=(reject|quarantine|none)'
  p=quarantine

If the mailing list simply forwards your mails, they will fail 
validation, and thus be held back by the receiving mail servers. To 
avoid this, Mailman changes the "From:" header to "Douglas Royds via 
Openembedded-core ".


Now the email address seen by patchwork is the same for everyone with 
either "reject" or "quarantine" as DMARC policy. I guess patchwork has 
that address associated with Andrii (maybe he came first).


More info from mailman: https://wiki.list.org/DEV/DMARC

--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix ApS
https://geanix.com
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] systemd: enable portabled by default and install utilities in systemd-container

[Martin Hundebøll]

Hi Luca,

On 10/05/2019 12.45, Luca Boccassi wrote:

On Fri, 2019-05-10 at 07:23 +0200, Martin Hundebøll wrote:

Hi Luca,

On 09/05/2019 23.40,
luca.bocca...@gmail.com
  wrote:

From: Luca Boccassi <
luca.bocca...@microsoft.com




portable services have been declared production ready with v241, so
enable
by default and install the files in the container package.


The systemd portables feature is not really related to
containers/VM's,
so I think it should stay in the primary package; or alternatively in
a
separate package.

// Martin


Hello Martin,

IMHO if VM tools and containers tool can go together in this package,
then portable services utilities can too - they are at different
positions on the same scale after all :-)

But I've got no strong opinion, so I'll send a v2 with them moved to
the main package.


I am planning to use portable services instead of going full container, 
so I would prefer not having the container/vm utils on target :)


// Martin


Signed-off-by: Luca Boccassi <
luca.bocca...@microsoft.com



---
   meta/recipes-core/systemd/systemd_242.bb | 10 ++
   1 file changed, 10 insertions(+)

diff --git a/meta/recipes-core/systemd/systemd_242.bb
b/meta/recipes-core/systemd/systemd_242.bb
index 73e03c7a77..83b00ba785 100644
--- a/meta/recipes-core/systemd/systemd_242.bb
+++ b/meta/recipes-core/systemd/systemd_242.bb
@@ -81,6 +81,7 @@ PACKAGECONFIG ??= " \
   nss \
   nss-mymachines \
   nss-resolve \
+portabled \
   quotacheck \
   randomseed \
   resolved \
@@ -403,8 +404,10 @@ SYSTEMD_SERVICE_${PN}-remote = "systemd-
journal-remote.socket"
   
   FILES_${PN}-container = "${sysconfdir}/dbus-

1/system.d/org.freedesktop.import1.conf \
${sysconfdir}/dbus-
1/system.d/org.freedesktop.machine1.conf \
+ ${sysconfdir}/dbus-
1/system.d/org.freedesktop.portable1.conf \
${sysconfdir}/systemd/system/multi-
user.target.wants/machines.target \
${base_bindir}/machinectl \
+ ${base_bindir}/portablectl \
${bindir}/systemd-nspawn \
${nonarch_libdir}/systemd/import-
pubring.gpg \
${systemd_system_unitdir}/busnames.targe
t.wants/org.freedesktop.import1.busname \
@@ -418,21 +421,28 @@ FILES_${PN}-container = "${sysconfdir}/dbus-
1/system.d/org.freedesktop.import1.c
${systemd_system_unitdir}/org.freedeskto
p.machine1.busname \
${systemd_system_unitdir}/systemd-
importd.service \
${systemd_system_unitdir}/systemd-
machined.service \
+ ${systemd_system_unitdir}/systemd-
portabled.service \
${systemd_system_unitdir}/dbus-
org.freedesktop.machine1.service \
+ ${systemd_system_unitdir}/dbus-
org.freedesktop.portable1.service \
${systemd_system_unitdir}/var-lib-
machines.mount \
${rootlibexecdir}/systemd/systemd-import
\
${rootlibexecdir}/systemd/systemd-
importd \
${rootlibexecdir}/systemd/systemd-
machined \
+ ${rootlibexecdir}/systemd/systemd-
portabled \
${rootlibexecdir}/systemd/systemd-pull \
+ ${exec_prefix}/lib/tmpfiles.d/portables.c
onf \
${exec_prefix}/lib/tmpfiles.d/systemd-
nspawn.conf \
${systemd_system_unitdir}/systemd-
nspawn@.service \
${libdir}/libnss_mymachines.so.2 \
${datadir}/dbus-1/system-
services/org.freedesktop.import1.service \
${datadir}/dbus-1/system-
services/org.freedesktop.machine1.service \
+ ${datadir}/dbus-1/system-
services/org.freedesktop.portable1.service \
${datadir}/dbus-
1/system.d/org.freedesktop.import1.conf \
${datadir}/dbus-
1/system.d/org.freedesktop.machine1.conf \
+ ${datadir}/dbus-
1/system.d/org.freedesktop.portable1.conf \
${datadir}/polkit-
1/actions/org.freedesktop.import1.policy \
${datadir}/polkit-
1/actions/org.freedesktop.machine1.policy \
+ ${datadir}/polkit-
1/actions/org.freedesktop.portable1.policy \
   "
   
   RRECOMMENDS_${PN}-container += "\







--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix ApS
https://geanix.com
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] systemd: enable portabled by default and install utilities in systemd-container

[Martin Hundebøll]

Hi Luca,

On 09/05/2019 23.40, luca.bocca...@gmail.com wrote:

From: Luca Boccassi 

portable services have been declared production ready with v241, so enable
by default and install the files in the container package.


The systemd portables feature is not really related to containers/VM's, 
so I think it should stay in the primary package; or alternatively in a 
separate package.


// Martin


Signed-off-by: Luca Boccassi 
---
  meta/recipes-core/systemd/systemd_242.bb | 10 ++
  1 file changed, 10 insertions(+)

diff --git a/meta/recipes-core/systemd/systemd_242.bb 
b/meta/recipes-core/systemd/systemd_242.bb
index 73e03c7a77..83b00ba785 100644
--- a/meta/recipes-core/systemd/systemd_242.bb
+++ b/meta/recipes-core/systemd/systemd_242.bb
@@ -81,6 +81,7 @@ PACKAGECONFIG ??= " \
  nss \
  nss-mymachines \
  nss-resolve \
+portabled \
  quotacheck \
  randomseed \
  resolved \
@@ -403,8 +404,10 @@ SYSTEMD_SERVICE_${PN}-remote = 
"systemd-journal-remote.socket"
  
  FILES_${PN}-container = "${sysconfdir}/dbus-1/system.d/org.freedesktop.import1.conf \

   
${sysconfdir}/dbus-1/system.d/org.freedesktop.machine1.conf \
+ 
${sysconfdir}/dbus-1/system.d/org.freedesktop.portable1.conf \
   
${sysconfdir}/systemd/system/multi-user.target.wants/machines.target \
   ${base_bindir}/machinectl \
+ ${base_bindir}/portablectl \
   ${bindir}/systemd-nspawn \
   ${nonarch_libdir}/systemd/import-pubring.gpg \
   
${systemd_system_unitdir}/busnames.target.wants/org.freedesktop.import1.busname 
\
@@ -418,21 +421,28 @@ FILES_${PN}-container = 
"${sysconfdir}/dbus-1/system.d/org.freedesktop.import1.c
   
${systemd_system_unitdir}/org.freedesktop.machine1.busname \
   ${systemd_system_unitdir}/systemd-importd.service \
   ${systemd_system_unitdir}/systemd-machined.service \
+ ${systemd_system_unitdir}/systemd-portabled.service \
   
${systemd_system_unitdir}/dbus-org.freedesktop.machine1.service \
+ 
${systemd_system_unitdir}/dbus-org.freedesktop.portable1.service \
   ${systemd_system_unitdir}/var-lib-machines.mount \
   ${rootlibexecdir}/systemd/systemd-import \
   ${rootlibexecdir}/systemd/systemd-importd \
   ${rootlibexecdir}/systemd/systemd-machined \
+ ${rootlibexecdir}/systemd/systemd-portabled \
   ${rootlibexecdir}/systemd/systemd-pull \
+ ${exec_prefix}/lib/tmpfiles.d/portables.conf \
   ${exec_prefix}/lib/tmpfiles.d/systemd-nspawn.conf \
   ${systemd_system_unitdir}/systemd-nspawn@.service \
   ${libdir}/libnss_mymachines.so.2 \
   
${datadir}/dbus-1/system-services/org.freedesktop.import1.service \
   
${datadir}/dbus-1/system-services/org.freedesktop.machine1.service \
+ 
${datadir}/dbus-1/system-services/org.freedesktop.portable1.service \
   
${datadir}/dbus-1/system.d/org.freedesktop.import1.conf \
   
${datadir}/dbus-1/system.d/org.freedesktop.machine1.conf \
+ 
${datadir}/dbus-1/system.d/org.freedesktop.portable1.conf \
   
${datadir}/polkit-1/actions/org.freedesktop.import1.policy \
   
${datadir}/polkit-1/actions/org.freedesktop.machine1.policy \
+ 
${datadir}/polkit-1/actions/org.freedesktop.portable1.policy \
  "
  
  RRECOMMENDS_${PN}-container += "\




--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix ApS
https://geanix.com
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v4] linux-libc-headers: Fix build failure by using fixed temporary file instead of pipe

[Martin Hundebøll]

 recipe and 
only
patch if required ?


I have not tried but we can prepend the do_patch of linux-libc-headers to
check the host binutils' version and modify the SRCURI there depending on
the result.

But the users may never want to upgrade their host binutils and thus need this
workaround forever. So even we know what the version is on the host and
warn them to upgrade, it seems we still don't know when we can drop the
workaround. I'm not sure the policy. Can we drop it when all supported
distributions have included the fix?


That is my suggestion (that we have a way to eventually drop the
patch), so either
we have to manually keep track of the host binutils versions and
revisit the patch
each release, we have some sort of check and conditional apply or we submit the
change to the upstream binutils -stable in the hopes that any distro
still tracking
v2.3x would bump to a 2.3x version with the change, and then we'd no longer have
to carry it anymore ... (but as you say in the commit it isn't a
problem in other
binutils, just this version, so there's likely no way to submit it
upstream and have
it do any good .. the problem is solved already).

That being said, I was more interested in hearing if you had looked into
conditional application, or if you had considered how we could
eventually drop the
patch, since I agree that as it currently stands .. we may have to
carry it for a long
time. We've had that discussion now, so we are covered.

Since the patch is simple enough, I'm not insisting on the complexity
of a dynamic
check, and I can't see another way to fix it.

So if you could double check the link to the upstream commit, I can
ack the patch
and it can go in as-is.

Cheers,

Bruce



Thanks,
Zhe



Bruce



 +
 +Signed-off-by: He Zhe mailto:zhe...@windriver.com>>
 +---
 + scripts/gcc-goto.sh | 7 ++-
 + 1 file changed, 6 insertions(+), 1 deletion(-)
 +
 +diff --git a/scripts/gcc-goto.sh b/scripts/gcc-goto.sh
 +index 083c526..0aaf1b4 100755
 +--- a/scripts/gcc-goto.sh
  b/scripts/gcc-goto.sh
 +@@ -3,7 +3,9 @@
 + # Test for gcc 'asm goto' support
 + # Copyright (C) 2010, Jason Baron mailto:jba...@redhat.com>>
 +
 +-cat << "END" | $@ -x c - -c -o /dev/null >/dev/null 2>&1 && echo "y"
 ++TMPFILE=`mktemp -p .`
 ++
 ++cat << "END" > ${TMPFILE}
 + int main(void)
 + {
 + #if defined(__arm__) || defined(__aarch64__)
 +@@ -20,3 +22,6 @@ entry:
 +   return 0;
 + }
 + END
 ++
 ++$@ -x c ${TMPFILE} -c -o /dev/null && echo "y"
 ++rm ${TMPFILE}
 +--
 +2.7.4
 +
 diff --git a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.18.bb 
<http://linux-libc-headers_4.18.bb> 
b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.18.bb 
<http://linux-libc-headers_4.18.bb>
 index eb7bee7..00420aa 100644
 --- a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.18.bb 
<http://linux-libc-headers_4.18.bb>
 +++ b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.18.bb 
<http://linux-libc-headers_4.18.bb>
 @@ -9,5 +9,9 @@ SRC_URI_append_libc-musl = "\
  file://0001-include-linux-stddef.h-in-swab.h-uapi-header.patch \
 "

 +SRC_URI_append = "\
 +
file://0001-scripts-Use-fixed-temporary-file-instead-of-pipe-for.patch \
 +"
 +
  SRC_URI[md5sum] = "bee5fe53ee1c3142b8f0c12c0d3348f9"
  SRC_URI[sha256sum] = 
"19d8bcf49ef530cd4e364a45b4a22fa70714b70349c8100e7308488e26f1eaf1"
 --
 2.7.4

 --
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org 
<mailto:Openembedded-core@lists.openembedded.org>
 http://lists.openembedded.org/mailman/listinfo/openembedded-core



--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at 
its end
- "Use the force Harry" - Gandalf, Star Trek II








--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix IVS
https://geanix.com
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/4] ccache.bbclass: Refactor it to make it more reliable

[Martin Hundebøll]

Hi Robert,

Just a small nit-pick below..

On 10/12/2018 03.24, Robert Yang wrote:

The previous ccache.bbclass has the following problems:
- It uses host's ccache for native recipes, but this may not work on some
   hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always
   build failures (m4-native failed at do_configure, and others will also be
   failed if I disable CCACHE for m4-native)

- native/nativesdk/cross/crosssdk recipes use host's ccahe, but target uses
   ccache-native, this may confuse user.

- The target recipes may use both host's ccache and ccache-native, this may
   cause unexpected problems and hard to debug. This is because ccache-native is
   in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when
   rebuild target recipes, and then it would use hosttools/ccache, but the
   previous ccache files were generated by ccache-native.

- Target recipes can't use ccache when no ccahe is installed on the host:
   CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"

After refactored:
All types recipes (native, target and others) will use ccache-native except
ccache-native's dependencies, host's cache won't be used any more. It is more
reliable now, which will work everywhere when ccache-native can be built.

And now we need use "CCACHE_DISABLE = '1'" to disable ccahe for the recipe
rather than "CCACHE = ''" since we set CCACHE in anonymous function, and
d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we
can't check whether CCACHE is set or not in anonymous function since it is
always set. Use CCACHE_DISABLE to disable it would be more clear.

Signed-off-by: Robert Yang 
---
  meta/classes/ccache.bbclass | 25 ++---
  meta/conf/bitbake.conf  | 21 -
  meta/conf/layer.conf|  1 -
  meta/lib/oe/utils.py|  3 +++
  meta/recipes-devtools/ccache/ccache.inc |  2 ++
  5 files changed, 43 insertions(+), 9 deletions(-)






diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index 8a584d6..9c41705 100644
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -363,6 +363,9 @@ def host_gcc_version(d, taskcontextonly=False):
  return
  
  compiler = d.getVar("BUILD_CC")

+# Get rid of ccache since it is not present when parsing.
+if compiler.startswith('ccache '):
+compiler = compiler[7:]


You can avoid the if-condition and hard-coded 7-value by doing:

  # Get rid of ccache since it is no present when parsing.
  compiler = d.getVar("BUILD_CC").lstrip("ccache-")

// Martin


  try:
  env = os.environ.copy()
  env["PATH"] = d.getVar("PATH")
diff --git a/meta/recipes-devtools/ccache/ccache.inc 
b/meta/recipes-devtools/ccache/ccache.inc
index 6566328..259c196 100644
--- a/meta/recipes-devtools/ccache/ccache.inc
+++ b/meta/recipes-devtools/ccache/ccache.inc
@@ -9,6 +9,8 @@ LICENSE = "GPLv3+"
  
  DEPENDS = "zlib"
  
+DEPENDS_class-native = "${CCACHE_NATIVE_DEPENDS}"

+
  SRC_URI = "https://download.samba.org/pub/${BPN}/${BP}.tar.xz;
  
  inherit autotools




--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix IVS
https://geanix.com
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH][thud 3/3] openssl-1.1.1: remove build path from version info

[Martin Hundebøll]

The openssl build system generates buildinf.h containing the full
compiler command line used to compile objects. This breaks
reproducibility, as the compile command is baked into libcrypto, where
it is used when running `openssl version -f`.

Add stripped build variables for the compiler and cflags lines, and use
those when generating buildinfo.h.

This is based on a similar patch for older openssl versions:
https://patchwork.openembedded.org/patch/147229/

Signed-off-by: Martin Hundebøll 
Signed-off-by: Richard Purdie 
---
 ...sysroot-and-debug-prefix-map-from-co.patch | 70 +++
 .../openssl/openssl_1.1.1.bb  |  1 +
 2 files changed, 71 insertions(+)
 create mode 100644 
meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch

diff --git 
a/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 
b/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
new file mode 100644
index 00..80b62ab18c
--- /dev/null
+++ 
b/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
@@ -0,0 +1,70 @@
+From 3e1d00481093e10775eaf69d619c45b32a4aa7dc Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Martin=20Hundeb=C3=B8ll?= 
+Date: Tue, 6 Nov 2018 14:50:47 +0100
+Subject: [PATCH] buildinfo: strip sysroot and debug-prefix-map from compiler
+ info
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+The openssl build system generates buildinf.h containing the full
+compiler command line used to compile objects. This breaks
+reproducibility, as the compile command is baked into libcrypto, where
+it is used when running `openssl version -f`.
+
+Add stripped build variables for the compiler and cflags lines, and use
+those when generating buildinfo.h.
+
+This is based on a similar patch for older openssl versions:
+https://patchwork.openembedded.org/patch/147229/
+
+Upstream-Status: Inappropriate [OE specific]
+Signed-off-by: Martin Hundebøll 
+---
+ Configurations/unix-Makefile.tmpl | 10 +-
+ crypto/build.info |  2 +-
+ 2 files changed, 10 insertions(+), 2 deletions(-)
+
+diff --git a/Configurations/unix-Makefile.tmpl 
b/Configurations/unix-Makefile.tmpl
+index 16af4d2087..54c162784c 100644
+--- a/Configurations/unix-Makefile.tmpl
 b/Configurations/unix-Makefile.tmpl
+@@ -317,13 +317,21 @@ BIN_LDFLAGS={- join(' ', $target{bin_lflags} || (),
+  '$(CNF_LDFLAGS)', '$(LDFLAGS)') -}
+ BIN_EX_LIBS=$(CNF_EX_LIBS) $(EX_LIBS)
+ 
+-# CPPFLAGS_Q is used for one thing only: to build up buildinf.h
++# *_Q variables are used for one thing only: to build up buildinf.h
+ CPPFLAGS_Q={- $cppflags1 =~ s|([\\"])|\\$1|g;
+   $cppflags2 =~ s|([\\"])|\\$1|g;
+   $lib_cppflags =~ s|([\\"])|\\$1|g;
+   join(' ', $lib_cppflags || (), $cppflags2 || (),
+ $cppflags1 || ()) -}
+ 
++CFLAGS_Q={- for (@{$config{CFLAGS}}) {
++  s|-fdebug-prefix-map=[^ ]+|-fdebug-prefix-map=|g;
++}
++join(' ', @{$config{CFLAGS}}) -}
++
++CC_Q={- $config{CC} =~ s|--sysroot=[^ ]+|--sysroot=recipe-sysroot|g;
++join(' ', $config{CC}) -}
++
+ PERLASM_SCHEME= {- $target{perlasm_scheme} -}
+ 
+ # For x86 assembler: Set PROCESSOR to 386 if you want to support
+diff --git a/crypto/build.info b/crypto/build.info
+index b515b7318e..8c9cee2a09 100644
+--- a/crypto/build.info
 b/crypto/build.info
+@@ -10,7 +10,7 @@ EXTRA=  ../ms/uplink-x86.pl ../ms/uplink.c ../ms/applink.c \
+ ppccpuid.pl pariscid.pl alphacpuid.pl arm64cpuid.pl armv4cpuid.pl
+ 
+ DEPEND[cversion.o]=buildinf.h
+-GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(LIB_CFLAGS) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
++GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC_Q) $(CFLAGS_Q) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
+ DEPEND[buildinf.h]=../configdata.pm
+ 
+ GENERATE[uplink-x86.s]=../ms/uplink-x86.pl $(PERLASM_SCHEME)
+-- 
+2.19.1
+
diff --git a/meta/recipes-connectivity/openssl/openssl_1.1.1.bb 
b/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
index b44089e82e..1234b64b86 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
@@ -17,6 +17,7 @@ SRC_URI = "http://www.openssl.org/source/openssl-${PV}.tar.gz 
\
file://0001-skip-test_symbol_presence.patch \
file://0002-fix-CVE-2018-0734.patch \
file://0003-fix-CVE-2018-0735.patch \
+   
file://0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch \
"
 
 SRC_URI_append_class-nativesdk = " \
-- 
2.19.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH][thud 2/3] busybox: make busybox.links.{suid, nosuid} reproducible

[Martin Hundebøll]

The busybox.link.* files are generated from autoconf.h and applets.h,
which are both auto-generated by the build system. The contents of the
two files might be in different order, and so the link files are not
reproducble as is.

Fix this by sorting the lists using `sort`.

Signed-off-by: Martin Hundebøll 
Signed-off-by: Richard Purdie 
---
 meta/recipes-core/busybox/busybox.inc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/busybox/busybox.inc 
b/meta/recipes-core/busybox/busybox.inc
index 10582a7534..09433dd824 100644
--- a/meta/recipes-core/busybox/busybox.inc
+++ b/meta/recipes-core/busybox/busybox.inc
@@ -184,7 +184,8 @@ do_compile() {
oe_runmake busybox_unstripped
mv busybox_unstripped busybox.$s
oe_runmake busybox.links
-   mv busybox.links busybox.links.$s
+   sort busybox.links > busybox.links.$s
+   rm busybox.links
done
 
# hard fail if sh is being linked to the suid busybox (detects 
bug 10346)
-- 
2.19.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH][thud 1/3] shadow: improve reproducibility by hard-coding shell path

[Martin Hundebøll]

The shadow configure script tries really hard to detect the running
shell to make sure it doesn't do unsupported calls.

On my system the shell is detected as /bin/sh, while a build in an
ubuntu docker it resolves to /bin/bash. And since the shell path is
baked into the target binaries through config.h, the build becomes
inreproducible.

Fix reproducibility by hard-coding the shell to be /bin/sh

Signed-off-by: Martin Hundebøll 
Signed-off-by: Richard Purdie 
---
 meta/recipes-extended/shadow/shadow.inc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-extended/shadow/shadow.inc 
b/meta/recipes-extended/shadow/shadow.inc
index 0fa80a282a..09c37ef8a3 100644
--- a/meta/recipes-extended/shadow/shadow.inc
+++ b/meta/recipes-extended/shadow/shadow.inc
@@ -45,6 +45,8 @@ PAM_SRC_URI = "file://pam.d/chfn \
 
 inherit autotools gettext
 
+export CONFIG_SHELL="/bin/sh"
+
 EXTRA_OECONF += "--without-audit \
  --without-libcrack \
  --without-selinux \
-- 
2.19.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] scons.bbclass: Force rebuild if necessary

[Martin Hundebøll]

On 13/11/2018 13.22, Andreas Müller wrote:

On Tue, Nov 13, 2018 at 12:06 PM Burton, Ross  wrote:


This make me look at scons a bit closer, somewhat in shock that it
doesn't have a native way of doing out of tree builds.  Apparently
abusing the repository argument will sometimes work for out of tree
(so you could just put B=$WORKDIR/build and add B to [cleandirs]) but
abusing sounds like 'doesn't always work' to me.

I then discovered that the one recipe in oe-core that uses scons
doesn't in fact use the class, and switching to it breaks because
there's no standard way of passing install paths (serf tries to
install to /usr on the host).

So how is your experience with the scons class? Is it actually useful?

Ross
On Tue, 13 Nov 2018 at 10:18, Andreas Müller  wrote:


Signed-off-by: Andreas Müller 
---
  meta/classes/scons.bbclass | 11 ++-
  1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/meta/classes/scons.bbclass b/meta/classes/scons.bbclass
index b9ae19d582..286843f74c 100644
--- a/meta/classes/scons.bbclass
+++ b/meta/classes/scons.bbclass
@@ -2,7 +2,16 @@ DEPENDS += "python-scons-native"

  EXTRA_OESCONS ?= ""

-do_configure[noexec] = "1"
+do_configure() {
+   if [ -n "${CONFIGURESTAMPFILE}" ]; then
+   if [ -e "${CONFIGURESTAMPFILE}" -a "`cat ${CONFIGURESTAMPFILE}`" != 
"${BB_TASKHASH}" -a "${CLEANBROKEN}" != "1" ]; then
+   ${STAGING_BINDIR_NATIVE}/scons --clean PREFIX=${prefix} 
prefix=${prefix} ${EXTRA_OESCONS}
+   fi
+
+   mkdir -p `dirname ${CONFIGURESTAMPFILE}`
+   echo ${BB_TASKHASH} > ${CONFIGURESTAMPFILE}
+   fi
+}

  scons_do_compile() {
  ${STAGING_BINDIR_NATIVE}/scons ${PARALLEL_MAKE} PREFIX=${prefix} 
prefix=${prefix} ${EXTRA_OESCONS} || \
--
2.14.5


Have few recipes in my layers inheriting scons. I build them
frequently in my running 'world image' [1]:


I use gpsd from meta-oe, which "works" too...

// Martin


meta-games/recipes-games/pingus/pingus_0.7.6.bb -> Builds 'fine' currently
meta-qt5-extra/recipes-misc/recipes-multimedia/mixxx/mixxx_git.bb
There are others I don't use:
meta-openembedded/meta-oe/recipes-dbs/mongodb/mongodb_git.bb
meta-openembedded/meta-oe/recipes-navigation/gpsd/gpsd_3.17.bb

Mixxx (has QA issues I am trying to fix) is the one on which I found
out that rebuild is not performed in case recipes are changed on which
mixxx depends on.

So yes I consider this class useful :)

[1] 
https://github.com/schnitzeltony/meta-mortsgna/blob/master/recipes-image/allgui/allgui-full-image.bb

Andreas



--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix IVS
https://geanix.com
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] busybox: make busybox.links.{suid, nosuid} reproducible

[Martin Hundebøll]

The busybox.link.* files are generated from autoconf.h and applets.h,
which are both auto-generated by the build system. The contents of the
two files might be in different order, and so the link files are not
reproducble as is.

Fix this by sorting the lists using `sort`.

Signed-off-by: Martin Hundebøll 
---
 meta/recipes-core/busybox/busybox.inc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/busybox/busybox.inc 
b/meta/recipes-core/busybox/busybox.inc
index 10582a7534..09433dd824 100644
--- a/meta/recipes-core/busybox/busybox.inc
+++ b/meta/recipes-core/busybox/busybox.inc
@@ -184,7 +184,8 @@ do_compile() {
oe_runmake busybox_unstripped
mv busybox_unstripped busybox.$s
oe_runmake busybox.links
-   mv busybox.links busybox.links.$s
+   sort busybox.links > busybox.links.$s
+   rm busybox.links
done
 
# hard fail if sh is being linked to the suid busybox (detects 
bug 10346)
-- 
2.19.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2] openssl-1.1.1: remove build path from version info

[Martin Hundebøll]

The openssl build system generates buildinf.h containing the full
compiler command line used to compile objects. This breaks
reproducibility, as the compile command is baked into libcrypto, where
it is used when running `openssl version -f`.

Add stripped build variables for the compiler and cflags lines, and use
those when generating buildinfo.h.

This is based on a similar patch for older openssl versions:
https://patchwork.openembedded.org/patch/147229/

Signed-off-by: Martin Hundebøll 
---

Changes since v1:
 - rebased on current master

 ...sysroot-and-debug-prefix-map-from-co.patch | 70 +++
 .../openssl/openssl_1.1.1.bb  |  1 +
 2 files changed, 71 insertions(+)
 create mode 100644 
meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch

diff --git 
a/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 
b/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
new file mode 100644
index 00..80b62ab18c
--- /dev/null
+++ 
b/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
@@ -0,0 +1,70 @@
+From 3e1d00481093e10775eaf69d619c45b32a4aa7dc Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Martin=20Hundeb=C3=B8ll?= 
+Date: Tue, 6 Nov 2018 14:50:47 +0100
+Subject: [PATCH] buildinfo: strip sysroot and debug-prefix-map from compiler
+ info
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+The openssl build system generates buildinf.h containing the full
+compiler command line used to compile objects. This breaks
+reproducibility, as the compile command is baked into libcrypto, where
+it is used when running `openssl version -f`.
+
+Add stripped build variables for the compiler and cflags lines, and use
+those when generating buildinfo.h.
+
+This is based on a similar patch for older openssl versions:
+https://patchwork.openembedded.org/patch/147229/
+
+Upstream-Status: Inappropriate [OE specific]
+Signed-off-by: Martin Hundebøll 
+---
+ Configurations/unix-Makefile.tmpl | 10 +-
+ crypto/build.info |  2 +-
+ 2 files changed, 10 insertions(+), 2 deletions(-)
+
+diff --git a/Configurations/unix-Makefile.tmpl 
b/Configurations/unix-Makefile.tmpl
+index 16af4d2087..54c162784c 100644
+--- a/Configurations/unix-Makefile.tmpl
 b/Configurations/unix-Makefile.tmpl
+@@ -317,13 +317,21 @@ BIN_LDFLAGS={- join(' ', $target{bin_lflags} || (),
+  '$(CNF_LDFLAGS)', '$(LDFLAGS)') -}
+ BIN_EX_LIBS=$(CNF_EX_LIBS) $(EX_LIBS)
+ 
+-# CPPFLAGS_Q is used for one thing only: to build up buildinf.h
++# *_Q variables are used for one thing only: to build up buildinf.h
+ CPPFLAGS_Q={- $cppflags1 =~ s|([\\"])|\\$1|g;
+   $cppflags2 =~ s|([\\"])|\\$1|g;
+   $lib_cppflags =~ s|([\\"])|\\$1|g;
+   join(' ', $lib_cppflags || (), $cppflags2 || (),
+ $cppflags1 || ()) -}
+ 
++CFLAGS_Q={- for (@{$config{CFLAGS}}) {
++  s|-fdebug-prefix-map=[^ ]+|-fdebug-prefix-map=|g;
++}
++join(' ', @{$config{CFLAGS}}) -}
++
++CC_Q={- $config{CC} =~ s|--sysroot=[^ ]+|--sysroot=recipe-sysroot|g;
++join(' ', $config{CC}) -}
++
+ PERLASM_SCHEME= {- $target{perlasm_scheme} -}
+ 
+ # For x86 assembler: Set PROCESSOR to 386 if you want to support
+diff --git a/crypto/build.info b/crypto/build.info
+index b515b7318e..8c9cee2a09 100644
+--- a/crypto/build.info
 b/crypto/build.info
+@@ -10,7 +10,7 @@ EXTRA=  ../ms/uplink-x86.pl ../ms/uplink.c ../ms/applink.c \
+ ppccpuid.pl pariscid.pl alphacpuid.pl arm64cpuid.pl armv4cpuid.pl
+ 
+ DEPEND[cversion.o]=buildinf.h
+-GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(LIB_CFLAGS) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
++GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC_Q) $(CFLAGS_Q) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
+ DEPEND[buildinf.h]=../configdata.pm
+ 
+ GENERATE[uplink-x86.s]=../ms/uplink-x86.pl $(PERLASM_SCHEME)
+-- 
+2.19.1
+
diff --git a/meta/recipes-connectivity/openssl/openssl_1.1.1.bb 
b/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
index b44089e82e..1234b64b86 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
@@ -17,6 +17,7 @@ SRC_URI = "http://www.openssl.org/source/openssl-${PV}.tar.gz 
\
file://0001-skip-test_symbol_presence.patch \
file://0002-fix-CVE-2018-0734.patch \
file://0003-fix-CVE-2018-0735.patch \
+   
file://0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch \
"
 
 SRC_URI_append_class-nativesdk = " \
-- 
2.19.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2] shadow: improve reproducibility by hard-coding shell path

[Martin Hundebøll]

The shadow configure script tries really hard to detect the running
shell to make sure it doesn't do unsupported calls.

On my system the shell is detected as /bin/sh, while a build in an
ubuntu docker it resolves to /bin/bash. And since the shell path is
baked into the target binaries through config.h, the build becomes
inreproducible.

Fix reproducibility by hard-coding the shell to be /bin/sh

Signed-off-by: Martin Hundebøll 
---

Changes since v1:
- changed hard-coded path from /bin/bash to /bin/sh

 meta/recipes-extended/shadow/shadow.inc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-extended/shadow/shadow.inc 
b/meta/recipes-extended/shadow/shadow.inc
index 0fa80a282a..09c37ef8a3 100644
--- a/meta/recipes-extended/shadow/shadow.inc
+++ b/meta/recipes-extended/shadow/shadow.inc
@@ -45,6 +45,8 @@ PAM_SRC_URI = "file://pam.d/chfn \
 
 inherit autotools gettext
 
+export CONFIG_SHELL="/bin/sh"
+
 EXTRA_OECONF += "--without-audit \
  --without-libcrack \
  --without-selinux \
-- 
2.19.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] openssl-1.1.1: remove build path from version info

[Martin Hundebøll]

The openssl build system generates buildinf.h containing the full
compiler command line used to compile objects. This breaks
reproducibility, as the compile command is baked into libcrypto, where
it is used when running `openssl version -f`.

Add stripped build variables for the compiler and cflags lines, and use
those when generating buildinfo.h.

This is based on a similar patch for older openssl versions:
https://patchwork.openembedded.org/patch/147229/

Signed-off-by: Martin Hundebøll 
---
 ...sysroot-and-debug-prefix-map-from-co.patch | 70 +++
 .../openssl/openssl_1.1.1.bb  |  1 +
 2 files changed, 71 insertions(+)
 create mode 100644 
meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch

diff --git 
a/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
 
b/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
new file mode 100644
index 00..80b62ab18c
--- /dev/null
+++ 
b/meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
@@ -0,0 +1,70 @@
+From 3e1d00481093e10775eaf69d619c45b32a4aa7dc Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Martin=20Hundeb=C3=B8ll?= 
+Date: Tue, 6 Nov 2018 14:50:47 +0100
+Subject: [PATCH] buildinfo: strip sysroot and debug-prefix-map from compiler
+ info
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+The openssl build system generates buildinf.h containing the full
+compiler command line used to compile objects. This breaks
+reproducibility, as the compile command is baked into libcrypto, where
+it is used when running `openssl version -f`.
+
+Add stripped build variables for the compiler and cflags lines, and use
+those when generating buildinfo.h.
+
+This is based on a similar patch for older openssl versions:
+https://patchwork.openembedded.org/patch/147229/
+
+Upstream-Status: Inappropriate [OE specific]
+Signed-off-by: Martin Hundebøll 
+---
+ Configurations/unix-Makefile.tmpl | 10 +-
+ crypto/build.info |  2 +-
+ 2 files changed, 10 insertions(+), 2 deletions(-)
+
+diff --git a/Configurations/unix-Makefile.tmpl 
b/Configurations/unix-Makefile.tmpl
+index 16af4d2087..54c162784c 100644
+--- a/Configurations/unix-Makefile.tmpl
 b/Configurations/unix-Makefile.tmpl
+@@ -317,13 +317,21 @@ BIN_LDFLAGS={- join(' ', $target{bin_lflags} || (),
+  '$(CNF_LDFLAGS)', '$(LDFLAGS)') -}
+ BIN_EX_LIBS=$(CNF_EX_LIBS) $(EX_LIBS)
+ 
+-# CPPFLAGS_Q is used for one thing only: to build up buildinf.h
++# *_Q variables are used for one thing only: to build up buildinf.h
+ CPPFLAGS_Q={- $cppflags1 =~ s|([\\"])|\\$1|g;
+   $cppflags2 =~ s|([\\"])|\\$1|g;
+   $lib_cppflags =~ s|([\\"])|\\$1|g;
+   join(' ', $lib_cppflags || (), $cppflags2 || (),
+ $cppflags1 || ()) -}
+ 
++CFLAGS_Q={- for (@{$config{CFLAGS}}) {
++  s|-fdebug-prefix-map=[^ ]+|-fdebug-prefix-map=|g;
++}
++join(' ', @{$config{CFLAGS}}) -}
++
++CC_Q={- $config{CC} =~ s|--sysroot=[^ ]+|--sysroot=recipe-sysroot|g;
++join(' ', $config{CC}) -}
++
+ PERLASM_SCHEME= {- $target{perlasm_scheme} -}
+ 
+ # For x86 assembler: Set PROCESSOR to 386 if you want to support
+diff --git a/crypto/build.info b/crypto/build.info
+index b515b7318e..8c9cee2a09 100644
+--- a/crypto/build.info
 b/crypto/build.info
+@@ -10,7 +10,7 @@ EXTRA=  ../ms/uplink-x86.pl ../ms/uplink.c ../ms/applink.c \
+ ppccpuid.pl pariscid.pl alphacpuid.pl arm64cpuid.pl armv4cpuid.pl
+ 
+ DEPEND[cversion.o]=buildinf.h
+-GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(LIB_CFLAGS) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
++GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC_Q) $(CFLAGS_Q) 
$(CPPFLAGS_Q)" "$(PLATFORM)"
+ DEPEND[buildinf.h]=../configdata.pm
+ 
+ GENERATE[uplink-x86.s]=../ms/uplink-x86.pl $(PERLASM_SCHEME)
+-- 
+2.19.1
+
diff --git a/meta/recipes-connectivity/openssl/openssl_1.1.1.bb 
b/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
index af9038abd5..93750c3afb 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
@@ -15,6 +15,7 @@ SRC_URI = "http://www.openssl.org/source/openssl-${PV}.tar.gz 
\
file://run-ptest \
file://openssl-c_rehash.sh \
file://0001-skip-test_symbol_presence.patch \
+   
file://0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch \
"
 
 SRC_URI_append_class-nativesdk = " \
-- 
2.19.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] shadow: improve reproducibility by hard-coding shell path

[Martin Hundebøll]

The shadow configure script tries really hard to detect the running
shell to make sure it doesn't do unsupported calls.

On my system the shell is detected as /bin/sh, while a build in an
ubuntu docker it resolves to /bin/bash. And since the shell path is
baked into the target binaries through config.h, the build becomes
inreproducible.

Fix reproducibility by hard-coding the shell to be /bin/bash

Signed-off-by: Martin Hundebøll 
---

I wasn't whether to use /bin/bash or /bin/sh; comments are welcome...

 meta/recipes-extended/shadow/shadow.inc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-extended/shadow/shadow.inc 
b/meta/recipes-extended/shadow/shadow.inc
index 0fa80a282a..8eacca6809 100644
--- a/meta/recipes-extended/shadow/shadow.inc
+++ b/meta/recipes-extended/shadow/shadow.inc
@@ -45,6 +45,8 @@ PAM_SRC_URI = "file://pam.d/chfn \
 
 inherit autotools gettext
 
+export CONFIG_SHELL="/bin/bash"
+
 EXTRA_OECONF += "--without-audit \
  --without-libcrack \
  --without-selinux \
-- 
2.19.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCHv2] glibc: make ld-2.28.so reproducible on arm

[Martin Hundebøll]

Hi Armin,

On 06/11/2018 20.41, akuster808 wrote:

Martin,


On 11/6/18 2:04 AM, Martin Hundebøll wrote:

Play the whack-a-mole game and add the .file directive to another
assembly file that otherwise shows itself in ld-2.28.so debug file,
which in turns alters the build-id of ld-2.28.so on target.

Signed-off-by: Martin Hundebøll 
---

Changes since v1:

  - Added signed-of-by tag

  .../0034-inject-file-assembly-directives.patch  | 13 +
  1 file changed, 13 insertions(+)

diff --git 
a/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch 
b/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch
index 3f9f8cf189..55eba2d43e 100644
--- a/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch
+++ b/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch
@@ -257,3 +257,16 @@ index 244d87fb6d..14d3e2ce14 100644
   /* Initial entry point code for the dynamic linker.
  The C function `_dl_start' is the real entry point;
  its return value is the user program's entry point.   */


Please add Upsteam-Status: and Signed-off-by: in the patch itself.


It was Richard's patch, so I chose the less intrusive approach. I can do 
a refresh of the entire patch?



You may want to review
https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines

This seems like a backport candidate for Thud too?


Yes, I was planning on bugging you was it hits master :)

Thanks,
Martin


regards,

Armin


+diff --git a/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S 
b/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S
+index d26ad1f8d3..a0de10bf81 100644
+--- a/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S
 b/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S
+@@ -27,6 +27,8 @@
+ARM unwind tables for register to register moves, the actual opcodes
+are not defined.  */
+
++  .file "libc-do-syscall.S"
++
+ #if defined(__thumb__)
+   .thumb
+   .syntax unified


--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix IVS
https://geanix.com
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2] glibc: make ld-2.28.so reproducible on arm

[Martin Hundebøll]

Play the whack-a-mole game and add the .file directive to another
assembly file that otherwise shows itself in ld-2.28.so debug file,
which in turns alters the build-id of ld-2.28.so on target.

Signed-off-by: Martin Hundebøll 
---

Changes since v1:

 - Added signed-of-by tag

 .../0034-inject-file-assembly-directives.patch  | 13 +
 1 file changed, 13 insertions(+)

diff --git 
a/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch 
b/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch
index 3f9f8cf189..55eba2d43e 100644
--- a/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch
+++ b/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch
@@ -257,3 +257,16 @@ index 244d87fb6d..14d3e2ce14 100644
  /* Initial entry point code for the dynamic linker.
 The C function `_dl_start' is the real entry point;
 its return value is the user program's entry point.*/
+diff --git a/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S 
b/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S
+index d26ad1f8d3..a0de10bf81 100644
+--- a/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S
 b/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S
+@@ -27,6 +27,8 @@
+ARM unwind tables for register to register moves, the actual opcodes
+are not defined.  */
+ 
++  .file "libc-do-syscall.S"
++
+ #if defined(__thumb__)
+   .thumb
+   .syntax unified
-- 
2.19.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] glibc: make ld-2.28.so reproducible on arm

[Martin Hundebøll]

Play the whack-a-mole game and add the .file directive to another
assembly file that otherwise shows itself in ld-2.28.so debug file,
which in turns alters the build-id of ld-2.28.so on target.
---
 .../0034-inject-file-assembly-directives.patch  | 13 +
 1 file changed, 13 insertions(+)

diff --git 
a/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch 
b/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch
index 3f9f8cf189..55eba2d43e 100644
--- a/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch
+++ b/meta/recipes-core/glibc/glibc/0034-inject-file-assembly-directives.patch
@@ -257,3 +257,16 @@ index 244d87fb6d..14d3e2ce14 100644
  /* Initial entry point code for the dynamic linker.
 The C function `_dl_start' is the real entry point;
 its return value is the user program's entry point.*/
+diff --git a/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S 
b/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S
+index d26ad1f8d3..a0de10bf81 100644
+--- a/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S
 b/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S
+@@ -27,6 +27,8 @@
+ARM unwind tables for register to register moves, the actual opcodes
+are not defined.  */
+ 
++  .file "libc-do-syscall.S"
++
+ #if defined(__thumb__)
+   .thumb
+   .syntax unified
-- 
2.19.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] systemtap: Fix issues from 4.0 upgrade

[Martin Hundebøll]

Hi Richard,

On 16/10/2018 21.25, Richard Purdie wrote:

4.0 adds systemd support but installs the unit files in the wrong location.
Fix this in do_install and inherit systemd to package them correctly.

Also fix ownership of files manually copied in to root:root to avoid QA 
warnings.

Signed-off-by: Richard Purdie 
---
  meta/recipes-kernel/systemtap/systemtap_git.bb | 10 +-
  1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/systemtap/systemtap_git.bb 
b/meta/recipes-kernel/systemtap/systemtap_git.bb
index 0b7833e901b..3c45bfa9374 100644
--- a/meta/recipes-kernel/systemtap/systemtap_git.bb
+++ b/meta/recipes-kernel/systemtap/systemtap_git.bb
@@ -25,7 +25,7 @@ PACKAGECONFIG[sqlite] = 
"--enable-sqlite,--disable-sqlite,sqlite3"
  PACKAGECONFIG[monitor] = "--enable-monitor,--disable-monitor,ncurses json-c"
  PACKAGECONFIG[python3-probes] = 
"--with-python3-probes,--without-python3-probes,python3-setuptools-native"
  
-inherit autotools gettext pkgconfig distutils3-base

+inherit autotools gettext pkgconfig distutils3-base systemd
  
  do_configure_prepend () {

  # Improve reproducibility for c++ object files
@@ -39,6 +39,14 @@ do_install_append () {
rm -rf ${D}${datadir}/${PN}
rm ${D}${libexecdir}/${PN}/stap-env
 fi
+
+   # Fix makefile hardcoded path assumptions for systemd (assumes $prefix)
+   install -d `dirname ${D}${systemd_unitdir}`
+   mv ${D}${prefix}/lib/systemd `dirname ${D}${systemd_unitdir}`
+   rmdir ${D}${prefix}/lib --ignore-fail-on-non-empty
+
+   # Ensure correct ownership for files copied in
+   chown root.root ${D}${sysconfdir}/stap-exporter/* -R


root:root ?


  }
  
  BBCLASSEXTEND = "nativesdk"




--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix IVS
https://geanix.com
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC 3/3] linux-firmware: MACHINEOVERRIDES for BCM43430 NVRAM

[Martin Hundebøll]

30-nvram-mur1dx() {
+       ( ln -sf ${PWD}/brcmfmac43430-sdio.MUR1DX.txt 
${D}${nonarch_base_libdir}/firmware/brcm/brcmfmac43430-sdio.txt)


... assuming PWD is available to the recipe. There will be a proper 
Yocto variable I can use, of course, but I can't think of it right now.


Either way, Martin's example doesn't work for me. And adding the 
absolute path of the original file doesn't seem any neater or clearer 
than following the TI example from the do_install a few lines up in the 
recipe. But I'm happy to do it either way, so long as it works.




--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix IVS
https://geanix.com
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 4/9] dos2unix.bbclass: Move to oe-core

[Martin Hundebøll]

Hi Khem,

On 19/08/2018 19.23, Khem Raj wrote:

- Import from meta-oe layer

- This is useful for many packages where CR-LF
   needs to be adjusted, many recipes depend on it
   e.g. meta-multimedia libebml and so on.

Signed-off-by: Khem Raj 
---
  meta/classes/dos2unix.bbclass | 14 ++
  1 file changed, 14 insertions(+)
  create mode 100644 meta/classes/dos2unix.bbclass

diff --git a/meta/classes/dos2unix.bbclass b/meta/classes/dos2unix.bbclass
new file mode 100644
index 00..3fc17e2196
--- /dev/null
+++ b/meta/classes/dos2unix.bbclass
@@ -0,0 +1,14 @@
+# Class for use to convert all CRLF line terminators to LF
+# provided that some projects are being developed/maintained
+# on Windows so they have different line terminators(CRLF) vs
+# on Linux(LF), which can cause annoying patching errors during
+# git push/checkout processes.
+
+do_convert_crlf_to_lf[depends] += "dos2unix-native:do_populate_sysroot"


Shouldn't the dos2unix recipe live in oe-core also, when a class depends 
upon it?


// Martin


+
+# Convert CRLF line terminators to LF
+do_convert_crlf_to_lf () {
+   find ${S} -type f -exec dos2unix {} \;
+}
+
+addtask convert_crlf_to_lf after do_unpack before do_patch



--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix IVS
https://geanix.com
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] openssh: fix wrong volatile dir for sshd host keys on read-only rootfs

[Martin Hundebøll]

Hi Andre,

On 15/08/2018 21.47, Andre McCurdy wrote:

On Wed, Aug 15, 2018 at 4:59 AM, Martin Hundebøll  wrote:

When the read-only-rootfs image feature is enabled, and openssh is
installed into an image, the ssh daemon is reconfigured to use
/var/run/ssh when generating host keys.

Fix up the creation of the volatile dir to actually match what sshd is
configured to.

Signed-off-by: Martin Hundebøll 
---
  meta/recipes-connectivity/openssh/openssh/volatiles.99_sshd | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-connectivity/openssh/openssh/volatiles.99_sshd 
b/meta/recipes-connectivity/openssh/openssh/volatiles.99_sshd
index a0d2af3c65..fcbc5ae9d5 100644
--- a/meta/recipes-connectivity/openssh/openssh/volatiles.99_sshd
+++ b/meta/recipes-connectivity/openssh/openssh/volatiles.99_sshd
@@ -1,2 +1,2 @@
-d root root 0755 /var/run/sshd none
+d root root 0755 /var/run/ssh none


This doesn't look right.

/var/run/sshd is the directory used for privilege separation (grep for
--with-privsep-path ), so it's not correct to remove it.


I see - didn't know about openssh chrooting to do privilege separation.


Note that sshd_check_keys script runs "mkdir -p $SYSCONFDIR" (ie
/var/run/ssh in the read-only rootfs case) at run time before creating
any keys.


Yes, it works without the volatile folder; for openssh at least.


What exactly was the problem that this patch tries to fix?


I am running a custom image with the read-only-rootfs feature enabled, 
and wanted to make the ssh host keys persistent across reboots.


At first, I tried adding a bind-mount entry to fstab from /data/ssh to 
/var/run/ssh, but the latter don't exist when mountall.sh is executed by 
RC (/data is the mountpoint of a persistent partition).


I then looked at the volatile entries and noticed that it created the 
(empty) /var/run/sshd, so changed it to (wrongly) create /var/run/ssh 
instead.


That wasn't enough though, since populate-volatiles.sh comes after 
mountall.sh.


In the end I simply added a new entry to volatiles to create a symlink 
from /var/run/ssh to /data/ssh, which works for me :)


Maybe I should change the patch to add a comment about the /var/run/sshd 
entry, so we don't end up doing mistakes like the 
debian-predictable-keys story.


// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] openssh: fix wrong volatile dir for sshd host keys on read-only rootfs

[Martin Hundebøll]

When the read-only-rootfs image feature is enabled, and openssh is
installed into an image, the ssh daemon is reconfigured to use
/var/run/ssh when generating host keys.

Fix up the creation of the volatile dir to actually match what sshd is
configured to.

Signed-off-by: Martin Hundebøll 
---
 meta/recipes-connectivity/openssh/openssh/volatiles.99_sshd | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-connectivity/openssh/openssh/volatiles.99_sshd 
b/meta/recipes-connectivity/openssh/openssh/volatiles.99_sshd
index a0d2af3c65..fcbc5ae9d5 100644
--- a/meta/recipes-connectivity/openssh/openssh/volatiles.99_sshd
+++ b/meta/recipes-connectivity/openssh/openssh/volatiles.99_sshd
@@ -1,2 +1,2 @@
-d root root 0755 /var/run/sshd none
+d root root 0755 /var/run/ssh none
 f root root 0644 /var/log/lastlog none
-- 
2.18.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe-core][PATCH v4 1/3] udev-extraconf/mount.sh: add support to systemd

[Martin Hundebøll]

Hi Hongzhi,

I think duplicating all of 'automount' when the only difference is the 
call to 'mount' is unnecessary.


It should be possible to check for system inside the existing 
'automount' (see diff below).


When reading the script I couldn't help cleaning up the special 
treatment of mount from util-linux (busybox accepts '-o silent' too), 
and remove the 'rm_dir' function, which can be replaced by 'rmdir'.


// Martin

diff --git a/meta/recipes-core/udev/udev-extraconf/mount.sh 
b/meta/recipes-core/udev/udev-extraconf/mount.sh

index d760328a09..6f013c5afb 100644
--- a/meta/recipes-core/udev/udev-extraconf/mount.sh
+++ b/meta/recipes-core/udev/udev-extraconf/mount.sh
@@ -20,11 +20,12 @@ done
 automount() {  
name="`basename "$DEVNAME"`"

-   ! test -d "/run/media/$name" && mkdir -p "/run/media/$name"
-   # Silent util-linux's version of mounting auto
-   if [ "x`readlink $MOUNT`" = "x/bin/mount.util-linux" ] ;
+   mkdir -p "/run/media/$name"
+
+   # Use systemd-mount to avoid being trapped in udev's mount-namespace
+   if [ "$(readlink /sbin/init)" = "/lib/systemd/systemd" ];
then
-   MOUNT="$MOUNT -o silent"
+   MOUNT="/usr/bin/systemd-mount --no-block"
fi

# If filesystem type is vfat, change the ownership group to 'disk', and
@@ -41,22 +42,12 @@ automount() {
if ! $MOUNT -t auto $DEVNAME "/run/media/$name"
then
 		#logger "mount.sh/automount" "$MOUNT -t auto $DEVNAME 
\"/run/media/$name\" failed!"

-   rm_dir "/run/media/$name"
+   rmdir "/run/media/$name"
else
 		logger "mount.sh/automount" "Auto-mount of [/run/media/$name] 
successful"

touch "/tmp/.automount-$name"
fi
 }
-   
-rm_dir() {
-   # We do not want to rm -r populated directories
-   if test "`find "$1" | wc -l | tr -d " "`" -lt 2 -a -d "$1"
-   then
-   ! test -z "$1" && rm -r "$1"
-   else
-   logger "mount.sh/automount" "Not removing non-empty directory 
[$1]"
-   fi
-}

 # No ID_FS_TYPE for cdrom device, yet it should be mounted
 name="`basename "$DEVNAME"`"

On 2018-08-02 04:50, Hongzhi.Song wrote:

Udev-extraconf works correctly with sysvinit in the aspect of automounting
block devices. But it has a serious problem in case of systemd. Block devices
automounted by udev is unaccessible to host space(out of udevd's private
namespace). For example, we cannot format those block devices.

e.g.
 root@qemux86:~# mkfs.ext4 /dev/sda1
 mke2fs 1.43.8
 /dev/sda1 contains a ext4 file system
 last mounted on Tue Apr
 Proceed anyway? (y,N) y
 /dev/sda1 is apparently in use by the system; will not make a filesystem 
here!

Other distributions has no such problem, because they use a series of rules to
manager block devices. Different types of block devices match different rules.
But udev-extraconf just use one rule, automount.rules, which results in this
problem.

The 'systemd-mount' command is recommended by the systemd community to solve 
such
problems.

This patch makes use of 'systemd-mount' to solve the above problem.

[YOCTO #12644]

Signed-off-by: Hongzhi.Song 
---
  meta/recipes-core/udev/udev-extraconf/mount.sh | 55 +++---
  1 file changed, 50 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-core/udev/udev-extraconf/mount.sh 
b/meta/recipes-core/udev/udev-extraconf/mount.sh
index d760328a09..3a72c455e0 100644
--- a/meta/recipes-core/udev/udev-extraconf/mount.sh
+++ b/meta/recipes-core/udev/udev-extraconf/mount.sh
@@ -4,10 +4,26 @@
  #
  # Attempt to mount any added block devices and umount any removed devices
  
+BASE_INIT="`readlink "/sbin/init"`"

+INIT_SYSTEMD="/lib/systemd/systemd"
+
+if [ "x$BASE_INIT" = "x$INIT_SYSTEMD" ];then
+MOUNT="/usr/bin/systemd-mount"
+UMOUNT="/usr/bin/systemd-umount"
+
+if [ -x $MOUNT ] && [ -x $UMOUNT ];
+then
+logger "Using systemd-mount to finish mount"
+else
+logger "Linux init is using systemd, so please install 
systemd-mount to finish mount"
+fi
+else
+MOUNT="/bin/mount"
+UMOUNT="/bin/umount"
+fi
  
-MOUNT="/bin/mount"

  PMOUNT="/usr/bin/pmount"
-UMOUNT="/bin/umount"
+
  for line in `grep -h -v ^# /etc/udev/mount.blacklist 
/etc/udev/mount.blacklist.d/*`
  do
if [ ` expr match "$DEVNAME" "$line" ` -gt 0 ];
@@ -17,6 +33,33 @@ do
fi
  done
  
+automount_systemd() {

+   name="`basename "$DEVNAME"`"
+
+! test -d "/run/media/$name" && mkdir -p "/run/media/$name"
+# Silent util-linux's version of mounting auto
+MOUNT="$MOUNT -o silent"
+
+# If filesystem type is vfat, change the ownership group to 'disk', and
+# grant it with  w/r/x permissions.
+case $ID_FS_TYPE in
+vfat|fat)
+MOUNT="$MOUNT -o umask=007,gid=`awk -F':' '/^disk/{print $3}' 
/etc/group`"
+ 

[OE-core] [PATCH][sumo] python2: Fix build with gcc8

[Martin Hundebøll]

From: Ross Burton 

(From OE-Core rev: 910f68c9c8dc26e12d28ef29e956af63d100f121)

Signed-off-by: Ross Burton 
Signed-off-by: Khem Raj 
Signed-off-by: Ross Burton 
Signed-off-by: Richard Purdie 
Signed-off-by: Martin Hundebøll 
---
 .../python/python-native_2.7.14.bb|  1 +
 .../python/python/fix-gc-alignment.patch  | 43 +++
 meta/recipes-devtools/python/python_2.7.14.bb |  1 +
 3 files changed, 45 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python/fix-gc-alignment.patch

diff --git a/meta/recipes-devtools/python/python-native_2.7.14.bb 
b/meta/recipes-devtools/python/python-native_2.7.14.bb
index 7c491fa3e0..b822583c53 100644
--- a/meta/recipes-devtools/python/python-native_2.7.14.bb
+++ b/meta/recipes-devtools/python/python-native_2.7.14.bb
@@ -16,6 +16,7 @@ SRC_URI += "\
 file://builddir.patch \
 file://parallel-makeinst-create-bindir.patch \
 file://revert_use_of_sysconfigdata.patch \
+file://fix-gc-alignment.patch \
"
 
 S = "${WORKDIR}/Python-${PV}"
diff --git a/meta/recipes-devtools/python/python/fix-gc-alignment.patch 
b/meta/recipes-devtools/python/python/fix-gc-alignment.patch
new file mode 100644
index 00..b63cd08747
--- /dev/null
+++ b/meta/recipes-devtools/python/python/fix-gc-alignment.patch
@@ -0,0 +1,43 @@
+Upstream-Status: Submitted
+Signed-off-by: Ross Burton 
+
+Fix for over-aligned GC info
+Patch by Florian Weimer
+
+See: https://bugzilla.redhat.com/show_bug.cgi?id=1540316
+Upstream discussion: 
https://mail.python.org/pipermail/python-dev/2018-January/152000.html
+
+diff --git a/Include/objimpl.h b/Include/objimpl.h
+index 55e83eced6..aa906144dc 100644
+--- a/Include/objimpl.h
 b/Include/objimpl.h
+@@ -248,6 +248,18 @@ PyAPI_FUNC(PyVarObject *) _PyObject_GC_Resize(PyVarObject 
*, Py_ssize_t);
+ /* for source compatibility with 2.2 */
+ #define _PyObject_GC_Del PyObject_GC_Del
+ 
++/* Former over-aligned definition of PyGC_Head, used to compute the
++   size of the padding for the new version below. */
++union _gc_head;
++union _gc_head_old {
++struct {
++union _gc_head *gc_next;
++union _gc_head *gc_prev;
++Py_ssize_t gc_refs;
++} gc;
++long double dummy;
++};
++
+ /* GC information is stored BEFORE the object structure. */
+ typedef union _gc_head {
+ struct {
+@@ -255,7 +267,8 @@ typedef union _gc_head {
+ union _gc_head *gc_prev;
+ Py_ssize_t gc_refs;
+ } gc;
+-long double dummy;  /* force worst-case alignment */
++double dummy;  /* force worst-case alignment */
++char dummy_padding[sizeof(union _gc_head_old)];
+ } PyGC_Head;
+ 
+ extern PyGC_Head *_PyGC_generation0;
+ 
\ No newline at end of file
diff --git a/meta/recipes-devtools/python/python_2.7.14.bb 
b/meta/recipes-devtools/python/python_2.7.14.bb
index 3044c3d363..b923b9237b 100644
--- a/meta/recipes-devtools/python/python_2.7.14.bb
+++ b/meta/recipes-devtools/python/python_2.7.14.bb
@@ -30,6 +30,7 @@ SRC_URI += "\
   file://add-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch \
   file://pass-missing-libraries-to-Extension-for-mul.patch \
   file://support_SOURCE_DATE_EPOCH_in_py_compile_2.7.patch \
+  file://fix-gc-alignment.patch \
 "
 
 S = "${WORKDIR}/Python-${PV}"
-- 
2.18.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] initscripts: remove warnings on read-only-rootfs (again)

[Martin Hundebøll]

When the populate-volatile.sh initscript tests if a configured symlink
is already in place, it uses readlink with the '-f' (follow) option:

> [ "$(readlink -f $source)" = "$dest" ]

If the test fails, it proceeds to delete the exisiting folder/file, and
create the configured symlink.

However, the '-f' option to readlink makes it follow symlinks pointing
at symlinks. If the $dest argument is a symlink, the above test fails,
and warnings are printed due to changing a read-only rootfs.

This is the case for /tmp, and /etc/resolv.conf:
> /tmp -> /var/tmp -> /var/volatile/tmp
> /etc/resolv.conf -> /var/run/resolv.conf -> /run/resolv.conf

Fix the warnings by removing the '-f' option, so that the test matches
the configuration.

Signed-off-by: Martin Hundebøll 
---
 .../initscripts/initscripts-1.0/populate-volatile.sh| 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/initscripts/initscripts-1.0/populate-volatile.sh 
b/meta/recipes-core/initscripts/initscripts-1.0/populate-volatile.sh
index 35316ec2ba..e1fce02906 100755
--- a/meta/recipes-core/initscripts/initscripts-1.0/populate-volatile.sh
+++ b/meta/recipes-core/initscripts/initscripts-1.0/populate-volatile.sh
@@ -80,7 +80,7 @@ mk_dir() {
 link_file() {
EXEC="
if [ -L \"$2\" ]; then
-   [ \"\$(readlink -f \"$2\")\" != \"$1\" ] && { rm -f \"$2\"; ln 
-sf \"$1\" \"$2\"; };
+   [ \"\$(readlink \"$2\")\" != \"$1\" ] && { rm -f \"$2\"; ln -sf 
\"$1\" \"$2\"; };
elif [ -d \"$2\" ]; then
if awk '\$2 == \"$2\" {exit 1}' /proc/mounts; then
cp -a $2/* $1 2>/dev/null;
-- 
2.18.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 2/2] recipes-support/nspr: Add RISC-V support

[Martin Hundebøll]

== 32)
++#define _PR_SI_ARCHITECTURE "riscv32"
++#elif defined(__riscv) && (__riscv_xlen == 64)
++#define _PR_SI_ARCHITECTURE "riscv64"
+ #else
+ #error "Unknown CPU architecture"
+ #endif
+ #define PR_DLL_SUFFIX ".so"
+
+ #define _PR_VMBASE  0x3000
+ #define _PR_STACK_VMBASE  0x5000
+ #define _MD_DEFAULT_STACK_SIZE65536L
+
diff --git a/meta/recipes-support/nspr/nspr_4.19.bb 
b/meta/recipes-support/nspr/nspr_4.19.bb
index de2c87131f..b43c78081d 100644
--- a/meta/recipes-support/nspr/nspr_4.19.bb
+++ b/meta/recipes-support/nspr/nspr_4.19.bb
@@ -11,6 +11,7 @@ SRC_URI = 
"http://ftp.mozilla.org/pub/nspr/releases/v${PV}/src/nspr-${PV}.tar.gz
 file://remove-srcdir-from-configure-in.patch \
 file://0002-Add-nios2-support.patch \
 file://0001-md-Fix-build-with-musl.patch \
+   file://0003-Add-type-definitions-for-the-RISC-V-architecture.patch \
 file://nspr.pc.in \
  "
  



--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix IVS
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] wic: allow bitbake variables in kickstarter files

[Martin Hundebøll]

On 2018-03-22 14:44, Rasmus Villemoes wrote:

image_types_wic.bbclass has a mechanism for doing variable substitution
on .wks files by simply letting the input file be called
.wks.in. However, that doesn't allow using variables in files included
via the include directive.

This adds (somewhat naive) support for variable substitution in all
files parsed by wic. The user should add all required variables to
WICVARS to get them exported appropriately.

Signed-off-by: Rasmus Villemoes 


ping³
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [OE-Core][PATCH 1/2] kernel-fitimage: Allow setting of DTB/DTBO relocation address

[Martin Hundebøll]

Hi Alex,

On 2018-06-16 09:17, Alex Kiernan wrote:

Introduce UBOOT_DTB_LOADADDRESS and UBOOT_DTBO_LOADADDRESS so that you
can set where U-Boot loads full and overlay DTBs. This is required when
using bootm's overlay support to construct the final DTB.

Signed-off-by: Alex Kiernan 
---
This commit conflicts with this patch:

https://patchwork.openembedded.org/series/12589/

Either that needs applying first and this needs rebasing on top of it, or
use this one in its place.

  meta/classes/kernel-fitimage.bbclass | 10 ++
  1 file changed, 10 insertions(+)

diff --git a/meta/classes/kernel-fitimage.bbclass 
b/meta/classes/kernel-fitimage.bbclass
index 50a91e1..f3c2ff0 100644
--- a/meta/classes/kernel-fitimage.bbclass
+++ b/meta/classes/kernel-fitimage.bbclass
@@ -135,6 +135,15 @@ fitimage_emit_section_dtb() {
  
  	dtb_csum="sha1"
  
+	dtb_loadline=""

+   dtb_ext=${DTB##*.}
+   if [ "${dtb_ext}" = "dtbo" ]; then
+   if [ -n "${UBOOT_DTBO_LOADADDRESS}" ]; then
+   dtb_loadline="load = <${UBOOT_DTBO_LOADADDRESS}>;"
+   fi
+   elif [ -n "${UBOOT_DTB_LOADADDRESS}" ]; then
+   dtb_loadline="load = <${UBOOT_DTB_LOADADDRESS}>;"
+   fi


I haven't looked into the use of DTB overlays in fitimages, but 
shouldn't it be possible to specify multiple overlays?


From the way I read your patch, only a single overlay is supported...

// Martin


cat << EOF >> ${1}
  fdt@${2} {
  description = "Flattened Device Tree blob";
@@ -142,6 +151,7 @@ fitimage_emit_section_dtb() {
  type = "flat_dt";
  arch = "${UBOOT_ARCH}";
  compression = "none";
+${dtb_loadline}
      hash@1 {
  algo = "${dtb_csum}";
  };



--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix IVS
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] [PATCH] [PATCH] bind:9.10.6 -> 9.10.7

[Martin Hundebøll]

Hi Hong,

On 2018-06-05 10:10, Hong Liu wrote:

1.Upgrade bind from 9.10.6 to 9.10.7.


There is a pending patch for updating bind to 9.12.1:
https://patchwork.openembedded.org/patch/151135/

// Martin


2.Update the checksum of LIC_FILES_CHKSUM, since bind has been changed. The 
lincese has been modified time.

Signed-off-by: Hong Liu 
---
  .../bind/{bind_9.10.6.bb => bind_9.10.7.bb} | 13 ++---
  1 file changed, 6 insertions(+), 7 deletions(-)
  rename meta/recipes-connectivity/bind/{bind_9.10.6.bb => bind_9.10.7.bb} (90%)

diff --git a/meta/recipes-connectivity/bind/bind_9.10.6.bb 
b/meta/recipes-connectivity/bind/bind_9.10.7.bb
similarity index 90%
rename from meta/recipes-connectivity/bind/bind_9.10.6.bb
rename to meta/recipes-connectivity/bind/bind_9.10.7.bb
index 20c8d7b..124968b 100644
--- a/meta/recipes-connectivity/bind/bind_9.10.6.bb
+++ b/meta/recipes-connectivity/bind/bind_9.10.7.bb
@@ -3,7 +3,7 @@ HOMEPAGE = "http://www.isc.org/sw/bind/;
  SECTION = "console/network"
  
  LICENSE = "ISC & BSD"

-LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=dba46507446198119bcde32a4feaab43"
+LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=c767ea4f3cf1017290d5b799fa929f80"
  
  DEPENDS = "openssl libcap"
  
@@ -15,16 +15,14 @@ SRC_URI = "https://ftp.isc.org/isc/bind9/${PV}/${BPN}-${PV}.tar.gz \

 file://named.service \
 file://bind9 \
 file://init.d-add-support-for-read-only-rootfs.patch \
-   file://bind-confgen-build-unix.o-once.patch \
 file://0001-build-use-pkg-config-to-find-libxml2.patch \
 file://bind-ensure-searching-for-json-headers-searches-sysr.patch \
 file://0001-gen.c-extend-DIRNAMESIZE-from-256-to-512.patch \
 file://0001-lib-dns-gen.c-fix-too-long-error.patch \
-   file://use-python3-and-fix-install-lib-path.patch \
 "
  
-SRC_URI[md5sum] = "84e663284b17aee0df1ce6f248b137d7"

-SRC_URI[sha256sum] = 
"17bbcd2bd7b1d32f5ba4b30d5dbe8a39bce200079048073d1e0d050fdf47e69d"
+SRC_URI[md5sum] = "320b9ef5bdf19ef2718a823d9b4243ba"
+SRC_URI[sha256sum] = 
"e15e70982d966cd4b194ff483f5cc636c0017b054ae2e332ca3f28fe53f11660"
  
  UPSTREAM_CHECK_URI = "https://ftp.isc.org/isc/bind9/;

  UPSTREAM_CHECK_REGEX = "(?P9(\.\d+)+(-P\d+)*)/"
@@ -36,6 +34,7 @@ EXTRA_OECONF = " ${ENABLE_IPV6} --with-libtool 
--enable-threads \
   --with-gssapi=no --with-ecdsa=yes \
   --sysconfdir=${sysconfdir}/bind \
   --with-openssl=${STAGING_DIR_HOST}${prefix} \
+   --with-eddsa=no \
 "
  
  inherit autotools update-rc.d systemd useradd pkgconfig python3-dir

@@ -67,6 +66,7 @@ PACKAGE_BEFORE_PN += "${PN}-utils"
  FILES_${PN}-utils = "${bindir}/host ${bindir}/dig"
  FILES_${PN}-dev += "${bindir}/isc-config.h"
  FILES_${PN} += "${sbindir}/generate-rndc-key.sh ${PYTHON_SITEPACKAGES_DIR}"
+FILES_${PN} += "/usr/lib/python2.7/site-packages/*"
  
  PACKAGE_BEFORE_PN += "${PN}-libs"

  FILES_${PN}-libs = "${libdir}/*.so*"
@@ -88,7 +88,7 @@ do_install_append() {
install -d "${D}${sysconfdir}/init.d"
install -m 644 ${S}/conf/* "${D}${sysconfdir}/bind/"
install -m 755 "${S}/init.d" "${D}${sysconfdir}/init.d/bind"
-   sed -i -e '1s,#!.*python3,#! /usr/bin/python3,' 
${D}${sbindir}/dnssec-coverage ${D}${sbindir}/dnssec-checkds
+   sed -i -e '1s,#!.*python,#! /usr/bin/python3,' 
${D}${sbindir}/dnssec-coverage ${D}${sbindir}/dnssec-checkds
  
  	# Install systemd related files

install -d ${D}${sbindir}
@@ -120,4 +120,3 @@ CONFFILES_${PN} = " \
${sysconfdir}/bind/db.local \
${sysconfdir}/bind/db.root \
"
-



--
Kind regards,
Martin Hundebøll
Embedded Linux Consultant

+45 61 65 54 61
mar...@geanix.com

Geanix IVS
DK39600706
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [v2][PATCH] bind: update to 9.12.1

[Martin Hundebøll]

Hi Armin,

Thanks for picking this up.

On 2018-05-23 05:07, Armin Kuster wrote:

[v2]
Remove python3 in default PACKACKECONFIG
include https://patchwork.openembedded.org/patch/148257/
Break the hard runtime dependency on python3-core by adding a
(default-on) packageconfig option for python.


Because of the newly added dependency on meta-python, the python(3) 
packageconfig is no longer "default-on".




Signed-off-by: Martin Hundebøll <m...@prevas.dk>

LIC_FILES_CHKSUM changed do to updated year
Requires python-ply package in core

removed:
dont-test-on-host.patch, no longer impliemented
drop use-python3-and-fix-install-lib-path.patch, they added the ability to pass 
in lib dir loctions
drop bind-confgen-build-unix.o-once.patch, fix included in update

Refresh other patches:
add python3 flag for PACKAGECONFIG to pull in python
add new config option --with-eddsa=

Signed-off-by: Armin Kuster <akuster...@gmail.com>
---
  ...0001-build-use-pkg-config-to-find-libxml2.patch | 13 +++---
  ...-gen.c-extend-DIRNAMESIZE-from-256-to-512.patch | 13 +++---
  .../0001-lib-dns-gen.c-fix-too-long-error.patch| 13 +++---
  .../bind/bind/bind-confgen-build-unix.o-once.patch | 48 --
  ...-searching-for-json-headers-searches-sysr.patch | 13 +++---
  .../bind/bind/dont-test-on-host.patch  | 17 
  .../use-python3-and-fix-install-lib-path.patch | 36 
  .../bind/{bind_9.10.6.bb => bind_9.12.1.bb}| 31 --
  8 files changed, 39 insertions(+), 145 deletions(-)
  delete mode 100644 
meta/recipes-connectivity/bind/bind/bind-confgen-build-unix.o-once.patch
  delete mode 100644 meta/recipes-connectivity/bind/bind/dont-test-on-host.patch
  delete mode 100644 
meta/recipes-connectivity/bind/bind/use-python3-and-fix-install-lib-path.patch
  rename meta/recipes-connectivity/bind/{bind_9.10.6.bb => bind_9.12.1.bb} (79%)






diff --git a/meta/recipes-connectivity/bind/bind_9.10.6.bb 
b/meta/recipes-connectivity/bind/bind_9.12.1.bb
similarity index 79%
rename from meta/recipes-connectivity/bind/bind_9.10.6.bb
rename to meta/recipes-connectivity/bind/bind_9.12.1.bb
index 20c8d7b..52480f4 100644
--- a/meta/recipes-connectivity/bind/bind_9.10.6.bb
+++ b/meta/recipes-connectivity/bind/bind_9.12.1.bb
@@ -3,42 +3,38 @@ HOMEPAGE = "http://www.isc.org/sw/bind/;
  SECTION = "console/network"
  
  LICENSE = "ISC & BSD"

-LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=dba46507446198119bcde32a4feaab43"
+LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=6ba7c9fe0c888a943c79c93e6de744fb"
  
-DEPENDS = "openssl libcap"

+DEPENDS = "openssl libcap zlib"
  
  SRC_URI = "https://ftp.isc.org/isc/bind9/${PV}/${BPN}-${PV}.tar.gz \

 file://conf.patch \
 file://make-etc-initd-bind-stop-work.patch \
-   file://dont-test-on-host.patch \
 file://generate-rndc-key.sh \
 file://named.service \
 file://bind9 \
 file://init.d-add-support-for-read-only-rootfs.patch \
-   file://bind-confgen-build-unix.o-once.patch \
 file://0001-build-use-pkg-config-to-find-libxml2.patch \
 file://bind-ensure-searching-for-json-headers-searches-sysr.patch \
 file://0001-gen.c-extend-DIRNAMESIZE-from-256-to-512.patch \
 file://0001-lib-dns-gen.c-fix-too-long-error.patch \
-   file://use-python3-and-fix-install-lib-path.patch \
 "
  
-SRC_URI[md5sum] = "84e663284b17aee0df1ce6f248b137d7"

-SRC_URI[sha256sum] = 
"17bbcd2bd7b1d32f5ba4b30d5dbe8a39bce200079048073d1e0d050fdf47e69d"
+SRC_URI[md5sum] = "ec20f2d6aff50651296fad962d19ccfa"
+SRC_URI[sha256sum] = 
"16e446425c35e48b651ddf1171d2115ebf32b07670b652e1030a174038937510"
  
  UPSTREAM_CHECK_URI = "https://ftp.isc.org/isc/bind9/;

  UPSTREAM_CHECK_REGEX = "(?P9(\.\d+)+(-P\d+)*)/"
  
-

  ENABLE_IPV6 = "--enable-ipv6=${@bb.utils.contains('DISTRO_FEATURES', 'ipv6', 
'yes', 'no', d)}"
  EXTRA_OECONF = " ${ENABLE_IPV6} --with-libtool --enable-threads \
   --disable-devpoll --enable-epoll --with-gost=no \
- --with-gssapi=no --with-ecdsa=yes \
+ --with-gssapi=no --with-ecdsa=yes --with-eddsa=no\
   --sysconfdir=${sysconfdir}/bind \
   --with-openssl=${STAGING_DIR_HOST}${prefix} \
 "
  
-inherit autotools update-rc.d systemd useradd pkgconfig python3-dir

+inherit autotools update-rc.d systemd useradd pkgconfig
  
  export PYTHON_SITEPACKAGES_DIR
  
@@ -48,6 +44,13 @@ PACKAGECONFIG[httpstats] = "--with-libxml2,--without-libxml2,libxml2"

  PACKAGECONFIG[readline] = "--with-readline=-lreadline,,readline"
  PACKAGECONFIG[libedit] = "--with-readline=-ledit,,libedit"
  PACKAGECONFIG[urandom] = 
"--with-randomdev=/dev/ur

Re: [OE-core] [PATCH 0/2] bind update req. new pkg in core

[Martin Hundebøll]

On 2018-05-22 03:06, Khem Raj wrote:


On Mon, May 21, 2018 at 3:44 PM Armin Kuster <akuster...@gmail.com 
<mailto:akuster...@gmail.com>> wrote:


With this update, the python-ply package is required.
I copied the one from meta-python to core. Once this hits master,
I will send a patch to remove the same recipe from
meta-python.


Can this be turned on/off using packageconfig ?
If so then it would be ideal


I did send a patch for (some of) this some time ago:
https://patchwork.openembedded.org/patch/148257/

// Martin

If not then please send the removal patches for meta-python regardless 
so it can be tested together




Armin Kuster (2):
   bind: update to 9.12.1
   python3-ply: add package needed by bind 9.12 update

  ...0001-build-use-pkg-config-to-find-libxml2.patch | 13 +++---
  ...-gen.c-extend-DIRNAMESIZE-from-256-to-512.patch | 13 +++---
  .../0001-lib-dns-gen.c-fix-too-long-error.patch    | 13 +++---
  .../bind/bind/bind-confgen-build-unix.o-once.patch | 48
--
  ...-searching-for-json-headers-searches-sysr.patch | 13 +++---
  .../bind/bind/dont-test-on-host.patch              | 17 
  .../use-python3-and-fix-install-lib-path.patch     | 36

  .../bind/{bind_9.10.6.bb <http://bind_9.10.6.bb> => bind_9.12.1.bb
<http://bind_9.12.1.bb>}        | 21 --
  meta/recipes-devtools/python/python-ply.inc        | 18 
  meta/recipes-devtools/python/python-ply_3.11.bb
<http://python-ply_3.11.bb>    |  2 +
  10 files changed, 49 insertions(+), 145 deletions(-)
  delete mode 100644
meta/recipes-connectivity/bind/bind/bind-confgen-build-unix.o-once.patch
  delete mode 100644
meta/recipes-connectivity/bind/bind/dont-test-on-host.patch
  delete mode 100644

meta/recipes-connectivity/bind/bind/use-python3-and-fix-install-lib-path.patch
  rename meta/recipes-connectivity/bind/{bind_9.10.6.bb
<http://bind_9.10.6.bb> => bind_9.12.1.bb <http://bind_9.12.1.bb>} (86%)
  create mode 100644 meta/recipes-devtools/python/python-ply.inc
  create mode 100644 meta/recipes-devtools/python/python-ply_3.11.bb
<http://python-ply_3.11.bb>

-- 
2.7.4


-- 
___

Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
<mailto:Openembedded-core@lists.openembedded.org>
http://lists.openembedded.org/mailman/listinfo/openembedded-core





--
MARTIN HUNDEBØLL, Prevas A/S
Software Developer

Hedeager 3, DK-8200 Aarhus N
Phone +45 87438070
Mobile +45 25562438
martin.hundeb...@prevas.dk
www.prevas.com
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] u-boot: Upgrade to 2018.03 release

[Martin Hundebøll]

On April 30, 2018 7:25:10 PM GMT+02:00, Marek Vasut  wrote:
>On 04/30/2018 02:17 PM, Alexander Kanavin wrote:
>> On 04/16/2018 11:40 AM, Marek Vasut wrote:
 Isn't it possible to solve your issue in the OE side?

>>>
>>> Can we NOT compile DTC alongside U-Boot somehow ? :)
>>> That'd be the most desired solution IMO.
>> 
>> Yes please. Can you adjust the recipes and classes in oe-core, and
>> remove the dtc recipe altogether?
>
>Remove dtc recipe ? Not sure I understand what you're implying.

I guess Alexander misunderstood the "alongside U-Boot" part of your suggestion.

I agree that building dtc/libfdt in the U-Boot build instead of just using 
whatever is already available on the host/OE is undesirable.

But please elaborate if I am mistaken.

// Martin


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Martin Hundebøll]

Hi Lukasz,

On 2018-04-27 16:51, Lukasz Majewski wrote:

This commit provides the ability to generate u-boot environment(s) as
images, which afterwards can be used to produce image (with wic) for
flashing (eMMC or SPI-NOR).

This change removes the need to run "env default" during production phase,
as proper environment (including redundant one) is already stored on
persistent memory (the CRC is also correct).


I think we should create a separate recipe to install the native 
mkenvimage binary (e.g. u-boot-mkenvimage_%.bb) or update 
u-boot-mkimage_%.bb install it.


Then a new recipe to create the environment images can depend on 
u-boot-mkenvimage-native.


Also note the recently added upstream support for external environment 
definitions:

http://git.denx.de/?p=u-boot.git;a=commit;h=f3d8f7dd73ac5dde258eb786d4a01869395b56d7

For our usecase we need the ability to generate environment images in 
yocto from such external definitions.


// Martin



Signed-off-by: Lukasz Majewski 

---
This patch depends on "u-boot: Upgrade to 2018.03 release"
https://patchwork.openembedded.org/patch/149998/
---
  meta/recipes-bsp/u-boot/u-boot.inc | 35 +++
  1 file changed, 35 insertions(+)

diff --git a/meta/recipes-bsp/u-boot/u-boot.inc 
b/meta/recipes-bsp/u-boot/u-boot.inc
index c2bcf99840..2796e503cf 100644
--- a/meta/recipes-bsp/u-boot/u-boot.inc
+++ b/meta/recipes-bsp/u-boot/u-boot.inc
@@ -305,3 +305,38 @@ do_deploy () {
  }
  
  addtask deploy before do_build after do_compile

+
+# Create new rules to extract default envs
+UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
+DEFAULT_ENVS ?= "u-boot-env-default.txt"
+DEFAULT_ENVS_SIZE ?= "65536"
+
+# Generate default environment
+do_gen_default_envs[doc] = "Generate image with default U-Boot environment(s)"
+do_gen_default_envs () {
+   ${B}/source/scripts/get_default_envs.sh ${B} > ${B}/${DEFAULT_ENVS}
+
+   # Generate env image
+   ${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o 
${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}
+
+   # Generate redundant env image
+   ${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o 
${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}
+
+   rm ${B}/${DEFAULT_ENVS}
+}
+
+addtask gen_default_envs before do_deploy_default_envs after do_compile
+
+# Deploy default environment
+do_deploy_default_envs[doc] = "Copy images with default U-Boot environment to 
deployment directory"
+do_deploy_default_envs () {
+   install -d ${DEPLOYDIR}
+
+   install ${B}/${UBOOT_ENVS_DEFAULT} ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
+   install ${B}/${UBOOT_ENVS_DEFAULT}_r 
${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r
+
+   rm ${B}/${UBOOT_ENVS_DEFAULT}
+   rm ${B}/${UBOOT_ENVS_DEFAULT}_r
+}
+
+addtask deploy_default_envs before do_deploy after do_gen_default_envs


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] u-boot: Upgrade to 2018.03 release

[Martin Hundebøll]

Hi,

On 2018-04-11 04:28, yamada.masah...@socionext.com wrote:

Hi.


I had to patch up our own u-boot recipe as shown in the attached patch
to make v2018.03 compile for qemu-x86.

The thing is that the build of pylibfdt became unconditional since
15b97f5c5e ('pylibfdt: move pylibfdt to scripts/dtc/pylibfdt and
refactor makefile')

In my case u-boot/pylibfdt failed to find the correct (native) headers,
because python setuptools / distutils looks into STAGING_{LIB,INC}DIR
when compiling the native extension for libfdt. I didn't find any other
way to specificy this in either the u-boot Makefile or some other magic
environment variable.


CCing Yamada-san, maybe he has an idea.
Also, do not top post.




The build of pylibfdt is conditional.
It is built only when CONFIG_PYLIBFDT=y,
which is selected by CONFIG_DTOC.

If you really do not need pylibfdt,
you can disable it by tweaking the configuration.

If you need pylibfdt but you cannot build it,
it is a different problem.

Thanks.


Correct. But I was building u-boot.rom for qemu-x86, which depends on 
binman and thus pylibfdt. Before the mentioned commit, one could avoid 
the building of pylibfdt by installing it on the host, as the makerule 
checked if python could already import pylibfdt. This check is now removed.


Or am I missing something?

// Martin




-Original Message-
From: Marek Vasut [mailto:ma...@denx.de]
Sent: Tuesday, April 10, 2018 6:55 PM
To: Martin Hundebøll <m...@prevas.dk>; Khem Raj <raj.k...@gmail.com>
Cc: Otavio Salvador <ota...@ossystems.com.br>;
openembedded-core@lists.openembedded.org; Yamada, Masahiro/山田 真弘
<yamada.masah...@socionext.com>
Subject: Re: [OE-core] [PATCH] u-boot: Upgrade to 2018.03 release

On 04/10/2018 11:49 AM, Martin Hundebøll wrote:

Hi,


Hi,


I had to patch up our own u-boot recipe as shown in the attached patch
to make v2018.03 compile for qemu-x86.

The thing is that the build of pylibfdt became unconditional since
15b97f5c5e ('pylibfdt: move pylibfdt to scripts/dtc/pylibfdt and
refactor makefile')

In my case u-boot/pylibfdt failed to find the correct (native) headers,
because python setuptools / distutils looks into STAGING_{LIB,INC}DIR
when compiling the native extension for libfdt. I didn't find any other
way to specificy this in either the u-boot Makefile or some other magic
environment variable.


CCing Yamada-san, maybe he has an idea.
Also, do not top post.


// Martin

On 2018-04-09 02:54, Khem Raj wrote:


On Sun, Apr 8, 2018 at 4:10 PM Marek Vasut <ma...@denx.de
<mailto:ma...@denx.de>> wrote:

     This upgrades the U-Boot from 2018.01 to 2018.03 release and drops
     patches accepted upstream, getting the patch count to zero.



It fails for me if I have libfdt installed on build host
It seems build is doing the include dance which goes all the way into
hosts system include directories



     Signed-off-by: Marek Vasut <ma...@denx.de <mailto:ma...@denx.de>>
     Cc: Otavio Salvador <ota...@ossystems.com.br
     <mailto:ota...@ossystems.com.br>>
     Cc: Ross Burton <ross.bur...@intel.com
<mailto:ross.bur...@intel.com>>
     Cc: Richard Purdie <richard.pur...@linuxfoundation.org
     <mailto:richard.pur...@linuxfoundation.org>>
     ---
   .../u-boot/files/MPC8315ERDB-enable-DHCP.patch        | 19
     ---
   ...t-common_2018.01.inc => u-boot-common_2018.03.inc} |  6

++

   ...fw-utils_2018.01.bb <http://fw-utils_2018.01.bb> =>
     u-boot-fw-utils_2018.03.bb <http://u-boot-fw-utils_2018.03.bb>}

|  0

   ...t-mkimage_2018.01.bb <http://t-mkimage_2018.01.bb> =>
     u-boot-mkimage_2018.03.bb <http://u-boot-mkimage_2018.03.bb>}

|  0

   .../u-boot/{u-boot_2018.01.bb <http://u-boot_2018.01.bb> =>
     u-boot_2018.03.bb <http://u-boot_2018.03.bb>}   |  0
   5 files changed, 2 insertions(+), 23 deletions(-)
   delete mode 100644
     meta/recipes-bsp/u-boot/files/MPC8315ERDB-enable-DHCP.patch
   rename meta/recipes-bsp/u-boot/{u-boot-common_2018.01.inc =>
     u-boot-common_2018.03.inc} (68%)
   rename meta/recipes-bsp/u-boot/{u-boot-fw-utils_2018.01.bb
     <http://u-boot-fw-utils_2018.01.bb> =>

u-boot-fw-utils_2018.03.bb

     <http://u-boot-fw-utils_2018.03.bb>} (100%)
   rename meta/recipes-bsp/u-boot/{u-boot-mkimage_2018.01.bb
     <http://u-boot-mkimage_2018.01.bb> => u-boot-mkimage_2018.03.bb
     <http://u-boot-mkimage_2018.03.bb>} (100%)
   rename meta/recipes-bsp/u-boot/{u-boot_2018.01.bb
     <http://u-boot_2018.01.bb> => u-boot_2018.03.bb
     <http://u-boot_2018.03.bb>} (100%)

     diff --git
     a/meta/recipes-bsp/u-boot/files/MPC8315ERDB-enable-DHCP.patch
     b/meta/recipes-bsp/u-boot/files/MPC8315ERDB-enable-DHCP.patch
     deleted file mode 100644
     index ecaa1796a4..00
     ---

a/meta/recipes-bsp/u-boot/files/MP

Re: [OE-core] [PATCH] u-boot: Upgrade to 2018.03 release

[Martin Hundebøll]

Hi,

I had to patch up our own u-boot recipe as shown in the attached patch 
to make v2018.03 compile for qemu-x86.


The thing is that the build of pylibfdt became unconditional since 
15b97f5c5e ('pylibfdt: move pylibfdt to scripts/dtc/pylibfdt and 
refactor makefile')


In my case u-boot/pylibfdt failed to find the correct (native) headers, 
because python setuptools / distutils looks into STAGING_{LIB,INC}DIR 
when compiling the native extension for libfdt. I didn't find any other 
way to specificy this in either the u-boot Makefile or some other magic 
environment variable.


// Martin

On 2018-04-09 02:54, Khem Raj wrote:


On Sun, Apr 8, 2018 at 4:10 PM Marek Vasut > wrote:


This upgrades the U-Boot from 2018.01 to 2018.03 release and drops
patches accepted upstream, getting the patch count to zero.



It fails for me if I have libfdt installed on build host
It seems build is doing the include dance which goes all the way into 
hosts system include directories




Signed-off-by: Marek Vasut >
Cc: Otavio Salvador >
Cc: Ross Burton >
Cc: Richard Purdie >
---
  .../u-boot/files/MPC8315ERDB-enable-DHCP.patch        | 19
---
  ...t-common_2018.01.inc => u-boot-common_2018.03.inc} |  6 ++
  ...fw-utils_2018.01.bb  =>
u-boot-fw-utils_2018.03.bb } |  0
  ...t-mkimage_2018.01.bb  =>
u-boot-mkimage_2018.03.bb } |  0
  .../u-boot/{u-boot_2018.01.bb  =>
u-boot_2018.03.bb }   |  0
  5 files changed, 2 insertions(+), 23 deletions(-)
  delete mode 100644
meta/recipes-bsp/u-boot/files/MPC8315ERDB-enable-DHCP.patch
  rename meta/recipes-bsp/u-boot/{u-boot-common_2018.01.inc =>
u-boot-common_2018.03.inc} (68%)
  rename meta/recipes-bsp/u-boot/{u-boot-fw-utils_2018.01.bb
 => u-boot-fw-utils_2018.03.bb
} (100%)
  rename meta/recipes-bsp/u-boot/{u-boot-mkimage_2018.01.bb
 => u-boot-mkimage_2018.03.bb
} (100%)
  rename meta/recipes-bsp/u-boot/{u-boot_2018.01.bb
 => u-boot_2018.03.bb
} (100%)

diff --git
a/meta/recipes-bsp/u-boot/files/MPC8315ERDB-enable-DHCP.patch
b/meta/recipes-bsp/u-boot/files/MPC8315ERDB-enable-DHCP.patch
deleted file mode 100644
index ecaa1796a4..00
--- a/meta/recipes-bsp/u-boot/files/MPC8315ERDB-enable-DHCP.patch
+++ /dev/null
@@ -1,19 +0,0 @@
-Enabled dhcp client functionality for Yocto reference
-hardware MPC8315E-RDB.
-
-Upstream-Status: Pending
-
-Signed-off-by: Ed Bartosh >
-
-Index: git/configs/MPC8315ERDB_defconfig
-===
 git.orig/configs/MPC8315ERDB_defconfig
-+++ git/configs/MPC8315ERDB_defconfig
-@@ -12,6 +12,7 @@ CONFIG_CMD_PCI=y
- CONFIG_CMD_SATA=y
- CONFIG_CMD_USB=y
- # CONFIG_CMD_SETEXPR is not set
-+CONFIG_CMD_DHCP=y
- CONFIG_CMD_MII=y
- CONFIG_CMD_PING=y
- CONFIG_CMD_DATE=y
diff --git a/meta/recipes-bsp/u-boot/u-boot-common_2018.01.inc
b/meta/recipes-bsp/u-boot/u-boot-common_2018.03.inc
similarity index 68%
rename from meta/recipes-bsp/u-boot/u-boot-common_2018.01.inc
rename to meta/recipes-bsp/u-boot/u-boot-common_2018.03.inc
index d2073ea0c7..8ffae365a9 100644
--- a/meta/recipes-bsp/u-boot/u-boot-common_2018.01.inc
+++ b/meta/recipes-bsp/u-boot/u-boot-common_2018.03.inc
@@ -7,10 +7,8 @@ PE = "1"

  # We use the revision in order to avoid having to fetch it from the
  # repo during parse
-SRCREV = "f3dd87e0b98999a78e500e8c6d2b063ebadf535a"
+SRCREV = "f95ab1fb6e37f0601f397091bb011edf7a98b890"

-SRC_URI = "git://git.denx.de/u-boot.git
 \
-    file://MPC8315ERDB-enable-DHCP.patch \
-"
+SRC_URI = "git://git.denx.de/u-boot.git
"

  S = "${WORKDIR}/git"
diff --git a/meta/recipes-bsp/u-boot/u-boot-fw-utils_2018.01.bb

b/meta/recipes-bsp/u-boot/u-boot-fw-utils_2018.03.bb

similarity index 100%
rename from meta/recipes-bsp/u-boot/u-boot-fw-utils_2018.01.bb

rename to 

Re: [OE-core] [RFC PATCH] devicetree.bbclass: User/BSP device tree source compilation class

[Martin Hundebøll]

Acked-by: Martin Hundebøll <m...@prevas.dk>

Thanks Nathan; this could be very useful for us.

// Martin

On 2018-04-05 12:48, Nathan Rossi wrote:

This bbclass implements the device tree compilation for user provided
device trees. In order to use this class, it should be inherited in a
BSP recipe which provides the sources. The default setup enables
inclusion of kernel device tree sources (though can be disabled by the
recipe by overriding DT_INCLUDE or KERNEL_INCLUDE).

This provides an additional mechanism for BSPs to provide device trees
and device tree overlays for their target machines. Whilst still
enabling access to the kernel device trees for base SoC includes and
headers.

This approach to providing device trees has benefits for certain use
cases over patching the device trees into the kernel source.

* device trees are separated from kernel source, allows for selection of
kernel and or kernel versions without needing to explicitly patch the
kernel (or appending to the kernel recipes).

* providing device trees from separate sources, from the layer,
generated by the recipe or other recipes.

This class also implements some additional features that are not
available in the kernel-devicetree flow. This includes population of
device tree blobs into the sysroot which allows for other recipes to
consume built dtbs (e.g. U-Boot with EXT_DTB compilation), device tree
overlay compilation and customizing DTC compilation args (boot
cpu/padding/etc.).
---
This patch is an RFC for adding devicetree.bbclass to oe-core. With the
plan to submit it for inclusion in the next release.

In addition to these changes it is my intention to also submit
documentation to the Yocto BSP guide that covers both methods (kernel
source patch and this bbclass) for how to provide user device trees for a
BSP layer.

The meta-xilinx layer has a recipe (device-tree.bb) that behaves exactly
like this class for some time. It has been very useful for out-of-kernel
device tree compilation as well as for a mechanism so that users of
Xilinx SoCs can provide their customizations (FPGA devices) and board
specific configuration. This compilation flow also makes sense for other
BSPs, which is the reason for creating this bbclass for common shared
use by other BSPs.

Examples of this classes usage can be seen for meta-xilinx:
https://github.com/nathanrossi/meta-xilinx/blob/nrossi/devicetree-classify/meta-xilinx-bsp/recipes-bsp/device-tree/device-tree.bb

And for a custom BSP:
https://github.com/nathanrossi/gps-clock-project/blob/wip-devicetree-classify/meta-gps-clock/recipes/device-tree/device-tree.bb
---
  meta/classes/devicetree.bbclass | 140 
  1 file changed, 140 insertions(+)
  create mode 100644 meta/classes/devicetree.bbclass

diff --git a/meta/classes/devicetree.bbclass b/meta/classes/devicetree.bbclass
new file mode 100644
index 00..dbc83f2a1d
--- /dev/null
+++ b/meta/classes/devicetree.bbclass
@@ -0,0 +1,140 @@
+# This bbclass implements device tree compliation for user provided device tree
+# sources. The compilation of the device tree sources is the same as the kernel
+# device tree compilation process, this includes being able to include sources
+# from the kernel such as soc dtsi files or header files such as gpio.h. In
+# addition to device trees this bbclass also handles compilation of device tree
+# overlays.
+#
+# The output of this class behaves similar to how kernel-devicetree.bbclass
+# operates in that the output files are installed into /boot/devicetree.
+# However this class on purpose separates the deployed device trees into the
+# 'devicetree' subdirectory. This prevents clashes with the kernel-devicetree
+# output. Additionally the device trees are populated into the sysroot for
+# access via the sysroot from within other recipes.
+
+SECTION ?= "bsp"
+
+# The default inclusion of kernel device tree includes and headers means that
+# device trees built with them are at least GPLv2 (and in some cases dual
+# licensed). Default to GPLv2 if the recipe does not specify a license.
+LICENSE ?= "GPLv2"
+LIC_FILES_CHKSUM ?= 
"file://${COMMON_LICENSE_DIR}/GPL-2.0;md5=801f80980d171dd6425610833a22dbe6"
+
+INHIBIT_DEFAULT_DEPS = "1"
+DEPENDS += "dtc-native"
+
+inherit deploy kernel-arch
+
+COMPATIBLE_MACHINE ?= "^$"
+
+PACKAGE_ARCH = "${MACHINE_ARCH}"
+
+SYSROOT_DIRS += "/boot/devicetree"
+FILES_${PN} = "/boot/devicetree/*.dtb /boot/devicetree/*.dtbo"
+
+S = "${WORKDIR}"
+B = "${WORKDIR}/build"
+
+# Default kernel includes, these represent what are normally used for in-kernel
+# sources.
+KERNEL_INCLUDE ??= " \
+${STAGING_KERNEL_DIR}/arch/${ARCH}/boot/dts \
+${STAGING_KERNEL_DIR}/arch/${ARCH}/boot/dts/* \
+${STAGING_KERNEL_DIR}/scripts/dtc/include-prefixes \
+"
+
+DT_INCLUDE[doc] = "Search paths to be made available to

Re: [OE-core] [PATCH] kernel.bbclass: avoid duplicates in KERNEL_IMAGETYPE_FOR_MAKE

[Martin Hundebøll]

Hi Andre,

On 2018-03-13 00:19, Andre McCurdy wrote:

Currently if KERNEL_IMAGETYPES contains both vmlinux and vmlinux.gz,
KERNEL_IMAGETYPE_FOR_MAKE will end up containing two copies of
vmlinux, which will result in two calls to "make vmlinux" from
kernel_do_compile().

Avoid duplicating vmlinux in KERNEL_IMAGETYPE_FOR_MAKE plus some
minor non-functional updates to formatting and comments.

Signed-off-by: Andre McCurdy
---
  meta/classes/kernel.bbclass | 28 +---
  1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index dc0152f..4bcaa61 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -62,31 +62,37 @@ python __anonymous () {
  type = d.getVar('KERNEL_IMAGETYPE') or ""
  alttype = d.getVar('KERNEL_ALT_IMAGETYPE') or ""
  types = d.getVar('KERNEL_IMAGETYPES') or ""
-kname = d.getVar('KERNEL_PACKAGE_NAME') or "kernel"
  if type not in types.split():
  types = (type + ' ' + types).strip()
  if alttype not in types.split():
  types = (alttype + ' ' + types).strip()
  d.setVar('KERNEL_IMAGETYPES', types)
  
-# some commonly used kernel images aren't generated by the kernel build system, such as vmlinux.gz

-# typeformake lists only valid kernel make targets, and post processing 
can be done after the kernel
-# is built (such as using gzip to compress vmlinux)
-typeformake = types.replace('vmlinux.gz', 'vmlinux')
+# KERNEL_IMAGETYPES may contain a mixture of image types supported directly
+# by the kernel build system and others which are created by 
post-processing
+# (e.g. compressing vmlinux -> vmlinux.gz in kernel_do_compile()).
+# KERNEL_IMAGETYPE_FOR_MAKE should contain only image types supported
+# directly by the kernel build system.
+
+typeformake = ""
+for type in types.split():
+if type == 'vmlinux.gz':
+type = 'vmlinux'
+if type in types.split():
+continue
+typeformake = (typeformake + ' ' + type).strip()
+
  d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', typeformake)


I think this hunk go be better implemented using a python set(), e.g. 
something like this (untested) piece:


> typeformake = set()
> post_extensions = ('gz', 'xz', 'bz2')
> for type in types.split():
> extsplit = type.rsplit('.', 1)
> if len(extsplit) == 2 and extsplit[1] in post_extensions:
> typeformake.add(extsplit[0])
> else:
> typeformake.add(type)
>
> d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', " ".join(typeformake))

This should allow easier addition of other OE-supported extensions...

// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] kernel-fitimage.bbclass: Deploy fitImage without Initramfs

[Martin Hundebøll]

Hi Vineeth,

On 2018-03-07 15:41, Vineeth Chowdary Karumanchi wrote:

This patch deploys fitImage (linux.bin + dtb ).
The use case is to have 2 partioned sd card, with
FAT partition having bootloader + fitImage and
ext4 partion having rootfs.

Signen-off-by: Vineeth Chowdary Karumanchi 
---
  meta/classes/kernel-fitimage.bbclass | 5 +
  1 file changed, 5 insertions(+)

diff --git a/meta/classes/kernel-fitimage.bbclass 
b/meta/classes/kernel-fitimage.bbclass
index 50a91e1..25884d5 100644
--- a/meta/classes/kernel-fitimage.bbclass
+++ b/meta/classes/kernel-fitimage.bbclass
@@ -464,6 +464,10 @@ kernel_do_deploy_append() {
 
linux_bin_base_name="fitImage-linux.bin-${PV}-${PR}-${MACHINE}-${DATETIME}"
 linux_bin_symlink_name=fitImage-linux.bin-${MACHINE}
 install -m 0644 linux.bin 
${DEPLOYDIR}/${linux_bin_base_name}.bin
+   fitimage_base_name="fitImage-${PV}-${PR}-${MACHINE}-${DATETIME}"
+   fitimage_symlink_name=fitImage-${MACHINE}
+   install -m 0644 arch/${ARCH}/boot/fitImage 
${DEPLOYDIR}/${fitimage_base_name}.bin
+


Isn't this already handled by the "linux_bin_base_name" variable just 
above (i.e. fitImahe-linux.bin-${MACHINE}). I have various fitimages, 
and as far as I recall, a non-initramfs fitimage is already deployed.


// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] bind: add packageconfig for python-support

[Martin Hundebøll]

From: Martin Hundeboll <m...@prevas.dk>

Break the hard runtime dependency on python3-core by adding a
(default-on) packageconfig option for python.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>

---
Changes in v2:

- Remove broken and unneeded build-dependency on python3(-core)

 meta/recipes-connectivity/bind/bind_9.10.6.bb | 14 ++
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-connectivity/bind/bind_9.10.6.bb 
b/meta/recipes-connectivity/bind/bind_9.10.6.bb
index 8b8835ba803..7af92a4fdab 100644
--- a/meta/recipes-connectivity/bind/bind_9.10.6.bb
+++ b/meta/recipes-connectivity/bind/bind_9.10.6.bb
@@ -38,16 +38,19 @@ EXTRA_OECONF = " ${ENABLE_IPV6} --with-libtool 
--enable-threads \
  --with-openssl=${STAGING_LIBDIR}/.. \
"
 
-inherit autotools update-rc.d systemd useradd pkgconfig python3-dir
+inherit autotools update-rc.d systemd useradd pkgconfig
 
 export PYTHON_SITEPACKAGES_DIR
 
 # PACKAGECONFIGs readline and libedit should NOT be set at same time
-PACKAGECONFIG ?= "readline"
+PACKAGECONFIG ?= "readline python"
 PACKAGECONFIG[httpstats] = "--with-libxml2,--without-libxml2,libxml2"
 PACKAGECONFIG[readline] = "--with-readline=-lreadline,,readline"
 PACKAGECONFIG[libedit] = "--with-readline=-ledit,,libedit"
 PACKAGECONFIG[urandom] = 
"--with-randomdev=/dev/urandom,--with-randomdev=/dev/random,,"
+PACKAGECONFIG[python] = 
"--with-python=${PYTHON},--without-python,,python3-core"
+
+inherit ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3native', '', 
d)}
 
 USERADD_PACKAGES = "${PN}"
 USERADD_PARAM_${PN} = "--system --home ${localstatedir}/cache/bind 
--no-create-home \
@@ -60,7 +63,6 @@ SYSTEMD_SERVICE_${PN} = "named.service"
 
 PARALLEL_MAKE = ""
 
-RDEPENDS_${PN} = "python3-core"
 RDEPENDS_${PN}-dev = ""
 
 PACKAGE_BEFORE_PN += "${PN}-utils"
@@ -88,7 +90,11 @@ do_install_append() {
install -d "${D}${sysconfdir}/init.d"
install -m 644 ${S}/conf/* "${D}${sysconfdir}/bind/"
install -m 755 "${S}/init.d" "${D}${sysconfdir}/init.d/bind"
-   sed -i -e '1s,#!.*python3,#! /usr/bin/python3,' 
${D}${sbindir}/dnssec-coverage ${D}${sbindir}/dnssec-checkds
+
+   if [ -n "${@bb.utils.contains('PACKAGECONFIG', 'python', 'python', '', 
d)}" ]; then
+   sed -i -e '1s,#!.*python3,#! /usr/bin/python3,' \
+   ${D}${sbindir}/dnssec-coverage 
${D}${sbindir}/dnssec-checkds
+   fi
 
# Install systemd related files
install -d ${D}${sbindir}
-- 
2.16.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] bind: add packageconfig for python-support

[Martin Hundebøll]

Ping

On 2018-02-05 13:25, Martin Hundebøll wrote:

Break the hard runtime dependency on python3-core by adding a
(default-on) packageconfig option for python.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>
---
  meta/recipes-connectivity/bind/bind_9.10.6.bb | 14 ++
  1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-connectivity/bind/bind_9.10.6.bb 
b/meta/recipes-connectivity/bind/bind_9.10.6.bb
index 8b8835ba80..2ec7ab44f0 100644
--- a/meta/recipes-connectivity/bind/bind_9.10.6.bb
+++ b/meta/recipes-connectivity/bind/bind_9.10.6.bb
@@ -38,16 +38,19 @@ EXTRA_OECONF = " ${ENABLE_IPV6} --with-libtool 
--enable-threads \
   --with-openssl=${STAGING_LIBDIR}/.. \
 "
  
-inherit autotools update-rc.d systemd useradd pkgconfig python3-dir

+inherit autotools update-rc.d systemd useradd pkgconfig
  
  export PYTHON_SITEPACKAGES_DIR
  
  # PACKAGECONFIGs readline and libedit should NOT be set at same time

-PACKAGECONFIG ?= "readline"
+PACKAGECONFIG ?= "readline python"
  PACKAGECONFIG[httpstats] = "--with-libxml2,--without-libxml2,libxml2"
  PACKAGECONFIG[readline] = "--with-readline=-lreadline,,readline"
  PACKAGECONFIG[libedit] = "--with-readline=-ledit,,libedit"
  PACKAGECONFIG[urandom] = 
"--with-randomdev=/dev/urandom,--with-randomdev=/dev/random,,"
+PACKAGECONFIG[python] = 
"--with-python=${PYTHON},--without-python,python3-core,python3-core"
+
+inherit ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3native', '', 
d)}
  
  USERADD_PACKAGES = "${PN}"

  USERADD_PARAM_${PN} = "--system --home ${localstatedir}/cache/bind 
--no-create-home \
@@ -60,7 +63,6 @@ SYSTEMD_SERVICE_${PN} = "named.service"
  
  PARALLEL_MAKE = ""
  
-RDEPENDS_${PN} = "python3-core"

  RDEPENDS_${PN}-dev = ""
  
  PACKAGE_BEFORE_PN += "${PN}-utils"

@@ -88,7 +90,11 @@ do_install_append() {
install -d "${D}${sysconfdir}/init.d"
install -m 644 ${S}/conf/* "${D}${sysconfdir}/bind/"
install -m 755 "${S}/init.d" "${D}${sysconfdir}/init.d/bind"
-   sed -i -e '1s,#!.*python3,#! /usr/bin/python3,' 
${D}${sbindir}/dnssec-coverage ${D}${sbindir}/dnssec-checkds
+
+   if [ -n "${@bb.utils.contains('PACKAGECONFIG', 'python', 'python', '', 
d)}" ]; then
+   sed -i -e '1s,#!.*python3,#! /usr/bin/python3,' \
+   ${D}${sbindir}/dnssec-coverage 
${D}${sbindir}/dnssec-checkds
+   fi
  
  	# Install systemd related files

install -d ${D}${sbindir}


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] bind: add packageconfig for python-support

[Martin Hundebøll]

On 2018-02-05 13:49, Richard Purdie wrote:

On Mon, 2018-02-05 at 13:25 +0100, Martin Hundebøll wrote:

Break the hard runtime dependency on python3-core by adding a
(default-on) packageconfig option for python.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>
---
  meta/recipes-connectivity/bind/bind_9.10.6.bb | 14 ++
  1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-connectivity/bind/bind_9.10.6.bb
b/meta/recipes-connectivity/bind/bind_9.10.6.bb
index 8b8835ba80..2ec7ab44f0 100644
--- a/meta/recipes-connectivity/bind/bind_9.10.6.bb
+++ b/meta/recipes-connectivity/bind/bind_9.10.6.bb
@@ -38,16 +38,19 @@ EXTRA_OECONF = " ${ENABLE_IPV6} --with-libtool --
enable-threads \
   --with-openssl=${STAGING_LIBDIR}/.. \
 "
  
-inherit autotools update-rc.d systemd useradd pkgconfig python3-dir

+inherit autotools update-rc.d systemd useradd pkgconfig
  
  export PYTHON_SITEPACKAGES_DIR
  
  # PACKAGECONFIGs readline and libedit should NOT be set at same time

-PACKAGECONFIG ?= "readline"
+PACKAGECONFIG ?= "readline python"
  PACKAGECONFIG[httpstats] = "--with-libxml2,--without-
libxml2,libxml2"
  PACKAGECONFIG[readline] = "--with-readline=-lreadline,,readline"
  PACKAGECONFIG[libedit] = "--with-readline=-ledit,,libedit"
  PACKAGECONFIG[urandom] = "--with-randomdev=/dev/urandom,--with-
randomdev=/dev/random,,"
+PACKAGECONFIG[python] = "--with-python=${PYTHON},--without-
python,python3-core,python3-core"
+
+inherit ${@bb.utils.contains('PACKAGECONFIG', 'python',
'python3native', '', d)}
  
  USERADD_PACKAGES = "${PN}"

  USERADD_PARAM_${PN} = "--system --home ${localstatedir}/cache/bind
--no-create-home \
@@ -60,7 +63,6 @@ SYSTEMD_SERVICE_${PN} = "named.service"
  
  PARALLEL_MAKE = ""
  
-RDEPENDS_${PN} = "python3-core"

  RDEPENDS_${PN}-dev = ""
  
  PACKAGE_BEFORE_PN += "${PN}-utils"

@@ -88,7 +90,11 @@ do_install_append() {
    install -d "${D}${sysconfdir}/init.d"
    install -m 644 ${S}/conf/* "${D}${sysconfdir}/bind/"
    install -m 755 "${S}/init.d" "${D}${sysconfdir}/init.d/bind"
-   sed -i -e '1s,#!.*python3,#! /usr/bin/python3,'
${D}${sbindir}/dnssec-coverage ${D}${sbindir}/dnssec-checkds
+
+   if [ -n "${@bb.utils.contains('PACKAGECONFIG', 'python',
'python', '', d)}" ]; then
+   sed -i -e '1s,#!.*python3,#! /usr/bin/python3,' \
+   ${D}${sbindir}/dnssec-coverage
${D}${sbindir}/dnssec-checkds
+   fi


Rather than having a broken file, shouldn't it be deleted in the "no
python" case?


Bind is clever enough to avoid installing it in that case...

// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] usbutils: Split out lsusb.py

[Martin Hundebøll]

The pretty-printing "lsusb.py" script shipped by usbutils is currently
useless, as it doesn't runtime depend on python, and has unversioned
python in the shebang.

Avoid adding a python dependency to current configurations with usbutils
buy splitting lsusb.py into a usbutils-python package, and make it
runtime depend on python3-core.

Make the script usable by replacing the shebang with a direct call to
${bindir}/python3.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>
---
 meta/recipes-bsp/usbutils/usbutils_009.bb | 8 
 1 file changed, 8 insertions(+)

diff --git a/meta/recipes-bsp/usbutils/usbutils_009.bb 
b/meta/recipes-bsp/usbutils/usbutils_009.bb
index 30522fd0f1..a7cbe45406 100644
--- a/meta/recipes-bsp/usbutils/usbutils_009.bb
+++ b/meta/recipes-bsp/usbutils/usbutils_009.bb
@@ -22,3 +22,11 @@ FILES_${PN}-dev += "${datadir}/pkgconfig"
 
 RRECOMMENDS_${PN} = "udev-hwdb"
 RDEPENDS_${PN}-ptest = "libboost-system libboost-thread"
+
+PACKAGE_BEFORE_PN =+ "${PN}-python"
+FILES_${PN}-python += "${bindir}/lsusb.py"
+RDEPENDS_${PN}-python = "python3-core"
+
+do_install_append() {
+sed -i -E '1s,#!.+python,#!${bindir}/python3,' ${D}${bindir}/lsusb.py
+}
-- 
2.16.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] bind: add packageconfig for python-support

[Martin Hundebøll]

Break the hard runtime dependency on python3-core by adding a
(default-on) packageconfig option for python.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>
---
 meta/recipes-connectivity/bind/bind_9.10.6.bb | 14 ++
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-connectivity/bind/bind_9.10.6.bb 
b/meta/recipes-connectivity/bind/bind_9.10.6.bb
index 8b8835ba80..2ec7ab44f0 100644
--- a/meta/recipes-connectivity/bind/bind_9.10.6.bb
+++ b/meta/recipes-connectivity/bind/bind_9.10.6.bb
@@ -38,16 +38,19 @@ EXTRA_OECONF = " ${ENABLE_IPV6} --with-libtool 
--enable-threads \
  --with-openssl=${STAGING_LIBDIR}/.. \
"
 
-inherit autotools update-rc.d systemd useradd pkgconfig python3-dir
+inherit autotools update-rc.d systemd useradd pkgconfig
 
 export PYTHON_SITEPACKAGES_DIR
 
 # PACKAGECONFIGs readline and libedit should NOT be set at same time
-PACKAGECONFIG ?= "readline"
+PACKAGECONFIG ?= "readline python"
 PACKAGECONFIG[httpstats] = "--with-libxml2,--without-libxml2,libxml2"
 PACKAGECONFIG[readline] = "--with-readline=-lreadline,,readline"
 PACKAGECONFIG[libedit] = "--with-readline=-ledit,,libedit"
 PACKAGECONFIG[urandom] = 
"--with-randomdev=/dev/urandom,--with-randomdev=/dev/random,,"
+PACKAGECONFIG[python] = 
"--with-python=${PYTHON},--without-python,python3-core,python3-core"
+
+inherit ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3native', '', 
d)}
 
 USERADD_PACKAGES = "${PN}"
 USERADD_PARAM_${PN} = "--system --home ${localstatedir}/cache/bind 
--no-create-home \
@@ -60,7 +63,6 @@ SYSTEMD_SERVICE_${PN} = "named.service"
 
 PARALLEL_MAKE = ""
 
-RDEPENDS_${PN} = "python3-core"
 RDEPENDS_${PN}-dev = ""
 
 PACKAGE_BEFORE_PN += "${PN}-utils"
@@ -88,7 +90,11 @@ do_install_append() {
install -d "${D}${sysconfdir}/init.d"
install -m 644 ${S}/conf/* "${D}${sysconfdir}/bind/"
install -m 755 "${S}/init.d" "${D}${sysconfdir}/init.d/bind"
-   sed -i -e '1s,#!.*python3,#! /usr/bin/python3,' 
${D}${sbindir}/dnssec-coverage ${D}${sbindir}/dnssec-checkds
+
+   if [ -n "${@bb.utils.contains('PACKAGECONFIG', 'python', 'python', '', 
d)}" ]; then
+   sed -i -e '1s,#!.*python3,#! /usr/bin/python3,' \
+   ${D}${sbindir}/dnssec-coverage 
${D}${sbindir}/dnssec-checkds
+   fi
 
# Install systemd related files
install -d ${D}${sbindir}
-- 
2.16.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v8] do_image: Implement IMAGE_ROOTFS_EXCLUDE_PATH feature.

[Martin Hundebøll]

Acked-by: Martin Hundebøll <m...@prevas.dk>

On 2018-01-26 10:56, Kristian Amlie wrote:

This is a direct followup from the earlier 6602392db3d39 commit in
wic. It works more or less the same way: The variable specifies a list
of directories relative to the root of the rootfs, and these
directories will be excluded from the resulting rootfs image. If an
entry ends with a slash, only the contents are omitted, not the
directory itself.

Since the intended use of the variable is to exclude certain
directories from the rootfs, and then include said directories in
other partitions, it is not natural for this variable to be respected
for image creators that create multi partition images. These can turn
the feature off locally by defining:

   do_image_myfs[respect_exclude_path] = "0"

Specifically, "wic" and "multiubi" come with the feature disabled.

Signed-off-by: Kristian Amlie <kristian.am...@northern.tech>
---
  meta/classes/image.bbclass   | 84 +++-
  meta/classes/image_types.bbclass |  1 +
  meta/classes/image_types_wic.bbclass |  1 +
  3 files changed, 84 insertions(+), 2 deletions(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 4531aa2..6f61db3 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -117,7 +117,8 @@ def rootfs_variables(d):
   
'IMAGE_ROOTFS_MAXSIZE','IMAGE_NAME','IMAGE_LINK_NAME','IMAGE_MANIFEST','DEPLOY_DIR_IMAGE','IMAGE_FSTYPES','IMAGE_INSTALL_COMPLEMENTARY','IMAGE_LINGUAS',
   
'MULTILIBRE_ALLOW_REP','MULTILIB_TEMP_ROOTFS','MULTILIB_VARIANTS','MULTILIBS','ALL_MULTILIB_PACKAGE_ARCHS','MULTILIB_GLOBAL_VARIANTS','BAD_RECOMMENDATIONS','NO_RECOMMENDATIONS',
   
'PACKAGE_ARCHS','PACKAGE_CLASSES','TARGET_VENDOR','TARGET_ARCH','TARGET_OS','OVERRIDES','BBEXTENDVARIANT','FEED_DEPLOYDIR_BASE_URI','INTERCEPT_DIR','USE_DEVFS',
- 'CONVERSIONTYPES', 'IMAGE_GEN_DEBUGFS', 'ROOTFS_RO_UNNEEDED', 
'IMGDEPLOYDIR', 'PACKAGE_EXCLUDE_COMPLEMENTARY', 
'REPRODUCIBLE_TIMESTAMP_ROOTFS']
+ 'CONVERSIONTYPES', 'IMAGE_GEN_DEBUGFS', 'ROOTFS_RO_UNNEEDED', 
'IMGDEPLOYDIR', 'PACKAGE_EXCLUDE_COMPLEMENTARY', 
'REPRODUCIBLE_TIMESTAMP_ROOTFS',
+ 'IMAGE_ROOTFS_EXCLUDE_PATH']
  variables.extend(rootfs_command_variables(d))
  variables.extend(variable_depends(d))
  return " ".join(variables)
@@ -508,8 +509,9 @@ python () {
  d.setVarFlag(task, 'func', '1')
  d.setVarFlag(task, 'fakeroot', '1')
  
-d.appendVarFlag(task, 'prefuncs', ' ' + debug + ' set_image_size')

+d.appendVarFlag(task, 'prefuncs', ' ' + debug + ' set_image_size 
prepare_excluded_directories')
  d.prependVarFlag(task, 'postfuncs', ' create_symlinks')
+d.appendVarFlag(task, 'postfuncs', ' cleanup_excluded_directories')
  d.appendVarFlag(task, 'subimages', ' ' + ' '.join(subimages))
  d.appendVarFlag(task, 'vardeps', ' ' + ' '.join(vardeps))
  d.appendVarFlag(task, 'vardepsexclude', 'DATETIME DATE ' + ' 
'.join(vardepsexclude))
@@ -518,6 +520,84 @@ python () {
  bb.build.addtask(task, 'do_image_complete', after, d)
  }
  
+python prepare_excluded_directories() {

+exclude_var = d.getVar('IMAGE_ROOTFS_EXCLUDE_PATH')
+if not exclude_var:
+return
+
+taskname = d.getVar("BB_CURRENTTASK")
+
+if d.getVarFlag('do_%s' % taskname, 'respect_exclude_path') == '0':
+bb.debug(1, "'IMAGE_ROOTFS_EXCLUDE_PATH' is set but 'respect_exclude_path' 
variable flag is 0 for this image type, so ignoring it")
+return
+
+import shutil
+from oe.path import copyhardlinktree
+
+exclude_list = exclude_var.split()
+
+rootfs_orig = d.getVar('IMAGE_ROOTFS')
+# We need a new rootfs directory we can delete files from. Copy to
+# workdir.
+new_rootfs = os.path.realpath(oe.path.join(d.getVar("WORKDIR"), 
"rootfs.%s" % taskname))
+
+if os.path.lexists(new_rootfs):
+shutil.rmtree(new_rootfs)
+
+copyhardlinktree(rootfs_orig, new_rootfs)
+
+for orig_path in exclude_list:
+path = orig_path
+if os.path.isabs(path):
+bb.fatal("IMAGE_ROOTFS_EXCLUDE_PATH: Must be relative: %s" % 
orig_path)
+
+full_path = os.path.realpath(oe.path.join(new_rootfs, path))
+
+# Disallow climbing outside of parent directory using '..',
+# because doing so could be quite disastrous (we will delete the
+# directory).
+if not full_path.startswith(new_rootfs):
+bb.fatal("'%s' points to a path outside the rootfs" % orig_path)
+
+if path.endswith(os.sep):
+# Delete content only.
+for entry in os.listdir(full_path):
+full_entry = oe.path.join(full_path, entry)
+if os.path.isdir(full_entry) and not 
os.path.islink(f

Re: [OE-core] [PATCH v7] do_image: Implement IMAGE_ROOTFS_EXCLUDE_PATH feature.

[Martin Hundebøll]

  # Disallow climbing outside of parent directory using '..',
+# because doing so could be quite disastrous (we will delete the
+# directory).
+if not full_path.startswith(new_rootfs):
+bb.fatal("'%s' points to a path outside the rootfs" % orig_path)
+
+if path.endswith(os.sep):
+# Delete content only.
+for entry in os.listdir(full_path):
+full_entry = os.path.join(full_path, entry)
+if os.path.isdir(full_entry) and not 
os.path.islink(full_entry):
+shutil.rmtree(full_entry)
+else:
+os.remove(full_entry)
+else:
+# Delete whole directory.
+shutil.rmtree(full_path)
+
+# Save old value for cleanup later.
+d.setVar('IMAGE_ROOTFS_ORIG', rootfs_orig)
+d.setVar('IMAGE_ROOTFS', new_rootfs)
+}
+
+python cleanup_excluded_directories() {
+exclude_var = d.getVar('IMAGE_ROOTFS_EXCLUDE_PATH')
+if not exclude_var:
+return
+
+taskname = d.getVar("BB_CURRENTTASK")
+
+if d.getVarFlag('do_%s' % taskname, 'respect_exclude_path') == '0':
+return
+
+import shutil
+
+rootfs_dirs_excluded = d.getVar('IMAGE_ROOTFS')
+rootfs_orig = d.getVar('IMAGE_ROOTFS_ORIG')
+# This should never happen, since we should have set it to a different
+# directory in the prepare function.
+assert rootfs_dirs_excluded != rootfs_orig
+
+shutil.rmtree(rootfs_dirs_excluded)
+d.setVar('IMAGE_ROOTFS', rootfs_orig)
+}


Are we sure the cleanup is needed? I can image cases where I would like 
to have a look in "rootfs_dirs_excluded" to see what how it differs from 
"rootfs_orig".



+
  #
  # Compute the rootfs size
  #
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index c736f92..8cd8fd5 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -201,6 +201,7 @@ IMAGE_CMD_multiubi () {
multiubi_mkfs "${mkubifs_args}" "${ubinize_args}" "${name}"
done
  }
+do_image_multiubi[respect_exclude_path] = "0"
  
  IMAGE_CMD_ubi () {

multiubi_mkfs "${MKUBIFS_ARGS}" "${UBINIZE_ARGS}"
diff --git a/meta/classes/image_types_wic.bbclass 
b/meta/classes/image_types_wic.bbclass
index dcf620c..452d7f3 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -34,6 +34,7 @@ IMAGE_CMD_wic () {
rm -rf "$out/"
  }
  IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES TOPDIR"
+do_image_wic[respect_exclude_path] = "0"
  
  # Rebuild when the wks file or vars in WICVARS change

  USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s' 
% c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}"



--
MARTIN HUNDEBØLL, Prevas A/S
Software Developer

Hedeager 3, DK-8200 Aarhus N
Phone +45 87438070
Mobile +45 25562438
martin.hundeb...@prevas.dk
www.prevas.com
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] glib-2.0: fix EXTRA_OECONF append for PTEST

[Martin Hundebøll]

The order of '_append' and '_' matters: '_append' must be
before overrides to have any effect.

Fix this in glib-2.0, so that EXTRA_OECONF is appended instead of
overwritten.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>
---
 meta/recipes-core/glib-2.0/glib.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/glib-2.0/glib.inc 
b/meta/recipes-core/glib-2.0/glib.inc
index 354d95cfe3..af65d80d76 100644
--- a/meta/recipes-core/glib-2.0/glib.inc
+++ b/meta/recipes-core/glib-2.0/glib.inc
@@ -49,7 +49,7 @@ EXTRA_OECONF = "${PRINTF} ${CORECONF}"
 EXTRA_OECONF_class-native = "${CORECONF} --disable-selinux"
 
 # Tell configure that we'll have dbus-daemon on the target for the tests
-EXTRA_OECONF_class-target_append = " ${@bb.utils.contains('PTEST_ENABLED', 
'1', ' ac_cv_prog_DBUS_DAEMON=dbus-daemon', '', d)}"
+EXTRA_OECONF_append_class-target = " ${@bb.utils.contains('PTEST_ENABLED', 
'1', ' ac_cv_prog_DBUS_DAEMON=dbus-daemon', '', d)}"
 
 do_configure_prepend() {
sed -i -e '1s,#!.*,#!${USRBINPATH}/env python3,' 
${S}/gio/gdbus-2.0/codegen/gdbus-codegen.in
-- 
2.15.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH][rocko] wic: support filesystem label for rawcopy

[Martin Hundebøll]

Hi Armin,

Any idea about when this will land in the rocko branch?

Thanks,
Martin

On 2017-12-11 08:12, Martin Hundebøll wrote:

The '--label' argument should work for '--source rawcopy' as it does for
'--source rootfs', so add a method in RawCopyPlugin to update the label
on the temporary filesystem images.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>
Signed-off-by: Ross Burton <ross.bur...@intel.com>
---

Hi Armin,

Please consider this for Rocko.

Thanks

  scripts/lib/wic/plugins/source/rawcopy.py | 22 ++
  1 file changed, 22 insertions(+)

diff --git a/scripts/lib/wic/plugins/source/rawcopy.py 
b/scripts/lib/wic/plugins/source/rawcopy.py
index 424ed26ed6..e86398ac8f 100644
--- a/scripts/lib/wic/plugins/source/rawcopy.py
+++ b/scripts/lib/wic/plugins/source/rawcopy.py
@@ -32,6 +32,25 @@ class RawCopyPlugin(SourcePlugin):
  
  name = 'rawcopy'
  
+@staticmethod

+def do_image_label(fstype, dst, label):
+if fstype.startswith('ext'):
+cmd = 'tune2fs -L %s %s' % (label, dst)
+elif fstype in ('msdos', 'vfat'):
+cmd = 'dosfslabel %s %s' % (dst, label)
+elif fstype == 'btrfs':
+cmd = 'btrfs filesystem label %s %s' % (dst, label)
+elif fstype == 'swap':
+cmd = 'mkswap -L %s %s' % (label, dst)
+elif fstype == 'squashfs':
+raise WicError("It's not possible to update a squashfs "
+   "filesystem label '%s'" % (label))
+else:
+raise WicError("Cannot update filesystem label: "
+   "Unknown fstype: '%s'" % (fstype))
+
+exec_cmd(cmd)
+
  @classmethod
  def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
   oe_builddir, bootimg_dir, kernel_dir,
@@ -66,4 +85,7 @@ class RawCopyPlugin(SourcePlugin):
  if filesize > part.size:
  part.size = filesize
  
+if part.label:

+RawCopyPlugin.do_image_label(part.fstype, dst, part.label)
+
  part.source_file = dst


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH RFC 01/18] fs-uuid.bbclass: rewrite get_fs_uuid function

[Martin Hundebøll]

Hi,

On 2017-12-19 23:58, California Sullivan wrote:

I couldn't find anywhere it was being used, and in trying to use it it
just spit out an error.

It now generates a random UUID and puts it in the deploy dir for
consumption.

Also use DISK_SIGNATURE_UUID for the replacement function, since that
is user-overridable.

Signed-off-by: California Sullivan <california.l.sulli...@intel.com>
---
  meta/classes/fs-uuid.bbclass | 32 ++--
  1 file changed, 22 insertions(+), 10 deletions(-)

diff --git a/meta/classes/fs-uuid.bbclass b/meta/classes/fs-uuid.bbclass
index 9b53dfba7ab..5a96054294e 100644
--- a/meta/classes/fs-uuid.bbclass
+++ b/meta/classes/fs-uuid.bbclass
@@ -1,16 +1,26 @@
+inherit deploy
+
  # Extract UUID from ${ROOTFS}, which must have been built
  # by the time that this function gets called. Only works
  # on ext file systems and depends on tune2fs.
  def get_rootfs_uuid(d):
-import subprocess
-rootfs = d.getVar('ROOTFS')
-output = subprocess.check_output(['tune2fs', '-l', rootfs])
-for line in output.split('\n'):
-if line.startswith('Filesystem UUID:'):
-uuid = line.split()[-1]
-bb.note('UUID of %s: %s' % (rootfs, uuid))
-return uuid
-bb.fatal('Could not determine filesystem UUID of %s' % rootfs)
+import os
+import uuid
+deploydir = d.getVar('DEPLOY_DIR_IMAGE')
+bootfs_uuid_file = deploydir + "/bootfs-uuid"


Maybe use a default assigned class variable here instead of the 
hard-coded "bootfs-uuid". The class is named "get_rootfs_uuid", so I 
would expect it to handle multiple cases.



+uuid_str = "deadbeef-dead-beef-dead-beefdeadbeef"
+if d.getVar('BUILD_REPRODUCIBLE_BINARIES', True) != "1":
+if os.path.exists(bootfs_uuid_file):
+with open(bootfs_uuid_file, "r+") as file:
+uuid_str = file.read()
+else:
+# deploydir might not exist yet...creating it here could be bad?
+if not os.path.exists(deploydir):
+os.makedirs(deploydir)
+uuid_str = str(uuid.uuid4())
+with open(bootfs_uuid_file, "w") as file:
+file.write(uuid_str)
+return uuid_str
  
  # Replace the special <> inside a string (like the

  # root= APPEND string in a syslinux.cfg or systemd-boot entry) with the
@@ -19,6 +29,8 @@ def get_rootfs_uuid(d):
  def replace_rootfs_uuid(d, string):
  UUID_PLACEHOLDER = '<>'
  if UUID_PLACEHOLDER in string:
-uuid = get_rootfs_uuid(d)
+uuid = d.getVar('DISK_SIGNATURE_UUID', True)
  string = string.replace(UUID_PLACEHOLDER, uuid)
  return string
+
+DISK_SIGNATURE_UUID ?= "${@get_rootfs_uuid(d)}"



--
MARTIN HUNDEBØLL, Prevas A/S
Software Developer

Hedeager 3, DK-8200 Aarhus N
Phone +45 87438070
Mobile +45 25562438
martin.hundeb...@prevas.dk
www.prevas.com
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] wic: plugins: rawcopy: Add support to download the raw image from a http server

[Martin Hundebøll]

On 2017-12-11 12:45, Otavio Salvador wrote:

On Mon, Dec 11, 2017 at 3:30 AM, Kevin Hao  wrote:

On Sun, Dec 10, 2017 at 10:53:43PM -0200, Otavio Salvador wrote:

On Sat, Dec 9, 2017 at 12:45 AM, Kevin Hao  wrote:

Some firmwares are critical for the basic functions of the board, and may
be used by the bootloader. Due to the license issue, we can't integrate
it. But we do need it when creating a booting image by using WIC.
So add the support for the WIC to download a raw file from http
server and write it to a specific part. In order to restrict the
downloading behaviour, we introduce a "WKS_ALLOW_DOWNLOAD" variable and
only do the downloading when this variable is set.

Signed-off-by: Kevin Hao 


This does not make much sense.

It does not allow to be downloaded? so build system can download it
and use it as a normal artifact.


In some cases, it doesn't make sense to do so. Because:
a) These firmwares are just raw binary files, they don't need any build or
process. They are only useful when we try to create a bootable image.


In any case, keep them on the download cache is a good thing as it
avoids build issues due network instability and avoids as well
undesired file content changes.


b) There firmwares may be licensed under a proprietary license. We may not
integrate them to a fully open source build system. But we do want to 
provide
an easy way for the customer to build a bootable image.


Sure and I agree, but this is what layers are for.



I agree with Otavio here. Images built with OE / Bitbake should list all 
their sources in SRC_URIs.


Our (internal) releases includes images with all the sources needed to 
build the final target images, and we have had issues before with make 
systems pulling in sources in do_configure.


I don't personally think do_image_wic is any different than do_configure 
in this regard.


// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH][rocko] wic: support filesystem label for rawcopy

[Martin Hundebøll]

The '--label' argument should work for '--source rawcopy' as it does for
'--source rootfs', so add a method in RawCopyPlugin to update the label
on the temporary filesystem images.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>
Signed-off-by: Ross Burton <ross.bur...@intel.com>
---

Hi Armin,

Please consider this for Rocko.

Thanks

 scripts/lib/wic/plugins/source/rawcopy.py | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/scripts/lib/wic/plugins/source/rawcopy.py 
b/scripts/lib/wic/plugins/source/rawcopy.py
index 424ed26ed6..e86398ac8f 100644
--- a/scripts/lib/wic/plugins/source/rawcopy.py
+++ b/scripts/lib/wic/plugins/source/rawcopy.py
@@ -32,6 +32,25 @@ class RawCopyPlugin(SourcePlugin):
 
 name = 'rawcopy'
 
+@staticmethod
+def do_image_label(fstype, dst, label):
+if fstype.startswith('ext'):
+cmd = 'tune2fs -L %s %s' % (label, dst)
+elif fstype in ('msdos', 'vfat'):
+cmd = 'dosfslabel %s %s' % (dst, label)
+elif fstype == 'btrfs':
+cmd = 'btrfs filesystem label %s %s' % (dst, label)
+elif fstype == 'swap':
+cmd = 'mkswap -L %s %s' % (label, dst)
+elif fstype == 'squashfs':
+raise WicError("It's not possible to update a squashfs "
+   "filesystem label '%s'" % (label))
+else:
+raise WicError("Cannot update filesystem label: "
+   "Unknown fstype: '%s'" % (fstype))
+
+exec_cmd(cmd)
+
 @classmethod
 def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
  oe_builddir, bootimg_dir, kernel_dir,
@@ -66,4 +85,7 @@ class RawCopyPlugin(SourcePlugin):
 if filesize > part.size:
 part.size = filesize
 
+if part.label:
+RawCopyPlugin.do_image_label(part.fstype, dst, part.label)
+
 part.source_file = dst
-- 
2.15.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] wic: support filesystem label for rawcopy

[Martin Hundebøll]

The '--label' argument should work for '--source rawcopy' as it does for
'--source rootfs', so add a method in RawCopyPlugin to update the label
on the temporary filesystem images.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>
---
 scripts/lib/wic/plugins/source/rawcopy.py | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/scripts/lib/wic/plugins/source/rawcopy.py 
b/scripts/lib/wic/plugins/source/rawcopy.py
index 424ed26ed6..e86398ac8f 100644
--- a/scripts/lib/wic/plugins/source/rawcopy.py
+++ b/scripts/lib/wic/plugins/source/rawcopy.py
@@ -32,6 +32,25 @@ class RawCopyPlugin(SourcePlugin):
 
 name = 'rawcopy'
 
+@staticmethod
+def do_image_label(fstype, dst, label):
+if fstype.startswith('ext'):
+cmd = 'tune2fs -L %s %s' % (label, dst)
+elif fstype in ('msdos', 'vfat'):
+cmd = 'dosfslabel %s %s' % (dst, label)
+elif fstype == 'btrfs':
+cmd = 'btrfs filesystem label %s %s' % (dst, label)
+elif fstype == 'swap':
+cmd = 'mkswap -L %s %s' % (label, dst)
+elif fstype == 'squashfs':
+raise WicError("It's not possible to update a squashfs "
+   "filesystem label '%s'" % (label))
+else:
+raise WicError("Cannot update filesystem label: "
+   "Unknown fstype: '%s'" % (fstype))
+
+exec_cmd(cmd)
+
 @classmethod
 def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
  oe_builddir, bootimg_dir, kernel_dir,
@@ -66,4 +85,7 @@ class RawCopyPlugin(SourcePlugin):
 if filesize > part.size:
 part.size = filesize
 
+if part.label:
+RawCopyPlugin.do_image_label(part.fstype, dst, part.label)
+
 part.source_file = dst
-- 
2.15.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] systemd: allow dots in arguments to template units

[Martin Hundebøll]

When installing systemd template units with an argument, the current code
removes characters between the '@' and the '.' from service names in
SYSTEMD_SERVICE_${PN}, e.g.:

  getty@tty1.service -> getty@.service

This fails for services with dots in the argument (which is perfectly
legal in systemd), since the code searches only until the first dot.
E.g.:

  vlan@eth0.1.service -> vlan@1.service

This is obviously wrong, and fails in systemd_populate_packages(), where
it fails to find the unit file.

Fix this by reworking the removal of the argument part of the service
name, so that parts before '@' and after teh last '.' are used as base
name.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>
---
 meta/classes/systemd.bbclass | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes/systemd.bbclass b/meta/classes/systemd.bbclass
index c4b4bb9b70..1b134322fb 100644
--- a/meta/classes/systemd.bbclass
+++ b/meta/classes/systemd.bbclass
@@ -154,8 +154,10 @@ python systemd_populate_packages() {
 # Deal with adding, for example, 'ifplugd@eth0.service' from
 # 'ifplugd@.service'
 base = None
-if service.find('@') != -1:
-base = re.sub('@[^.]+.', '@.', service)
+at = service.find('@')
+if at != -1:
+ext = service.rfind('.')
+base = service[:at] + '@' + service[ext:]
 
 for path in searchpaths:
 if os.path.exists(oe.path.join(d.getVar("D"), path, 
service)):
-- 
2.15.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] wic: fallback to dd, if sparse_copy does not work

[Martin Hundebøll]

On 2017-11-16 12:49, Dogukan Ergun wrote:

Docker's aufs filesystem doesn't support file ioctl operations like FIGETBSZ
or FIEMAP.
Sparse_copy operation will fail if those ioctls are not supported.
If sparse_copy fails while generating wic images, fallback to dd for copying
filesystems on final image.

Signed-off-by: Dogukan Ergun 
---
  scripts/lib/wic/plugins/imager/direct.py | 10 --
  1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/wic/plugins/imager/direct.py 
b/scripts/lib/wic/plugins/imager/direct.py
index da1c061..70f93ee 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -577,8 +577,14 @@ class PartitionedImage():
  for part in self.partitions:
  source = part.source_file
  if source:
-# install source_file contents into a partition
-sparse_copy(source, self.path, seek=part.start * 
self.sector_size)
+try:
+# install source_file contents into a partition
+sparse_copy(source, self.path, seek=part.start * 
self.sector_size)
+except:


I guess a failing sparse_copy() would raise an IOError. Shouldn't you 
limit the except to catching only those?


// Martin


+# Sparse_copy failed, fallback to dd method
+dd_cmd = "dd if=%s of=%s bs=%d seek=%d count=%d 
conv=notrunc" % \
+(source, self.path, self.sector_size, part.start, 
part.size_sec)
+exec_cmd(dd_cmd)
  
  logger.debug("Installed %s in partition %d, sectors %d-%d, "

   "size %d sectors", source, part.num, part.start,


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/7] meson: add a recipe and class from meta-oe

[Martin Hundebøll]

Hi,

On 2017-11-15 12:08, Alexander Kanavin wrote:

The original recipe has been provided and improved by:

Ross Burton 
Ricardo Ribalda Delgado 
Adam C. Foltzer 
Peter Kjellerstedt 
Linus Svensson 

I have added a couple patches to fix up gtk-doc and
gobject-introspection in cross-compilation environments.

Signed-off-by: Alexander Kanavin 
---
  meta/classes/meson.bbclass | 103 +++
  ...ix-issues-that-arise-when-cross-compiling.patch | 113 +
  ...rospection-determine-g-ir-scanner-and-g-i.patch |  41 
  meta/recipes-devtools/meson/meson_0.43.0.bb|  19 
  4 files changed, 276 insertions(+)
  create mode 100644 meta/classes/meson.bbclass
  create mode 100644 
meta/recipes-devtools/meson/meson/0001-gtkdoc-fix-issues-that-arise-when-cross-compiling.patch
  create mode 100644 
meta/recipes-devtools/meson/meson/0002-gobject-introspection-determine-g-ir-scanner-and-g-i.patch
  create mode 100644 meta/recipes-devtools/meson/meson_0.43.0.bb

diff --git a/meta/classes/meson.bbclass b/meta/classes/meson.bbclass
new file mode 100644
index 000..f6cf24c33a7
--- /dev/null
+++ b/meta/classes/meson.bbclass
@@ -0,0 +1,103 @@
+inherit python3native
+
+DEPENDS_append = " meson-native ninja-native"
+
+# As Meson enforces out-of-tree builds we can just use cleandirs
+B = "${WORKDIR}/build"
+do_configure[cleandirs] = "${B}"
+
+# Where the meson.build build configuration is
+MESON_SOURCEPATH = "${S}"
+
+# These variables in the environment override meson's *native* tools settings.
+# We have to unset them, so that meson doesn't pick up the cross tools and
+# use them for native builds.
+unset CC
+unset CXX
+unset AR
+
+def noprefix(var, d):
+return d.getVar(var, True).replace(d.getVar('prefix', True) + '/', '', 1)
+
+MESONOPTS = " --prefix ${prefix} \
+  --bindir ${@noprefix('bindir', d)} \
+  --sbindir ${@noprefix('sbindir', d)} \
+  --datadir ${@noprefix('datadir', d)} \
+  --libdir ${@noprefix('libdir', d)} \
+  --libexecdir ${@noprefix('libexecdir', d)} \
+  --includedir ${@noprefix('includedir', d)} \
+  --mandir ${@noprefix('mandir', d)} \
+  --infodir ${@noprefix('infodir', d)} \
+  --sysconfdir ${sysconfdir} \
+  --localstatedir ${localstatedir} \
+  --sharedstatedir ${sharedstatedir}"
+
+MESON_C_ARGS = "${TARGET_CC_ARCH}${TOOLCHAIN_OPTIONS}"
+MESON_LINK_ARGS = "${MESON_C_ARGS} ${LDFLAGS}"
+
+MESON_HOST_ENDIAN = "${@bb.utils.contains('SITEINFO_ENDIANNESS', 'be', 'big', 
'little', d)}"
+MESON_TARGET_ENDIAN = "${@bb.utils.contains('TUNE_FEATURES', 'bigendian', 'big', 
'little', d)}"
+
+EXTRA_OEMESON += "${PACKAGECONFIG_CONFARGS}"
+
+MESON_CROSS_FILE = ""
+MESON_CROSS_FILE_class-target = "--cross-file ${WORKDIR}/meson.cross"


Shouldn't nativesdk builds use a meson.cross file too?


+
+def meson_array(var, d):
+return "', '".join(d.getVar(var, True).split()).join(("'", "'"))
+
+addtask write_config before do_configure
+do_write_config[vardeps] += "MESON_C_ARGS TOOLCHAIN_OPTIONS"
+do_write_config() {
+# This needs to be Py to split the args into single-element lists
+cat >${WORKDIR}/meson.cross <

Is meson.cross also written for native builds? If so, isn't there a way 
to avoid that?



+
+CONFIGURE_FILES = "meson.build"
+
+meson_do_configure() {
+if ! meson ${MESONOPTS} "${MESON_SOURCEPATH}" "${B}" ${MESON_CROSS_FILE} 
${EXTRA_OEMESON}; then
+cat ${B}/meson-logs/meson-log.txt
+bbfatal_log meson failed
+fi
+}
+
+do_compile[progress] = "outof:^\[(\d+)/(\d+)\]\s+"
+meson_do_compile() {
+ninja ${PARALLEL_MAKE}
+}
+
+meson_do_install() {
+DESTDIR='${D}' ninja ${PARALLEL_MAKEINST} install
+}
+
+EXPORT_FUNCTIONS do_configure do_compile do_install


// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] sanity.bbclass: uppercase check for machine and distro

[Martin Hundebøll]

On 2017-09-24 21:52, Yeoh Ee Peng wrote:

Uppercase in machine and distro will result in unexpected
behavior.


Wouldn't other checks make sense as well; e.g. underscore or starting 
with "pn-" ?


I'd prefer a whitelist check instead of an ever growing number of 
blacklist checks.


// Martin


QA check will produce error message if it verified that
machine or distro is uppercase.

Signed-off-by: Yeoh Ee Peng <ee.peng.y...@intel.com>
---
  meta/classes/sanity.bbclass | 9 +
  1 file changed, 9 insertions(+)

diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
index 1feb794..82efaef 100644
--- a/meta/classes/sanity.bbclass
+++ b/meta/classes/sanity.bbclass
@@ -694,6 +694,13 @@ def sanity_check_locale(d):
  except locale.Error:
  raise_sanity_error("You system needs to support the en_US.UTF-8 
locale.", d)
  
+def check_name_uppercase(type, name):

+import re
+messages = ""
+if re.search('[A-Z]', name):
+messages = messages + "%s: '%s' is uppercase. Please correct it to 
lowercase to avoid unexpected behavior.\n" % (type, name)
+return messages
+
  def check_sanity_everybuild(status, d):
  import os, stat
  # Sanity tests which test the users environment so need to run at each 
build (or are so cheap
@@ -723,6 +730,7 @@ def check_sanity_everybuild(status, d):
  # need to take into account DISTRO renaming DISTRO
  distro = d.getVar('DISTRO')
  if distro and distro != "nodistro":
+status.addresult(check_name_uppercase('DISTRO', d.getVar('DISTRO')))
  if not ( check_conf_exists("conf/distro/${DISTRO}.conf", d) or 
check_conf_exists("conf/distro/include/${DISTRO}.inc", d) ):
  status.addresult("DISTRO '%s' not found. Please set a valid DISTRO in your 
local.conf\n" % d.getVar("DISTRO"))
  
@@ -743,6 +751,7 @@ def check_sanity_everybuild(status, d):

  # Check that the MACHINE is valid, if it is set
  machinevalid = True
  if d.getVar('MACHINE'):
+status.addresult(check_name_uppercase('MACHINE', d.getVar('MACHINE')))
  if not check_conf_exists("conf/machine/${MACHINE}.conf", d):
  status.addresult('MACHINE=%s is invalid. Please set a valid 
MACHINE in your local.conf, environment or other configuration file.\n' % 
(d.getVar('MACHINE')))
  machinevalid = False



--
MARTIN HUNDEBØLL, Prevas A/S
Software Developer

Hedeager 3, DK-8200 Aarhus N
Phone +45 87438070
Mobile +45 25562438
martin.hundeb...@prevas.dk
www.prevas.com
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [pyro][PATCH] image.bbclass: Sorted ctypes to avoid basehash error

[Martin Hundebøll]

From: Gerson Fernando Budke 

When selected multiple subimages a similar error could happend:
  Variable do_image_cpio[subimages] value changed \
from 'cpio.gz.u-boot cpio.gz' to 'cpio.gz cpio.gz.u-boot'
To avoid this, 'ctypes' should be sorted at 'gen_conversion_cmds'.

This garantee that 'CONVERSION_CMD_xxx' are always written in tha same
order and consequently 'do_image_cpio' have the same hash.

(From OE-Core rev: 271f1a5f65b8685a1e3645026876251122ef3974)

Signed-off-by: Gerson Fernando Budke 
Signed-off-by: Ross Burton 
Signed-off-by: Richard Purdie 
---
 meta/classes/image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index ef2b38aeaf..9c9f14a99a 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -453,7 +453,7 @@ python () {
 
 rm_tmp_images = set()
 def gen_conversion_cmds(bt):
-for ctype in ctypes:
+for ctype in sorted(ctypes):
 if bt.endswith("." + ctype):
 type = bt[0:-len(ctype) - 1]
 if type.startswith("debugfs_"):
-- 
2.14.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] image.bbclass: Sorted ctypes to avoid basehash error

[Martin Hundebøll]

On 2017-09-19 19:15, Gerson Fernando Budke wrote:

When selected multiple subimages a similar error could happend:
   Variable do_image_cpio[subimages] value changed \
 from 'cpio.gz.u-boot cpio.gz' to 'cpio.gz cpio.gz.u-boot'
To avoid this, 'ctypes' should be sorted at 'gen_conversion_cmds'.

This garantee that 'CONVERSION_CMD_xxx' are always written in tha same
order and consequently 'do_image_cpio' have the same hash.

Signed-off-by: Gerson Fernando Budke <nando...@gmail.com>


Tested-by: Martin Hundebøll <m...@prevas.dk>


---
  meta/classes/image.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 3639aa4..bef634b 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -452,7 +452,7 @@ python () {
  
  rm_tmp_images = set()

  def gen_conversion_cmds(bt):
-for ctype in ctypes:
+for ctype in sorted(ctypes):
  if bt.endswith("." + ctype):
  type = bt[0:-len(ctype) - 1]
  if type.startswith("debugfs_"):


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] image.bbclass: Correct chaining compression support

[Martin Hundebøll]

Hi Tom,

On 2017-07-22 00:06, Tom Rini wrote:

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index de535ce6fcff..bd6a5b7b810a 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -453,7 +453,7 @@ python () {
  rm_tmp_images = set()
  def gen_conversion_cmds(bt):
  for ctype in ctypes:
-if bt[bt.find('.') + 1:] == ctype:
+if bt.endswith("." + ctype):
  type = bt[0:-len(ctype) - 1]
  if type.startswith("debugfs_"):
  type = type[8:]


This is the change that in fact messes with our cpio.gz.u-boot image 
types, causing base hash changes.


I suspect the changed if-check to be too permissive. In our case, it now 
matches any ctype that ends with ".u-boot", and not just "gz.u-boot" as 
set in IMAGE_FSTYPES.


Wouldn't it be correct to match on the entire bt variable? I.e.

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass 


index ef2b38aeaf..818932f7f1 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -454,7 +454,7 @@ python () {
 rm_tmp_images = set()
 def gen_conversion_cmds(bt):
 for ctype in ctypes:
-if bt.endswith("." + ctype):
+if bt.split('.', 1) == [t, ctype]:
 type = bt[0:-len(ctype) - 1]
 if type.startswith("debugfs_"):
 type = type[8:]

This works for me, preliminary tested with the following fs types 
configured:


IMAGE_FSTYPES = "cpio.gz.u-boot"
IMAGE_FSTYPES = "tar cpio cpio.gz cpio.gz.u-boot"
	IMAGE_FSTYPES = "tar cpio cpio.gz cpio.gz.u-boot tar.bz2 jffs2 wic 
wic.bmap"


// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] image: Fix "metadata is not deterministic" when chaining 2+ CONVERSION_CMDs

[Martin Hundebøll]

On 2017-09-15 20:20, Martin Hundebøll wrote:

On 2017-09-15 14:16, Tom Rini wrote:

This commit has resulted in more or less persistent basehash issues
for my cpio.gz.u-boot image:

ERROR: When reparsing 
/home/mnhu/projects/pil/yocto/meta-pil/recipes-filesystems/rescue-rootfs.bb.do_image_cpio, 


the basehash value changed from ac9907bbb8ee2ee6d9ca90f7a7e265f5 to
8c7b717add6b1d1ba6ff761ebfdedf01. The metadata is not deterministic
and this needs to be fixed.

The error occurs repeatedly when doing `bitbake -c clean
rescue-rootfs && bitbake rescue-rootfs`. Reverting the commit makes
the error go away.

Also, the issue only appears for cpio.gz.u-boot image types, but not
for cpio.gz

I am far from an expert in bitbake signature handling, so please let
me know what to do, to debug this further.

Can you let me know what layers and machine I need to reproduce this?
I'll take a look, thanks!


I am able to reproduce with the following:

   - Ubuntu 16.04 based on the attached Dockerfile
   - A clean clone of poky (commit 072430b9b3)
   - meta, meta-poky, and meta-yocto-bsp (see build-config.log)
   - beaglebone machine and gpio.gz.u-boot image type (see local.conf)


Sorry, this seems to be caused by your earlier patch: 88eec32150 
('image.bbclass: Correct chaining compression support'). My first 
conclusion was probably caused by caching playing me a trick.


I'll reply to that patch instead.

// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] image: Fix "metadata is not deterministic" when chaining 2+ CONVERSION_CMDs

[Martin Hundebøll]

Hi Tom,

On 2017-09-15 14:16, Tom Rini wrote:

This commit has resulted in more or less persistent basehash issues
for my cpio.gz.u-boot image:

ERROR: When reparsing 
/home/mnhu/projects/pil/yocto/meta-pil/recipes-filesystems/rescue-rootfs.bb.do_image_cpio,
the basehash value changed from ac9907bbb8ee2ee6d9ca90f7a7e265f5 to
8c7b717add6b1d1ba6ff761ebfdedf01. The metadata is not deterministic
and this needs to be fixed.

The error occurs repeatedly when doing `bitbake -c clean
rescue-rootfs && bitbake rescue-rootfs`. Reverting the commit makes
the error go away.

Also, the issue only appears for cpio.gz.u-boot image types, but not
for cpio.gz

I am far from an expert in bitbake signature handling, so please let
me know what to do, to debug this further.

Can you let me know what layers and machine I need to reproduce this?
I'll take a look, thanks!


I am able to reproduce with the following:

  - Ubuntu 16.04 based on the attached Dockerfile
  - A clean clone of poky (commit 072430b9b3)
  - meta, meta-poky, and meta-yocto-bsp (see build-config.log)
  - beaglebone machine and gpio.gz.u-boot image type (see local.conf)

// Martin

FROM ubuntu:16.04
MAINTAINER Martin Hundebøll <m...@prevas.dk>

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN apt-get install -y python python3 build-essential locales
RUN apt-get install -y chrpath cpio diffstat gawk git texinfo wget iputils-ping 
openssh-client
RUN apt-get install -y zsh vim less
RUN locale-gen en_US.UTF-8
ENV LANG=en_US.UTF-8

RUN groupadd admin; \
useradd -m -p $(openssl passwd -crypt foobar) -s /bin/zsh -G admin mnhu;
USER mnhu
WORKDIR /home/mnhu
CMD ["/bin/zsh"]

#
# This file is your local configuration file and is where all local user 
settings
# are placed. The comments in this file give some guide to the options a new 
user
# to the system might want to change but pretty much any configuration option 
can
# be set in this file. More adventurous users can look at local.conf.extended
# which contains other examples of configuration which can be placed in this 
file
# but new users likely won't need any of them initially.
#
# Lines starting with the '#' character are commented out and in some cases the
# default values are provided as comments to show people example syntax. 
Enabling
# the option is a question of removing the # character and making any change to 
the
# variable as required.

#
# Machine Selection
#
# You need to select a specific machine to target the build with. There are a 
selection
# of emulated machines available which can boot and run in the QEMU emulator:
#
#MACHINE ?= "qemuarm"
#MACHINE ?= "qemuarm64"
#MACHINE ?= "qemumips"
#MACHINE ?= "qemumips64"
#MACHINE ?= "qemuppc"
#MACHINE ?= "qemux86"
#MACHINE ?= "qemux86-64"
#
# There are also the following hardware board target machines included for 
# demonstration purposes:
#
#MACHINE ?= "beaglebone"
#MACHINE ?= "genericx86"
#MACHINE ?= "genericx86-64"
#MACHINE ?= "mpc8315e-rdb"
#MACHINE ?= "edgerouter"
#
# This sets the default machine to be qemux86 if no other machine is selected:
MACHINE ??= "beaglebone"

#
# Where to place downloads
#
# During a first build the system will download many different source code 
tarballs
# from various upstream projects. This can take a while, particularly if your 
network
# connection is slow. These are all stored in DL_DIR. When wiping and 
rebuilding you
# can preserve this directory to speed up this part of subsequent builds. This 
directory
# is safe to share between multiple builds on the same machine too.
#
# The default is a downloads directory under TOPDIR which is the build 
directory.
#
#DL_DIR ?= "${TOPDIR}/downloads"

#
# Where to place shared-state files
#
# BitBake has the capability to accelerate builds based on previously built 
output.
# This is done using "shared state" files which can be thought of as cache 
objects
# and this option determines where those files are placed.
#
# You can wipe out TMPDIR leaving this directory intact and the build would 
regenerate
# from these files if no changes were made to the configuration. If changes 
were made
# to the configuration, only shared state files where the state was still valid 
would
# be used (done using checksums).
#
# The default is a sstate-cache directory under TOPDIR.
#
#SSTATE_DIR ?= "${TOPDIR}/sstate-cache"

#
# Where to place the build output
#
# This option specifies where the bulk of the building work should be done and
# where BitBake should place its temporary files and output. Keep in mind that
# this includes the extraction and compilation of many applications and the 
toolchain
# which can use Gigabytes of hard disk space.
#
# The default is a tmp directory under TOPDIR.
#
#TMPDIR = "${TOPDIR}/tmp"

#
# Default policy config
#
# The distribution setting contr

[OE-core] [PATCHv2] Revert "kernel-module-split: rrecommend kernel-image instead of rdepend"

[Martin Hundebøll]

This reverts commit e0ed52c51464855e9a6a37ea49df7efde7e91076.

Commit e0ed52c514 ('kernel-module-split: rrecommend kernel-image instead
of rdepend') changed kernel modules to rrecommend kernel-image instead
of rdepend on kernel. This broke existing setups, where the kernel is
omitted by setting RDEPEND_kernel-base = "".

Revert the patch, as the existing way of omitting kernel-image in images
works just fine.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>

---

Changes in v2:
 * Added signed-of-by line

---
 meta/classes/kernel-module-split.bbclass | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index 78843c5474..1035525dac 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -120,11 +120,6 @@ python split_kernel_module_packages () {
 rdepends[dep] = []
 d.setVar('RDEPENDS_' + pkg, bb.utils.join_deps(rdepends, 
commasep=False))
 
-# let kernel modules runtime recommend kernel image
-rrecommends = bb.utils.explode_dep_versions2(d.getVar('RRECOMMENDS_' + 
pkg) or "")
-rrecommends['kernel-image'] = []
-d.setVar('RRECOMMENDS_' + pkg, bb.utils.join_deps(rrecommends, 
commasep=False))
-
 # Avoid automatic -dev recommendations for modules ending with -dev.
 d.setVarFlag('RRECOMMENDS_' + pkg, 'nodeprrecs', 1)
 
@@ -143,7 +138,7 @@ python split_kernel_module_packages () {
 postinst = d.getVar('pkg_postinst_modules')
 postrm = d.getVar('pkg_postrm_modules')
 
-modules = do_split_packages(d, root='${nonarch_base_libdir}/modules', 
file_regex=module_regex, output_pattern=module_pattern, description='%s kernel 
module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, 
extra_depends='')
+modules = do_split_packages(d, root='${nonarch_base_libdir}/modules', 
file_regex=module_regex, output_pattern=module_pattern, description='%s kernel 
module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, 
extra_depends='kernel-%s' % (d.getVar("KERNEL_VERSION")))
 if modules:
 metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE')
 d.appendVar('RDEPENDS_' + metapkg, ' '+' '.join(modules))
-- 
2.14.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-module-split: rrecommend kernel-image instead of rdepend

[Martin Hundebøll]

On 2017-09-11 11:26, Richard Purdie wrote:

On Mon, 2017-09-11 at 09:02 +0200, Martin Hundebøll wrote:


On 2017-09-08 23:29, Paul Barker wrote:


On Fri, Sep 8, 2017 at 10:07 PM, Martin Hundebøll
<martin.hundeb...@prevas.dk> wrote:


  

No, you'd need it to be set in a conf file so it takes effect when
the
kernel-base package is created.

I can see the issue here though. Can you set `RDEPENDS_kernel-base
=
""` globally and explicitly install kernel-image into the
appropriate
image? If not then yes we'll need to find an alternative solution
to
simply reverting the change.


Seems like the inverted approach of disabling install of kernel-
image
globally, and enabling it per-image works just fine.

Reverting the patch is fine by me - sorry for the noise.


Are you sure? I can't imagine being able to influence this at all from
an image recipe, either the dependency is present or its not and it
would be set from the global configuration, not from an image recipe...


Pretty sure, yes.

conf/distro/basic.conf:
RDEPENDS_kernel-base = ""

recipes-filesystems/rootfs.bb:
IMAGE_INSTALL += "kernel-image kernel-module-foo"

recipes-filesystems/rescuefs.bb:
IMAGE_INSTALL += "kernel-module-foo"

// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] Revert "kernel-module-split: rrecommend kernel-image instead of rdepend"

[Martin Hundebøll]

This reverts commit e0ed52c51464855e9a6a37ea49df7efde7e91076.

Commit e0ed52c514 ('kernel-module-split: rrecommend kernel-image instead
of rdepend') changed kernel modules to rrecommend kernel-image instead
of rdepend on kernel. This broke existing setups, where the kernel is
omitted by setting RDEPEND_kernel-base = "".

Revert the patch, as the existing way of omitting kernel-image in images
works just fine.
---
 meta/classes/kernel-module-split.bbclass | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index 78843c5474..1035525dac 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -120,11 +120,6 @@ python split_kernel_module_packages () {
 rdepends[dep] = []
 d.setVar('RDEPENDS_' + pkg, bb.utils.join_deps(rdepends, 
commasep=False))
 
-# let kernel modules runtime recommend kernel image
-rrecommends = bb.utils.explode_dep_versions2(d.getVar('RRECOMMENDS_' + 
pkg) or "")
-rrecommends['kernel-image'] = []
-d.setVar('RRECOMMENDS_' + pkg, bb.utils.join_deps(rrecommends, 
commasep=False))
-
 # Avoid automatic -dev recommendations for modules ending with -dev.
 d.setVarFlag('RRECOMMENDS_' + pkg, 'nodeprrecs', 1)
 
@@ -143,7 +138,7 @@ python split_kernel_module_packages () {
 postinst = d.getVar('pkg_postinst_modules')
 postrm = d.getVar('pkg_postrm_modules')
 
-modules = do_split_packages(d, root='${nonarch_base_libdir}/modules', 
file_regex=module_regex, output_pattern=module_pattern, description='%s kernel 
module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, 
extra_depends='')
+modules = do_split_packages(d, root='${nonarch_base_libdir}/modules', 
file_regex=module_regex, output_pattern=module_pattern, description='%s kernel 
module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, 
extra_depends='kernel-%s' % (d.getVar("KERNEL_VERSION")))
 if modules:
 metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE')
 d.appendVar('RDEPENDS_' + metapkg, ' '+' '.join(modules))
-- 
2.14.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] kernel-module-split: rrecommend kernel-image instead of rdepend

[Martin Hundebøll]

On 2017-09-08 23:29, Paul Barker wrote:

On Fri, Sep 8, 2017 at 10:07 PM, Martin Hundebøll
<martin.hundeb...@prevas.dk> wrote:



On September 8, 2017 10:37:26 PM GMT+02:00, Denys Dmytriyenko <de...@denix.org> 
wrote:

On Fri, Sep 08, 2017 at 09:06:17PM +0100, Paul Barker wrote:


This breaks the method described in the kernel dev FAQ to omit the
kernel image from the rootfs which is to set the following in a conf
file:

 RDEPENDS_kernel-base = ""

See

http://www.yoctoproject.org/docs/2.3.1/kernel-dev/kernel-dev.html#idm45690538384880.


By recommending kernel-image directly instead of recommending the
top-level kernel- package which was actually removed from
extra_depends, we skip the inclusion of the kernel-base package. I'm
not familiar enough with the details of kernel module loading to know
if the 2 files in kernel-base (modules.builtin and modules.order) are
strictly needed but I think they should at least be included in via
recommendation.

As we already have a documented way to omit the kernel image, can we
revert this?

As an alternative I can send in a patch which changes the recommend

to

the top-level kernel- package instead of kernel-image.


FWIW, this was discussed at #yocto on IRC today. When I first saw the
change
several days ago, I was puzzled as well...
The existing method has been in use by many of us for a very long time
and
seems to be well documented - I also vote for reverting the change.


Does setting `RDEPENDS_kernel-base = ""` in an image recipe work too?



No, you'd need it to be set in a conf file so it takes effect when the
kernel-base package is created.

I can see the issue here though. Can you set `RDEPENDS_kernel-base =
""` globally and explicitly install kernel-image into the appropriate
image? If not then yes we'll need to find an alternative solution to
simply reverting the change.



Seems like the inverted approach of disabling install of kernel-image 
globally, and enabling it per-image works just fine.


Reverting the patch is fine by me - sorry for the noise.

// Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] kernel-module-split: rrecommend kernel-image instead of rdepend

[Martin Hundebøll]

Hard depending on the kernel makes it impossible to install kernel
modules without getting the kernel image installed too. This is
inconvenient in e.g. initramdisks, where the kernel is loaded from
outside the initramdisk.

Making the kernel modules rrecommend kernel-image- instead of
rdepending on it, makes it possible to install kernel modules without
the kernel image by setting "kernel-image" in BAD_RECOMMENDATIONS.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>
---
 meta/classes/kernel-module-split.bbclass | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index 1035525dac..78843c5474 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -120,6 +120,11 @@ python split_kernel_module_packages () {
 rdepends[dep] = []
 d.setVar('RDEPENDS_' + pkg, bb.utils.join_deps(rdepends, 
commasep=False))
 
+# let kernel modules runtime recommend kernel image
+rrecommends = bb.utils.explode_dep_versions2(d.getVar('RRECOMMENDS_' + 
pkg) or "")
+rrecommends['kernel-image'] = []
+d.setVar('RRECOMMENDS_' + pkg, bb.utils.join_deps(rrecommends, 
commasep=False))
+
 # Avoid automatic -dev recommendations for modules ending with -dev.
 d.setVarFlag('RRECOMMENDS_' + pkg, 'nodeprrecs', 1)
 
@@ -138,7 +143,7 @@ python split_kernel_module_packages () {
 postinst = d.getVar('pkg_postinst_modules')
 postrm = d.getVar('pkg_postrm_modules')
 
-modules = do_split_packages(d, root='${nonarch_base_libdir}/modules', 
file_regex=module_regex, output_pattern=module_pattern, description='%s kernel 
module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, 
extra_depends='kernel-%s' % (d.getVar("KERNEL_VERSION")))
+modules = do_split_packages(d, root='${nonarch_base_libdir}/modules', 
file_regex=module_regex, output_pattern=module_pattern, description='%s kernel 
module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, 
extra_depends='')
 if modules:
 metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE')
 d.appendVar('RDEPENDS_' + metapkg, ' '+' '.join(modules))
-- 
2.14.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] kernel-module-split: rrecommend kernel-image instead of rdepend

[Martin Hundebøll]

Hard depending on the kernel makes it impossible to install kernel
modules without getting the kernel image installed too. This is
inconvenient in e.g. initramdisks, where the kernel is loaded from
outside the initramdisk.

Making the kernel modules rrecommend kernel-image- instead of
rdepending on it, makes it possible to install kernel modules without
the kernel image by setting "kernel-image" in BAD_RECOMMENDATIONS.

Signed-off-by: Martin Hundebøll <m...@prevas.dk>
---
 meta/classes/kernel-module-split.bbclass | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel-module-split.bbclass 
b/meta/classes/kernel-module-split.bbclass
index ba9cc1517e..7a0260e3db 100644
--- a/meta/classes/kernel-module-split.bbclass
+++ b/meta/classes/kernel-module-split.bbclass
@@ -120,6 +120,11 @@ python split_kernel_module_packages () {
 rdepends[dep] = []
 d.setVar('RDEPENDS_' + pkg, bb.utils.join_deps(rdepends, 
commasep=False))
 
+# let kernel modules runtime recommend kernel image
+rrecommends = bb.utils.explode_dep_versions2(d.getVar('RRECOMMENDS_' + 
pkg) or "")
+rrecommends['kernel-image'] = []
+d.setVar('RRECOMMENDS_' + pkg, bb.utils.join_deps(rrecommends, 
commasep=False))
+
 # Avoid automatic -dev recommendations for modules ending with -dev.
 d.setVarFlag('RRECOMMENDS_' + pkg, 'nodeprrecs', 1)
 
@@ -138,7 +143,7 @@ python split_kernel_module_packages () {
 postinst = d.getVar('pkg_postinst_modules')
 postrm = d.getVar('pkg_postrm_modules')
 
-modules = do_split_packages(d, root='/lib/modules', 
file_regex=module_regex, output_pattern=module_pattern, description='%s kernel 
module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, 
extra_depends='kernel-%s' % (d.getVar("KERNEL_VERSION")))
+modules = do_split_packages(d, root='/lib/modules', 
file_regex=module_regex, output_pattern=module_pattern, description='%s kernel 
module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, 
extra_depends='')
 if modules:
 metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE')
 d.appendVar('RDEPENDS_' + metapkg, ' '+' '.join(modules))
-- 
2.14.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] Use env in python-shebangs

[Martin Hundebøll]

To make fake env or virtualenv possible, make sure python scripts use
 #!/usr/bin/env python as shebang.

Comments to previous patches/rfcs rejected the idea of either explicit
use of python2 in the env-shebang or a python2-symlink in the bitbake
environment, so this is dropped.

Still we would like to make the lifes easier for those unfortunate souls
that live in a python3-first world/distro :)

Thanks for the previous comments!

Martin Hundebøll (1):
  scripts: use '/usr/bin/env' in shebangs with python

 bitbake/lib/progressbar.py| 2 +-
 scripts/lib/wic/conf.py   | 2 +-
 scripts/lib/wic/creator.py| 2 +-
 scripts/lib/wic/imager/baseimager.py  | 2 +-
 scripts/lib/wic/kickstart/__init__.py | 2 +-
 scripts/lib/wic/kickstart/custom_commands/micboot.py  | 2 +-
 scripts/lib/wic/kickstart/custom_commands/micpartition.py | 2 +-
 scripts/lib/wic/msger.py  | 2 +-
 scripts/lib/wic/plugin.py | 2 +-
 scripts/lib/wic/pluginbase.py | 2 +-
 scripts/lib/wic/utils/errors.py   | 2 +-
 scripts/lib/wic/utils/fs_related.py   | 2 +-
 scripts/lib/wic/utils/misc.py | 2 +-
 scripts/lib/wic/utils/partitionedfs.py| 2 +-
 scripts/lib/wic/utils/runner.py   | 2 +-
 scripts/pybootchartgui/pybootchartgui.py  | 2 +-
 16 files changed, 16 insertions(+), 16 deletions(-)

-- 
2.1.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] scripts: use '/usr/bin/env' in shebangs with python

[Martin Hundebøll]

To support yocto on systems with python3 as default version, scripts
should use /usr/bin/env python in the shebang, as this allows the use of
a fake env to mimic python2 as default version.

This patch simply replaces occurrences of #!/usr/bin/python with
 #!/usr/bin/env python and was done with this oneliner:

 git grep -lE '^#!/usr/bin/python' | xargs \
 sed -i 's|/usr/bin/python|/usr/bin/env python|'

Signed-off-by: Martin Hundebøll mar...@hundeboll.net
---
 bitbake/lib/progressbar.py| 2 +-
 scripts/lib/wic/conf.py   | 2 +-
 scripts/lib/wic/creator.py| 2 +-
 scripts/lib/wic/imager/baseimager.py  | 2 +-
 scripts/lib/wic/kickstart/__init__.py | 2 +-
 scripts/lib/wic/kickstart/custom_commands/micboot.py  | 2 +-
 scripts/lib/wic/kickstart/custom_commands/micpartition.py | 2 +-
 scripts/lib/wic/msger.py  | 2 +-
 scripts/lib/wic/plugin.py | 2 +-
 scripts/lib/wic/pluginbase.py | 2 +-
 scripts/lib/wic/utils/errors.py   | 2 +-
 scripts/lib/wic/utils/fs_related.py   | 2 +-
 scripts/lib/wic/utils/misc.py | 2 +-
 scripts/lib/wic/utils/partitionedfs.py| 2 +-
 scripts/lib/wic/utils/runner.py   | 2 +-
 scripts/pybootchartgui/pybootchartgui.py  | 2 +-
 16 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/bitbake/lib/progressbar.py b/bitbake/lib/progressbar.py
index b668647..114cdc1 100644
--- a/bitbake/lib/progressbar.py
+++ b/bitbake/lib/progressbar.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 # -*- coding: iso-8859-1 -*-
 #
 # progressbar  - Text progressbar library for python.
diff --git a/scripts/lib/wic/conf.py b/scripts/lib/wic/conf.py
index d5419f8..be34355 100644
--- a/scripts/lib/wic/conf.py
+++ b/scripts/lib/wic/conf.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2011 Intel, Inc.
 #
diff --git a/scripts/lib/wic/creator.py b/scripts/lib/wic/creator.py
index a4b19ac..2219377 100644
--- a/scripts/lib/wic/creator.py
+++ b/scripts/lib/wic/creator.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2011 Intel, Inc.
 #
diff --git a/scripts/lib/wic/imager/baseimager.py 
b/scripts/lib/wic/imager/baseimager.py
index 5bcd2f7..e830527 100644
--- a/scripts/lib/wic/imager/baseimager.py
+++ b/scripts/lib/wic/imager/baseimager.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2007 Red Hat  Inc.
 # Copyright (c) 2009, 2010, 2011 Intel, Inc.
diff --git a/scripts/lib/wic/kickstart/__init__.py 
b/scripts/lib/wic/kickstart/__init__.py
index 4f5b778..6000982 100644
--- a/scripts/lib/wic/kickstart/__init__.py
+++ b/scripts/lib/wic/kickstart/__init__.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2007 Red Hat, Inc.
 # Copyright (c) 2009, 2010, 2011 Intel, Inc.
diff --git a/scripts/lib/wic/kickstart/custom_commands/micboot.py 
b/scripts/lib/wic/kickstart/custom_commands/micboot.py
index 66d1678..d162142 100644
--- a/scripts/lib/wic/kickstart/custom_commands/micboot.py
+++ b/scripts/lib/wic/kickstart/custom_commands/micboot.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2008, 2009, 2010 Intel, Inc.
 #
diff --git a/scripts/lib/wic/kickstart/custom_commands/micpartition.py 
b/scripts/lib/wic/kickstart/custom_commands/micpartition.py
index 59a87fb..43d04f1 100644
--- a/scripts/lib/wic/kickstart/custom_commands/micpartition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/micpartition.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Marko Saukko marko.sau...@cybercom.com
 #
diff --git a/scripts/lib/wic/msger.py b/scripts/lib/wic/msger.py
index 9afc85b..9f557e7 100644
--- a/scripts/lib/wic/msger.py
+++ b/scripts/lib/wic/msger.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 # vim: ai ts=4 sts=4 et sw=4
 #
 # Copyright (c) 2009, 2010, 2011 Intel, Inc.
diff --git a/scripts/lib/wic/plugin.py b/scripts/lib/wic/plugin.py
index 61c5859..41a8017 100644
--- a/scripts/lib/wic/plugin.py
+++ b/scripts/lib/wic/plugin.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2011 Intel, Inc.
 #
diff --git a/scripts/lib/wic/pluginbase.py b/scripts/lib/wic/pluginbase.py
index b8b3a46..e3de9ba 100644
--- a/scripts/lib/wic/pluginbase.py
+++ b/scripts/lib/wic/pluginbase.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2011 Intel, Inc.
 #
diff --git a/scripts/lib/wic/utils/errors.py b/scripts/lib/wic/utils/errors.py
index 86e230a..9410311 100644
--- a/scripts/lib/wic/utils/errors.py
+++ b/scripts/lib/wic/utils/errors.py
@@ -1,4 +1,4 @@
-#!/usr

[OE-core] [RFCv2 2/2] scripts: modify environment to use python2 instead

[Martin Hundebøll]

In distributions where /usr/bin/python points at python3, the check for
python version fails even when python2 is available at /usr/bin/python2.

The usual way to circumvent this is to manually create a fake
environment, where python is a symlink to python2. Since openembedded
already sets up a semi fake environment, we might as well add a python
symlink to this.

This patch adds a check in oe-buildenv-internal, that runs through a
list of possible python names, and check for their existance and
version. Once a compatible version is found, a symlink is created in
./bitbake/bin/python.

If no compatible version is found, an error is printed and
oe-init-build-env fails.

Signed-off-by: Martin Hundebøll mar...@hundeboll.net
---
 scripts/oe-buildenv-internal | 30 ++---
 scripts/python-env.sh| 46 
 2 files changed, 56 insertions(+), 20 deletions(-)
 create mode 100755 scripts/python-env.sh

diff --git a/scripts/oe-buildenv-internal b/scripts/oe-buildenv-internal
index bba6f8f..3841ccb 100755
--- a/scripts/oe-buildenv-internal
+++ b/scripts/oe-buildenv-internal
@@ -29,26 +29,6 @@ if [ ! -z $OECORE_SDK_VERSION ]; then
 return 1
 fi
 
-# Make sure we're not using python v3.x. This check can't go into
-# sanity.bbclass because bitbake's source code doesn't even pass
-# parsing stage when used with python v3, so we catch it here so we
-# can offer a meaningful error message.
-py_v3_check=`/usr/bin/env python --version 21 | grep Python 3`
-if [ $py_v3_check !=  ]; then
-   echo 2 Bitbake is not compatible with python v3
-   echo 2 Please set up python v2 as your default python interpreter
-   return 1
-fi
-
-# Similarly, we now have code that doesn't parse correctly with older
-# versions of Python, and rather than fixing that and being eternally
-# vigilant for any other new feature use, just check the version here.
-py_v26_check=`python -c 'import sys; print sys.version_info = (2,7,3)'`
-if [ $py_v26_check != True ]; then
-   echo 2 BitBake requires Python 2.7.3 or later
-   return 1
-fi
-
 if [ x$BDIR = x ]; then
 if [ x$1 = x ]; then
 BDIR=build
@@ -94,6 +74,16 @@ if ! (test -d $BITBAKEDIR); then
 return 1
 fi
 
+# Make sure we're not using python v3.x. This check can't go into
+# sanity.bbclass because bitbake's source code doesn't even pass
+# parsing stage when used with python v3, so we catch it here so we
+# can offer a meaningful error message.
+source ${OEROOT}/scripts/python-env.sh
+if ! env_set_python; then
+echo env_set_python failed
+return 1
+fi
+
 # Make sure our paths are at the beginning of $PATH
 NEWPATHS=${OEROOT}/scripts:$BITBAKEDIR/bin:
 PATH=$NEWPATHS$(echo $PATH | sed -e s|:$NEWPATHS|:|g -e s|^$NEWPATHS||)
diff --git a/scripts/python-env.sh b/scripts/python-env.sh
new file mode 100755
index 000..d88587e
--- /dev/null
+++ b/scripts/python-env.sh
@@ -0,0 +1,46 @@
+function env_set_python() {
+# List of possible python executables to check in environment
+py_names=(python python2 python2.7)
+
+# Minimum supported python version
+py_minver=(2,7,3)
+
+# Maximum supported python version
+py_maxver=(2,9,9)
+
+# For each python name listed abobe
+for py_name in ${py_names[@]}; do
+# Get the full path from the shell environment
+py=$(command -v $py_name)
+
+# Skip this name if it doesn't exist
+if [ -z $py ]; then
+continue
+fi
+
+# Skip this name if version is too old
+minver=$($py_name -c import sys; print(sys.version_info = 
$py_minver))
+if [ $minver != True ]; then
+continue
+fi
+
+# Skip this name if version is too new
+maxver=$($py_name -c import sys; print(sys.version_info = 
$py_maxver))
+if [ $maxver != True ]; then
+continue
+fi
+
+# Found a compatible python name. Install it in the environment
+echo Setting $py as python env
+ln -sf $py $BITBAKEDIR/bin/python
+
+# Return with success
+return 0
+done
+
+# None of the listed python name were usable. Complain and exit with error
+echo 2 No compatible python interpreter was found.
+echo 2 Please install python v2 and make sure it is available in your 
$$PATH
+
+return 1
+}
-- 
2.1.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFCv2 0/2] Support systems with python3 as default

[Martin Hundebøll]

Since more distributions plan to move to python3, steps should be taken
to support such systems.

My previous attemt was pretty intrusive and changed all occurrences of
/usr/bin/{env ,}python to /usr/bin/env python2. Such apporach might break
on systems with no python2 alias in env, so here is a different approach.

Paul Braker suggested to auto-detect the available python version and
setup a fake environment, so I have given it a shot. This is quite simple
and much less intrusive than my first attempt. It utilizes the existing
bitbake env and simply tests a few different python names and, if a
compatible version is found, creates a symlink in ./bitbate/bin/python

Comments/suggestions are welcome!

Martin Hundebøll (2):
  scripts: use '/usr/bin/env' in shebangs with python
  scripts: modify environment to use python2 instead

 bitbake/lib/progressbar.py |  2 +-
 scripts/lib/wic/conf.py|  2 +-
 scripts/lib/wic/creator.py |  2 +-
 scripts/lib/wic/imager/baseimager.py   |  2 +-
 scripts/lib/wic/kickstart/__init__.py  |  2 +-
 .../lib/wic/kickstart/custom_commands/micboot.py   |  2 +-
 .../wic/kickstart/custom_commands/micpartition.py  |  2 +-
 scripts/lib/wic/msger.py   |  2 +-
 scripts/lib/wic/plugin.py  |  2 +-
 scripts/lib/wic/pluginbase.py  |  2 +-
 scripts/lib/wic/utils/errors.py|  2 +-
 scripts/lib/wic/utils/fs_related.py|  2 +-
 scripts/lib/wic/utils/misc.py  |  2 +-
 scripts/lib/wic/utils/partitionedfs.py |  2 +-
 scripts/lib/wic/utils/runner.py|  2 +-
 scripts/oe-buildenv-internal   | 30 +-
 scripts/pybootchartgui/pybootchartgui.py   |  2 +-
 scripts/python-env.sh  | 46 ++
 18 files changed, 72 insertions(+), 36 deletions(-)
 create mode 100755 scripts/python-env.sh

-- 
2.1.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFCv2 1/2] scripts: use '/usr/bin/env' in shebangs with python

[Martin Hundebøll]

To support yocto on systems with python3 as default version, scripts
should use /usr/bin/env python in the shebang, as this allows the use of
a fake env to mimic python2 as default version.

This patch simply replaces occurrences of #!/usr/bin/python with
 #!/usr/bin/env python and was done with this oneliner:

 git grep -lE '^#!/usr/bin/python' | xargs \
 sed -i 's|/usr/bin/python|/usr/bin/env python|'

Signed-off-by: Martin Hundebøll mar...@hundeboll.net
---
 bitbake/lib/progressbar.py| 2 +-
 scripts/lib/wic/conf.py   | 2 +-
 scripts/lib/wic/creator.py| 2 +-
 scripts/lib/wic/imager/baseimager.py  | 2 +-
 scripts/lib/wic/kickstart/__init__.py | 2 +-
 scripts/lib/wic/kickstart/custom_commands/micboot.py  | 2 +-
 scripts/lib/wic/kickstart/custom_commands/micpartition.py | 2 +-
 scripts/lib/wic/msger.py  | 2 +-
 scripts/lib/wic/plugin.py | 2 +-
 scripts/lib/wic/pluginbase.py | 2 +-
 scripts/lib/wic/utils/errors.py   | 2 +-
 scripts/lib/wic/utils/fs_related.py   | 2 +-
 scripts/lib/wic/utils/misc.py | 2 +-
 scripts/lib/wic/utils/partitionedfs.py| 2 +-
 scripts/lib/wic/utils/runner.py   | 2 +-
 scripts/pybootchartgui/pybootchartgui.py  | 2 +-
 16 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/bitbake/lib/progressbar.py b/bitbake/lib/progressbar.py
index b668647..114cdc1 100644
--- a/bitbake/lib/progressbar.py
+++ b/bitbake/lib/progressbar.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 # -*- coding: iso-8859-1 -*-
 #
 # progressbar  - Text progressbar library for python.
diff --git a/scripts/lib/wic/conf.py b/scripts/lib/wic/conf.py
index d5419f8..be34355 100644
--- a/scripts/lib/wic/conf.py
+++ b/scripts/lib/wic/conf.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2011 Intel, Inc.
 #
diff --git a/scripts/lib/wic/creator.py b/scripts/lib/wic/creator.py
index a4b19ac..2219377 100644
--- a/scripts/lib/wic/creator.py
+++ b/scripts/lib/wic/creator.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2011 Intel, Inc.
 #
diff --git a/scripts/lib/wic/imager/baseimager.py 
b/scripts/lib/wic/imager/baseimager.py
index 5bcd2f7..e830527 100644
--- a/scripts/lib/wic/imager/baseimager.py
+++ b/scripts/lib/wic/imager/baseimager.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2007 Red Hat  Inc.
 # Copyright (c) 2009, 2010, 2011 Intel, Inc.
diff --git a/scripts/lib/wic/kickstart/__init__.py 
b/scripts/lib/wic/kickstart/__init__.py
index 4f5b778..6000982 100644
--- a/scripts/lib/wic/kickstart/__init__.py
+++ b/scripts/lib/wic/kickstart/__init__.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2007 Red Hat, Inc.
 # Copyright (c) 2009, 2010, 2011 Intel, Inc.
diff --git a/scripts/lib/wic/kickstart/custom_commands/micboot.py 
b/scripts/lib/wic/kickstart/custom_commands/micboot.py
index 66d1678..d162142 100644
--- a/scripts/lib/wic/kickstart/custom_commands/micboot.py
+++ b/scripts/lib/wic/kickstart/custom_commands/micboot.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2008, 2009, 2010 Intel, Inc.
 #
diff --git a/scripts/lib/wic/kickstart/custom_commands/micpartition.py 
b/scripts/lib/wic/kickstart/custom_commands/micpartition.py
index 59a87fb..43d04f1 100644
--- a/scripts/lib/wic/kickstart/custom_commands/micpartition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/micpartition.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Marko Saukko marko.sau...@cybercom.com
 #
diff --git a/scripts/lib/wic/msger.py b/scripts/lib/wic/msger.py
index 9afc85b..9f557e7 100644
--- a/scripts/lib/wic/msger.py
+++ b/scripts/lib/wic/msger.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 # vim: ai ts=4 sts=4 et sw=4
 #
 # Copyright (c) 2009, 2010, 2011 Intel, Inc.
diff --git a/scripts/lib/wic/plugin.py b/scripts/lib/wic/plugin.py
index 61c5859..41a8017 100644
--- a/scripts/lib/wic/plugin.py
+++ b/scripts/lib/wic/plugin.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2011 Intel, Inc.
 #
diff --git a/scripts/lib/wic/pluginbase.py b/scripts/lib/wic/pluginbase.py
index b8b3a46..e3de9ba 100644
--- a/scripts/lib/wic/pluginbase.py
+++ b/scripts/lib/wic/pluginbase.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python -tt
 #
 # Copyright (c) 2011 Intel, Inc.
 #
diff --git a/scripts/lib/wic/utils/errors.py b/scripts/lib/wic/utils/errors.py
index 86e230a..9410311 100644
--- a/scripts/lib/wic/utils/errors.py
+++ b/scripts/lib/wic/utils/errors.py
@@ -1,4 +1,4 @@
-#!/usr

[OE-core] [RFC 2/2] meta: rm __pycache__ in glib.inc::do_install_append()

[Martin Hundebøll]

Python3 places bytecode files in./ __pycache__ instead of simple .pyc
and .pyo in the script folder.

With python3 as the default python binary (i.e. /usr/bin/python -
python3), the building glib-2.0 fails in do_package() due to the mentioned
python cache files not being installed/removed. Fix this by removing the
__pycache__ folder during do_install_append(), as is done with *.pyc and
*.pyo.

Signed-off-by: Martin Hundebøll mar...@hundeboll.net
---
 meta/recipes-core/glib-2.0/glib.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-core/glib-2.0/glib.inc 
b/meta/recipes-core/glib-2.0/glib.inc
index 993320b..77644a9 100644
--- a/meta/recipes-core/glib-2.0/glib.inc
+++ b/meta/recipes-core/glib-2.0/glib.inc
@@ -63,6 +63,7 @@ do_install_append () {
# Remove some unpackaged files
rm -f ${D}${datadir}/glib-2.0/codegen/*.pyc
rm -f ${D}${datadir}/glib-2.0/codegen/*.pyo
+   rm -rf ${D}${datadir}/glib-2.0/codegen/__pycache__/
 
# Some distros have both /bin/perl and /usr/bin/perl, but we set perl 
location
# for target as /usr/bin/perl, so fix it to /usr/bin/perl.
-- 
2.1.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC 0/2] Explicit use of python 2

[Martin Hundebøll]

Hi,

I am new to yocto/openembedded, and wanted to work with yocto on Arch
Linux. As you might know, Arch has python3 as it default version, but
seamlessly supports python2 scripts by using /usr/bin/python2 instead.

This will also be needed as more and more distributions start moving to
python3. Also, it would be a reasonable guideline to require the use of
/usr/bin/env python{2,3} for future patches.

I changed (almost) all uses of #!/usr/bin/python and #!/usr/bin/env python
to /usr/bin/env python2 and did a few patches to core packages that
assumes python2.

Feedback is welcome. I especially expect problems on older Ubuntu/Debian
releases, which didn't ship with the /usr/bin/python2 symlink by default.

// Martin

Martin Hundebøll (2):
  scripts: use python2 explicitly in shebangs
  meta: rm __pycache__ in glib.inc::do_install_append()

 bitbake/bin/bitbake  |  2 +-
 bitbake/bin/bitbake-diffsigs |  2 +-
 bitbake/bin/bitbake-dumpsig  |  2 +-
 bitbake/bin/bitbake-layers   |  2 +-
 bitbake/bin/bitbake-prserv   |  2 +-
 bitbake/bin/bitbake-selftest |  2 +-
 bitbake/bin/bitbake-worker   |  2 +-
 bitbake/bin/bitdoc   |  2 +-
 bitbake/bin/image-writer |  2 +-
 bitbake/bin/toaster  | 14 +++---
 bitbake/contrib/dump_cache.py|  2 +-
 bitbake/lib/bb/cooker.py |  2 +-
 bitbake/lib/bb/cookerdata.py |  2 +-
 bitbake/lib/bb/monitordisk.py|  2 +-
 bitbake/lib/bb/namedtuple_with_abc.py|  2 +-
 bitbake/lib/bb/parse/parse_py/BBHandler.py   |  2 +-
 bitbake/lib/bb/parse/parse_py/ConfHandler.py |  2 +-
 bitbake/lib/bb/parse/parse_py/__init__.py|  2 +-
 bitbake/lib/bb/pysh/lsprof.py|  2 +-
 bitbake/lib/bb/runqueue.py   |  2 +-
 bitbake/lib/bb/taskdata.py   |  2 +-
 bitbake/lib/bb/ui/crumbs/builddetailspage.py |  2 +-
 bitbake/lib/bb/ui/crumbs/builder.py  |  2 +-
 bitbake/lib/bb/ui/crumbs/hobpages.py |  2 +-
 bitbake/lib/bb/ui/crumbs/imageconfigurationpage.py   |  2 +-
 bitbake/lib/bb/ui/crumbs/imagedetailspage.py |  2 +-
 bitbake/lib/bb/ui/crumbs/packageselectionpage.py |  2 +-
 bitbake/lib/bb/ui/crumbs/recipeselectionpage.py  |  2 +-
 bitbake/lib/bb/ui/crumbs/sanitycheckpage.py  |  2 +-
 bitbake/lib/bb/ui/hob.py |  2 +-
 bitbake/lib/progressbar.py   |  2 +-
 bitbake/lib/toaster/manage.py|  2 +-
 .../lib/toaster/orm/fixtures/orm_views_testdata.json |  2 +-
 meta/classes/package_rpm.bbclass |  2 +-
 meta/lib/oeqa/runexported.py |  2 +-
 meta/recipes-connectivity/bind/bind_9.9.5.bb |  2 +-
 meta/recipes-core/glib-2.0/glib-2.0_2.38.2.bb|  1 +
 meta/recipes-core/glib-2.0/glib.inc  |  3 ++-
 .../pseudo/files/0002-pseudo_force_python2.patch | 20 
 meta/recipes-devtools/pseudo/pseudo_1.5.1.bb |  1 +
 meta/recipes-devtools/python/python-native_2.7.3.bb  |  4 ++--
 meta/recipes-devtools/python/python/cgi_py.patch |  2 +-
 meta/recipes-devtools/python/python3-native_3.3.3.bb |  2 +-
 meta/recipes-devtools/python/python3/cgi_py.patch|  4 ++--
 meta/recipes-devtools/qemu/qemu.inc  |  2 +-
 meta/recipes-gnome/gnome/gnome-doc-utils.inc |  2 +-
 .../use-usr-bin-env-for-python-in-xml2po.patch   |  4 ++--
 .../libglade/libglade-2.6.4/python_environment.patch |  2 +-
 .../builder/files/please_wait_dialog.py  |  2 +-
 ...001-rt-tests-Allow-for-user-specified-PYLIB.patch |  4 ++--
 ...hon-scripts-should-use-interpreter-from-env.patch |  8 
 .../createrepo/createrepo/rpm-createsolvedb.py   |  2 +-
 meta/recipes-support/createrepo/createrepo_0.4.11.bb |  4 ++--
 scripts/bitbake-whatchanged  |  2 +-
 scripts/buildhistory-collect-srcrevs |  2 +-
 scripts/buildhistory-diff|  2 +-
 scripts/cleanup-workdir  |  2 +-
 scripts/combo-layer  |  2 +-
 scripts/contrib/bbvars.py|  2 +-
 scripts/contrib/graph-tool   |  2 +-
 scripts/contrib/list-packageconfig-flags.py  |  2 +-
 scripts/contrib/python/generate-manifest-2.7.py  |  2 +-
 scripts/contrib/python/generate-manifest-3.3.py  |  2 +-
 scripts/cp-noerror   |  2 +-
 scripts/create-recipe| 12 ++--
 scripts/jhbuild/jhbuild2oe.py|  2

[OE-core] [RFC 1/2] scripts: use python2 explicitly in shebangs

[Martin Hundebøll]

As distributions start moving to python3, scripts should explicity
request python2 in the shebang. This patch changes all occurences of
#!/usr/bin/python and #!/usr/bin/env python
to
#!/usr/bin/env python2

The change was done using the a massive oneline[1]; followed by some
manual replacements of direct shell calls to 'python and wrong
replacements (e.g. in patches).

This change was tested on Arch Linux, and succeeds in building
core-image-minimal. More extended builds will likely fail due to the use
of python[^23], but should easily be fixed.

To the best of my knowledge, recent versions of Ubuntu ship with
/usr/bin/python2 pointing to python2.7 (see [2]), but users of older
versions might need to create this symlink manually.

Comments and suggestions are welcome!

[1] git grep -l -E /usr/bin/env python|/usr/bin/python \
| xargs sed -Ei '/python3/!s/\/usr\/bin\/env \
python(.*)/\/usr\/bin\/env \
python2\1/;/python3/!s/\/usr\/bin\/python(.*)/\
\/usr\/bin\/env python2\1/'
[2] https://wiki.ubuntu.com/Python/3

Signed-off-by: Martin Hundebøll mar...@hundeboll.net
---
 bitbake/bin/bitbake  |  2 +-
 bitbake/bin/bitbake-diffsigs |  2 +-
 bitbake/bin/bitbake-dumpsig  |  2 +-
 bitbake/bin/bitbake-layers   |  2 +-
 bitbake/bin/bitbake-prserv   |  2 +-
 bitbake/bin/bitbake-selftest |  2 +-
 bitbake/bin/bitbake-worker   |  2 +-
 bitbake/bin/bitdoc   |  2 +-
 bitbake/bin/image-writer |  2 +-
 bitbake/bin/toaster  | 14 +++---
 bitbake/contrib/dump_cache.py|  2 +-
 bitbake/lib/bb/cooker.py |  2 +-
 bitbake/lib/bb/cookerdata.py |  2 +-
 bitbake/lib/bb/monitordisk.py|  2 +-
 bitbake/lib/bb/namedtuple_with_abc.py|  2 +-
 bitbake/lib/bb/parse/parse_py/BBHandler.py   |  2 +-
 bitbake/lib/bb/parse/parse_py/ConfHandler.py |  2 +-
 bitbake/lib/bb/parse/parse_py/__init__.py|  2 +-
 bitbake/lib/bb/pysh/lsprof.py|  2 +-
 bitbake/lib/bb/runqueue.py   |  2 +-
 bitbake/lib/bb/taskdata.py   |  2 +-
 bitbake/lib/bb/ui/crumbs/builddetailspage.py |  2 +-
 bitbake/lib/bb/ui/crumbs/builder.py  |  2 +-
 bitbake/lib/bb/ui/crumbs/hobpages.py |  2 +-
 bitbake/lib/bb/ui/crumbs/imageconfigurationpage.py   |  2 +-
 bitbake/lib/bb/ui/crumbs/imagedetailspage.py |  2 +-
 bitbake/lib/bb/ui/crumbs/packageselectionpage.py |  2 +-
 bitbake/lib/bb/ui/crumbs/recipeselectionpage.py  |  2 +-
 bitbake/lib/bb/ui/crumbs/sanitycheckpage.py  |  2 +-
 bitbake/lib/bb/ui/hob.py |  2 +-
 bitbake/lib/progressbar.py   |  2 +-
 bitbake/lib/toaster/manage.py|  2 +-
 .../lib/toaster/orm/fixtures/orm_views_testdata.json |  2 +-
 meta/classes/package_rpm.bbclass |  2 +-
 meta/lib/oeqa/runexported.py |  2 +-
 meta/recipes-connectivity/bind/bind_9.9.5.bb |  2 +-
 meta/recipes-core/glib-2.0/glib-2.0_2.38.2.bb|  1 +
 meta/recipes-core/glib-2.0/glib.inc  |  2 +-
 .../pseudo/files/0002-pseudo_force_python2.patch | 20 
 meta/recipes-devtools/pseudo/pseudo_1.5.1.bb |  1 +
 meta/recipes-devtools/python/python-native_2.7.3.bb  |  4 ++--
 meta/recipes-devtools/python/python/cgi_py.patch |  2 +-
 meta/recipes-devtools/python/python3-native_3.3.3.bb |  2 +-
 meta/recipes-devtools/python/python3/cgi_py.patch|  4 ++--
 meta/recipes-devtools/qemu/qemu.inc  |  2 +-
 meta/recipes-gnome/gnome/gnome-doc-utils.inc |  2 +-
 .../use-usr-bin-env-for-python-in-xml2po.patch   |  4 ++--
 .../libglade/libglade-2.6.4/python_environment.patch |  2 +-
 .../builder/files/please_wait_dialog.py  |  2 +-
 ...001-rt-tests-Allow-for-user-specified-PYLIB.patch |  4 ++--
 ...hon-scripts-should-use-interpreter-from-env.patch |  8 
 .../createrepo/createrepo/rpm-createsolvedb.py   |  2 +-
 meta/recipes-support/createrepo/createrepo_0.4.11.bb |  4 ++--
 scripts/bitbake-whatchanged  |  2 +-
 scripts/buildhistory-collect-srcrevs |  2 +-
 scripts/buildhistory-diff|  2 +-
 scripts/cleanup-workdir  |  2 +-
 scripts/combo-layer  |  2 +-
 scripts/contrib/bbvars.py|  2 +-
 scripts/contrib/graph-tool   |  2 +-
 scripts/contrib/list-packageconfig-flags.py  |  2 +-
 scripts/contrib/python/generate-manifest-2.7.py

Re: [OE-core] [RFC 1/2] scripts: use python2 explicitly in shebangs

[Martin Hundebøll]

Hi Paul,

On 2014-10-29 14:05, Paul Barker wrote:

On 29 October 2014 11:18, Martin Hundebøll mar...@hundeboll.net wrote:

As distributions start moving to python3, scripts should explicity
request python2 in the shebang. This patch changes all occurences of
 #!/usr/bin/python and #!/usr/bin/env python
to
 #!/usr/bin/env python2



snip lots


diff --git a/scripts/oe-buildenv-internal b/scripts/oe-buildenv-internal
index bba6f8f..b514aa5 100755
--- a/scripts/oe-buildenv-internal
+++ b/scripts/oe-buildenv-internal
@@ -33,7 +33,7 @@ fi
  # sanity.bbclass because bitbake's source code doesn't even pass
  # parsing stage when used with python v3, so we catch it here so we
  # can offer a meaningful error message.
-py_v3_check=`/usr/bin/env python --version 21 | grep Python 3`
+py_v3_check=`/usr/bin/env python2 --version 21 | grep Python 3`
  if [ $py_v3_check !=  ]; then
 echo 2 Bitbake is not compatible with python v3
 echo 2 Please set up python v2 as your default python interpreter
@@ -43,7 +43,7 @@ fi
  # Similarly, we now have code that doesn't parse correctly with older
  # versions of Python, and rather than fixing that and being eternally
  # vigilant for any other new feature use, just check the version here.
-py_v26_check=`python -c 'import sys; print sys.version_info = (2,7,3)'`
+py_v26_check=`python2 -c 'import sys; print sys.version_info = (2,7,3)'`
  if [ $py_v26_check != True ]; then
 echo 2 BitBake requires Python 2.7.3 or later
 return 1


This is the section of shell script which checks the python version
when you run 'oe-init-build-env'. If you want to change the way python
is ran I'd start by looking at this code. If this script could
automatically try 'python2' if 'python' is found to be Python 3, and
ensure the correct interpreter is used in all commands that run after
the environment is setup that would be fantastic. It'd probably be a
mess to implement though.


True and true. It would be nice with some auto-config which sets up the 
environment to use the right python binary/symlink, and shouldn't be too 
messy if it's doable through something similar to virtualenv. I've seen 
similar approaches done with autoconf in gnome's jhbuild (although it 
failed on Arch too).


Yet, it's still needed to correct all the places using /usr/bin/python 
directly instead of depending on the environment.



The change as it currently stands looks confusing as py_v3_check is
set by running python2.


It becomes kind of redundant, yes. Maybe spit out a warning and some 
hints if default is python v3? A lot of packages might still depend on 
python2 as default, so users should be warned...



I used to use a virtualenv to temporarily set Python 2 as the default
for the shell in which I ran bitbake commands. That was a while ago
though, all my current builds are running on Ubuntu 12.04/14.04 server
instances which I access via ssh from my Arch Linux box.


Using a compatible distro-install is of course the easiest way to get 
going. Could also work with a virtual machine.


If this has interest, I will work on a python snippet to detect the 
possible python version, and setup the environment correspondingly.


Thanks,
Martin
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core