Re: [OE-core] [PATCH] flex: Upgrade to 2.6.4

[Fan Xin]

OK, send this to meta-oe for test.

Fan Xin

On 2017年06月09日 10:42, Khem Raj wrote:

On Thu, Jun 8, 2017 at 6:39 PM, Fan Xin  wrote:

1. Upgrade flex from 2.6.0 to 2.6.4

2. Change the SRC_URI
Change SRC_URI from sourceforge to github due to the flex on sourceforge 
has not been updated more than one year.



how has it been tested ?
I think it needs to be tested with meta-oe before it gets into OE-core.


Signed-off-by: Fan Xin 
---
  meta/recipes-devtools/flex/{flex_2.6.0.bb => flex_2.6.4.bb} | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)
  rename meta/recipes-devtools/flex/{flex_2.6.0.bb => flex_2.6.4.bb} (90%)

diff --git a/meta/recipes-devtools/flex/flex_2.6.0.bb 
b/meta/recipes-devtools/flex/flex_2.6.4.bb
similarity index 90%
rename from meta/recipes-devtools/flex/flex_2.6.0.bb
rename to meta/recipes-devtools/flex/flex_2.6.4.bb
index ab35b09..038aa37 100644
--- a/meta/recipes-devtools/flex/flex_2.6.0.bb
+++ b/meta/recipes-devtools/flex/flex_2.6.4.bb
@@ -10,7 +10,7 @@ BBCLASSEXTEND = "native nativesdk"

  LIC_FILES_CHKSUM = "file://COPYING;md5=e4742cf92e89040b39486a6219b68067"

-SRC_URI = "${SOURCEFORGE_MIRROR}/flex/flex-${PV}.tar.bz2 \
+SRC_URI = "https://github.com/westes/flex/files/981163/${PN}-${PV}.tar.gz \
 file://run-ptest \
 file://do_not_create_pdf_doc.patch \
 
file://0001-tests-add-a-target-for-building-tests-without-runnin.patch \
@@ -19,8 +19,8 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/flex/flex-${PV}.tar.bz2 \
 ${@bb.utils.contains('PTEST_ENABLED', '1', '', 
'file://disable-tests.patch', d)} \
 "

-SRC_URI[md5sum] = "266270f13c48ed043d95648075084d59"
-SRC_URI[sha256sum] = 
"24e611ef5a4703a191012f80c1027dc9d12555183ce0ecd46f3636e587e9b8e9"
+SRC_URI[md5sum] = "2882e3179748cc9f9c23ec593d6adc8d"
+SRC_URI[sha256sum] = 
"e87aae032bf07c26f85ac0ed3250998c37621d95f8bd748b31f15b33c45ee995"

  # Flex has moved to github from 2.6.1 onwards
  UPSTREAM_CHECK_URI = "https://github.com/westes/flex/releases;
--
1.9.1

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


--
=
株式会社富士通コンピュータテクノロジーズ
組込みシステム技術統括部　第一ファームウェア技術部
樊　昕 Fan Xin
fan@jp.fujitsu.com

┏┓ 
┗■ 【ubinux V15】のリリースを開始しました！
「SDN(Open vSwitch)」や「クラウド管理(OpenStack Heat）」などに対応
-
詳細＞＞http://elsc.utsfd.cs.fujitsu.co.jp/location_elsc.php?id=0024
※"ubinux"は組込み装置向け当社独自のLinuxディストリビューションです
━━━
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] rng-tools: Fix textrels on 32bit x86

[Khem Raj]

When testing core-image-sato with hardening flags, it fails with
SIGSEGV in libc.so during relocation time

This is due to relocations in .text [textrel]
build QA points it out clearly during qemux86 build as well

AssertionError: 2 != 0 : Log: 
/mnt/a/oe/build/tmp/work/qemux86-bec-linux-musl/core-image-sato/1.0-r0/dmesg_output.log
---
Central error: [   19.043597] rngd[525]: segfault at 80098bb7 ip b77b14fc sp 
bfe9b380 error 7 in libc.so[b774c000+97000]

Signed-off-by: Khem Raj 
---
 .../rng-tools-5-fix-textrels-on-PIC-x86.patch  | 104 +
 meta/recipes-support/rng-tools/rng-tools_5.bb  |   1 +
 2 files changed, 105 insertions(+)
 create mode 100644 
meta/recipes-support/rng-tools/rng-tools/rng-tools-5-fix-textrels-on-PIC-x86.patch

diff --git 
a/meta/recipes-support/rng-tools/rng-tools/rng-tools-5-fix-textrels-on-PIC-x86.patch
 
b/meta/recipes-support/rng-tools/rng-tools/rng-tools-5-fix-textrels-on-PIC-x86.patch
new file mode 100644
index 00..90c9d8c515
--- /dev/null
+++ 
b/meta/recipes-support/rng-tools/rng-tools/rng-tools-5-fix-textrels-on-PIC-x86.patch
@@ -0,0 +1,104 @@
+From: Francisco Blas Izquierdo Riera (klondike) 
+Subject: [PATCH] Fix assemby textrels on rdrand_asm.S on PIC x86
+
+This patch updates the fixes in the assembly in rdrand_asm.S in
+sys-apps/rng-tools-5 so it won't generate textrels on PIC systems.
+The main fixes are in the use of leal in SETPTR for such systems, the rest is
+the usual PIC support stuff.
+
+This should fix Gentoo bug #469962 and help fix #518210
+
+This patch is released under the GPLv2 or a higher version license as is the
+original file as long as the author and the tester are credited.
+
+Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=469962
+Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=518210
+Upstream-status: Not sent yet
+Signed-off-by: Francisco Blas Izquierdo Riera (klondike) 
+Reported-by: cilly 
+Reported-by: Manuel Rüger 
+Tested-by: Anthony Basile 
+
+Upstream-Status: Pending
+
+Index: rng-tools-5/rdrand_asm.S
+===
+--- rng-tools-5.orig/rdrand_asm.S
 rng-tools-5/rdrand_asm.S
+@@ -2,6 +2,7 @@
+  * Copyright (c) 2011-2014, Intel Corporation
+  * Authors: Fenghua Yu ,
+  *  H. Peter Anvin 
++ * PIC code by: Francisco Blas Izquierdo Riera (klondike) 

+  *
+  * This program is free software; you can redistribute it and/or modify it
+  * under the terms and conditions of the GNU General Public License,
+@@ -174,7 +175,19 @@ ENTRY(x86_rdseed_or_rdrand_bytes)
+   jmp 4b
+ ENDPROC(x86_rdseed_or_rdrand_bytes)
+ 
++#if defined(__PIC__)
++#define INIT_PIC() \
++  pushl   %ebx ; \
++  call__x86.get_pc_thunk.bx ; \
++  addl$_GLOBAL_OFFSET_TABLE_, %ebx
++#define END_PIC() \
++  popl%ebx
++#define SETPTR(var,ptr) leal (var)@GOTOFF(%ebx),ptr
++#else
++#define INIT_PIC()
++#define END_PIC()
+ #define SETPTR(var,ptr)   movl $(var),ptr
++#endif
+ #define PTR0  %eax
+ #define PTR1  %edx
+ #define PTR2  %ecx
+@@ -190,6 +203,7 @@ ENTRY(x86_aes_mangle)
+   movl8(%ebp), %eax
+   movl12(%ebp), %edx
+   push%esi
++  INIT_PIC()
+ #endif
+   movl$512, CTR3  /* Number of rounds */
+   
+@@ -280,6 +294,7 @@ offset = offset + 16
+   movdqa  %xmm7, (7*16)(PTR1)
+ 
+ #ifdef __i386__
++  END_PIC()
+   pop %esi
+   pop %ebp
+ #endif
+@@ -294,6 +309,7 @@ ENTRY(x86_aes_expand_key)
+   push%ebp
+   mov %esp, %ebp
+   movl8(%ebp), %eax
++  INIT_PIC()
+ #endif
+ 
+   SETPTR(aes_round_keys, PTR1)
+@@ -323,6 +339,7 @@ ENTRY(x86_aes_expand_key)
+   call1f
+ 
+ #ifdef __i386__
++  END_PIC()
+   pop %ebp
+ #endif
+   ret
+@@ -343,6 +360,16 @@ ENTRY(x86_aes_expand_key)
+ 
+ ENDPROC(x86_aes_expand_key)
+ 
++#if defined(__i386__) && defined(__PIC__)
++  .section
.text.__x86.get_pc_thunk.bx,"axG",@progbits,__x86.get_pc_thunk.bx,comdat
++  .globl  __x86.get_pc_thunk.bx
++  .hidden __x86.get_pc_thunk.bx
++  .type   __x86.get_pc_thunk.bx, @function
++__x86.get_pc_thunk.bx:
++  movl(%esp), %ebx
++  ret
++#endif
++
+   .bss
+   .balign 64
+ aes_round_keys:
diff --git a/meta/recipes-support/rng-tools/rng-tools_5.bb 
b/meta/recipes-support/rng-tools/rng-tools_5.bb
index 9329e8ad31..e2acaba91a 100644
--- a/meta/recipes-support/rng-tools/rng-tools_5.bb
+++ b/meta/recipes-support/rng-tools/rng-tools_5.bb
@@ -7,6 +7,7 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/gkernel/${BP}.tar.gz \
file://0002-Add-argument-to-control-the-libargp-dependency.patch \
file://underquote.patch \
file://uclibc-libuargp-configure.patch \
+   

[OE-core] [PATCH 1/3] testimage.bbclass: Correct the comment to state right dir for test cases

[Khem Raj]

Signed-off-by: Khem Raj 
---
 meta/classes/testimage.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 1185593a1b..589ef5db93 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -13,7 +13,7 @@
 
 # You can set (or append to) TEST_SUITES in local.conf to select the tests
 # which you want to run for your target.
-# The test names are the module names in meta/lib/oeqa/runtime.
+# The test names are the module names in meta/lib/oeqa/runtime/cases.
 # Each name in TEST_SUITES represents a required test for the image. (no 
skipping allowed)
 # Appending "auto" means that it will try to run all tests that are suitable 
for the image (each test decides that on it's own).
 # Note that order in TEST_SUITES is relevant: tests are run in an order such 
that
-- 
2.13.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] pulseaudio: disable PIE flags when hardened flags are enabled

[Khem Raj]

Fixes

WARNING: pulseaudio-10.0-r0 do_package_qa: QA Issue: ELF binary 
'/mnt/a/oe/build/tmp/work/i586-bec-linux-musl/pulseaudio/10.0-r0/packages-split/pulseaudio-server/usr/bin/pulseaudio'
 has relocations in .text [textrel]

This also makes bitbake -c testimage core-image-sato (hardened)
build to pass all tests

Fixes

AssertionError: 1 != 0 : Log: 
/mnt/a/oe/build/tmp/work/qemux86-bec-linux-musl/core-image-sato/1.0-r0/dmesg_output.log
---
Central error: [   20.726960] pulseaudio[729]: segfault at 80052b6c ip b771b4fc 
sp bfc97940 error 7 in libc.so[b76b6000+97000]

Signed-off-by: Khem Raj 
---
 meta/recipes-multimedia/pulseaudio/pulseaudio_10.0.bb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-multimedia/pulseaudio/pulseaudio_10.0.bb 
b/meta/recipes-multimedia/pulseaudio/pulseaudio_10.0.bb
index f3a85737fc..3ea35e592a 100644
--- a/meta/recipes-multimedia/pulseaudio/pulseaudio_10.0.bb
+++ b/meta/recipes-multimedia/pulseaudio/pulseaudio_10.0.bb
@@ -8,6 +8,8 @@ SRC_URI = 
"http://freedesktop.org/software/pulseaudio/releases/${BP}.tar.xz \
 SRC_URI[md5sum] = "4950d2799bf55ab91f6b7f990b7f0971"
 SRC_URI[sha256sum] = 
"a3186824de9f0d2095ded5d0d0db0405dc73133983c2fbb37291547e37462f57"
 
+SECURITY_CFLAGS = "${SECURITY_NO_PIE_CFLAGS}"
+
 do_compile_prepend() {
 mkdir -p ${S}/libltdl
 cp ${STAGING_LIBDIR}/libltdl* ${S}/libltdl
-- 
2.13.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] Fix imagetests with harderning flags

[Khem Raj]

This patchset is fixing packages to build without textrels when
security flags are turned on. When testing the image built with musl
it clearly segfaults and results in failures in imagetest

Tested with MACHINE=qemux86 TCLIBC=musl bitbake -ctestimage core-image-sato

ore-image-sato-1.0-r0 do_testimage_auto: SUMMARY:
core-image-sato-1.0-r0 do_testimage_auto: core-image-sato () - Ran 13 tests in 
19.907s
core-image-sato-1.0-r0 do_testimage_auto: core-image-sato - OK - All required 
tests passed
core-image-sato-1.0-r0 do_testimage_auto: RESULTS:
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - 
connman.ConnmanTest.test_connmand_help - Testcase 961: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - 
connman.ConnmanTest.test_connmand_running - Testcase 221: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - date.DateTest.test_date - 
Testcase 211: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - df.DfTest.test_df - 
Testcase 234: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - 
oe_syslog.SyslogTest.test_syslog_running - Testcase 201: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - 
oe_syslog.SyslogTestConfig.test_syslog_logger - Testcase 1149: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - 
oe_syslog.SyslogTestConfig.test_syslog_restart - Testcase 1150: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - 
oe_syslog.SyslogTestConfig.test_syslog_startup_config - Testcase 202: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - 
parselogs.ParseLogsTest.test_parselogs - Testcase 1059: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - ping.PingTest.test_ping - 
Testcase 964: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - scp.ScpTest.test_scp_file - 
Testcase 220: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - ssh.SSHTest.test_ssh - 
Testcase 224: PASSED
core-image-sato-1.0-r0 do_testimage_auto: RESULTS - 
xorg.XorgTest.test_xorg_running - Testcase 1151: PASSED
NOTE: Tasks Summary: Attempted 5732 tasks of which 5717 didn't need to be rerun 
and all succeeded.

The following changes since commit 186882ca62bf683b93cd7a250963921b89ba071f:

  buildhistory: skip tests if GitPython module is missing (2017-06-07 16:00:43 
+0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib kraj/hardening-fixes
  
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=kraj/hardening-fixes

Khem Raj (3):
  testimage.bbclass: Correct the comment to state right dir for test
cases
  pulseaudio: disable PIE flags when hardened flags are enabled
  rng-tools: Fix textrels on 32bit x86

 meta/classes/testimage.bbclass |   2 +-
 .../pulseaudio/pulseaudio_10.0.bb  |   2 +
 .../rng-tools-5-fix-textrels-on-PIC-x86.patch  | 104 +
 meta/recipes-support/rng-tools/rng-tools_5.bb  |   1 +
 4 files changed, 108 insertions(+), 1 deletion(-)
 create mode 100644 
meta/recipes-support/rng-tools/rng-tools/rng-tools-5-fix-textrels-on-PIC-x86.patch

-- 
2.13.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] flex: Upgrade to 2.6.4

[Khem Raj]

On Thu, Jun 8, 2017 at 6:39 PM, Fan Xin  wrote:
> 1. Upgrade flex from 2.6.0 to 2.6.4
>
> 2. Change the SRC_URI
>Change SRC_URI from sourceforge to github due to the flex on sourceforge 
> has not been updated more than one year.
>

how has it been tested ?
I think it needs to be tested with meta-oe before it gets into OE-core.

> Signed-off-by: Fan Xin 
> ---
>  meta/recipes-devtools/flex/{flex_2.6.0.bb => flex_2.6.4.bb} | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>  rename meta/recipes-devtools/flex/{flex_2.6.0.bb => flex_2.6.4.bb} (90%)
>
> diff --git a/meta/recipes-devtools/flex/flex_2.6.0.bb 
> b/meta/recipes-devtools/flex/flex_2.6.4.bb
> similarity index 90%
> rename from meta/recipes-devtools/flex/flex_2.6.0.bb
> rename to meta/recipes-devtools/flex/flex_2.6.4.bb
> index ab35b09..038aa37 100644
> --- a/meta/recipes-devtools/flex/flex_2.6.0.bb
> +++ b/meta/recipes-devtools/flex/flex_2.6.4.bb
> @@ -10,7 +10,7 @@ BBCLASSEXTEND = "native nativesdk"
>
>  LIC_FILES_CHKSUM = "file://COPYING;md5=e4742cf92e89040b39486a6219b68067"
>
> -SRC_URI = "${SOURCEFORGE_MIRROR}/flex/flex-${PV}.tar.bz2 \
> +SRC_URI = "https://github.com/westes/flex/files/981163/${PN}-${PV}.tar.gz \
> file://run-ptest \
> file://do_not_create_pdf_doc.patch \
> 
> file://0001-tests-add-a-target-for-building-tests-without-runnin.patch \
> @@ -19,8 +19,8 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/flex/flex-${PV}.tar.bz2 \
> ${@bb.utils.contains('PTEST_ENABLED', '1', '', 
> 'file://disable-tests.patch', d)} \
> "
>
> -SRC_URI[md5sum] = "266270f13c48ed043d95648075084d59"
> -SRC_URI[sha256sum] = 
> "24e611ef5a4703a191012f80c1027dc9d12555183ce0ecd46f3636e587e9b8e9"
> +SRC_URI[md5sum] = "2882e3179748cc9f9c23ec593d6adc8d"
> +SRC_URI[sha256sum] = 
> "e87aae032bf07c26f85ac0ed3250998c37621d95f8bd748b31f15b33c45ee995"
>
>  # Flex has moved to github from 2.6.1 onwards
>  UPSTREAM_CHECK_URI = "https://github.com/westes/flex/releases;
> --
> 1.9.1
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] flex: Upgrade to 2.6.4

[Fan Xin]

1. Upgrade flex from 2.6.0 to 2.6.4

2. Change the SRC_URI
   Change SRC_URI from sourceforge to github due to the flex on sourceforge has 
not been updated more than one year.

Signed-off-by: Fan Xin 
---
 meta/recipes-devtools/flex/{flex_2.6.0.bb => flex_2.6.4.bb} | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
 rename meta/recipes-devtools/flex/{flex_2.6.0.bb => flex_2.6.4.bb} (90%)

diff --git a/meta/recipes-devtools/flex/flex_2.6.0.bb 
b/meta/recipes-devtools/flex/flex_2.6.4.bb
similarity index 90%
rename from meta/recipes-devtools/flex/flex_2.6.0.bb
rename to meta/recipes-devtools/flex/flex_2.6.4.bb
index ab35b09..038aa37 100644
--- a/meta/recipes-devtools/flex/flex_2.6.0.bb
+++ b/meta/recipes-devtools/flex/flex_2.6.4.bb
@@ -10,7 +10,7 @@ BBCLASSEXTEND = "native nativesdk"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=e4742cf92e89040b39486a6219b68067"
 
-SRC_URI = "${SOURCEFORGE_MIRROR}/flex/flex-${PV}.tar.bz2 \
+SRC_URI = "https://github.com/westes/flex/files/981163/${PN}-${PV}.tar.gz \
file://run-ptest \
file://do_not_create_pdf_doc.patch \

file://0001-tests-add-a-target-for-building-tests-without-runnin.patch \
@@ -19,8 +19,8 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/flex/flex-${PV}.tar.bz2 \
${@bb.utils.contains('PTEST_ENABLED', '1', '', 
'file://disable-tests.patch', d)} \
"
 
-SRC_URI[md5sum] = "266270f13c48ed043d95648075084d59"
-SRC_URI[sha256sum] = 
"24e611ef5a4703a191012f80c1027dc9d12555183ce0ecd46f3636e587e9b8e9"
+SRC_URI[md5sum] = "2882e3179748cc9f9c23ec593d6adc8d"
+SRC_URI[sha256sum] = 
"e87aae032bf07c26f85ac0ed3250998c37621d95f8bd748b31f15b33c45ee995"
 
 # Flex has moved to github from 2.6.1 onwards
 UPSTREAM_CHECK_URI = "https://github.com/westes/flex/releases;
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] dpkg: Add missing RDEPENDS for dpkg-perl

[Tom Rini]

In order to be able to use dpkg-perl on a system various stock perl
modules must also be installed on the system.  Create the list of
required modules based on a read of the code and testing with additional
utilities and list them in RDEPENDS_${PN}-perl.

Signed-off-by: Tom Rini 
---
 meta/recipes-devtools/dpkg/dpkg.inc | 19 +++
 1 file changed, 19 insertions(+)

diff --git a/meta/recipes-devtools/dpkg/dpkg.inc 
b/meta/recipes-devtools/dpkg/dpkg.inc
index a61839b21540..836bb1a82252 100644
--- a/meta/recipes-devtools/dpkg/dpkg.inc
+++ b/meta/recipes-devtools/dpkg/dpkg.inc
@@ -69,6 +69,25 @@ RPROVIDES_update-alternatives-dpkg += "update-alternatives"
 PACKAGES += "${PN}-perl"
 FILES_${PN}-perl = "${libdir}/perl/${PERLVERSION}"
 
+RDEPENDS_${PN}-perl += "perl-module-carp perl-module-constant \
+perl-module-cwd perl-module-digest \
+perl-module-digest-md5 perl-module-errno \
+perl-module-exporter perl-module-fcntl \
+perl-module-feature perl-module-file-basename \
+perl-module-file-compare perl-module-file-copy \
+perl-module-file-find perl-module-file-path \
+perl-module-file-spec perl-module-file-temp \
+perl-module-list-util perl-module-overload \
+perl-module-parent perl-module-storable \
+perl-module-filehandle perl-module-io \
+perl-module-io-handle perl-module-io-seekable \
+perl-module-posix perl-module-scalar-util \
+perl-module-selectsaver perl-module-symbol \
+perl-module-term-ansicolor perl-module-tie-handle \
+perl-module-tie-hash perl-module-storable \
+perl-module-time-hires perl-module-time-piece \
+perl-module-xsloader"
+
 # Split out start-stop-daemon to its own package.  Note that it
 # is installed in a different directory than the one used for
 # the bitbake version.
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] dpkg: Fix perl modules by moving them to the versioned perl directory

[Tom Rini]

In order for the dpkg perl modules to be used the must reside in the
versioned perl library directory (as to be in the default include path).
Be explicit about this location in our FILES_${PN}-perl directive, so
that if this breaks in the future, the recipe will fail).  We can now
drop the custom do_configure as it wasn't fixing this problem.

Signed-off-by: Tom Rini 
---
 meta/recipes-devtools/dpkg/dpkg.inc | 14 --
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/meta/recipes-devtools/dpkg/dpkg.inc 
b/meta/recipes-devtools/dpkg/dpkg.inc
index 870117a3250c..a61839b21540 100644
--- a/meta/recipes-devtools/dpkg/dpkg.inc
+++ b/meta/recipes-devtools/dpkg/dpkg.inc
@@ -9,7 +9,7 @@ RDEPENDS_${PN}_class-native = ""
 
 UPSTREAM_CHECK_URI = "${DEBIAN_MIRROR}/main/d/dpkg/"
 
-inherit autotools gettext perlnative pkgconfig systemd
+inherit autotools gettext perlnative pkgconfig systemd perl-version
 
 python () {
 if not bb.utils.contains('DISTRO_FEATURES', 'sysvinit', True, False, d):
@@ -20,8 +20,8 @@ python () {
 export PERL = "${bindir}/perl"
 PERL_class-native = "${STAGING_BINDIR_NATIVE}/perl-native/perl"
 
-export PERL_LIBDIR = "${libdir}/perl"
-PERL_LIBDIR_class-native = "${libdir}/perl-native/perl"
+export PERL_LIBDIR = "${libdir}/perl/${PERLVERSION}"
+PERL_LIBDIR_class-native = "${libdir}/perl-native/perl/${PERLVERSION}"
 
 EXTRA_OECONF = "\
--disable-dselect \
@@ -37,12 +37,6 @@ PACKAGECONFIG[liblzma] = "--with-liblzma,--without-liblzma, 
xz"
 EXTRA_OECONF += "TAR=tar"
 EXTRA_OECONF_append_class-target = " DEB_HOST_ARCH=${DPKG_ARCH}"
 
-do_configure () {
-echo >> ${S}/m4/compiler.m4
-sed -i -e 's#PERL_LIBDIR=.*$#PERL_LIBDIR="${libdir}/perl"#' ${S}/configure
-autotools_do_configure
-}
-
 do_install_append () {
if [ "${PN}" = "dpkg-native" ]; then
# update-alternatives doesn't have an offline mode
@@ -73,7 +67,7 @@ FILES_update-alternatives-dpkg = 
"${bindir}/update-alternatives ${localstatedir}
 RPROVIDES_update-alternatives-dpkg += "update-alternatives"
 
 PACKAGES += "${PN}-perl"
-FILES_${PN}-perl = "${libdir}/perl"
+FILES_${PN}-perl = "${libdir}/perl/${PERLVERSION}"
 
 # Split out start-stop-daemon to its own package.  Note that it
 # is installed in a different directory than the one used for
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] cpan-base.bbclass: Move PERLVERSION logic to a new file

[Tom Rini]

It is possible for non-CPAN recipes to contain perl modules.  These perl
modules must reside in the versioned perl library directory in order to
work in normal circumstances..  Export this logic to a separate class so
that it can be reused without the rest of the cpan logic.

Without this, dpkg will not export its perl code to the correct location
and will not be found by utilities that expect to use it.

Signed-off-by: Tom Rini 
---
 meta/classes/cpan-base.bbclass| 24 +---
 meta/classes/perl-version.bbclass | 24 
 2 files changed, 25 insertions(+), 23 deletions(-)
 create mode 100644 meta/classes/perl-version.bbclass

diff --git a/meta/classes/cpan-base.bbclass b/meta/classes/cpan-base.bbclass
index 55ac0526953a..577fcd63ce9f 100644
--- a/meta/classes/cpan-base.bbclass
+++ b/meta/classes/cpan-base.bbclass
@@ -7,27 +7,7 @@ FILES_${PN} += "${libdir}/perl ${datadir}/perl"
 DEPENDS  += "${@["perl", "perl-native"][(bb.data.inherits_class('native', 
d))]}"
 RDEPENDS_${PN} += "${@["perl", ""][(bb.data.inherits_class('native', d))]}"
 
-PERL_OWN_DIR = "${@["", "/perl-native"][(bb.data.inherits_class('native', 
d))]}"
-
-# Determine the staged version of perl from the perl configuration file
-# Assign vardepvalue, because otherwise signature is changed before and after
-# perl is built (from None to real version in config.sh).
-get_perl_version[vardepvalue] = "${PERL_OWN_DIR}"
-def get_perl_version(d):
-import re
-cfg = d.expand('${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/config.sh')
-try:
-f = open(cfg, 'r')
-except IOError:
-return None
-l = f.readlines();
-f.close();
-r = re.compile("^version='(\d*\.\d*\.\d*)'")
-for s in l:
-m = r.match(s)
-if m:
-return m.group(1)
-return None
+inherit perl-version
 
 def is_target(d):
 if not bb.data.inherits_class('native', d):
@@ -36,5 +16,3 @@ def is_target(d):
 
 PERLLIBDIRS = "${libdir}/perl"
 PERLLIBDIRS_class-native = "${libdir}/perl-native"
-PERLVERSION := "${@get_perl_version(d)}"
-PERLVERSION[vardepvalue] = ""
diff --git a/meta/classes/perl-version.bbclass 
b/meta/classes/perl-version.bbclass
new file mode 100644
index ..fafe68a775f7
--- /dev/null
+++ b/meta/classes/perl-version.bbclass
@@ -0,0 +1,24 @@
+PERL_OWN_DIR = "${@["", "/perl-native"][(bb.data.inherits_class('native', 
d))]}"
+
+# Determine the staged version of perl from the perl configuration file
+# Assign vardepvalue, because otherwise signature is changed before and after
+# perl is built (from None to real version in config.sh).
+get_perl_version[vardepvalue] = "${PERL_OWN_DIR}"
+def get_perl_version(d):
+import re
+cfg = d.expand('${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/config.sh')
+try:
+f = open(cfg, 'r')
+except IOError:
+return None
+l = f.readlines();
+f.close();
+r = re.compile("^version='(\d*\.\d*\.\d*)'")
+for s in l:
+m = r.match(s)
+if m:
+return m.group(1)
+return None
+
+PERLVERSION := "${@get_perl_version(d)}"
+PERLVERSION[vardepvalue] = ""
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCHv2 0/2] Multi-threaded RPM support

[Leonardo Sandoval]

On Thu, 2017-06-08 at 17:42 +0300, Alexander Kanavin wrote:
> Changes in version 2: use openmp for multi-processing instead of NSPR 
> library, as
> requested by upstream. Add a link to the upstream submission to patches.
> 
> These two patches add multi-threaded features to RPM to speed up the 
> do_package_write_rpm task.
> 
> Specifically:
> 
> 1) Multi-threaded xz compressor is used instead of default single-threaded 
> gz. This has the most
> dramatic effect when a recipe produces a smaller number of large-sized 
> packages.
> 
> 2) Packages creation is run in multiple threads via thread pools. This is 
> most beneficial when
> a recipe produces a large amount of small packages.
> 
> Some not very scientific benchmarks for time and .rpm sizes
> (time is measured for do_package_write_rpm tasks only):
> 

Tried v2 again and I am getting this problem (segmentation faults):

http://errors.yoctoproject.org/Errors/Build/38812/


The idea I have is to get some numbers from buildstats and have better
insight of the impact of this series. BTW, I tried core-image-sato.



> webkitgtk
> before: 9m12s 1550M
> after: 1m40s 858M
> 
> glibc-locale
> before: 2m52s 125M
> after: 30s 56M
> 
> glibc
> before: 46s 54M
> after: 13s 38M
> 
> perl
> before: 1m09s 63M
> after: 45s 42M
> 
> python3
> before: 30s 38M
> after: 18s 24M
> 
> The following changes since commit 576821ea0a7558b626ccc87e9ae0e9ee40864956:
> 
>   bitbake: bitbake-layers: check layer dependencies before adding (2017-06-06 
> 19:52:51 +0100)
> 
> are available in the git repository at:
> 
>   git://git.yoctoproject.org/poky-contrib akanavin/parallel-rpm
>   
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=akanavin/parallel-rpm
> 
> Alexander Kanavin (2):
>   package_rpm.bbclass: use multithreaded xz compression
>   rpm: run binary package generation via thread pools
> 
>  meta/classes/package_rpm.bbclass   |   2 +
>  ...y-package-building-into-a-separate-functi.patch |  83 
>  ...-binary-package-creation-via-thread-pools.patch | 125 
>  ...c-make-operations-over-string-pools-threa.patch | 207 
>  ...c-remove-static-local-variables-from-buil.patch | 216 
> +
>  meta/recipes-devtools/rpm/rpm_git.bb   |   4 +
>  6 files changed, 637 insertions(+)
>  create mode 100644 
> meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
>  create mode 100644 
> meta/recipes-devtools/rpm/files/0002-Run-binary-package-creation-via-thread-pools.patch
>  create mode 100644 
> meta/recipes-devtools/rpm/files/0003-rpmstrpool.c-make-operations-over-string-pools-threa.patch
>  create mode 100644 
> meta/recipes-devtools/rpm/files/0004-build-pack.c-remove-static-local-variables-from-buil.patch
> 
> -- 
> 2.11.0
> 


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] context: allow test execution with buildhistory unless version-going-backwards is present

[leonardo . sandoval . gonzalez]

From: Leonardo Sandoval 

Users constantly running oe-selftest find annoying to disable buildhistory 
before running
'oe-selftest' then enabling after it. To lessen the buildhistory restriction, 
allow buildhistory
unless 'version-going-backwards' is present on ERROR_QA bitbake variable.

[YOCTO #11580]

Signed-off-by: Leonardo Sandoval 
---
 meta/lib/oeqa/selftest/context.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/context.py 
b/meta/lib/oeqa/selftest/context.py
index ca73070c0b..f061b987b0 100644
--- a/meta/lib/oeqa/selftest/context.py
+++ b/meta/lib/oeqa/selftest/context.py
@@ -150,7 +150,8 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
 _check_required_env_variables(["BUILDDIR"])
 _check_presence_meta_selftest()
 
-if "buildhistory.bbclass" in self.tc.td["BBINCLUDED"]:
+# buildhistory introduce QA 'version-going-backwards' errors so stop 
if the latter are enabled
+if "buildhistory.bbclass" in self.tc.td["BBINCLUDED"] and 
'version-going-backwards' in self.tc.td['ERROR_QA']:
 self.tc.logger.error("You have buildhistory enabled already and 
this isn't recommended for selftest, please disable it first.")
 raise OEQAPreRun
 
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCHv2 2/2] rpm: run binary package generation via thread pools

[Mark Hatle]

On 6/8/17 12:23 PM, Alexander Kanavin wrote:
> On 06/08/2017 07:55 PM, Mark Hatle wrote:
>>> +@@ -565,8 +559,8 @@ static rpmRC packageBinary(rpmSpec spec, Package pkg, 
>>> const char *cookie, int ch
>>> +   headerCopyTags(spec->packages->header, pkg->header, copyTags);
>>> +   
>>> +   headerPutString(pkg->header, RPMTAG_RPMVERSION, VERSION);
>>> +-  headerPutString(pkg->header, RPMTAG_BUILDHOST, buildHost());
>>> +-  headerPutUint32(pkg->header, RPMTAG_BUILDTIME, getBuildTime(), 1);
>>> ++  headerPutString(pkg->header, RPMTAG_BUILDHOST, buildHost);
>>> ++  headerPutUint32(pkg->header, RPMTAG_BUILDTIME, , 1);
>>
>> I don't see what the advantage of this change over the current approach is --
>> unless the first call is in a thread so the variable can clash.  If that is 
>> the
>> case, it may make more sense to ensure that these two are run very early in 
>> the
>> build process before it threads.
> 
> They're run from threads, and they do clash, and I had spent a whole day 
> tracing data corruption coredumps back to these two.
> 
> This is explained in the commit message:
> 
> "Their use is causing difficult to diagnoze data races when building 
> multiple packages in parallel..."
> 
> And I had changed the code so that they're run early in the build 
> process, and just once. See below...

Yes, but you were no longer calling them and moved to passing in a variable.
The point of my comment is two fold.

One, by changing the behavior of the function you now HAVE to record the value
or if you run it again you will get a different result.  (Likely not what is
desired in the general case...)

..and two you also had to change the function calls to pass in this single 
value.

Assuming that moving the call earlier (like you did) works, and the static
values are retrievable by the function.  You could have fixed this with the
snipped below (without storing the values).  This would simplify the patch.

(If there is something in the way the threads work with openmp that makes the
static variable not transfer from the main execution thread to the children
threads -- that is a different issue and should be stated in the commit.)

> 
>>> +@@ -629,6 +623,8 @@ static struct binaryPackageTaskData* 
>>> runBinaryPackageTasks(rpmSpec spec, const c
>>> + struct binaryPackageTaskData *tasks = NULL;
>>> + struct binaryPackageTaskData *task = NULL;
>>> + struct binaryPackageTaskData *prev = NULL;
>>> ++rpm_time_t buildTime = getBuildTime();
>>> ++char *host = buildHost();
>>> +
>>> + for (Package pkg = spec->packages; pkg != NULL; pkg = pkg->next) {
>>> + task = rcalloc(1, sizeof(*task));
> 
> 
>  right there.
> 
> Please read the code more carefully :-)
> 
> 
> Alex
> 

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/2] Yocto Compatible 2.0 support code

[Patrick Ohly]

On Thu, 2017-06-08 at 10:28 -0500, Joshua Watt wrote:
> Sure. I wouldn't suggest using an if statement for "just anything", you
> can surely do terrible things that way. It would (by convention) be
> restricted to the same sorts of things that the conditional includes
> allow now. On a similar token, you can do the same sorts of terrible
> things with conditional includes as currently proposed because it has
> the same enforcement policy (i.e. "by convention").

I'm starting to wonder whether this "convention" can be strengthened
with additional warnings. The code which currently evaluates the include
parameter could record in the datastore the original expression and what
it evaluated to, then later when the recipe gets finalized, an event
handler can check whether evaluating the expression still gives the same
result.

This would also be useful for "inherit". I remember struggling to
understand why certain image type classes kept getting inherited despite
changing IMAGE_FSTYPES - it turned out, that change had to be made
earlier.

That's neither an argument for nor against the "if" check - the same
could be done for that. Just something that occurred to me.

> On the other hand, perhaps the range of terrible things that can be
> done extends to more than just how you conditionally include something.
> *What* is conditionally included might also require some scrutiny. As
> you have alluded to, overrides are probably the best option for
> variables, so putting them in a conditional include file is probably
> not ideal. Forcing people to move the things that have to be
> conditional to a separate file might actually be detrimental in a
> number of ways:
>  1) It might encourage recipe writers to do more in the include file
> than they maybe should so that they don't have to make a plethora of
> files.
>  2) It might make it harder to verify that what the recipe writers did
> is correct since the context of what they are doing is removed from the
> parent recipe.
> 
> IIRC the conditional syntax (if or conditional include) is really
> mostly needed for the parts of bitbake that don't allow overrides
> (addtask and such). If that is the desired restriction, it would not be
> difficult to have bitbake enforce that by only allowing the subset of
> things that don't support overrides to be in the body of a if
> statement. This would be more difficult with conditional includes
> unless some other bitbake syntax was added.

There's some truth to that IMHO, but I'm uncertain whether it warrants
introducing entirely new syntax. In refkit, I only ran into one
particular case were an include file was necessary.

> If that's the consensus, than I'm fine with that. From my perspective,
> conditional includes are just another (more difficult to use) form of
> an "if" statement, and making it difficult to do things conditionally
> doesn't necessarily make it better for anyone.

Making it hard sends the message that it shouldn't be used lightly.
Documentation will have to make clear that conditional includes are the
last resort when everything else isn't usable.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 5/6] linux-yocto/4.9: update to 4.9.31

[Bruce Ashfield]

Updating to the latest 4.9 -stable which comprises the following
commits:

   f1aa865ae5d4 Linux 4.9.31
   11214bd292ec xfs: Fix off-by-in in loop termination in 
xfs_find_get_desired_pgoff()
   75c5afd58d46 xfs: fix unaligned access in xfs_btree_visit_blocks
   7fb8ab8f0a38 xfs: avoid mount-time deadlock in CoW extent recovery
   e40c145c023d xfs: xfs_trans_alloc_empty
   0e542792a046 xfs: bad assertion for delalloc an extent that start at i_size
   f60d76efa91a xfs: BMAPX shouldn't barf on inline-format directories
   53c44c236f21 xfs: fix indlen accounting error on partial delalloc conversion
   54894ea3c542 xfs: fix use-after-free in xfs_finish_page_writeback
   d457f822817f xfs: reserve enough blocks to handle btree splits when remapping
   0ba833fe73d2 xfs: wait on new inodes during quotaoff dquot release
   2ea882d8ebc7 xfs: update ag iterator to support wait on new inodes
   e86b616b5b9e xfs: support ability to wait on new inodes
   10f0b2c3c225 xfs: fix up quotacheck buffer list error handling
   95487d4be1e9 xfs: prevent multi-fsb dir readahead from reading random blocks
   93bd169845e5 xfs: handle array index overrun in xfs_dir2_leaf_readbuf()
   99226b890d63 xfs: fix integer truncation in xfs_bmap_remap_alloc
   4e2762878a59 xfs: drop iolock from reclaim context to appease lockdep
   4e8163fc8159 xfs: actually report xattr extents via iomap
   de417ea6b0a6 xfs: fix over-copying of getbmap parameters from userspace
   c2ad2dc3d264 xfs: use dedicated log worker wq to avoid deadlock with cil wq
   3890d83805fe xfs: fix kernel memory exposure problems
   ca659e086fb7 xfs: rework the inline directory verifiers
   815414e7648b xfs: verify inline directory data forks
   11b485477285 xfs: fix off-by-one on max nr_pages in 
xfs_find_get_desired_pgoff()
   9c795fff53f9 xfs: use ->b_state to fix buffer I/O accounting release race
   c9eab63b9e62 xfs: Fix missed holes in SEEK_HOLE implementation
   670821b9482d drm/gma500/psb: Actually use VBT mode when it is found
   74b416367b4e mm/slub.c: trace free objects at KERN_INFO
   c1bb2a899b5f slub/memcg: cure the brainless abuse of sysfs attributes
   873f3b0ebbfe ksm: prevent crash after write_protect_page fails
   d5ecb4ca0da7 x86/boot: Use CROSS_COMPILE prefix for readelf
   d1cff071 RDMA/qib,hfi1: Fix MR reference count leak on write with 
immediate
   292f70cd9649 mm: consider memblock reservations for deferred memory 
initialization sizing
   1163e785b150 mlock: fix mlock count can not decrease in race condition
   d494cab70697 mm/migrate: fix refcount handling when 
!hugepage_migration_supported()
   7d8ef0e0bc1e ALSA: hda - apply STAC_9200_DELL_M22 quirk for Dell Latitude 
D430
   da856d05645c pcmcia: remove left-over %Z format
   ebd4c110fd0b drm/radeon: Fix vram_size/visible values in DRM_RADEON_GEM_INFO 
ioctl
   acc771fdaec7 drm/radeon: Unbreak HPD handling for r600+
   c8d25fcb5980 drm/radeon/ci: disable mclk switching for high refresh rates 
(v2)
   9869fb485cc6 scsi: mpt3sas: Force request partial completion alignment
   21f33b157721 nvme: avoid to use blk_mq_abort_requeue_list()
   510b0ec7f60f nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
   ae0578089242 nvme-rdma: support devices with queue size < 32
   34808d76dd77 HID: wacom: Have wacom_tpc_irq guard against possible NULL 
dereference
   69b1d90e6a0f ibmvscsis: Fix the incorrect req_lim_delta
   80569d0e09ad ibmvscsis: Clear left-over abort_cmd pointers
   49d33fd10070 iscsi-target: Always wait for kthread_should_stop() before 
kthread exit
   ecbf0f48d65e mmc: sdhci-iproc: suppress spurious interrupt with Multiblock 
read
   8735cf2291cd Revert "ACPI / button: Change default behavior to 
lid_init_state=open"
   32d8077f1e9b acpi, nfit: Fix the memory error check in nfit_handle_mce()
   68c83a379106 x86/MCE: Export memory_error()
   4472887cbd13 crypto: skcipher - Add missing API setkey checks
   63399974effb i2c: i2c-tiny-usb: fix buffer not being DMA capable
   d3b2d9ca90c2 drivers/tty: 8250: only call fintek_8250_probe when doing port 
I/O
   1d74fc36f3ec powerpc/spufs: Fix hash faults for kernel regions
   68a056175512 fs/ufs: Set UFS default maximum bytes per file
   1a658771d5e1 sparc/ftrace: Fix ftrace graph time measurement
   45ceb845ef34 sparc: Fix -Wstringop-overflow warning
   c1133c671a04 bpf: add bpf_clone_redirect to bpf_helper_changes_pkt_data
   988b9792b856 ipv4: add reference counting to metrics
   1de51502a025 sctp: fix ICMP processing if skb is non-linear
   4b81271ed1c3 tcp: avoid fastopen API to be used on AF_UNSPEC
   9e056584770b virtio-net: enable TSO/checksum offloads for Q-in-Q vlans
   9c6cfd5811bd be2net: Fix offload features for Q-in-Q packets
   5f595d529796 vlan: Fix tcp checksum offloads in Q-in-Q vlans
   cc6773b51bf3 net: phy: marvell: Limit errata to 88m1101
   4fb5fd27dec0 net/mlx5: Avoid using pending command interface slots
   1730a2b9e5b5 bonding: fix accounting of active ports in 3ad
   304b41014acb ipv6: fix out of bound writes in 

[OE-core] [PATCH 4/6] linux-yocto/4.10: update to v4.10.17

[Bruce Ashfield]

Updating to the latest 4.10 -stable releases, which comprise the
following changes:

   17a4d4803381 Linux 4.10.17
   291e716bb382 pstore: Shut down worker when unregistering
   e5590e3d9275 pstore: Fix flags to enable dumps on powerpc
   e3d4daa7f6d4 libnvdimm, pfn: fix 'npfns' vs section alignment
   116ada1a98ab libnvdimm: fix nvdimm_bus_lock() vs device_lock() ordering
   f92a2fe7cdb1 libnvdimm, pmem: fix a NULL pointer BUG in nd_pmem_notify
   72393c00f00a libnvdimm, region: fix flush hint detection crash
   b821a605977e ipmi: Fix kernel panic at ipmi_ssif_thread()
   c8e4805dd634 Bluetooth: hci_intel: add missing tty-device sanity check
   a8620f066675 Bluetooth: hci_bcm: add missing tty-device sanity check
   9a3054df3cf6 Bluetooth: Fix user channel for 32bit userspace on 64bit kernel
   58d479441029 tty: pty: Fix ldisc flush after userspace become aware of the 
data already
   9e3b9909bce3 serial: omap: suspend device on probe errors
   c1ce1f427e0a serial: omap: fix runtime-pm handling on unbind
   2578dd75ad12 serial: samsung: Use right device for DMA-mapping calls
   a78ddcd2a858 fscrypt: fix context consistency check when key(s) unavailable
   659ccd97668a f2fs: fix fs corruption due to zero inode page
   717946b469cf mm: fix data corruption due to stale mmap reads
   35223d76e2cf dax: prevent invalidation of mapped DAX entries
   fa7043b3a2e0 device-dax: fix sysfs attribute deadlock
   e1a19ef52919 device-dax: fix cdev leak
   81845f520179 md/raid1: avoid reusing a resync bio after error handling.
   23ebf6aa650d padata: free correct variable
   586aa5a6537f ovl: do not set overlay.opaque on non-dir create
   cf95696518f5 CIFS: add misssing SFM mapping for doublequote
   582fb96084c3 cifs: fix CIFS_IOC_GET_MNT_INFO oops
   4452b80eaef8 CIFS: fix oplock break deadlocks
   cd01b53b cifs: fix CIFS_ENUMERATE_SNAPSHOTS oops
   6ec05086dca9 cifs: fix leak in FSCTL_ENUM_SNAPS response handling
   b1b295efad9f CIFS: fix mapping of SFM_SPACE and SFM_PERIOD
   ae6c2182b853 SMB3: Work around mount failure when using SMB3 dialect to Macs
   6716949b0029 Set unicode flag on cifs echo request to avoid Mac error
   b7174f403828 Fix match_prepath()
   93697e1e5099 mm: prevent potential recursive reclaim due to clearing 
PF_MEMALLOC
   3302d94ab6f9 fs/block_dev: always invalidate cleancache in invalidate_bdev()
   f174092ec373 ceph: fix memory leak in __ceph_setxattr()
   594d4eca1c49 fs/xattr.c: zero out memory copied to userspace in getxattr
   49302d531325 orangefs: do not check possibly stale size on truncate
   42d86d92af64 orangefs: do not set getattr_time on orangefs_lookup
   d2c326c7ff6d orangefs: clean up oversize xattr validation
   4af222e1d681 orangefs: fix bounds check for listxattr
   e3e77f8ba5f6 ext4: evict inline data when writing to memory map
   fd469456ad6d jbd2: fix dbench4 performance regression for 'nobarrier' mounts
   e2e596f2888c perf annotate s390: Implement jump types for perf annotate
   d122da54d33e perf annotate s390: Fix perf annotate error -95 (4.10 
regression)
   ba6006004353 perf auxtrace: Fix no_size logic in 
addr_filter__resolve_kernel_syms()
   d187c9e135d3 IB/hfi1: Prevent kernel QP post send hard lockups
   04692adb3aac IB/mlx4: Reduce SRIOV multicast cleanup warning message to 
debug level
   e4e17bce1672 IB/mlx4: Fix ib device initialization error flow
   5d691b80ca4d IB/IPoIB: ibX: failed to create mcg debug file
   53bd2ccebd51 IB/core: For multicast functions, verify that LIDs are 
multicast LIDs
   b40c7a502b1e IB/core: Fix sysfs registration error flow
   f269df7bad86 iov_iter: don't revert iov buffer if csum error
   fc483680829a vfio/type1: Remove locked page accounting workqueue
   c85990cf511d dm thin: fix a memory leak when passing discard bio down
   bd0db3b70b59 dm rq: check blk_mq_register_dev() return value in 
dm_mq_init_request_queue()
   5d953aa1cd2e dm era: save spacemap metadata root after the pre-commit
   4c1dad842bfc dm crypt: rewrite (wipe) key in crypto layer using random data
   bce0767157c3 crypto: ccp - Change ISR handler method for a v5 CCP
   f106cd8575c4 crypto: ccp - Change ISR handler method for a v3 CCP
   595c7ad3c64b crypto: ccp - Disable interrupts early on unload
   e1adc5e04af5 crypto: ccp - Use only the relevant interrupt bits
   7ae1df9048db crypto: algif_aead - Require setkey before accept(2)
   fe51605c9512 crypto: s5p-sss - Close possible race for completed requests
   635aff41e59a block: fix blk_integrity_register to use template's 
interval_exp if not 0
   5c5d86be4f3f arm64: KVM: Fix decoding of Rt/Rt2 when trapping AArch32 CP 
accesses
   8348ffba88e5 KVM: arm/arm64: fix races in kvm_psci_vcpu_on
   74cbcb5afa75 KVM: x86: fix user triggerable warning in 
kvm_apic_accept_events()
   f22d13c45f2d perf/x86: Fix Broadwell-EP DRAM RAPL events
   29d07bb20ee5 um: Fix PTRACE_POKEUSER on x86_64
   efbd8cc8f6f8 x86, pmem: Fix cache flushing for iovec write < 8 bytes
   f0896a0d1e6d selftests/x86/ldt_gdt_32: Work around a 

[OE-core] [PATCH 6/6] linux-yocto/4.4: update to v4.4.71

[Bruce Ashfield]

Updating the the latest 4.4 -stable release which comprises the
following commits:

4bbbc7696405 Linux 4.4.71
9d65be36a7cc xfs: only return -errno or success from attr ->put_listent
1b03d85a4f37 xfs: in _attrlist_by_handle, copy the cursor back to userspace
c56605c69ba6 xfs: fix unaligned access in xfs_btree_visit_blocks
9f7b5da0570f xfs: bad assertion for delalloc an extent that start at i_size
3ba13d7f5b2b xfs: fix indlen accounting error on partial delalloc conversion
1d41dd5c1fd6 xfs: wait on new inodes during quotaoff dquot release
9d97d6a15265 xfs: update ag iterator to support wait on new inodes
8e25af0dc5ad xfs: support ability to wait on new inodes
cf55c35974e1 xfs: fix up quotacheck buffer list error handling
a76647a71c8e xfs: prevent multi-fsb dir readahead from reading random blocks
8caa9a54b32b xfs: handle array index overrun in xfs_dir2_leaf_readbuf()
0ace12c11401 xfs: fix over-copying of getbmap parameters from userspace
fe705621b9b4 xfs: fix off-by-one on max nr_pages in xfs_find_get_desired_pgoff()
b9a7816997a3 xfs: Fix missed holes in SEEK_HOLE implementation
03489bfc7830 mlock: fix mlock count can not decrease in race condition
7e13bab109ea mm/migrate: fix refcount handling when 
!hugepage_migration_supported()
4e4b72c0ee3d drm/gma500/psb: Actually use VBT mode when it is found
14bfe118dd7d slub/memcg: cure the brainless abuse of sysfs attributes
023a8b0925be ALSA: hda - apply STAC_9200_DELL_M22 quirk for Dell Latitude D430
85ddc41a6c4a pcmcia: remove left-over %Z format
69877793e23d drm/radeon: Unbreak HPD handling for r600+
15de2e4c90b7 drm/radeon/ci: disable mclk switching for high refresh rates (v2)
3529600b1601 scsi: mpt3sas: Force request partial completion alignment
58b7cb10f6e2 HID: wacom: Have wacom_tpc_irq guard against possible NULL 
dereference
c0fd730b678d mmc: sdhci-iproc: suppress spurious interrupt with Multiblock read
2ca57fc82436 i2c: i2c-tiny-usb: fix buffer not being DMA capable
1b5286ba9f13 vlan: Fix tcp checksum offloads in Q-in-Q vlans
e989f9bf2a9d net: phy: marvell: Limit errata to 88m1101
605b6b2b4d8a netem: fix skb_orphan_partial()
338f665acb4b ipv4: add reference counting to metrics
97f54575ff57 sctp: fix ICMP processing if skb is non-linear
fe22b6005538 tcp: avoid fastopen API to be used on AF_UNSPEC
d7ed7fcecf20 virtio-net: enable TSO/checksum offloads for Q-in-Q vlans
8380f16d0702 be2net: Fix offload features for Q-in-Q packets
38f02f2ce0ca ipv6: fix out of bound writes in __ip6_append_data()
3a854210f9a5 bridge: start hello_timer when enabling KERNEL_STP in br_stp_start
b543ccc4f627 qmi_wwan: add another Lenovo EM74xx device ID
94c0bf3cbb99 bridge: netlink: check vlan_default_pvid range
f76d54a8882e ipv6: Check ip6_find_1stfragopt() return value properly.
017fabead5c2 ipv6: Prevent overrun when parsing v6 header options
640bfcf232a9 net: Improve handling of failures on link and route dumps
7ede5c90fcdd tcp: eliminate negative reordering in tcp_clean_rtx_queue
ffa551def59c sctp: do not inherit ipv6_{mc|ac|fl}_list from parent
704e6c6b8651 sctp: fix src address selection if using secondary addresses for 
ipv6
90e3f8a55871 tcp: avoid fragmenting peculiar skbs in SACK
182abc4e74a1 s390/qeth: avoid null pointer dereference on OSN
21b871582375 s390/qeth: unbreak OSM and OSN support
2ac37098ee3d s390/qeth: handle sysfs error during initialization
d1428ee54073 ipv6/dccp: do not inherit ipv6_mc_list from parent
5f67a1663c03 dccp/tcp: do not inherit mc_list from parent
b9978c27454c sparc: Fix -Wstringop-overflow warning
b409ba3b0535 Linux 4.4.70
837bfdb41337 drivers: char: mem: Check for address space wraparound with mmap()
52cf24769487 nfsd: encoders mustn't use unitialized values in error cases
da922dc48dcf drm/edid: Add 10 bpc quirk for LGD 764 panel in HP zBook 17 G2
bc428e94070e PCI: Freeze PME scan before suspending devices
5f36c8b4e4a3 PCI: Fix pci_mmap_fits() for HAVE_PCI_RESOURCE_TO_USER platforms
6384f782a69c tracing/kprobes: Enforce kprobes teardown after testing
d5fb96b955ff osf_wait4(): fix infoleak
e07db0d720d3 genirq: Fix chained interrupt data ordering
1736f2b3de62 uwb: fix device quirk on big-endian hosts
ca19dd15e7bb metag/uaccess: Check access_ok in strncpy_from_user
2d9b2e780832 metag/uaccess: Fix access_ok()
98d5e84363ea iommu/vt-d: Flush the IOTLB to get rid of the initial kdump 
mappings
cb89b1f9dff9 staging: rtl8192e: rtl92e_get_eeprom_size Fix read size of 
EPROM_CMD.
427907e599fa staging: rtl8192e: fix 2 byte alignment of register BSSIDR.
8b26f53bf026 mm/huge_memory.c: respect FOLL_FORCE/FOLL_COW for thp
f03484fd5a3a xc2028: Fix use-after-free bug properly
e0188a556da6 arm64: documentation: document tagged pointer stack constraints
06dd8281a7d3 arm64: uaccess: ensure extension of access_ok() addr
c23fee69f5b5 arm64: xchg: hazard against entire exchange variable
acbab784a9b6 ARM: dts: at91: sama5d3_xplained: not all ADC channels are 
available
6ae3be7167b7 ARM: dts: at91: sama5d3_xplained: fix ADC vref
1ab43a598996 powerpc/64e: Fix hang when 

[OE-core] [PATCH 1/6] linux-yocto/meta: configuration changes (wifi, kexec and nft)

[Bruce Ashfield]

Making the following kernel configuration fragment changes available:

  c9f07d79f01d nftables: add more configuration options.
  8d3cf0ad6c0c common-pc-wifi.cfg: add CONFIG_MT7601U module
  55c9bf749c6d ktypes/developer: Enable CONFIG_KEXEC in config

Signed-off-by: Bruce Ashfield 
---
 meta/recipes-kernel/linux/linux-yocto-rt_4.10.bb   | 2 +-
 meta/recipes-kernel/linux/linux-yocto-rt_4.9.bb| 2 +-
 meta/recipes-kernel/linux/linux-yocto-tiny_4.10.bb | 2 +-
 meta/recipes-kernel/linux/linux-yocto-tiny_4.9.bb  | 2 +-
 meta/recipes-kernel/linux/linux-yocto_4.10.bb  | 2 +-
 meta/recipes-kernel/linux/linux-yocto_4.9.bb   | 2 +-
 6 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_4.10.bb 
b/meta/recipes-kernel/linux/linux-yocto-rt_4.10.bb
index fc235c98d435..46b84c99304e 100644
--- a/meta/recipes-kernel/linux/linux-yocto-rt_4.10.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_4.10.bb
@@ -12,7 +12,7 @@ python () {
 }
 
 SRCREV_machine ?= "d2c1ed3c0cbc0d8dcf429346f7ab2e96d470b504"
-SRCREV_meta ?= "79a4c1b415207961ba47c5d58f7625000218e98b"
+SRCREV_meta ?= "fdfa9de38ddb3362f9d9ff6b35e14dcb93da43b1"
 
 SRC_URI = 
"git://git.yoctoproject.org/linux-yocto-4.10.git;branch=${KBRANCH};name=machine 
\

git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.10;destsuffix=${KMETA}"
diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_4.9.bb 
b/meta/recipes-kernel/linux/linux-yocto-rt_4.9.bb
index c87c01c19337..f35ae5531294 100644
--- a/meta/recipes-kernel/linux/linux-yocto-rt_4.9.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_4.9.bb
@@ -12,7 +12,7 @@ python () {
 }
 
 SRCREV_machine ?= "095698fd99b452d95188ff6f2cce3df5127beb5c"
-SRCREV_meta ?= "fcc729412013996e17f7aa3953b21cb0a1521108"
+SRCREV_meta ?= "6300f53e21a119c8cd97988e3c44e0bb5e8594ce"
 
 SRC_URI = 
"git://git.yoctoproject.org/linux-yocto-4.9.git;branch=${KBRANCH};name=machine \

git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.9;destsuffix=${KMETA}"
diff --git a/meta/recipes-kernel/linux/linux-yocto-tiny_4.10.bb 
b/meta/recipes-kernel/linux/linux-yocto-tiny_4.10.bb
index 21ca972e252a..ee42155bac31 100644
--- a/meta/recipes-kernel/linux/linux-yocto-tiny_4.10.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-tiny_4.10.bb
@@ -10,7 +10,7 @@ KMETA = "kernel-meta"
 KCONF_BSP_AUDIT_LEVEL = "2"
 
 SRCREV_machine ?= "d2c1ed3c0cbc0d8dcf429346f7ab2e96d470b504"
-SRCREV_meta ?= "79a4c1b415207961ba47c5d58f7625000218e98b"
+SRCREV_meta ?= "fdfa9de38ddb3362f9d9ff6b35e14dcb93da43b1"
 
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
diff --git a/meta/recipes-kernel/linux/linux-yocto-tiny_4.9.bb 
b/meta/recipes-kernel/linux/linux-yocto-tiny_4.9.bb
index 919a25847cd1..c3b39cfd8dd5 100644
--- a/meta/recipes-kernel/linux/linux-yocto-tiny_4.9.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-tiny_4.9.bb
@@ -10,7 +10,7 @@ KMETA = "kernel-meta"
 KCONF_BSP_AUDIT_LEVEL = "2"
 
 SRCREV_machine ?= "2c31d7a45ae75159a7d991abdeb7002a4493af7f"
-SRCREV_meta ?= "fcc729412013996e17f7aa3953b21cb0a1521108"
+SRCREV_meta ?= "6300f53e21a119c8cd97988e3c44e0bb5e8594ce"
 
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
diff --git a/meta/recipes-kernel/linux/linux-yocto_4.10.bb 
b/meta/recipes-kernel/linux/linux-yocto_4.10.bb
index cbcd38e6d4d2..07454cf01fe1 100644
--- a/meta/recipes-kernel/linux/linux-yocto_4.10.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_4.10.bb
@@ -19,7 +19,7 @@ SRCREV_machine_qemux86 ?= 
"d2c1ed3c0cbc0d8dcf429346f7ab2e96d470b504"
 SRCREV_machine_qemux86-64 ?= "d2c1ed3c0cbc0d8dcf429346f7ab2e96d470b504"
 SRCREV_machine_qemumips64 ?= "229abf6f63652058db1308a0c07e59652d3e6a23"
 SRCREV_machine ?= "d2c1ed3c0cbc0d8dcf429346f7ab2e96d470b504"
-SRCREV_meta ?= "79a4c1b415207961ba47c5d58f7625000218e98b"
+SRCREV_meta ?= "fdfa9de38ddb3362f9d9ff6b35e14dcb93da43b1"
 
 SRC_URI = 
"git://git.yoctoproject.org/linux-yocto-4.10.git;name=machine;branch=${KBRANCH};
 \

git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.10;destsuffix=${KMETA}"
diff --git a/meta/recipes-kernel/linux/linux-yocto_4.9.bb 
b/meta/recipes-kernel/linux/linux-yocto_4.9.bb
index b986d11c3226..c3cb8594963b 100644
--- a/meta/recipes-kernel/linux/linux-yocto_4.9.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_4.9.bb
@@ -19,7 +19,7 @@ SRCREV_machine_qemux86 ?= 
"2c31d7a45ae75159a7d991abdeb7002a4493af7f"
 SRCREV_machine_qemux86-64 ?= "2c31d7a45ae75159a7d991abdeb7002a4493af7f"
 SRCREV_machine_qemumips64 ?= "bfdb9d2cf36f5ded1cd8d03e58465d9f6250bba8"
 SRCREV_machine ?= "2c31d7a45ae75159a7d991abdeb7002a4493af7f"
-SRCREV_meta ?= "fcc729412013996e17f7aa3953b21cb0a1521108"
+SRCREV_meta ?= "6300f53e21a119c8cd97988e3c44e0bb5e8594ce"
 
 SRC_URI = 
"git://git.yoctoproject.org/linux-yocto-4.9.git;name=machine;branch=${KBRANCH}; 
\

git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.9;destsuffix=${KMETA}"
-- 

[OE-core] [PATCH 3/6] linux-yocto-rt: 4.9-rt18

[Bruce Ashfield]

Integrating the 4.9-rt18 port that Paul Gortmaker has prepared:

 f458d12de7f1 v4.9.27-rt18
 82fcbd9f7e1f futex/rtmutex: Cure RT double blocking issue
 b4352b9fb961 futex: backported patches
 0874a0a35318 random: avoid preempt_disable()ed section
 07481e7a0e04 v4.9.27-rt17
 76c86c0787fe v4.9.20-rt16
 d394d677d188 rwsem/rt: Lift single reader restriction
 ad7b0ccf9f1f rtmutex: Provide locked slowpath
 f363d238326d rtmutex: Provide rt_mutex_lock_state()
 b3b4c3aa793e rtmutex: Make lock_killable work
 e3fa9e78c860 v4.9.20-rt15
 1e458a1a876d v4.9.18-rt14
 f57fd2fcd3e2 Add the rtmutex rework
 3829b7532ca9 lockdep: Fix per-cpu static objects
 bc1065b64955 v4.9.18-rt13
 095698fd99b4 Merge branch 'standard/base' into standard/preempt-rt/base

Signed-off-by: Bruce Ashfield 
---
 meta/recipes-kernel/linux/linux-yocto-rt_4.9.bb   | 4 ++--
 meta/recipes-kernel/linux/linux-yocto-tiny_4.9.bb | 2 +-
 meta/recipes-kernel/linux/linux-yocto_4.9.bb  | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_4.9.bb 
b/meta/recipes-kernel/linux/linux-yocto-rt_4.9.bb
index f35ae5531294..cf584be7eef6 100644
--- a/meta/recipes-kernel/linux/linux-yocto-rt_4.9.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_4.9.bb
@@ -11,8 +11,8 @@ python () {
 raise bb.parse.SkipPackage("Set PREFERRED_PROVIDER_virtual/kernel to 
linux-yocto-rt to enable it")
 }
 
-SRCREV_machine ?= "095698fd99b452d95188ff6f2cce3df5127beb5c"
-SRCREV_meta ?= "6300f53e21a119c8cd97988e3c44e0bb5e8594ce"
+SRCREV_machine ?= "f458d12de7f157fe050afc7c1fee08022876eae3"
+SRCREV_meta ?= "de57de86e23493d9356fc4cc0e92a3df58a29032"
 
 SRC_URI = 
"git://git.yoctoproject.org/linux-yocto-4.9.git;branch=${KBRANCH};name=machine \

git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.9;destsuffix=${KMETA}"
diff --git a/meta/recipes-kernel/linux/linux-yocto-tiny_4.9.bb 
b/meta/recipes-kernel/linux/linux-yocto-tiny_4.9.bb
index c3b39cfd8dd5..49047e9c9954 100644
--- a/meta/recipes-kernel/linux/linux-yocto-tiny_4.9.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-tiny_4.9.bb
@@ -10,7 +10,7 @@ KMETA = "kernel-meta"
 KCONF_BSP_AUDIT_LEVEL = "2"
 
 SRCREV_machine ?= "2c31d7a45ae75159a7d991abdeb7002a4493af7f"
-SRCREV_meta ?= "6300f53e21a119c8cd97988e3c44e0bb5e8594ce"
+SRCREV_meta ?= "de57de86e23493d9356fc4cc0e92a3df58a29032"
 
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
diff --git a/meta/recipes-kernel/linux/linux-yocto_4.9.bb 
b/meta/recipes-kernel/linux/linux-yocto_4.9.bb
index c3cb8594963b..e9fe07761de3 100644
--- a/meta/recipes-kernel/linux/linux-yocto_4.9.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_4.9.bb
@@ -19,7 +19,7 @@ SRCREV_machine_qemux86 ?= 
"2c31d7a45ae75159a7d991abdeb7002a4493af7f"
 SRCREV_machine_qemux86-64 ?= "2c31d7a45ae75159a7d991abdeb7002a4493af7f"
 SRCREV_machine_qemumips64 ?= "bfdb9d2cf36f5ded1cd8d03e58465d9f6250bba8"
 SRCREV_machine ?= "2c31d7a45ae75159a7d991abdeb7002a4493af7f"
-SRCREV_meta ?= "6300f53e21a119c8cd97988e3c44e0bb5e8594ce"
+SRCREV_meta ?= "de57de86e23493d9356fc4cc0e92a3df58a29032"
 
 SRC_URI = 
"git://git.yoctoproject.org/linux-yocto-4.9.git;name=machine;branch=${KBRANCH}; 
\

git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.9;destsuffix=${KMETA}"
-- 
2.5.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/6] kern-tools: avoid syntax errors when inheriting meta data

[Bruce Ashfield]

It is possible to inherit meta data for either patches + config
or for just config.

It is possible that the patch queue contains invalid (when
sourced) shell characters in the patch names, which throws a
syntax error and aborts processing.

The patch + config case was fixed some time ago, but we recently
stumbled onto the config-only case which was still not properly
quoted and hence safe.

This commit brings the config-only inherit in line with the
patch + config processing and we won't abort processing if
characters like () are in patch names.

Signed-off-by: Bruce Ashfield 
---
 meta/recipes-kernel/kern-tools/kern-tools-native_git.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb 
b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
index 5e65469a69ca..2217a31076a2 100644
--- a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
+++ b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = 
"file://git/tools/kgit;beginline=5;endline=9;md5=a6c2fa8aef1b
 
 DEPENDS = "git-native"
 
-SRCREV = "bd7447cd6274d764a129dcdc246cdbfd8c47b991"
+SRCREV = "9cd2b626d652bec10c6bc75275b35bfee74d447c"
 PR = "r12"
 PV = "0.2+git${SRCPV}"
 
-- 
2.5.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/6] kernel-yocto: consolidated pull request

[Bruce Ashfield]

Hi all,

This pull request includes -stable updates for all the kernels that are
currently in master.  I'll be dropping one kernel as I move to 4.12+ for
the fall release, but for now, they all continue to get updates.

I've done smoke build/boot testing on the updates, but the build matrix
is huge, so I'll keep an eye on any autobuilder results just in case
something slipped through.

We also have some configuration changes and a bug fix to the kern-tools
when a BSP includes a file and inhibits patches.

Cheers,

Bruce

The following changes since commit 4a7612c7a12b9a381fb8343ba9586272b889fc15:

  buildhistory: skip tests if GitPython module is missing (2017-06-07 16:00:49 
+0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib zedd/kernel
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel

Bruce Ashfield (6):
  linux-yocto/meta: configuration changes (wifi, kexec and nft)
  kern-tools: avoid syntax errors when inheriting meta data
  linux-yocto-rt: 4.9-rt18
  linux-yocto/4.10: update to v4.10.17
  linux-yocto/4.9: update to 4.9.31
  linux-yocto/4.4: update to v4.4.71

 .../kern-tools/kern-tools-native_git.bb  |  2 +-
 meta/recipes-kernel/linux/linux-yocto-rt_4.10.bb |  6 +++---
 meta/recipes-kernel/linux/linux-yocto-rt_4.4.bb  |  6 +++---
 meta/recipes-kernel/linux/linux-yocto-rt_4.9.bb  |  6 +++---
 meta/recipes-kernel/linux/linux-yocto-tiny_4.10.bb   |  6 +++---
 meta/recipes-kernel/linux/linux-yocto-tiny_4.4.bb|  6 +++---
 meta/recipes-kernel/linux/linux-yocto-tiny_4.9.bb|  6 +++---
 meta/recipes-kernel/linux/linux-yocto_4.10.bb| 20 ++--
 meta/recipes-kernel/linux/linux-yocto_4.4.bb | 20 ++--
 meta/recipes-kernel/linux/linux-yocto_4.9.bb | 20 ++--
 10 files changed, 49 insertions(+), 49 deletions(-)

-- 
2.5.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH][morty] linux-yocto/4.8: update to 4.8.24

[Bruce Ashfield]

Integration the -stable updates to 4.8, which comprise the following
commits:

   ae51b3b4efe6 Linux 4.8.24
   accb950219c0 nvme/pci: Disable on removal when disconnected
   995be68699ef padata: avoid race in reordering
   bea2fdcc6d3e blk: improve order of bio handling in generic_make_request()
   26690f5a8bdf mm: workingset: fix premature shadow node shrinking with cgroups
   f52ade55d6e9 MIPS: Lantiq: Fix cascaded IRQ setup
   feae78cdcaef ARM: dts: BCM5301X: Correct GIC_PPI interrupt flags
   e36a1a1a9695 ARM: BCM5301X: Add back handler ignoring external imprecise 
aborts
   26fbe60844a9 mm, hugetlb: use pte_present() instead of pmd_present() in 
follow_huge_pmd()
   d15202219abe mm: rmap: fix huge file mmap accounting in the memcg stats
   72f38e33793c x86/mce: Fix copy/paste error in exception table entries
   9e08a6f8e64e x86/mm/KASLR: Exclude EFI region from KASLR VA space 
randomization
   100aa553e31b x86/mm/64: Enable KASLR for vmemmap memory region
   2e5c45495433 drm/etnaviv: (re-)protect fence allocation with GPU mutex
   7f160b149090 drm/vc4: Allocate the right amount of space for boot-time CRTC 
state.
   38cdfd6153cf drm/radeon: Override fpfn for all VRAM placements in 
radeon_evict_flags
   ccd0fc5589ca KVM: kvm_io_bus_unregister_dev() should never fail
   332e2ac55d3b KVM: x86: clear bus pointer when destroyed
   fcbdbe1290d6 serial: mxs-auart: Fix baudrate calculation
   3b7bb3391f62 USB: fix linked-list corruption in rh_call_control()
   5f004babfd46 tty/serial: atmel: fix race condition (TX+DMA)
   4509ea7b4349 ACPI: Do not create a platform_device for IOAPIC/IOxAPIC
   12fddf4d255f ACPI: Fix incompatibility with mcount-based function graph 
tracing
   8805bbdea121 nfsd: map the ENOKEY to nfserr_perm for avoiding warning
   1fa18f8fad45 mmc: sdhci-of-at91: fix MMC_DDR_52 timing selection
   3a0df02f0387 mmc: sdhci: Disable runtime pm when the sdio_irq is enabled
   d742580939f5 ASoC: Intel: Skylake: fix invalid memory access due to wrong 
reference of pointer
   726514355250 ASoC: atmel-classd: fix audio clock rate
   0dd77376bd45 ALSA: hda - fix a problem for lineout on a Dell AIO machine
   12788ff3a917 ALSA: seq: Fix race during FIFO resize
   5ccedc21345f PCI: iproc: Save host bridge window resource in struct 
iproc_pcie
   61b133e83718 scsi: scsi_dh_alua: Ensure that alua_activate() calls the 
completion function
   ae4a71a0b8b4 scsi: scsi_dh_alua: Check scsi_device_get() return value
   1b82bcc866c3 scsi: libsas: fix ata xfer length
   0b9b5b603f27 scsi: sg: check length passed to SG_NEXT_CMD_LEN
   8b02a4b8e3b3 xfs: Use xfs_icluster_size_fsb() to calculate inode alignment 
mask
   a8eded7e1484 xfs: tune down agno asserts in the bmap code
   7650b45a6dd8 xfs: Use xfs_icluster_size_fsb() to calculate inode chunk 
alignment
   e9d4a4b55c86 xfs: don't reserve blocks for right shift transactions
   81f456210999 xfs: split indlen reservations fairly when under reserved
   842c9d864561 xfs: handle indlen shortage on delalloc extent merge
   f90db61ed148 xfs: don't fail xfs_extent_busy allocation
   590e612677e2 xfs: reset b_first_retry_time when clear the retry status of 
xfs_buf_t
   c493142b3a2c xfs: verify free block header fields
   4dda58920fb9 xfs: check for obviously bad level values in the bmbt root
   0faf06ef8b24 xfs: filter out obviously bad btree pointers
   6974d828ce0c xfs: fail _dir_open when readahead fails
   365ec079aeeb xfs: fix toctou race when locking an inode to access the data 
map
   06a1dbe68823 xfs: fix eofblocks race with file extending async dio writes
   fc1ff6abfb5d xfs: pull up iolock from xfs_free_eofblocks()
   2de02a1aa708 xen/setup: Don't relocate p2m over existing one
   bd37e332bc35 libceph: force GFP_NOIO for socket allocations
   14d57fe1fcd3 sched/rt: Add a missing rescheduling point
   461aa14a19df metag/ptrace: Reject partial NT_METAG_RPIPE writes
   35d078d3b1f5 metag/ptrace: Provide default TXSTATUS for short NT_PRSTATUS
   9519f4083b0d metag/ptrace: Preserve previous registers for short regset write
   b2d2e181ebe4 sparc/ptrace: Preserve previous registers for short regset write
   0a89a38ebbae mips/ptrace: Preserve previous registers for short regset write
   a303cc3a707b h8300/ptrace: Fix incorrect register transfer count
   576d881764eb c6x/ptrace: Remove useless PTRACE_SETREGSET implementation
   1661ca89ebd1 pinctrl: qcom: Don't clear status bit on irq_unmask
   5c52be1e1c55 virtio_balloon: init 1st buffer in stats vq
   171bd47e04fa KVM: x86: cleanup the page tracking SRCU instance
   37c063abc015 xfrm_user: validate XFRM_MSG_NEWAE incoming ESN size harder
   f5e8896582a0 xfrm_user: validate XFRM_MSG_NEWAE XFRMA_REPLAY_ESN_VAL 
replay_window
   ef704417015e xfrm: policy: init locks early
   54b82b92b678 crypto: algif_hash - avoid zero-sized array
   35a836ab3d9b fbcon: Fix vc attr at deinit
   d64422cfc106 drm: reference count event->completion
   5b8d397ea73a drm/bridge: analogix dp: Fix runtime PM state on driver bind
   

Re: [OE-core] [PATCHv2 2/2] rpm: run binary package generation via thread pools

[Alexander Kanavin]

On 06/08/2017 07:55 PM, Mark Hatle wrote:

+@@ -565,8 +559,8 @@ static rpmRC packageBinary(rpmSpec spec, Package pkg, 
const char *cookie, int ch
+   headerCopyTags(spec->packages->header, pkg->header, copyTags);
+   
+   headerPutString(pkg->header, RPMTAG_RPMVERSION, VERSION);
+-  headerPutString(pkg->header, RPMTAG_BUILDHOST, buildHost());
+-  headerPutUint32(pkg->header, RPMTAG_BUILDTIME, getBuildTime(), 1);
++  headerPutString(pkg->header, RPMTAG_BUILDHOST, buildHost);
++  headerPutUint32(pkg->header, RPMTAG_BUILDTIME, , 1);


I don't see what the advantage of this change over the current approach is --
unless the first call is in a thread so the variable can clash.  If that is the
case, it may make more sense to ensure that these two are run very early in the
build process before it threads.


They're run from threads, and they do clash, and I had spent a whole day 
tracing data corruption coredumps back to these two.


This is explained in the commit message:

"Their use is causing difficult to diagnoze data races when building 
multiple packages in parallel..."


And I had changed the code so that they're run early in the build 
process, and just once. See below...




+@@ -629,6 +623,8 @@ static struct binaryPackageTaskData* 
runBinaryPackageTasks(rpmSpec spec, const c
+ struct binaryPackageTaskData *tasks = NULL;
+ struct binaryPackageTaskData *task = NULL;
+ struct binaryPackageTaskData *prev = NULL;
++rpm_time_t buildTime = getBuildTime();
++char *host = buildHost();
+
+ for (Package pkg = spec->packages; pkg != NULL; pkg = pkg->next) {
+ task = rcalloc(1, sizeof(*task));



 right there.

Please read the code more carefully :-)


Alex
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCHv2 2/2] rpm: run binary package generation via thread pools

[Mark Hatle]

On 6/8/17 9:42 AM, Alexander Kanavin wrote:
> This greatly reduces build times when there is a large amount of small
> rpm packages to produce. The patches are rather invasive,
> and so will be submitted upstream.
> 
> Signed-off-by: Alexander Kanavin 
> ---
>  ...y-package-building-into-a-separate-functi.patch |  83 
>  ...-binary-package-creation-via-thread-pools.patch | 125 
>  ...c-make-operations-over-string-pools-threa.patch | 207 
>  ...c-remove-static-local-variables-from-buil.patch | 216 
> +
>  meta/recipes-devtools/rpm/rpm_git.bb   |   4 +
>  5 files changed, 635 insertions(+)
>  create mode 100644 
> meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
>  create mode 100644 
> meta/recipes-devtools/rpm/files/0002-Run-binary-package-creation-via-thread-pools.patch
>  create mode 100644 
> meta/recipes-devtools/rpm/files/0003-rpmstrpool.c-make-operations-over-string-pools-threa.patch
>  create mode 100644 
> meta/recipes-devtools/rpm/files/0004-build-pack.c-remove-static-local-variables-from-buil.patch
> 
> diff --git 
> a/meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
>  
> b/meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
> new file mode 100644
> index 000..3d8b12144e7
> --- /dev/null
> +++ 
> b/meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
> @@ -0,0 +1,83 @@
> +From b841b699e519438a66b661247c94efff63d0700e Mon Sep 17 00:00:00 2001
> +From: Alexander Kanavin 
> +Date: Thu, 25 May 2017 18:15:27 +0300
> +Subject: [PATCH 01/14] Split binary package building into a separate function
> +
> +So that it can be run as a thread pool task.
> +
> +Upstream-Status: Submitted 
> [https://github.com/rpm-software-management/rpm/pull/226]
> +Signed-off-by: Alexander Kanavin 
> +---
> + build/pack.c | 33 +
> + 1 file changed, 21 insertions(+), 12 deletions(-)
> +
> +diff --git a/build/pack.c b/build/pack.c
> +index 497300b96..891e6bdc3 100644
> +--- a/build/pack.c
>  b/build/pack.c
> +@@ -546,18 +546,13 @@ static rpmRC checkPackages(char *pkgcheck)
> + return RPMRC_OK;
> + }
> + 
> +-rpmRC packageBinaries(rpmSpec spec, const char *cookie, int cheating)
> ++static rpmRC packageBinary(rpmSpec spec, Package pkg, const char *cookie, 
> int cheating, char** filename)
> + {
> +-rpmRC rc;
> +-const char *errorString;
> +-Package pkg;
> +-char *pkglist = NULL;
> +-
> +-for (pkg = spec->packages; pkg != NULL; pkg = pkg->next) {
> +-char *fn;
> ++const char *errorString;
> ++rpmRC rc = RPMRC_OK;

The above change appears to be changing the spacing of the code.  RPM is
somewhat unique in how it does spacing.

Generally it's:

four spaces
   tab
   tab + four spaces
   tab + tab
repeat

(Note there are a lot of inconsistencies here, so I'd say follow what the
function itself is using, as much as you can.)

> + 
> + if (pkg->fileList == NULL)
> +-continue;
> ++return rc;
> + 
> + if ((rc = processScriptFiles(spec, pkg)))
> + return rc;
> +@@ -591,7 +586,7 @@ rpmRC packageBinaries(rpmSpec spec, const char *cookie, 
> int cheating)
> +  headerGetString(pkg->header, RPMTAG_NAME), errorString);
> + return RPMRC_FAIL;
> + }
> +-fn = rpmGetPath("%{_rpmdir}/", binRpm, NULL);
> ++*filename = rpmGetPath("%{_rpmdir}/", binRpm, NULL);

I'm not sure if it would be a cleaner change if you did (top of function):

char *fn = *filename;

and then kept this code as it is.  Might be up to the RPM community for that
type of comment.

> + if ((binDir = strchr(binRpm, '/')) != NULL) {
> + struct stat st;
> + char *dn;
> +@@ -613,14 +608,28 @@ rpmRC packageBinaries(rpmSpec spec, const char 
> *cookie, int cheating)
> + free(binRpm);
> + }
> + 
> +-rc = writeRPM(pkg, NULL, fn, NULL);
> ++rc = writeRPM(pkg, NULL, *filename, NULL);
> + if (rc == RPMRC_OK) {
> + /* Do check each written package if enabled */
> +-char *pkgcheck = rpmExpand("%{?_build_pkgcheck} ", fn, NULL);
> ++char *pkgcheck = rpmExpand("%{?_build_pkgcheck} ", *filename, NULL);
> + if (pkgcheck[0] != ' ') {
> + rc = checkPackages(pkgcheck);
> + }
> + free(pkgcheck);
> ++}
> ++return rc;
> ++}
> ++
> ++rpmRC packageBinaries(rpmSpec spec, const char *cookie, int cheating)
> ++{
> ++rpmRC rc;
> ++Package pkg;
> ++char *pkglist = NULL;
> ++
> ++for (pkg = spec->packages; pkg != NULL; pkg = pkg->next) {
> ++char *fn = NULL;
> ++rc = packageBinary(spec, pkg, cookie, cheating, );
> ++if (rc == RPMRC_OK) {

Re: [OE-core] [PATCH v3] runtime/dnf: Add new dnf test cases

[Alexander Kanavin]

On 06/07/2017 08:31 PM, Jose Perez Carranza wrote:

When I tried this test image was built without using states hence
"busybox*.rpm" and "curl*.rpm"where present under
"tmp/deploy/rpm/core2_64" hence the repo i taking tha packages form
there to add them to repo and used on the image, but when using sstates
those packages are not being built so are not present on local build
directory. is there any way to force those packages to be built to have
them available on tmp/deploy/rpm/core2_64?


At least one of the failures has nothing to do with sstate or repositories:

  File 
"/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x32/build/meta/lib/oeqa/runtime/cases/dnf.py", 
line 136, in test_dnf_exclude

if self.dnf('list curl %s' % excludepkg ,0):
  File 
"/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x32/build/meta/lib/oeqa/runtime/cases/dnf.py", 
line 18, in dnf

self.assertEqual(status, expected, message)
AssertionError: 1 != 0 : dnf list curl curl-dev
Error: No matching Packages to list


That line is asking if curl package is installed on the image, and, in 
core-image-sato's case it isn't, and so failure occurs. The repositories 
are not even accessed.


I haven't been able to reproduce the installroot failure, works fine here.


Alex

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 7/7] oeqa/core/loader: Fix filtering on test modules with submodules

[Aníbal Limón]

Our filtering allows to specify which tests to run using,

.[test_class].[test_name]

But the module name logic was restricted to only accept one level,
for example: runtime_test vs oelib.types, to support multiple
submodules use only the first part for filtering.

This allows to run the whole tests in a module with more than tree
levels.

Due to the ambiguity on the test filtering options with test cases
with more than tree levels the supported sytnax is,



or

.[submoduleN].[test_class].[test_name]

[YOCTO #11632]

Signed-off-by: Aníbal Limón 
---
 meta/lib/oeqa/core/loader.py | 70 +---
 1 file changed, 34 insertions(+), 36 deletions(-)

diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
index b9385ea..d110881 100644
--- a/meta/lib/oeqa/core/loader.py
+++ b/meta/lib/oeqa/core/loader.py
@@ -164,8 +164,11 @@ class OETestLoader(unittest.TestLoader):
 """
 Returns True if test case must be filtered, False otherwise.
 """
-# Filters by module.class.name
-module_name = case.__module__
+# XXX; If the module has more than one namespace only use
+# the first to support run the whole module specifying the
+# .[test_class].[test_name]
+module_name = case.__module__.split('.')[0]
+
 class_name = case.__class__.__name__
 test_name = case._testMethodName
 
@@ -280,6 +283,33 @@ class OETestLoader(unittest.TestLoader):
 
 return self.suiteClass(cases) if cases else big_suite
 
+def _filterModule(self, module):
+if module.__name__ in sys.builtin_module_names:
+msg = 'Tried to import %s test module but is a built-in'
+raise ImportError(msg % module.__name__)
+
+# XXX; If the module has more than one namespace only use
+# the first to support run the whole module specifying the
+# .[test_class].[test_name]
+module_name = module.__name__.split('.')[0]
+
+# Normal test modules are loaded if no modules were specified,
+# if module is in the specified module list or if 'all' is in
+# module list.
+# Underscore modules are loaded only if specified in module list.
+load_module = True if not module_name.startswith('_') \
+  and (not self.modules \
+   or module_name in self.modules \
+   or 'all' in self.modules) \
+   else False
+
+load_underscore = True if module_name.startswith('_') \
+  and module_name in self.modules \
+   else False
+
+return (load_module, load_underscore)
+
+
 # XXX After Python 3.5, remove backward compatibility hacks for
 # use_load_tests deprecation via *args and **kws.  See issue 16662.
 if sys.version_info >= (3,5):
@@ -287,23 +317,7 @@ class OETestLoader(unittest.TestLoader):
 """
 Returns a suite of all tests cases contained in module.
 """
-if module.__name__ in sys.builtin_module_names:
-msg = 'Tried to import %s test module but is a built-in'
-raise ImportError(msg % module.__name__)
-
-# Normal test modules are loaded if no modules were specified,
-# if module is in the specified module list or if 'all' is in
-# module list.
-# Underscore modules are loaded only if specified in module list.
-load_module = True if not module.__name__.startswith('_') \
-  and (not self.modules \
-   or module.__name__ in self.modules \
-   or 'all' in self.modules) \
-   else False
-
-load_underscore = True if module.__name__.startswith('_') \
-  and module.__name__ in self.modules \
-   else False
+load_module, load_underscore = self._filterModule(module)
 
 if load_module or load_underscore:
 return super(OETestLoader, self).loadTestsFromModule(
@@ -315,23 +329,7 @@ class OETestLoader(unittest.TestLoader):
 """
 Returns a suite of all tests cases contained in module.
 """
-if module.__name__ in sys.builtin_module_names:
-msg = 'Tried to import %s test module but is a built-in'
-raise ImportError(msg % module.__name__)
-
-# Normal test modules are loaded if no modules were specified,
-# if module is in the specified module list or if 'all' is in
-# module list.
-# Underscore modules are loaded only if specified in module list.
-load_module = True if not module.__name__.startswith('_') \

[OE-core] [PATCH 6/7] oeqa/cases/oelib: Change default case class to unittest.case.TestCase

[Aníbal Limón]

Some tests doesn't need call bitbake so it is better to use the
basic unittest case class.

[YOCTO #10828]

Signed-off-by: Aníbal Limón 
---
 meta/lib/oeqa/selftest/cases/oelib/elf.py | 4 ++--
 meta/lib/oeqa/selftest/cases/oelib/license.py | 6 +++---
 meta/lib/oeqa/selftest/cases/oelib/path.py| 4 ++--
 meta/lib/oeqa/selftest/cases/oelib/types.py   | 6 +++---
 meta/lib/oeqa/selftest/cases/oelib/utils.py   | 6 +++---
 5 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/oelib/elf.py 
b/meta/lib/oeqa/selftest/cases/oelib/elf.py
index 0451eba..74ee6a1 100644
--- a/meta/lib/oeqa/selftest/cases/oelib/elf.py
+++ b/meta/lib/oeqa/selftest/cases/oelib/elf.py
@@ -1,7 +1,7 @@
-from oeqa.selftest.case import OESelftestTestCase
+from unittest.case import TestCase
 import oe.qa
 
-class TestElf(OESelftestTestCase):
+class TestElf(TestCase):
 def test_machine_name(self):
 """
 Test elf_machine_to_string()
diff --git a/meta/lib/oeqa/selftest/cases/oelib/license.py 
b/meta/lib/oeqa/selftest/cases/oelib/license.py
index a6d9c9a..bfd9ed9 100644
--- a/meta/lib/oeqa/selftest/cases/oelib/license.py
+++ b/meta/lib/oeqa/selftest/cases/oelib/license.py
@@ -1,4 +1,4 @@
-from oeqa.selftest.case import OESelftestTestCase
+from unittest.case import TestCase
 import oe.license
 
 class SeenVisitor(oe.license.LicenseVisitor):
@@ -9,7 +9,7 @@ class SeenVisitor(oe.license.LicenseVisitor):
 def visit_Str(self, node):
 self.seen.append(node.s)
 
-class TestSingleLicense(OESelftestTestCase):
+class TestSingleLicense(TestCase):
 licenses = [
 "GPLv2",
 "LGPL-2.0",
@@ -37,7 +37,7 @@ class TestSingleLicense(OESelftestTestCase):
 self.parse(license)
 self.assertEqual(cm.exception.license, license)
 
-class TestSimpleCombinations(OESelftestTestCase):
+class TestSimpleCombinations(TestCase):
 tests = {
 "FOO": ["FOO", "BAR"],
 "BAZ & MOO": ["BAZ", "MOO"],
diff --git a/meta/lib/oeqa/selftest/cases/oelib/path.py 
b/meta/lib/oeqa/selftest/cases/oelib/path.py
index 2ae5eaf..75a27c0 100644
--- a/meta/lib/oeqa/selftest/cases/oelib/path.py
+++ b/meta/lib/oeqa/selftest/cases/oelib/path.py
@@ -1,11 +1,11 @@
-from oeqa.selftest.case import OESelftestTestCase
+from unittest.case import TestCase
 import oe, oe.path
 import tempfile
 import os
 import errno
 import shutil
 
-class TestRealPath(OESelftestTestCase):
+class TestRealPath(TestCase):
 DIRS = [ "a", "b", "etc", "sbin", "usr", "usr/bin", "usr/binX", 
"usr/sbin", "usr/include", "usr/include/gdbm" ]
 FILES = [ "etc/passwd", "b/file" ]
 LINKS = [
diff --git a/meta/lib/oeqa/selftest/cases/oelib/types.py 
b/meta/lib/oeqa/selftest/cases/oelib/types.py
index 99c8404..6b53aa6 100644
--- a/meta/lib/oeqa/selftest/cases/oelib/types.py
+++ b/meta/lib/oeqa/selftest/cases/oelib/types.py
@@ -1,7 +1,7 @@
-from oeqa.selftest.case import OESelftestTestCase
+from unittest.case import TestCase
 from oe.maketype import create
 
-class TestBooleanType(OESelftestTestCase):
+class TestBooleanType(TestCase):
 def test_invalid(self):
 self.assertRaises(ValueError, create, '', 'boolean')
 self.assertRaises(ValueError, create, 'foo', 'boolean')
@@ -31,7 +31,7 @@ class TestBooleanType(OESelftestTestCase):
 self.assertEqual(create('y', 'boolean'), True)
 self.assertNotEqual(create('y', 'boolean'), False)
 
-class TestList(OESelftestTestCase):
+class TestList(TestCase):
 def assertListEqual(self, value, valid, sep=None):
 obj = create(value, 'list', separator=sep)
 self.assertEqual(obj, valid)
diff --git a/meta/lib/oeqa/selftest/cases/oelib/utils.py 
b/meta/lib/oeqa/selftest/cases/oelib/utils.py
index 5bc5fff..9fb6c15 100644
--- a/meta/lib/oeqa/selftest/cases/oelib/utils.py
+++ b/meta/lib/oeqa/selftest/cases/oelib/utils.py
@@ -1,7 +1,7 @@
-from oeqa.selftest.case import OESelftestTestCase
+from unittest.case import TestCase
 from oe.utils import packages_filter_out_system, trim_version
 
-class TestPackagesFilterOutSystem(OESelftestTestCase):
+class TestPackagesFilterOutSystem(TestCase):
 def test_filter(self):
 """
 Test that oe.utils.packages_filter_out_system works.
@@ -31,7 +31,7 @@ class TestPackagesFilterOutSystem(OESelftestTestCase):
 self.assertEqual(pkgs, ["foo-data"])
 
 
-class TestTrimVersion(OESelftestTestCase):
+class TestTrimVersion(TestCase):
 def test_version_exception(self):
 with self.assertRaises(TypeError):
 trim_version(None, 2)
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 5/7] oeqa/core/loader: Allow unittest.TestCase's to be executed

[Aníbal Limón]

Currently there was a restriction to only execute tests that's
inherits from OETestCase but in some circunstancies the features
from the OEQA framework isn't needed so we need to support
basic unittests.

[YOCTO #10828]

Signed-off-by: Aníbal Limón 
---
 meta/lib/oeqa/core/loader.py | 12 +++-
 meta/lib/oeqa/core/runner.py | 25 ++---
 2 files changed, 21 insertions(+), 16 deletions(-)

diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
index 7cc4d4c..b9385ea 100644
--- a/meta/lib/oeqa/core/loader.py
+++ b/meta/lib/oeqa/core/loader.py
@@ -182,7 +182,7 @@ class OETestLoader(unittest.TestLoader):
 return True
 
 # Decorator filters
-if self.filters:
+if self.filters and isinstance(case, OETestCase):
 filters = self.filters.copy()
 case_decorators = [cd for cd in case.decorators
if cd.__class__ in self.used_filters]
@@ -200,7 +200,8 @@ class OETestLoader(unittest.TestLoader):
 return False
 
 def _getTestCase(self, testCaseClass, tcName):
-if not hasattr(testCaseClass, '__oeqa_loader'):
+if not hasattr(testCaseClass, '__oeqa_loader') and \
+issubclass(testCaseClass, OETestCase):
 # In order to support data_vars validation
 # monkey patch the default setUp/tearDown{Class} to use
 # the ones provided by OETestCase
@@ -227,7 +228,8 @@ class OETestLoader(unittest.TestLoader):
 setattr(testCaseClass, '__oeqa_loader', True)
 
 case = testCaseClass(tcName)
-setattr(case, 'decorators', [])
+if isinstance(case, OETestCase):
+setattr(case, 'decorators', [])
 
 return case
 
@@ -239,9 +241,9 @@ class OETestLoader(unittest.TestLoader):
 raise TypeError("Test cases should not be derived from TestSuite." 
\
 " Maybe you meant to derive %s from TestCase?" 
\
 % testCaseClass.__name__)
-if not issubclass(testCaseClass, self.caseClass):
+if not issubclass(testCaseClass, unittest.case.TestCase):
 raise TypeError("Test %s is not derived from %s" % \
-(testCaseClass.__name__, self.caseClass.__name__))
+(testCaseClass.__name__, unittest.case.TestCase.__name__))
 
 testCaseNames = self.getTestCaseNames(testCaseClass)
 if not testCaseNames and hasattr(testCaseClass, 'runTest'):
diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py
index 7ce718e..532b25b 100644
--- a/meta/lib/oeqa/core/runner.py
+++ b/meta/lib/oeqa/core/runner.py
@@ -121,9 +121,10 @@ class OETestResult(_TestResult):
 break
 
 oeid = -1
-for d in case.decorators:
-if hasattr(d, 'oeid'):
-oeid = d.oeid
+if hasattr(case, 'decorators'):
+for d in case.decorators:
+if hasattr(d, 'oeid'):
+oeid = d.oeid
 
 if fail:
 self.tc.logger.info("RESULTS - %s - Testcase %s: %s" % 
(case.id(),
@@ -188,9 +189,10 @@ class OETestRunner(_TestRunner):
 def _list_cases_without_id(logger, case):
 
 found_id = False
-for d in case.decorators:
-if isinstance(d, OETestID):
-found_id = True
+if hasattr(case, 'decorators'):
+for d in case.decorators:
+if isinstance(d, OETestID):
+found_id = True
 
 if not found_id:
 logger.info('oeid missing for %s' % case.id())
@@ -199,11 +201,12 @@ class OETestRunner(_TestRunner):
 oeid = None
 oetag = None
 
-for d in case.decorators:
-if isinstance(d, OETestID):
-oeid = d.oeid
-elif isinstance(d, OETestTag):
-oetag = d.oetag
+if hasattr(case, 'decorators'):
+for d in case.decorators:
+if isinstance(d, OETestID):
+oeid = d.oeid
+elif isinstance(d, OETestTag):
+oetag = d.oetag
 
 logger.info("%s\t%s\t\t%s" % (oeid, oetag, case.id()))
 
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/7] oeqa: Change the order to logDetails and logSummary

[Aníbal Limón]

Is better to log the summary at end to see in an easy way
the actual result of the test run.

[YOCTO #11622]

Signed-off-by: Aníbal Limón 
---
 meta/classes/testimage.bbclass| 2 +-
 meta/classes/testsdk.bbclass  | 4 ++--
 meta/lib/oeqa/core/context.py | 2 +-
 meta/lib/oeqa/selftest/context.py | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 1185593..6c33e16 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -292,8 +292,8 @@ def testimage_main(d):
 # Show results (if we have them)
 if not results:
 bb.fatal('%s - FAILED - tests were interrupted during execution' % pn)
-results.logSummary(pn)
 results.logDetails()
+results.logSummary(pn)
 if not results.wasSuccessful():
 bb.fatal('%s - FAILED - check the task log and the ssh log' % pn)
 
diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index 8a9e680..6b51a33 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -72,8 +72,8 @@ def testsdk_main(d):
 component = "%s %s" % (pn, OESDKTestContextExecutor.name)
 context_msg = "%s:%s" % (os.path.basename(tcname), 
os.path.basename(sdk_env))
 
-result.logSummary(component, context_msg)
 result.logDetails()
+result.logSummary(component, context_msg)
 
 if not result.wasSuccessful():
 fail = True
@@ -176,8 +176,8 @@ def testsdkext_main(d):
 component = "%s %s" % (pn, OESDKExtTestContextExecutor.name)
 context_msg = "%s:%s" % (os.path.basename(tcname), 
os.path.basename(sdk_env))
 
-result.logSummary(component, context_msg)
 result.logDetails()
+result.logSummary(component, context_msg)
 
 if not result.wasSuccessful():
 fail = True
diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
index 0dbf5c3..2d543ff 100644
--- a/meta/lib/oeqa/core/context.py
+++ b/meta/lib/oeqa/core/context.py
@@ -158,8 +158,8 @@ class OETestContextExecutor(object):
 else:
 self._pre_run()
 rc = self.tc.runTests(**self.tc_kwargs['run'])
-rc.logSummary(self.name)
 rc.logDetails()
+rc.logSummary(self.name)
 
 output_link = os.path.join(os.path.dirname(args.output_log),
 "%s-results.log" % self.name)
diff --git a/meta/lib/oeqa/selftest/context.py 
b/meta/lib/oeqa/selftest/context.py
index ca73070..8c8439b 100644
--- a/meta/lib/oeqa/selftest/context.py
+++ b/meta/lib/oeqa/selftest/context.py
@@ -179,8 +179,8 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
 else:
 self._pre_run()
 rc = self.tc.runTests(**self.tc_kwargs['run'])
-rc.logSummary(self.name)
 rc.logDetails()
+rc.logSummary(self.name)
 
 return rc
 
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/7] OEQA, devtool and YP compatible fixes

[Aníbal Limón]

The following changes since commit 4a7612c7a12b9a381fb8343ba9586272b889fc15:

  buildhistory: skip tests if GitPython module is missing (2017-06-07 16:00:49 
+0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib alimon/simple_patches
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=alimon/simple_patches

Aníbal Limón (7):
  devtool/standard: Fix lock in _prep_extract_operation
  scripts/yocto-compat-layer.py: Return non-zero when layer test fail
  scripts/yocto-compat-layer-wrapper: Use realpath of output_log
  oeqa: Change the order to logDetails and logSummary
  oeqa/core/loader: Allow unittest.TestCase's to be executed
  oeqa/cases/oelib: Change default case class to unittest.case.TestCase
  oeqa/core/loader: Fix filtering on test modules with submodules

 meta/classes/testimage.bbclass|  2 +-
 meta/classes/testsdk.bbclass  |  4 +-
 meta/lib/oeqa/core/context.py |  2 +-
 meta/lib/oeqa/core/loader.py  | 82 +--
 meta/lib/oeqa/core/runner.py  | 25 
 meta/lib/oeqa/selftest/cases/oelib/elf.py |  4 +-
 meta/lib/oeqa/selftest/cases/oelib/license.py |  6 +-
 meta/lib/oeqa/selftest/cases/oelib/path.py|  4 +-
 meta/lib/oeqa/selftest/cases/oelib/types.py   |  6 +-
 meta/lib/oeqa/selftest/cases/oelib/utils.py   |  6 +-
 meta/lib/oeqa/selftest/context.py |  2 +-
 scripts/lib/devtool/standard.py   |  1 +
 scripts/yocto-compat-layer-wrapper| 18 +-
 scripts/yocto-compat-layer.py |  5 +-
 14 files changed, 95 insertions(+), 72 deletions(-)

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/7] scripts/yocto-compat-layer.py: Return non-zero when layer test fail

[Aníbal Limón]

If whatever layer tested fails returns 2 to indicate the
failure.

[YOCTO #11482]

Signed-off-by: Aníbal Limón 
---
 scripts/yocto-compat-layer.py | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/scripts/yocto-compat-layer.py b/scripts/yocto-compat-layer.py
index 0d5700b..30c55a9 100755
--- a/scripts/yocto-compat-layer.py
+++ b/scripts/yocto-compat-layer.py
@@ -178,16 +178,19 @@ def main():
 results_status[layer['name']] = 'PASS' if 
results[layer['name']].wasSuccessful() else 'FAIL'
 layers_tested = layers_tested + 1
 
+ret = 0
 if layers_tested:
 logger.info('')
 logger.info('Summary of results:')
 logger.info('')
 for layer_name in results_status:
 logger.info('%s ... %s' % (layer_name, results_status[layer_name]))
+if not results[layer_name].wasSuccessful():
+ret = 2 # ret = 1 used for initialization errors
 
 cleanup_bblayers(None, None)
 
-return 0
+return ret
 
 if __name__ == '__main__':
 try:
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/7] scripts/yocto-compat-layer-wrapper: Use realpath of output_log

[Aníbal Limón]

We are using a temp directory, use the realpath for output log
to store the results in the original BUILDDIR.

[YOCTO #11571]

Signed-off-by: Aníbal Limón 
---
 scripts/yocto-compat-layer-wrapper | 18 +-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/scripts/yocto-compat-layer-wrapper 
b/scripts/yocto-compat-layer-wrapper
index db4b687..b6baeb4 100755
--- a/scripts/yocto-compat-layer-wrapper
+++ b/scripts/yocto-compat-layer-wrapper
@@ -13,13 +13,29 @@ if [ -z "$BUILDDIR" ]; then
exit 2
 fi
 
+# since we are using a temp directory, use the realpath for output
+# log option
+output_log=''
+while getopts o: name
+do
+   case $name in
+   o) output_log=$(realpath "$OPTARG")
+   esac
+done
+shift $(($OPTIND - 1))
+
+# generate a temp directory to run compat layer script
 base_dir=$(realpath $BUILDDIR/../)
 cd $base_dir
 
 build_dir=$(mktemp -p $base_dir -d -t build-)
 
 source oe-init-build-env $build_dir
-yocto-compat-layer.py "$@"
+if [[ $output_log != '' ]]; then
+   yocto-compat-layer.py -o "$output_log" "$*"
+else
+   yocto-compat-layer.py "$@"
+fi
 retcode=$?
 
 rm -rf $build_dir
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/7] devtool/standard: Fix lock in _prep_extract_operation

[Aníbal Limón]

If for any reason the parse_recipe fail in extract command
the process gets locked because Cooker is expecting the
finish event by tinfoil.

For example:

$ devtool extract remake /tmp/remake

ERROR: remake is unavailable:
  remake was skipped: PREFERRED_PROVIDER_virtual/make set to make, not remake

Signed-off-by: Aníbal Limón 
---
 scripts/lib/devtool/standard.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 2ecef99..7e342e7 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -436,6 +436,7 @@ def _prep_extract_operation(config, basepath, recipename, 
tinfoil=None):
 
 rd = parse_recipe(config, tinfoil, recipename, True)
 if not rd:
+tinfoil.shutdown()
 return None
 
 if bb.data.inherits_class('kernel-yocto', rd):
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 21/25] wic: add help and usage content for 'wic rm'

[Ed Bartosh]

Added wic_rm_help and wic_rm_usage variables to help.py.
These variables contain help content that will be used in
'wic rm help' and 'wic rm --help' output.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/help.py | 59 -
 1 file changed, 58 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index e93ac4b..23d943c 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -395,7 +395,7 @@ DESCRIPTION
 The second form of the command copies file or directory to the specified 
directory
 on the vfat partition:
$ wic cp test 
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/efi/
-   $ wic ls 
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/eti/
+   $ wic ls 
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/efi/
Volume in drive : is boot
 Volume Serial Number is DB4C-FD4C
Directory for ::/efi
@@ -411,6 +411,63 @@ DESCRIPTION
 containing the tools(parted and mtools) to use.
 """
 
+wic_rm_usage = """
+
+ Remove files or directories from the vfat partitions
+
+ usage: wic rm : [--native-sysroot ]
+
+ This command  removes files or directories from the vfat partitions of 
partitioned
+ image.
+
+ See 'wic help rm' for more detailed instructions.
+
+"""
+
+wic_rm_help = """
+
+NAME
+wic rm - remove files or directories from the vfat partitions
+
+SYNOPSIS
+wic rm  :
+wic rm  : --native-sysroot 
+
+DESCRIPTION
+This command removes files or directories from the vfat partition of the
+wic image:
+
+$ wic ls 
./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1
+Volume in drive : is boot
+ Volume Serial Number is 11D0-DE21
+Directory for ::/
+
+libcom32 c32186500 2017-06-02  15:15
+libutil  c32 24148 2017-06-02  15:15
+syslinux cfg   209 2017-06-02  15:15
+vesamenu c32 27104 2017-06-02  15:15
+vmlinuz6926384 2017-06-02  15:15
+5 files   7 164 345 bytes
+ 16 582 656 bytes free
+
+$ wic rm 
./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/libutil.c32
+
+$ wic ls 
./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1
+Volume in drive : is boot
+ Volume Serial Number is 11D0-DE21
+Directory for ::/
+
+libcom32 c32186500 2017-06-02  15:15
+syslinux cfg   209 2017-06-02  15:15
+vesamenu c32 27104 2017-06-02  15:15
+vmlinuz6926384 2017-06-02  15:15
+4 files   7 140 197 bytes
+ 16 607 232 bytes free
+
+The -n option is used to specify the path to the native sysroot
+containing the tools(parted and mtools) to use.
+"""
+
 wic_plugins_help = """
 
 NAME
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 14/25] filemap: change signature of sparse_copy function

[Ed Bartosh]

Renamed parameter offset->skip to match names of dd
parameters.

Changed affected sparse_copy calls.

Added explanation of the parameters to docstring.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/filemap.py   | 15 ---
 scripts/lib/wic/plugins/imager/direct.py |  2 +-
 2 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/scripts/lib/wic/filemap.py b/scripts/lib/wic/filemap.py
index 8fe302a..8719f44 100644
--- a/scripts/lib/wic/filemap.py
+++ b/scripts/lib/wic/filemap.py
@@ -530,9 +530,18 @@ def filemap(image, log=None):
 except ErrorNotSupp:
 return FilemapSeek(image, log)
 
-def sparse_copy(src_fname, dst_fname, offset=0, skip=0,
+def sparse_copy(src_fname, dst_fname, skip=0, seek=0,
 length=0, api=None):
-"""Efficiently copy sparse file to or into another file."""
+"""
+Efficiently copy sparse file to or into another file.
+
+src_fname: path to source file
+dst_fname: path to destination file
+skip: skip N bytes at thestart of src
+seek: seek N bytes from the start of dst
+length: read N bytes from src and write them to dst
+api: FilemapFiemap or FilemapSeek object
+"""
 if not api:
 api = filemap
 fmap = api(src_fname)
@@ -554,7 +563,7 @@ def sparse_copy(src_fname, dst_fname, offset=0, skip=0,
 start = skip
 
 fmap._f_image.seek(start, os.SEEK_SET)
-dst_file.seek(offset + start - skip, os.SEEK_SET)
+dst_file.seek(seek + start - skip, os.SEEK_SET)
 
 chunk_size = 1024 * 1024
 to_read = end - start
diff --git a/scripts/lib/wic/plugins/imager/direct.py 
b/scripts/lib/wic/plugins/imager/direct.py
index f2e6127..3cdedd1 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -550,7 +550,7 @@ class PartitionedImage():
 source = part.source_file
 if source:
 # install source_file contents into a partition
-sparse_copy(source, self.path, part.start * self.sector_size)
+sparse_copy(source, self.path, seek=part.start * 
self.sector_size)
 
 logger.debug("Installed %s in partition %d, sectors %d-%d, "
  "size %d sectors", source, part.num, part.start,
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 13/25] wic: add mcopy property

[Ed Bartosh]

Added property that points to the mcopy executable.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/engine.py | 5 +
 1 file changed, 5 insertions(+)

diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index e3701c4..f24fb9a 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -237,6 +237,7 @@ class Disk:
 self.native_sysroot = native_sysroot
 self._partitions = None
 self._mdir = None
+self._mcopy = None
 self._partimages = {}
 
 # find parted
@@ -280,6 +281,10 @@ class Disk:
 def mdir(self):
 return self._prop('mdir')
 
+@property
+def mcopy(self):
+return self._prop("mcopy")
+
 def _get_part_image(self, pnum):
 if pnum not in self.partitions:
 raise WicError("Partition %s is not in the image")
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 23/25] wic: implement removing files

[Ed Bartosh]

Added implementation of Disk.del method and wic_r
function that removes files from the vfat partition
using mdel utility.

[YOCTO #11283]

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/engine.py | 16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index 9a8055c..6fc8bb7 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -238,6 +238,7 @@ class Disk:
 self._partitions = None
 self._mdir = None
 self._mcopy = None
+self._mdel = None
 self._partimages = {}
 
 # find parted
@@ -285,6 +286,10 @@ class Disk:
 def mcopy(self):
 return self._prop("mcopy")
 
+@property
+def mdel(self):
+return self._prop("mdel")
+
 def _get_part_image(self, pnum):
 if pnum not in self.partitions:
 raise WicError("Partition %s is not in the image")
@@ -318,6 +323,14 @@ class Disk:
 exec_cmd(cmd)
 self._put_part_image(pnum)
 
+def remove(self, pnum, path):
+"""Remove files/dirs from the partition."""
+cmd = "{} -i {} ::{}".format(self.mdel,
+ self._get_part_image(pnum),
+ path)
+exec_cmd(cmd)
+self._put_part_image(pnum)
+
 def wic_ls(args, native_sysroot):
 """List contents of partitioned image or vfat partition."""
 disk = Disk(args.path.image, native_sysroot)
@@ -345,7 +358,8 @@ def wic_rm(args, native_sysroot):
 Remove files or directories from the vfat partition of
 partitioned image.
 """
-pass
+disk = Disk(args.path.image, native_sysroot)
+disk.remove(args.path.part, args.path.path)
 
 def find_canned(scripts_path, file_name):
 """
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 25/25] selftest: add test_wic_rm test case

[Ed Bartosh]

Added test case for "wic rm" functionality.
- remove file from vfat partition
- remove directory from vfat partition

Signed-off-by: Ed Bartosh 
---
 meta/lib/oeqa/selftest/cases/wic.py | 30 ++
 1 file changed, 30 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/wic.py 
b/meta/lib/oeqa/selftest/cases/wic.py
index 5034587..76cf8d0 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -855,3 +855,33 @@ part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 
--exclude-path bin/ --r
 self.assertEqual(0, result.status)
 self.assertEqual(8, len(result.output.split('\n')))
 self.assertTrue(os.path.basename(testdir) in result.output)
+
+def test_wic_rm(self):
+"""Test removing files and directories from the the wic image."""
+self.assertEqual(0, runCmd("wic create mkefidisk "
+   "--image-name=core-image-minimal "
+   "-D -o %s" % self.resultdir).status)
+images = glob(self.resultdir + "mkefidisk-*.direct")
+self.assertEqual(1, len(images))
+
+sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools')
+
+# list directory content of the first partition
+result = runCmd("wic ls %s:1 -n %s" % (images[0], sysroot))
+self.assertEqual(0, result.status)
+self.assertTrue('\nbzimage' in result.output)
+self.assertTrue('\nefi   ' in result.output)
+
+# remove file
+result = runCmd("wic rm %s:1/bzimage -n %s" % (images[0], sysroot))
+self.assertEqual(0, result.status)
+
+# remove directory
+result = runCmd("wic rm %s:1/efi -n %s" % (images[0], sysroot))
+self.assertEqual(0, result.status)
+
+# check if they're removed
+result = runCmd("wic ls %s:1 -n %s" % (images[0], sysroot))
+self.assertEqual(0, result.status)
+self.assertFalse('\nbzimage' in result.output)
+self.assertFalse('\nefi   ' in result.output)
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 20/25] wic: add wic_init_parser_rm

[Ed Bartosh]

Add parser for 'wic rm' subcommand.

Signed-off-by: Ed Bartosh 
---
 scripts/wic | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/scripts/wic b/scripts/wic
index 5e81fad..fc192ec 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -379,6 +379,12 @@ def wic_init_parser_cp(subparser):
 subparser.add_argument("-n", "--native-sysroot",
 help="path to the native sysroot containing the tools")
 
+def wic_init_parser_rm(subparser):
+subparser.add_argument("path", type=imgpathtype,
+help="path: :")
+subparser.add_argument("-n", "--native-sysroot",
+help="path to the native sysroot containing the tools")
+
 def wic_init_parser_help(subparser):
 helpparsers = subparser.add_subparsers(dest='help_topic', 
help=hlp.wic_usage)
 for helptopic in helptopics:
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 18/25] wic: fully implement 'wic cp'

[Ed Bartosh]

Added implementation of Disk.copy method and wic_cp
function that copies files/directories to the vfat partition
of the partitioned image.

[YOCTO #11283]

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/engine.py | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index 6a4f55f..a48c4ad 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -310,6 +310,14 @@ class Disk:
self._get_part_image(pnum),
path))
 
+def copy(self, src, pnum, path):
+"""Copy partition image into wic image."""
+cmd = "{} -i {} -snop {} ::{}".format(self.mcopy,
+  self._get_part_image(pnum),
+  src, path)
+exec_cmd(cmd)
+self._put_part_image(pnum)
+
 def wic_ls(args, native_sysroot):
 """List contents of partitioned image or vfat partition."""
 disk = Disk(args.path.image, native_sysroot)
@@ -329,7 +337,8 @@ def wic_cp(args, native_sysroot):
 Copy local file or directory to the vfat partition of
 partitioned image.
 """
-pass
+disk = Disk(args.dest.image, native_sysroot)
+disk.copy(args.src, args.dest.part, args.dest.path)
 
 def find_canned(scripts_path, file_name):
 """
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 19/25] selftest: add test_wic_cp test case

[Ed Bartosh]

Added test case for "wic cp" functionality.
 - copy file to vfat partition
 - copy directory to vfat partition

Signed-off-by: Ed Bartosh 
---
 meta/lib/oeqa/selftest/cases/wic.py | 46 -
 1 file changed, 45 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/cases/wic.py 
b/meta/lib/oeqa/selftest/cases/wic.py
index 5d67395..5034587 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -28,7 +28,7 @@ import sys
 import unittest
 
 from glob import glob
-from shutil import rmtree
+from shutil import rmtree, copy
 from functools import wraps, lru_cache
 from tempfile import NamedTemporaryFile
 
@@ -811,3 +811,47 @@ part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 
--exclude-path bin/ --r
 result = runCmd("wic ls %s:1/ -n %s" % (images[0], sysroot))
 self.assertEqual(0, result.status)
 self.assertEqual(6, len(result.output.split('\n')))
+
+def test_wic_cp(self):
+"""Test copy files and directories to the the wic image."""
+self.assertEqual(0, runCmd("wic create wictestdisk "
+   "--image-name=core-image-minimal "
+   "-D -o %s" % self.resultdir).status)
+images = glob(self.resultdir + "wictestdisk-*.direct")
+self.assertEqual(1, len(images))
+
+sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools')
+
+# list directory content of the first partition
+result = runCmd("wic ls %s:1/ -n %s" % (images[0], sysroot))
+self.assertEqual(0, result.status)
+self.assertEqual(6, len(result.output.split('\n')))
+
+with NamedTemporaryFile("w", suffix=".wic-cp") as testfile:
+testfile.write("test")
+
+# copy file to the partition
+result = runCmd("wic cp %s %s:1/ -n %s" % (testfile.name, 
images[0], sysroot))
+self.assertEqual(0, result.status)
+
+# check if file is there
+result = runCmd("wic ls %s:1/ -n %s" % (images[0], sysroot))
+self.assertEqual(0, result.status)
+self.assertEqual(7, len(result.output.split('\n')))
+self.assertTrue(os.path.basename(testfile.name) in result.output)
+
+# prepare directory
+testdir = os.path.join(self.resultdir, 'wic-test-cp-dir')
+testsubdir = os.path.join(testdir, 'subdir')
+os.makedirs(os.path.join(testsubdir))
+copy(testfile.name, testdir)
+
+# copy directory to the partition
+result = runCmd("wic cp %s %s:1/ -n %s" % (testdir, images[0], 
sysroot))
+self.assertEqual(0, result.status)
+
+# check if directory is there
+result = runCmd("wic ls %s:1/ -n %s" % (images[0], sysroot))
+self.assertEqual(0, result.status)
+self.assertEqual(8, len(result.output.split('\n')))
+self.assertTrue(os.path.basename(testdir) in result.output)
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 22/25] wic: add 'wic rm' command

[Ed Bartosh]

Added empty 'wic rm' command that does nothing.
The functionality will be added by the next commits.

[YOCTO #11283]

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/engine.py |  7 +++
 scripts/wic   | 14 ++
 2 files changed, 21 insertions(+)

diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index a48c4ad..9a8055c 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -340,6 +340,13 @@ def wic_cp(args, native_sysroot):
 disk = Disk(args.dest.image, native_sysroot)
 disk.copy(args.src, args.dest.part, args.dest.path)
 
+def wic_rm(args, native_sysroot):
+"""
+Remove files or directories from the vfat partition of
+partitioned image.
+"""
+pass
+
 def find_canned(scripts_path, file_name):
 """
 Find a file either by its path or by name in the canned files dir.
diff --git a/scripts/wic b/scripts/wic
index fc192ec..da14f47 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -248,6 +248,13 @@ def wic_cp_subcommand(args, usage_str):
 """
 engine.wic_cp(args, args.native_sysroot)
 
+def wic_rm_subcommand(args, usage_str):
+"""
+Command-line handling for removing files/dirs from images.
+The real work is done by engine.wic_rm()
+"""
+engine.wic_rm(args, args.native_sysroot)
+
 def wic_help_subcommand(args, usage_str):
 """
 Command-line handling for help subcommand to keep the current
@@ -286,6 +293,9 @@ helptopics = {
 "cp":[wic_help_topic_subcommand,
   wic_help_topic_usage,
   hlp.wic_cp_help],
+"rm":[wic_help_topic_subcommand,
+  wic_help_topic_usage,
+  hlp.wic_rm_help],
 "list":  [wic_help_topic_subcommand,
   wic_help_topic_usage,
   hlp.wic_list_help]
@@ -409,6 +419,10 @@ subcommands = {
   hlp.wic_cp_usage,
   hlp.wic_cp_help,
   wic_init_parser_cp],
+"rm":[wic_rm_subcommand,
+  hlp.wic_rm_usage,
+  hlp.wic_rm_help,
+  wic_init_parser_rm],
 "help":  [wic_help_subcommand,
   wic_help_topic_usage,
   hlp.wic_help_help,
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 12/25] wic: add Disk._prop helper

[Ed Bartosh]

Added generic helper to use in property methods to
access commands in a lazy manner.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/engine.py | 15 ++-
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index f8f2844..e3701c4 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -267,13 +267,18 @@ class Disk:
 
 return self._partitions
 
+def _prop(self, name):
+"""Get path to the executable in a lazy way."""
+aname = "_%s" % name
+if getattr(self, aname) is None:
+setattr(self, aname, find_executable(name, self.paths))
+if not getattr(self, aname):
+raise WicError("Can't find executable {}".format(name))
+return getattr(self, aname)
+
 @property
 def mdir(self):
-if self._mdir is None:
-self._mdir = find_executable("mdir", self.paths)
-if not self._mdir:
-raise WicError("Can't find executable mdir")
-return self._mdir
+return self._prop('mdir')
 
 def _get_part_image(self, pnum):
 if pnum not in self.partitions:
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 17/25] wic: add Disk._put_part_image method

[Ed Bartosh]

This method copies partition image into the wic image.
It will be used in 'wic cp' and 'wic rm' subcommands
to copy changed partition back into wic image.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/engine.py | 5 +
 1 file changed, 5 insertions(+)

diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index f24fb9a..6a4f55f 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -300,6 +300,11 @@ class Disk:
 
 return self._partimages[pnum]
 
+def _put_part_image(self, pnum):
+"""Put partition image into partitioned image."""
+sparse_copy(self._partimages[pnum], self.imagepath,
+seek=self.partitions[pnum].start)
+
 def dir(self, pnum, path):
 return exec_cmd("{} -i {} ::{}".format(self.mdir,
self._get_part_image(pnum),
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 11/25] wic: add 'wic cp' command

[Ed Bartosh]

Added empty 'wic cp' command that does nothing.
The functionality will be added by the next commits.

[YOCTO #11283]

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/engine.py |  7 +++
 scripts/wic   | 13 +
 2 files changed, 20 insertions(+)

diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index 95c8d1c..f8f2844 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -309,6 +309,13 @@ def wic_ls(args, native_sysroot):
 path = args.path.path or '/'
 print(disk.dir(args.path.part, path))
 
+def wic_cp(args, native_sysroot):
+"""
+Copy local file or directory to the vfat partition of
+partitioned image.
+"""
+pass
+
 def find_canned(scripts_path, file_name):
 """
 Find a file either by its path or by name in the canned files dir.
diff --git a/scripts/wic b/scripts/wic
index 945dda8..5e81fad 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -241,6 +241,12 @@ def wic_ls_subcommand(args, usage_str):
 """
 engine.wic_ls(args, args.native_sysroot)
 
+def wic_cp_subcommand(args, usage_str):
+"""
+Command-line handling for copying files/dirs to images.
+The real work is done by engine.wic_cp()
+"""
+engine.wic_cp(args, args.native_sysroot)
 
 def wic_help_subcommand(args, usage_str):
 """
@@ -277,6 +283,9 @@ helptopics = {
 "ls":[wic_help_topic_subcommand,
   wic_help_topic_usage,
   hlp.wic_ls_help],
+"cp":[wic_help_topic_subcommand,
+  wic_help_topic_usage,
+  hlp.wic_cp_help],
 "list":  [wic_help_topic_subcommand,
   wic_help_topic_usage,
   hlp.wic_list_help]
@@ -390,6 +399,10 @@ subcommands = {
   hlp.wic_ls_usage,
   hlp.wic_ls_help,
   wic_init_parser_ls],
+"cp":[wic_cp_subcommand,
+  hlp.wic_cp_usage,
+  hlp.wic_cp_help,
+  wic_init_parser_cp],
 "help":  [wic_help_subcommand,
   wic_help_topic_usage,
   hlp.wic_help_help,
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 15/25] filemap: check if dest is written for every block

[Ed Bartosh]

If lenght parameter is provided to sparse_copy call
it's mandatory to check if the output file is fully
written after reading unmapped block from input file.

If it's not done then sparse_copy can write more data
than specified length.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/filemap.py | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/scripts/lib/wic/filemap.py b/scripts/lib/wic/filemap.py
index 8719f44..764dbbe 100644
--- a/scripts/lib/wic/filemap.py
+++ b/scripts/lib/wic/filemap.py
@@ -563,6 +563,13 @@ def sparse_copy(src_fname, dst_fname, skip=0, seek=0,
 start = skip
 
 fmap._f_image.seek(start, os.SEEK_SET)
+
+written += start - skip - written
+if length and written >= length:
+dst_file.seek(seek + length, os.SEEK_SET)
+dst_file.close()
+return
+
 dst_file.seek(seek + start - skip, os.SEEK_SET)
 
 chunk_size = 1024 * 1024
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 16/25] filemap: calculate dst size correctly

[Ed Bartosh]

Fixed calculation of the dst file size using skip, seek and
length parameters. Current code does it incorrectly which
causes sparse_copy API to create unnecessary big output files.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/filemap.py | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/wic/filemap.py b/scripts/lib/wic/filemap.py
index 764dbbe..6d11355 100644
--- a/scripts/lib/wic/filemap.py
+++ b/scripts/lib/wic/filemap.py
@@ -549,7 +549,11 @@ def sparse_copy(src_fname, dst_fname, skip=0, seek=0,
 dst_file = open(dst_fname, 'r+b')
 except IOError:
 dst_file = open(dst_fname, 'wb')
-dst_file.truncate(os.path.getsize(src_fname))
+if length:
+dst_size = length + seek
+else:
+dst_size = os.path.getsize(src_fname) + seek - skip
+dst_file.truncate(dst_size)
 
 written = 0
 for first, last in fmap.get_mapped_ranges(0, fmap.blocks_cnt):
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 24/25] wic: implement removing directories

[Ed Bartosh]

Added support for removing directories using mdeltree
utility to Disk.del method

[YOCTO #11283]

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/engine.py | 22 ++
 1 file changed, 18 insertions(+), 4 deletions(-)

diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index 6fc8bb7..2c899dd 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -239,6 +239,7 @@ class Disk:
 self._mdir = None
 self._mcopy = None
 self._mdel = None
+self._mdeltree = None
 self._partimages = {}
 
 # find parted
@@ -290,6 +291,10 @@ class Disk:
 def mdel(self):
 return self._prop("mdel")
 
+@property
+def mdeltree(self):
+return self._prop("mdeltree")
+
 def _get_part_image(self, pnum):
 if pnum not in self.partitions:
 raise WicError("Partition %s is not in the image")
@@ -325,10 +330,19 @@ class Disk:
 
 def remove(self, pnum, path):
 """Remove files/dirs from the partition."""
-cmd = "{} -i {} ::{}".format(self.mdel,
- self._get_part_image(pnum),
- path)
-exec_cmd(cmd)
+partimg = self._get_part_image(pnum)
+cmd = "{} -i {} ::{}".format(self.mdel, partimg, path)
+try:
+exec_cmd(cmd)
+except WicError as err:
+if "not found" in str(err) or "non empty" in str(err):
+# mdel outputs 'File ... not found' or 'directory .. non empty"
+# try to use mdeltree as path could be a directory
+cmd = "{} -i {} ::{}".format(self.mdeltree,
+ partimg, path)
+exec_cmd(cmd)
+else:
+raise err
 self._put_part_image(pnum)
 
 def wic_ls(args, native_sysroot):
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 09/25] wic: add wic_init_parser_cp

[Ed Bartosh]

Add parser for 'wic cp' subcommand and a custom argument type.

Signed-off-by: Ed Bartosh 
---
 scripts/wic | 16 
 1 file changed, 16 insertions(+)

diff --git a/scripts/wic b/scripts/wic
index 4161f80..945dda8 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -341,6 +341,8 @@ def imgtype(arg):
 image, part = image.split(':')
 if '/' in part:
 part, path = part.split('/', 1)
+if not path:
+path = '/'
 
 if not os.path.isfile(image):
 err = "%s is not a regular file or symlink" % image
@@ -354,6 +356,20 @@ def wic_init_parser_ls(subparser):
 subparser.add_argument("-n", "--native-sysroot",
 help="path to the native sysroot containing the tools")
 
+def imgpathtype(arg):
+img = imgtype(arg)
+if img.part is None:
+raise argparse.ArgumentTypeError("partition number is not specified")
+return img
+
+def wic_init_parser_cp(subparser):
+subparser.add_argument("src",
+help="source spec")
+subparser.add_argument("dest", type=imgpathtype,
+help="image spec: :[]")
+subparser.add_argument("-n", "--native-sysroot",
+help="path to the native sysroot containing the tools")
+
 def wic_init_parser_help(subparser):
 helpparsers = subparser.add_subparsers(dest='help_topic', 
help=hlp.wic_usage)
 for helptopic in helptopics:
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 06/25] wic: add 'wic ls' command

[Ed Bartosh]

Added empty 'wic ls' command that does nothing.
The functionality will be added by the next commits.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/engine.py |  4 
 scripts/wic   | 15 +++
 2 files changed, 19 insertions(+)

diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index 6473582..e58beb7 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -225,6 +225,10 @@ def wic_list(args, scripts_path):
 
 return False
 
+def wic_ls(args, native_sysroot):
+"""List contents of partitioned image or vfat partition."""
+pass
+
 def find_canned(scripts_path, file_name):
 """
 Find a file either by its path or by name in the canned files dir.
diff --git a/scripts/wic b/scripts/wic
index 6c9a30d..4161f80 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -234,6 +234,14 @@ def wic_list_subcommand(args, usage_str):
 raise WicError("Bad list arguments, exiting")
 
 
+def wic_ls_subcommand(args, usage_str):
+"""
+Command-line handling for list content of images.
+The real work is done by engine.wic_ls()
+"""
+engine.wic_ls(args, args.native_sysroot)
+
+
 def wic_help_subcommand(args, usage_str):
 """
 Command-line handling for help subcommand to keep the current
@@ -266,6 +274,9 @@ helptopics = {
 "create":[wic_help_topic_subcommand,
   wic_help_topic_usage,
   hlp.wic_create_help],
+"ls":[wic_help_topic_subcommand,
+  wic_help_topic_usage,
+  hlp.wic_ls_help],
 "list":  [wic_help_topic_subcommand,
   wic_help_topic_usage,
   hlp.wic_list_help]
@@ -359,6 +370,10 @@ subcommands = {
   hlp.wic_list_usage,
   hlp.wic_list_help,
   wic_init_parser_list],
+"ls":[wic_ls_subcommand,
+  hlp.wic_ls_usage,
+  hlp.wic_ls_help,
+  wic_init_parser_ls],
 "help":  [wic_help_subcommand,
   wic_help_topic_usage,
   hlp.wic_help_help,
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 10/25] wic: add help and usage content for 'wic cp'

[Ed Bartosh]

Added wic_cp_help and wic_cp_usage variables to
help.py. These variables contain help content that
will be used in 'wic cp help' and 'wic cp --help'
output.

[YOCTO #11283]

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/help.py | 62 +
 1 file changed, 62 insertions(+)

diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index bb3c749..e93ac4b 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -349,6 +349,68 @@ DESCRIPTION
 
 """
 
+wic_cp_usage = """
+
+ Copy files and directories to the vfat partitions
+
+ usage: wic cp  :[] [--native-sysroot ]
+
+ This command  copies local files or directories to the vfat partitions of 
partitioned
+ image.
+
+ See 'wic help cp' for more detailed instructions.
+
+"""
+
+wic_cp_help = """
+
+NAME
+wic cp - copy files and directories to the vfat partitions
+
+SYNOPSIS
+wic cp  :
+wic cp  :
+wic cp  : --native-sysroot 
+
+DESCRIPTION
+This command copies files and directories to the vfat partition of the
+wic image.
+
+The first form of it copies file or directory to the root directory of
+the vfat partition:
+$ wic cp test.wks 
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1
+$ wic ls 
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1
+Volume in drive : is boot
+ Volume Serial Number is DB4C-FD4C
+Directory for ::/
+
+efi   2017-05-24  18:15
+loader2017-05-24  18:15
+startup  nsh26 2017-05-24  18:15
+vmlinuz6926384 2017-05-24  18:15
+test wks   628 2017-05-24  21:22
+5 files   6 927 038 bytes
+ 15 677 440 bytes free
+
+The second form of the command copies file or directory to the specified 
directory
+on the vfat partition:
+   $ wic cp test 
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/efi/
+   $ wic ls 
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/eti/
+   Volume in drive : is boot
+Volume Serial Number is DB4C-FD4C
+   Directory for ::/efi
+
+   . 2017-05-24  18:15
+   ..2017-05-24  18:15
+   boot  2017-05-24  18:15
+   test  2017-05-24  21:27
+   4 files   0 bytes
+15 675 392 bytes free
+
+The -n option is used to specify the path to the native sysroot
+containing the tools(parted and mtools) to use.
+"""
+
 wic_plugins_help = """
 
 NAME
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 08/25] selftest: add new test case test_wic_ls

[Ed Bartosh]

Tested 'wic ls' functionality:
 - list of image partitions
 - list of directory content of vfat partition

Signed-off-by: Ed Bartosh 
---
 meta/lib/oeqa/selftest/cases/wic.py | 20 
 1 file changed, 20 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/wic.py 
b/meta/lib/oeqa/selftest/cases/wic.py
index 4040cf7..5d67395 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -791,3 +791,23 @@ part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 
--exclude-path bin/ --r
 # 8 blocks is 4K (physical sector size)
 self.assertEqual(dest_stat.st_blocks, 8)
 os.unlink(dest)
+
+def test_wic_ls(self):
+"""Test listing image content using 'wic ls'"""
+self.assertEqual(0, runCmd("wic create wictestdisk "
+   "--image-name=core-image-minimal "
+   "-D -o %s" % self.resultdir).status)
+images = glob(self.resultdir + "wictestdisk-*.direct")
+self.assertEqual(1, len(images))
+
+sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools')
+
+# list partitions
+result = runCmd("wic ls %s -n %s" % (images[0], sysroot))
+self.assertEqual(0, result.status)
+self.assertEqual(3, len(result.output.split('\n')))
+
+# list directory content of the first partition
+result = runCmd("wic ls %s:1/ -n %s" % (images[0], sysroot))
+self.assertEqual(0, result.status)
+self.assertEqual(6, len(result.output.split('\n')))
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 07/25] engine: implement listing wic images

[Ed Bartosh]

Implemented 'wic ls' functionality:
 - list image partitions
 - list directory content of vfat partitions

[YOCTO #11283]

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/engine.py | 84 +--
 1 file changed, 82 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index e58beb7..95c8d1c 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -30,10 +30,15 @@
 
 import logging
 import os
+import tempfile
+
+from collections import namedtuple, OrderedDict
+from distutils.spawn import find_executable
 
 from wic import WicError
+from wic.filemap import sparse_copy
 from wic.pluginbase import PluginMgr
-from wic.utils.misc import get_bitbake_var
+from wic.utils.misc import get_bitbake_var, exec_cmd
 
 logger = logging.getLogger('wic')
 
@@ -225,9 +230,84 @@ def wic_list(args, scripts_path):
 
 return False
 
+
+class Disk:
+def __init__(self, imagepath, native_sysroot):
+self.imagepath = imagepath
+self.native_sysroot = native_sysroot
+self._partitions = None
+self._mdir = None
+self._partimages = {}
+
+# find parted
+self.paths = "/bin:/usr/bin:/usr/sbin:/sbin/"
+if native_sysroot:
+for path in self.paths.split(':'):
+self.paths = "%s%s:%s" % (native_sysroot, path, self.paths)
+
+self.parted = find_executable("parted", self.paths)
+if not self.parted:
+raise WicError("Can't find executable parted")
+
+def __del__(self):
+for path in self._partimages.values():
+os.unlink(path)
+
+@property
+def partitions(self):
+if self._partitions is None:
+self._partitions = OrderedDict()
+out = exec_cmd("%s -sm %s unit B print" % (self.parted, 
self.imagepath))
+parttype = namedtuple("Part", "pnum start end size fstype")
+for line in out.splitlines()[2:]:
+pnum, start, end, size, fstype = line.split(':')[:5]
+partition = parttype(pnum, int(start[:-1]), int(end[:-1]),
+ int(size[:-1]), fstype)
+self._partitions[pnum] = partition
+
+return self._partitions
+
+@property
+def mdir(self):
+if self._mdir is None:
+self._mdir = find_executable("mdir", self.paths)
+if not self._mdir:
+raise WicError("Can't find executable mdir")
+return self._mdir
+
+def _get_part_image(self, pnum):
+if pnum not in self.partitions:
+raise WicError("Partition %s is not in the image")
+part = self.partitions[pnum]
+if not part.fstype.startswith("fat"):
+raise WicError("Not supported fstype: {}".format(part.fstype))
+if pnum not in self._partimages:
+tmpf = tempfile.NamedTemporaryFile(prefix="wic-part")
+dst_fname = tmpf.name
+tmpf.close()
+sparse_copy(self.imagepath, dst_fname, skip=part.start, 
length=part.size)
+self._partimages[pnum] = dst_fname
+
+return self._partimages[pnum]
+
+def dir(self, pnum, path):
+return exec_cmd("{} -i {} ::{}".format(self.mdir,
+   self._get_part_image(pnum),
+   path))
+
 def wic_ls(args, native_sysroot):
 """List contents of partitioned image or vfat partition."""
-pass
+disk = Disk(args.path.image, native_sysroot)
+if not args.path.part:
+if disk.partitions:
+print('Num StartEnd  Size  Fstype')
+for part in disk.partitions.values():
+print("{:2s}  {:12d} {:12d} {:12d}  {}".format(\
+  part.pnum, part.start, part.end,
+  part.size, part.fstype))
+else:
+path = args.path.path or '/'
+print(disk.dir(args.path.part, path))
 
 def find_canned(scripts_path, file_name):
 """
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 03/25] bootimg-pcbios: make boot image file unique

[Ed Bartosh]

Plugin code uses boot.img file name for an image file. If there are
two partitions that use bootimg-pcbios wic breaks with an error
"file already exists: boot.img"

Made image file name unique by adding wks like number to it to fix
the issue.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/plugins/source/bootimg-pcbios.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py 
b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index 5890c12..98ad88b 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -186,7 +186,7 @@ class BootimgPcbiosPlugin(SourcePlugin):
  extra_blocks, part.mountpoint, blocks)
 
 # dosfs image, created by mkdosfs
-bootimg = "%s/boot.img" % cr_workdir
+bootimg = "%s/boot%s.img" % (cr_workdir, part.lineno)
 
 dosfs_cmd = "mkdosfs -n boot -S 512 -C %s %d" % (bootimg, blocks)
 exec_native_cmd(dosfs_cmd, native_sysroot)
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 01/25] filemap: fix skip logic

[Ed Bartosh]

Fixed bug in processing 'skip' parameter:
   don't read input file if end of bmap block is less than skip

Simplified logic of positioning to the start of data inside a
partially skipped bmap block.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/filemap.py | 11 +++
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/scripts/lib/wic/filemap.py b/scripts/lib/wic/filemap.py
index 1f1aacc..585b7ea 100644
--- a/scripts/lib/wic/filemap.py
+++ b/scripts/lib/wic/filemap.py
@@ -545,11 +545,14 @@ def sparse_copy(src_fname, dst_fname, offset=0, skip=0, 
api=None):
 start = first * fmap.block_size
 end = (last + 1) * fmap.block_size
 
+if skip >= end:
+continue
+
 if start < skip < end:
-fmap._f_image.seek(skip, os.SEEK_SET)
-else:
-fmap._f_image.seek(start, os.SEEK_SET)
-dst_file.seek(offset + start, os.SEEK_SET)
+start = skip
+
+fmap._f_image.seek(start, os.SEEK_SET)
+dst_file.seek(offset + start - skip, os.SEEK_SET)
 
 chunk_size = 1024 * 1024
 to_read = end - start
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 04/25] wic: add wic_init_parser_ls

[Ed Bartosh]

Added parser for 'wic ls' command.

Signed-off-by: Ed Bartosh 
---
 scripts/wic | 25 +
 1 file changed, 25 insertions(+)

diff --git a/scripts/wic b/scripts/wic
index 49cad86..6c9a30d 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -35,6 +35,8 @@ import os
 import sys
 import argparse
 import logging
+
+from collections import namedtuple
 from distutils import spawn
 
 # External modules
@@ -317,6 +319,29 @@ def wic_init_parser_list(subparser):
  "defined inside the .wks file")
 return
 
+def imgtype(arg):
+"""
+Custom type for ArgumentParser
+Converts path spec to named tuple: (image, partition, path)
+"""
+image = arg
+part = path = None
+if ':' in image:
+image, part = image.split(':')
+if '/' in part:
+part, path = part.split('/', 1)
+
+if not os.path.isfile(image):
+err = "%s is not a regular file or symlink" % image
+raise argparse.ArgumentTypeError(err)
+
+return namedtuple('ImgType', 'image part path')(image, part, path)
+
+def wic_init_parser_ls(subparser):
+subparser.add_argument("path", type=imgtype,
+help="image spec: [:[]]")
+subparser.add_argument("-n", "--native-sysroot",
+help="path to the native sysroot containing the tools")
 
 def wic_init_parser_help(subparser):
 helpparsers = subparser.add_subparsers(dest='help_topic', 
help=hlp.wic_usage)
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 05/25] wic: add help and usage content for 'wic ls'

[Ed Bartosh]

Added wic_ls_help and wic_ls_usage variables to
help.py. These variables contain help content that
will be used in 'wic ls help' and 'wic ls --help'
output.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/help.py | 65 +
 1 file changed, 65 insertions(+)

diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index f9f7268..bb3c749 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -284,6 +284,71 @@ DESCRIPTION
 details.
 """
 
+wic_ls_usage = """
+
+ List content of a partitioned image
+
+ usage: wic ls [:[]] [--native-sysroot ]
+
+ This command  outputs either list of image partitions or directory contents
+ of vfat partitions.
+
+ See 'wic help ls' for more detailed instructions.
+
+"""
+
+wic_ls_help = """
+
+NAME
+wic ls - List contents of partitioned image or vfat partitions
+
+SYNOPSIS
+wic ls 
+wic ls :
+wic ls :
+wic ls : --native-sysroot 
+
+DESCRIPTION
+This command lists either partitions of the image or directory contents
+of vfat partitions.
+
+The first form it lists partitions of the image.
+For example:
+$ wic ls tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic
+Num StartEnd  Size  Fstype
+11048576 24438783 23390208  fat16
+2   25165824 50315263 25149440  ext4
+
+Second and third form list directory content of vfat partition:
+$ wic ls 
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1
+Volume in drive : is boot
+ Volume Serial Number is 2DF2-5F02
+Directory for ::/
+
+efi   2017-05-11  10:54
+startup  nsh26 2017-05-11  10:54
+vmlinuz6922288 2017-05-11  10:54
+3 files   6 922 314 bytes
+ 15 818 752 bytes free
+
+
+$ wic ls 
tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/EFI/boot/
+Volume in drive : is boot
+ Volume Serial Number is 2DF2-5F02
+Directory for ::/EFI/boot
+
+. 2017-05-11  10:54
+..2017-05-11  10:54
+grub cfg   679 2017-05-11  10:54
+bootx64  efi571392 2017-05-11  10:54
+4 files 572 071 bytes
+ 15 818 752 bytes free
+
+The -n option is used to specify the path to the native sysroot
+containing the tools(parted and mtools) to use.
+
+"""
+
 wic_plugins_help = """
 
 NAME
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 02/25] filemap: add parameter 'length' to sparse_copy

[Ed Bartosh]

Added parameter 'length' to specify amount of data
to write into destination file. This is useful when only
part of source file should be written into destination file.

Signed-off-by: Ed Bartosh 
---
 scripts/lib/wic/filemap.py | 15 ---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/scripts/lib/wic/filemap.py b/scripts/lib/wic/filemap.py
index 585b7ea..8fe302a 100644
--- a/scripts/lib/wic/filemap.py
+++ b/scripts/lib/wic/filemap.py
@@ -530,7 +530,8 @@ def filemap(image, log=None):
 except ErrorNotSupp:
 return FilemapSeek(image, log)
 
-def sparse_copy(src_fname, dst_fname, offset=0, skip=0, api=None):
+def sparse_copy(src_fname, dst_fname, offset=0, skip=0,
+length=0, api=None):
 """Efficiently copy sparse file to or into another file."""
 if not api:
 api = filemap
@@ -541,6 +542,7 @@ def sparse_copy(src_fname, dst_fname, offset=0, skip=0, 
api=None):
 dst_file = open(dst_fname, 'wb')
 dst_file.truncate(os.path.getsize(src_fname))
 
+written = 0
 for first, last in fmap.get_mapped_ranges(0, fmap.blocks_cnt):
 start = first * fmap.block_size
 end = (last + 1) * fmap.block_size
@@ -561,7 +563,14 @@ def sparse_copy(src_fname, dst_fname, offset=0, skip=0, 
api=None):
 while read < to_read:
 if read + chunk_size > to_read:
 chunk_size = to_read - read
-chunk = fmap._f_image.read(chunk_size)
+size = chunk_size
+if length and written + size > length:
+size = length - written
+chunk = fmap._f_image.read(size)
 dst_file.write(chunk)
-read += chunk_size
+read += size
+written += size
+if written == length:
+dst_file.close()
+return
 dst_file.close()
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 00/25] #11283 wic ls & cp & rm

[Ed Bartosh]

Hi,

This is a consolidated patchet made of previously sent 'wic ls' 'wic cp' and
'wic rm' patchset. It doesn't depend on generic efi implementation, which
made these 3 patchsets pending.

Please, review and merge.

Thanks

The following changes since commit 4a7612c7a12b9a381fb8343ba9586272b889fc15:

  buildhistory: skip tests if GitPython module is missing (2017-06-07 16:00:49 
+0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib ed/wip
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/wip

Ed Bartosh (25):
  filemap: fix skip logic
  filemap: add parameter 'length' to sparse_copy
  bootimg-pcbios: make boot image file unique
  wic: add wic_init_parser_ls
  wic: add help and usage content for 'wic ls'
  wic: add 'wic ls' command
  engine: implement listing wic images
  selftest: add new test case test_wic_ls
  wic: add wic_init_parser_cp
  wic: add help and usage content for 'wic cp'
  wic: add 'wic cp' command
  wic: add Disk._prop helper
  wic: add mcopy property
  filemap: change signature of sparse_copy function
  filemap: check if dest is written for every block
  filemap: calculate dst size correctly
  wic: add Disk._put_part_image method
  wic: fully implement 'wic cp'
  selftest: add test_wic_cp test case
  wic: add wic_init_parser_rm
  wic: add help and usage content for 'wic rm'
  wic: add 'wic rm' command
  wic: implement removing files
  wic: implement removing directories
  selftest: add test_wic_rm test case

 meta/lib/oeqa/selftest/cases/wic.py  |  96 +++-
 scripts/lib/wic/engine.py| 152 ++-
 scripts/lib/wic/filemap.py   |  50 --
 scripts/lib/wic/help.py  | 184 +++
 scripts/lib/wic/plugins/imager/direct.py |   2 +-
 scripts/lib/wic/plugins/source/bootimg-pcbios.py |   2 +-
 scripts/wic  |  89 +++
 7 files changed, 562 insertions(+), 13 deletions(-)

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/2] Yocto Compatible 2.0 support code

[Joshua Watt]

On Thu, 2017-06-08 at 15:33 +0100, Richard Purdie wrote:
> On Thu, 2017-06-08 at 08:55 -0500, Joshua Watt wrote:
> > On Thu, 2017-06-08 at 09:56 +0100, Richard Purdie wrote:
> > > 
> > > On Wed, 2017-06-07 at 10:43 -0500, Joshua Watt wrote:
> > > > 
> > > > On Wed, 2017-06-07 at 17:31 +0200, Patrick Ohly wrote:
> > > > > 
> > > > > 
> > > > > As discussed in the "[Openembedded-architecture] Yocto
> > > > > Compatible
> > > > > 2.0
> > > > > + signature changes" mail thread, changes in a .bbappend
> > > > > cannot
> > > > > be
> > > > > done unconditionally. Making _append and _remove depend on
> > > > > overrides
> > > > > which get set based on DISTRO_FEATURES is one way of
> > > > > achieving
> > > > > this.
> > > > > 
> > > > > The oe.utils.optional_includes() helper function has not been
> > > > > discussed before. It's an attempt to address concerns by
> > > > > developers
> > > > > that having to write code for (potentially complex) condition
> > > > > checking
> > > > > is error prone and hard to read.
> > > > 
> > > > I promise I'm not trying to start a flame war here, and perhaps
> > > > there
> > > > is history behind this that I'm not aware of but...
> > > > 
> > > > Why doesn't bitbake support some sort of "if" statement? It
> > > > seems
> > > > like most of what we are trying to do could be accomplished
> > > > with
> > > > much
> > > > less fuss if one could simply do this in the bb file:
> > > > 
> > > >  if bb.utils.contains('DISTRO_FEATURES', 'my-feature', d):
> > > > include foo.inc
> > > 
> > > This wouldn't actually solve as much of the problem as you think
> > > it
> > > might at first glance and probably causes others, at least as I
> > > understand it (as someone who's worked on bitbake's override
> > > code).
> > > 
> > > For example, at what point does this get evaluated? Most bitbake
> > > variables are expanded at usage time, not parse time but here,
> > > the
> > > way
> > > the parser works today, it would have to do an immediate
> > > expansion
> > > of
> > > DISTRO_FEATURES to decide whether to include this file (or code
> > > block).
> > 
> > Doesn't this same argument apply to doing a conditional include of
> > a
> > file? When bitbake goes to resolve the file name while evaluating
> > the
> > AST, it has to evaluate DISTRO_FEATURES which might not be
> > complete.
> > If
> > the conditional in an "if" statement were also evaluated when
> > evaluating the AST, I believe the following snipet:
> > 
> >  require ${@ oe.utils.optional_includes(d, 'foo-feature:bar.inc') }
> > 
> > Would be (functionally) identical to something (sort of) like:
> > 
> >  if oe.utils.optional_includes(d, 'foo-feature:True'):
> >  
> > 
> > Without requiring splitting the recipe content up into multiple
> > files.
> 
> I did say the problem applied to the require syntax, yes.
> 
> Put another way, my big worry is that the if syntax will make people
> start to want if syntax for things other than include style
> operations
> and try and do other things other than "inclusion" type work with it.
> We'll also need to then start dealing with nesting and most likely
> other complications as well as pushing us to having to deal with
> immediate expansion problems.
> 
> One of the strengths of the current syntax we have to day is that it
> makes most things possible but does try and encourage you to do
> things
> "the right way". In adding an if syntax like this I suspect we're on
> a
> path which won't lead to a good place. I appreciate this isn't an
> exact
> science answer :/.
> 
> 
> To recap on how we get here, there is a problem of selective content
> inclusion in distros/layers. Right now you tend to have to buy into
> everything in a layer or nothing. This is bad for usability and
> adoption of components in layers. Sometimes its not practical to
> separate everything into isolated layers.
> 
> We've therefore tried to come up with a way of handling this adding
> minimal changes but allowing the configuration we need.
> 
> We do need to try and limit the scope of the usage of this as there
> is
> a fundamental issue, namely immediate expansion. I know most users
> will
> not realise there is even a problem with this.
> 
> *If* we limit the scope to DISTRO_FEATURES, we stand a reasonable
> chance of being able to limit the occasions a user runs into this.
> 
> On the other hand, if we add a generic if syntax, encourage usage of
> any variable and so on I think we're setting ourselves up for
> failure.

Sure. I wouldn't suggest using an if statement for "just anything", you
can surely do terrible things that way. It would (by convention) be
restricted to the same sorts of things that the conditional includes
allow now. On a similar token, you can do the same sorts of terrible
things with conditional includes as currently proposed because it has
the same enforcement policy (i.e. "by convention").

On the other hand, perhaps the range of terrible things that can be
done extends to more than 

[OE-core] [PATCH v6] gzip: add ptest

[Denys Zagorui]

Signed-off-by: Denys Zagorui 
---
 .../0001-gzip-port-zdiff-zless-to-Busybox.patch| 59 ++
 meta/recipes-extended/gzip/files/run-ptest |  6 +++
 meta/recipes-extended/gzip/gzip_1.8.bb | 20 +++-
 3 files changed, 84 insertions(+), 1 deletion(-)
 create mode 100644 
meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch
 create mode 100644 meta/recipes-extended/gzip/files/run-ptest

diff --git 
a/meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch 
b/meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch
new file mode 100644
index 000..20d5a19
--- /dev/null
+++ 
b/meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch
@@ -0,0 +1,59 @@
+From 5f712621829ed81a758077431226a86df37fbc3b Mon Sep 17 00:00:00 2001
+From: Denys Zagorui 
+Date: Thu, 8 Jun 2017 16:05:50 +0300
+Subject: [PATCH] gzip: port zdiff, zless to Busybox
+
+Problem reported by Denys Zagorui (Bug#26088).
+* tests/zdiff: Check that diff uses POSIX-format output.
+* zless.in (less_version): Don't exit merely because 'less -V'
+fails; instead, assume 'less' is compatible with an old version of
+the original 'less'.  Busybox 'less -V' fails, but apparently its
+'less' works anyway somehow.
+
+Signed-off-by: Denys Zagorui 
+
+Upstream-Status: Accepted
+---
+ tests/zdiff | 4 +++-
+ zless.in| 2 +-
+ 2 files changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/tests/zdiff b/tests/zdiff
+index 0bb7c7d..9cd4fd4 100755
+--- a/tests/zdiff
 b/tests/zdiff
+@@ -22,7 +22,6 @@
+ 
+ echo a > a || framework_failure_
+ echo b > b || framework_failure_
+-gzip a b || framework_failure_
+ 
+ cat < exp
+ 1c1
+@@ -31,7 +30,10 @@ cat < exp
+ > b
+ EOF
+ 
++diff a b | diff exp - || skip_ "diff output format is incompatible with POSIX"
++
+ fail=0
++gzip a b || fail=1
+ zdiff a.gz b.gz > out 2>&1
+ test $? = 1 || fail=1
+ 
+diff --git a/zless.in b/zless.in
+index e634af6..9759ae6 100644
+--- a/zless.in
 b/zless.in
+@@ -47,7 +47,7 @@ if test "${LESSMETACHARS+set}" != set; then
+   export LESSMETACHARS
+ fi
+ 
+-less_version=`less -V` || exit
++less_version=`less -V 2>/dev/null`
+ case $less_version in
+ less' '45[1-9]* | \
+ less' '4[6-9][0-9]* | \
+-- 
+1.9.1
+
diff --git a/meta/recipes-extended/gzip/files/run-ptest 
b/meta/recipes-extended/gzip/files/run-ptest
new file mode 100644
index 000..cf7c649
--- /dev/null
+++ b/meta/recipes-extended/gzip/files/run-ptest
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+cd src/tests
+
+make check
+
diff --git a/meta/recipes-extended/gzip/gzip_1.8.bb 
b/meta/recipes-extended/gzip/gzip_1.8.bb
index 11be846..a0dec14 100644
--- a/meta/recipes-extended/gzip/gzip_1.8.bb
+++ b/meta/recipes-extended/gzip/gzip_1.8.bb
@@ -2,7 +2,9 @@ require gzip.inc
 
 LICENSE = "GPLv3+"
 
-SRC_URI = "${GNU_MIRROR}/gzip/${BP}.tar.gz"
+SRC_URI = "${GNU_MIRROR}/gzip/${BP}.tar.gz \
+file://0001-gzip-port-zdiff-zless-to-Busybox.patch \
+file://run-ptest"
 SRC_URI_append_class-target = " file://wrong-path-fix.patch"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
@@ -12,6 +14,22 @@ PROVIDES_append_class-native = " gzip-replacement-native"
 
 BBCLASSEXTEND = "native"
 
+inherit ptest
+
+do_install_ptest() {
+   mkdir -p ${D}${PTEST_PATH}/src/build-aux
+   cp ${S}/build-aux/test-driver ${D}${PTEST_PATH}/src/build-aux/
+   mkdir -p ${D}${PTEST_PATH}/src/tests
+   cp -r ${S}/tests/* ${D}${PTEST_PATH}/src/tests
+   sed -e 's/^abs_srcdir = ..*/abs_srcdir = \.\./' \
+-e 's/^top_srcdir = ..*/top_srcdir = \.\./' \
+-e 's/^GREP = ..*/GREP = grep/' \
+-e 's/^AWK = ..*/AWK = awk/'\
+-e 's/^srcdir = ..*/srcdir = \./'   \
+-e 's/^Makefile: ..*/Makefile: /'   \
+${B}/tests/Makefile > ${D}${PTEST_PATH}/src/tests/Makefile
+}
+
 SRC_URI[md5sum] = "732553152814b22dc35aa0267df5286c"
 SRC_URI[sha256sum] = 
"1ff7aedb3d66a0d73f442f6261e4b3860df6fd6c94025c2cb31a202c9c60fe0e"
 
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/2] Yocto Compatible 2.0 support code

[Patrick Ohly]

On Thu, 2017-06-08 at 15:33 +0100, Richard Purdie wrote:
> Patrick for example mentioned IMAGE_FEATURES. This one is fraught with
> problems since:
> 
> a) Its a recipe level setting so using it in a base configuration
> context would end badly

We cannot even rely on DISTRO_FEATURES in the middle of the base
configuration construction; basically conditional includes in the base
configuration are not reliable at all.

> b) Users change this in a variety of places some of which would be
> bitten by the immediate expansion problem even just in recipe context

I agree. The same had already occurred to me, with a slightly different
failure mode:

c) base image recipe does the conditional include based on
   IMAGE_FEATURES, bbappend is used to change IMAGE_FEATURES => include
   done based on something other than the final IMAGE_FEATURES

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 1/2] package_rpm.bbclass: use multithreaded xz compression

[Alexander Kanavin]

RPM's default is single-threaded gz; the change greatly helps with
both buildtimes (when there is a small number of large-sized packages)
and disk space taken by resulting rpms.

Signed-off-by: Alexander Kanavin 
---
 meta/classes/package_rpm.bbclass | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index a844c4d4360..dc241975c3b 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -644,6 +644,8 @@ python do_package_rpm () {
 cmd = cmd + " --define '_build_name_fmt 
%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm'"
 cmd = cmd + " --define '_use_internal_dependency_generator 0'"
 cmd = cmd + " --define '_binaries_in_noarch_packages_terminate_build 0'"
+cmd = cmd + " --define '_binary_payload w6T.xzdio'"
+cmd = cmd + " --define '_source_payload w6T.xzdio'"
 if perfiledeps:
 cmd = cmd + " --define '__find_requires " + outdepends + "'"
 cmd = cmd + " --define '__find_provides " + outprovides + "'"
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 2/2] rpm: run binary package generation via thread pools

[Alexander Kanavin]

This greatly reduces build times when there is a large amount of small
rpm packages to produce. The patches are rather invasive,
and so will be submitted upstream.

Signed-off-by: Alexander Kanavin 
---
 ...y-package-building-into-a-separate-functi.patch |  83 
 ...-binary-package-creation-via-thread-pools.patch | 125 
 ...c-make-operations-over-string-pools-threa.patch | 207 
 ...c-remove-static-local-variables-from-buil.patch | 216 +
 meta/recipes-devtools/rpm/rpm_git.bb   |   4 +
 5 files changed, 635 insertions(+)
 create mode 100644 
meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
 create mode 100644 
meta/recipes-devtools/rpm/files/0002-Run-binary-package-creation-via-thread-pools.patch
 create mode 100644 
meta/recipes-devtools/rpm/files/0003-rpmstrpool.c-make-operations-over-string-pools-threa.patch
 create mode 100644 
meta/recipes-devtools/rpm/files/0004-build-pack.c-remove-static-local-variables-from-buil.patch

diff --git 
a/meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
 
b/meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
new file mode 100644
index 000..3d8b12144e7
--- /dev/null
+++ 
b/meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
@@ -0,0 +1,83 @@
+From b841b699e519438a66b661247c94efff63d0700e Mon Sep 17 00:00:00 2001
+From: Alexander Kanavin 
+Date: Thu, 25 May 2017 18:15:27 +0300
+Subject: [PATCH 01/14] Split binary package building into a separate function
+
+So that it can be run as a thread pool task.
+
+Upstream-Status: Submitted 
[https://github.com/rpm-software-management/rpm/pull/226]
+Signed-off-by: Alexander Kanavin 
+---
+ build/pack.c | 33 +
+ 1 file changed, 21 insertions(+), 12 deletions(-)
+
+diff --git a/build/pack.c b/build/pack.c
+index 497300b96..891e6bdc3 100644
+--- a/build/pack.c
 b/build/pack.c
+@@ -546,18 +546,13 @@ static rpmRC checkPackages(char *pkgcheck)
+ return RPMRC_OK;
+ }
+ 
+-rpmRC packageBinaries(rpmSpec spec, const char *cookie, int cheating)
++static rpmRC packageBinary(rpmSpec spec, Package pkg, const char *cookie, int 
cheating, char** filename)
+ {
+-rpmRC rc;
+-const char *errorString;
+-Package pkg;
+-char *pkglist = NULL;
+-
+-for (pkg = spec->packages; pkg != NULL; pkg = pkg->next) {
+-  char *fn;
++const char *errorString;
++rpmRC rc = RPMRC_OK;
+ 
+   if (pkg->fileList == NULL)
+-  continue;
++  return rc;
+ 
+   if ((rc = processScriptFiles(spec, pkg)))
+   return rc;
+@@ -591,7 +586,7 @@ rpmRC packageBinaries(rpmSpec spec, const char *cookie, 
int cheating)
+headerGetString(pkg->header, RPMTAG_NAME), errorString);
+   return RPMRC_FAIL;
+   }
+-  fn = rpmGetPath("%{_rpmdir}/", binRpm, NULL);
++  *filename = rpmGetPath("%{_rpmdir}/", binRpm, NULL);
+   if ((binDir = strchr(binRpm, '/')) != NULL) {
+   struct stat st;
+   char *dn;
+@@ -613,14 +608,28 @@ rpmRC packageBinaries(rpmSpec spec, const char *cookie, 
int cheating)
+   free(binRpm);
+   }
+ 
+-  rc = writeRPM(pkg, NULL, fn, NULL);
++  rc = writeRPM(pkg, NULL, *filename, NULL);
+   if (rc == RPMRC_OK) {
+   /* Do check each written package if enabled */
+-  char *pkgcheck = rpmExpand("%{?_build_pkgcheck} ", fn, NULL);
++  char *pkgcheck = rpmExpand("%{?_build_pkgcheck} ", *filename, NULL);
+   if (pkgcheck[0] != ' ') {
+   rc = checkPackages(pkgcheck);
+   }
+   free(pkgcheck);
++  }
++return rc;
++}
++
++rpmRC packageBinaries(rpmSpec spec, const char *cookie, int cheating)
++{
++rpmRC rc;
++Package pkg;
++char *pkglist = NULL;
++
++for (pkg = spec->packages; pkg != NULL; pkg = pkg->next) {
++  char *fn = NULL;
++rc = packageBinary(spec, pkg, cookie, cheating, );
++  if (rc == RPMRC_OK) {
+   rstrcat(, fn);
+   rstrcat(, " ");
+   }
+-- 
+2.11.0
+
diff --git 
a/meta/recipes-devtools/rpm/files/0002-Run-binary-package-creation-via-thread-pools.patch
 
b/meta/recipes-devtools/rpm/files/0002-Run-binary-package-creation-via-thread-pools.patch
new file mode 100644
index 000..549148930fa
--- /dev/null
+++ 
b/meta/recipes-devtools/rpm/files/0002-Run-binary-package-creation-via-thread-pools.patch
@@ -0,0 +1,125 @@
+From f8a05339a1e7b307e8de8c858e6e963782fc5925 Mon Sep 17 00:00:00 2001
+From: Alexander Kanavin 
+Date: Thu, 25 May 2017 19:30:20 +0300
+Subject: [PATCH 1/3] Run binary package creation via thread pools.
+
+Upstream-Status: Submitted 

[OE-core] [PATCHv2 0/2] Multi-threaded RPM support

[Alexander Kanavin]

Changes in version 2: use openmp for multi-processing instead of NSPR library, 
as
requested by upstream. Add a link to the upstream submission to patches.

These two patches add multi-threaded features to RPM to speed up the 
do_package_write_rpm task.

Specifically:

1) Multi-threaded xz compressor is used instead of default single-threaded gz. 
This has the most
dramatic effect when a recipe produces a smaller number of large-sized packages.

2) Packages creation is run in multiple threads via thread pools. This is most 
beneficial when
a recipe produces a large amount of small packages.

Some not very scientific benchmarks for time and .rpm sizes
(time is measured for do_package_write_rpm tasks only):

webkitgtk
before: 9m12s 1550M
after: 1m40s 858M

glibc-locale
before: 2m52s 125M
after: 30s 56M

glibc
before: 46s 54M
after: 13s 38M

perl
before: 1m09s 63M
after: 45s 42M

python3
before: 30s 38M
after: 18s 24M

The following changes since commit 576821ea0a7558b626ccc87e9ae0e9ee40864956:

  bitbake: bitbake-layers: check layer dependencies before adding (2017-06-06 
19:52:51 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib akanavin/parallel-rpm
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=akanavin/parallel-rpm

Alexander Kanavin (2):
  package_rpm.bbclass: use multithreaded xz compression
  rpm: run binary package generation via thread pools

 meta/classes/package_rpm.bbclass   |   2 +
 ...y-package-building-into-a-separate-functi.patch |  83 
 ...-binary-package-creation-via-thread-pools.patch | 125 
 ...c-make-operations-over-string-pools-threa.patch | 207 
 ...c-remove-static-local-variables-from-buil.patch | 216 +
 meta/recipes-devtools/rpm/rpm_git.bb   |   4 +
 6 files changed, 637 insertions(+)
 create mode 100644 
meta/recipes-devtools/rpm/files/0001-Split-binary-package-building-into-a-separate-functi.patch
 create mode 100644 
meta/recipes-devtools/rpm/files/0002-Run-binary-package-creation-via-thread-pools.patch
 create mode 100644 
meta/recipes-devtools/rpm/files/0003-rpmstrpool.c-make-operations-over-string-pools-threa.patch
 create mode 100644 
meta/recipes-devtools/rpm/files/0004-build-pack.c-remove-static-local-variables-from-buil.patch

-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] bitbake.conf: DISTRO_FEATURES as overrides

[Patrick Ohly]

On Thu, 2017-06-08 at 13:16 +, Peter Kjellerstedt wrote:
> > My feelings are:
> > 
> > * We need to use a prefix. We've been burnt too many times in the past
> >   when not using these. "df-" is fine, users will just have to deal
> >   with it.

Fine with me.

> > * We should filter the list of overrides to those which the distro
> >   wants to nominate. I really don't want to encourage wider spread of
> >   things than we need to, these need to be conscious decisions.
> 
> Isn't there a risk that will be confusing? I.e., recipes that use, e.g., 
> bb.utils.contains() to check if a distro feature is set will be affected 
> as soon as the feature is added to DISTRO_FEATURES, but recipes that 
> use the override will only be affected if the feature has also been 
> added to some filter variable.

I'm not sure. I guess it boils down to proper documentation.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe-core] DISK_SIGNATURE no longer functional

[Davis, Michael]

Jonathan,

Yep that fixed it.

Thanks,
Mike


-Original Message-
From: Jonathan Liu [mailto:net...@gmail.com] 
Sent: Thursday, June 08, 2017 6:08 AM
To: Davis, Michael
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [oe-core] DISK_SIGNATURE no longer functional

Hi Michael,

I have sent a patch to the mailing list and CC-ed you.
Let me know if it fixes the issue.

Regards,
Jonathan

On 8 June 2017 at 02:12, Davis, Michael  wrote:
> I tried using  ROOT_VM = "root=PARTUUID=${DISK_SIGNATURE}-02"
>
> The resulting PARTUUID in syslinux append line still doesn’t match the one
> on the actual disk.
>
>
>
>
>
> From: Jonathan Liu [mailto:net...@gmail.com]
> Sent: Tuesday, June 06, 2017 5:01 PM
>
>
> To: Davis, Michael
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [oe-core] DISK_SIGNATURE no longer functional
>
>
>
> Hi Michael,
>
>
>
> See https://patchwork.openembedded.org/patch/128565/. SYSLINUX_ROOT was
> renamed to ROOT_VM in Krogoth.
>
>
>
> Regards,
>
> Jonathan
>
>
>
> On Tue., 6 Jun. 2017 at 11:29 pm, Davis, Michael 
> wrote:
>
> Pyro.
>
> Guessing something with all the new wic additions killed it.
>
>
> -Original Message-
> From: Jonathan Liu [mailto:net...@gmail.com]
> Sent: Monday, June 05, 2017 7:17 PM
> To: Davis, Michael
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [oe-core] DISK_SIGNATURE no longer functional
>
> Hi Michael,
>
> On 21 May 2017 at 03:16, Davis, Michael  wrote:
>> I am trying to create a generic MSDOS PC_BIOS hdddirect image with a root
>> partition set by the DISK_SIGNATURE.
>>
>> SYSLINUX_ROOT = "root=PARTUUID=${DISK_SIGNATURE}-02"
>>
>>
>>
>> However the signature generated in syslinux.cfg and the one on the disk
>> don’t match.
>>
>> I tried defining my own signature and the one in syslinux changes, but the
>> one on the disk doesn’t match.
>>
>>
>>
>> root=PARTUUID=22aaf83b-02
>>
>>
>>
>> NAME   PARTUUID
>>
>> sdd
>>
>> ├─sdd2 341e84a7-02
>>
>> ├─sdd3 341e84a7-03
>>
>> └─sdd1 341e84a7-01
>
> Which release did DISK_SIGNATURE stop working?
>
> Regards,
> Jonathan
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] utils.py: helper function for optional include files

[Patrick Ohly]

On Thu, 2017-06-08 at 10:20 +0100, Richard Purdie wrote:
> On Wed, 2017-06-07 at 17:31 +0200, Patrick Ohly wrote:
> > By using oe.utils.optional_includes(), developers can simplify the
> > code which selects which additional include files need to be included
> > in a .bbappend.
> > 
> > In the simple case (one distro feature and one include file) the code
> > is not shorter, but the intent is clearer than corresponding code
> > using bb.utils.contains():
> > 
> >require ${@ oe.utils.optional_includes(d, 'foo-feature:bar.inc') }
> > 
> > More complex cases are also supported, in particular include files
> > that are required for one of several distro features or multiple
> > different include files.
> > 
> > To keep the common use case simple, DISTRO_FEATURES are checked by
> > default. Checking IMAGE_FEATURES might also be useful.
> > 
> > The DISTRO_FEATURES default and the intended usage make this more
> > suitable for OE-core than bitbake.
> 
> I'm honestly not sure this actually aids readability.

Fair enough.

> Taking your example:
> 
> require ${@oe.utils.optional_includes(d, "foo,bar:foo-or-bar.inc 
> xyz:x.inc,y.inc,z.inc")}
> 
> I think I actually prefer:
> 
> require ${@oe.utils.optional_includes(d, "foo,bar:foo-or-bar.inc")}
> require ${@oe.utils.optional_includes(d, "xyz:x.inc,y.inc,z.inc")}
> 
> since its more explict. Spelling this out with the existing syntax
> isn't so bad either, I probably still think that this is clearer too:
> 
> require ${@bb.utils.contains(d, "DISTRO_FEATURES", "foo bar", 
> "foo-or-bar.inc", "")}

That's the "foo and bar" case, not "foo or bar". It's an actual mistake
that people have made and that didn't get caught during code review.

> require ${@bb.utils.contains(d, "DISTRO_FEATURES", "xyz", "x.inc y.inc 
> z.inc", "")}
> 
> How about simply:
> 
> require ${@oe.utils.distro_features(d, "foo bar", "foo-or-bar.inc")}
> require ${@bb.utils.distro_features(d, "xyz", "x.inc y.inc z.inc")}

That works for me, I just wonder about the exact semantic of listing
multiple features: "and" as in contains(), or "any"?

I'm leaning towards two functions where that is explicit:
  oe.utils.all_distro_features(d, "foo bar", "foo-and-bar.inc")
  oe.utils.any_distro_features(d, "foo bar", "foo-or-bar.inc")

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/2] Yocto Compatible 2.0 support code

[Richard Purdie]

On Thu, 2017-06-08 at 08:55 -0500, Joshua Watt wrote:
> On Thu, 2017-06-08 at 09:56 +0100, Richard Purdie wrote:
> > 
> > On Wed, 2017-06-07 at 10:43 -0500, Joshua Watt wrote:
> > > 
> > > On Wed, 2017-06-07 at 17:31 +0200, Patrick Ohly wrote:
> > > > 
> > > > 
> > > > As discussed in the "[Openembedded-architecture] Yocto
> > > > Compatible
> > > > 2.0
> > > > + signature changes" mail thread, changes in a .bbappend cannot
> > > > be
> > > > done unconditionally. Making _append and _remove depend on
> > > > overrides
> > > > which get set based on DISTRO_FEATURES is one way of achieving
> > > > this.
> > > > 
> > > > The oe.utils.optional_includes() helper function has not been
> > > > discussed before. It's an attempt to address concerns by
> > > > developers
> > > > that having to write code for (potentially complex) condition
> > > > checking
> > > > is error prone and hard to read.
> > > I promise I'm not trying to start a flame war here, and perhaps
> > > there
> > > is history behind this that I'm not aware of but...
> > > 
> > > Why doesn't bitbake support some sort of "if" statement? It seems
> > > like most of what we are trying to do could be accomplished with
> > > much
> > > less fuss if one could simply do this in the bb file:
> > > 
> > >  if bb.utils.contains('DISTRO_FEATURES', 'my-feature', d):
> > > include foo.inc
> > This wouldn't actually solve as much of the problem as you think it
> > might at first glance and probably causes others, at least as I
> > understand it (as someone who's worked on bitbake's override code).
> > 
> > For example, at what point does this get evaluated? Most bitbake
> > variables are expanded at usage time, not parse time but here, the
> > way
> > the parser works today, it would have to do an immediate expansion
> > of
> > DISTRO_FEATURES to decide whether to include this file (or code
> > block).
> Doesn't this same argument apply to doing a conditional include of a
> file? When bitbake goes to resolve the file name while evaluating the
> AST, it has to evaluate DISTRO_FEATURES which might not be complete.
> If
> the conditional in an "if" statement were also evaluated when
> evaluating the AST, I believe the following snipet:
> 
>  require ${@ oe.utils.optional_includes(d, 'foo-feature:bar.inc') }
> 
> Would be (functionally) identical to something (sort of) like:
> 
>  if oe.utils.optional_includes(d, 'foo-feature:True'):
>  
> 
> Without requiring splitting the recipe content up into multiple
> files.

I did say the problem applied to the require syntax, yes.

Put another way, my big worry is that the if syntax will make people
start to want if syntax for things other than include style operations
and try and do other things other than "inclusion" type work with it.
We'll also need to then start dealing with nesting and most likely
other complications as well as pushing us to having to deal with
immediate expansion problems.

One of the strengths of the current syntax we have to day is that it
makes most things possible but does try and encourage you to do things
"the right way". In adding an if syntax like this I suspect we're on a
path which won't lead to a good place. I appreciate this isn't an exact
science answer :/.


To recap on how we get here, there is a problem of selective content
inclusion in distros/layers. Right now you tend to have to buy into
everything in a layer or nothing. This is bad for usability and
adoption of components in layers. Sometimes its not practical to
separate everything into isolated layers.

We've therefore tried to come up with a way of handling this adding
minimal changes but allowing the configuration we need.

We do need to try and limit the scope of the usage of this as there is
a fundamental issue, namely immediate expansion. I know most users will
not realise there is even a problem with this.

*If* we limit the scope to DISTRO_FEATURES, we stand a reasonable
chance of being able to limit the occasions a user runs into this.

On the other hand, if we add a generic if syntax, encourage usage of
any variable and so on I think we're setting ourselves up for failure.

Patrick for example mentioned IMAGE_FEATURES. This one is fraught with
problems since:

a) Its a recipe level setting so using it in a base configuration
context would end badly

b) Users change this in a variety of places some of which would be
bitten by the immediate expansion problem even just in recipe context

So no, I really don't like the idea of the if syntax, attractive as it
may look at first.

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] ✗ patchtest: failure for gzip: add ptest (rev5)

[Patchwork]

== Series Details ==

Series: gzip: add ptest (rev5)
Revision: 5
URL   : https://patchwork.openembedded.org/series/5728/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Patch[v5] gzip: add ptest
 Issue Patch is missing Signed-off-by [test_signed_off_by_presence] 
  Suggested fixSign off the patch (either manually or with "git commit 
--amend -s")



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3] oeqa/selftest: Adds verification for the integrity compressed images

[Leonardo Sandoval]

On Thu, 2017-06-08 at 11:51 +0100, Burton, Ross wrote:
> 
> On 6 June 2017 at 20:29, Francisco Pedraza
>  wrote:
> Added verification for images compression divided in 3 steps.
> 1. Add IMAGE_FSTYPES += "[type(s)]" to ~/conf/local.conf
> 2. Build a "core-image-minimal" using this configuration.
> 3. Verify that the image compressed file is present in rootfs
> directory.
> YOCTO[#10745]
> 
> Still fails on the main AB:
> 
> 
> https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/911/steps/Running%20oe-selftest/logs/stdio
> 

perhaps the fails are expected as commented on 

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10745#c9






> 
> Ross
> -- 
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v5] gzip: add ptest

[Denys Zagorui]

Signed-of-by: Denys Zagorui 
---
 .../0001-gzip-port-zdiff-zless-to-Busybox.patch| 59 ++
 meta/recipes-extended/gzip/files/run-ptest |  6 +++
 meta/recipes-extended/gzip/gzip_1.8.bb | 20 +++-
 3 files changed, 84 insertions(+), 1 deletion(-)
 create mode 100644 
meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch
 create mode 100644 meta/recipes-extended/gzip/files/run-ptest

diff --git 
a/meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch 
b/meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch
new file mode 100644
index 000..20d5a19
--- /dev/null
+++ 
b/meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch
@@ -0,0 +1,59 @@
+From 5f712621829ed81a758077431226a86df37fbc3b Mon Sep 17 00:00:00 2001
+From: Denys Zagorui 
+Date: Thu, 8 Jun 2017 16:05:50 +0300
+Subject: [PATCH] gzip: port zdiff, zless to Busybox
+
+Problem reported by Denys Zagorui (Bug#26088).
+* tests/zdiff: Check that diff uses POSIX-format output.
+* zless.in (less_version): Don't exit merely because 'less -V'
+fails; instead, assume 'less' is compatible with an old version of
+the original 'less'.  Busybox 'less -V' fails, but apparently its
+'less' works anyway somehow.
+
+Signed-off-by: Denys Zagorui 
+
+Upstream-Status: Accepted
+---
+ tests/zdiff | 4 +++-
+ zless.in| 2 +-
+ 2 files changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/tests/zdiff b/tests/zdiff
+index 0bb7c7d..9cd4fd4 100755
+--- a/tests/zdiff
 b/tests/zdiff
+@@ -22,7 +22,6 @@
+ 
+ echo a > a || framework_failure_
+ echo b > b || framework_failure_
+-gzip a b || framework_failure_
+ 
+ cat < exp
+ 1c1
+@@ -31,7 +30,10 @@ cat < exp
+ > b
+ EOF
+ 
++diff a b | diff exp - || skip_ "diff output format is incompatible with POSIX"
++
+ fail=0
++gzip a b || fail=1
+ zdiff a.gz b.gz > out 2>&1
+ test $? = 1 || fail=1
+ 
+diff --git a/zless.in b/zless.in
+index e634af6..9759ae6 100644
+--- a/zless.in
 b/zless.in
+@@ -47,7 +47,7 @@ if test "${LESSMETACHARS+set}" != set; then
+   export LESSMETACHARS
+ fi
+ 
+-less_version=`less -V` || exit
++less_version=`less -V 2>/dev/null`
+ case $less_version in
+ less' '45[1-9]* | \
+ less' '4[6-9][0-9]* | \
+-- 
+1.9.1
+
diff --git a/meta/recipes-extended/gzip/files/run-ptest 
b/meta/recipes-extended/gzip/files/run-ptest
new file mode 100644
index 000..cf7c649
--- /dev/null
+++ b/meta/recipes-extended/gzip/files/run-ptest
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+cd src/tests
+
+make check
+
diff --git a/meta/recipes-extended/gzip/gzip_1.8.bb 
b/meta/recipes-extended/gzip/gzip_1.8.bb
index 11be846..a0dec14 100644
--- a/meta/recipes-extended/gzip/gzip_1.8.bb
+++ b/meta/recipes-extended/gzip/gzip_1.8.bb
@@ -2,7 +2,9 @@ require gzip.inc
 
 LICENSE = "GPLv3+"
 
-SRC_URI = "${GNU_MIRROR}/gzip/${BP}.tar.gz"
+SRC_URI = "${GNU_MIRROR}/gzip/${BP}.tar.gz \
+file://0001-gzip-port-zdiff-zless-to-Busybox.patch \
+file://run-ptest"
 SRC_URI_append_class-target = " file://wrong-path-fix.patch"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
@@ -12,6 +14,22 @@ PROVIDES_append_class-native = " gzip-replacement-native"
 
 BBCLASSEXTEND = "native"
 
+inherit ptest
+
+do_install_ptest() {
+   mkdir -p ${D}${PTEST_PATH}/src/build-aux
+   cp ${S}/build-aux/test-driver ${D}${PTEST_PATH}/src/build-aux/
+   mkdir -p ${D}${PTEST_PATH}/src/tests
+   cp -r ${S}/tests/* ${D}${PTEST_PATH}/src/tests
+   sed -e 's/^abs_srcdir = ..*/abs_srcdir = \.\./' \
+-e 's/^top_srcdir = ..*/top_srcdir = \.\./' \
+-e 's/^GREP = ..*/GREP = grep/' \
+-e 's/^AWK = ..*/AWK = awk/'\
+-e 's/^srcdir = ..*/srcdir = \./'   \
+-e 's/^Makefile: ..*/Makefile: /'   \
+${B}/tests/Makefile > ${D}${PTEST_PATH}/src/tests/Makefile
+}
+
 SRC_URI[md5sum] = "732553152814b22dc35aa0267df5286c"
 SRC_URI[sha256sum] = 
"1ff7aedb3d66a0d73f442f6261e4b3860df6fd6c94025c2cb31a202c9c60fe0e"
 
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] ✗ patchtest: failure for gzip: add ptest (rev4)

[Patchwork]

== Series Details ==

Series: gzip: add ptest (rev4)
Revision: 4
URL   : https://patchwork.openembedded.org/series/5728/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Patch[v4] gzip: add ptest
 Issue Patch is missing Signed-off-by [test_signed_off_by_presence] 
  Suggested fixSign off the patch (either manually or with "git commit 
--amend -s")



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/2] Yocto Compatible 2.0 support code

[Joshua Watt]

On Thu, 2017-06-08 at 09:56 +0100, Richard Purdie wrote:
> On Wed, 2017-06-07 at 10:43 -0500, Joshua Watt wrote:
> > On Wed, 2017-06-07 at 17:31 +0200, Patrick Ohly wrote:
> > > 
> > > As discussed in the "[Openembedded-architecture] Yocto Compatible
> > > 2.0
> > > + signature changes" mail thread, changes in a .bbappend cannot
> > > be
> > > done unconditionally. Making _append and _remove depend on
> > > overrides
> > > which get set based on DISTRO_FEATURES is one way of achieving
> > > this.
> > > 
> > > The oe.utils.optional_includes() helper function has not been
> > > discussed before. It's an attempt to address concerns by
> > > developers
> > > that having to write code for (potentially complex) condition
> > > checking
> > > is error prone and hard to read.
> > 
> > I promise I'm not trying to start a flame war here, and perhaps
> > there
> > is history behind this that I'm not aware of but...
> > 
> > Why doesn't bitbake support some sort of "if" statement? It seems
> > like most of what we are trying to do could be accomplished with
> > much
> > less fuss if one could simply do this in the bb file:
> > 
> >  if bb.utils.contains('DISTRO_FEATURES', 'my-feature', d):
> > include foo.inc
> 
> This wouldn't actually solve as much of the problem as you think it
> might at first glance and probably causes others, at least as I
> understand it (as someone who's worked on bitbake's override code).
> 
> For example, at what point does this get evaluated? Most bitbake
> variables are expanded at usage time, not parse time but here, the
> way
> the parser works today, it would have to do an immediate expansion of
> DISTRO_FEATURES to decide whether to include this file (or code
> block).

Doesn't this same argument apply to doing a conditional include of a
file? When bitbake goes to resolve the file name while evaluating the
AST, it has to evaluate DISTRO_FEATURES which might not be complete. If
the conditional in an "if" statement were also evaluated when
evaluating the AST, I believe the following snipet:

 require ${@ oe.utils.optional_includes(d, 'foo-feature:bar.inc') }

Would be (functionally) identical to something (sort of) like:

 if oe.utils.optional_includes(d, 'foo-feature:True'):
 

Without requiring splitting the recipe content up into multiple files.


> So ok, lets assume we change bitbake massively and defer the
> expansion
> somehow. What if foo.inc influences the contents of DISTRO_FEATURES?
> Should it then "unparse" foo.inc if my-feature was removed? or error?
> or silently ignore that?
> 
> bitbake's main conditional today is through overrides and these do
> allow a controlled delayed expansion of metadata in most cases. In
> some
> cases such as include and inherit statements there is still the
> immediate expansion issue above but at least there aren't huge
> changes
> to the parser required to make it work so its the best of both
> worlds.

I was curious as to what it would it would actually take to make "if"
statements like the one I described above work (and I wanted to learn
more about the bitbake internals), so I did a proof of concept on
GitHub:
https://github.com/JPEWdev/poky/commit/998a00f122154bb509d22b412fba0773
97f6e433

It's actually not particularly terrible IMHO, but I'm sure it could be
better.

I can repost it to the bitbake mailing list as an RFC if you think that
would be helpful.

> 
> > One could even eliminate the separate inc file and simply put its
> > contents under the conditional (as much fun as it seems to have to
> > open
> > a new file just to see what a recipe is doing with a distro
> > feature...)
> > 
> > It would also appear that this could make a lot of other things
> > simpler as well (and may even negate the need to backfill
> > DISTRO_FEATURES into overrides?)
> 
> See if the above gives food for thought on that...
> 
> The big problems are the corner cases. If we do add new syntax it
> needs
> to avoid these as we already have some pretty nasty ones, thankfully
> most people don't hit them though.

That's fine. I"m not particularly trying to say that an "if" statement
is the magic bullet for corner cases, but I think it is equivalent
functionality to conditional includes and more readable and
maintainable for people writing recipes. Maybe that means
DISTRO_FEATURES still need to become OVERRIDES, IDK.

> 
> Cheers,
> 
> Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v4] gzip: add ptest

[Denys Zagorui]

---
 .../0001-gzip-port-zdiff-zless-to-Busybox.patch| 59 ++
 meta/recipes-extended/gzip/files/run-ptest |  6 +++
 meta/recipes-extended/gzip/gzip_1.8.bb | 20 +++-
 3 files changed, 84 insertions(+), 1 deletion(-)
 create mode 100644 
meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch
 create mode 100644 meta/recipes-extended/gzip/files/run-ptest

diff --git 
a/meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch 
b/meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch
new file mode 100644
index 000..20d5a19
--- /dev/null
+++ 
b/meta/recipes-extended/gzip/files/0001-gzip-port-zdiff-zless-to-Busybox.patch
@@ -0,0 +1,59 @@
+From 5f712621829ed81a758077431226a86df37fbc3b Mon Sep 17 00:00:00 2001
+From: Denys Zagorui 
+Date: Thu, 8 Jun 2017 16:05:50 +0300
+Subject: [PATCH] gzip: port zdiff, zless to Busybox
+
+Problem reported by Denys Zagorui (Bug#26088).
+* tests/zdiff: Check that diff uses POSIX-format output.
+* zless.in (less_version): Don't exit merely because 'less -V'
+fails; instead, assume 'less' is compatible with an old version of
+the original 'less'.  Busybox 'less -V' fails, but apparently its
+'less' works anyway somehow.
+
+Signed-off-by: Denys Zagorui 
+
+Upstream-Status: Accepted
+---
+ tests/zdiff | 4 +++-
+ zless.in| 2 +-
+ 2 files changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/tests/zdiff b/tests/zdiff
+index 0bb7c7d..9cd4fd4 100755
+--- a/tests/zdiff
 b/tests/zdiff
+@@ -22,7 +22,6 @@
+ 
+ echo a > a || framework_failure_
+ echo b > b || framework_failure_
+-gzip a b || framework_failure_
+ 
+ cat < exp
+ 1c1
+@@ -31,7 +30,10 @@ cat < exp
+ > b
+ EOF
+ 
++diff a b | diff exp - || skip_ "diff output format is incompatible with POSIX"
++
+ fail=0
++gzip a b || fail=1
+ zdiff a.gz b.gz > out 2>&1
+ test $? = 1 || fail=1
+ 
+diff --git a/zless.in b/zless.in
+index e634af6..9759ae6 100644
+--- a/zless.in
 b/zless.in
+@@ -47,7 +47,7 @@ if test "${LESSMETACHARS+set}" != set; then
+   export LESSMETACHARS
+ fi
+ 
+-less_version=`less -V` || exit
++less_version=`less -V 2>/dev/null`
+ case $less_version in
+ less' '45[1-9]* | \
+ less' '4[6-9][0-9]* | \
+-- 
+1.9.1
+
diff --git a/meta/recipes-extended/gzip/files/run-ptest 
b/meta/recipes-extended/gzip/files/run-ptest
new file mode 100644
index 000..cf7c649
--- /dev/null
+++ b/meta/recipes-extended/gzip/files/run-ptest
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+cd src/tests
+
+make check
+
diff --git a/meta/recipes-extended/gzip/gzip_1.8.bb 
b/meta/recipes-extended/gzip/gzip_1.8.bb
index 11be846..a0dec14 100644
--- a/meta/recipes-extended/gzip/gzip_1.8.bb
+++ b/meta/recipes-extended/gzip/gzip_1.8.bb
@@ -2,7 +2,9 @@ require gzip.inc
 
 LICENSE = "GPLv3+"
 
-SRC_URI = "${GNU_MIRROR}/gzip/${BP}.tar.gz"
+SRC_URI = "${GNU_MIRROR}/gzip/${BP}.tar.gz \
+file://0001-gzip-port-zdiff-zless-to-Busybox.patch \
+file://run-ptest"
 SRC_URI_append_class-target = " file://wrong-path-fix.patch"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
@@ -12,6 +14,22 @@ PROVIDES_append_class-native = " gzip-replacement-native"
 
 BBCLASSEXTEND = "native"
 
+inherit ptest
+
+do_install_ptest() {
+   mkdir -p ${D}${PTEST_PATH}/src/build-aux
+   cp ${S}/build-aux/test-driver ${D}${PTEST_PATH}/src/build-aux/
+   mkdir -p ${D}${PTEST_PATH}/src/tests
+   cp -r ${S}/tests/* ${D}${PTEST_PATH}/src/tests
+   sed -e 's/^abs_srcdir = ..*/abs_srcdir = \.\./' \
+-e 's/^top_srcdir = ..*/top_srcdir = \.\./' \
+-e 's/^GREP = ..*/GREP = grep/' \
+-e 's/^AWK = ..*/AWK = awk/'\
+-e 's/^srcdir = ..*/srcdir = \./'   \
+-e 's/^Makefile: ..*/Makefile: /'   \
+${B}/tests/Makefile > ${D}${PTEST_PATH}/src/tests/Makefile
+}
+
 SRC_URI[md5sum] = "732553152814b22dc35aa0267df5286c"
 SRC_URI[sha256sum] = 
"1ff7aedb3d66a0d73f442f6261e4b3860df6fd6c94025c2cb31a202c9c60fe0e"
 
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH V2] python2/3: Remove building host path from packages

[Burton, Ross]

On 5 June 2017 at 10:15, Li Zhou  wrote:

> @@ -185,9 +185,14 @@ py_package_preprocess () {
> install -m 0644 ${B}/Makefile.orig ${PKGD}/${libdir}/python${
> PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}/Makefile
> # Remove references to buildmachine paths in target Makefile and
> _sysconfigdata
> sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e
> s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \
> +   -e 's:${STAGING_DIR_NATIVE}::g' \
> +   -e 's:${STAGING_DIR_HOST}::g' \
> +   -e 's:${WORKDIR}::g' \
> +   -e 's:${TMPDIR}::g' \
> ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile \
> ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config-${
> PYTHON_MAJMIN}${PYTHON_ABI}/Makefile \
> -   ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py
> +   ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py
> \
> +   ${PKGD}/${bindir}/python${PYTHON_MAJMIN}${PYTHON_ABI}-
> config
>  }
>

The problem with doing this is py_package_preprocess is that it doesn't
touch the .pyc files, so they'll have to be recompiled:

ross@flashheart
/data/poky-master/tmp/work/corei7-64-poky-linux/python3/3.5.3-r1.0/packages-split
$ grep -r /data/poky-master
Binary file
python3-core/usr/lib/python3.5/__pycache__/_sysconfigdata.cpython-35.opt-2.pyc
matches
Binary file
python3-core/usr/lib/python3.5/__pycache__/_sysconfigdata.cpython-35.opt-1.pyc
matches
Binary file
python3-core/usr/lib/python3.5/__pycache__/_sysconfigdata.cpython-35.pyc
matches

I think the seds need to be moved earlier in the build, or that py file
recompiled.

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] alsa-utils: 1.1.3 -> 1.1.4

[Tanu Kaskinen]

Changelog: http://alsa-project.org/main/index.php/Changes_v1.1.3_v1.1.4

Signed-off-by: Tanu Kaskinen 
---
 .../alsa/{alsa-utils-scripts_1.1.3.bb => alsa-utils-scripts_1.1.4.bb} | 0
 .../alsa/{alsa-utils_1.1.3.bb => alsa-utils_1.1.4.bb} | 4 ++--
 2 files changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-multimedia/alsa/{alsa-utils-scripts_1.1.3.bb => 
alsa-utils-scripts_1.1.4.bb} (100%)
 rename meta/recipes-multimedia/alsa/{alsa-utils_1.1.3.bb => 
alsa-utils_1.1.4.bb} (97%)

diff --git a/meta/recipes-multimedia/alsa/alsa-utils-scripts_1.1.3.bb 
b/meta/recipes-multimedia/alsa/alsa-utils-scripts_1.1.4.bb
similarity index 100%
rename from meta/recipes-multimedia/alsa/alsa-utils-scripts_1.1.3.bb
rename to meta/recipes-multimedia/alsa/alsa-utils-scripts_1.1.4.bb
diff --git a/meta/recipes-multimedia/alsa/alsa-utils_1.1.3.bb 
b/meta/recipes-multimedia/alsa/alsa-utils_1.1.4.bb
similarity index 97%
rename from meta/recipes-multimedia/alsa/alsa-utils_1.1.3.bb
rename to meta/recipes-multimedia/alsa/alsa-utils_1.1.4.bb
index e6dd5b0a87..c749f32120 100644
--- a/meta/recipes-multimedia/alsa/alsa-utils_1.1.3.bb
+++ b/meta/recipes-multimedia/alsa/alsa-utils_1.1.4.bb
@@ -23,8 +23,8 @@ SRC_URI = 
"ftp://ftp.alsa-project.org/pub/utils/alsa-utils-${PV}.tar.bz2 \

file://0001-alsactl-don-t-let-systemd-unit-restore-the-volume-wh.patch \
   "
 
-SRC_URI[md5sum] = "2bf94d3e3410dcc74bb0dae10d46a979"
-SRC_URI[sha256sum] = 
"127217a54eea0f9a49700a2f239a2d4f5384aa094d68df04a8eb80132eb6167c"
+SRC_URI[md5sum] = "01e3934ca5bd22a80c27289d1b0adcdc"
+SRC_URI[sha256sum] = 
"a7831044de92c5bf33bf3365a3f36e49397f4191e934df460ae1ca15138c9d9d"
 
 # On build machines with python-docutils (not python3-docutils !!) installed
 # rst2man (not rst2man.py) is detected and compile fails with
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] alsa-plugins: 1.1.1 -> 1.1.4

[Tanu Kaskinen]

Changelog: http://alsa-project.org/main/index.php/Changes_v1.1.3_v1.1.4

Signed-off-by: Tanu Kaskinen 
---
 .../alsa/{alsa-plugins_1.1.1.bb => alsa-plugins_1.1.4.bb} | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-multimedia/alsa/{alsa-plugins_1.1.1.bb => 
alsa-plugins_1.1.4.bb} (96%)

diff --git a/meta/recipes-multimedia/alsa/alsa-plugins_1.1.1.bb 
b/meta/recipes-multimedia/alsa/alsa-plugins_1.1.4.bb
similarity index 96%
rename from meta/recipes-multimedia/alsa/alsa-plugins_1.1.1.bb
rename to meta/recipes-multimedia/alsa/alsa-plugins_1.1.4.bb
index 16686a0445..b7f79b7535 100644
--- a/meta/recipes-multimedia/alsa/alsa-plugins_1.1.1.bb
+++ b/meta/recipes-multimedia/alsa/alsa-plugins_1.1.4.bb
@@ -21,8 +21,8 @@ LIC_FILES_CHKSUM = "\
 "
 
 SRC_URI = "ftp://ftp.alsa-project.org/pub/plugins/${BP}.tar.bz2;
-SRC_URI[md5sum] = "69f9f3e2de3c97fc71d496e91e271fe5"
-SRC_URI[sha256sum] = 
"8ea4d1e082c36528a896a2581e5eb62d4dc2683238e353050d0d624e65f901f1"
+SRC_URI[md5sum] = "de51130a7444b79b2dd3c25e28420754"
+SRC_URI[sha256sum] = 
"530d1c3bdaeb058f2a03607a33b9e16ee5369bfd30a96bc09bd2c69b4ddd1a8a"
 
 DEPENDS += "alsa-lib"
 
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] Update the alsa recipes

[Tanu Kaskinen]

The following changes since commit 186882ca62bf683b93cd7a250963921b89ba071f:

  buildhistory: skip tests if GitPython module is missing (2017-06-07 16:00:43 
+0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib tanuk/updates
  http://cgit.openembedded.org/openembedded-core-contrib/log/?h=tanuk/updates

Tanu Kaskinen (3):
  alsa-lib: 1.1.3 -> 1.1.4.1
  alsa-utils: 1.1.3 -> 1.1.4
  alsa-plugins: 1.1.1 -> 1.1.4

 .../alsa-lib/0001-ucm-parser-needs-limits.h.patch  | 33 --
 .../{alsa-lib_1.1.3.bb => alsa-lib_1.1.4.1.bb} |  5 ++--
 ...alsa-plugins_1.1.1.bb => alsa-plugins_1.1.4.bb} |  4 +--
 ...cripts_1.1.3.bb => alsa-utils-scripts_1.1.4.bb} |  0
 .../{alsa-utils_1.1.3.bb => alsa-utils_1.1.4.bb}   |  4 +--
 5 files changed, 6 insertions(+), 40 deletions(-)
 delete mode 100644 
meta/recipes-multimedia/alsa/alsa-lib/0001-ucm-parser-needs-limits.h.patch
 rename meta/recipes-multimedia/alsa/{alsa-lib_1.1.3.bb => alsa-lib_1.1.4.1.bb} 
(87%)
 rename meta/recipes-multimedia/alsa/{alsa-plugins_1.1.1.bb => 
alsa-plugins_1.1.4.bb} (96%)
 rename meta/recipes-multimedia/alsa/{alsa-utils-scripts_1.1.3.bb => 
alsa-utils-scripts_1.1.4.bb} (100%)
 rename meta/recipes-multimedia/alsa/{alsa-utils_1.1.3.bb => 
alsa-utils_1.1.4.bb} (97%)

-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] alsa-lib: 1.1.3 -> 1.1.4.1

[Tanu Kaskinen]

Changelogs:
http://alsa-project.org/main/index.php/Changes_v1.1.3_v1.1.4
http://alsa-project.org/main/index.php/Changes_v1.1.4_v1.1.4.1

Dropped backported patch 0001-ucm-parser-needs-limits.h.patch.

Signed-off-by: Tanu Kaskinen 
---
 .../alsa-lib/0001-ucm-parser-needs-limits.h.patch  | 33 --
 .../{alsa-lib_1.1.3.bb => alsa-lib_1.1.4.1.bb} |  5 ++--
 2 files changed, 2 insertions(+), 36 deletions(-)
 delete mode 100644 
meta/recipes-multimedia/alsa/alsa-lib/0001-ucm-parser-needs-limits.h.patch
 rename meta/recipes-multimedia/alsa/{alsa-lib_1.1.3.bb => alsa-lib_1.1.4.1.bb} 
(87%)

diff --git 
a/meta/recipes-multimedia/alsa/alsa-lib/0001-ucm-parser-needs-limits.h.patch 
b/meta/recipes-multimedia/alsa/alsa-lib/0001-ucm-parser-needs-limits.h.patch
deleted file mode 100644
index 4edaf4d961..00
--- a/meta/recipes-multimedia/alsa/alsa-lib/0001-ucm-parser-needs-limits.h.patch
+++ /dev/null
@@ -1,33 +0,0 @@
-From 005ac9d2fa000912c8fb8257020a0471b1c6176c Mon Sep 17 00:00:00 2001
-From: Gustavo Zacarias 
-Date: Wed, 21 Dec 2016 19:46:34 -0300
-Subject: [PATCH] ucm: parser needs limits.h
-
-It's using PATH_MAX which is defined there, otherwise the build fails on
-musl libc.
-
-Signed-off-by: Gustavo Zacarias 
-Signed-off-by: Takashi Iwai 
-
-Upstream-Status: Accepted [expected in 1.1.4]
-
-Signed-off-by: Tanu Kaskinen 

- src/ucm/parser.c | 1 +
- 1 file changed, 1 insertion(+)
-
-diff --git a/src/ucm/parser.c b/src/ucm/parser.c
-index c98373a9..f520abc5 100644
 a/src/ucm/parser.c
-+++ b/src/ucm/parser.c
-@@ -32,6 +32,7 @@
- 
- #include "ucm_local.h"
- #include 
-+#include 
- 
- /** The name of the environment variable containing the UCM directory */
- #define ALSA_CONFIG_UCM_VAR "ALSA_CONFIG_UCM"
--- 
-2.11.0
-
diff --git a/meta/recipes-multimedia/alsa/alsa-lib_1.1.3.bb 
b/meta/recipes-multimedia/alsa/alsa-lib_1.1.4.1.bb
similarity index 87%
rename from meta/recipes-multimedia/alsa/alsa-lib_1.1.3.bb
rename to meta/recipes-multimedia/alsa/alsa-lib_1.1.4.1.bb
index 191a0367de..947f7d69e1 100644
--- a/meta/recipes-multimedia/alsa/alsa-lib_1.1.3.bb
+++ b/meta/recipes-multimedia/alsa/alsa-lib_1.1.4.1.bb
@@ -11,10 +11,9 @@ BBCLASSEXTEND = "native nativesdk"
 SRC_URI = "ftp://ftp.alsa-project.org/pub/lib/${BP}.tar.bz2 \
file://Check-if-wordexp-function-is-supported.patch \
file://avoid-including-sys-poll.h-directly.patch \
-   file://0001-ucm-parser-needs-limits.h.patch \
 "
-SRC_URI[md5sum] = "eefe5992567ba00d6110a540657aaf5c"
-SRC_URI[sha256sum] = 
"71282502184c592c1a008e256c22ed0ba5728ca65e05273ceb480c70f515969c"
+SRC_URI[md5sum] = "29fa3e69122d3cf3e8f0e01a0cb1d183"
+SRC_URI[sha256sum] = 
"91bb870c14d1c7c269213285eeed874fa3d28112077db061a3af8010d0885b76"
 
 inherit autotools pkgconfig
 
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] bitbake.conf: DISTRO_FEATURES as overrides

[Peter Kjellerstedt]

> -Original Message-
> From: Richard Purdie [mailto:richard.pur...@linuxfoundation.org]
> Sent: den 8 juni 2017 12:45
> To: Patrick Ohly ; Peter Kjellerstedt
> 
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 1/2] bitbake.conf: DISTRO_FEATURES as
> overrides
> 
> On Thu, 2017-06-08 at 08:04 +0200, Patrick Ohly wrote:
> > On Wed, 2017-06-07 at 16:11 +, Peter Kjellerstedt wrote:
> > >
> > > Rather than requiring that the wanted DISTRO_FEATURES that should
> > > be
> > > available as overrides are defined in DISTRO_FEATURES_OVERRIDES
> > > (which
> > > should not be confused with the similarly named
> > > DISTROFEATURESOVERRIDES
> > > variable that you also add...),
> > I had thought about those names and in the end went ahead with the
> > similar names because the customizable one made sense to me and the
> > internal one is similar to the other entries in OVERRIDES.
> >
> > >
> > >  why not add them all but with a prefix.
> > > I.e., similar to how package names are available as overrides
> > > prefixed
> > > with "pn-", how about all distro features are made available as
> > > overrides with a "df-" prefix?
> > That would be fine with me.
> >
> > I just have a few concerns:
> >   * How performance-sensitive is OVERRIDES? How can the impact of
> > both approaches be benchmarked? The idea behind the
> > configurable
> > subset was to add only a few new overrides. We currently have
> > almost 70 individual entries in DISTRO_FEATURES.
> >   * I've seen confusion about the pn- prefix. At least df- would
> > be
> > named appropriately (in contrast to PN, which is historic),
> > but
> > it's yet another convention that might not be immediately
> > obvious. The same is true for selecting a subset with the
> > same
> > name as the feature, though.
> >   * Can distro features contain characters that are invalid in an
> > override? _ and : would have to be avoided, for example by
> > mapping them to -.
> 
> My feelings are:
> 
> * We need to use a prefix. We've been burnt too many times in the past
>   when not using these. "df-" is fine, users will just have to deal
>   with it.
> 
> * We should filter the list of overrides to those which the distro
>   wants to nominate. I really don't want to encourage wider spread of
>   things than we need to, these need to be conscious decisions.

Isn't there a risk that will be confusing? I.e., recipes that use, e.g., 
bb.utils.contains() to check if a distro feature is set will be affected 
as soon as the feature is added to DISTRO_FEATURES, but recipes that 
use the override will only be affected if the feature has also been 
added to some filter variable.

> Cheers,
> 
> Richard

//Peter

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3] runtime/dnf: Add new dnf test cases

[Jose Perez Carranza]

On 06/08/2017 07:49 AM, Alexander Kanavin wrote:

On 06/08/2017 03:49 PM, Jose Perez Carranza wrote:


1. bitbake core-image-sato using shared states (I'm using local sstates)


When you do this, tmp/deploy/rpm should get re-populated with rpm 
packages, regardless of whether they're taken from sstate, or rebuilt.
All those steps are done on a fresh build directory, so the first time 
that image is built packages are not being populated when rebuilding the 
image packages are get pupulated, but AB only built once the image and 
then execute testimage.


Can you check why this does not happen on your side?

Alex


--
Saludos
José

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] opkg-utils: if Python support is disabled, delete the Python scripts

[Ross Burton]

To make it obvious what is lost when python is disabled, actually delete the
scripts so the user can't attempt to use them.

Signed-off-by: Ross Burton 
---
 meta/recipes-devtools/opkg-utils/opkg-utils_git.bb | 5 +
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb 
b/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb
index 06710b5..ed5980b 100644
--- a/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb
+++ b/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb
@@ -19,6 +19,7 @@ S = "${WORKDIR}/git"
 
 TARGET_CC_ARCH += "${LDFLAGS}"
 
+# For native builds we use the host Python
 PYTHONRDEPS = "python3 python3-shell python3-io python3-math python3-crypt 
python3-logging python3-fcntl python3-subprocess python3-pickle 
python3-compression python3-textutils python3-stringold"
 PYTHONRDEPS_class-native = ""
 
@@ -31,6 +32,10 @@ do_install() {
if ! ${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 
'true', 'false', d)}; then
rm -f "${D}${bindir}/update-alternatives"
fi
+
+if ! ${@bb.utils.contains('PACKAGECONFIG', 'python', 'true', 'false', d)}; 
then
+grep -lZ "/usr/bin/env.*python" ${D}${bindir}/* | xargs -0 rm
+fi
 }
 
 do_install_append_class-target() {
-- 
2.8.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] State of bitbake world, Failed tasks 2017-06-06

[Martin Jansa]

http://www.openembedded.org/wiki/Bitbake_World_Status

== Number of issues - stats ==
{| class='wikitable'
!|Date   !!colspan='3'|Failed tasks 
!!|Signatures !!colspan='14'|QA !!Comment
|-
||  ||qemuarm   ||qemux86   ||qemux86_64||all   
||already-stripped  ||libdir||textrel   ||build-deps
||file-rdeps||version-going-backwards   ||host-user-contaminated
||installed-vs-shipped  ||unknown-configure-option  ||symlink-to-sysroot
||invalid-pkgconfig ||pkgname   ||ldflags   ||compile-host-path 
||  
|-
||2017-06-06||3 ||2 ||3 ||0 ||0 ||1 
||2 ||0 ||0 ||6 ||5 
||0 ||0 ||0 ||0 ||0 
||0 ||0 ||  
|}

== Failed tasks 2017-06-06 ==

INFO: jenkins-job.sh-1.8.21 Complete log available at 
http://logs.nslu2-linux.org/buildlogs/oe/world/rocko/log.report.20170607_102110.log

=== common (0) ===

=== common-x86 (2) ===
* 
meta-openembedded/meta-oe/recipes-support/espeak/espeak_1.48.04.bb:do_compile
* meta-qt5/recipes-qt/qt5/qtwebengine_git.bb:do_compile

=== qemuarm (3) ===
* meta-qt5/recipes-qt/qt5/qtlocation_git.bb:do_compile
* meta-qt5/recipes-qt/qt5/qtremoteobjects_git.bb:do_compile
* 
openembedded-core/meta/recipes-support/libunwind/libunwind_1.2.bb:do_compile

=== qemux86 (0) ===

=== qemux86_64 (0) ===

=== Number of failed tasks (8) ===
{| class=wikitable
|-
|| qemuarm  || 3 || 
http://logs.nslu2-linux.org/buildlogs/oe/world/rocko/log.world.qemuarm.20170606_034118.log/
 || http://errors.yoctoproject.org/Errors/Build/38558/
|-
|| qemux86  || 2 || 
http://logs.nslu2-linux.org/buildlogs/oe/world/rocko/log.world.qemux86.20170606_043640.log/
 || http://errors.yoctoproject.org/Errors/Build/38561/
|-
|| qemux86_64   || 3 || 
http://logs.nslu2-linux.org/buildlogs/oe/world/rocko/log.world.qemux86-64.20170606_151725.log/
 || http://errors.yoctoproject.org/Errors/Build/38651/
|}

=== PNBLACKLISTs (15) ===

=== QA issues (14) ===
{| class=wikitable
!| Count||Issue
|-
||0 ||already-stripped
|-
||0 ||build-deps
|-
||0 ||compile-host-path
|-
||0 ||file-rdeps
|-
||0 ||installed-vs-shipped
|-
||0 ||invalid-pkgconfig
|-
||0 ||ldflags
|-
||0 ||pkgname
|-
||0 ||symlink-to-sysroot
|-
||0 ||unknown-configure-option
|-
||1 ||libdir
|-
||2 ||textrel
|-
||5 ||host-user-contaminated
|-
||6 ||version-going-backwards
|}



=== Incorrect PACKAGE_ARCH or sstate signatures (0) ===

Complete log: 
http://logs.nslu2-linux.org/buildlogs/oe/world/rocko/log.signatures.20170606_103657.log/


* ERROR: Nothing PROVIDES 'qtwebengine' (but 
/home/jenkins/oe/world/shr-core/meta-qt5/recipes-qt/qt5/qtwebview_git.bb 
DEPENDS on or otherwise requires it)
* ERROR: qtwebengine was skipped: incompatible with machine qemuarm (not in 
COMPATIBLE_MACHINE)
* ERROR: Required build target 'meta-world-pkgdata' has no buildable 
providers.


PNBLACKLISTs:
openembedded-core/:
meta-browser:
recipes-browser/chromium/cef3_280796.bb:PNBLACKLIST[cef3] ?= "BROKEN: fails to 
build with gcc-6"
meta-openembedded:
meta-networking/recipes-support/lksctp-tools/lksctp-tools_1.0.17.bb:PNBLACKLIST[lksctp-tools]
 ?= "${@bb.utils.contains('DISTRO_FEATURES', 'ld-is-gold', "BROKEN: fails to 
link against sctp_connectx symbol", '', d)}"
meta-oe/recipes-connectivity/bluez/bluez-hcidump_2.5.bb:PNBLACKLIST[bluez-hcidump]
 ?= "${@bb.utils.contains('DISTRO_FEATURES', 'bluez5', 'bluez5 conflicts with 
bluez4 and bluez5 is selected in DISTRO_FEATURES', '', d)}"
meta-oe/recipes-connectivity/bluez/bluez4_4.101.bb:PNBLACKLIST[bluez4] ?= 
"${@bb.utils.contains('DISTRO_FEATURES', 'bluez5', 'bluez5 conflicts with 
bluez4 and bluez5 is selected in DISTRO_FEATURES', '', d)}"
meta-oe/recipes-connectivity/bluez/gst-plugin-bluetooth_4.101.bb:PNBLACKLIST[gst-plugin-bluetooth]
 ?= "${@bb.utils.contains('DISTRO_FEATURES', 'bluez5', 'bluez5 conflicts with 
bluez4 and bluez5 is selected in DISTRO_FEATURES', '', d)}"
meta-oe/recipes-core/dbus/libdbus-c++_0.9.0.bb:PNBLACKLIST[libdbus-c++] ?= 
"Fails to build with RSS http://errors.yoctoproject.org/Errors/Details/130644/ 
- the recipe will be removed on 2017-09-01 unless the issue is fixed"
meta-oe/recipes-graphics/libsexy/libsexy_0.1.11.bb:PNBLACKLIST[libsexy] ?= 
"Fails to build with RSS http://errors.yoctoproject.org/Errors/Details/130607/ 
- the recipe will be removed on 2017-09-01 unless the issue is fixed"
meta-oe/recipes-graphics/xorg-driver/xf86-video-geode_2.11.16.bb:PNBLACKLIST[xf86-video-geode]
 ?= "BROKEN, fails to build - the recipe will be removed on 2017-09-01 unless 
the issue is fixed"
meta-oe/recipes-navigation/foxtrotgps/foxtrotgps_1.1.1.bb:PNBLACKLIST[foxtrotgps]
 ?= 

[OE-core] [PATCH v2] openssl: Upgrade 1.0.2k -> 1.0.2l

[Changhyeok Bae]

Dropped obsolete patches, because the new version contains them:
  - fix-cipher-des-ede3-cfb1.patch
  - openssl-avoid-NULL-pointer-dereference-in-EVP_DigestInit_ex.patch

Signed-off-by: Changhyeok Bae 
---
 .../openssl/openssl/fix-cipher-des-ede3-cfb1.patch | 21 
 ...-pointer-dereference-in-EVP_DigestInit_ex.patch | 23 --
 .../{openssl_1.0.2k.bb => openssl_1.0.2l.bb}   |  8 +++-
 3 files changed, 3 insertions(+), 49 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/openssl/openssl/fix-cipher-des-ede3-cfb1.patch
 delete mode 100644 
meta/recipes-connectivity/openssl/openssl/openssl-avoid-NULL-pointer-dereference-in-EVP_DigestInit_ex.patch
 rename meta/recipes-connectivity/openssl/{openssl_1.0.2k.bb => 
openssl_1.0.2l.bb} (86%)

diff --git 
a/meta/recipes-connectivity/openssl/openssl/fix-cipher-des-ede3-cfb1.patch 
b/meta/recipes-connectivity/openssl/openssl/fix-cipher-des-ede3-cfb1.patch
deleted file mode 100644
index 2a318a4..000
--- a/meta/recipes-connectivity/openssl/openssl/fix-cipher-des-ede3-cfb1.patch
+++ /dev/null
@@ -1,21 +0,0 @@
-Upstream-Status: Submitted
-
-This patch adds the fix for one of the ciphers used in openssl, namely
-the cipher des-ede3-cfb1. Complete bug log and patch is present here:
-http://rt.openssl.org/Ticket/Display.html?id=2867
-
-Signed-off-by: Muhammad Shakeel 
-
-Index: openssl-1.0.2/crypto/evp/e_des3.c
-===
 openssl-1.0.2.orig/crypto/evp/e_des3.c
-+++ openssl-1.0.2/crypto/evp/e_des3.c
-@@ -211,7 +211,7 @@ static int des_ede3_cfb1_cipher(EVP_CIPH
- size_t n;
- unsigned char c[1], d[1];
- 
--for (n = 0; n < inl; ++n) {
-+for (n = 0; n * 8 < inl; ++n) {
- c[0] = (in[n / 8] & (1 << (7 - n % 8))) ? 0x80 : 0;
- DES_ede3_cfb_encrypt(c, d, 1, 1,
-  (ctx)->ks1, (ctx)->ks2,
diff --git 
a/meta/recipes-connectivity/openssl/openssl/openssl-avoid-NULL-pointer-dereference-in-EVP_DigestInit_ex.patch
 
b/meta/recipes-connectivity/openssl/openssl/openssl-avoid-NULL-pointer-dereference-in-EVP_DigestInit_ex.patch
deleted file mode 100644
index f736e5c..000
--- 
a/meta/recipes-connectivity/openssl/openssl/openssl-avoid-NULL-pointer-dereference-in-EVP_DigestInit_ex.patch
+++ /dev/null
@@ -1,23 +0,0 @@
-openssl: avoid NULL pointer dereference in EVP_DigestInit_ex()
-
-We should avoid accessing the type pointer if it's NULL,
-this could happen if ctx->digest is not NULL.
-
-Upstream-Status: Submitted
-http://www.mail-archive.com/openssl-dev@openssl.org/msg32860.html
-
-Signed-off-by: Xufeng Zhang 

-Index: openssl-1.0.2h/crypto/evp/digest.c
-===
 openssl-1.0.2h.orig/crypto/evp/digest.c
-+++ openssl-1.0.2h/crypto/evp/digest.c
-@@ -211,7 +211,7 @@ int EVP_DigestInit_ex(EVP_MD_CTX *ctx, c
- type = ctx->digest;
- }
- #endif
--if (ctx->digest != type) {
-+if (type && (ctx->digest != type)) {
- if (ctx->digest && ctx->digest->ctx_size) {
- OPENSSL_free(ctx->md_data);
- ctx->md_data = NULL;
diff --git a/meta/recipes-connectivity/openssl/openssl_1.0.2k.bb 
b/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
similarity index 86%
rename from meta/recipes-connectivity/openssl/openssl_1.0.2k.bb
rename to meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
index 83d1a50..a2ef2ac 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.0.2k.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
@@ -7,7 +7,7 @@ DEPENDS += "cryptodev-linux"
 CFLAG += "-DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS"
 CFLAG_append_class-native = " -fPIC"
 
-LIC_FILES_CHKSUM = "file://LICENSE;md5=27ffa5d74bb5a337056c14b2ef93fbf6"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=057d9218c6180e1d9ee407572b2dd225"
 
 export DIRS = "crypto ssl apps engines"
 export OE_LDFLAGS="${LDFLAGS}"
@@ -32,8 +32,6 @@ SRC_URI += "file://find.pl;subdir=${BP}/util/ \
 file://debian1.0.2/version-script.patch \
 file://debian1.0.2/soname.patch \
 file://openssl_fix_for_x32.patch \
-file://fix-cipher-des-ede3-cfb1.patch \
-
file://openssl-avoid-NULL-pointer-dereference-in-EVP_DigestInit_ex.patch \
 file://openssl-fix-des.pod-error.patch \
 file://Makefiles-ptest.patch \
 file://ptest-deps.patch \
@@ -45,8 +43,8 @@ SRC_URI += "file://find.pl;subdir=${BP}/util/ \
 file://Use-SHA256-not-MD5-as-default-digest.patch \
 file://0001-Fix-build-with-clang-using-external-assembler.patch \
 "
-SRC_URI[md5sum] = "f965fc0bf01bf882b31314b61391ae65"
-SRC_URI[sha256sum] = 
"6b3977c61f2aedf0f96367dcfb5c6e578cf37e7b8d913b4ecb6643c3cb88d8c0"
+SRC_URI[md5sum] = "f85123cd390e864dfbe517e7616e6566"
+SRC_URI[sha256sum] = 

Re: [OE-core] [PATCH v3] runtime/dnf: Add new dnf test cases

[Alexander Kanavin]

On 06/08/2017 03:49 PM, Jose Perez Carranza wrote:


1. bitbake core-image-sato using shared states (I'm using local sstates)


When you do this, tmp/deploy/rpm should get re-populated with rpm 
packages, regardless of whether they're taken from sstate, or rebuilt.


Can you check why this does not happen on your side?

Alex
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH V2 2/3] kernel-fitimage: set replacementtype to vmlinux

[Anders Darander]

Hi,


* C.R. Guo  [170608 11:44]:
> Can you apply these patches?

No, I can't.


> -Original Message-
> From: C.R. Guo [mailto:chunrong@nxp.com] 
> Sent: Tuesday, June 06, 2017 4:05 PM

Besides, wait a little longer for feedback / merging, before you start
to send reminders. Everyone has lots of other things to do.

Cheers,
Anders

-- 
Anders Darander, Senior System Architect
ChargeStorm AB / eStorm AB
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3] runtime/dnf: Add new dnf test cases

[Jose Perez Carranza]

On 06/08/2017 06:31 AM, Alexander Kanavin wrote:

On 06/07/2017 08:31 PM, Jose Perez Carranza wrote:

When I tried this test image was built without using states hence
"busybox*.rpm" and "curl*.rpm"where present under
"tmp/deploy/rpm/core2_64" hence the repo i taking tha packages form
there to add them to repo and used on the image, but when using sstates
those packages are not being built so are not present on local build
directory. is there any way to force those packages to be built to have
them available on tmp/deploy/rpm/core2_64?


I can try to help if you provide steps to reproduce: if I merely do

rm -rf build/tmp
bitbake core-image-sato

then the build/tmp/deploy/rpm directory gets fully re-populated.


Sure I'm using this branch [1]

The steps that Iḿ following are :

1. bitbake core-image-sato using shared states (I'm using local sstates)
2. Add to the local.conf
INHERIT += "testimage"
TEST_SUITES += " ping ssh dnf"
3. bitbake core-image-sato -c testimage

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=jperez/dnf-updates


Alex



--
Saludos
José

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] python3: make readline, gdbm, and db support conditional.

[Alexander Kanavin]

On 06/08/2017 03:35 PM, Puustinen, Ismo wrote:


db should be taken out from this patch, it's no longer a gplv3
concern
now that we're providing db 5.x only :)


I can leave db out of the patch then. Is db 5.x a long-term solution or
do we have to change to 6.x at some point? Is anyone maintaining db 5
(security fixes especially) anymore?


We have no plans to change to 6.x.

5.x is what all major Linux distros are using, so we can take security 
fixes from them as needed. 6.x has been more or less rejected by open 
source community.


Alex
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] python3: make readline, gdbm, and db support conditional.

[Puustinen, Ismo]

On Thu, 2017-06-08 at 13:51 +0300, Alexander Kanavin wrote:
> On 06/07/2017 04:49 PM, Ismo Puustinen wrote:
> > The three libraries' (readline, gdbm, and db) licenses belong to
> > the
> > GPLv3 family. Add them to to PACKAGECONFIG so they can be switched
> > off
> > if the licensing doesn't allow using them. Python build system
> > autodetects the dependencies but doesn't allow them to be
> > explicitly
> > disabled, so just deal with the dependencies.
> 
> db should be taken out from this patch, it's no longer a gplv3
> concern 
> now that we're providing db 5.x only :)

I can leave db out of the patch then. Is db 5.x a long-term solution or
do we have to change to 6.x at some point? Is anyone maintaining db 5
(security fixes especially) anymore?

Ismo
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] ✗ patchtest: failure for gzip: add ptest (rev3)

[Patchwork]

== Series Details ==

Series: gzip: add ptest (rev3)
Revision: 3
URL   : https://patchwork.openembedded.org/series/5728/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue Added patch file is missing Upstream-Status in the header 
[test_upstream_status_presence] 
  Suggested fixAdd Upstream-Status:  to the header of 
meta/recipes-extended/gzip/files/0001-maint-use-returns_-rather-than-explicit-comparison-w.patch
 (possible values: Pending, Submitted, Accepted, Backport, Denied, 
Inappropriate)

* Issue A patch file has been added, but does not have a 
Signed-off-by tag [test_signed_off_by_presence] 
  Suggested fixSign off the added patch file 
(meta/recipes-extended/gzip/files/0001-maint-use-returns_-rather-than-explicit-comparison-w.patch)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3] gzip: add ptest

[Denys Zagorui]

Signed-off-by: Denys Zagorui 
---
 ...eturns_-rather-than-explicit-comparison-w.patch | 48 +++
 .../0002-gzip-port-zdiff-zless-to-Busybox.patch| 55 ++
 meta/recipes-extended/gzip/files/run-ptest |  6 +++
 meta/recipes-extended/gzip/gzip_1.8.bb | 20 +++-
 4 files changed, 128 insertions(+), 1 deletion(-)
 create mode 100644 
meta/recipes-extended/gzip/files/0001-maint-use-returns_-rather-than-explicit-comparison-w.patch
 create mode 100644 
meta/recipes-extended/gzip/files/0002-gzip-port-zdiff-zless-to-Busybox.patch
 create mode 100644 meta/recipes-extended/gzip/files/run-ptest

diff --git 
a/meta/recipes-extended/gzip/files/0001-maint-use-returns_-rather-than-explicit-comparison-w.patch
 
b/meta/recipes-extended/gzip/files/0001-maint-use-returns_-rather-than-explicit-comparison-w.patch
new file mode 100644
index 000..e5e8dca
--- /dev/null
+++ 
b/meta/recipes-extended/gzip/files/0001-maint-use-returns_-rather-than-explicit-comparison-w.patch
@@ -0,0 +1,48 @@
+From 9994edfb55cc58c02e4a4b7308a3f514fd971d96 Mon Sep 17 00:00:00 2001
+From: Jim Meyering 
+Date: Sun, 30 Oct 2016 12:16:42 -0700
+Subject: [PATCH 1/2] maint: use "returns_" rather than explicit comparison
+ with "$?"
+
+* tests/zdiff: Use "returns_ 1" rather than testing $? = 1.
+* tests/hufts: Likewise.
+* tests/timestamp: Likewise, but s/1/2/.
+
+[Denys Zagorui ] remove part of
+this patch that applied to tests/timestamp (not exist)
+---
+ tests/hufts | 3 +--
+ tests/zdiff | 3 +--
+ 2 files changed, 2 insertions(+), 4 deletions(-)
+
+diff --git a/tests/hufts b/tests/hufts
+index 5832a21..1793b89 100755
+--- a/tests/hufts
 b/tests/hufts
+@@ -23,8 +23,7 @@ printf '\n...: invalid compressed data--format violated\n' > 
exp \
+   || framework_failure_
+
+ fail=0
+-gzip -dc "$abs_srcdir/hufts-segv.gz" > out 2> err
+-test $? = 1 || fail=1
++returns_ 1 gzip -dc "$abs_srcdir/hufts-segv.gz" > out 2> err || fail=1
+
+ compare /dev/null out || fail=1
+
+diff --git a/tests/zdiff b/tests/zdiff
+index 0bb7c7d..eee47b3 100755
+--- a/tests/zdiff
 b/tests/zdiff
+@@ -32,8 +32,7 @@ cat < exp
+ EOF
+
+ fail=0
+-zdiff a.gz b.gz > out 2>&1
+-test $? = 1 || fail=1
++returns_ 1 zdiff a.gz b.gz > out 2>&1 || fail=1
+
+ compare exp out || fail=1
+
+--
+1.9.1
+
diff --git 
a/meta/recipes-extended/gzip/files/0002-gzip-port-zdiff-zless-to-Busybox.patch 
b/meta/recipes-extended/gzip/files/0002-gzip-port-zdiff-zless-to-Busybox.patch
new file mode 100644
index 000..6ae1809
--- /dev/null
+++ 
b/meta/recipes-extended/gzip/files/0002-gzip-port-zdiff-zless-to-Busybox.patch
@@ -0,0 +1,55 @@
+From 2c3577657e79445493211d7f58a6038b672e4f62 Mon Sep 17 00:00:00 2001
+From: Paul Eggert 
+Date: Mon, 13 Mar 2017 16:35:55 -0700
+Subject: [PATCH 2/2] gzip: port zdiff, zless to Busybox
+
+Problem reported by Denys Zagorui (Bug#26088).
+* tests/zdiff: Check that diff uses POSIX-format output.
+* zless.in (less_version): Don't exit merely because 'less -V'
+fails; instead, assume 'less' is compatible with an old version of
+the original 'less'.  Busybox 'less -V' fails, but apparently its
+'less' works anyway somehow.
+---
+ tests/zdiff | 4 +++-
+ zless.in| 2 +-
+ 2 files changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/tests/zdiff b/tests/zdiff
+index eee47b3..944e4c4 100755
+--- a/tests/zdiff
 b/tests/zdiff
+@@ -22,7 +22,6 @@
+
+ echo a > a || framework_failure_
+ echo b > b || framework_failure_
+-gzip a b || framework_failure_
+
+ cat < exp
+ 1c1
+@@ -31,7 +30,10 @@ cat < exp
+ > b
+ EOF
+
++diff a b | diff exp - || skip_ "diff output format is incompatible with POSIX"
++
+ fail=0
++gzip a b || fail=1
+ returns_ 1 zdiff a.gz b.gz > out 2>&1 || fail=1
+
+ compare exp out || fail=1
+diff --git a/zless.in b/zless.in
+index e634af6..9759ae6 100644
+--- a/zless.in
 b/zless.in
+@@ -47,7 +47,7 @@ if test "${LESSMETACHARS+set}" != set; then
+   export LESSMETACHARS
+ fi
+
+-less_version=`less -V` || exit
++less_version=`less -V 2>/dev/null`
+ case $less_version in
+ less' '45[1-9]* | \
+ less' '4[6-9][0-9]* | \
+--
+1.9.1
+
diff --git a/meta/recipes-extended/gzip/files/run-ptest 
b/meta/recipes-extended/gzip/files/run-ptest
new file mode 100644
index 000..cf7c649
--- /dev/null
+++ b/meta/recipes-extended/gzip/files/run-ptest
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+cd src/tests
+
+make check
+
diff --git a/meta/recipes-extended/gzip/gzip_1.8.bb 
b/meta/recipes-extended/gzip/gzip_1.8.bb
index 11be846..ab467be 100644
--- a/meta/recipes-extended/gzip/gzip_1.8.bb
+++ b/meta/recipes-extended/gzip/gzip_1.8.bb
@@ -2,7 +2,10 @@ require gzip.inc
 
 LICENSE = "GPLv3+"
 
-SRC_URI = "${GNU_MIRROR}/gzip/${BP}.tar.gz"
+SRC_URI = "${GNU_MIRROR}/gzip/${BP}.tar.gz \
+
file://0001-maint-use-returns_-rather-than-explicit-comparison-w.patch \
+file://0002-gzip-port-zdiff-zless-to-Busybox.patch \
+

Re: [OE-core] [master][pyro][PATCH] image-vm: Avoid use of fold, tac and paste commands for DISK_SIGNATURE

[Burton, Ross]

On 8 June 2017 at 12:21, Jonathan Liu  wrote:

> For example:
> $ echo $(false)
>
> $ echo $?
> 0
>

I stand corrected.  Thanks, sh.

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3] runtime/dnf: Add new dnf test cases

[Alexander Kanavin]

On 06/07/2017 08:31 PM, Jose Perez Carranza wrote:

When I tried this test image was built without using states hence
"busybox*.rpm" and "curl*.rpm"where present under
"tmp/deploy/rpm/core2_64" hence the repo i taking tha packages form
there to add them to repo and used on the image, but when using sstates
those packages are not being built so are not present on local build
directory. is there any way to force those packages to be built to have
them available on tmp/deploy/rpm/core2_64?


I can try to help if you provide steps to reproduce: if I merely do

rm -rf build/tmp
bitbake core-image-sato

then the build/tmp/deploy/rpm directory gets fully re-populated.


Alex

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [master][pyro][PATCH] image-vm: Avoid use of fold, tac and paste commands for DISK_SIGNATURE

[Jonathan Liu]

Hi Ross,

It is the shell not awk.

For example:
$ echo $(false)

$ echo $?
0

Regards,
Jonathan

On 8 June 2017 at 21:18, Burton, Ross  wrote:
>
> On 8 June 2017 at 12:07, Jonathan Liu  wrote:
>>
>> These commands are not whitelisted by the HOSTTOOLS variable which
>> silently prevents the MBR disk signature from being written to the
>> image.
>
>
> Silently, ouch.  Thanks, awk.
>
> Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [master][pyro][PATCH] image-vm: Avoid use of fold, tac and paste commands for DISK_SIGNATURE

[Burton, Ross]

On 8 June 2017 at 12:07, Jonathan Liu  wrote:

> These commands are not whitelisted by the HOSTTOOLS variable which
> silently prevents the MBR disk signature from being written to the
> image.
>

Silently, ouch.  Thanks, awk.

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] libxml2: Fix CVE-2017-8872

[Jussi Kukkonen]

On 7 June 2017 at 11:51, Fan Xin  wrote:
>
> CVE: CVE-2017-8872
> The htmlParseTryOrFinish function in HTMLparser.c in libxml2 2.9.4 allows
attackers
> to cause a denial of service (buffer over-read) or information disclosure.
>
> External References:
> https://bugzilla.gnome.org/show_bug.cgi?id=77520


This should be
https://bugzilla.gnome.org/show_bug.cgi?id=775200

I have the same question that was asked in the upstream bug comment 6 about
two weeks ago: The patch doesn't seem to have any effect (because the goto
will happen anyway since 'avail' is 0), am I missing something?

Jussi


>
>
> Signed-off-by: Fan Xin 
> ---
>  .../libxml/libxml2/libxml2-CVE-2017-8872.patch | 23
++
>  meta/recipes-core/libxml/libxml2_2.9.4.bb  |  1 +
>  2 files changed, 24 insertions(+)
>  create mode 100644
meta/recipes-core/libxml/libxml2/libxml2-CVE-2017-8872.patch
>
> diff --git a/meta/recipes-core/libxml/libxml2/libxml2-CVE-2017-8872.patch
b/meta/recipes-core/libxml/libxml2/libxml2-CVE-2017-8872.patch
> new file mode 100644
> index 000..df05e06
> --- /dev/null
> +++ b/meta/recipes-core/libxml/libxml2/libxml2-CVE-2017-8872.patch
> @@ -0,0 +1,23 @@
> +libxml2-2.9.4: Fix CVE-2017-8872
> +
> +Bug 775200 - (CVE-2017-8872) global-buffer-overflow in
htmlParseTryOrFinish (HTMLparser.c:5403)
> + - [https://bugzilla.gnome.org/show_bug.cgi?id=775200]
> +
> +CVE: CVE-2017-8872
> +Upstream-Status: Submitted
> +
> +Signed-off-by: Fan Xin 
> +
> +Index: libxml2-2.9.4/HTMLparser.c
> +===
> +--- libxml2-2.9.4.orig/HTMLparser.c
>  libxml2-2.9.4/HTMLparser.c
> +@@ -5396,6 +5396,8 @@ htmlParseTryOrFinish(htmlParserCtxtPtr c
> +   ctxt->instate = XML_PARSER_EOF;
> +   if ((ctxt->sax) && (ctxt->sax->endDocument != NULL))
> +   ctxt->sax->endDocument(ctxt->userData);
> ++
> ++  goto done;
> +   }
> +   }
> + if (avail < 1)
> diff --git a/meta/recipes-core/libxml/libxml2_2.9.4.bb
b/meta/recipes-core/libxml/libxml2_2.9.4.bb
> index ea0d3b8..0b4cbca 100644
> --- a/meta/recipes-core/libxml/libxml2_2.9.4.bb
> +++ b/meta/recipes-core/libxml/libxml2_2.9.4.bb
> @@ -24,6 +24,7 @@ SRC_URI = "
ftp://xmlsoft.org/libxml2/libxml2-${PV}.tar.gz;name=libtar \
> file://libxml2-CVE-2016-4658.patch \
> file://libxml2-fix_NULL_pointer_derefs.patch \
> file://CVE-2016-9318.patch \
> +  file://libxml2-CVE-2017-8872.patch \
>"
>
>  SRC_URI[libtar.md5sum] = "ae249165c173b1ff386ee8ad676815f5"
> --
> 1.9.1
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe-core] DISK_SIGNATURE no longer functional

[Jonathan Liu]

Hi Michael,

I have sent a patch to the mailing list and CC-ed you.
Let me know if it fixes the issue.

Regards,
Jonathan

On 8 June 2017 at 02:12, Davis, Michael  wrote:
> I tried using  ROOT_VM = "root=PARTUUID=${DISK_SIGNATURE}-02"
>
> The resulting PARTUUID in syslinux append line still doesn’t match the one
> on the actual disk.
>
>
>
>
>
> From: Jonathan Liu [mailto:net...@gmail.com]
> Sent: Tuesday, June 06, 2017 5:01 PM
>
>
> To: Davis, Michael
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [oe-core] DISK_SIGNATURE no longer functional
>
>
>
> Hi Michael,
>
>
>
> See https://patchwork.openembedded.org/patch/128565/. SYSLINUX_ROOT was
> renamed to ROOT_VM in Krogoth.
>
>
>
> Regards,
>
> Jonathan
>
>
>
> On Tue., 6 Jun. 2017 at 11:29 pm, Davis, Michael 
> wrote:
>
> Pyro.
>
> Guessing something with all the new wic additions killed it.
>
>
> -Original Message-
> From: Jonathan Liu [mailto:net...@gmail.com]
> Sent: Monday, June 05, 2017 7:17 PM
> To: Davis, Michael
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [oe-core] DISK_SIGNATURE no longer functional
>
> Hi Michael,
>
> On 21 May 2017 at 03:16, Davis, Michael  wrote:
>> I am trying to create a generic MSDOS PC_BIOS hdddirect image with a root
>> partition set by the DISK_SIGNATURE.
>>
>> SYSLINUX_ROOT = "root=PARTUUID=${DISK_SIGNATURE}-02"
>>
>>
>>
>> However the signature generated in syslinux.cfg and the one on the disk
>> don’t match.
>>
>> I tried defining my own signature and the one in syslinux changes, but the
>> one on the disk doesn’t match.
>>
>>
>>
>> root=PARTUUID=22aaf83b-02
>>
>>
>>
>> NAME   PARTUUID
>>
>> sdd
>>
>> ├─sdd2 341e84a7-02
>>
>> ├─sdd3 341e84a7-03
>>
>> └─sdd1 341e84a7-01
>
> Which release did DISK_SIGNATURE stop working?
>
> Regards,
> Jonathan
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [master][pyro][PATCH] image-vm: Avoid use of fold, tac and paste commands for DISK_SIGNATURE

[Jonathan Liu]

These commands are not whitelisted by the HOSTTOOLS variable which
silently prevents the MBR disk signature from being written to the
image.

Reported-by: Michael Davis 
Signed-off-by: Jonathan Liu 
---
 meta/classes/image-vm.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image-vm.bbclass b/meta/classes/image-vm.bbclass
index 35c9244e9b..98bd92000e 100644
--- a/meta/classes/image-vm.bbclass
+++ b/meta/classes/image-vm.bbclass
@@ -93,7 +93,7 @@ build_boot_dd() {
 
parted $IMAGE print
 
-   awk "BEGIN { printf \"$(echo ${DISK_SIGNATURE} | fold -w 2 | tac | 
paste -sd '' | sed 's/\(..\)/\\x&/g')\" }" | \
+   awk "BEGIN { printf \"$(echo ${DISK_SIGNATURE} | sed 
's/\(..\)\(..\)\(..\)\(..\)/\\x\4\\x\3\\x\2\\x\1/')\" }" | \
dd of=$IMAGE bs=1 seek=440 conv=notrunc
 
OFFSET=`expr $END2 / 512`
-- 
2.12.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] openssl: Upgrade 1.0.2k -> 1.0.2l

[Jussi Kukkonen]

On 8 June 2017 at 11:18, Changhyeok Bae  wrote:

> Patch fix-cipher-des-ede3-cfb1.patch was removed as it is included
> in this release.
>
> Signed-off-by: Changhyeok Bae 
> ---
>  .../openssl/openssl/fix-cipher-des-ede3-cfb1.patch  | 21
> -
>  .../{openssl_1.0.2k.bb => openssl_1.0.2l.bb}|  7 +++
>  2 files changed, 3 insertions(+), 25 deletions(-)
>  delete mode 100644 meta/recipes-connectivity/openssl/openssl/fix-cipher-
> des-ede3-cfb1.patch
>  rename meta/recipes-connectivity/openssl/{openssl_1.0.2k.bb =>
> openssl_1.0.2l.bb} (90%)
>
> diff --git 
> a/meta/recipes-connectivity/openssl/openssl/fix-cipher-des-ede3-cfb1.patch
> b/meta/recipes-connectivity/openssl/openssl/fix-cipher-des-ede3-cfb1.patch
> deleted file mode 100644
> index 2a318a4..000
> --- a/meta/recipes-connectivity/openssl/openssl/fix-cipher-
> des-ede3-cfb1.patch
> +++ /dev/null
> @@ -1,21 +0,0 @@
> -Upstream-Status: Submitted
> -
> -This patch adds the fix for one of the ciphers used in openssl, namely
> -the cipher des-ede3-cfb1. Complete bug log and patch is present here:
> -http://rt.openssl.org/Ticket/Display.html?id=2867
> -
> -Signed-off-by: Muhammad Shakeel 
> -
> -Index: openssl-1.0.2/crypto/evp/e_des3.c
> -===
>  openssl-1.0.2.orig/crypto/evp/e_des3.c
> -+++ openssl-1.0.2/crypto/evp/e_des3.c
> -@@ -211,7 +211,7 @@ static int des_ede3_cfb1_cipher(EVP_CIPH
> - size_t n;
> - unsigned char c[1], d[1];
> -
> --for (n = 0; n < inl; ++n) {
> -+for (n = 0; n * 8 < inl; ++n) {
> - c[0] = (in[n / 8] & (1 << (7 - n % 8))) ? 0x80 : 0;
> - DES_ede3_cfb_encrypt(c, d, 1, 1,
> -  (ctx)->ks1, (ctx)->ks2,
> diff --git a/meta/recipes-connectivity/openssl/openssl_1.0.2k.bb
> b/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
> similarity index 90%
> rename from meta/recipes-connectivity/openssl/openssl_1.0.2k.bb
> rename to meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
> index 83d1a50..6390746 100644
> --- a/meta/recipes-connectivity/openssl/openssl_1.0.2k.bb
> +++ b/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
> @@ -7,7 +7,7 @@ DEPENDS += "cryptodev-linux"
>  CFLAG += "-DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS"
>  CFLAG_append_class-native = " -fPIC"
>
> -LIC_FILES_CHKSUM = "file://LICENSE;md5=27ffa5d74bb5a337056c14b2ef93fbf6"
> +LIC_FILES_CHKSUM = "file://LICENSE;md5=057d9218c6180e1d9ee407572b2dd225"
>
>  export DIRS = "crypto ssl apps engines"
>  export OE_LDFLAGS="${LDFLAGS}"
> @@ -32,7 +32,6 @@ SRC_URI += "file://find.pl;subdir=${BP}/util/ \
>  file://debian1.0.2/version-script.patch \
>  file://debian1.0.2/soname.patch \
>  file://openssl_fix_for_x32.patch \
> -file://fix-cipher-des-ede3-cfb1.patch \
>  
> file://openssl-avoid-NULL-pointer-dereference-in-EVP_DigestInit_ex.patch
> \
>

I think the patch mentioned above has been obsolete for many releases as
well: see
https://github.com/openssl/openssl/commit/ff2459b91877959bc1a3a8a927c6b5f473530f86

Cheers,
  Jussi


 file://openssl-fix-des.pod-error.patch \
>  file://Makefiles-ptest.patch \
> @@ -45,8 +44,8 @@ SRC_URI += "file://find.pl;subdir=${BP}/util/ \
>  file://Use-SHA256-not-MD5-as-default-digest.patch \
>  file://0001-Fix-build-with-clang-using-external-assembler.patch
> \
>  "
> -SRC_URI[md5sum] = "f965fc0bf01bf882b31314b61391ae65"
> -SRC_URI[sha256sum] = "6b3977c61f2aedf0f96367dcfb5c6e
> 578cf37e7b8d913b4ecb6643c3cb88d8c0"
> +SRC_URI[md5sum] = "f85123cd390e864dfbe517e7616e6566"
> +SRC_URI[sha256sum] = "ce07195b659e75f4e1db4355286007
> 0061f156a98bb37b672b101ba6e3ddf30c"
>
>  PACKAGES =+ "${PN}-engines"
>  FILES_${PN}-engines = "${libdir}/ssl/engines/*.so ${libdir}/engines"
> --
> 2.7.4
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core