Re: [OE-core] [PATCH 2/2] linux-yocto: Add scsi_debug module when ptest is in DISTRO_FEATURES

[Mariano Lopez]

On 6/15/19 8:02 PM, Bruce Ashfield wrote:

On Sat, Jun 15, 2019 at 3:47 PM Mariano Lopez
 wrote:


On 6/15/19 10:15 AM, Richard Purdie wrote:

On Thu, 2019-06-13 at 22:32 -0500, Mariano López wrote:

util-linux ptest requires the scsi_debug module to perform eject/mount
tests. This will conditionally add scsi_debug module when ptest is in
DISTRO_FEATURES.

This doesn't include linux-yocto-tiny because the resulting image will
be too big and do_image would complain about this.

[YOCTO #13301]

Signed-off-by: Mariano López 
---
   meta/recipes-kernel/linux/linux-yocto-rt_4.19.bb | 1 +
   meta/recipes-kernel/linux/linux-yocto-rt_5.0.bb  | 1 +
   meta/recipes-kernel/linux/linux-yocto_4.19.bb| 1 +
   meta/recipes-kernel/linux/linux-yocto_5.0.bb | 1 +
   4 files changed, 4 insertions(+)

diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_4.19.bb 
b/meta/recipes-kernel/linux/linux-yocto-rt_4.19.bb
index 6604bdf90c..0836dc7ea0 100644
--- a/meta/recipes-kernel/linux/linux-yocto-rt_4.19.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_4.19.bb
@@ -41,3 +41,4 @@ KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
   KERNEL_FEATURES_append_qemuall=" cfg/virtio.scc"
   KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
   KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
+KERNEL_FEATURES_append = "${@bb.utils.contains("DISTRO_FEATURES", "ptest", " 
features/scsi/scsi-debug.scc", "" ,d)}"
diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_5.0.bb 
b/meta/recipes-kernel/linux/linux-yocto-rt_5.0.bb
index 1fe28b1bf0..b5e415f935 100644
--- a/meta/recipes-kernel/linux/linux-yocto-rt_5.0.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_5.0.bb
@@ -41,3 +41,4 @@ KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
   KERNEL_FEATURES_append_qemuall=" cfg/virtio.scc"
   KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
   KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
+KERNEL_FEATURES_append = "${@bb.utils.contains("DISTRO_FEATURES", "ptest", " 
features/scsi/scsi-debug.scc", "" ,d)}"
diff --git a/meta/recipes-kernel/linux/linux-yocto_4.19.bb 
b/meta/recipes-kernel/linux/linux-yocto_4.19.bb
index a5fdafe6b2..cda4ecf89e 100644
--- a/meta/recipes-kernel/linux/linux-yocto_4.19.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_4.19.bb
@@ -47,3 +47,4 @@ KERNEL_FEATURES_append_qemuall=" cfg/virtio.scc"
   KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
   KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
   KERNEL_FEATURES_append = " ${@bb.utils.contains("TUNE_FEATURES", "mx32", " cfg/x32.scc", 
"" ,d)}"
+KERNEL_FEATURES_append = " ${@bb.utils.contains("DISTRO_FEATURES", "ptest", " 
features/scsi/scsi-debug.scc", "" ,d)}"
diff --git a/meta/recipes-kernel/linux/linux-yocto_5.0.bb 
b/meta/recipes-kernel/linux/linux-yocto_5.0.bb
index da795d97aa..8aec315084 100644
--- a/meta/recipes-kernel/linux/linux-yocto_5.0.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_5.0.bb
@@ -50,3 +50,4 @@ KERNEL_FEATURES_append_qemuall=" cfg/virtio.scc"
   KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
   KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
   KERNEL_FEATURES_append = " ${@bb.utils.contains("TUNE_FEATURES", "mx32", " cfg/x32.scc", 
"" ,d)}"
+KERNEL_FEATURES_append = " ${@bb.utils.contains("DISTRO_FEATURES", "ptest", " 
features/scsi/scsi-debug.scc", "" ,d)}"

I merged this but have just realised its causing warnings:

https://autobuilder.yoctoproject.org/typhoon/#/builders/93/builds/31

which suggest its not quite doing what we hope/need? :/

Cheers,

Richard

Indeed, it seems the configuration is not applied for the qemuarm64
MACHINE:(


It would be applied, but it may be missing some dependency on that
machine and is hence not enabled in the final kernel .config.

Bruce


SCSI support is not enabled by default for ARM architectures, I just 
sent the patch for this:

http://lists.openembedded.org/pipermail/openembedded-core/2019-June/283775.html


Thanks!
Mariano




Let me look into this and send another patch

Thanks,
Mairano




--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] linux-yocto: Add scsi_debug module when ptest is in DISTRO_FEATURES

[Mariano Lopez]

On 6/15/19 10:15 AM, Richard Purdie wrote:

On Thu, 2019-06-13 at 22:32 -0500, Mariano López wrote:

util-linux ptest requires the scsi_debug module to perform eject/mount
tests. This will conditionally add scsi_debug module when ptest is in
DISTRO_FEATURES.

This doesn't include linux-yocto-tiny because the resulting image will
be too big and do_image would complain about this.

[YOCTO #13301]

Signed-off-by: Mariano López 
---
  meta/recipes-kernel/linux/linux-yocto-rt_4.19.bb | 1 +
  meta/recipes-kernel/linux/linux-yocto-rt_5.0.bb  | 1 +
  meta/recipes-kernel/linux/linux-yocto_4.19.bb| 1 +
  meta/recipes-kernel/linux/linux-yocto_5.0.bb | 1 +
  4 files changed, 4 insertions(+)

diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_4.19.bb 
b/meta/recipes-kernel/linux/linux-yocto-rt_4.19.bb
index 6604bdf90c..0836dc7ea0 100644
--- a/meta/recipes-kernel/linux/linux-yocto-rt_4.19.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_4.19.bb
@@ -41,3 +41,4 @@ KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
  KERNEL_FEATURES_append_qemuall=" cfg/virtio.scc"
  KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
  KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
+KERNEL_FEATURES_append = "${@bb.utils.contains("DISTRO_FEATURES", "ptest", " 
features/scsi/scsi-debug.scc", "" ,d)}"
diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_5.0.bb 
b/meta/recipes-kernel/linux/linux-yocto-rt_5.0.bb
index 1fe28b1bf0..b5e415f935 100644
--- a/meta/recipes-kernel/linux/linux-yocto-rt_5.0.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_5.0.bb
@@ -41,3 +41,4 @@ KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
  KERNEL_FEATURES_append_qemuall=" cfg/virtio.scc"
  KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
  KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
+KERNEL_FEATURES_append = "${@bb.utils.contains("DISTRO_FEATURES", "ptest", " 
features/scsi/scsi-debug.scc", "" ,d)}"
diff --git a/meta/recipes-kernel/linux/linux-yocto_4.19.bb 
b/meta/recipes-kernel/linux/linux-yocto_4.19.bb
index a5fdafe6b2..cda4ecf89e 100644
--- a/meta/recipes-kernel/linux/linux-yocto_4.19.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_4.19.bb
@@ -47,3 +47,4 @@ KERNEL_FEATURES_append_qemuall=" cfg/virtio.scc"
  KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
  KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
  KERNEL_FEATURES_append = " ${@bb.utils.contains("TUNE_FEATURES", "mx32", " cfg/x32.scc", 
"" ,d)}"
+KERNEL_FEATURES_append = " ${@bb.utils.contains("DISTRO_FEATURES", "ptest", " 
features/scsi/scsi-debug.scc", "" ,d)}"
diff --git a/meta/recipes-kernel/linux/linux-yocto_5.0.bb 
b/meta/recipes-kernel/linux/linux-yocto_5.0.bb
index da795d97aa..8aec315084 100644
--- a/meta/recipes-kernel/linux/linux-yocto_5.0.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_5.0.bb
@@ -50,3 +50,4 @@ KERNEL_FEATURES_append_qemuall=" cfg/virtio.scc"
  KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
  KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
  KERNEL_FEATURES_append = " ${@bb.utils.contains("TUNE_FEATURES", "mx32", " cfg/x32.scc", 
"" ,d)}"
+KERNEL_FEATURES_append = " ${@bb.utils.contains("DISTRO_FEATURES", "ptest", " 
features/scsi/scsi-debug.scc", "" ,d)}"

I merged this but have just realised its causing warnings:

https://autobuilder.yoctoproject.org/typhoon/#/builders/93/builds/31

which suggest its not quite doing what we hope/need? :/

Cheers,

Richard


Indeed, it seems the configuration is not applied for the qemuarm64 
MACHINE:(


Let me look into this and send another patch

Thanks,
Mairano

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v4 2/4] ptest.bbclass: Add feature to populate a binary directory

[Mariano Lopez]

On 4/11/19 3:11 PM, Richard Purdie wrote:

On Tue, 2019-04-09 at 00:44 -0500, Mariano Lopez wrote:

This adds the functionality to create a binary directory within
PTEST_PATH directory. This directory will be populated with
symlinks pointing to the binaries installed by the package and
then renamed by update-alternatives. This way the ptest only needs
to source this binary directory in order to use the expected
binaries.

To enable this feature just add PTEST_BINDIR = "1" to the recipe.

[YOCTO #12597]
[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
  meta/classes/ptest.bbclass | 32 
  1 file changed, 32 insertions(+)

diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index 97865c9338..e87a9659cb 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -65,6 +65,38 @@ do_install_ptest_base() {
  done
  }
  
+PTEST_BINDIR_PKGD_PATH = "${PKGD}${PTEST_PATH}/bin"

+
+# This function needs to run after apply_update_alternative_renames
because the
+# aforementioned function will update the ALTERNATIVE_LINK_NAME
flag. Append is
+# used here to make this function to run as late as possible.
+PACKAGE_PREPROCESS_FUNCS_append = "${@bb.utils.contains('PTEST_BINDI
R', '1', \
+bb.utils.contains('PTEST_ENABLED
', '1', ' ptest_update_alternatives', '', d), '', d)}"
+
+python ptest_update_alternatives() {
+"""
+This function will generate the symlinks in the
PTEST_BINDIR_PKGD_PATH
+to match the renamed binaries by update-alternatives.
+"""
+
+if not bb.data.inherits_class('update-alternatives', d) \
+   or not update_alternatives_enabled(d):
+return
+
+bb.note("Generating symlinks for ptest")
+bin_paths = { os.environ["bindir"], os.environ["base_bindir"],

I'm going to merge this but is there a good reason you're using
os.environ[] here? If not we should merge a small fixup to use
d.getVar() since ultimately we don't really want these things in the
environment if we can help it.

Cheers,

Richard


Sorry about that, I just sent the fix for this

Mariano

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] ptest.bbclass: Use d.getVar instead of os.environ

[Mariano Lopez]

[YOCTO #12597]
[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/classes/ptest.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index e87a9659cb..936bf82736 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -84,8 +84,8 @@ python ptest_update_alternatives() {
 return
 
 bb.note("Generating symlinks for ptest")
-bin_paths = { os.environ["bindir"], os.environ["base_bindir"],
-  os.environ["sbindir"], os.environ["base_sbindir"] }
+bin_paths = { d.getVar("bindir"), d.getVar("base_bindir"),
+   d.getVar("sbindir"), d.getVar("base_sbindir") }
 ptest_bindir = d.getVar("PTEST_BINDIR_PKGD_PATH")
 os.mkdir(ptest_bindir)
 for pkg in (d.getVar('PACKAGES') or "").split():
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v4 3/4] util-linux: Use PTEST binary directory

[Mariano Lopez]

Some binaries generated by util-linux will be replaced by core-utils
in the final image by update-alternatives, so use a dedicated directory
with symlinks to avoid using a binary generated by another package.

This will solve the issue with the ptest runner timing out when
running the kill ptests for util-linux.

[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/recipes-core/util-linux/util-linux.inc   | 5 +++--
 meta/recipes-core/util-linux/util-linux/run-ptest | 4 
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/util-linux/util-linux.inc 
b/meta/recipes-core/util-linux/util-linux.inc
index 18c3af240e..a67318e84e 100644
--- a/meta/recipes-core/util-linux/util-linux.inc
+++ b/meta/recipes-core/util-linux/util-linux.inc
@@ -284,6 +284,7 @@ ALTERNATIVE_LINK_NAME[wall.1] = "${mandir}/man1/wall.1"
 
 BBCLASSEXTEND = "native nativesdk"
 
+PTEST_BINDIR = "1"
 do_compile_ptest() {
 oe_runmake buildtest-TESTS
 }
@@ -312,8 +313,8 @@ do_install_ptest() {
 
'/^\tif[[:space:]]\[[[:space:]]![[:space:]]-x[[:space:]]"$1"/s|$1|`which $1 
2>/dev/null`|g' \
  ${D}${PTEST_PATH}/tests/functions.sh
 
-# "kill -L" behaves differently than "/bin/kill -L" so we need an 
additional fix
+# Running "kill" without the the complete path would use the shell's 
built-in kill
 sed -i -e \
- '/^TS_CMD_KILL/ s|kill|/bin/kill|g' \
+ '/^TS_CMD_KILL/ s|kill|${PTEST_PATH}/bin/kill|g' \
  ${D}${PTEST_PATH}/tests/commands.sh
 }
diff --git a/meta/recipes-core/util-linux/util-linux/run-ptest 
b/meta/recipes-core/util-linux/util-linux/run-ptest
index fbc2f9b56a..2178ab8fef 100644
--- a/meta/recipes-core/util-linux/util-linux/run-ptest
+++ b/meta/recipes-core/util-linux/util-linux/run-ptest
@@ -1,5 +1,9 @@
 #!/bin/sh
 
+current_path=$(readlink -f $0)
+export bindir=$(dirname $current_path)
+export PATH=$bindir/bin:$PATH
+
 cd tests || exit 1  
 
 comps=$(find ts/ -type f -perm -111 -regex ".*/[^\.~]*" |  sort)
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v4 2/4] ptest.bbclass: Add feature to populate a binary directory

[Mariano Lopez]

This adds the functionality to create a binary directory within
PTEST_PATH directory. This directory will be populated with
symlinks pointing to the binaries installed by the package and
then renamed by update-alternatives. This way the ptest only needs
to source this binary directory in order to use the expected
binaries.

To enable this feature just add PTEST_BINDIR = "1" to the recipe.

[YOCTO #12597]
[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/classes/ptest.bbclass | 32 
 1 file changed, 32 insertions(+)

diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index 97865c9338..e87a9659cb 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -65,6 +65,38 @@ do_install_ptest_base() {
 done
 }
 
+PTEST_BINDIR_PKGD_PATH = "${PKGD}${PTEST_PATH}/bin"
+
+# This function needs to run after apply_update_alternative_renames because the
+# aforementioned function will update the ALTERNATIVE_LINK_NAME flag. Append is
+# used here to make this function to run as late as possible.
+PACKAGE_PREPROCESS_FUNCS_append = "${@bb.utils.contains('PTEST_BINDIR', '1', \
+bb.utils.contains('PTEST_ENABLED', '1', ' 
ptest_update_alternatives', '', d), '', d)}"
+
+python ptest_update_alternatives() {
+"""
+This function will generate the symlinks in the PTEST_BINDIR_PKGD_PATH
+to match the renamed binaries by update-alternatives.
+"""
+
+if not bb.data.inherits_class('update-alternatives', d) \
+   or not update_alternatives_enabled(d):
+return
+
+bb.note("Generating symlinks for ptest")
+bin_paths = { os.environ["bindir"], os.environ["base_bindir"],
+  os.environ["sbindir"], os.environ["base_sbindir"] }
+ptest_bindir = d.getVar("PTEST_BINDIR_PKGD_PATH")
+os.mkdir(ptest_bindir)
+for pkg in (d.getVar('PACKAGES') or "").split():
+alternatives = update_alternatives_alt_targets(d, pkg)
+for alt_name, alt_link, alt_target, _ in alternatives:
+# Some alternatives are for man pages,
+# check if the alternative is in PATH
+if os.path.dirname(alt_link) in bin_paths:
+os.symlink(alt_target, os.path.join(ptest_bindir, alt_name))
+}
+
 do_configure_ptest_base[dirs] = "${B}"
 do_compile_ptest_base[dirs] = "${B}"
 do_install_ptest_base[dirs] = "${B}"
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v4 4/4] busybox: Use PTEST binary directory

[Mariano Lopez]

This will generate the symlinks in the ptest binary directory using the
ptest class functionality instead of generating them manually. Because
the ptest class uses update-alternatives to get the metadata for the
symlinks it will respect the use of BUSYBOX_SPLIT_SUID automatically.

[YOCTO #12597]

Signed-off-by: Mariano Lopez 
---
 meta/recipes-core/busybox/busybox.inc | 16 ++--
 1 file changed, 2 insertions(+), 14 deletions(-)

diff --git a/meta/recipes-core/busybox/busybox.inc 
b/meta/recipes-core/busybox/busybox.inc
index c9d25ff1ca..174ce5a8c0 100644
--- a/meta/recipes-core/busybox/busybox.inc
+++ b/meta/recipes-core/busybox/busybox.inc
@@ -347,24 +347,12 @@ do_install () {
fi
 }
 
+PTEST_BINDIR = "1"
+
 do_install_ptest () {
cp -r ${B}/testsuite ${D}${PTEST_PATH}/
cp ${B}/.config  ${D}${PTEST_PATH}/
ln -s /bin/busybox   ${D}${PTEST_PATH}/busybox
-
-   mkdir ${D}${PTEST_PATH}/bin
-   if [ "${BUSYBOX_SPLIT_SUID}" = "1" ]; then
-   while read link; do
-   ln -s ${base_bindir}/busybox.suid 
${D}${PTEST_PATH}/bin/$(basename $link)
-   done <${D}${sysconfdir}/busybox.links.suid
-   while read link; do
-   ln -s ${base_bindir}/busybox.nosuid 
${D}${PTEST_PATH}/bin/$(basename $link)
-   done <${D}${sysconfdir}/busybox.links.nosuid
-   else
-   while read link; do
-   ln -s ${base_bindir}/busybox 
${D}${PTEST_PATH}/bin/$(basename $link)
-   done <${D}${sysconfdir}/busybox.links
-   fi
 }
 
 inherit update-alternatives
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v4 1/4] update-alternatives.bbclass: Add function to get metadata

[Mariano Lopez]

This adds update_alternatives_alt_targets function to get the metadata
for a package. This is for code reuse because the metadata would help
other classes that needs to be aware of how update-alternatives modify
the final package.

[YOCTO #12597]
[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/classes/update-alternatives.bbclass | 70 +++-
 1 file changed, 45 insertions(+), 25 deletions(-)

diff --git a/meta/classes/update-alternatives.bbclass 
b/meta/classes/update-alternatives.bbclass
index 537e85d9a3..b702e77ee5 100644
--- a/meta/classes/update-alternatives.bbclass
+++ b/meta/classes/update-alternatives.bbclass
@@ -216,42 +216,62 @@ python apply_update_alternative_renames () {
 update_files(alt_target, alt_target_rename, pkg, d)
 }
 
+def update_alternatives_alt_targets(d, pkg):
+"""
+Returns the update-alternatives metadata for a package.
+
+The returned format is a list of tuples where the tuple contains:
+alt_name: The binary name
+alt_link: The path for the binary (Shared by different packages)
+alt_target:   The path for the renamed binary (Unique per package)
+alt_priority: The priority of the alt_target
+
+All the alt_targets will be installed into the sysroot. The alt_link is
+a symlink pointing to the alt_target with the highest priority.
+"""
+
+pn = d.getVar('BPN')
+pkgdest = d.getVar('PKGD')
+updates = list()
+for alt_name in (d.getVar('ALTERNATIVE_%s' % pkg) or "").split():
+alt_link = d.getVarFlag('ALTERNATIVE_LINK_NAME', alt_name)
+alt_target   = d.getVarFlag('ALTERNATIVE_TARGET_%s' % pkg, alt_name) 
or \
+   d.getVarFlag('ALTERNATIVE_TARGET', alt_name) or \
+   d.getVar('ALTERNATIVE_TARGET_%s' % pkg) or \
+   d.getVar('ALTERNATIVE_TARGET') or \
+   alt_link
+alt_priority = d.getVarFlag('ALTERNATIVE_PRIORITY_%s' % pkg,  
alt_name) or \
+   d.getVarFlag('ALTERNATIVE_PRIORITY',  alt_name) or \
+   d.getVar('ALTERNATIVE_PRIORITY_%s' % pkg) or  \
+   d.getVar('ALTERNATIVE_PRIORITY')
+
+# This shouldn't trigger, as it should have been resolved earlier!
+if alt_link == alt_target:
+bb.note('alt_link == alt_target: %s == %s -- correcting, this 
should not happen!' % (alt_link, alt_target))
+alt_target = '%s.%s' % (alt_target, pn)
+
+if not os.path.lexists('%s/%s' % (pkgdest, alt_target)):
+bb.warn('%s: NOT adding alternative provide %s: %s does not exist' 
% (pn, alt_link, alt_target))
+continue
+
+alt_target = os.path.normpath(alt_target)
+updates.append( (alt_name, alt_link, alt_target, alt_priority) )
+
+return updates
+
 PACKAGESPLITFUNCS_prepend = "populate_packages_updatealternatives "
 
 python populate_packages_updatealternatives () {
 if not update_alternatives_enabled(d):
 return
 
-pn = d.getVar('BPN')
-
 # Do actual update alternatives processing
-pkgdest = d.getVar('PKGD')
 for pkg in (d.getVar('PACKAGES') or "").split():
 # Create post install/removal scripts
 alt_setup_links = ""
 alt_remove_links = ""
-for alt_name in (d.getVar('ALTERNATIVE_%s' % pkg) or "").split():
-alt_link = d.getVarFlag('ALTERNATIVE_LINK_NAME', alt_name)
-alt_target   = d.getVarFlag('ALTERNATIVE_TARGET_%s' % pkg, 
alt_name) or d.getVarFlag('ALTERNATIVE_TARGET', alt_name)
-alt_target   = alt_target or d.getVar('ALTERNATIVE_TARGET_%s' % 
pkg) or d.getVar('ALTERNATIVE_TARGET') or alt_link
-# Sometimes alt_target is specified as relative to the link name.
-alt_target   = os.path.join(os.path.dirname(alt_link), alt_target)
-
-alt_priority = d.getVarFlag('ALTERNATIVE_PRIORITY_%s' % pkg,  
alt_name) or d.getVarFlag('ALTERNATIVE_PRIORITY',  alt_name)
-alt_priority = alt_priority or d.getVar('ALTERNATIVE_PRIORITY_%s' 
% pkg) or d.getVar('ALTERNATIVE_PRIORITY')
-
-# This shouldn't trigger, as it should have been resolved earlier!
-if alt_link == alt_target:
-bb.note('alt_link == alt_target: %s == %s -- correcting, this 
should not happen!' % (alt_link, alt_target))
-alt_target = '%s.%s' % (alt_target, pn)
-
-if not os.path.lexists('%s/%s' % (pkgdest, alt_target)):
-bb.warn('%s: NOT adding alternative provide %s: %s does not 
exist' % (pn, alt_link, alt_target))
-continue
-
-# Default to generate shell script.. eventually we may want to 
change this...
-alt_target = os.path.normpath(alt_target)
-
+updates = update_alternatives_alt_targets(d, pkg)
+for alt_name, alt_link, alt_target, alt_pr

[OE-core] [PATCH v4 0/4] Feature to use symbolic links to binaries for ptest

[Mariano Lopez]

Some binaries are renamed by update-alternatives class at build time
and some ptest run against a binary belonging to another package. Take
for example busybox or util-linux.

This series adds a directory within the ptest with symlinks to the
binaries produced by the package, so just adding this directory to the
PATH will test the correct binaries.

To use this feature just add PTEST_BINDIR = "1" to the recipe.

v2 changes:

Add busybox recipe to use this method of symlinks generation.

Refactor the way the binary ptest directory is generated. Now the
directory will contain only symlinks to binaries that were renamed by
update-alternatives and not symlinks to all the package's binaries. This
have some advantages:
- Less lines and the code is cleaner.
- There is no need to have symlinks for every binary.
- Reuse existing update-alternatives functionality, don't need to add
  new features to the class.

v3 changes:

Fix white space mismatch

v4 changes:

Check for PTEST_ENABLED to add the functionality, this will solve the
issue when building nativesdk packages

The following changes since commit ffa5a1bda6741f5dc9f1b8db1bb37b0c6f103c99:

  asciidoc: specify XML catalogue to use (2019-04-03 14:56:27 +0100)

are available in the Git repository at:

  git://github.com/justanotherboy/poky bug13238v4
  https://github.com/justanotherboy/poky/tree/bug13238v4

Mariano Lopez (4):
  update-alternatives.bbclass: Add function to get metadata
  ptest.bbclass: Add feature to populate a binary directory
  util-linux: Use PTEST binary directory
  busybox: Use PTEST binary directory

 meta/classes/ptest.bbclass| 32 +
 meta/classes/update-alternatives.bbclass  | 70 ---
 meta/recipes-core/busybox/busybox.inc | 16 +
 meta/recipes-core/util-linux/util-linux.inc   |  5 +-
 .../util-linux/util-linux/run-ptest   |  4 ++
 5 files changed, 86 insertions(+), 41 deletions(-)

-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 1/4] update-alternatives.bbclass: Add function to get metadata

[Mariano Lopez]

On 4/7/19 4:32 PM, Richard Purdie wrote:

On Sun, 2019-04-07 at 12:15 -0500, Mariano Lopez wrote:

-
-# Default to generate shell script.. eventually we may want to 
change this...
-alt_target = os.path.normpath(alt_target)
-
-alt_setup_links  += '\tupdate-alternatives --install %s %s %s 
%s\n' % (alt_link, alt_name, alt_target, alt_priority)
-alt_remove_links += '\tupdate-alternatives --remove  %s %s\n' % 
(alt_name, alt_target)
+updates = update_alternatives_alt_targets(d, pkg)
+for alt_name, alt_link, alt_target, alt_priority in updates:
+   alt_setup_links  += '\tupdate-alternatives --install %s %s %s %s\n' 
% (alt_link, alt_name, alt_target, alt_priority)
+   alt_remove_links += '\tupdate-alternatives --remove  %s %s\n' % 
(alt_name, alt_target)

I think the above is three whitespaces, not four? I know its minor but
it jumped out as I was comparing the code before/after!

Cheers,

Richard



Good catch! I have sent the the fixed version but I messed up with the 
cover letter, you can find the correct version here:

https://patchwork.openembedded.org/patch/160236/


I think I don't even need to put that link there :)

Thanks,

Mariano

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3 2/4] ptest.bbclass: Add feature to populate a binary directory

[Mariano Lopez]

This adds the functionality to create a binary directory within
PTEST_PATH directory. This directory will be populated with
symlinks pointing to the binaries installed by the package and
then renamed by update-alternatives. This way the ptest only needs
to source this binary directory in order to use the expected
binaries.

To enable this feature just add PTEST_BINDIR = "1" to the recipe.

[YOCTO #12597]
[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/classes/ptest.bbclass | 31 +++
 1 file changed, 31 insertions(+)

diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index 97865c9338..0c37447881 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -65,6 +65,37 @@ do_install_ptest_base() {
 done
 }
 
+PTEST_BINDIR_PKGD_PATH = "${PKGD}${PTEST_PATH}/bin"
+
+# This function needs to run after apply_update_alternative_renames because the
+# aforementioned function will update the ALTERNATIVE_LINK_NAME flag. Append is
+# used here to make this function to run as late as possible.
+PACKAGE_PREPROCESS_FUNCS_append = "${@bb.utils.contains("PTEST_BINDIR", "1", " 
ptest_update_alternatives", "", d)}"
+
+python ptest_update_alternatives() {
+"""
+This function will generate the symlinks in the PTEST_BINDIR_PKGD_PATH
+to match the renamed binaries by update-alternatives.
+"""
+
+if not bb.data.inherits_class('update-alternatives', d) \
+   or not update_alternatives_enabled(d):
+return
+
+bb.note("Generating symlinks for ptest")
+bin_paths = { os.environ["bindir"], os.environ["base_bindir"],
+  os.environ["sbindir"], os.environ["base_sbindir"] }
+ptest_bindir = d.getVar("PTEST_BINDIR_PKGD_PATH")
+os.mkdir(ptest_bindir)
+for pkg in (d.getVar('PACKAGES') or "").split():
+alternatives = update_alternatives_alt_targets(d, pkg)
+for alt_name, alt_link, alt_target, _ in alternatives:
+# Some alternatives are for man pages,
+# check if the alternative is in PATH
+if os.path.dirname(alt_link) in bin_paths:
+os.symlink(alt_target, os.path.join(ptest_bindir, alt_name))
+}
+
 do_configure_ptest_base[dirs] = "${B}"
 do_compile_ptest_base[dirs] = "${B}"
 do_install_ptest_base[dirs] = "${B}"
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3 4/4] busybox: Use PTEST binary directory

[Mariano Lopez]

This will generate the symlinks in the ptest binary directory using the
ptest class functionality instead of generating them manually. Because
the ptest class uses update-alternatives to get the metadata for the
symlinks it will respect the use of BUSYBOX_SPLIT_SUID automatically.

[YOCTO #12597]

Signed-off-by: Mariano Lopez 
---
 meta/recipes-core/busybox/busybox.inc | 16 ++--
 1 file changed, 2 insertions(+), 14 deletions(-)

diff --git a/meta/recipes-core/busybox/busybox.inc 
b/meta/recipes-core/busybox/busybox.inc
index c9d25ff1ca..174ce5a8c0 100644
--- a/meta/recipes-core/busybox/busybox.inc
+++ b/meta/recipes-core/busybox/busybox.inc
@@ -347,24 +347,12 @@ do_install () {
fi
 }
 
+PTEST_BINDIR = "1"
+
 do_install_ptest () {
cp -r ${B}/testsuite ${D}${PTEST_PATH}/
cp ${B}/.config  ${D}${PTEST_PATH}/
ln -s /bin/busybox   ${D}${PTEST_PATH}/busybox
-
-   mkdir ${D}${PTEST_PATH}/bin
-   if [ "${BUSYBOX_SPLIT_SUID}" = "1" ]; then
-   while read link; do
-   ln -s ${base_bindir}/busybox.suid 
${D}${PTEST_PATH}/bin/$(basename $link)
-   done <${D}${sysconfdir}/busybox.links.suid
-   while read link; do
-   ln -s ${base_bindir}/busybox.nosuid 
${D}${PTEST_PATH}/bin/$(basename $link)
-   done <${D}${sysconfdir}/busybox.links.nosuid
-   else
-   while read link; do
-   ln -s ${base_bindir}/busybox 
${D}${PTEST_PATH}/bin/$(basename $link)
-   done <${D}${sysconfdir}/busybox.links
-   fi
 }
 
 inherit update-alternatives
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3 1/4] update-alternatives.bbclass: Add function to get metadata

[Mariano Lopez]

This adds update_alternatives_alt_targets function to get the metadata
for a package. This is for code reuse because the metadata would help
other classes that needs to be aware of how update-alternatives modify
the final package.

[YOCTO #12597]
[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/classes/update-alternatives.bbclass | 70 +++-
 1 file changed, 45 insertions(+), 25 deletions(-)

diff --git a/meta/classes/update-alternatives.bbclass 
b/meta/classes/update-alternatives.bbclass
index 537e85d9a3..b702e77ee5 100644
--- a/meta/classes/update-alternatives.bbclass
+++ b/meta/classes/update-alternatives.bbclass
@@ -216,42 +216,62 @@ python apply_update_alternative_renames () {
 update_files(alt_target, alt_target_rename, pkg, d)
 }
 
+def update_alternatives_alt_targets(d, pkg):
+"""
+Returns the update-alternatives metadata for a package.
+
+The returned format is a list of tuples where the tuple contains:
+alt_name: The binary name
+alt_link: The path for the binary (Shared by different packages)
+alt_target:   The path for the renamed binary (Unique per package)
+alt_priority: The priority of the alt_target
+
+All the alt_targets will be installed into the sysroot. The alt_link is
+a symlink pointing to the alt_target with the highest priority.
+"""
+
+pn = d.getVar('BPN')
+pkgdest = d.getVar('PKGD')
+updates = list()
+for alt_name in (d.getVar('ALTERNATIVE_%s' % pkg) or "").split():
+alt_link = d.getVarFlag('ALTERNATIVE_LINK_NAME', alt_name)
+alt_target   = d.getVarFlag('ALTERNATIVE_TARGET_%s' % pkg, alt_name) 
or \
+   d.getVarFlag('ALTERNATIVE_TARGET', alt_name) or \
+   d.getVar('ALTERNATIVE_TARGET_%s' % pkg) or \
+   d.getVar('ALTERNATIVE_TARGET') or \
+   alt_link
+alt_priority = d.getVarFlag('ALTERNATIVE_PRIORITY_%s' % pkg,  
alt_name) or \
+   d.getVarFlag('ALTERNATIVE_PRIORITY',  alt_name) or \
+   d.getVar('ALTERNATIVE_PRIORITY_%s' % pkg) or  \
+   d.getVar('ALTERNATIVE_PRIORITY')
+
+# This shouldn't trigger, as it should have been resolved earlier!
+if alt_link == alt_target:
+bb.note('alt_link == alt_target: %s == %s -- correcting, this 
should not happen!' % (alt_link, alt_target))
+alt_target = '%s.%s' % (alt_target, pn)
+
+if not os.path.lexists('%s/%s' % (pkgdest, alt_target)):
+bb.warn('%s: NOT adding alternative provide %s: %s does not exist' 
% (pn, alt_link, alt_target))
+continue
+
+alt_target = os.path.normpath(alt_target)
+updates.append( (alt_name, alt_link, alt_target, alt_priority) )
+
+return updates
+
 PACKAGESPLITFUNCS_prepend = "populate_packages_updatealternatives "
 
 python populate_packages_updatealternatives () {
 if not update_alternatives_enabled(d):
 return
 
-pn = d.getVar('BPN')
-
 # Do actual update alternatives processing
-pkgdest = d.getVar('PKGD')
 for pkg in (d.getVar('PACKAGES') or "").split():
 # Create post install/removal scripts
 alt_setup_links = ""
 alt_remove_links = ""
-for alt_name in (d.getVar('ALTERNATIVE_%s' % pkg) or "").split():
-alt_link = d.getVarFlag('ALTERNATIVE_LINK_NAME', alt_name)
-alt_target   = d.getVarFlag('ALTERNATIVE_TARGET_%s' % pkg, 
alt_name) or d.getVarFlag('ALTERNATIVE_TARGET', alt_name)
-alt_target   = alt_target or d.getVar('ALTERNATIVE_TARGET_%s' % 
pkg) or d.getVar('ALTERNATIVE_TARGET') or alt_link
-# Sometimes alt_target is specified as relative to the link name.
-alt_target   = os.path.join(os.path.dirname(alt_link), alt_target)
-
-alt_priority = d.getVarFlag('ALTERNATIVE_PRIORITY_%s' % pkg,  
alt_name) or d.getVarFlag('ALTERNATIVE_PRIORITY',  alt_name)
-alt_priority = alt_priority or d.getVar('ALTERNATIVE_PRIORITY_%s' 
% pkg) or d.getVar('ALTERNATIVE_PRIORITY')
-
-# This shouldn't trigger, as it should have been resolved earlier!
-if alt_link == alt_target:
-bb.note('alt_link == alt_target: %s == %s -- correcting, this 
should not happen!' % (alt_link, alt_target))
-alt_target = '%s.%s' % (alt_target, pn)
-
-if not os.path.lexists('%s/%s' % (pkgdest, alt_target)):
-bb.warn('%s: NOT adding alternative provide %s: %s does not 
exist' % (pn, alt_link, alt_target))
-continue
-
-# Default to generate shell script.. eventually we may want to 
change this...
-alt_target = os.path.normpath(alt_target)
-
+updates = update_alternatives_alt_targets(d, pkg)
+for alt_name, alt_link, alt_target, alt_pr

[OE-core] [PATCH v3 3/4] util-linux: Use PTEST binary directory

[Mariano Lopez]

Some binaries generated by util-linux will be replaced by core-utils
in the final image by update-alternatives, so use a dedicated directory
with symlinks to avoid using a binary generated by another package.

This will solve the issue with the ptest runner timing out when
running the kill ptests for util-linux.

[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/recipes-core/util-linux/util-linux.inc   | 5 +++--
 meta/recipes-core/util-linux/util-linux/run-ptest | 4 
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/util-linux/util-linux.inc 
b/meta/recipes-core/util-linux/util-linux.inc
index 18c3af240e..a67318e84e 100644
--- a/meta/recipes-core/util-linux/util-linux.inc
+++ b/meta/recipes-core/util-linux/util-linux.inc
@@ -284,6 +284,7 @@ ALTERNATIVE_LINK_NAME[wall.1] = "${mandir}/man1/wall.1"
 
 BBCLASSEXTEND = "native nativesdk"
 
+PTEST_BINDIR = "1"
 do_compile_ptest() {
 oe_runmake buildtest-TESTS
 }
@@ -312,8 +313,8 @@ do_install_ptest() {
 
'/^\tif[[:space:]]\[[[:space:]]![[:space:]]-x[[:space:]]"$1"/s|$1|`which $1 
2>/dev/null`|g' \
  ${D}${PTEST_PATH}/tests/functions.sh
 
-# "kill -L" behaves differently than "/bin/kill -L" so we need an 
additional fix
+# Running "kill" without the the complete path would use the shell's 
built-in kill
 sed -i -e \
- '/^TS_CMD_KILL/ s|kill|/bin/kill|g' \
+ '/^TS_CMD_KILL/ s|kill|${PTEST_PATH}/bin/kill|g' \
  ${D}${PTEST_PATH}/tests/commands.sh
 }
diff --git a/meta/recipes-core/util-linux/util-linux/run-ptest 
b/meta/recipes-core/util-linux/util-linux/run-ptest
index fbc2f9b56a..2178ab8fef 100644
--- a/meta/recipes-core/util-linux/util-linux/run-ptest
+++ b/meta/recipes-core/util-linux/util-linux/run-ptest
@@ -1,5 +1,9 @@
 #!/bin/sh
 
+current_path=$(readlink -f $0)
+export bindir=$(dirname $current_path)
+export PATH=$bindir/bin:$PATH
+
 cd tests || exit 1  
 
 comps=$(find ts/ -type f -perm -111 -regex ".*/[^\.~]*" |  sort)
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/3] Feature to use symbolic links to binaries for ptest

[Mariano Lopez]

Thanks for pointing that out! The busybox recipe provided me
insight on how to implement this series in a cleaner way.
I refactor the code and the v2 of the series can be found here:
http://lists.openembedded.org/pipermail/openembedded-core/2019-April/280981.html

Please do not integrate this series

Thanks again Ross!

On Fri, Apr 5, 2019 at 1:49 PM Mariano Lopez 
wrote:

> Sure, can you integrate this in the while I work on the busybox one? I'll
> do it as part of bug 12597 that is also assigned to me
>
> Mariano
>
> On Fri, Apr 5, 2019 at 9:37 AM Burton, Ross  wrote:
>
>> Can you also port busybox to this instead of its own hand-coded logic?
>>
>> Ross
>>
>> On Fri, 5 Apr 2019 at 15:27, Mariano Lopez
>>  wrote:
>> >
>> > Some binaries are renamed by update-alternatives class at build time
>> > and some ptest run against a binary belonging to another package. Take
>> > for example busybox or util-linux.
>> >
>> > This series adds a directory within the ptest path with symlinks to the
>> > binaries produced by the package, so just adding this directory to the
>> > PATH will test the correct binaries.
>> >
>> > To use this feature just add PTEST_BINDIR = "1" to the recipe.
>> >
>> > This will help to solve bug 13238.
>> >
>> > [YOCTO #13238]
>> >
>> > The following changes since commit
>> ffa5a1bda6741f5dc9f1b8db1bb37b0c6f103c99:
>> >
>> >   asciidoc: specify XML catalogue to use (2019-04-03 14:56:27 +0100)
>> >
>> > are available in the Git repository at:
>> >
>> >   git://github.com/justanotherboy/poky bug12597
>> >   https://github.com/justanotherboy/poky/tree/bug12597
>> >
>> > Mariano Lopez (3):
>> >   update-alternatives.bbclass: Add function to get renamed binaries
>> >   ptest.bbclass: Add feature to populate a binary directory
>> >   util-linux: Use PTEST binary directory
>> >
>> >
>> > --
>> > 2.19.2
>> >
>> > --
>> > ___
>> > Openembedded-core mailing list
>> > Openembedded-core@lists.openembedded.org
>> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 3/4] util-linux: Use PTEST binary directory

[Mariano Lopez]

Some binaries generated by util-linux will be replaced by core-utils
in the final image by update-alternatives, so use a dedicated directory
with symlinks to avoid using a binary generated by another package.

This will solve the issue with the ptest runner timing out when
running the kill ptests for util-linux.

[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/recipes-core/util-linux/util-linux.inc   | 5 +++--
 meta/recipes-core/util-linux/util-linux/run-ptest | 4 
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/util-linux/util-linux.inc 
b/meta/recipes-core/util-linux/util-linux.inc
index 18c3af240e..a67318e84e 100644
--- a/meta/recipes-core/util-linux/util-linux.inc
+++ b/meta/recipes-core/util-linux/util-linux.inc
@@ -284,6 +284,7 @@ ALTERNATIVE_LINK_NAME[wall.1] = "${mandir}/man1/wall.1"
 
 BBCLASSEXTEND = "native nativesdk"
 
+PTEST_BINDIR = "1"
 do_compile_ptest() {
 oe_runmake buildtest-TESTS
 }
@@ -312,8 +313,8 @@ do_install_ptest() {
 
'/^\tif[[:space:]]\[[[:space:]]![[:space:]]-x[[:space:]]"$1"/s|$1|`which $1 
2>/dev/null`|g' \
  ${D}${PTEST_PATH}/tests/functions.sh
 
-# "kill -L" behaves differently than "/bin/kill -L" so we need an 
additional fix
+# Running "kill" without the the complete path would use the shell's 
built-in kill
 sed -i -e \
- '/^TS_CMD_KILL/ s|kill|/bin/kill|g' \
+ '/^TS_CMD_KILL/ s|kill|${PTEST_PATH}/bin/kill|g' \
  ${D}${PTEST_PATH}/tests/commands.sh
 }
diff --git a/meta/recipes-core/util-linux/util-linux/run-ptest 
b/meta/recipes-core/util-linux/util-linux/run-ptest
index fbc2f9b56a..2178ab8fef 100644
--- a/meta/recipes-core/util-linux/util-linux/run-ptest
+++ b/meta/recipes-core/util-linux/util-linux/run-ptest
@@ -1,5 +1,9 @@
 #!/bin/sh
 
+current_path=$(readlink -f $0)
+export bindir=$(dirname $current_path)
+export PATH=$bindir/bin:$PATH
+
 cd tests || exit 1  
 
 comps=$(find ts/ -type f -perm -111 -regex ".*/[^\.~]*" |  sort)
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 4/4] busybox: Use PTEST binary directory

[Mariano Lopez]

This will generate the symlinks in the ptest binary directory using the
ptest class functionality instead of generating them manually. Because
the ptest class uses update-alternatives to get the metadata for the
symlinks it will respect the use of BUSYBOX_SPLIT_SUID automatically.

[YOCTO #12597]

Signed-off-by: Mariano Lopez 
---
 meta/recipes-core/busybox/busybox.inc | 16 ++--
 1 file changed, 2 insertions(+), 14 deletions(-)

diff --git a/meta/recipes-core/busybox/busybox.inc 
b/meta/recipes-core/busybox/busybox.inc
index c9d25ff1ca..174ce5a8c0 100644
--- a/meta/recipes-core/busybox/busybox.inc
+++ b/meta/recipes-core/busybox/busybox.inc
@@ -347,24 +347,12 @@ do_install () {
fi
 }
 
+PTEST_BINDIR = "1"
+
 do_install_ptest () {
cp -r ${B}/testsuite ${D}${PTEST_PATH}/
cp ${B}/.config  ${D}${PTEST_PATH}/
ln -s /bin/busybox   ${D}${PTEST_PATH}/busybox
-
-   mkdir ${D}${PTEST_PATH}/bin
-   if [ "${BUSYBOX_SPLIT_SUID}" = "1" ]; then
-   while read link; do
-   ln -s ${base_bindir}/busybox.suid 
${D}${PTEST_PATH}/bin/$(basename $link)
-   done <${D}${sysconfdir}/busybox.links.suid
-   while read link; do
-   ln -s ${base_bindir}/busybox.nosuid 
${D}${PTEST_PATH}/bin/$(basename $link)
-   done <${D}${sysconfdir}/busybox.links.nosuid
-   else
-   while read link; do
-   ln -s ${base_bindir}/busybox 
${D}${PTEST_PATH}/bin/$(basename $link)
-   done <${D}${sysconfdir}/busybox.links
-   fi
 }
 
 inherit update-alternatives
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 1/4] update-alternatives.bbclass: Add function to get metadata

[Mariano Lopez]

This adds update_alternatives_alt_targets function to get the metadata
for a package. This is for code reuse because the metadata would help
other classes that needs to be aware of how update-alternatives modify
the final package.

[YOCTO #12597]
[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/classes/update-alternatives.bbclass | 74 +++-
 1 file changed, 47 insertions(+), 27 deletions(-)

diff --git a/meta/classes/update-alternatives.bbclass 
b/meta/classes/update-alternatives.bbclass
index 537e85d9a3..bc19373348 100644
--- a/meta/classes/update-alternatives.bbclass
+++ b/meta/classes/update-alternatives.bbclass
@@ -216,44 +216,64 @@ python apply_update_alternative_renames () {
 update_files(alt_target, alt_target_rename, pkg, d)
 }
 
+def update_alternatives_alt_targets(d, pkg):
+"""
+Returns the update-alternatives metadata for a package.
+
+The returned format is a list of tuples where the tuple contains:
+alt_name: The binary name
+alt_link: The path for the binary (Shared by different packages)
+alt_target:   The path for the renamed binary (Unique per package)
+alt_priority: The priority of the alt_target
+
+All the alt_targets will be installed into the sysroot. The alt_link is
+a symlink pointing to the alt_target with the highest priority.
+"""
+
+pn = d.getVar('BPN')
+pkgdest = d.getVar('PKGD')
+updates = list()
+for alt_name in (d.getVar('ALTERNATIVE_%s' % pkg) or "").split():
+alt_link = d.getVarFlag('ALTERNATIVE_LINK_NAME', alt_name)
+alt_target   = d.getVarFlag('ALTERNATIVE_TARGET_%s' % pkg, alt_name) 
or \
+   d.getVarFlag('ALTERNATIVE_TARGET', alt_name) or \
+   d.getVar('ALTERNATIVE_TARGET_%s' % pkg) or \
+   d.getVar('ALTERNATIVE_TARGET') or \
+   alt_link
+alt_priority = d.getVarFlag('ALTERNATIVE_PRIORITY_%s' % pkg,  
alt_name) or \
+   d.getVarFlag('ALTERNATIVE_PRIORITY',  alt_name) or \
+   d.getVar('ALTERNATIVE_PRIORITY_%s' % pkg) or  \
+   d.getVar('ALTERNATIVE_PRIORITY')
+
+# This shouldn't trigger, as it should have been resolved earlier!
+if alt_link == alt_target:
+bb.note('alt_link == alt_target: %s == %s -- correcting, this 
should not happen!' % (alt_link, alt_target))
+alt_target = '%s.%s' % (alt_target, pn)
+
+if not os.path.lexists('%s/%s' % (pkgdest, alt_target)):
+bb.warn('%s: NOT adding alternative provide %s: %s does not exist' 
% (pn, alt_link, alt_target))
+continue
+
+alt_target = os.path.normpath(alt_target)
+updates.append( (alt_name, alt_link, alt_target, alt_priority) )
+
+return updates
+
 PACKAGESPLITFUNCS_prepend = "populate_packages_updatealternatives "
 
 python populate_packages_updatealternatives () {
 if not update_alternatives_enabled(d):
 return
 
-pn = d.getVar('BPN')
-
 # Do actual update alternatives processing
-pkgdest = d.getVar('PKGD')
 for pkg in (d.getVar('PACKAGES') or "").split():
 # Create post install/removal scripts
 alt_setup_links = ""
 alt_remove_links = ""
-for alt_name in (d.getVar('ALTERNATIVE_%s' % pkg) or "").split():
-alt_link = d.getVarFlag('ALTERNATIVE_LINK_NAME', alt_name)
-alt_target   = d.getVarFlag('ALTERNATIVE_TARGET_%s' % pkg, 
alt_name) or d.getVarFlag('ALTERNATIVE_TARGET', alt_name)
-alt_target   = alt_target or d.getVar('ALTERNATIVE_TARGET_%s' % 
pkg) or d.getVar('ALTERNATIVE_TARGET') or alt_link
-# Sometimes alt_target is specified as relative to the link name.
-alt_target   = os.path.join(os.path.dirname(alt_link), alt_target)
-
-alt_priority = d.getVarFlag('ALTERNATIVE_PRIORITY_%s' % pkg,  
alt_name) or d.getVarFlag('ALTERNATIVE_PRIORITY',  alt_name)
-alt_priority = alt_priority or d.getVar('ALTERNATIVE_PRIORITY_%s' 
% pkg) or d.getVar('ALTERNATIVE_PRIORITY')
-
-# This shouldn't trigger, as it should have been resolved earlier!
-if alt_link == alt_target:
-bb.note('alt_link == alt_target: %s == %s -- correcting, this 
should not happen!' % (alt_link, alt_target))
-alt_target = '%s.%s' % (alt_target, pn)
-
-if not os.path.lexists('%s/%s' % (pkgdest, alt_target)):
-bb.warn('%s: NOT adding alternative provide %s: %s does not 
exist' % (pn, alt_link, alt_target))
-continue
-
-# Default to generate shell script.. eventually we may want to 
change this...
-alt_target = os.path.normpath(alt_target)
-
-alt_setup_links  += '\tupdate-alternatives --install %s %s %s 
%s\n' % (alt_link, alt_name, alt_tar

[OE-core] [PATCH v2 0/4] Feature to use symbolic links to binaries for ptest

[Mariano Lopez]

Some binaries are renamed by update-alternatives class at build time
and some ptest run against a binary belonging to another package. Take
for example busybox or util-linux.

This series adds a directory within the ptest with symlinks to the
binaries produced by the package, so just adding this directory to the
PATH will test the correct binaries.

v2 changes:

Add busybox recipe to use this method of symlinks generation.

Refactor the way the binary ptest directory is generated. Now the
directory will contain only symlinks to binaries that were renamed by
update-alternatives and not symlinks to all the package's binaries. This
have some advantages:
- Less lines and the code is cleaner.
- There is no need to have symlinks for every binary.
- Reuse existing update-alternatives functionality, don't need to add
  new features to the class.

To use this feature just add PTEST_BINDIR = "1" to the recipe.

The following changes since commit ffa5a1bda6741f5dc9f1b8db1bb37b0c6f103c99:

  asciidoc: specify XML catalogue to use (2019-04-03 14:56:27 +0100)

are available in the Git repository at:

  git://github.com/justanotherboy/poky bug13238
  https://github.com/justanotherboy/poky/tree/bug13238

Mariano Lopez (4):
  update-alternatives.bbclass: Add function to get metadata
  ptest.bbclass: Add feature to populate a binary directory
  util-linux: Use PTEST binary directory
  busybox: Use PTEST binary directory

 meta/classes/ptest.bbclass| 31 
 meta/classes/update-alternatives.bbclass  | 74 ---
 meta/recipes-core/busybox/busybox.inc | 16 +---
 meta/recipes-core/util-linux/util-linux.inc   |  5 +-
 .../util-linux/util-linux/run-ptest   |  4 +
 5 files changed, 87 insertions(+), 43 deletions(-)

-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 2/4] ptest.bbclass: Add feature to populate a binary directory

[Mariano Lopez]

This adds the functionality to create a binary directory within
PTEST_PATH directory. This directory will be populated with
symlinks pointing to the binaries installed by the package and
then renamed by update-alternatives. This way the ptest only needs
to source this binary directory in order to use the expected
binaries.

To enable this feature just add PTEST_BINDIR = "1" to the recipe.

[YOCTO #12597]
[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/classes/ptest.bbclass | 31 +++
 1 file changed, 31 insertions(+)

diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index 97865c9338..0c37447881 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -65,6 +65,37 @@ do_install_ptest_base() {
 done
 }
 
+PTEST_BINDIR_PKGD_PATH = "${PKGD}${PTEST_PATH}/bin"
+
+# This function needs to run after apply_update_alternative_renames because the
+# aforementioned function will update the ALTERNATIVE_LINK_NAME flag. Append is
+# used here to make this function to run as late as possible.
+PACKAGE_PREPROCESS_FUNCS_append = "${@bb.utils.contains("PTEST_BINDIR", "1", " 
ptest_update_alternatives", "", d)}"
+
+python ptest_update_alternatives() {
+"""
+This function will generate the symlinks in the PTEST_BINDIR_PKGD_PATH
+to match the renamed binaries by update-alternatives.
+"""
+
+if not bb.data.inherits_class('update-alternatives', d) \
+   or not update_alternatives_enabled(d):
+return
+
+bb.note("Generating symlinks for ptest")
+bin_paths = { os.environ["bindir"], os.environ["base_bindir"],
+  os.environ["sbindir"], os.environ["base_sbindir"] }
+ptest_bindir = d.getVar("PTEST_BINDIR_PKGD_PATH")
+os.mkdir(ptest_bindir)
+for pkg in (d.getVar('PACKAGES') or "").split():
+alternatives = update_alternatives_alt_targets(d, pkg)
+for alt_name, alt_link, alt_target, _ in alternatives:
+# Some alternatives are for man pages,
+# check if the alternative is in PATH
+if os.path.dirname(alt_link) in bin_paths:
+os.symlink(alt_target, os.path.join(ptest_bindir, alt_name))
+}
+
 do_configure_ptest_base[dirs] = "${B}"
 do_compile_ptest_base[dirs] = "${B}"
 do_install_ptest_base[dirs] = "${B}"
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/3] Feature to use symbolic links to binaries for ptest

[Mariano Lopez]

Sure, can you integrate this in the while I work on the busybox one? I'll
do it as part of bug 12597 that is also assigned to me

Mariano

On Fri, Apr 5, 2019 at 9:37 AM Burton, Ross  wrote:

> Can you also port busybox to this instead of its own hand-coded logic?
>
> Ross
>
> On Fri, 5 Apr 2019 at 15:27, Mariano Lopez
>  wrote:
> >
> > Some binaries are renamed by update-alternatives class at build time
> > and some ptest run against a binary belonging to another package. Take
> > for example busybox or util-linux.
> >
> > This series adds a directory within the ptest path with symlinks to the
> > binaries produced by the package, so just adding this directory to the
> > PATH will test the correct binaries.
> >
> > To use this feature just add PTEST_BINDIR = "1" to the recipe.
> >
> > This will help to solve bug 13238.
> >
> > [YOCTO #13238]
> >
> > The following changes since commit
> ffa5a1bda6741f5dc9f1b8db1bb37b0c6f103c99:
> >
> >   asciidoc: specify XML catalogue to use (2019-04-03 14:56:27 +0100)
> >
> > are available in the Git repository at:
> >
> >   git://github.com/justanotherboy/poky bug12597
> >   https://github.com/justanotherboy/poky/tree/bug12597
> >
> > Mariano Lopez (3):
> >   update-alternatives.bbclass: Add function to get renamed binaries
> >   ptest.bbclass: Add feature to populate a binary directory
> >   util-linux: Use PTEST binary directory
> >
> >
> > --
> > 2.19.2
> >
> > --
> > ___
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] util-linux: Use PTEST binary directory

[Mariano Lopez]

Some binaries generated by util-linux will be replaced by core-utils
in the final image by update-alternatives, so use a dedicated directory
with symlinks to avoid using a binary generated by another package.

This will solve the issue with the ptest runner timing out when running
the kill ptests for util-linux.

[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/recipes-core/util-linux/util-linux.inc   | 5 +++--
 meta/recipes-core/util-linux/util-linux/run-ptest | 4 
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/util-linux/util-linux.inc 
b/meta/recipes-core/util-linux/util-linux.inc
index 18c3af240e..a67318e84e 100644
--- a/meta/recipes-core/util-linux/util-linux.inc
+++ b/meta/recipes-core/util-linux/util-linux.inc
@@ -284,6 +284,7 @@ ALTERNATIVE_LINK_NAME[wall.1] = "${mandir}/man1/wall.1"
 
 BBCLASSEXTEND = "native nativesdk"
 
+PTEST_BINDIR = "1"
 do_compile_ptest() {
 oe_runmake buildtest-TESTS
 }
@@ -312,8 +313,8 @@ do_install_ptest() {
 
'/^\tif[[:space:]]\[[[:space:]]![[:space:]]-x[[:space:]]"$1"/s|$1|`which $1 
2>/dev/null`|g' \
  ${D}${PTEST_PATH}/tests/functions.sh
 
-# "kill -L" behaves differently than "/bin/kill -L" so we need an 
additional fix
+# Running "kill" without the the complete path would use the shell's 
built-in kill
 sed -i -e \
- '/^TS_CMD_KILL/ s|kill|/bin/kill|g' \
+ '/^TS_CMD_KILL/ s|kill|${PTEST_PATH}/bin/kill|g' \
  ${D}${PTEST_PATH}/tests/commands.sh
 }
diff --git a/meta/recipes-core/util-linux/util-linux/run-ptest 
b/meta/recipes-core/util-linux/util-linux/run-ptest
index fbc2f9b56a..2178ab8fef 100644
--- a/meta/recipes-core/util-linux/util-linux/run-ptest
+++ b/meta/recipes-core/util-linux/util-linux/run-ptest
@@ -1,5 +1,9 @@
 #!/bin/sh
 
+current_path=$(readlink -f $0)
+export bindir=$(dirname $current_path)
+export PATH=$bindir/bin:$PATH
+
 cd tests || exit 1  
 
 comps=$(find ts/ -type f -perm -111 -regex ".*/[^\.~]*" |  sort)
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] Feature to use symbolic links to binaries for ptest

[Mariano Lopez]

Some binaries are renamed by update-alternatives class at build time
and some ptest run against a binary belonging to another package. Take
for example busybox or util-linux.

This series adds a directory within the ptest path with symlinks to the
binaries produced by the package, so just adding this directory to the
PATH will test the correct binaries.

To use this feature just add PTEST_BINDIR = "1" to the recipe.

This will help to solve bug 13238.

[YOCTO #13238]

The following changes since commit ffa5a1bda6741f5dc9f1b8db1bb37b0c6f103c99:

  asciidoc: specify XML catalogue to use (2019-04-03 14:56:27 +0100)

are available in the Git repository at:

  git://github.com/justanotherboy/poky bug12597
  https://github.com/justanotherboy/poky/tree/bug12597

Mariano Lopez (3):
  update-alternatives.bbclass: Add function to get renamed binaries
  ptest.bbclass: Add feature to populate a binary directory
  util-linux: Use PTEST binary directory


-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] ptest.bbclass: Add feature to populate a binary directory

[Mariano Lopez]

This adds a functionality to populate a binary directory with symlinks
within the PTEST_PATH directory to the binaries build by the package. This
is helpful for binaries that are renamed by update-alternatives.

This will take all the package binaries' that are in bindir, sbindir
base_bindir, and base_sbindir and create symlinks in PTEST_BINDIR_PATH. To
enable this feature just add PTEST_BINDIR = "1" to the recipe.

[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/classes/ptest.bbclass | 52 ++
 1 file changed, 52 insertions(+)

diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index 97865c9338..735057c44d 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -63,6 +63,58 @@ do_install_ptest_base() {
 fi
 done
 done
+
+if [ "${PTEST_BINDIR}" == "1" ]; then
+install_ptest_bindir
+fi
+}
+
+PTEST_BINDIR_PATH="${D}${PTEST_PATH}/bin"
+
+install_ptest_bindir() {
+# Create ${PTEST_PATH}/bin to create symlinks to the package's binaries
+# this way the path can be added to PATH and execute the binaries easier
+# from ptest-runner.
+bbdebug 1 "Generating PTEST's bin directory"
+binary_paths="${bindir} ${sbindir} ${base_bindir} ${base_sbindir}"
+mkdir -p ${PTEST_BINDIR_PATH}
+
+for path in ${binary_paths}; do
+for src in ${D}${path}/*; do
+binary=`basename ${src}`
+ln -s ${path}/${binary} ${PTEST_BINDIR_PATH}/${binary}
+done
+done
+}
+
+# This function needs to run after apply_update_alternative_renames because the
+# aforementioned function will update the ALTERNATIVE_LINK_NAME flag. Append is
+# used here to make this function to run as late as possible.
+PACKAGE_PREPROCESS_FUNCS_append = "${@bb.utils.contains("PTEST_BINDIR", "1", " 
ptest_update_alternatives", "", d)}"
+
+python ptest_update_alternatives() {
+"""
+This function will fix the symlinks in the PTEST_BINDIR that
+were broken by the renaming of update-alternatives
+"""
+
+if not bb.data.inherits_class('update-alternatives', d) \
+   or not update_alternatives_enabled(d):
+return
+
+bb.note("Updating PTEST symlinks after the renaming of 
update-alternatives")
+
+ptest_pkgd_bindir = os.path.join(d.getVar("PKGD"),
+ d.getVar("PTEST_PATH")[1:],
+ "bin")
+links_dict = { os.path.join(ptest_pkgd_bindir, link):
+   os.readlink(os.path.join(ptest_pkgd_bindir, link))
+   for link in os.listdir(ptest_pkgd_bindir) }
+for filename, link in links_dict.items():
+alt_link = update_alternatives_get_alt_target(d, link)
+if alt_link:
+os.unlink(filename)
+os.symlink(alt_link, filename)
 }
 
 do_configure_ptest_base[dirs] = "${B}"
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] update-alternatives.bbclass: Add function to get renamed binaries

[Mariano Lopez]

This adds a function to get the new name for renamed binaries by the
update-alternatives class. The function uses the build metadata because
some update-alternatives binaries/scripts lack the feature to provide
this information.

This also adds the function _ua_alt_target for code reuse.

[YOCTO #13238]

Signed-off-by: Mariano Lopez 
---
 meta/classes/update-alternatives.bbclass | 43 +---
 1 file changed, 38 insertions(+), 5 deletions(-)

diff --git a/meta/classes/update-alternatives.bbclass 
b/meta/classes/update-alternatives.bbclass
index 537e85d9a3..d601dfc700 100644
--- a/meta/classes/update-alternatives.bbclass
+++ b/meta/classes/update-alternatives.bbclass
@@ -216,6 +216,43 @@ python apply_update_alternative_renames () {
 update_files(alt_target, alt_target_rename, pkg, d)
 }
 
+def update_alternatives_get_alt_target(d, link):
+"""
+Returns the renamed binary modified by update-alternatives class.
+
+Unfortunately not all update-alternative binaries/scripts implement a way
+to provide a list of binaries for a link, so we need to use the build
+metadata to get the information.
+
+NOTE: This function needs to run after apply_update_alternative_renames
+"""
+for pkg in (d.getVar('PACKAGES') or "").split():
+for alt_name in (d.getVar('ALTERNATIVE_%s' % pkg) or "").split():
+alt_link   = d.getVarFlag('ALTERNATIVE_LINK_NAME', alt_name)
+
+# Just care for the link being asked for
+if link != alt_link:
+continue
+
+alt_target = _ua_alt_target(d, alt_name, alt_link, pkg)
+if alt_target != alt_link:
+return alt_target
+else:
+return None
+
+return None
+
+def _ua_alt_target(d, alt_name, alt_link, pkg):
+alt_target = d.getVarFlag('ALTERNATIVE_TARGET_%s' % pkg, alt_name) or \
+ d.getVarFlag('ALTERNATIVE_TARGET', alt_name) or \
+ d.getVar('ALTERNATIVE_TARGET_%s' % pkg) or \
+ d.getVar('ALTERNATIVE_TARGET') or \
+ alt_link
+
+# Sometimes alt_target is specified as relative to the link name.
+alt_target   = os.path.join(os.path.dirname(alt_link), alt_target)
+return alt_target
+
 PACKAGESPLITFUNCS_prepend = "populate_packages_updatealternatives "
 
 python populate_packages_updatealternatives () {
@@ -232,11 +269,7 @@ python populate_packages_updatealternatives () {
 alt_remove_links = ""
 for alt_name in (d.getVar('ALTERNATIVE_%s' % pkg) or "").split():
 alt_link = d.getVarFlag('ALTERNATIVE_LINK_NAME', alt_name)
-alt_target   = d.getVarFlag('ALTERNATIVE_TARGET_%s' % pkg, 
alt_name) or d.getVarFlag('ALTERNATIVE_TARGET', alt_name)
-alt_target   = alt_target or d.getVar('ALTERNATIVE_TARGET_%s' % 
pkg) or d.getVar('ALTERNATIVE_TARGET') or alt_link
-# Sometimes alt_target is specified as relative to the link name.
-alt_target   = os.path.join(os.path.dirname(alt_link), alt_target)
-
+alt_target   = _ua_alt_target(d, alt_name, alt_link, pkg)
 alt_priority = d.getVarFlag('ALTERNATIVE_PRIORITY_%s' % pkg,  
alt_name) or d.getVarFlag('ALTERNATIVE_PRIORITY',  alt_name)
 alt_priority = alt_priority or d.getVar('ALTERNATIVE_PRIORITY_%s' 
% pkg) or d.getVar('ALTERNATIVE_PRIORITY')
 
-- 
2.19.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/1] package_ipk.bbclass: Replace empty lines in DESCRIPTION with '.'

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This will deal with empty lines in DESCRIPTION while creating ipk packages.

Changes in v2:

- Firts version of this patch would fail do_package_write_ipk task if found an
  empty line in DESCRIPTION, this version will replace an empty line with a '.'
  as suggested by Ross.

The following changes since commit eff56e4f0d59b1d965a68e4f009b7f07717b7edd:

  bitbake: bitbake-user-manual: Fixed special character in -D output 
(2017-04-03 22:32:02 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/bug10677v2
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug10677v2

Mariano Lopez (1):
  package_ipk.bbclass: Replace empty lines in DESCRIPTION with '.'

 meta/classes/package_ipk.bbclass | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

-- 
2.10.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] package_ipk.bbclass: Replace empty lines in DESCRIPTION with '.'

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

opkg uses empty lines as separator for next package and if an ipk file was
packaged with empty lines in DESCRIPTION opkg won't be able to handle such ipk
file, this happens at execution time.

This commit will replace empty lines in DESCRIPTION with a '.' when generating
an ipk package to avoid this issue.

[YOCTO #10677]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/package_ipk.bbclass | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index fa47385..c7cec9d 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -145,7 +145,11 @@ python do_package_ipk () {
 # We don't limit the width when manually indent, but 
we do
 # need the textwrap.fill() to set the initial_indent 
and
 # subsequent_indent, so set a large width
-ctrlfile.write('%s\n' % textwrap.fill(t.strip(), 
width=10, initial_indent=' ', subsequent_indent=' '))
+line = textwrap.fill(t.strip(),
+ width=10,
+ initial_indent=' ',
+ subsequent_indent=' ') or '.'
+ctrlfile.write('%s\n' % line)
 else:
 # Auto indent
 ctrlfile.write('%s\n' % textwrap.fill(description, 
width=74, initial_indent=' ', subsequent_indent=' '))
-- 
2.10.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] oeqa/runtime/cases: Migrate underscore cases

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

There were two missing cases to be migrated to the new framework: _qemutiny and
_ptest.

qemutiny was straightforward.

ptest on the other hand wasn't working even in previous releases; it has been
migrated from smart to dnf, and how ptest packages are gathered to be
installed, adapted to use unicode, and removed a lot of code that wasn't needed
anymore.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/runtime/cases/_ptest.py| 130 +--
 meta/lib/oeqa/runtime/cases/_qemutiny.py |  11 ++-
 2 files changed, 59 insertions(+), 82 deletions(-)

diff --git a/meta/lib/oeqa/runtime/cases/_ptest.py 
b/meta/lib/oeqa/runtime/cases/_ptest.py
index a243251..aaed9a5 100644
--- a/meta/lib/oeqa/runtime/cases/_ptest.py
+++ b/meta/lib/oeqa/runtime/cases/_ptest.py
@@ -1,30 +1,39 @@
-import unittest, os, shutil
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
+import os
+import shutil
+import subprocess
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotDataVar, skipIfNotFeature
+from oeqa.runtime.decorator.package import OEHasPackage
+
+from oeqa.runtime.cases.dnf import DnfTest
 from oeqa.utils.logparser import *
 from oeqa.utils.httpserver import HTTPService
-import bb
-import glob
-from oe.package_manager import RpmPkgsList
-import subprocess
 
-def setUpModule():
-if not oeRuntimeTest.hasFeature("package-management"):
-skipModule("Image doesn't have package management feature")
-if not oeRuntimeTest.hasPackage("smartpm"):
-skipModule("Image doesn't have smart installed")
-if "package_rpm" != 
oeRuntimeTest.tc.d.getVar("PACKAGE_CLASSES").split()[0]:
-skipModule("Rpm is not the primary package manager")
+class PtestRunnerTest(DnfTest):
+
+@classmethod
+def setUpClass(cls):
+rpm_deploy = os.path.join(cls.tc.td['DEPLOY_DIR'], 'rpm')
+cls.repo_server = HTTPService(rpm_deploy, cls.tc.target.server_ip)
+cls.repo_server.start()
 
-class PtestRunnerTest(oeRuntimeTest):
+@classmethod
+def tearDownClass(cls):
+cls.repo_server.stop()
 
 # a ptest log parser
 def parse_ptest(self, logfile):
-parser = Lparser(test_0_pass_regex="^PASS:(.+)", 
test_0_fail_regex="^FAIL:(.+)", section_0_begin_regex="^BEGIN: .*/(.+)/ptest", 
section_0_end_regex="^END: .*/(.+)/ptest")
+parser = Lparser(test_0_pass_regex="^PASS:(.+)",
+ test_0_fail_regex="^FAIL:(.+)",
+ section_0_begin_regex="^BEGIN: .*/(.+)/ptest",
+ section_0_end_regex="^END: .*/(.+)/ptest")
 parser.init()
 result = Result()
 
-with open(logfile) as f:
+with open(logfile, errors='replace') as f:
 for line in f:
 result_tuple = parser.parse_line(line)
 if not result_tuple:
@@ -50,70 +59,39 @@ class PtestRunnerTest(oeRuntimeTest):
 result.sort_tests()
 return result
 
-@classmethod
-def setUpClass(self):
-#note the existing channels that are on the board before creating new 
ones
-#self.existingchannels = set()
-#(status, result) = oeRuntimeTest.tc.target.run('smart channel --show 
| grep "\["', 0)
-#for x in result.split("\n"):
-#self.existingchannels.add(x)
-self.repo_server = 
HTTPService(oeRuntimeTest.tc.d.getVar('DEPLOY_DIR'), 
oeRuntimeTest.tc.target.server_ip)
-self.repo_server.start()
-
-@classmethod
-def tearDownClass(self):
-self.repo_server.stop()
-#remove created channels to be able to repeat the tests on same image
-#(status, result) = oeRuntimeTest.tc.target.run('smart channel --show 
| grep "\["', 0)
-#for x in result.split("\n"):
-#if x not in self.existingchannels:
-#oeRuntimeTest.tc.target.run('smart channel --remove 
'+x[1:-1]+' -y', 0)
-
-def add_smart_channel(self):
-image_pkgtype = self.tc.d.getVar('IMAGE_PKGTYPE')
-deploy_url = 'http://%s:%s/%s' %(self.target.server_ip, 
self.repo_server.port, image_pkgtype)
-pkgarchs = self.tc.d.getVar('PACKAGE_ARCHS').replace("-","_").split()
-for arch in os.listdir('%s/%s' % (self.repo_server.root_dir, 
image_pkgtype)):
-if arch in pkgarchs:
-self.target.run('smart channel -y --add {a} type=rpm-md 
baseurl={u}/{a}'.format(a=arch, u=deploy_url), 0)
-self.target.run('smart update', 0)
-
-def install_complementary(self, globs=None):
-in

[OE-core] [PATCHv2 1/1] oeqa/core/loader.py: Do not import underscore modules by default

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Underscore modules are meant to be run only when manually added to the test
suite, so far another mechanisms are in place to make this happen with
runtime, sdk, and esdk (mostly in test* bbclasses).

This will add such functionality in the core framework so other specific
frameworks can take use this without adding something else.

[YOCTO #10980]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/core/loader.py | 34 ++
 1 file changed, 30 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
index 74f1117..63a1703 100644
--- a/meta/lib/oeqa/core/loader.py
+++ b/meta/lib/oeqa/core/loader.py
@@ -223,8 +223,21 @@ class OETestLoader(unittest.TestLoader):
 msg = 'Tried to import %s test module but is a built-in'
 raise ImportError(msg % module.__name__)
 
-if not self.modules or "all" in self.modules or \
-module.__name__ in self.modules:
+# Normal test modules are loaded if no modules were specified,
+# if module is in the specified module list or if 'all' is in
+# module list.
+# Underscore modules are loaded only if specified in module list.
+load_module = True if not module.__name__.startswith('_') \
+  and (not self.modules \
+   or module.__name__ in self.modules \
+   or 'all' in self.modules) \
+   else False
+
+load_underscore = True if module.__name__.startswith('_') \
+  and module.__name__ in self.modules \
+   else False
+
+if load_module or load_underscore:
 return super(OETestLoader, self).loadTestsFromModule(
 module, *args, pattern=pattern, **kws)
 else:
@@ -238,8 +251,21 @@ class OETestLoader(unittest.TestLoader):
 msg = 'Tried to import %s test module but is a built-in'
 raise ImportError(msg % module.__name__)
 
-if not self.modules or "all" in self.modules or \
-module.__name__ in self.modules:
+# Normal test modules are loaded if no modules were specified,
+# if module is in the specified module list or if 'all' is in
+# module list.
+# Underscore modules are loaded only if specified in module list.
+load_module = True if not module.__name__.startswith('_') \
+  and (not self.modules \
+   or module.__name__ in self.modules \
+   or 'all' in self.modules) \
+   else False
+
+load_underscore = True if module.__name__.startswith('_') \
+  and module.__name__ in self.modules \
+   else False
+
+if load_module or load_underscore:
 return super(OETestLoader, self).loadTestsFromModule(
 module, use_load_tests)
 else:
-- 
2.10.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] oeqa/core/loader.py: Do not import underscore modules by default

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Underscore modules are meant to be run only when manually added to the test
suite, so far another mechanisms are in place to make this happen with
runtime, sdk, and esdk (mostly in test* bbclasses).

This will add such functionality in the core framework so other specific
frameworks can take use this without adding something else.

[YOCTO 10980]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/core/loader.py | 34 ++
 1 file changed, 30 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
index 74f1117..63a1703 100644
--- a/meta/lib/oeqa/core/loader.py
+++ b/meta/lib/oeqa/core/loader.py
@@ -223,8 +223,21 @@ class OETestLoader(unittest.TestLoader):
 msg = 'Tried to import %s test module but is a built-in'
 raise ImportError(msg % module.__name__)
 
-if not self.modules or "all" in self.modules or \
-module.__name__ in self.modules:
+# Normal test modules are loaded if no modules were specified,
+# if module is in the specified module list or if 'all' is in
+# module list.
+# Underscore modules are loaded only if specified in module list.
+load_module = True if not module.__name__.startswith('_') \
+  and (not self.modules \
+   or module.__name__ in self.modules \
+   or 'all' in self.modules) \
+   else False
+
+load_underscore = True if module.__name__.startswith('_') \
+  and module.__name__ in self.modules \
+   else False
+
+if load_module or load_underscore:
 return super(OETestLoader, self).loadTestsFromModule(
 module, *args, pattern=pattern, **kws)
 else:
@@ -238,8 +251,21 @@ class OETestLoader(unittest.TestLoader):
 msg = 'Tried to import %s test module but is a built-in'
 raise ImportError(msg % module.__name__)
 
-if not self.modules or "all" in self.modules or \
-module.__name__ in self.modules:
+# Normal test modules are loaded if no modules were specified,
+# if module is in the specified module list or if 'all' is in
+# module list.
+# Underscore modules are loaded only if specified in module list.
+load_module = True if not module.__name__.startswith('_') \
+  and (not self.modules \
+   or module.__name__ in self.modules \
+   or 'all' in self.modules) \
+   else False
+
+load_underscore = True if module.__name__.startswith('_') \
+  and module.__name__ in self.modules \
+   else False
+
+if load_module or load_underscore:
 return super(OETestLoader, self).loadTestsFromModule(
 module, use_load_tests)
 else:
-- 
2.10.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/3] oeqa/runtime: Import custom targets

[Mariano Lopez]

On 27/03/17 07:38, Richard Purdie wrote:
> On Fri, 2017-03-24 at 15:06 -0700, mariano.lo...@linux.intel.com wrote:
>> From: Mariano Lopez <mariano.lo...@linux.intel.com>
>>
>> This series allows to import custom targets to be used with runtime
>> testing.
>> Now is possible to have a target in /lib/oeqa/core/target and
>> testimage and testexport can use such target using TEST_TARGET
>> variable.
>>
>> To register a custom target you just need use decorate a target class
>> with
>> registerTarget, and set "targetName" attribute to the name that will
>> be used
>> by TEST_TARGET variable.
> Unfortunately something about this series break oe-selftest:
>
> https://autobuilder.yocto.io/builders/nightly-oe-selftest/builds/229/st
> eps/Running%20oe-selftest/logs/stdio
>
> It doesn't do it in isolation, that test run by itself is fine. When
> run as part of oe-selftest it breaks though. I did bisect it down to
> this series and things have all built fine after I dropped these 3
> patches.
>
> So we'll have to figure out what is happening here...

I'm very sorry that you have to do a bisect to find this out :(

I did run selftest in my workstation to avoid this thing but it seems
it wasn't enough.

The problem is when importing the modules with
oe-test (outside bitbake). I did check it with python 3.4 but I didn't
test it with 3.5 so, I'll start with that.

Regards,
Mariano
>
> Cheers,
>
> Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] oeqa/runtime/context.py: Add automatic target loading

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This adds automatic target loading for runtime testing. It will look for
targets in 'lib/oeqa/core/target' in every layer used in the build.

Introspection was used to achieve this functionality, the idea is to load all
target modules and they will be registered with registerTarget decorator, so
it is less error prone than previous implementation.

testimage and testexport bbclasses were modified to support this. testexport
required more work to find the targets and copy them into the exporting
directory. The executor also needed to remove current choices (simpleremote,
qemu) because now we can have custom targets.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/testexport.bbclass   |  19 ++-
 meta/classes/testimage.bbclass|   4 +-
 meta/lib/oeqa/core/target/base.py |  26 +
 meta/lib/oeqa/runtime/context.py  | 109 --
 4 files changed, 68 insertions(+), 90 deletions(-)

diff --git a/meta/classes/testexport.bbclass b/meta/classes/testexport.bbclass
index 56edda9..25d66ca 100644
--- a/meta/classes/testexport.bbclass
+++ b/meta/classes/testexport.bbclass
@@ -58,8 +58,8 @@ def testexport_main(d):
 logger = logging.getLogger("BitBake")
 
 target = OERuntimeTestContextExecutor.getTarget(
-d.getVar("TEST_TARGET"), None, d.getVar("TEST_TARGET_IP"),
-d.getVar("TEST_SERVER_IP"))
+d.getVar("TEST_TARGET"), d.getVar('BBLAYERS').split(), None,
+d.getVar("TEST_TARGET_IP"), d.getVar("TEST_SERVER_IP"))
 
 host_dumper = OERuntimeTestContextExecutor.getHostDumper(
 d.getVar("testimage_dump_host"), d.getVar("TESTIMAGE_DUMP_DIR"))
@@ -108,6 +108,21 @@ def copy_needed_files(d, tc):
 else:
 shutil.copy2(src, dst)
 
+# Get all target files.
+t_path = os.path.join('lib', 'oeqa', 'core', 'target')
+targets_to_copy = [os.path.join(root, filename)
+   for path in d.getVar('BBLAYERS').split()
+   for root, _, files in os.walk(os.path.join(path, 
t_path))
+   for filename in files
+   if filename.endswith('.py') ]
+
+# Copy all targets.
+export_target_path = os.path.join(export_path, t_path)
+oe.path.remove(export_target_path)
+bb.utils.mkdirhier(export_target_path)
+for f in targets_to_copy:
+shutil.copy2(f, export_target_path)
+
 # Remove cases and just copy the ones specified
 cases_path = os.path.join(export_path, 'lib', 'oeqa', 'runtime', 'cases')
 oe.path.remove(cases_path)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index bfe5714..3199ac4 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -242,8 +242,8 @@ def testimage_main(d):
 
 # the robot dance
 target = OERuntimeTestContextExecutor.getTarget(
-d.getVar("TEST_TARGET"), None, d.getVar("TEST_TARGET_IP"),
-d.getVar("TEST_SERVER_IP"), **target_kwargs)
+d.getVar("TEST_TARGET"), d.getVar('BBLAYERS').split(), None,
+d.getVar("TEST_TARGET_IP"), d.getVar("TEST_SERVER_IP"), 
**target_kwargs)
 
 # test context
 tc = OERuntimeTestContext(td, logger, target, host_dumper,
diff --git a/meta/lib/oeqa/core/target/base.py 
b/meta/lib/oeqa/core/target/base.py
index c886491..426a577 100644
--- a/meta/lib/oeqa/core/target/base.py
+++ b/meta/lib/oeqa/core/target/base.py
@@ -54,3 +54,29 @@ class OETarget(object):
 @abstractmethod
 def copyDirTo(self, localSrc, remoteDst):
 pass
+
+def discover_targets(layer_paths):
+"""
+Imports modules found in 'lib/oeqa/core/target'.
+
+This is used to register targets using registerTarget decorator.
+"""
+
+target_path = 'lib/oeqa/core/target'
+paths = [os.path.join(p, target_path) for p in layer_paths]
+for path in paths:
+files_python = [os.path.join(root, filename)
+for root, _, filenames in os.walk(path)
+for filename in filenames
+if filename.endswith('.py')]
+for f in files_python:
+if '__init__.py' in f:
+continue
+abs_path = os.path.abspath(f)
+for sys_path in sys.path:
+if sys_path in abs_path:
+rel_path = os.path.relpath(abs_path, sys_path)
+break
+
+name = rel_path.replace('.py', '').replace(os.path.sep, '.')
+importlib.import_module(name)
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index c4cd76c..5482a39 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -51,6 +51,10 @@ class

[OE-core] [PATCH 1/3] oeqa: Remove __init__.py files.

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

>From python 3.3 and above, if a module path contains an __init__.py file, just
that path will be checked for module imports; this behavior makes impossible
to extend core functionality using other layers with the current __init__.py
files. You can check a more detailed explanation on:
http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_traps.html

This patch removes the __init__.py files from just from the new framework.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/core/__init__.py| 0
 meta/lib/oeqa/core/cases/__init__.py  | 0
 meta/lib/oeqa/core/decorator/{__init__.py => base.py} | 0
 meta/lib/oeqa/core/decorator/data.py  | 3 +--
 meta/lib/oeqa/core/decorator/depends.py   | 4 +---
 meta/lib/oeqa/core/decorator/oeid.py  | 2 +-
 meta/lib/oeqa/core/decorator/oetag.py | 2 +-
 meta/lib/oeqa/core/decorator/oetimeout.py | 2 +-
 meta/lib/oeqa/core/loader.py  | 2 +-
 meta/lib/oeqa/core/target/{__init__.py => base.py}| 0
 meta/lib/oeqa/core/target/qemu.py | 2 +-
 meta/lib/oeqa/core/target/ssh.py  | 2 +-
 meta/lib/oeqa/core/tests/__init__.py  | 0
 meta/lib/oeqa/core/utils/__init__.py  | 0
 meta/lib/oeqa/runtime/decorator/package.py| 2 +-
 meta/lib/oeqa/runtime/utils/__init__.py   | 0
 meta/lib/oeqa/sdk/__init__.py | 0
 meta/lib/oeqa/sdk/utils/__init__.py   | 0
 meta/lib/oeqa/sdkext/__init__.py  | 0
 19 files changed, 9 insertions(+), 12 deletions(-)
 delete mode 100644 meta/lib/oeqa/core/__init__.py
 delete mode 100644 meta/lib/oeqa/core/cases/__init__.py
 rename meta/lib/oeqa/core/decorator/{__init__.py => base.py} (100%)
 rename meta/lib/oeqa/core/target/{__init__.py => base.py} (100%)
 delete mode 100644 meta/lib/oeqa/core/tests/__init__.py
 delete mode 100644 meta/lib/oeqa/core/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/runtime/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/sdk/__init__.py
 delete mode 100644 meta/lib/oeqa/sdk/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/sdkext/__init__.py

diff --git a/meta/lib/oeqa/core/__init__.py b/meta/lib/oeqa/core/__init__.py
deleted file mode 100644
index e69de29..000
diff --git a/meta/lib/oeqa/core/cases/__init__.py 
b/meta/lib/oeqa/core/cases/__init__.py
deleted file mode 100644
index e69de29..000
diff --git a/meta/lib/oeqa/core/decorator/__init__.py 
b/meta/lib/oeqa/core/decorator/base.py
similarity index 100%
rename from meta/lib/oeqa/core/decorator/__init__.py
rename to meta/lib/oeqa/core/decorator/base.py
diff --git a/meta/lib/oeqa/core/decorator/data.py 
b/meta/lib/oeqa/core/decorator/data.py
index ff7bdd9..658e8a2 100644
--- a/meta/lib/oeqa/core/decorator/data.py
+++ b/meta/lib/oeqa/core/decorator/data.py
@@ -2,8 +2,7 @@
 # Released under the MIT license (see COPYING.MIT)
 
 from oeqa.core.exception import OEQAMissingVariable
-
-from . import OETestDecorator, registerDecorator
+from oeqa.core.decorator.base import OETestDecorator, registerDecorator
 
 def has_feature(td, feature):
 """
diff --git a/meta/lib/oeqa/core/decorator/depends.py 
b/meta/lib/oeqa/core/decorator/depends.py
index 195711c..608b06a 100644
--- a/meta/lib/oeqa/core/decorator/depends.py
+++ b/meta/lib/oeqa/core/decorator/depends.py
@@ -2,10 +2,8 @@
 # Released under the MIT license (see COPYING.MIT)
 
 from unittest import SkipTest
-
 from oeqa.core.exception import OEQADependency
-
-from . import OETestDiscover, registerDecorator
+from oeqa.core.decorator.base import OETestDiscover, registerDecorator
 
 def _add_depends(registry, case, depends):
 module_name = case.__module__
diff --git a/meta/lib/oeqa/core/decorator/oeid.py 
b/meta/lib/oeqa/core/decorator/oeid.py
index ea8017a..a5bee52 100644
--- a/meta/lib/oeqa/core/decorator/oeid.py
+++ b/meta/lib/oeqa/core/decorator/oeid.py
@@ -1,8 +1,8 @@
 # Copyright (C) 2016 Intel Corporation
 # Released under the MIT license (see COPYING.MIT)
 
-from . import OETestFilter, registerDecorator
 from oeqa.core.utils.misc import intToList
+from oeqa.core.decorator.base import OETestFilter, registerDecorator
 
 def _idFilter(oeid, filters):
  return False if oeid in filters else True
diff --git a/meta/lib/oeqa/core/decorator/oetag.py 
b/meta/lib/oeqa/core/decorator/oetag.py
index ad38ab7..6a36428 100644
--- a/meta/lib/oeqa/core/decorator/oetag.py
+++ b/meta/lib/oeqa/core/decorator/oetag.py
@@ -1,8 +1,8 @@
 # Copyright (C) 2016 Intel Corporation
 # Released under the MIT license (see COPYING.MIT)
 
-from . import OETestFilter, registerDecorator
 from oeqa.core.utils.misc import strToList
+from oeqa.core.decorator.base import OETestFilter, registerDecorator
 
 def _tagFilte

[OE-core] [PATCH 2/3] oeqa/target: Add decorator to register targets

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This adds a way to register targets that can be used with runtime testing. To
do this just decorate a target class with registerTarget, and set "targetName"
attribute to the name that will be used by TEST_TARGET variable.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/core/target/base.py | 23 +++
 meta/lib/oeqa/core/target/qemu.py |  5 +
 meta/lib/oeqa/core/target/ssh.py  |  6 +-
 3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/core/target/base.py 
b/meta/lib/oeqa/core/target/base.py
index d2468bc..c886491 100644
--- a/meta/lib/oeqa/core/target/base.py
+++ b/meta/lib/oeqa/core/target/base.py
@@ -1,8 +1,31 @@
 # Copyright (C) 2016 Intel Corporation
 # Released under the MIT license (see COPYING.MIT)
+import os
+import sys
+import importlib
 
 from abc import abstractmethod
 
+# Used to keep record of registered targets, it
+# uses class' targetName as the key to the class.
+targetClasses = {}
+
+def registerTarget(obj):
+""" Use as decorator to register targets for runtime testing """
+
+if (obj.targetName in targetClasses and
+obj.__name__ != targetClasses[obj.targetName].__name__):
+
+msg = ('Tried to register %s as "%s" that is used by %s' %
+   (obj, obj.targetName, targetClasses[obj.targetName]))
+raise ImportError(msg)
+
+if not issubclass(obj, OETarget):
+raise TypeError('%s must inherit from OETarget' % obj)
+
+targetClasses[obj.targetName] = obj
+return obj
+
 class OETarget(object):
 
 def __init__(self, logger, *args, **kwargs):
diff --git a/meta/lib/oeqa/core/target/qemu.py 
b/meta/lib/oeqa/core/target/qemu.py
index 261c466..d1b6bf5 100644
--- a/meta/lib/oeqa/core/target/qemu.py
+++ b/meta/lib/oeqa/core/target/qemu.py
@@ -6,12 +6,17 @@ import sys
 import signal
 import time
 
+from oeqa.core.target.base import registerTarget
 from oeqa.core.target.ssh import OESSHTarget
 from oeqa.utils.qemurunner import QemuRunner
 
 supported_fstypes = ['ext3', 'ext4', 'cpio.gz', 'wic', 'elf']
 
+@registerTarget
 class OEQemuTarget(OESSHTarget):
+
+targetName = 'qemu'
+
 def __init__(self, logger, ip, server_ip, timeout=300, user='root',
 port=None, machine='', rootfs='', kernel='', kvm=False,
 dump_dir='', dump_host_cmds='', display='', bootlog='',
diff --git a/meta/lib/oeqa/core/target/ssh.py b/meta/lib/oeqa/core/target/ssh.py
index 035965a..d973058 100644
--- a/meta/lib/oeqa/core/target/ssh.py
+++ b/meta/lib/oeqa/core/target/ssh.py
@@ -7,9 +7,13 @@ import select
 import logging
 import subprocess
 
-from oeqa.core.target.base import OETarget
+from oeqa.core.target.base import OETarget, registerTarget
 
+@registerTarget
 class OESSHTarget(OETarget):
+
+targetName = 'simpleremote'
+
 def __init__(self, logger, ip, server_ip, timeout=300, user='root',
  port=None, **kwargs):
 if not logger:
-- 
2.10.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] oeqa/runtime: Import custom targets

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This series allows to import custom targets to be used with runtime testing.
Now is possible to have a target in /lib/oeqa/core/target and
testimage and testexport can use such target using TEST_TARGET variable.

To register a custom target you just need use decorate a target class with
registerTarget, and set "targetName" attribute to the name that will be used
by TEST_TARGET variable.

The following changes since commit 9fe7a69535f9443175da8289633a761f58c372ff:

  bitbake: bitbake-diffsigs: Add debug support (2017-03-23 13:19:50 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/oeqa_target
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/oeqa_target

Mariano Lopez (3):
  oeqa: Remove __init__.py files.
  oeqa/target: Add decorator to register targets
  oeqa/runtime/context.py: Add automatic target loading

 meta/classes/testexport.bbclass|  19 +++-
 meta/classes/testimage.bbclass |   4 +-
 meta/lib/oeqa/core/__init__.py |   0
 meta/lib/oeqa/core/cases/__init__.py   |   0
 .../oeqa/core/decorator/{__init__.py => base.py}   |   0
 meta/lib/oeqa/core/decorator/data.py   |   3 +-
 meta/lib/oeqa/core/decorator/depends.py|   4 +-
 meta/lib/oeqa/core/decorator/oeid.py   |   2 +-
 meta/lib/oeqa/core/decorator/oetag.py  |   2 +-
 meta/lib/oeqa/core/decorator/oetimeout.py  |   2 +-
 meta/lib/oeqa/core/loader.py   |   2 +-
 meta/lib/oeqa/core/target/__init__.py  |  33 ---
 meta/lib/oeqa/core/target/base.py  |  82 
 meta/lib/oeqa/core/target/qemu.py  |   7 +-
 meta/lib/oeqa/core/target/ssh.py   |   6 +-
 meta/lib/oeqa/core/tests/__init__.py   |   0
 meta/lib/oeqa/core/utils/__init__.py   |   0
 meta/lib/oeqa/runtime/context.py   | 109 +
 meta/lib/oeqa/runtime/decorator/package.py |   2 +-
 meta/lib/oeqa/runtime/utils/__init__.py|   0
 meta/lib/oeqa/sdk/__init__.py  |   0
 meta/lib/oeqa/sdk/utils/__init__.py|   0
 meta/lib/oeqa/sdkext/__init__.py   |   0
 23 files changed, 142 insertions(+), 135 deletions(-)
 delete mode 100644 meta/lib/oeqa/core/__init__.py
 delete mode 100644 meta/lib/oeqa/core/cases/__init__.py
 rename meta/lib/oeqa/core/decorator/{__init__.py => base.py} (100%)
 delete mode 100644 meta/lib/oeqa/core/target/__init__.py
 create mode 100644 meta/lib/oeqa/core/target/base.py
 delete mode 100644 meta/lib/oeqa/core/tests/__init__.py
 delete mode 100644 meta/lib/oeqa/core/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/runtime/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/sdk/__init__.py
 delete mode 100644 meta/lib/oeqa/sdk/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/sdkext/__init__.py

-- 
2.10.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 0/2] OEQA loader improvment

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

The first adds more verbosity when a class doesn't inherit
from the expected test class.

The second patch raises an error when trying to import a test
with the same module name as a built-in module.

Change in v2:

- Rebased to current master

The following changes since commit 3c83b56309ab419f8cda72c0711479f60f61439a:

  bitbake: fetch2/svn: change 'rsh' parameter to 'ssh' (2017-02-23 12:50:17 
-0800)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/bug10978
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug10978

Mariano Lopez (2):
  oeqa/core/loader.py: Give meaningful error when failed to load classes
  oeqa/core/loader.py: Avoid importing tests with built-ins name

 meta/lib/oeqa/core/loader.py | 19 +++
 1 file changed, 15 insertions(+), 4 deletions(-)

-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 2/2] oeqa/core/loader.py: Avoid importing tests with built-ins name

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

If importing a test with the same name as a built-in module,
it will silently import the built-in and check for tests in
built-in module. This happened with syslog module in debian
based machines, so add a raise to avoid this behavior.

[YOCTO #10978]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/core/loader.py | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
index b9ba923..74f1117 100644
--- a/meta/lib/oeqa/core/loader.py
+++ b/meta/lib/oeqa/core/loader.py
@@ -216,6 +216,13 @@ class OETestLoader(unittest.TestLoader):
 # use_load_tests deprecation via *args and **kws.  See issue 16662.
 if sys.version_info >= (3,5):
 def loadTestsFromModule(self, module, *args, pattern=None, **kws):
+"""
+Returns a suite of all tests cases contained in module.
+"""
+if module.__name__ in sys.builtin_module_names:
+msg = 'Tried to import %s test module but is a built-in'
+raise ImportError(msg % module.__name__)
+
 if not self.modules or "all" in self.modules or \
 module.__name__ in self.modules:
 return super(OETestLoader, self).loadTestsFromModule(
@@ -227,6 +234,10 @@ class OETestLoader(unittest.TestLoader):
 """
 Returns a suite of all tests cases contained in module.
 """
+if module.__name__ in sys.builtin_module_names:
+msg = 'Tried to import %s test module but is a built-in'
+raise ImportError(msg % module.__name__)
+
 if not self.modules or "all" in self.modules or \
 module.__name__ in self.modules:
 return super(OETestLoader, self).loadTestsFromModule(
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 1/2] oeqa/core/loader.py: Give meaningful error when failed to load classes

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

With this we get the class that is actually having the problem,
not just a TypeError with an unknown class causing the error.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/core/loader.py | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
index a380325..b9ba923 100644
--- a/meta/lib/oeqa/core/loader.py
+++ b/meta/lib/oeqa/core/loader.py
@@ -171,11 +171,11 @@ class OETestLoader(unittest.TestLoader):
 """
 if issubclass(testCaseClass, unittest.suite.TestSuite):
 raise TypeError("Test cases should not be derived from TestSuite." 
\
-" Maybe you meant to derive from TestCase?")
+" Maybe you meant to derive %s from TestCase?" 
\
+% testCaseClass.__name__)
 if not issubclass(testCaseClass, self.caseClass):
-raise TypeError("Test cases need to be derived from %s" % \
-self.caseClass.__name__)
-
+raise TypeError("Test %s is not derived from %s" % \
+(testCaseClass.__name__, self.caseClass.__name__))
 
 testCaseNames = self.getTestCaseNames(testCaseClass)
 if not testCaseNames and hasattr(testCaseClass, 'runTest'):
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] selftest/eSDK.py: Cleanup when there is an error in setUpClass

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Lately autobuilders are experiencing hangs with selftest,
it seems it is cause if an error happens in setUpClass
method of oeSDKExtSelfTest class because HTTP server
keeps running in background.

This patch will ensure tearDownClass will be run if there
is an error in setUpClass.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/selftest/eSDK.py | 46 +++---
 1 file changed, 25 insertions(+), 21 deletions(-)

diff --git a/meta/lib/oeqa/selftest/eSDK.py b/meta/lib/oeqa/selftest/eSDK.py
index ee1ca6a..b68e997 100644
--- a/meta/lib/oeqa/selftest/eSDK.py
+++ b/meta/lib/oeqa/selftest/eSDK.py
@@ -64,7 +64,7 @@ class oeSDKExtSelfTest(oeSelfTest):
 runCmd("%s -y -d \"%s\"" % (cls.ext_sdk_path, cls.tmpdir_eSDKQA))
 
 cls.env_eSDK = oeSDKExtSelfTest.get_esdk_environment('', 
cls.tmpdir_eSDKQA)
-   
+
 sstate_config="""
 SDK_LOCAL_CONF_WHITELIST = "SSTATE_MIRRORS"
 SSTATE_MIRRORS =  "file://.* http://%s/PATH;
@@ -73,37 +73,41 @@ CORE_IMAGE_EXTRA_INSTALL = "perl"
 
 with open(os.path.join(cls.tmpdir_eSDKQA, 'conf', 'local.conf'), 'a+') 
as f:
 f.write(sstate_config)
-
 
 @classmethod
 def setUpClass(cls):
-# Start to serve sstate dir
-sstate_dir = get_bb_var('SSTATE_DIR')
-cls.http_service = HTTPService(sstate_dir)
-cls.http_service.start()
+# If there is an exception in setUpClass it will not run tearDownClass
+# method and it leaves HTTP server running forever, so we need to be
+# sure tearDownClass is run.
+try:
+cls.tmpdir_eSDKQA = tempfile.mkdtemp(prefix='eSDKQA')
 
-cls.http_url = "http://127.0.0.1:%d; % cls.http_service.port
- 
-cls.image = 'core-image-minimal'
+# Start to serve sstate dir
+sstate_dir = get_bb_var('SSTATE_DIR')
+cls.http_service = HTTPService(sstate_dir)
+cls.http_url = "http://127.0.0.1:%d; % cls.http_service.port
+cls.http_service.start()
 
-cls.tmpdir_eSDKQA = tempfile.mkdtemp(prefix='eSDKQA')
-oeSDKExtSelfTest.generate_eSDK(cls.image)
+cls.image = 'core-image-minimal'
+oeSDKExtSelfTest.generate_eSDK(cls.image)
 
-# Install eSDK
-cls.ext_sdk_path = oeSDKExtSelfTest.get_eSDK_toolchain(cls.image)
-runCmd("%s -y -d \"%s\"" % (cls.ext_sdk_path, cls.tmpdir_eSDKQA))
+# Install eSDK
+cls.ext_sdk_path = oeSDKExtSelfTest.get_eSDK_toolchain(cls.image)
+runCmd("%s -y -d \"%s\"" % (cls.ext_sdk_path, cls.tmpdir_eSDKQA))
 
-cls.env_eSDK = oeSDKExtSelfTest.get_esdk_environment('', 
cls.tmpdir_eSDKQA)
+cls.env_eSDK = oeSDKExtSelfTest.get_esdk_environment('', 
cls.tmpdir_eSDKQA)
 
-# Configure eSDK to use sstate mirror from poky
-sstate_config="""
+# Configure eSDK to use sstate mirror from poky
+sstate_config="""
 SDK_LOCAL_CONF_WHITELIST = "SSTATE_MIRRORS"
 SSTATE_MIRRORS =  "file://.* http://%s/PATH;
-""" % cls.http_url
-with open(os.path.join(cls.tmpdir_eSDKQA, 'conf', 'local.conf'), 'a+') 
as f:
-f.write(sstate_config)
+""" % cls.http_url
+with open(os.path.join(cls.tmpdir_eSDKQA, 'conf', 'local.conf'), 
'a+') as f:
+f.write(sstate_config)
+except:
+cls.tearDownClass()
+raise
 
-  
 @classmethod
 def tearDownClass(cls):
 shutil.rmtree(cls.tmpdir_eSDKQA)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 2/2] selftest: Optimize get_bb_var use

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

get_bb_var calls bitbake every time it is used and every call
would take about 7 seconds. There are tests that calls get_bb_var
several times when they can use get_bb_vars. Also there are tests
that calls it to fetch the same variable over and over again.

This will optimize the use of get_bb_var and get_bb_vars for a
little speed up in the tests.

[YOCTO #11037]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/selftest/archiver.py |  7 +--
 meta/lib/oeqa/selftest/bbtests.py  | 24 +
 meta/lib/oeqa/selftest/buildhistory.py |  5 +-
 meta/lib/oeqa/selftest/buildoptions.py | 10 ++--
 meta/lib/oeqa/selftest/devtool.py  | 57 -
 meta/lib/oeqa/selftest/eSDK.py | 21 
 meta/lib/oeqa/selftest/liboe.py| 21 
 meta/lib/oeqa/selftest/manifest.py |  7 +--
 meta/lib/oeqa/selftest/pkgdata.py  | 11 ++--
 meta/lib/oeqa/selftest/prservice.py|  9 ++--
 meta/lib/oeqa/selftest/recipetool.py   | 34 +++--
 meta/lib/oeqa/selftest/runtime-test.py | 15 --
 meta/lib/oeqa/selftest/signing.py  | 12 +++--
 meta/lib/oeqa/selftest/sstate.py   | 20 +---
 meta/lib/oeqa/selftest/sstatetests.py  | 92 ++
 meta/lib/oeqa/selftest/tinfoil.py  |  2 +-
 meta/lib/oeqa/selftest/wic.py  | 39 +++---
 17 files changed, 208 insertions(+), 178 deletions(-)

diff --git a/meta/lib/oeqa/selftest/archiver.py 
b/meta/lib/oeqa/selftest/archiver.py
index 97b6f5b..d7f215c 100644
--- a/meta/lib/oeqa/selftest/archiver.py
+++ b/meta/lib/oeqa/selftest/archiver.py
@@ -1,5 +1,5 @@
 from oeqa.selftest.base import oeSelfTest
-from oeqa.utils.commands import bitbake, get_bb_var
+from oeqa.utils.commands import bitbake, get_bb_vars
 from oeqa.utils.decorators import testcase
 import glob
 import os
@@ -28,10 +28,11 @@ class Archiver(oeSelfTest):
 features += 'COPYLEFT_PN_EXCLUDE = "%s"\n' % exclude_recipe
 self.write_config(features)
 
-shutil.rmtree(get_bb_var('TMPDIR'))
+bitbake('-c clean %s %s' % (include_recipe, exclude_recipe))
 bitbake("%s %s" % (include_recipe, exclude_recipe))
 
-src_path = os.path.join(get_bb_var('DEPLOY_DIR_SRC'), 
get_bb_var('TARGET_SYS'))
+bb_vars = get_bb_vars(['DEPLOY_DIR_SRC', 'TARGET_SYS'])
+src_path = os.path.join(bb_vars['DEPLOY_DIR_SRC'], 
bb_vars['TARGET_SYS'])
 
 # Check that include_recipe was included
 included_present = len(glob.glob(src_path + '/%s-*' % include_recipe))
diff --git a/meta/lib/oeqa/selftest/bbtests.py 
b/meta/lib/oeqa/selftest/bbtests.py
index 7675563..4d9eec1 100644
--- a/meta/lib/oeqa/selftest/bbtests.py
+++ b/meta/lib/oeqa/selftest/bbtests.py
@@ -3,7 +3,7 @@ import re
 
 import oeqa.utils.ftools as ftools
 from oeqa.selftest.base import oeSelfTest
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars
 from oeqa.utils.decorators import testcase
 
 class BitbakeTests(oeSelfTest):
@@ -78,9 +78,10 @@ class BitbakeTests(oeSelfTest):
 # test 1 from bug 5875
 test_recipe = 'zlib'
 test_data = "Microsoft Made No Profit From Anyone's Zunes Yo"
-image_dir = get_bb_var('D', test_recipe)
-pkgsplit_dir = get_bb_var('PKGDEST', test_recipe)
-man_dir = get_bb_var('mandir', test_recipe)
+bb_vars = get_bb_vars(['D', 'PKGDEST', 'mandir'], test_recipe)
+image_dir = bb_vars['D']
+pkgsplit_dir = bb_vars['PKGDEST']
+man_dir = bb_vars['mandir']
 
 bitbake('-c clean %s' % test_recipe)
 bitbake('-c package -f %s' % test_recipe)
@@ -121,8 +122,9 @@ class BitbakeTests(oeSelfTest):
 @testcase(899)
 def test_image_manifest(self):
 bitbake('core-image-minimal')
-deploydir = get_bb_var("DEPLOY_DIR_IMAGE", target="core-image-minimal")
-imagename = get_bb_var("IMAGE_LINK_NAME", target="core-image-minimal")
+bb_vars = get_bb_vars(["DEPLOY_DIR_IMAGE", "IMAGE_LINK_NAME"], 
"core-image-minimal")
+deploydir = bb_vars["DEPLOY_DIR_IMAGE"]
+imagename = bb_vars["IMAGE_LINK_NAME"]
 manifest = os.path.join(deploydir, imagename + ".manifest")
 self.assertTrue(os.path.islink(manifest), msg="No manifest file 
created for image. It should have been created in %s" % manifest)
 
@@ -160,8 +162,9 @@ SSTATE_DIR = \"${TOPDIR}/download-selftest\"
 result = bitbake('-c fetch aspell', ignore_status=True)
 self.delete_recipeinc('aspell')
 self.assertEqual(result.status, 0, msg = "Couldn't fetch aspell. %s" % 
result.output)
-self.assertTrue(os.path.isfile(os.path.join(get_bb_var("DL_DIR"), 
'test-as

[OE-core] [PATCHv2 1/2] selftest: Avoid sstate corruption by calling cleansstate

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Currently selftest doesn't use sstates because some tests
clean sstate cache; using sstates would give a performance
boost instead of building everything from scratch.

With this sstates are not corrupted using different methods
depending on tests:

devtool: These tests needed to delete the cache so SSTATE_DIR
as SSTATE_MIRRORS and set a temporal SSTATE_DIR.

sstatetests: This module already used a temporal SSTATE_DIR, so
just set up the SSTATE_MIRRORS.

Rest: Removed cleansstate, some of them required to force a
certain task, others were just removed or changed for another
task.

[YOCTO #10929]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/selftest/bbtests.py  |  6 ++
 meta/lib/oeqa/selftest/buildoptions.py | 22 ++
 meta/lib/oeqa/selftest/devtool.py  | 18 +-
 meta/lib/oeqa/selftest/oescripts.py|  8 ++--
 meta/lib/oeqa/selftest/prservice.py|  3 ---
 meta/lib/oeqa/selftest/recipetool.py   |  5 -
 meta/lib/oeqa/selftest/signing.py  |  8 
 meta/lib/oeqa/selftest/sstatetests.py  |  6 +++---
 8 files changed, 38 insertions(+), 38 deletions(-)

diff --git a/meta/lib/oeqa/selftest/bbtests.py 
b/meta/lib/oeqa/selftest/bbtests.py
index fccb395..7675563 100644
--- a/meta/lib/oeqa/selftest/bbtests.py
+++ b/meta/lib/oeqa/selftest/bbtests.py
@@ -37,7 +37,6 @@ class BitbakeTests(oeSelfTest):
 
 @testcase(103)
 def test_local_sstate(self):
-bitbake('m4-native -ccleansstate')
 bitbake('m4-native')
 bitbake('m4-native -cclean')
 result = bitbake('m4-native')
@@ -83,8 +82,8 @@ class BitbakeTests(oeSelfTest):
 pkgsplit_dir = get_bb_var('PKGDEST', test_recipe)
 man_dir = get_bb_var('mandir', test_recipe)
 
-bitbake('-c cleansstate %s' % test_recipe)
-bitbake(test_recipe)
+bitbake('-c clean %s' % test_recipe)
+bitbake('-c package -f %s' % test_recipe)
 self.add_command_to_tearDown('bitbake -c clean %s' % test_recipe)
 
 man_file = os.path.join(image_dir + man_dir, 'man3/zlib.3')
@@ -103,7 +102,6 @@ class BitbakeTests(oeSelfTest):
 # test 2 from bug 5875
 test_recipe = 'zlib'
 
-bitbake('-c cleansstate %s' % test_recipe)
 bitbake(test_recipe)
 self.add_command_to_tearDown('bitbake -c clean %s' % test_recipe)
 
diff --git a/meta/lib/oeqa/selftest/buildoptions.py 
b/meta/lib/oeqa/selftest/buildoptions.py
index d40eb00..28734fd 100644
--- a/meta/lib/oeqa/selftest/buildoptions.py
+++ b/meta/lib/oeqa/selftest/buildoptions.py
@@ -36,9 +36,8 @@ class ImageOptionsTests(oeSelfTest):
 p = get_bb_var('SYSROOT_DESTDIR', 'ccache-native') + 
get_bb_var('bindir', 'ccache-native') + "/" + "ccache"
 self.assertTrue(os.path.isfile(p), msg = "No ccache found (%s)" % p)
 self.write_config('INHERIT += "ccache"')
-bitbake("m4 -c cleansstate")
-bitbake("m4 -c compile")
-self.addCleanup(bitbake, 'ccache-native -ccleansstate')
+self.add_command_to_tearDown('bitbake -c clean m4')
+bitbake("m4 -f -c compile")
 res = runCmd("grep ccache %s" % 
(os.path.join(get_bb_var("WORKDIR","m4"),"temp/log.do_compile")), 
ignore_status=True)
 self.assertEqual(0, res.status, msg="No match for ccache in m4 
log.do_compile. For further details: %s" % 
os.path.join(get_bb_var("WORKDIR","m4"),"temp/log.do_compile"))
 
@@ -75,14 +74,14 @@ class SanityOptionsTest(oeSelfTest):
 
 @testcase(927)
 def test_options_warnqa_errorqa_switch(self):
-bitbake("xcursor-transparent-theme -ccleansstate")
 
 self.write_config("INHERIT_remove = \"report-error\"")
 if "packages-list" not in get_bb_var("ERROR_QA"):
 self.append_config("ERROR_QA_append = \" packages-list\"")
 
 self.write_recipeinc('xcursor-transparent-theme', 'PACKAGES += 
\"${PN}-dbg\"')
-res = bitbake("xcursor-transparent-theme", ignore_status=True)
+self.add_command_to_tearDown('bitbake -c clean 
xcursor-transparent-theme')
+res = bitbake("xcursor-transparent-theme -f -c package", 
ignore_status=True)
 self.delete_recipeinc('xcursor-transparent-theme')
 line = self.getline(res, "QA Issue: xcursor-transparent-theme-dbg is 
listed in PACKAGES multiple times, this leads to packaging errors.")
 self.assertTrue(line and line.startswith("ERROR:"), msg=res.output)
@@ -90,8 +89,7 @@ class SanityOptionsTest(oeSelfTest):
 self.write_recipeinc('xcursor-transparent-theme', 'PACKAGES += 
\"${PN}-dbg\"')
 self.append_config('ERROR_QA_remove = "pa

[OE-core] [PATCHv2 0/2] Improvements to oe-selftest

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This series includes to main improvements;
- Remove use of cleansstate tasks from tests in order to use shared
  states in the autobuilders
- Optimize use of get_bb_var to avoid calling bitbake -e over and
  over again

This has been tested in local autobuilders.

This series requires:

- https://patchwork.openembedded.org/series/5191/

This series supersedes:

- https://patchwork.openembedded.org/series/5101/
- https://patchwork.openembedded.org/series/5122/

Changes in v2:
- Use 'bitbake -c clean ' after using --force to avoid
  complains from bitbake.

The following changes since commit d9b18c28568432e0710099868576b12f23d502cf:

  insane.bbclass: Add prepare_recipe_sysroot dependency for package_qa 
(2017-02-21 13:52:49 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/bug10929
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug10929

Mariano Lopez (2):
  selftest: Avoid sstate corruption by calling cleansstate
  selftest: Optimize get_bb_var use

 meta/lib/oeqa/selftest/archiver.py |  7 +--
 meta/lib/oeqa/selftest/bbtests.py  | 30 ++-
 meta/lib/oeqa/selftest/buildhistory.py |  5 +-
 meta/lib/oeqa/selftest/buildoptions.py | 32 +--
 meta/lib/oeqa/selftest/devtool.py  | 75 +-
 meta/lib/oeqa/selftest/eSDK.py | 21 
 meta/lib/oeqa/selftest/liboe.py| 21 
 meta/lib/oeqa/selftest/manifest.py |  7 +--
 meta/lib/oeqa/selftest/oescripts.py|  8 +--
 meta/lib/oeqa/selftest/pkgdata.py  | 11 ++--
 meta/lib/oeqa/selftest/prservice.py| 12 ++---
 meta/lib/oeqa/selftest/recipetool.py   | 39 +++---
 meta/lib/oeqa/selftest/runtime-test.py | 15 --
 meta/lib/oeqa/selftest/signing.py  | 20 +++
 meta/lib/oeqa/selftest/sstate.py   | 20 ---
 meta/lib/oeqa/selftest/sstatetests.py  | 98 ++
 meta/lib/oeqa/selftest/tinfoil.py  |  2 +-
 meta/lib/oeqa/selftest/wic.py  | 39 +++---
 18 files changed, 246 insertions(+), 216 deletions(-)

-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCHv2 1/1] insane.bbclass: Add missing dependencies

[Mariano Lopez]

On 15/02/17 22:02, Richard Purdie wrote:
> On Fri, 2017-02-10 at 14:19 -0600, mariano.lo...@linux.intel.com wrote:
>> From: Mariano Lopez <mariano.lo...@linux.intel.com>
>>
>> package_qa task requires some tools installed in sysroot; with
>> the introduction of recipe specific sysroot this task won't
>> have such tools installed if it's forced to run.
>>
>> Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
>> ---
>>  meta/classes/insane.bbclass | 5 -
>>  1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/insane.bbclass
>> b/meta/classes/insane.bbclass
>> index 3cf86c1..0167b14 100644
>> --- a/meta/classes/insane.bbclass
>> +++ b/meta/classes/insane.bbclass
>> @@ -1223,9 +1223,12 @@ python do_package_qa () {
>>  bb.note("DONE with PACKAGE QA")
>>  }
>>  
>> +# binutils is used for most checks, so need to set as dependency
>> +# POPULATESYSROOTDEPS is defined in staging class.
>> +do_package_qa[depends] += "${POPULATESYSROOTDEPS}"
>>  do_package_qa[vardepsexclude] = "BB_TASKDEPDATA"
>>  do_package_qa[rdeptask] = "do_packagedata"
>> -addtask do_package_qa after do_packagedata do_package before
>> do_build
>> +addtask do_package_qa after do_prepare_recipe_sysroot do_packagedata
>> do_package before do_build
> This is close but I don't think you need this last addtask change, I
> think it should work just fine with the added [depends]?

Indeed, it works just with [depends]. Just sent v3 of this patch.

Mariano

>
> Cheers,
>
> Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv3 1/1] insane.bbclass: Add missing dependencies

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

package_qa task requires some tools installed in sysroot; with
the introduction of recipe specific sysroot this task won't
have such tools installed if it's forced to run.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/insane.bbclass | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index e8e54f0..6a34bd5 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -1221,6 +1221,9 @@ python do_package_qa () {
 bb.note("DONE with PACKAGE QA")
 }
 
+# binutils is used for most checks, so need to set as dependency
+# POPULATESYSROOTDEPS is defined in staging class.
+do_package_qa[depends] += "${POPULATESYSROOTDEPS}"
 do_package_qa[vardepsexclude] = "BB_TASKDEPDATA"
 do_package_qa[rdeptask] = "do_packagedata"
 addtask do_package_qa after do_packagedata do_package before do_build
-- 
2.10.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv3 0/1] insane.bbclass: Add missing dependencies

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Changes in v2:

- Add binutils as dependency too

Changes in v3:

- Remove prepare_recipe_sysroot task dependency as recommended by Richard 

The following changes since commit e436a6398684d2872cb541f1cfb0f67b3618d15a:

  layer.conf: bump version for change in eSDK selftest behaviour (2017-02-19 
09:39:03 -0800)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/insane_fix
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/insane_fix

Mariano Lopez (1):
  insane.bbclass: Add missing dependencies

 meta/classes/insane.bbclass | 3 +++
 1 file changed, 3 insertions(+)

-- 
2.10.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] oeqa/runtime/context.py: Fix use of getTarget() with testexport

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

The idea on getTarget is to use kwargs to send custom variables
to different targets, instead of this, a new variable was added
(just used for custom targets) and this broke testexport. So
in order to fix it, just add the custom variable to kwargs.

This fixes the use of getTarget() in testexport class that was
introduced in 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/testimage.bbclass|  6 +-
 meta/lib/oeqa/core/target/qemu.py |  2 +-
 meta/lib/oeqa/runtime/context.py  | 11 ++-
 3 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 09cc6d2..82557a8 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -229,6 +229,10 @@ def testimage_main(d):
   'kvm' : kvm,
 }
 
+# TODO: Currently BBPATH is needed for custom loading of targets.
+# It would be better to find these modules using instrospection.
+target_kwargs['target_modules_path'] = d.getVar('BBPATH')
+
 # runtime use network for download projects for build
 export_proxies(d)
 
@@ -239,7 +243,7 @@ def testimage_main(d):
 
 # the robot dance
 target = OERuntimeTestContextExecutor.getTarget(
-d.getVar("TEST_TARGET"), d.getVar("BBPATH"), None, 
d.getVar("TEST_TARGET_IP"),
+d.getVar("TEST_TARGET"), None, d.getVar("TEST_TARGET_IP"),
 d.getVar("TEST_SERVER_IP"), **target_kwargs)
 
 # test context
diff --git a/meta/lib/oeqa/core/target/qemu.py 
b/meta/lib/oeqa/core/target/qemu.py
index 9d3f68c..2dc521c 100644
--- a/meta/lib/oeqa/core/target/qemu.py
+++ b/meta/lib/oeqa/core/target/qemu.py
@@ -15,7 +15,7 @@ class OEQemuTarget(OESSHTarget):
 def __init__(self, logger, ip, server_ip, timeout=300, user='root',
 port=None, machine='', rootfs='', kernel='', kvm=False,
 dump_dir='', dump_host_cmds='', display='', bootlog='',
-tmpdir='', dir_image='', boottime=60):
+tmpdir='', dir_image='', boottime=60, **kwargs):
 
 super(OEQemuTarget, self).__init__(logger, ip, server_ip, timeout,
 user, port)
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index ea1b4a6..c4cd76c 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -89,7 +89,7 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
 help="Qemu boot configuration, only needed when target_type is 
QEMU.")
 
 @staticmethod
-def getTarget(target_type, target_modules_path, logger, target_ip, 
server_ip, **kwargs):
+def getTarget(target_type, logger, target_ip, server_ip, **kwargs):
 target = None
 
 if target_type == 'simpleremote':
@@ -97,8 +97,17 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
 elif target_type == 'qemu':
 target = OEQemuTarget(logger, target_ip, server_ip, **kwargs)
 else:
+# XXX: This code uses the old naming convention for controllers and
+# targets, the idea it is to leave just targets as the controller
+# most of the time was just a wrapper.
+# XXX: This code tries to import modules from lib/oeqa/controllers
+# directory and treat them as controllers, it will less error prone
+# to use introspection to load such modules.
+# XXX: Don't base your targets on this code it will be refactored
+# in the near future.
 # Custom target module loading
 try:
+target_modules_path = kwargs.get('target_modules_path', '')
 controller = 
OERuntimeTestContextExecutor.getControllerModule(target_type, 
target_modules_path)
 target = controller(logger, target_ip, server_ip, **kwargs)
 except ImportError as e:
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] oeqa/core/loader.py: Avoid importing tests with built-ins name

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

If importing a test with the same name as a built-in module,
it will silently import the built-in and check for tests in
built-in module. This happened with syslog module in debian
based machines, so add a raise to avoid this behavior.

[YOCTO #10978]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/core/loader.py | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
index bf13a5a..a28512b 100644
--- a/meta/lib/oeqa/core/loader.py
+++ b/meta/lib/oeqa/core/loader.py
@@ -217,6 +217,13 @@ class OETestLoader(unittest.TestLoader):
 # use_load_tests deprecation via *args and **kws.  See issue 16662.
 if sys.version_info >= (3,5):
 def loadTestsFromModule(self, module, *args, pattern=None, **kws):
+"""
+Returns a suite of all tests cases contained in module.
+"""
+if module.__name__ in sys.builtin_module_names:
+msg = 'Tried to import %s test module but is a built-in'
+raise ImportError(msg % module.__name__)
+
 if not self.modules or "all" in self.modules or \
 module.__name__ in self.modules:
 return super(OETestLoader, self).loadTestsFromModule(
@@ -228,6 +235,10 @@ class OETestLoader(unittest.TestLoader):
 """
 Returns a suite of all tests cases contained in module.
 """
+if module.__name__ in sys.builtin_module_names:
+msg = 'Tried to import %s test module but is a built-in'
+raise ImportError(msg % module.__name__)
+
 if not self.modules or "all" in self.modules or \
 module.__name__ in self.modules:
 return super(OETestLoader, self).loadTestsFromModule(
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] oeqa/core/loader.py: Give meaningful error when failed to load classes

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

With this we get the class that is actually having the problem,
not just a TypeError with an unknown class causing the error.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/core/loader.py | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
index c73ef9a..bf13a5a 100644
--- a/meta/lib/oeqa/core/loader.py
+++ b/meta/lib/oeqa/core/loader.py
@@ -171,10 +171,11 @@ class OETestLoader(unittest.TestLoader):
 """
 if issubclass(testCaseClass, unittest.suite.TestSuite):
 raise TypeError("Test cases should not be derived from TestSuite." 
\
-" Maybe you meant to derive from TestCase?")
+" Maybe you meant to derive %s from TestCase?" 
\
+% testCaseClass.__name__)
 if not issubclass(testCaseClass, self.caseClass):
-raise TypeError("Test cases need to be derived from %s" % \
-caseClass.__name__)
+raise TypeError("Test %s is not derived from %s" % \
+(testCaseClass.__name__, self.caseClass.__name__))
 
 
 testCaseNames = self.getTestCaseNames(testCaseClass)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/2] OEQA loader improvement

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

The first adds more verbosity when a class doesn't inherit
from the expected test class.

The second patch raises an error when trying to import a test
with the same module name as a built-in module.

The following changes since commit e758547db9048d4aa1c1415d6af8072f519fae24:

  nss: Fix nss-native so the checksum doesn't change with BUILD_ARCH 
(2017-02-09 10:52:03 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/bug10978
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug10978

Mariano Lopez (2):
  oeqa/core/loader.py: Give meaningful error when failed to load classes
  oeqa/core/loader.py: Avoid importing tests with built-ins name

 meta/lib/oeqa/core/loader.py | 18 +++---
 1 file changed, 15 insertions(+), 3 deletions(-)

-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 1/1] insane.bbclass: Add missing dependencies

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

package_qa task requires some tools installed in sysroot; with
the introduction of recipe specific sysroot this task won't
have such tools installed if it's forced to run.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/insane.bbclass | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 3cf86c1..0167b14 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -1223,9 +1223,12 @@ python do_package_qa () {
 bb.note("DONE with PACKAGE QA")
 }
 
+# binutils is used for most checks, so need to set as dependency
+# POPULATESYSROOTDEPS is defined in staging class.
+do_package_qa[depends] += "${POPULATESYSROOTDEPS}"
 do_package_qa[vardepsexclude] = "BB_TASKDEPDATA"
 do_package_qa[rdeptask] = "do_packagedata"
-addtask do_package_qa after do_packagedata do_package before do_build
+addtask do_package_qa after do_prepare_recipe_sysroot do_packagedata 
do_package before do_build
 
 SSTATETASKS += "do_package_qa"
 do_package_qa[sstate-inputdirs] = ""
-- 
2.10.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 0/1] insane.bbclass: Add missing dependencies

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Changes in v2:

- Add binutils as dependency too

The following changes since commit a624cf7f95c8cf4ff764cc997fd1db4601b97dcc:

  oeqa/selftest/pkgdata: use m4 instead of bash (2017-02-07 14:50:10 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/insane_fix
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/insane_fix

Mariano Lopez (1):
  insane.bbclass: Add missing dependencies

 meta/classes/insane.bbclass | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

-- 
2.10.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] testimage.bbclass: Add check for empty test suite

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

If for some reason (most likely incorrect TEST_SUITES var)
you end with an empty test suite, testimage will start the
target (qemu by default) and will run an empty suite, so
better have a check before starting the target.

[YOCTO #10979]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/testimage.bbclass | 4 
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index c6e4cec..1312202 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -149,6 +149,7 @@ def testimage_main(d):
 from oeqa.runtime.context import OERuntimeTestContext
 from oeqa.runtime.context import OERuntimeTestContextExecutor
 from oeqa.core.target.qemu import supported_fstypes
+from oeqa.core.utils.test import getSuiteCases
 from oeqa.utils import make_logger_bitbake_compatible
 
 def sigterm_exception(signum, stackframe):
@@ -250,6 +251,9 @@ def testimage_main(d):
 test_modules = d.getVar('TEST_SUITES')
 tc.loadTests(test_paths, modules=test_modules)
 
+if not getSuiteCases(tc.suites):
+bb.fatal('Empty test suite, please verify TEST_SUITES variable')
+
 package_extraction(d, tc.suites)
 
 bootparams = None
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] selftest: Optimize get_bb_var use

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

get_bb_var calls bitbake every time it is used and every call
would take about 7 seconds. There are tests that calls get_bb_var
several times when they can use get_bb_vars. Also there are tests
that calls it to fetch the same variable over and over again.

This will optimize the use of get_bb_var and get_bb_vars for a
little speed up in the tests.

[YOCTO #11037]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/selftest/archiver.py |  7 +--
 meta/lib/oeqa/selftest/bbtests.py  | 24 +
 meta/lib/oeqa/selftest/buildhistory.py |  5 +-
 meta/lib/oeqa/selftest/buildoptions.py | 10 ++--
 meta/lib/oeqa/selftest/devtool.py  | 57 -
 meta/lib/oeqa/selftest/eSDK.py | 21 
 meta/lib/oeqa/selftest/liboe.py| 21 
 meta/lib/oeqa/selftest/manifest.py |  7 +--
 meta/lib/oeqa/selftest/pkgdata.py  | 11 ++--
 meta/lib/oeqa/selftest/prservice.py|  9 ++--
 meta/lib/oeqa/selftest/recipetool.py   | 34 +++--
 meta/lib/oeqa/selftest/runtime-test.py | 15 --
 meta/lib/oeqa/selftest/signing.py  | 12 +++--
 meta/lib/oeqa/selftest/sstate.py   | 20 +---
 meta/lib/oeqa/selftest/sstatetests.py  | 92 ++
 meta/lib/oeqa/selftest/tinfoil.py  |  2 +-
 meta/lib/oeqa/selftest/wic.py  | 39 +++---
 17 files changed, 208 insertions(+), 178 deletions(-)

diff --git a/meta/lib/oeqa/selftest/archiver.py 
b/meta/lib/oeqa/selftest/archiver.py
index 97b6f5b..d7f215c 100644
--- a/meta/lib/oeqa/selftest/archiver.py
+++ b/meta/lib/oeqa/selftest/archiver.py
@@ -1,5 +1,5 @@
 from oeqa.selftest.base import oeSelfTest
-from oeqa.utils.commands import bitbake, get_bb_var
+from oeqa.utils.commands import bitbake, get_bb_vars
 from oeqa.utils.decorators import testcase
 import glob
 import os
@@ -28,10 +28,11 @@ class Archiver(oeSelfTest):
 features += 'COPYLEFT_PN_EXCLUDE = "%s"\n' % exclude_recipe
 self.write_config(features)
 
-shutil.rmtree(get_bb_var('TMPDIR'))
+bitbake('-c clean %s %s' % (include_recipe, exclude_recipe))
 bitbake("%s %s" % (include_recipe, exclude_recipe))
 
-src_path = os.path.join(get_bb_var('DEPLOY_DIR_SRC'), 
get_bb_var('TARGET_SYS'))
+bb_vars = get_bb_vars(['DEPLOY_DIR_SRC', 'TARGET_SYS'])
+src_path = os.path.join(bb_vars['DEPLOY_DIR_SRC'], 
bb_vars['TARGET_SYS'])
 
 # Check that include_recipe was included
 included_present = len(glob.glob(src_path + '/%s-*' % include_recipe))
diff --git a/meta/lib/oeqa/selftest/bbtests.py 
b/meta/lib/oeqa/selftest/bbtests.py
index 55c10f5..7ac4fc5 100644
--- a/meta/lib/oeqa/selftest/bbtests.py
+++ b/meta/lib/oeqa/selftest/bbtests.py
@@ -3,7 +3,7 @@ import re
 
 import oeqa.utils.ftools as ftools
 from oeqa.selftest.base import oeSelfTest
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars
 from oeqa.utils.decorators import testcase
 
 class BitbakeTests(oeSelfTest):
@@ -78,9 +78,10 @@ class BitbakeTests(oeSelfTest):
 # test 1 from bug 5875
 test_recipe = 'zlib'
 test_data = "Microsoft Made No Profit From Anyone's Zunes Yo"
-image_dir = get_bb_var('D', test_recipe)
-pkgsplit_dir = get_bb_var('PKGDEST', test_recipe)
-man_dir = get_bb_var('mandir', test_recipe)
+bb_vars = get_bb_vars(['D', 'PKGDEST', 'mandir'], test_recipe)
+image_dir = bb_vars['D']
+pkgsplit_dir = bb_vars['PKGDEST']
+man_dir = bb_vars['mandir']
 
 bitbake('-c clean %s' % test_recipe)
 bitbake('-c package -f %s' % test_recipe)
@@ -121,8 +122,9 @@ class BitbakeTests(oeSelfTest):
 @testcase(899)
 def test_image_manifest(self):
 bitbake('core-image-minimal')
-deploydir = get_bb_var("DEPLOY_DIR_IMAGE", target="core-image-minimal")
-imagename = get_bb_var("IMAGE_LINK_NAME", target="core-image-minimal")
+bb_vars = get_bb_vars(["DEPLOY_DIR_IMAGE", "IMAGE_LINK_NAME"], 
"core-image-minimal")
+deploydir = bb_vars["DEPLOY_DIR_IMAGE"]
+imagename = bb_vars["IMAGE_LINK_NAME"]
 manifest = os.path.join(deploydir, imagename + ".manifest")
 self.assertTrue(os.path.islink(manifest), msg="No manifest file 
created for image. It should have been created in %s" % manifest)
 
@@ -160,8 +162,9 @@ SSTATE_DIR = \"${TOPDIR}/download-selftest\"
 result = bitbake('-c fetch aspell', ignore_status=True)
 self.delete_recipeinc('aspell')
 self.assertEqual(result.status, 0, msg = "Couldn't fetch aspell. %s" % 
result.output)
-self.assertTrue(os.path.isfile(os.path.join(get_bb_var("DL_DIR"), 
'test-as

[OE-core] [PATCH 1/2] selftest: Avoid sstate corruption by calling cleansstate

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Currently selftest doesn't use sstates because some tests
clean sstate cache; using sstates would give a performance
boost instead of building everything from scratch.

With this sstates are not corrupted using different methods
depending on tests:

devtool: These tests needed to delete the cache so SSTATE_DIR
as SSTATE_MIRRORS and set a temporal SSTATE_DIR.

sstatetests: This module already used a temporal SSTATE_DIR, so
just set up the SSTATE_MIRRORS.

Rest: Removed cleansstate, some of them required to force a
certain task, others were just removed or changed for another
task.

[YOCTO #10929]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/selftest/bbtests.py  |  6 ++
 meta/lib/oeqa/selftest/buildoptions.py | 18 ++
 meta/lib/oeqa/selftest/devtool.py  | 18 +-
 meta/lib/oeqa/selftest/oescripts.py|  8 ++--
 meta/lib/oeqa/selftest/prservice.py|  3 ---
 meta/lib/oeqa/selftest/recipetool.py   |  5 -
 meta/lib/oeqa/selftest/signing.py  |  8 
 meta/lib/oeqa/selftest/sstatetests.py  |  6 +++---
 8 files changed, 34 insertions(+), 38 deletions(-)

diff --git a/meta/lib/oeqa/selftest/bbtests.py 
b/meta/lib/oeqa/selftest/bbtests.py
index 9dbac95..55c10f5 100644
--- a/meta/lib/oeqa/selftest/bbtests.py
+++ b/meta/lib/oeqa/selftest/bbtests.py
@@ -37,7 +37,6 @@ class BitbakeTests(oeSelfTest):
 
 @testcase(103)
 def test_local_sstate(self):
-bitbake('m4-native -ccleansstate')
 bitbake('m4-native')
 bitbake('m4-native -cclean')
 result = bitbake('m4-native')
@@ -83,8 +82,8 @@ class BitbakeTests(oeSelfTest):
 pkgsplit_dir = get_bb_var('PKGDEST', test_recipe)
 man_dir = get_bb_var('mandir', test_recipe)
 
-bitbake('-c cleansstate %s' % test_recipe)
-bitbake(test_recipe)
+bitbake('-c clean %s' % test_recipe)
+bitbake('-c package -f %s' % test_recipe)
 self.add_command_to_tearDown('bitbake -c clean %s' % test_recipe)
 
 man_file = os.path.join(image_dir + man_dir, 'man3/zlib.3')
@@ -103,7 +102,6 @@ class BitbakeTests(oeSelfTest):
 # test 2 from bug 5875
 test_recipe = 'zlib'
 
-bitbake('-c cleansstate %s' % test_recipe)
 bitbake(test_recipe)
 self.add_command_to_tearDown('bitbake -c clean %s' % test_recipe)
 
diff --git a/meta/lib/oeqa/selftest/buildoptions.py 
b/meta/lib/oeqa/selftest/buildoptions.py
index d40eb00..2889afa 100644
--- a/meta/lib/oeqa/selftest/buildoptions.py
+++ b/meta/lib/oeqa/selftest/buildoptions.py
@@ -36,9 +36,7 @@ class ImageOptionsTests(oeSelfTest):
 p = get_bb_var('SYSROOT_DESTDIR', 'ccache-native') + 
get_bb_var('bindir', 'ccache-native') + "/" + "ccache"
 self.assertTrue(os.path.isfile(p), msg = "No ccache found (%s)" % p)
 self.write_config('INHERIT += "ccache"')
-bitbake("m4 -c cleansstate")
-bitbake("m4 -c compile")
-self.addCleanup(bitbake, 'ccache-native -ccleansstate')
+bitbake("m4 -f -c compile")
 res = runCmd("grep ccache %s" % 
(os.path.join(get_bb_var("WORKDIR","m4"),"temp/log.do_compile")), 
ignore_status=True)
 self.assertEqual(0, res.status, msg="No match for ccache in m4 
log.do_compile. For further details: %s" % 
os.path.join(get_bb_var("WORKDIR","m4"),"temp/log.do_compile"))
 
@@ -75,14 +73,13 @@ class SanityOptionsTest(oeSelfTest):
 
 @testcase(927)
 def test_options_warnqa_errorqa_switch(self):
-bitbake("xcursor-transparent-theme -ccleansstate")
 
 self.write_config("INHERIT_remove = \"report-error\"")
 if "packages-list" not in get_bb_var("ERROR_QA"):
 self.append_config("ERROR_QA_append = \" packages-list\"")
 
 self.write_recipeinc('xcursor-transparent-theme', 'PACKAGES += 
\"${PN}-dbg\"')
-res = bitbake("xcursor-transparent-theme", ignore_status=True)
+res = bitbake("xcursor-transparent-theme -f -c package", 
ignore_status=True)
 self.delete_recipeinc('xcursor-transparent-theme')
 line = self.getline(res, "QA Issue: xcursor-transparent-theme-dbg is 
listed in PACKAGES multiple times, this leads to packaging errors.")
 self.assertTrue(line and line.startswith("ERROR:"), msg=res.output)
@@ -90,8 +87,7 @@ class SanityOptionsTest(oeSelfTest):
 self.write_recipeinc('xcursor-transparent-theme', 'PACKAGES += 
\"${PN}-dbg\"')
 self.append_config('ERROR_QA_remove = "packages-list"')
 self.append_config('WARN_QA_append = " packages-list"')
-bitbake("xcursor-transparent-theme -

[OE-core] [PATCH 0/2] Improvements to oe-selftest

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This series includes to main improvements;
- Remove use of cleansstate tasks from tests in order to use shared
  states in the autobuilders
- Optimize use of get_bb_var to avoid calling bitbake -e over and
  over again

This has been tested in local autobuilders.

This series requires:

- https://patchwork.openembedded.org/series/5191/

This series supersedes:

- https://patchwork.openembedded.org/series/5101/
- https://patchwork.openembedded.org/series/5122/

The following changes since commit 634e6f703ea1d7fa3c1ae668b0c9ae90a7ffc0b7:

  insane.bbclass: Add prepare_recipe_sysroot dependency for package_qa 
(2017-02-09 08:00:53 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/bug10929
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug10929

Mariano Lopez (2):
  selftest: Avoid sstate corruption by calling cleansstate
  selftest: Optimize get_bb_var use

 meta/lib/oeqa/selftest/archiver.py |  7 +--
 meta/lib/oeqa/selftest/bbtests.py  | 30 ++-
 meta/lib/oeqa/selftest/buildhistory.py |  5 +-
 meta/lib/oeqa/selftest/buildoptions.py | 28 +-
 meta/lib/oeqa/selftest/devtool.py  | 75 +-
 meta/lib/oeqa/selftest/eSDK.py | 21 
 meta/lib/oeqa/selftest/liboe.py| 21 
 meta/lib/oeqa/selftest/manifest.py |  7 +--
 meta/lib/oeqa/selftest/oescripts.py|  8 +--
 meta/lib/oeqa/selftest/pkgdata.py  | 11 ++--
 meta/lib/oeqa/selftest/prservice.py| 12 ++---
 meta/lib/oeqa/selftest/recipetool.py   | 39 +++---
 meta/lib/oeqa/selftest/runtime-test.py | 15 --
 meta/lib/oeqa/selftest/signing.py  | 20 +++
 meta/lib/oeqa/selftest/sstate.py   | 20 ---
 meta/lib/oeqa/selftest/sstatetests.py  | 98 ++
 meta/lib/oeqa/selftest/tinfoil.py  |  2 +-
 meta/lib/oeqa/selftest/wic.py  | 39 +++---
 18 files changed, 242 insertions(+), 216 deletions(-)

-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] insane.bbclass: Add prepare_recipe_sysroot dependency for package_qa

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

package_qa task requires some tools installed in sysroot; with
the introduction of recipe specific sysroot this task won't
have such tools installed if it's forced to run.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/insane.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 3cf86c1..64c6d14 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -1225,7 +1225,7 @@ python do_package_qa () {
 
 do_package_qa[vardepsexclude] = "BB_TASKDEPDATA"
 do_package_qa[rdeptask] = "do_packagedata"
-addtask do_package_qa after do_packagedata do_package before do_build
+addtask do_package_qa after do_prepare_recipe_sysroot do_packagedata 
do_package before do_build
 
 SSTATETASKS += "do_package_qa"
 do_package_qa[sstate-inputdirs] = ""
-- 
2.10.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] do_populate_cve_db: Error in executing cve-check-update

[Mariano Lopez]

On 06/02/17 09:17, Jussi Kukkonen wrote:
>
>
> On 6 February 2017 at 16:56, Burton, Ross  > wrote:
>
>
> On 6 February 2017 at 14:43, Sona Sarmadi  > wrote:
>
> bbdebug 2 "Updating cve-check-tool database located in
> $cve_dir"
> if cve-check-update -d "$cve_dir" ; then
> printf "CVE database was updated on %s UTC\n\n"
> "$(LANG=C date --utc +'%F %T')" > "$cve_file"
> else
> bbwarn "Error in executing cve-check-update" 
> <
>
>
> This definitely needs to be rewritten so you can see the output if
> it fails.  Just run cve-check-update -d  yourself and see
> what it says.  Last time I had this failing it was because the
> mitre servers were offline.
>
>
> Agreed about the error output.

Also you need to patch the tool, most of the time there is no output
from it; I think Ikey would integrate those patches without hesitation.

>
> I think recipe specific sysroots broke the setup somehow (so the tools
> are not actually in sysroot when they're needed). I'm taking a look at
> this tomorrow.

I tried today, but I'm having a hard time with the proxies (like always)
so I can't really verify this. Were you able to check?

Mariano
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv3 4/5] oeqa/core/context.py: Add validation for run-tests option

[mariano . lopez]

From: Aníbal Limón 

The run-tests option is optional so if isn't specified set
to None instead of crash on split().

Signed-off-by: Aníbal Limón 
---
 meta/lib/oeqa/core/context.py | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
index efed4e6..4476750 100644
--- a/meta/lib/oeqa/core/context.py
+++ b/meta/lib/oeqa/core/context.py
@@ -215,7 +215,11 @@ class OETestContextExecutor(object):
 else:
 self.tc_kwargs['init']['td'] = {}
 
-self.tc_kwargs['load']['modules'] = args.run_tests.split()
+
+if args.run_tests:
+self.tc_kwargs['load']['modules'] = args.run_tests.split()
+else:
+self.tc_kwargs['load']['modules'] = None
 
 self.module_paths = args.CASES_PATHS
 
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv3 5/5] oeqa/runtime/cases: Rename syslog module to oe_syslog

[mariano . lopez]

From: Aníbal Limón <anibal.li...@linux.intel.com>

Debian based distros has a builtin syslog module so when
try to load tests using unittest it references the builtin
module instead of runtime/cases.

[YOCTO #10964]

Signed-off-by: Aníbal Limón <anibal.li...@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/testimage.bbclass  | 2 +-
 meta/lib/oeqa/runtime/cases/{syslog.py => oe_syslog.py} | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)
 rename meta/lib/oeqa/runtime/cases/{syslog.py => oe_syslog.py} (92%)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 7495fe1..c6e4cec 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -38,7 +38,7 @@ TEST_PACKAGED_DIR ?= "${TEST_NEEDED_PACKAGES_DIR}/packaged"
 RPMTESTSUITE = "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'smart rpm', '', 
d)}"
 SYSTEMDSUITE = "${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'systemd', 
'', d)}"
 MINTESTSUITE = "ping"
-NETTESTSUITE = "${MINTESTSUITE} ssh df date scp syslog ${SYSTEMDSUITE}"
+NETTESTSUITE = "${MINTESTSUITE} ssh df date scp oe_syslog ${SYSTEMDSUITE}"
 DEVTESTSUITE = "gcc kernelmodule ldd"
 
 DEFAULT_TEST_SUITES = "${MINTESTSUITE} auto"
diff --git a/meta/lib/oeqa/runtime/cases/syslog.py 
b/meta/lib/oeqa/runtime/cases/oe_syslog.py
similarity index 92%
rename from meta/lib/oeqa/runtime/cases/syslog.py
rename to meta/lib/oeqa/runtime/cases/oe_syslog.py
index 537c519..005b697 100644
--- a/meta/lib/oeqa/runtime/cases/syslog.py
+++ b/meta/lib/oeqa/runtime/cases/oe_syslog.py
@@ -18,7 +18,7 @@ class SyslogTest(OERuntimeTestCase):
 class SyslogTestConfig(OERuntimeTestCase):
 
 @OETestID(1149)
-@OETestDepends(['syslog.SyslogTest.test_syslog_running'])
+@OETestDepends(['oe_syslog.SyslogTest.test_syslog_running'])
 def test_syslog_logger(self):
 status, output = self.target.run('logger foobar')
 msg = "Can't log into syslog. Output: %s " % output
@@ -35,7 +35,7 @@ class SyslogTestConfig(OERuntimeTestCase):
 self.assertEqual(status, 0, msg=msg)
 
 @OETestID(1150)
-@OETestDepends(['syslog.SyslogTest.test_syslog_running'])
+@OETestDepends(['oe_syslog.SyslogTest.test_syslog_running'])
 def test_syslog_restart(self):
 if "systemd" != self.tc.td.get("VIRTUAL-RUNTIME_init_manager", ""):
 (_, _) = self.target.run('/etc/init.d/syslog restart')
@@ -44,7 +44,7 @@ class SyslogTestConfig(OERuntimeTestCase):
 
 
 @OETestID(202)
-@OETestDepends(['syslog.SyslogTestConfig.test_syslog_logger'])
+@OETestDepends(['oe_syslog.SyslogTestConfig.test_syslog_logger'])
 @OEHasPackage(["!sysklogd", "busybox"])
 @skipIfDataVar('VIRTUAL-RUNTIME_init_manager', 'systemd',
'Not appropiate for systemd image')
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv3 2/5] runtime/cases/smart.py: Check for IMAGE_PKGTYPE instead of PACKAGE_CLASSES

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

smart test requires to build the image using rpm packages, this check was
included, but it checked for PACKAGE_CLASSES=='package_rpm', and this is
not true when building packages for rpm and deb/ipk. So this would check
IMAGE_PKGTYPE instead.

[YOCTO #10964]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/runtime/cases/smart.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/cases/smart.py 
b/meta/lib/oeqa/runtime/cases/smart.py
index 9b4d0d2..79bd9c8 100644
--- a/meta/lib/oeqa/runtime/cases/smart.py
+++ b/meta/lib/oeqa/runtime/cases/smart.py
@@ -23,7 +23,7 @@ class SmartBasicTest(SmartTest):
 
 @skipIfNotFeature('package-management',
   'Test requires package-management to be in 
IMAGE_FEATURES')
-@skipIfNotDataVar('PACKAGE_CLASSES', 'package_rpm',
+@skipIfNotDataVar('IMAGE_PKGTYPE', 'rpm',
   'RPM is not the primary package manager')
 @OEHasPackage(['smartpm'])
 @OETestID(716)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv3 3/5] testimage.bbclass: Allow to run tests on autobuilder's images

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

With the change to the new framework data store dependecy was
removed, instead a new file is generated and used in testimage.
When testing builds from the autobuilders the test data values
are from the autobuilder, including the paths.

Some tests require paths to current environment in order to run,
this commit will update such paths and fix the error of running
images donwloaded from autobuilders.

[YOCTO #10964]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/testimage.bbclass   | 18 +++---
 meta/lib/oeqa/core/utils/misc.py |  7 +++
 2 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 73a5c1a..7495fe1 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -49,11 +49,11 @@ DEFAULT_TEST_SUITES_pn-core-image-x11 = "${MINTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-core-image-lsb = "${NETTESTSUITE} pam parselogs 
${RPMTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-core-image-sato = "${NETTESTSUITE} connman xorg 
parselogs ${RPMTESTSUITE} \
 ${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python', '', d)}"
-DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "${NETTESTSUITE} connman xorg 
perl python \
-${DEVTESTSUITE} parselogs ${RPMTESTSUITE}"
+DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "${NETTESTSUITE} buildcvs 
buildiptables buildgalculator \
+connman ${DEVTESTSUITE} logrotate perl parselogs python ${RPMTESTSUITE} 
xorg"
 DEFAULT_TEST_SUITES_pn-core-image-lsb-dev = "${NETTESTSUITE} pam perl python 
parselogs ${RPMTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "${NETTESTSUITE} buildcvs 
buildiptables buildgalculator \
-connman ${DEVTESTSUITE} pam perl python parselogs ${RPMTESTSUITE}"
+connman ${DEVTESTSUITE} logrotate pam parselogs perl python 
${RPMTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-meta-toolchain = "auto"
 
 # aarch64 has no graphics
@@ -79,13 +79,15 @@ TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 
'rpm', 'python-smartp
 TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'ipk', 
'opkg-utils-native:do_populate_sysroot', '', d)}"
 TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'deb', 
'apt-native:do_populate_sysroot', '', d)}"
 TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 
'python-smartpm-native:do_populate_sysroot', '', d)}"
-
+TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 
'createrepo-native:do_populate_sysroot', '', d)}"
 
 TESTIMAGELOCK = "${TMPDIR}/testimage.lock"
 TESTIMAGELOCK_qemuall = ""
 
 TESTIMAGE_DUMP_DIR ?= "/tmp/oe-saved-tests/"
 
+TESTIMAGE_UPDATE_VARS ?= "DL_DIR WORKDIR DEPLOY_DIR"
+
 testimage_dump_target () {
 top -bn1
 ps
@@ -138,13 +140,12 @@ def testimage_sanity(d):
 
 def testimage_main(d):
 import os
-import signal
 import json
-import sys
+import signal
 import logging
-import time
 
 from bb.utils import export_proxies
+from oeqa.core.utils.misc import updateTestData
 from oeqa.runtime.context import OERuntimeTestContext
 from oeqa.runtime.context import OERuntimeTestContextExecutor
 from oeqa.core.target.qemu import supported_fstypes
@@ -166,6 +167,9 @@ def testimage_main(d):
 
 tdname = "%s.testdata.json" % image_name
 td = json.load(open(tdname, "r"))
+# Some variables need to be updates (mostly paths) with the
+# ones of the current environment because some tests require them.
+updateTestData(d, td, d.getVar('TESTIMAGE_UPDATE_VARS').split())
 
 image_manifest = "%s.manifest" % image_name
 image_packages = 
OERuntimeTestContextExecutor.readPackagesManifest(image_manifest)
diff --git a/meta/lib/oeqa/core/utils/misc.py b/meta/lib/oeqa/core/utils/misc.py
index 6ae58ad..d1eec13 100644
--- a/meta/lib/oeqa/core/utils/misc.py
+++ b/meta/lib/oeqa/core/utils/misc.py
@@ -35,3 +35,10 @@ def dataStoteToDict(d, variables):
 data[v] = d.getVar(v, True)
 
 return data
+
+def updateTestData(d, td, variables):
+"""
+Updates variables with values of data store to test data.
+"""
+for var in variables:
+td[var] = d.getVar(var)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv3 1/5] runtime/cases: Fix case numbers, missing cases and unused classes

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

connman: oeRuntimeTest class is not used anymore as part of runtime
migration, this particular case was missed, so fix it.

gcc: Removed unneded lines.

multilib: fixed case number.

syslog: added a missing test.

[YOCTO #10964]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/runtime/cases/connman.py  | 2 +-
 meta/lib/oeqa/runtime/cases/gcc.py  | 3 ---
 meta/lib/oeqa/runtime/cases/multilib.py | 2 +-
 meta/lib/oeqa/runtime/cases/syslog.py   | 9 +
 4 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/runtime/cases/connman.py 
b/meta/lib/oeqa/runtime/cases/connman.py
index 8b47108..12456b4 100644
--- a/meta/lib/oeqa/runtime/cases/connman.py
+++ b/meta/lib/oeqa/runtime/cases/connman.py
@@ -6,7 +6,7 @@ from oeqa.runtime.decorator.package import OEHasPackage
 class ConnmanTest(OERuntimeTestCase):
 
 def service_status(self, service):
-if oeRuntimeTest.hasFeature("systemd"):
+if 'systemd' in self.tc.td['DISTRO_FEATURES']:
 (_, output) = self.target.run('systemctl status -l %s' % service)
 return output
 else:
diff --git a/meta/lib/oeqa/runtime/cases/gcc.py 
b/meta/lib/oeqa/runtime/cases/gcc.py
index 064fa49..9110831 100644
--- a/meta/lib/oeqa/runtime/cases/gcc.py
+++ b/meta/lib/oeqa/runtime/cases/gcc.py
@@ -11,15 +11,12 @@ class GccCompileTest(OERuntimeTestCase):
 def setUpClass(cls):
 dst = '/tmp/'
 src = os.path.join(cls.tc.files_dir, 'test.c')
-#dst = '/tmp/test.c'
 cls.tc.target.copyTo(src, dst)
 
 src = os.path.join(cls.tc.runtime_files_dir, 'testmakefile')
-#dst = '/tmp/testmakefile'
 cls.tc.target.copyTo(src, dst)
 
 src = os.path.join(cls.tc.files_dir, 'test.cpp')
-#dst = '/tmp/test.cpp'
 cls.tc.target.copyTo(src, dst)
 
 @classmethod
diff --git a/meta/lib/oeqa/runtime/cases/multilib.py 
b/meta/lib/oeqa/runtime/cases/multilib.py
index 8f6d2b2..8c167f1 100644
--- a/meta/lib/oeqa/runtime/cases/multilib.py
+++ b/meta/lib/oeqa/runtime/cases/multilib.py
@@ -23,9 +23,9 @@ class MultilibTest(OERuntimeTestCase):
 msg = "%s isn't %s (is %s)" % (binary, arch, theclass)
 self.assertEqual(theclass, arch, msg=msg)
 
+@OETestID(1593)
 @skipIfNotInDataVar('MULTILIBS', 'multilib:lib32',
 "This isn't a multilib:lib32 image")
-@OETestID(201)
 @OETestDepends(['ssh.SSHTest.test_ssh'])
 def test_check_multilib_libc(self):
 """
diff --git a/meta/lib/oeqa/runtime/cases/syslog.py 
b/meta/lib/oeqa/runtime/cases/syslog.py
index 1016e67..537c519 100644
--- a/meta/lib/oeqa/runtime/cases/syslog.py
+++ b/meta/lib/oeqa/runtime/cases/syslog.py
@@ -34,6 +34,15 @@ class SyslogTestConfig(OERuntimeTestCase):
' Output: %s ' % output)
 self.assertEqual(status, 0, msg=msg)
 
+@OETestID(1150)
+@OETestDepends(['syslog.SyslogTest.test_syslog_running'])
+def test_syslog_restart(self):
+if "systemd" != self.tc.td.get("VIRTUAL-RUNTIME_init_manager", ""):
+(_, _) = self.target.run('/etc/init.d/syslog restart')
+else:
+(_, _) = self.target.run('systemctl restart syslog.service')
+
+
 @OETestID(202)
 @OETestDepends(['syslog.SyslogTestConfig.test_syslog_logger'])
 @OEHasPackage(["!sysklogd", "busybox"])
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 3/5] testimage.bbclass: Allow to run tests on autobuilder's images

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

With the change to the new framework data store dependecy was
removed, instead a new file is generated and used in testimage.
When testing builds from the autobuilders the test data values
are from the autobuilder, including the paths.

Some tests require paths to current environment in order to run,
this commit will update such paths and fix the error of running
images donwloaded from autobuilders.

[YOCTO #10964]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/testimage.bbclass   | 18 +++---
 meta/lib/oeqa/core/utils/misc.py |  7 +++
 2 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 73a5c1a..7495fe1 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -49,11 +49,11 @@ DEFAULT_TEST_SUITES_pn-core-image-x11 = "${MINTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-core-image-lsb = "${NETTESTSUITE} pam parselogs 
${RPMTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-core-image-sato = "${NETTESTSUITE} connman xorg 
parselogs ${RPMTESTSUITE} \
 ${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python', '', d)}"
-DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "${NETTESTSUITE} connman xorg 
perl python \
-${DEVTESTSUITE} parselogs ${RPMTESTSUITE}"
+DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "${NETTESTSUITE} buildcvs 
buildiptables buildgalculator \
+connman ${DEVTESTSUITE} logrotate perl parselogs python ${RPMTESTSUITE} 
xorg"
 DEFAULT_TEST_SUITES_pn-core-image-lsb-dev = "${NETTESTSUITE} pam perl python 
parselogs ${RPMTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "${NETTESTSUITE} buildcvs 
buildiptables buildgalculator \
-connman ${DEVTESTSUITE} pam perl python parselogs ${RPMTESTSUITE}"
+connman ${DEVTESTSUITE} logrotate pam parselogs perl python 
${RPMTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-meta-toolchain = "auto"
 
 # aarch64 has no graphics
@@ -79,13 +79,15 @@ TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 
'rpm', 'python-smartp
 TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'ipk', 
'opkg-utils-native:do_populate_sysroot', '', d)}"
 TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'deb', 
'apt-native:do_populate_sysroot', '', d)}"
 TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 
'python-smartpm-native:do_populate_sysroot', '', d)}"
-
+TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 
'createrepo-native:do_populate_sysroot', '', d)}"
 
 TESTIMAGELOCK = "${TMPDIR}/testimage.lock"
 TESTIMAGELOCK_qemuall = ""
 
 TESTIMAGE_DUMP_DIR ?= "/tmp/oe-saved-tests/"
 
+TESTIMAGE_UPDATE_VARS ?= "DL_DIR WORKDIR DEPLOY_DIR"
+
 testimage_dump_target () {
 top -bn1
 ps
@@ -138,13 +140,12 @@ def testimage_sanity(d):
 
 def testimage_main(d):
 import os
-import signal
 import json
-import sys
+import signal
 import logging
-import time
 
 from bb.utils import export_proxies
+from oeqa.core.utils.misc import updateTestData
 from oeqa.runtime.context import OERuntimeTestContext
 from oeqa.runtime.context import OERuntimeTestContextExecutor
 from oeqa.core.target.qemu import supported_fstypes
@@ -166,6 +167,9 @@ def testimage_main(d):
 
 tdname = "%s.testdata.json" % image_name
 td = json.load(open(tdname, "r"))
+# Some variables need to be updates (mostly paths) with the
+# ones of the current environment because some tests require them.
+updateTestData(d, td, d.getVar('TESTIMAGE_UPDATE_VARS').split())
 
 image_manifest = "%s.manifest" % image_name
 image_packages = 
OERuntimeTestContextExecutor.readPackagesManifest(image_manifest)
diff --git a/meta/lib/oeqa/core/utils/misc.py b/meta/lib/oeqa/core/utils/misc.py
index 6ae58ad..d1eec13 100644
--- a/meta/lib/oeqa/core/utils/misc.py
+++ b/meta/lib/oeqa/core/utils/misc.py
@@ -35,3 +35,10 @@ def dataStoteToDict(d, variables):
 data[v] = d.getVar(v, True)
 
 return data
+
+def updateTestData(d, td, variables):
+"""
+Updates variables with values of data store to test data.
+"""
+for var in variables:
+td[var] = d.getVar(var)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 5/5] oeqa/runtime/cases: Rename syslog module to oe_syslog

[mariano . lopez]

From: Aníbal Limón <anibal.li...@linux.intel.com>

Debian based distros has a builtin syslog module so when
try to load tests using unittest it references the builtin
module instead of runtime/cases.

[YOCTO ##10964]

Signed-off-by: Aníbal Limón <anibal.li...@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/testimage.bbclass  | 2 +-
 meta/lib/oeqa/runtime/cases/{syslog.py => oe_syslog.py} | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)
 rename meta/lib/oeqa/runtime/cases/{syslog.py => oe_syslog.py} (92%)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 7495fe1..c6e4cec 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -38,7 +38,7 @@ TEST_PACKAGED_DIR ?= "${TEST_NEEDED_PACKAGES_DIR}/packaged"
 RPMTESTSUITE = "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'smart rpm', '', 
d)}"
 SYSTEMDSUITE = "${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'systemd', 
'', d)}"
 MINTESTSUITE = "ping"
-NETTESTSUITE = "${MINTESTSUITE} ssh df date scp syslog ${SYSTEMDSUITE}"
+NETTESTSUITE = "${MINTESTSUITE} ssh df date scp oe_syslog ${SYSTEMDSUITE}"
 DEVTESTSUITE = "gcc kernelmodule ldd"
 
 DEFAULT_TEST_SUITES = "${MINTESTSUITE} auto"
diff --git a/meta/lib/oeqa/runtime/cases/syslog.py 
b/meta/lib/oeqa/runtime/cases/oe_syslog.py
similarity index 92%
rename from meta/lib/oeqa/runtime/cases/syslog.py
rename to meta/lib/oeqa/runtime/cases/oe_syslog.py
index 537c519..005b697 100644
--- a/meta/lib/oeqa/runtime/cases/syslog.py
+++ b/meta/lib/oeqa/runtime/cases/oe_syslog.py
@@ -18,7 +18,7 @@ class SyslogTest(OERuntimeTestCase):
 class SyslogTestConfig(OERuntimeTestCase):
 
 @OETestID(1149)
-@OETestDepends(['syslog.SyslogTest.test_syslog_running'])
+@OETestDepends(['oe_syslog.SyslogTest.test_syslog_running'])
 def test_syslog_logger(self):
 status, output = self.target.run('logger foobar')
 msg = "Can't log into syslog. Output: %s " % output
@@ -35,7 +35,7 @@ class SyslogTestConfig(OERuntimeTestCase):
 self.assertEqual(status, 0, msg=msg)
 
 @OETestID(1150)
-@OETestDepends(['syslog.SyslogTest.test_syslog_running'])
+@OETestDepends(['oe_syslog.SyslogTest.test_syslog_running'])
 def test_syslog_restart(self):
 if "systemd" != self.tc.td.get("VIRTUAL-RUNTIME_init_manager", ""):
 (_, _) = self.target.run('/etc/init.d/syslog restart')
@@ -44,7 +44,7 @@ class SyslogTestConfig(OERuntimeTestCase):
 
 
 @OETestID(202)
-@OETestDepends(['syslog.SyslogTestConfig.test_syslog_logger'])
+@OETestDepends(['oe_syslog.SyslogTestConfig.test_syslog_logger'])
 @OEHasPackage(["!sysklogd", "busybox"])
 @skipIfDataVar('VIRTUAL-RUNTIME_init_manager', 'systemd',
'Not appropiate for systemd image')
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 1/5] runtime/cases: Fix case numbers, missing cases and unused classes

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

connman: oeRuntimeTest class is not used anymore as part of runtime
migration, this particular case was missed, so fix it.

gcc: Removed unneded lines.

multilib: fixed case number.

syslog: added a missing test.

[YOCTO #10964]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/runtime/cases/connman.py  | 2 +-
 meta/lib/oeqa/runtime/cases/gcc.py  | 3 ---
 meta/lib/oeqa/runtime/cases/multilib.py | 2 +-
 meta/lib/oeqa/runtime/cases/syslog.py   | 9 +
 4 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/runtime/cases/connman.py 
b/meta/lib/oeqa/runtime/cases/connman.py
index 8b47108..12456b4 100644
--- a/meta/lib/oeqa/runtime/cases/connman.py
+++ b/meta/lib/oeqa/runtime/cases/connman.py
@@ -6,7 +6,7 @@ from oeqa.runtime.decorator.package import OEHasPackage
 class ConnmanTest(OERuntimeTestCase):
 
 def service_status(self, service):
-if oeRuntimeTest.hasFeature("systemd"):
+if 'systemd' in self.tc.td['DISTRO_FEATURES']:
 (_, output) = self.target.run('systemctl status -l %s' % service)
 return output
 else:
diff --git a/meta/lib/oeqa/runtime/cases/gcc.py 
b/meta/lib/oeqa/runtime/cases/gcc.py
index 064fa49..9110831 100644
--- a/meta/lib/oeqa/runtime/cases/gcc.py
+++ b/meta/lib/oeqa/runtime/cases/gcc.py
@@ -11,15 +11,12 @@ class GccCompileTest(OERuntimeTestCase):
 def setUpClass(cls):
 dst = '/tmp/'
 src = os.path.join(cls.tc.files_dir, 'test.c')
-#dst = '/tmp/test.c'
 cls.tc.target.copyTo(src, dst)
 
 src = os.path.join(cls.tc.runtime_files_dir, 'testmakefile')
-#dst = '/tmp/testmakefile'
 cls.tc.target.copyTo(src, dst)
 
 src = os.path.join(cls.tc.files_dir, 'test.cpp')
-#dst = '/tmp/test.cpp'
 cls.tc.target.copyTo(src, dst)
 
 @classmethod
diff --git a/meta/lib/oeqa/runtime/cases/multilib.py 
b/meta/lib/oeqa/runtime/cases/multilib.py
index 8f6d2b2..8c167f1 100644
--- a/meta/lib/oeqa/runtime/cases/multilib.py
+++ b/meta/lib/oeqa/runtime/cases/multilib.py
@@ -23,9 +23,9 @@ class MultilibTest(OERuntimeTestCase):
 msg = "%s isn't %s (is %s)" % (binary, arch, theclass)
 self.assertEqual(theclass, arch, msg=msg)
 
+@OETestID(1593)
 @skipIfNotInDataVar('MULTILIBS', 'multilib:lib32',
 "This isn't a multilib:lib32 image")
-@OETestID(201)
 @OETestDepends(['ssh.SSHTest.test_ssh'])
 def test_check_multilib_libc(self):
 """
diff --git a/meta/lib/oeqa/runtime/cases/syslog.py 
b/meta/lib/oeqa/runtime/cases/syslog.py
index 1016e67..537c519 100644
--- a/meta/lib/oeqa/runtime/cases/syslog.py
+++ b/meta/lib/oeqa/runtime/cases/syslog.py
@@ -34,6 +34,15 @@ class SyslogTestConfig(OERuntimeTestCase):
' Output: %s ' % output)
 self.assertEqual(status, 0, msg=msg)
 
+@OETestID(1150)
+@OETestDepends(['syslog.SyslogTest.test_syslog_running'])
+def test_syslog_restart(self):
+if "systemd" != self.tc.td.get("VIRTUAL-RUNTIME_init_manager", ""):
+(_, _) = self.target.run('/etc/init.d/syslog restart')
+else:
+(_, _) = self.target.run('systemctl restart syslog.service')
+
+
 @OETestID(202)
 @OETestDepends(['syslog.SyslogTestConfig.test_syslog_logger'])
 @OEHasPackage(["!sysklogd", "busybox"])
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 4/5] oeqa/core/context.py: Add validation for run-tests option

[mariano . lopez]

From: Aníbal Limón 

The run-tests option is optional so if isn't specified set
to None instead of crash on split().

Signed-off-by: Aníbal Limón 
---
 meta/lib/oeqa/core/context.py | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
index efed4e6..4476750 100644
--- a/meta/lib/oeqa/core/context.py
+++ b/meta/lib/oeqa/core/context.py
@@ -215,7 +215,11 @@ class OETestContextExecutor(object):
 else:
 self.tc_kwargs['init']['td'] = {}
 
-self.tc_kwargs['load']['modules'] = args.run_tests.split()
+
+if args.run_tests:
+self.tc_kwargs['load']['modules'] = args.run_tests.split()
+else:
+self.tc_kwargs['load']['modules'] = None
 
 self.module_paths = args.CASES_PATHS
 
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 0/5] Fix bugs found within runtime testing

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This will fix some bugs found in testimage when executing tests
from images downloaded from the autobuilders.

Changes in v2:

- Fix some test cases that were changed during the migration.
- Add createrepo-native as dependency of test image.
- Fix TEST_SUITES in core-image-sato-sdk and core-image-lsb-sdk.
- Renamed syslog test to oe_syslog to make it run on debian/ubuntu.

The following changes since commit 0150dddeaf901c91d32be093efd0348a344fb545:

  sanity.conf: Update minimum bitbake version requirements to 1.33.0 
(2017-01-27 10:42:39 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/bug10964
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug10964

Aníbal Limón (2):
  oeqa/core/context.py: Add validation for run-tests option
  oeqa/runtime/cases: Rename syslog module to oe_syslog

Mariano Lopez (3):
  runtime/cases: Fix case numbers, missing cases and unused classes
  runtime/cases/smart.py: Check for IMAGE_PKGTYPE instead of
PACKAGE_CLASSES
  testimage.bbclass: Allow to run tests on autobuilder's images

 meta/classes/testimage.bbclass   | 20 
 meta/lib/oeqa/core/context.py|  6 +-
 meta/lib/oeqa/core/utils/misc.py |  7 +++
 meta/lib/oeqa/runtime/cases/connman.py   |  2 +-
 meta/lib/oeqa/runtime/cases/gcc.py   |  3 ---
 meta/lib/oeqa/runtime/cases/multilib.py  |  2 +-
 .../oeqa/runtime/cases/{syslog.py => oe_syslog.py}   | 13 +++--
 meta/lib/oeqa/runtime/cases/smart.py |  2 +-
 8 files changed, 38 insertions(+), 17 deletions(-)
 rename meta/lib/oeqa/runtime/cases/{syslog.py => oe_syslog.py} (82%)

-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 2/5] runtime/cases/smart.py: Check for IMAGE_PKGTYPE instead of PACKAGE_CLASSES

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

smart test requires to build the image using rpm packages, this check was
included, but it checked for PACKAGE_CLASSES=='package_rpm', and this is
not true when building packages for rpm and deb/ipk. So this would check
IMAGE_PKGTYPE instead.

[YOCTO #10964]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/runtime/cases/smart.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/cases/smart.py 
b/meta/lib/oeqa/runtime/cases/smart.py
index 9b4d0d2..79bd9c8 100644
--- a/meta/lib/oeqa/runtime/cases/smart.py
+++ b/meta/lib/oeqa/runtime/cases/smart.py
@@ -23,7 +23,7 @@ class SmartBasicTest(SmartTest):
 
 @skipIfNotFeature('package-management',
   'Test requires package-management to be in 
IMAGE_FEATURES')
-@skipIfNotDataVar('PACKAGE_CLASSES', 'package_rpm',
+@skipIfNotDataVar('IMAGE_PKGTYPE', 'rpm',
   'RPM is not the primary package manager')
 @OEHasPackage(['smartpm'])
 @OETestID(716)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/1] package_ipk.bbclass: Add check for empty lines in DESCRIPTION

[Mariano Lopez]

On 25/01/17 05:56, Richard Purdie wrote:
> On Tue, 2017-01-24 at 11:52 -0700, Christopher Larson wrote:
>> On Tue, Jan 24, 2017 at 1:37 AM, <mariano.lo...@linux.intel.com>
>> wrote:
>>> From: Mariano Lopez <mariano.lo...@linux.intel.com>
>>>
>>> opkg uses empty lines as separator for next package and if an ipk
>>> file was packaged with empty lines in DESCRIPTION opkg won't be
>>> able to handle such ipk file, this happens at execution time.
>>>
>>> This commit will add a check for empty lines in DESCRIPTION when
>>> generating to ipk package to avoid this issue.
>>>
>>> [YOCTO #10677]
>>>
>>> Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
>>>
>> Could it handle a line with just a space, or a period? Could you not
>> replace the lines with that instead, rather than aborting?
> FWIW package_deb uses ".".

Also opkg will strip() the lines, so an empty space is not a option here.

>
> Also, please use bb.fatal() calls with caution, I just removed several
> from that class. In this case its perhaps ok but often the correct
> thing to do is raise an exception of some kind. This means bitbake will
> generate a usable traceback rather than simply printing the fatal
> message. In this case the message is probably fine but there were
> several cases where a traceback was a much better idea as the fatal
> message wasn't enough to debug with.

Fair enough, I will consider this advice when using bb.fatal()

Regards,
Mariano
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] runtime/cases/smart.py: Check for IMAGE_PKGTYPE instead of PACKAGE_CLASSES

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

smart test requires to build the image using rpm packages, this check was
included, but it checked for PACKAGE_CLASSES=='package_rpm', and this is
not true when building packages for rpm and deb/ipk. So this would check
IMAGE_PKGTYPE instead.

[YOCTO #10964]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/runtime/cases/smart.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/cases/smart.py 
b/meta/lib/oeqa/runtime/cases/smart.py
index 9b4d0d2..79bd9c8 100644
--- a/meta/lib/oeqa/runtime/cases/smart.py
+++ b/meta/lib/oeqa/runtime/cases/smart.py
@@ -23,7 +23,7 @@ class SmartBasicTest(SmartTest):
 
 @skipIfNotFeature('package-management',
   'Test requires package-management to be in 
IMAGE_FEATURES')
-@skipIfNotDataVar('PACKAGE_CLASSES', 'package_rpm',
+@skipIfNotDataVar('IMAGE_PKGTYPE', 'rpm',
   'RPM is not the primary package manager')
 @OEHasPackage(['smartpm'])
 @OETestID(716)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] Fix bugs found within runtime testing

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This will fix some bugs found in testimage when executing tests
from images downloaded from the autobuilders.

The following changes since commit aa7e2717d33c2d754ec486e9148f446c1d6a3b14:

  ref-manual: remove core-image-directfb reference (2017-01-25 12:07:43 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/bug10964
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug10964

Mariano Lopez (3):
  runtime/cases/connman.py: Stop using oeRuntimeTest
  runtime/cases/smart.py: Check for IMAGE_PKGTYPE instead of
PACKAGE_CLASSES
  testimage.bbclass: Allow to run tests on autobuilder's images

 meta/classes/testimage.bbclass | 11 +++
 meta/lib/oeqa/core/utils/misc.py   |  7 +++
 meta/lib/oeqa/runtime/cases/connman.py |  2 +-
 meta/lib/oeqa/runtime/cases/smart.py   |  2 +-
 4 files changed, 16 insertions(+), 6 deletions(-)

-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] runtime/cases/connman.py: Stop using oeRuntimeTest

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

oeRuntimeTest class is not used anymore as part of runtime
migration, this particular case was missed, so fix it.

[YOCTO #10964]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/runtime/cases/connman.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/cases/connman.py 
b/meta/lib/oeqa/runtime/cases/connman.py
index 8b47108..12456b4 100644
--- a/meta/lib/oeqa/runtime/cases/connman.py
+++ b/meta/lib/oeqa/runtime/cases/connman.py
@@ -6,7 +6,7 @@ from oeqa.runtime.decorator.package import OEHasPackage
 class ConnmanTest(OERuntimeTestCase):
 
 def service_status(self, service):
-if oeRuntimeTest.hasFeature("systemd"):
+if 'systemd' in self.tc.td['DISTRO_FEATURES']:
 (_, output) = self.target.run('systemctl status -l %s' % service)
 return output
 else:
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] testimage.bbclass: Allow to run tests on autobuilder's images

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

With the change to the new framework data store dependecy was
removed, instead a new file is generated and used in testimage.
When testing builds from the autobuilders the test data values
are from the autobuilder, including the paths.

Some tests require paths to current environment in order to run,
this commit will update such paths and fix the error of running
images donwloaded from autobuilders.

[YOCTO #10964]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/testimage.bbclass   | 11 +++
 meta/lib/oeqa/core/utils/misc.py |  7 +++
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 96e41c6..d65ac52 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -79,12 +79,13 @@ TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 
'ipk', 'opkg-utils-na
 TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'deb', 
'apt-native:do_populate_sysroot', '', d)}"
 TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 
'python-smartpm-native:do_populate_sysroot', '', d)}"
 
-
 TESTIMAGELOCK = "${TMPDIR}/testimage.lock"
 TESTIMAGELOCK_qemuall = ""
 
 TESTIMAGE_DUMP_DIR ?= "/tmp/oe-saved-tests/"
 
+TESTIMAGE_UPDATE_VARS ?= "WORKDIR DEPLOY_DIR"
+
 testimage_dump_target () {
 top -bn1
 ps
@@ -137,13 +138,12 @@ def testimage_sanity(d):
 
 def testimage_main(d):
 import os
-import signal
 import json
-import sys
+import signal
 import logging
-import time
 
 from bb.utils import export_proxies
+from oeqa.core.utils.misc import updateTestData
 from oeqa.runtime.context import OERuntimeTestContext
 from oeqa.runtime.context import OERuntimeTestContextExecutor
 from oeqa.core.target.qemu import supported_fstypes
@@ -165,6 +165,9 @@ def testimage_main(d):
 
 tdname = "%s.testdata.json" % image_name
 td = json.load(open(tdname, "r"))
+# Some variables need to be updates (mostly paths) with the
+# ones of the current environment because some tests require them.
+updateTestData(d, td, d.getVar('TESTIMAGE_UPDATE_VARS').split())
 
 image_manifest = "%s.manifest" % image_name
 image_packages = 
OERuntimeTestContextExecutor.readPackagesManifest(image_manifest)
diff --git a/meta/lib/oeqa/core/utils/misc.py b/meta/lib/oeqa/core/utils/misc.py
index 6ae58ad..d1eec13 100644
--- a/meta/lib/oeqa/core/utils/misc.py
+++ b/meta/lib/oeqa/core/utils/misc.py
@@ -35,3 +35,10 @@ def dataStoteToDict(d, variables):
 data[v] = d.getVar(v, True)
 
 return data
+
+def updateTestData(d, td, variables):
+"""
+Updates variables with values of data store to test data.
+"""
+for var in variables:
+td[var] = d.getVar(var)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] package_ipk.bbclass: Add check for empty lines in DESCRIPTION

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

opkg uses empty lines as separator for next package and if an ipk
file was packaged with empty lines in DESCRIPTION opkg won't be
able to handle such ipk file, this happens at execution time.

This commit will add a check for empty lines in DESCRIPTION when
generating to ipk package to avoid this issue.

[YOCTO #10677]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/package_ipk.bbclass | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index a76b235..a8a4ddc 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -32,6 +32,13 @@ python do_package_ipk () {
 bb.error("Variables incorrectly set, unable to package")
 return
 
+# opkg won't allow empty lines in package's description at execution
+# time, so is better to fail at build time instead when trying to install.
+description = d.getVar('DESCRIPTION').encode('UTF-8', 'ignore')
+description = description.decode('unicode_escape')
+if ([line for line in description.splitlines() if not line.strip()]):
+bb.fatal("opkg won't allow empty lines in DESCRIPTION")
+
 packages = d.getVar('PACKAGES')
 if not packages or packages == '':
 bb.debug(1, "No packages; nothing to do")
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] testimage.bbclass: Add systemd test automatically

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This adds systemd test automatically when an image
is build with systemd DISTRO_FEATURE

[YOCTO #10935]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/testimage.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 96e41c6..73a5c1a 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -36,8 +36,9 @@ TEST_EXTRACTED_DIR ?= "${TEST_NEEDED_PACKAGES_DIR}/extracted"
 TEST_PACKAGED_DIR ?= "${TEST_NEEDED_PACKAGES_DIR}/packaged"
 
 RPMTESTSUITE = "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'smart rpm', '', 
d)}"
+SYSTEMDSUITE = "${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'systemd', 
'', d)}"
 MINTESTSUITE = "ping"
-NETTESTSUITE = "${MINTESTSUITE} ssh df date scp syslog"
+NETTESTSUITE = "${MINTESTSUITE} ssh df date scp syslog ${SYSTEMDSUITE}"
 DEVTESTSUITE = "gcc kernelmodule ldd"
 
 DEFAULT_TEST_SUITES = "${MINTESTSUITE} auto"
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] lib/oe/package_manager.py: Fix extract for ipk and deb

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

With the move to use lists instead of strings in subprocess
calls, package extraction was broken for ipk and deb. This
fixes this issue.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oe/package_manager.py | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index a8644cc..a02bff4 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -1545,11 +1545,15 @@ class OpkgDpkgPM(PackageManager):
 tmp_dir = tempfile.mkdtemp()
 current_dir = os.getcwd()
 os.chdir(tmp_dir)
+if self.d.getVar('IMAGE_PKGTYPE') == 'deb':
+data_tar = 'data.tar.xz'
+else:
+data_tar = 'data.tar.gz'
 
 try:
 cmd = [ar_cmd, 'x', pkg_path]
 output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
-cmd = [tar_cmd, 'xf', 'data.tar.*']
+cmd = [tar_cmd, 'xf', data_tar]
 output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
 except subprocess.CalledProcessError as e:
 bb.utils.remove(tmp_dir, recurse=True)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] oeqa/utils/qemurunner.py: Be sure to stop qemu-system

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

When runqemu fails, qemu-system process would keep running
and won't be killed, setpgrp() was used when runqemu was
a shell script but it seems it doesn't work always with python.

This would kill qemu-system explicity and to avoid leaving
it behind.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/testimage.bbclass| 2 ++
 meta/lib/oeqa/utils/qemurunner.py | 9 +
 2 files changed, 11 insertions(+)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 770ec80..7eb4038 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -171,6 +171,8 @@ def testimage_main(d):
 bb.plain(msg)
 else:
 bb.fatal("%s - FAILED - check the task log and the ssh log" % pn)
+except BlockingIOError as err:
+bb.error('runqemu failed, shutting down...')
 finally:
 signal.signal(signal.SIGTERM, tc.origsigtermhandler)
 target.stop()
diff --git a/meta/lib/oeqa/utils/qemurunner.py 
b/meta/lib/oeqa/utils/qemurunner.py
index 6927456..21bc35a 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -296,6 +296,7 @@ class QemuRunner:
 
 def stop(self):
 self.stop_thread()
+self.stop_qemu_system()
 if hasattr(self, "origchldhandler"):
 signal.signal(signal.SIGCHLD, self.origchldhandler)
 if self.runqemu:
@@ -320,6 +321,14 @@ class QemuRunner:
 self.qemupid = None
 self.ip = None
 
+def stop_qemu_system(self):
+if self.qemupid:
+try:
+# qemu-system behaves well and a SIGTERM is enough
+os.kill(self.qemupid, signal.SIGTERM)
+except ProcessLookupError as e:
+logger.warn('qemu-system ended unexpectedly')
+
 def stop_thread(self):
 if self.thread and self.thread.is_alive():
 self.thread.stop()
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] oeqa/utils/qemurunner.py: Add missing sys module

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This adds the missing sys module used by the child process
to exit. It seems the exception was cached in testimage and
selftest. It seems nobody noticed this because the module
is only used for sys.exit().

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/utils/qemurunner.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/lib/oeqa/utils/qemurunner.py 
b/meta/lib/oeqa/utils/qemurunner.py
index 8f1b5b9..6927456 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -7,6 +7,7 @@
 
 import subprocess
 import os
+import sys
 import time
 import signal
 import re
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/2] Be sure to kill qemu-system on testimage

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

With refactor of runqemu to use python instead of shel scripting,
the qemu-system process keeps running if runqemu fails. This series
will fix this and will terminate qemu-system explicity.

The following changes since commit 81021bc0aa0f64e67535f6a9551e921a64fe4395:

  yocto-project-qs, ref-manual: Added note for "resources temporarily 
unavailable" error (2017-01-11 17:23:18 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/qemu_fix
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/qemu_fix

Mariano Lopez (2):
  oeqa/utils/qemurunner.py: Add missing sys module
  oeqa/utils/qemurunner.py: Be sure to stop qemu-system

 meta/classes/testimage.bbclass|  2 ++
 meta/lib/oeqa/utils/qemurunner.py | 10 ++
 2 files changed, 12 insertions(+)

-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 8/8] oeqa.utils.metadata: add bitbake revision information

[Mariano Lopez]

On 28/12/16 07:02, Markus Lehtonen wrote:
> [YOCTO #10590]
>
> Signed-off-by: Markus Lehtonen 
> ---
>  meta/lib/oeqa/utils/metadata.py | 32 +++-
>  1 file changed, 19 insertions(+), 13 deletions(-)
>
> diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
> index 6331c21..23449fc 100644
> --- a/meta/lib/oeqa/utils/metadata.py
> +++ b/meta/lib/oeqa/utils/metadata.py
> @@ -10,6 +10,8 @@ from collections.abc import MutableMapping
>  from xml.dom.minidom import parseString
>  from xml.etree.ElementTree import Element, tostring
>  
> +from git import Repo, InvalidGitRepositoryError, NoSuchPathError
> +

It seems not every user running selftest appreciated the requirement of
gitpython, so there was a patch to keep this dependency out. This will
introduce the need of gitpython again.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 7/8] oeqa.utils.metadata: have layer name as an attribute in xml

[Mariano Lopez]

On 28/12/16 07:02, Markus Lehtonen wrote:
> -def dict_to_XML(tag, dictionary):
> +def dict_to_XML(tag, dictionary, **kwargs):
>  """ Return XML element converting dicts recursively. """
>  
> -elem = Element(tag)
> +elem = Element(tag, **kwargs)
>  for key, val in dictionary.items():
> -if isinstance(val, MutableMapping):
> +if tag == 'layers':
> +child = (dict_to_XML('layer', val, name=key))
> +elif isinstance(val, MutableMapping):
>  child = (dict_to_XML(key, val))
>  else:
>  child = Element(key)

I was thinking that this function would be more generic, adding a
comparison to a tag name surely won't keep the function generic, is
there another way to implement this keeping this function generic enough?
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/8] oeqa.utils.metadata: drop 'unknown' git data elements

[Mariano Lopez]

On 28/12/16 07:02, Markus Lehtonen wrote:
> It's better just to not have the xml elements than to have elements with
> faux data. One could have git branch named 'unknown', for example.
>
>

I don't think is a good idea to completely remove the layer from the
metadata when such layer is not a repository, it would remove valuable
information for later analysis. And if you ask would do such a thing, I
would, I have my own debug layers that can mess with testing.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] oeqa/utils/commands.py: Fix get_bb_vars() when called without arguments

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Commit 9d55e9d489cd78be592fb9b4d6484f9060c62fdd broke calling get_bb_vars()
when called without arguments. This fix this issue.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/utils/commands.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/utils/commands.py b/meta/lib/oeqa/utils/commands.py
index 6acb24a..aecf8cf 100644
--- a/meta/lib/oeqa/utils/commands.py
+++ b/meta/lib/oeqa/utils/commands.py
@@ -149,7 +149,8 @@ def get_bb_vars(variables=None, target=None, 
postconfig=None):
 """Get values of multiple bitbake variables"""
 bbenv = get_bb_env(target, postconfig=postconfig)
 
-variables = variables.copy()
+if variables is not None:
+variables = variables.copy()
 var_re = re.compile(r'^(export )?(?P\w+)="(?P.*)"$')
 unset_re = re.compile(r'^unset (?P\w+)$')
 lastline = None
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/3] cve-check: allow recipes to override the product name

[Mariano Lopez]

On 07/12/16 10:50, Ross Burton wrote:
> Add a new variable CVE_PRODUCT for the product name to look up in the NVD
> database.  Default this to BPN, but allow recipes such as tiff (which is 
> libtiff
> in NVD) to override it.
>
> Signed-off-by: Ross Burton 
>
I like the idea to be able to override the name that cve-check-tool
checks. The only drawback would be the burden of adding these to needed
recipes. This is still better to have to guess the correct name, or to
check PROVIDES or RPROVIDES, there are just too much corner cases. So
this solution has my approval.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCHv2 3/3] oe-selftest: Add option to submit test result to a git repository.

[Mariano Lopez]

On Thursday, December 01, 2016 10:52:40 AM Benjamin Esquivel wrote:
> Hi Mariano
> 
> On Thu, 2016-12-01 at 09:37 -0600, mariano.lo...@linux.intel.com wrote:
> > From: Mariano Lopez <mariano.lo...@linux.intel.com>
> > 
> > This new option allows to commit the result to a git repository,
> > along with the results it will add a metadata file for information
> > of the current selftest run, such as: hostname, machine, distro,
> > distro version, host version, and layers.
> > 
> > This implementation will have a branch per different hostname,
> > testing branch, and machine.
> > 
> > To use this feature use:
> > 
> > oe-selftest  --repository 
> > 
> > [YOCTO #9954]

> I think this version is okay :) have you tested it in an Autobuilder
> task? 

Indeed, that is why it took me a bit longer to submit the patch. I
know this won't blow up current master.  But Ihaven't used the
option yet, we would need a remote repo and a new autobuilder
buildset.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 3/3] oe-selftest: Add option to submit test result to a git repository.

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This new option allows to commit the result to a git repository,
along with the results it will add a metadata file for information
of the current selftest run, such as: hostname, machine, distro,
distro version, host version, and layers.

This implementation will have a branch per different hostname,
testing branch, and machine.

To use this feature use:

oe-selftest  --repository 

[YOCTO #9954]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 scripts/oe-selftest | 102 
 1 file changed, 102 insertions(+)

diff --git a/scripts/oe-selftest b/scripts/oe-selftest
index deaa432..f4b861f 100755
--- a/scripts/oe-selftest
+++ b/scripts/oe-selftest
@@ -36,6 +36,7 @@ import re
 import fnmatch
 import collections
 import imp
+import git
 
 sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) + '/lib')
 import scriptpath
@@ -46,6 +47,7 @@ import argparse_oe
 import oeqa.selftest
 import oeqa.utils.ftools as ftools
 from oeqa.utils.commands import runCmd, get_bb_var, get_test_layer
+from oeqa.utils.metadata import metadata_from_bb, write_metadata_file
 from oeqa.selftest.base import oeSelfTest, get_available_machines
 
 try:
@@ -106,6 +108,8 @@ def get_args_parser():
help='List all tags that have been set to test cases.')
 parser.add_argument('--machine', required=False, dest='machine', 
choices=['random', 'all'], default=None,
 help='Run tests on different machines (random/all).')
+parser.add_argument('--repository', required=False, dest='repository', 
default='', action='store',
+help='Submit test results to a repository')
 return parser
 
 
@@ -572,6 +576,75 @@ def main():
 
 log.info("Finished")
 
+if args.repository:
+# Commit tests results to repository
+metadata = metadata_from_bb()
+git_dir = os.path.join(os.getcwd(), 'selftest')
+if not os.path.isdir(git_dir):
+os.mkdir(git_dir)
+
+log.debug('Checking for git repository in %s' % git_dir)
+try:
+repo = git.Repo(git_dir)
+except git.exc.InvalidGitRepositoryError:
+log.debug("Couldn't find git repository %s; "
+ "cloning from %s" % (git_dir, args.repository))
+repo = git.Repo.clone_from(args.repository, git_dir)
+
+r_branches = repo.git.branch(r=True)
+r_branches = set(r_branches.replace('origin/', '').split())
+l_branches = {str(branch) for branch in repo.branches}
+branch = '%s/%s/%s' % (metadata['hostname'],
+   metadata['layers']['meta']['branch'],
+   metadata['machine'])
+
+if branch in l_branches:
+log.debug('Found branch in local repository, checking out')
+repo.git.checkout(branch)
+elif branch in r_branches:
+log.debug('Found branch in remote repository, checking'
+  ' out and pulling')
+repo.git.checkout(branch)
+repo.git.pull()
+else:
+log.debug('New branch %s' % branch)
+repo.git.checkout('master')
+repo.git.checkout(b=branch)
+
+cleanResultsDir(repo)
+xml_dir = os.path.join(os.getcwd(), log_prefix)
+copyResultFiles(xml_dir, git_dir, repo)
+metadata_file = os.path.join(git_dir, 'metadata.xml')
+write_metadata_file(metadata_file, metadata)
+repo.index.add([metadata_file])
+repo.index.write()
+
+# Get information for commit message
+layer_info = ''
+for layer, values in metadata['layers'].items():
+layer_info = '%s%-17s = %s:%s\n' % (layer_info, layer,
+  values['branch'], values['revision'])
+msg = 'Selftest for build %s of %s %s for machine %s on %s\n\n%s' 
% (
+   log_prefix[12:], metadata['distro'], 
metadata['distro_version'],
+   metadata['machine'], metadata['hostname'], layer_info)
+
+log.debug('Commiting results to local repository')
+repo.index.commit(msg)
+if not repo.is_dirty():
+try:
+if branch in r_branches:
+log.debug('Pushing changes to remote repository')
+repo.git.push()
+else:
+log.debug('Pushing changes to remote repository '
+  'creating new branch')
+repo.git.push('-u', 'origin', branch)
+except GitCommandError:
+log.

[OE-core] [PATCHv2 1/3] oeqa/utils/commands.py: Make a copy of variables in get_bb_vars

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

The function get_bb_vars will remove items for the list passed
as the function argument, this will leave the caller with an
empty list and the function never says it will consume the items.

This hasn't been found before because only get_bb_var uses this
function.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/utils/commands.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/lib/oeqa/utils/commands.py b/meta/lib/oeqa/utils/commands.py
index 5cd0f74..e00c879 100644
--- a/meta/lib/oeqa/utils/commands.py
+++ b/meta/lib/oeqa/utils/commands.py
@@ -149,6 +149,7 @@ def get_bb_vars(variables=None, target=None, 
postconfig=None):
 """Get values of multiple bitbake variables"""
 bbenv = get_bb_env(target, postconfig=postconfig)
 
+variables = variables.copy()
 var_re = re.compile(r'^(export )?(?P\w+)="(?P.*)"$')
 unset_re = re.compile(r'^unset (?P\w+)$')
 lastline = None
-- 
2.7.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 2/3] oeqa/utils/metadata.py: Add metadata library

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Adds functions to get metadata from the host running the tests.

[YOCTO #9954]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/utils/metadata.py | 83 +
 1 file changed, 83 insertions(+)
 create mode 100644 meta/lib/oeqa/utils/metadata.py

diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
new file mode 100644
index 000..ecbe763
--- /dev/null
+++ b/meta/lib/oeqa/utils/metadata.py
@@ -0,0 +1,83 @@
+# Copyright (C) 2016 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+#
+# Functions to get metadata from the testing host used
+# for analytics of test results.
+
+from git import Repo, InvalidGitRepositoryError, NoSuchPathError
+from collections import OrderedDict
+from collections.abc import MutableMapping
+from xml.dom.minidom import parseString
+from xml.etree.ElementTree import Element, tostring
+
+from oe.lsb import distro_identifier
+from oeqa.utils.commands import runCmd, get_bb_var, get_bb_vars
+
+metadata_vars = ['MACHINE', 'DISTRO', 'DISTRO_VERSION']
+
+def metadata_from_bb():
+""" Returns test's metadata as OrderedDict.
+
+Data will be gathered using bitbake -e thanks to get_bb_vars.
+"""
+
+info_dict = OrderedDict()
+hostname = runCmd('hostname')
+info_dict['hostname'] = hostname.output
+data_dict = get_bb_vars(metadata_vars)
+for var in metadata_vars:
+info_dict[var.lower()] = data_dict[var]
+host_distro= distro_identifier()
+host_distro, _, host_distro_release = host_distro.partition('-')
+info_dict['host_distro'] = host_distro
+info_dict['host_distro_release'] = host_distro_release
+info_dict['layers'] = get_layers(get_bb_var('BBLAYERS'))
+return info_dict
+
+def metadata_from_data_store(d):
+""" Returns test's metadata as OrderedDict.
+
+Data will be collected from the provided data store.
+"""
+# TODO: Getting metadata from the data store would
+# be useful when running within bitbake.
+pass
+
+def get_layers(layers):
+""" Returns layer name, branch, and revision as OrderedDict. """
+
+layer_dict = OrderedDict()
+for layer in layers.split():
+layer_name = os.path.basename(layer)
+layer_dict[layer_name] = OrderedDict()
+try:
+repo = Repo(layer, search_parent_directories=True)
+revision, branch = repo.head.object.name_rev.split()
+layer_dict[layer_name]['branch'] = branch
+layer_dict[layer_name]['revision'] = revision
+except (InvalidGitRepositoryError, NoSuchPathError):
+layer_dict[layer_name]['branch'] = 'unknown'
+layer_dict[layer_name]['revision'] = 'unknown'
+return layer_dict
+
+def write_metadata_file(file_path, metadata):
+""" Writes metadata to a XML file in directory. """
+
+xml = dict_to_XML('metadata', metadata)
+xml_doc = parseString(tostring(xml).decode('UTF-8'))
+with open(file_path, 'w') as f:
+f.write(xml_doc.toprettyxml())
+
+def dict_to_XML(tag, dictionary):
+""" Return XML element converting dicts recursively. """
+
+elem = Element(tag)
+for key, val in dictionary.items():
+if isinstance(val, MutableMapping):
+child = (dict_to_XML(key, val))
+else:
+child = Element(key)
+child.text = str(val)
+elem.append(child)
+return elem
-- 
2.7.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 0/3] Add communication mechanism for sending test results

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

These changes add a mechanism for sending test result to a remote
repository for later consumption of the data from other clients.

These patches were tested in our local autobuilders.

Changes in v2:

- Give more priority to local branches, because it would be difficult
  to match a branch name (because it includes the hostname).
- Instead of throwing an execption when a push fails, log the error
  and return 1 to the caller.
- Instead to call 3 times bitbake -e, just call it one time to get
  needed variables.

The following changes since commit 9e63f81c78e284c9b325fe04a1b59e61c7ad8a1a:

  bitbake: ast: remove BBVERSIONS support (2016-11-30 15:48:10 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/bug9954v2
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug9954v2

Mariano Lopez (3):
  oeqa/utils/commands.py: Make a copy of variables in get_bb_vars
  oeqa/utils/metadata.py: Add metadata library
  oe-selftest: Add option to submit test result to a git repository.

 meta/lib/oeqa/utils/commands.py |   1 +
 meta/lib/oeqa/utils/metadata.py |  83 
 scripts/oe-selftest | 102 
 3 files changed, 186 insertions(+)
 create mode 100644 meta/lib/oeqa/utils/metadata.py

-- 
2.7.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] oeqa/utils/metadata.py: Add metadata library

[Mariano Lopez]

On Tuesday, November 29, 2016 04:17:25 PM Benjamin Esquivel wrote:
> On Tue, 2016-11-29 at 08:42 -0600, mariano.lo...@linux.intel.com wrote:
> > From: Mariano Lopez <mariano.lo...@linux.intel.com>
> > 
> > Adds functions to get metadata from the host running the tests.
> > 
> > [YOCTO #9954]
> > 
> > Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
> > ---
> >  meta/lib/oeqa/utils/metadata.py | 77
> > +
> >  1 file changed, 77 insertions(+)
> >  create mode 100644 meta/lib/oeqa/utils/metadata.py
> > 
> > diff --git a/meta/lib/oeqa/utils/metadata.py
> > b/meta/lib/oeqa/utils/metadata.py
> > new file mode 100644
> > index 000..3be805c
> > --- /dev/null
> > +++ b/meta/lib/oeqa/utils/metadata.py
> > @@ -0,0 +1,77 @@
> > +# Copyright (C) 2016 Intel Corporation
> > +#
> > +# Released under the MIT license (see COPYING.MIT)
> > +#
> > +# Functions to get metadata from the testing host used
> > +# for analytics of test results.
> > +
> > +from git import Repo
> > +from collections import OrderedDict
> > +from collections.abc import MutableMapping
> > +from xml.dom.minidom import parseString
> > +from xml.etree.ElementTree import Element, tostring
> > +
> > +from oe.lsb import distro_identifier
> > +from oeqa.utils.commands import runCmd, get_bb_var
> > +
> > +def metadata_from_bb():
> > +""" Returns test's metadata as OrderedDict.
> > +
> > +Data will be gathered using bitbake -e thanks to get_bb_var.
> > +"""
> > +
> > +info_dict = OrderedDict()
> > +hostname = runCmd('hostname')
> > +info_dict['hostname'] = hostname.output
> > +info_dict['machine'] = get_bb_var('MACHINE')
> > +info_dict['distro'] = get_bb_var('DISTRO')
> > +info_dict['distro_version'] = get_bb_var('DISTRO_VERSION')
> > +host_distro= distro_identifier()
> > +host_distro, _, host_distro_release = host_distro.partition('-')
> > +info_dict['host_distro'] = host_distro
> > +info_dict['host_distro_release'] = host_distro_release
> > +info_dict['layers'] = get_layers(get_bb_var('BBLAYERS'))
> is none of the upper statements going to throw exceptions? otherwise
> try/except as appropriate.
> > +return info_dict
> > +
> > +def metadata_from_data_store(d):
> > +""" Returns test's metadata as OrderedDict.
> > +
> > +Data will be collected from the provided data store.
> > +"""
> > +# TODO: Getting metadata from the data store would
> > +# be useful when running within bitbake.
> > +pass
> > +
> > +def get_layers(layers):
> > +""" Returns layer name, branch, and revision as OrderedDict. """
> > +
> > +layer_dict = OrderedDict()
> > +for layer in layers.split():
> > +layer_name = os.path.basename(layer)
> > +layer_dict[layer_name] = OrderedDict()
> > +repo = Repo(layer, search_parent_directories=True)
> > +revision, branch = repo.head.object.name_rev.split()
> > +layer_dict[layer_name]['branch'] = branch
> > +layer_dict[layer_name]['revision'] = revision
> same here for the try/except, did you test with usual cases of zero
> input and unexisting path's, etc?
> > +return layer_dict
> > +
> > +def write_metadata_file(file_path, metadata):
> > +""" Writes metadata to a XML file in directory. """
> > +
> > +xml = dict_to_XML('metadata', metadata)
> > +xml_doc = parseString(tostring(xml).decode('UTF-8'))
> > +with open(file_path, 'w') as f:
> > +f.write(xml_doc.toprettyxml())
> > +
> > +def dict_to_XML(tag, dictionary):
> > +""" Return XML element converting dicts recursively. """
> > +
> > +elem = Element(tag)
> > +for key, val in dictionary.items():
> > +if isinstance(val, MutableMapping):
> > +child = (dict_to_XML(key, val))
> > +else:
> > +child = Element(key)
> > +child.text = str(val)
> > +elem.append(child)
> > +return elem
> > -- 
> > 2.7.3
> > 

I'll implement the proposded changes and will send another version

Mariano

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] oe-selftest: Add option to submit test result to a git repository.

[Mariano Lopez]

On Tuesday, November 29, 2016 04:26:45 PM Benjamin Esquivel wrote:
> On Tue, 2016-11-29 at 08:42 -0600, mariano.lo...@linux.intel.com wrote:
> > From: Mariano Lopez <mariano.lo...@linux.intel.com>
> > 
> > This new option allows to commit the result to a git repository,
> > along with the results it will add a metadata file for information
> > of the current selftest run, such as: hostname, machine, distro,
> > distro version, host version, and layers.
> > 
> > This implementation will have a branch per different hostname,
> > testing branch, and machine.
> > 
> > To use this feature use:
> > 
> > oe-selftest  --repository 
> > 
> > [YOCTO #9954]
> > 
> > Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
> > ---
> >  scripts/oe-selftest | 98
> > +
> >  1 file changed, 98 insertions(+)
> > 
> > diff --git a/scripts/oe-selftest b/scripts/oe-selftest
> > index deaa432..81dfa3d 100755
> > --- a/scripts/oe-selftest
> > +++ b/scripts/oe-selftest
> > @@ -36,6 +36,7 @@ import re
> >  import fnmatch
> >  import collections
> >  import imp
> > +import git
> >  
> >  sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) +
> > '/lib')
> >  import scriptpath
> > @@ -46,6 +47,7 @@ import argparse_oe
> >  import oeqa.selftest
> >  import oeqa.utils.ftools as ftools
> >  from oeqa.utils.commands import runCmd, get_bb_var, get_test_layer
> > +from oeqa.utils.metadata import metadata_from_bb,
> > write_metadata_file
> >  from oeqa.selftest.base import oeSelfTest, get_available_machines
> >  
> >  try:
> > @@ -106,6 +108,8 @@ def get_args_parser():
> > help='List all tags that have been set to
> > test cases.')
> >  parser.add_argument('--machine', required=False, dest='machine',
> > choices=['random', 'all'], default=None,
> >  help='Run tests on different machines
> > (random/all).')
> > +parser.add_argument('--repository', required=False,
> > dest='repository', default='', action='store',
> > +help='Submit test results to a repository')
> >  return parser
> >  
> >  
> > @@ -572,6 +576,71 @@ def main():
> >  
> >  log.info("Finished")
> >  
> > +if args.repository:
> oe-selftest is already long and not so modular, is there a chance to
> send these actions to methods outside oe-selftest and import them? it
> would also make them accessible to other data flows that wish to do the
> same.

It would be possible to move this code to a module, although the specific
needs of every test would be different, for example the clean and add
functions must be different for everyone. I would leave it here and check
the needs of other test to check if this can be reused. I would do that
work, because I'm already familiar with this.

> > +# Commit tests results to repository
> > +metadata = metadata_from_bb()
> > +git_dir = os.path.join(os.getcwd(), 'selftest')
> > +if not os.path.isdir(git_dir):
> > +os.mkdir(git_dir)
> > +
> > +log.debug('Checking for git repository in %s' % git_dir)
> > +try:
> > +repo = git.Repo(git_dir)
> > +except git.exc.InvalidGitRepositoryError:
> > +log.debug("Couldn't find git repository %s; "
> > + "cloning from %s" % (git_dir,
> > args.repository))
> > +repo = git.Repo.clone_from(args.repository, git_dir)
> > +
> > +r_branches = repo.git.branch(r=True)
> > +r_branches = set(r_branches.replace('origin/',
> > '').split())
> > +l_branches = {str(branch) for branch in repo.branches}
> > +branch = '%s/%s/%s' % (metadata['hostname'],
> > +   metadata['layers']['meta']['branc
> > h'],
> > +   metadata['machine'])
> > +
> > +if branch in r_branches:
> > +log.debug('Found branch in remote repository,
> > checking'
> > +  ' out and pulling')
> > +repo.git.checkout(branch)
> > +repo.git.pull()
> > +elif branch in l_branches:
> > +log.debug('Found branch in local repository,
> > checking out')
> > +repo.git.checkout(b

Re: [OE-core] [PATCH 1/2] oeqa/utils/metadata.py: Add metadata library

[Mariano Lopez]

On Tuesday, November 29, 2016 04:17:25 PM Benjamin Esquivel wrote:
> On Tue, 2016-11-29 at 08:42 -0600, mariano.lo...@linux.intel.com wrote:
> > From: Mariano Lopez <mariano.lo...@linux.intel.com>
> > 
> > Adds functions to get metadata from the host running the tests.
> > 
> > [YOCTO #9954]
> > 
> > Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
> > ---
> >  meta/lib/oeqa/utils/metadata.py | 77
> > +
> >  1 file changed, 77 insertions(+)
> >  create mode 100644 meta/lib/oeqa/utils/metadata.py
> > 
> > diff --git a/meta/lib/oeqa/utils/metadata.py
> > b/meta/lib/oeqa/utils/metadata.py
> > new file mode 100644
> > index 000..3be805c
> > --- /dev/null
> > +++ b/meta/lib/oeqa/utils/metadata.py
> > @@ -0,0 +1,77 @@
> > +# Copyright (C) 2016 Intel Corporation
> > +#
> > +# Released under the MIT license (see COPYING.MIT)
> > +#
> > +# Functions to get metadata from the testing host used
> > +# for analytics of test results.
> > +
> > +from git import Repo
> > +from collections import OrderedDict
> > +from collections.abc import MutableMapping
> > +from xml.dom.minidom import parseString
> > +from xml.etree.ElementTree import Element, tostring
> > +
> > +from oe.lsb import distro_identifier
> > +from oeqa.utils.commands import runCmd, get_bb_var
> > +
> > +def metadata_from_bb():
> > +""" Returns test's metadata as OrderedDict.
> > +
> > +Data will be gathered using bitbake -e thanks to get_bb_var.
> > +"""
> > +
> > +info_dict = OrderedDict()
> > +hostname = runCmd('hostname')
> > +info_dict['hostname'] = hostname.output
> > +info_dict['machine'] = get_bb_var('MACHINE')
> > +info_dict['distro'] = get_bb_var('DISTRO')
> > +info_dict['distro_version'] = get_bb_var('DISTRO_VERSION')
> > +host_distro= distro_identifier()
> > +host_distro, _, host_distro_release = host_distro.partition('-')
> > +info_dict['host_distro'] = host_distro
> > +info_dict['host_distro_release'] = host_distro_release
> > +info_dict['layers'] = get_layers(get_bb_var('BBLAYERS'))
> is none of the upper statements going to throw exceptions? otherwise
> try/except as appropriate.

I really don't expect this code to throw an exception, these are bitbake
commands running in the host.

> > +return info_dict
> > +
> > +def metadata_from_data_store(d):
> > +""" Returns test's metadata as OrderedDict.
> > +
> > +Data will be collected from the provided data store.
> > +"""
> > +# TODO: Getting metadata from the data store would
> > +# be useful when running within bitbake.
> > +pass
> > +
> > +def get_layers(layers):
> > +""" Returns layer name, branch, and revision as OrderedDict. """
> > +
> > +layer_dict = OrderedDict()
> > +for layer in layers.split():
> > +layer_name = os.path.basename(layer)
> > +layer_dict[layer_name] = OrderedDict()
> > +repo = Repo(layer, search_parent_directories=True)
> > +revision, branch = repo.head.object.name_rev.split()
> > +layer_dict[layer_name]['branch'] = branch
> > +layer_dict[layer_name]['revision'] = revision
> same here for the try/except, did you test with usual cases of zero
> input and unexisting path's, etc?

If the function doesn't receive input, it won't enter in the loop, in case the
directory doesn't exists bitbake would complain long before reaching this
code. If the user gets creative enough to not pass a valid BBLAYER value
I think s/he deserve to be bitten by the exception.

> > +return layer_dict
> > +
> > +def write_metadata_file(file_path, metadata):
> > +""" Writes metadata to a XML file in directory. """
> > +
> > +xml = dict_to_XML('metadata', metadata)
> > +xml_doc = parseString(tostring(xml).decode('UTF-8'))
> > +with open(file_path, 'w') as f:
> > +f.write(xml_doc.toprettyxml())
> > +
> > +def dict_to_XML(tag, dictionary):
> > +""" Return XML element converting dicts recursively. """
> > +
> > +elem = Element(tag)
> > +for key, val in dictionary.items():
> > +if isinstance(val, MutableMapping):
> > +child = (dict_to_XML(key, val))
> > +else:
> > +child = Element(key)
> > +child.text = str(val)
> > +elem.append(child)
> > +return elem
> > -- 
> > 2.7.3
> > 

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] oe-selftest: Add option to submit test result to a git repository.

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This new option allows to commit the result to a git repository,
along with the results it will add a metadata file for information
of the current selftest run, such as: hostname, machine, distro,
distro version, host version, and layers.

This implementation will have a branch per different hostname,
testing branch, and machine.

To use this feature use:

oe-selftest  --repository 

[YOCTO #9954]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 scripts/oe-selftest | 98 +
 1 file changed, 98 insertions(+)

diff --git a/scripts/oe-selftest b/scripts/oe-selftest
index deaa432..81dfa3d 100755
--- a/scripts/oe-selftest
+++ b/scripts/oe-selftest
@@ -36,6 +36,7 @@ import re
 import fnmatch
 import collections
 import imp
+import git
 
 sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) + '/lib')
 import scriptpath
@@ -46,6 +47,7 @@ import argparse_oe
 import oeqa.selftest
 import oeqa.utils.ftools as ftools
 from oeqa.utils.commands import runCmd, get_bb_var, get_test_layer
+from oeqa.utils.metadata import metadata_from_bb, write_metadata_file
 from oeqa.selftest.base import oeSelfTest, get_available_machines
 
 try:
@@ -106,6 +108,8 @@ def get_args_parser():
help='List all tags that have been set to test cases.')
 parser.add_argument('--machine', required=False, dest='machine', 
choices=['random', 'all'], default=None,
 help='Run tests on different machines (random/all).')
+parser.add_argument('--repository', required=False, dest='repository', 
default='', action='store',
+help='Submit test results to a repository')
 return parser
 
 
@@ -572,6 +576,71 @@ def main():
 
 log.info("Finished")
 
+if args.repository:
+# Commit tests results to repository
+metadata = metadata_from_bb()
+git_dir = os.path.join(os.getcwd(), 'selftest')
+if not os.path.isdir(git_dir):
+os.mkdir(git_dir)
+
+log.debug('Checking for git repository in %s' % git_dir)
+try:
+repo = git.Repo(git_dir)
+except git.exc.InvalidGitRepositoryError:
+log.debug("Couldn't find git repository %s; "
+ "cloning from %s" % (git_dir, args.repository))
+repo = git.Repo.clone_from(args.repository, git_dir)
+
+r_branches = repo.git.branch(r=True)
+r_branches = set(r_branches.replace('origin/', '').split())
+l_branches = {str(branch) for branch in repo.branches}
+branch = '%s/%s/%s' % (metadata['hostname'],
+   metadata['layers']['meta']['branch'],
+   metadata['machine'])
+
+if branch in r_branches:
+log.debug('Found branch in remote repository, checking'
+  ' out and pulling')
+repo.git.checkout(branch)
+repo.git.pull()
+elif branch in l_branches:
+log.debug('Found branch in local repository, checking out')
+repo.git.checkout(branch)
+else:
+log.debug('New branch %s' % branch)
+repo.git.checkout('master')
+repo.git.checkout(b=branch)
+
+cleanResultsDir(repo)
+xml_dir = os.path.join(os.getcwd(), log_prefix)
+copyResultFiles(xml_dir, git_dir, repo)
+metadata_file = os.path.join(git_dir, 'metadata.xml')
+write_metadata_file(metadata_file, metadata)
+repo.index.add([metadata_file])
+repo.index.write()
+
+# Get information for commit message
+layer_info = ''
+for layer, values in metadata['layers'].items():
+layer_info = '%s%-17s = %s:%s\n' % (layer_info, layer,
+  values['branch'], values['revision'])
+msg = 'Selftest for build %s of %s %s for machine %s on %s\n\n%s' 
% (
+   log_prefix[12:], metadata['distro'], 
metadata['distro_version'],
+   metadata['machine'], metadata['hostname'], layer_info)
+
+log.debug('Commiting results to local repository')
+repo.index.commit(msg)
+if not repo.is_dirty():
+if branch in r_branches:
+log.debug('Pushing changes to remote repository')
+repo.git.push()
+else:
+log.debug('Pushing changes to remote repository '
+  'creating new branch')
+repo.git.push('-u', 'origin', branch)
+else:
+log.error('Local repository is dirty, not pushing commits')
+
 if result.wasSuccessful():

[OE-core] [PATCH 1/2] oeqa/utils/metadata.py: Add metadata library

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Adds functions to get metadata from the host running the tests.

[YOCTO #9954]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/lib/oeqa/utils/metadata.py | 77 +
 1 file changed, 77 insertions(+)
 create mode 100644 meta/lib/oeqa/utils/metadata.py

diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
new file mode 100644
index 000..3be805c
--- /dev/null
+++ b/meta/lib/oeqa/utils/metadata.py
@@ -0,0 +1,77 @@
+# Copyright (C) 2016 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+#
+# Functions to get metadata from the testing host used
+# for analytics of test results.
+
+from git import Repo
+from collections import OrderedDict
+from collections.abc import MutableMapping
+from xml.dom.minidom import parseString
+from xml.etree.ElementTree import Element, tostring
+
+from oe.lsb import distro_identifier
+from oeqa.utils.commands import runCmd, get_bb_var
+
+def metadata_from_bb():
+""" Returns test's metadata as OrderedDict.
+
+Data will be gathered using bitbake -e thanks to get_bb_var.
+"""
+
+info_dict = OrderedDict()
+hostname = runCmd('hostname')
+info_dict['hostname'] = hostname.output
+info_dict['machine'] = get_bb_var('MACHINE')
+info_dict['distro'] = get_bb_var('DISTRO')
+info_dict['distro_version'] = get_bb_var('DISTRO_VERSION')
+host_distro= distro_identifier()
+host_distro, _, host_distro_release = host_distro.partition('-')
+info_dict['host_distro'] = host_distro
+info_dict['host_distro_release'] = host_distro_release
+info_dict['layers'] = get_layers(get_bb_var('BBLAYERS'))
+return info_dict
+
+def metadata_from_data_store(d):
+""" Returns test's metadata as OrderedDict.
+
+Data will be collected from the provided data store.
+"""
+# TODO: Getting metadata from the data store would
+# be useful when running within bitbake.
+pass
+
+def get_layers(layers):
+""" Returns layer name, branch, and revision as OrderedDict. """
+
+layer_dict = OrderedDict()
+for layer in layers.split():
+layer_name = os.path.basename(layer)
+layer_dict[layer_name] = OrderedDict()
+repo = Repo(layer, search_parent_directories=True)
+revision, branch = repo.head.object.name_rev.split()
+layer_dict[layer_name]['branch'] = branch
+layer_dict[layer_name]['revision'] = revision
+return layer_dict
+
+def write_metadata_file(file_path, metadata):
+""" Writes metadata to a XML file in directory. """
+
+xml = dict_to_XML('metadata', metadata)
+xml_doc = parseString(tostring(xml).decode('UTF-8'))
+with open(file_path, 'w') as f:
+f.write(xml_doc.toprettyxml())
+
+def dict_to_XML(tag, dictionary):
+""" Return XML element converting dicts recursively. """
+
+elem = Element(tag)
+for key, val in dictionary.items():
+if isinstance(val, MutableMapping):
+child = (dict_to_XML(key, val))
+else:
+child = Element(key)
+child.text = str(val)
+elem.append(child)
+return elem
-- 
2.7.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/2] Add communication mechanism for sending test results

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>


These changes add a mechanism for sending test result to a remote
repository for later consumption of the data from other clients.

These patches were tested in our local autobuilders.

The following changes since commit 12a0ee049e453b6d0d2ce2f3fa981d1b6e02bd78:

  dev-manual: Added note about RPM not dealing with post-install (2016-11-23 
11:10:35 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/bug9954
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug9954

Mariano Lopez (2):
  oeqa/utils/metadata.py: Add metadata library
  oe-selftest: Add option to submit test result to a git repository.

 meta/lib/oeqa/utils/metadata.py | 77 
 scripts/oe-selftest | 98 +
 2 files changed, 175 insertions(+)
 create mode 100644 meta/lib/oeqa/utils/metadata.py

-- 
2.7.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv2 1/1] nfs-utils: 1.3.3 -> 1.3.4

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 .../files/nfs-utils-debianize-start-statd.patch|  23 +++--
 ...tatd-fix-a-segfault-caused-by-improper-us.patch | 113 -
 .../fix-protocol-minor-version-fall-back.patch |  55 --
 .../{nfs-utils_1.3.3.bb => nfs-utils_1.3.4.bb} |   6 +-
 4 files changed, 14 insertions(+), 183 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/nfs-utils/nfs-utils/0001-nfs-utils-statd-fix-a-segfault-caused-by-improper-us.patch
 delete mode 100644 
meta/recipes-connectivity/nfs-utils/nfs-utils/fix-protocol-minor-version-fall-back.patch
 rename meta/recipes-connectivity/nfs-utils/{nfs-utils_1.3.3.bb => 
nfs-utils_1.3.4.bb} (95%)

diff --git 
a/meta/recipes-connectivity/nfs-utils/files/nfs-utils-debianize-start-statd.patch
 
b/meta/recipes-connectivity/nfs-utils/files/nfs-utils-debianize-start-statd.patch
index 8500229..ede0dce 100644
--- 
a/meta/recipes-connectivity/nfs-utils/files/nfs-utils-debianize-start-statd.patch
+++ 
b/meta/recipes-connectivity/nfs-utils/files/nfs-utils-debianize-start-statd.patch
@@ -9,17 +9,18 @@ Signed-off-by: Li Wang <li.w...@windriver.com>
 Signed-off-by: Roy Li <rongqing...@windriver.com>
 Signed-off-by: Wenzong Fan <wenzong@windriver.com>
 ---
- utils/statd/start-statd | 9 -
- 1 file changed, 8 insertions(+), 1 deletion(-)
+ utils/statd/start-statd | 10 +-
+ 1 file changed, 9 insertions(+), 1 deletion(-)
 
 diff --git a/utils/statd/start-statd b/utils/statd/start-statd
-index ec9383b..3969b8c 100755
+index 2fd6039..f591b34 100755
 --- a/utils/statd/start-statd
 +++ b/utils/statd/start-statd
-@@ -6,6 +6,13 @@
- # site.
- PATH="/sbin:/usr/sbin:/bin:/usr/bin"
- 
+@@ -17,6 +17,14 @@ then
+ # statd already running - must have been slow to respond.
+ exit 0
+ fi
++
 +# Read config
 +DEFAULTFILE=/etc/default/nfs-common
 +NEED_IDMAPD=
@@ -28,14 +29,14 @@ index ec9383b..3969b8c 100755
 +fi
 +
  # First try systemd if it's installed.
- if systemctl --help >/dev/null 2>&1; then
+ if [ -d /run/systemd/system ]; then
  # Quit only if the call worked.
-@@ -13,4 +20,4 @@ if systemctl --help >/dev/null 2>&1; then
- fi
+@@ -25,4 +33,4 @@ fi
  
+ cd /
  # Fall back to launching it ourselves.
 -exec rpc.statd --no-notify
 +exec rpc.statd --no-notify $STATDOPTS
 -- 
-1.9.1
+2.6.6
 
diff --git 
a/meta/recipes-connectivity/nfs-utils/nfs-utils/0001-nfs-utils-statd-fix-a-segfault-caused-by-improper-us.patch
 
b/meta/recipes-connectivity/nfs-utils/nfs-utils/0001-nfs-utils-statd-fix-a-segfault-caused-by-improper-us.patch
deleted file mode 100644
index de0b045..000
--- 
a/meta/recipes-connectivity/nfs-utils/nfs-utils/0001-nfs-utils-statd-fix-a-segfault-caused-by-improper-us.patch
+++ /dev/null
@@ -1,113 +0,0 @@
-Upstream-Status: Pending
-
-Subject: nfs-utils/statd: fix a segfault caused by improper usage of RPC 
interface
-
-There is a hack which uses the bottom-level RPC improperly as below
-in the current statd implementation:
-insert a socket in the svc_fdset without a corresponding transport handle
-and passes the socket to the svc_getreqset subroutine, this usage causes
-a segfault of statd on a huge amount of sm-notifications.
-
-Fix the issue by separating the non-RPC-server sock from RPC dispatcher.
-
-Signed-off-by: Shan Hai <shan@windriver.com>
-Signed-off-by: Chen Qi <qi.c...@windriver.com>

- utils/statd/rmtcall.c | 1 -
- utils/statd/statd.c   | 5 +++--
- utils/statd/statd.h   | 2 +-
- utils/statd/svc_run.c | 8 ++--
- 4 files changed, 10 insertions(+), 6 deletions(-)
-
-diff --git a/utils/statd/rmtcall.c b/utils/statd/rmtcall.c
-index fd576d9..cde091b 100644
 a/utils/statd/rmtcall.c
-+++ b/utils/statd/rmtcall.c
-@@ -104,7 +104,6 @@ statd_get_socket(void)
-   if (sockfd < 0)
-   return -1;
- 
--  FD_SET(sockfd, _FDSET);
-   return sockfd;
- }
- 
-diff --git a/utils/statd/statd.c b/utils/statd/statd.c
-index 51a016e..e21a259 100644
 a/utils/statd/statd.c
-+++ b/utils/statd/statd.c
-@@ -247,6 +247,7 @@ int main (int argc, char **argv)
-   int port = 0, out_port = 0;
-   int nlm_udp = 0, nlm_tcp = 0;
-   struct rlimit rlim;
-+  int notify_sockfd;
- 
-   int pipefds[2] = { -1, -1};
-   char status;
-@@ -473,7 +474,7 @@ int main (int argc, char **argv)
-   }
- 
-   /* Make sure we have a privilege port for calling into the kernel */
--  if (statd_get_socket() < 0)
-+  if ((notify_sockfd = statd_get_socket()) < 0)
-   exit(1);
- 
-   /* If sm-notify didn't take all the state files, load
-@@ -528,7 +529,7 @@ int main (int argc, char **argv)
-* Handle incoming requests:  SM_NOTIFY socket requests, as
-* well as callbacks from lockd.
-*/
--  my_svc_run();   /* I rolled

[OE-core] [PATCH 1/1] scripst/runqemu: Allow to use qemu from host.

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

This will add support to use qemu from the running host,
with this is possible to put qemu-native in ASSUME_PROVIDED
variable.

By default it will try to get qemu from the build sysroot,
and only if it fails will try to use the host's qemu.

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 scripts/runqemu | 15 +++
 1 file changed, 15 insertions(+)

diff --git a/scripts/runqemu b/scripts/runqemu
index af25423..0a9cb94 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -906,6 +906,21 @@ class BaseConfig(object):
 raise Exception("Failed to boot, QB_SYSTEM_NAME is NULL!")
 
 qemu_bin = '%s/%s' % (self.get('STAGING_BINDIR_NATIVE'), qemu_system)
+
+# It is possible to have qemu-native in ASSUME_PROVIDED, and it won't
+# find QEMU in sysroot, it needs to use host's qemu.
+if not os.path.exists(qemu_bin):
+logger.info("QEMU binary not found in %s, trying host's QEMU" % 
qemu_bin)
+for path in (os.environ['PATH'] or '').split(':'):
+qemu_bin_tmp = os.path.join(path, qemu_system)
+logger.info("Trying: %s" % qemu_bin_tmp)
+if os.path.exists(qemu_bin_tmp):
+qemu_bin = qemu_bin_tmp
+if not os.path.isabs(qemu_bin):
+qemu_bin = os.path.abspath(qemu_bin)
+logger.info("Using host's QEMU: %s" % qemu_bin)
+break
+
 if not os.access(qemu_bin, os.X_OK):
 raise OEPathError("No QEMU binary '%s' could be found" % qemu_bin)
 
-- 
2.7.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] nfs-utils: 1.3.3 -> 1.3.4

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 .../files/nfs-utils-debianize-start-statd.patch|  23 +++--
 ...tatd-fix-a-segfault-caused-by-improper-us.patch | 113 -
 .../{nfs-utils_1.3.3.bb => nfs-utils_1.3.4.bb} |   5 +-
 3 files changed, 14 insertions(+), 127 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/nfs-utils/nfs-utils/0001-nfs-utils-statd-fix-a-segfault-caused-by-improper-us.patch
 rename meta/recipes-connectivity/nfs-utils/{nfs-utils_1.3.3.bb => 
nfs-utils_1.3.4.bb} (96%)

diff --git 
a/meta/recipes-connectivity/nfs-utils/files/nfs-utils-debianize-start-statd.patch
 
b/meta/recipes-connectivity/nfs-utils/files/nfs-utils-debianize-start-statd.patch
index 8500229..ede0dce 100644
--- 
a/meta/recipes-connectivity/nfs-utils/files/nfs-utils-debianize-start-statd.patch
+++ 
b/meta/recipes-connectivity/nfs-utils/files/nfs-utils-debianize-start-statd.patch
@@ -9,17 +9,18 @@ Signed-off-by: Li Wang <li.w...@windriver.com>
 Signed-off-by: Roy Li <rongqing...@windriver.com>
 Signed-off-by: Wenzong Fan <wenzong@windriver.com>
 ---
- utils/statd/start-statd | 9 -
- 1 file changed, 8 insertions(+), 1 deletion(-)
+ utils/statd/start-statd | 10 +-
+ 1 file changed, 9 insertions(+), 1 deletion(-)
 
 diff --git a/utils/statd/start-statd b/utils/statd/start-statd
-index ec9383b..3969b8c 100755
+index 2fd6039..f591b34 100755
 --- a/utils/statd/start-statd
 +++ b/utils/statd/start-statd
-@@ -6,6 +6,13 @@
- # site.
- PATH="/sbin:/usr/sbin:/bin:/usr/bin"
- 
+@@ -17,6 +17,14 @@ then
+ # statd already running - must have been slow to respond.
+ exit 0
+ fi
++
 +# Read config
 +DEFAULTFILE=/etc/default/nfs-common
 +NEED_IDMAPD=
@@ -28,14 +29,14 @@ index ec9383b..3969b8c 100755
 +fi
 +
  # First try systemd if it's installed.
- if systemctl --help >/dev/null 2>&1; then
+ if [ -d /run/systemd/system ]; then
  # Quit only if the call worked.
-@@ -13,4 +20,4 @@ if systemctl --help >/dev/null 2>&1; then
- fi
+@@ -25,4 +33,4 @@ fi
  
+ cd /
  # Fall back to launching it ourselves.
 -exec rpc.statd --no-notify
 +exec rpc.statd --no-notify $STATDOPTS
 -- 
-1.9.1
+2.6.6
 
diff --git 
a/meta/recipes-connectivity/nfs-utils/nfs-utils/0001-nfs-utils-statd-fix-a-segfault-caused-by-improper-us.patch
 
b/meta/recipes-connectivity/nfs-utils/nfs-utils/0001-nfs-utils-statd-fix-a-segfault-caused-by-improper-us.patch
deleted file mode 100644
index de0b045..000
--- 
a/meta/recipes-connectivity/nfs-utils/nfs-utils/0001-nfs-utils-statd-fix-a-segfault-caused-by-improper-us.patch
+++ /dev/null
@@ -1,113 +0,0 @@
-Upstream-Status: Pending
-
-Subject: nfs-utils/statd: fix a segfault caused by improper usage of RPC 
interface
-
-There is a hack which uses the bottom-level RPC improperly as below
-in the current statd implementation:
-insert a socket in the svc_fdset without a corresponding transport handle
-and passes the socket to the svc_getreqset subroutine, this usage causes
-a segfault of statd on a huge amount of sm-notifications.
-
-Fix the issue by separating the non-RPC-server sock from RPC dispatcher.
-
-Signed-off-by: Shan Hai <shan@windriver.com>
-Signed-off-by: Chen Qi <qi.c...@windriver.com>

- utils/statd/rmtcall.c | 1 -
- utils/statd/statd.c   | 5 +++--
- utils/statd/statd.h   | 2 +-
- utils/statd/svc_run.c | 8 ++--
- 4 files changed, 10 insertions(+), 6 deletions(-)
-
-diff --git a/utils/statd/rmtcall.c b/utils/statd/rmtcall.c
-index fd576d9..cde091b 100644
 a/utils/statd/rmtcall.c
-+++ b/utils/statd/rmtcall.c
-@@ -104,7 +104,6 @@ statd_get_socket(void)
-   if (sockfd < 0)
-   return -1;
- 
--  FD_SET(sockfd, _FDSET);
-   return sockfd;
- }
- 
-diff --git a/utils/statd/statd.c b/utils/statd/statd.c
-index 51a016e..e21a259 100644
 a/utils/statd/statd.c
-+++ b/utils/statd/statd.c
-@@ -247,6 +247,7 @@ int main (int argc, char **argv)
-   int port = 0, out_port = 0;
-   int nlm_udp = 0, nlm_tcp = 0;
-   struct rlimit rlim;
-+  int notify_sockfd;
- 
-   int pipefds[2] = { -1, -1};
-   char status;
-@@ -473,7 +474,7 @@ int main (int argc, char **argv)
-   }
- 
-   /* Make sure we have a privilege port for calling into the kernel */
--  if (statd_get_socket() < 0)
-+  if ((notify_sockfd = statd_get_socket()) < 0)
-   exit(1);
- 
-   /* If sm-notify didn't take all the state files, load
-@@ -528,7 +529,7 @@ int main (int argc, char **argv)
-* Handle incoming requests:  SM_NOTIFY socket requests, as
-* well as callbacks from lockd.
-*/
--  my_svc_run();   /* I rolled my own, Olaf made it better... */
-+  my_svc_run(notify_sockfd);  /* I rolled my own, Olaf made 
it better... */
- 
-   /* Only get here when simulating

Re: [OE-core] [PATCH 1/1] toolchain-shar-relocate.sh: Add error-handling

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

The relocation script of the SDK doesn't have enough
error handling when replacing host perl with SDK perl
or changing the symlinks. This will add those checks
along with a sanity check of xargs.

[YOCTO #10114]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/files/toolchain-shar-relocate.sh | 20 +++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/meta/files/toolchain-shar-relocate.sh 
b/meta/files/toolchain-shar-relocate.sh
index d4bcf0e..e3c1001 100644
--- a/meta/files/toolchain-shar-relocate.sh
+++ b/meta/files/toolchain-shar-relocate.sh
@@ -1,3 +1,8 @@
+if ! xargs --version > /dev/null 2>&1; then
+   echo "xargs is required by the relocation script, please install it 
first. Abort!"
+   exit 1
+fi
+
 # fix dynamic loader paths in all ELF SDK binaries
 native_sysroot=$($SUDO_EXEC cat $env_setup_script |grep 
'OECORE_NATIVE_SYSROOT='|cut -d'=' -f2|tr -d '"')
 dl_path=$($SUDO_EXEC find $native_sysroot/lib -name "ld-linux*")
@@ -7,6 +12,10 @@ if [ "$dl_path" = "" ] ; then
 fi
 executable_files=$($SUDO_EXEC find $native_sysroot -type f \
\( -perm -0100 -o -perm -0010 -o -perm -0001 \) -printf "'%h/%f' ")
+if [ "x$executable_files" = "x" ]; then
+   echo "SDK relocate failed, could not get executalbe files"
+   exit 1
+fi
 
 tdir=`mktemp -d`
 if [ x$tdir = x ] ; then
@@ -14,7 +23,7 @@ if [ x$tdir = x ] ; then
exit 1
 fi
 cat <> $tdir/relocate_sdk.sh
-#!/bin/bash
+#!/bin/sh
 for py in python python2 python3
 do
PYTHON=\`which \${py} 2>/dev/null\`
@@ -53,9 +62,18 @@ done | xargs -n100 file | grep 
":.*\(ASCII\|script\|source\).*text" | \
 -e "s:^#! */usr/bin/perl.*:#! /usr/bin/env perl:g" \
 -e "s: /usr/bin/perl: /usr/bin/env perl:g"
 
+if [ $? -ne 0 ]; then
+   echo "Failed to replace perl. Relocate script failed. Abort!"
+   exit 1
+fi
+
 # change all symlinks pointing to @SDKPATH@
 for l in $($SUDO_EXEC find $native_sysroot -type l); do
$SUDO_EXEC ln -sfn $(readlink $l|$SUDO_EXEC sed -e 
"s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:") $l
+   if [ $? -ne 0 ]; then
+   echo "Failed to setup symlinks. Relocate script failed. Abort!"
+   exit 1
+fi
 done
 
 echo done
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] testimage.bbclass: Add package manager dependency

[mariano . lopez]

From: Mariano Lopez <mariano.lo...@linux.intel.com>

The feature to install packages in the target requires to
build the package manager. It would fail, with very obtuse
errors, if a test requires to install something and the
package manager hasn't been build. This will add the package
manager as dependency for testimage.

[YOCTO #10260]

Signed-off-by: Mariano Lopez <mariano.lo...@linux.intel.com>
---
 meta/classes/testimage.bbclass | 4 
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 263d539..a908f92 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -68,6 +68,10 @@ TESTIMAGEDEPENDS_qemuall = "qemu-native:do_populate_sysroot 
qemu-helper-native:d
 TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 
'cpio-native:do_populate_sysroot', '', d)}"
 TESTIMAGEDEPENDS_qemuall += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 
'cpio-native:do_populate_sysroot', '', d)}"
 TESTIMAGEDEPENDS_qemuall += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 
'createrepo-native:do_populate_sysroot', '', d)}"
+TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 
'python-smartpm-native:do_populate_sysroot', '', d)}"
+TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'ipk', 
'opkg-utils-native:do_populate_sysroot', '', d)}"
+TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'deb', 
'apt-native:do_populate_sysroot', '', d)}"
+
 
 TESTIMAGELOCK = "${TMPDIR}/testimage.lock"
 TESTIMAGELOCK_qemuall = ""
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core