Re: [OE-core][PATCH v2] devicetree.bbclass: Allow selection of dts files to build

[Nathan Rossi]

On Fri, 21 Apr 2023 at 22:47, Petr Kubizňák - 2N  wrote:
>
> > kernel-devicetree.bbclass does the same translation dtb->dts
>
> I mean dts->dtb, of course. The point is dtb's are expected on the input.
>
> Petr
>
> 
> From: Petr Kubizňák - 2N 
> Sent: Friday, April 21, 2023 2:30 PM
> To: Nathan Rossi
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core][PATCH v2] devicetree.bbclass: Allow selection of dts 
> files to build
>
> Hi Nathan,
>
> Thank you for great suggestions. All of them are valid and I will update the 
> patch once we discuss the .dtb -> .dts mapping.
>
> > I am also curious why you need to handle this mapping of .dtb -> .dts.
> > Do you have an example recipe that is using this class in this way?
> > Because the ".dtb" in KERNEL_DEVICETREE is specifically because the
> > KERNEL_DEVICETREE variable is pointing at kernel makefile targets.
>
> I don't have an example of a particular recipe, but how about 
> EXTERNAL_KERNEL_DEVICETREE variable, used in kernel-fitimage.bbclass? Albeit 
> I don't use this variable in my recipes, I believe it could be used for both 
> classes now with simple assignment DT_FILES = EXTERNAL_KERNEL_DEVICETREE.
>
> And if for whatever reason someone uses a custom variable (like me), they 
> would still expect dtb's rather than dts's. I didn't find any example of 
> using lists of dts files. Contrary, kernel-devicetree.bbclass does the same 
> translation dtb->dts and it even warns about dts being provided (see 
> normalize_dtb()). I didn't want to be that strict here (if someone has a list 
> of dts files, while not to use it directly) but in general, I expect a list 
> of dtb files will be much more common.

So just trying to make the DT_FILES somewhat consistent with the
KERNEL_DEVICETREE, seems fine. I was only curious in case you had some
special path handling that might be better solved by overriding the
devicetree_do_compile and calling the associated compile functions
directly.

The v3 patch looks good :).

Thanks,
Nathan


>
> Kind Regards,
> Petr
>
> 
> From: Nathan Rossi 
> Sent: Friday, April 21, 2023 1:44 PM
> To: Petr Kubizňák - 2N
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core][PATCH v2] devicetree.bbclass: Allow selection of dts 
> files to build
>
> On Fri, 21 Apr 2023 at 18:41, Petr Kubizňák  wrote:
> >
> > Add DT_FILES variable to allow the user of the class to select specific
> > dts files to build. This is useful for packages featuring dts files
> > for multiple machines.
> >
> > Since many machine configs contain a list of dtb files
> > (e.g. KERNEL_DEVICETREE), DT_FILES works with both dts and dtb files.
> >
> > Signed-off-by: Petr Kubizňák 
> > ---
> >  meta/classes-recipe/devicetree.bbclass | 10 +-
> >  1 file changed, 9 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes-recipe/devicetree.bbclass 
> > b/meta/classes-recipe/devicetree.bbclass
> > index ed2a92e447..8f348f1370 100644
> > --- a/meta/classes-recipe/devicetree.bbclass
> > +++ b/meta/classes-recipe/devicetree.bbclass
> > @@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \
> >
> >  DT_INCLUDE[doc] = "Search paths to be made available to both the device 
> > tree compiler and preprocessor for inclusion."
> >  DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}"
> > -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select 
> > dts files that are not in source (e.g. generated)."
> > +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. 
> > Defaults to source directory."
> >  DT_FILES_PATH ?= "${S}"
> > +DT_FILES[doc] = "Space-separated list of dts or dtb files to build. If 
> > empty, all dts files are built."
>
> Worth noting in the doc string that DT_FILES paths are relative to
> DT_FILES_PATH.
>
> > +DT_FILES ?= ""
> >
> >  DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as 
> > extra space typically for additional properties during boot."
> >  DT_PADDING_SIZE ??= "0x3000"
> > @@ -125,7 +127,10 @@ def devicetree_compile(dtspath, includes, d):
> >  subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, 
> > stderr=subprocess.STDOUT)
> >
> >  python devicetree_do_compile() {
> > +import re
> >  includes = expand_includes("DT_INCLUDE", d)
> > +dtfiles = d.getVar("DT_FILES")
&g

Re: [OE-core][PATCH v2] devicetree.bbclass: Allow selection of dts files to build

[Nathan Rossi]

On Fri, 21 Apr 2023 at 18:41, Petr Kubizňák  wrote:
>
> Add DT_FILES variable to allow the user of the class to select specific
> dts files to build. This is useful for packages featuring dts files
> for multiple machines.
>
> Since many machine configs contain a list of dtb files
> (e.g. KERNEL_DEVICETREE), DT_FILES works with both dts and dtb files.
>
> Signed-off-by: Petr Kubizňák 
> ---
>  meta/classes-recipe/devicetree.bbclass | 10 +-
>  1 file changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes-recipe/devicetree.bbclass 
> b/meta/classes-recipe/devicetree.bbclass
> index ed2a92e447..8f348f1370 100644
> --- a/meta/classes-recipe/devicetree.bbclass
> +++ b/meta/classes-recipe/devicetree.bbclass
> @@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \
>
>  DT_INCLUDE[doc] = "Search paths to be made available to both the device tree 
> compiler and preprocessor for inclusion."
>  DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}"
> -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select 
> dts files that are not in source (e.g. generated)."
> +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. 
> Defaults to source directory."
>  DT_FILES_PATH ?= "${S}"
> +DT_FILES[doc] = "Space-separated list of dts or dtb files to build. If 
> empty, all dts files are built."

Worth noting in the doc string that DT_FILES paths are relative to
DT_FILES_PATH.

> +DT_FILES ?= ""
>
>  DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as 
> extra space typically for additional properties during boot."
>  DT_PADDING_SIZE ??= "0x3000"
> @@ -125,7 +127,10 @@ def devicetree_compile(dtspath, includes, d):
>  subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, 
> stderr=subprocess.STDOUT)
>
>  python devicetree_do_compile() {
> +import re
>  includes = expand_includes("DT_INCLUDE", d)
> +dtfiles = d.getVar("DT_FILES")

If this populates the variable dtfiles with DT_FILES already split it
can fall back to os.listdir if empty, this avoids the need for the "if
filter..." logic within the loop:

dtfiles = d.getVar("DT_FILES").split()
dtfiles = [ for dtfiles in dtfiles]
for dts in dtfiles or os.listdir(listpath):

> +filter = [ re.sub("dtbo?", "dts", dtfile) for dtfile in dtfiles.split() ]

This regex is a bit problematic, it will substitute any part of the
filename and not just the extension suffix. E.g. "test-dtbo.dts" ->
"test-dts.dts". The expression might need to be something similar to
"\.dtbo?$".

I am also curious why you need to handle this mapping of .dtb -> .dts.
Do you have an example recipe that is using this class in this way?
Because the ".dtb" in KERNEL_DEVICETREE is specifically because the
KERNEL_DEVICETREE variable is pointing at kernel makefile targets.

Regards,
Nathan

>  listpath = d.getVar("DT_FILES_PATH")
>  for dts in os.listdir(listpath):
>  dtspath = os.path.join(listpath, dts)
> @@ -134,6 +139,9 @@ python devicetree_do_compile() {
>  continue # skip non-.dts files and non-overlay files
>  except:
>  continue # skip if can't determine if overlay
> +if filter:
> +if not(dts in filter):
> +continue # skip if dts not in defined filter
>  devicetree_compile(dtspath, includes, d)
>  }
>
> --
> 2.30.2
>
>
> 
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#180283): 
https://lists.openembedded.org/g/openembedded-core/message/180283
Mute This Topic: https://lists.openembedded.org/mt/98406619/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH 4/4] package: Fix handling of minidebuginfo with newer binutils

[Nathan Rossi]

From: Nathan Rossi 

Newer versions of binutils (2.38+) have changed how the
"--only-keep-debug" of objcopy behaves when stripping non-debug sections
from an ELF.

  
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=68f543154e92ab0f5d6c569e0fa143f5e8bd2d80

This change causes associated sections to be correctly marked as NOBITS
with the section contents removed from the output. The side effect is
that this causes issues with objcopy's ability to perform symbol and
relocation stripping (-S/--strip-all) on the debug split ELF, such that
with some object files (e.g. kernel modules) objcopy fails to strip
symbols/relocations with an error like the following:

  .../.debug/nls_cp950.ko[.rodata]: file truncated

Because of this it is now problematic to generate minidebuginfo for
these types of ELF objects. However it is not typically useful to inject
minidebuginfo into these types of ELFs, and other distributions (e.g.
Fedora, referring to find-debuginfo.sh of debugedit) only insert
minidebuginfo into executables and shared libraries.

This change causes the minidebuginfo injection to only apply to EXEC/DYN
type ELFs, which limits the injection to executables and shared
libraires.

Additionally this change fixes the parsing of the sections from the
"readelf -W -S" output which was not accounting for the section index
column having leading spaces for single digit index values e.g. "[ 1]".

Signed-off-by: Nathan Rossi 
Signed-off-by: Alexandre Belloni 
(cherry picked from commit 2084cfcb3d15db3e02637f1cd63ab9c997f38a65)
---
 meta/classes/package.bbclass | 21 ++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 97e97d2703..8b11fdd155 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -484,16 +484,31 @@ def inject_minidebuginfo(file, dvar, dv, d):
 bb.debug(1, 'ELF file {} has no debuginfo, skipping minidebuginfo 
injection'.format(file))
 return
 
+# minidebuginfo does not make sense to apply to ELF objects other than
+# executables and shared libraries, skip applying the minidebuginfo
+# generation for objects like kernel modules.
+for line in subprocess.check_output([readelf, '-h', debugfile], 
universal_newlines=True).splitlines():
+if not line.strip().startswith("Type:"):
+continue
+elftype = line.split(":")[1].strip()
+if not any(elftype.startswith(i) for i in ["EXEC", "DYN"]):
+bb.debug(1, 'ELF file {} is not executable/shared, skipping 
minidebuginfo injection'.format(file))
+return
+break
+
 # Find non-allocated PROGBITS, NOTE, and NOBITS sections in the debuginfo.
 # We will exclude all of these from minidebuginfo to save space.
 remove_section_names = []
 for line in subprocess.check_output([readelf, '-W', '-S', debugfile], 
universal_newlines=True).splitlines():
-fields = line.split()
-if len(fields) < 8:
+# strip the leading "  [ 1]" section index to allow splitting on space
+if ']' not in line:
+continue
+fields = line[line.index(']') + 1:].split()
+if len(fields) < 7:
 continue
 name = fields[0]
 type = fields[1]
-flags = fields[7]
+flags = fields[6]
 # .debug_ sections will be removed by objcopy -S so no need to 
explicitly remove them
 if name.startswith('.debug_'):
 continue
---
2.38.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173231): 
https://lists.openembedded.org/g/openembedded-core/message/173231
Mute This Topic: https://lists.openembedded.org/mt/95014778/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH 3/4] glibc-locale: Do not INHIBIT_DEFAULT_DEPS

[Nathan Rossi]

From: Nathan Rossi 

The glibc-locale recipe already partially depends on the base depends in
order to satisfy the do_package dependency on binutils. However since
commit d6ffd683bf6 NM has defaulted to gcc-nm, meaning do_package
depends on gcc (for minidebuginfo).

Whilst the do_package task could handle having the dependencies
explicitly defined (either in glibc-locale or in package.bbclass),
setting these would require some amount of conditional dependency
configuration (cross/crosssdk/etc.). Since both binutils and gcc are
already dependencies of virtual/libc (although compilerlibs is not),
having glibc-locale not inhibit the default depends simplifies the
handling of this situation for both glibc-locale and package.bbclass.

Signed-off-by: Nathan Rossi 
Signed-off-by: Alexandre Belloni 
(cherry picked from commit 8a40d0a6039e87a5b4b26a0e84dd797fe5c75cba)
---
 meta/recipes-core/glibc/glibc-locale.inc | 11 +++
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
b/meta/recipes-core/glibc/glibc-locale.inc
index 7c14abfe99..7f70b3ca4f 100644
--- a/meta/recipes-core/glibc/glibc-locale.inc
+++ b/meta/recipes-core/glibc/glibc-locale.inc
@@ -5,14 +5,9 @@ SUMMARY = "Locale data from glibc"
 BPN = "glibc"
 LOCALEBASEPN = "${MLPREFIX}glibc"
 
-# glibc-collateral.inc inhibits all default deps, but do_package needs objcopy
-# ERROR: objcopy failed with exit code 127 (cmd was 'i586-webos-linux-objcopy' 
--only-keep-debug 'glibc-locale/2.17-r0/package/usr/lib/gconv/IBM1166.so' 
'glibc-locale/2.17-r0/package/usr/lib/gconv/.debug/IBM1166.so')
-# ERROR: Function failed: split_and_strip_files
-BINUTILSDEP = "virtual/${MLPREFIX}${TARGET_PREFIX}binutils:do_populate_sysroot"
-BINUTILSDEP:class-nativesdk = 
"virtual/${TARGET_PREFIX}binutils-crosssdk:do_populate_sysroot"
-do_package[depends] += "${BINUTILSDEP}"
-
-DEPENDS += "virtual/libc"
+# Do not inhibit default deps, do_package requires binutils/gcc for
+# objcopy/gcc-nm and glibc-locale depends on virtual/libc directly.
+INHIBIT_DEFAULT_DEPS = ""
 
 # Binary locales are generated at build time if ENABLE_BINARY_LOCALE_GENERATION
 # is set. The idea is to avoid running localedef on the target (at first boot)
---
2.38.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173230): 
https://lists.openembedded.org/g/openembedded-core/message/173230
Mute This Topic: https://lists.openembedded.org/mt/95014776/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH 2/4] oeqa/selftest/minidebuginfo: Create selftest for minidebuginfo

[Nathan Rossi]

From: Nathan Rossi 

Add a new selftest to validate minidebuginfo support. This selftest
builds a complete target image with PACKAGE_MINIDEBUGINFO enabled. ELFs
included in the image are expected to have minidebuginfo included in the
resulting executables and shared libraries, the self test validates this
by unpacking the image and checking for the associated ".gnu_debugdata"
section on busybox and libc ELFs.

Signed-off-by: Nathan Rossi 
Signed-off-by: Alexandre Belloni 
(cherry picked from commit 5063a31ad05b75ec6ac12158fe759e81fcdb1585)
---
The commit from master relies on the 'target_sys' parameter of runCmd,
this was introduced in afa3d3ba00b40fd29e9852eeaa2c2c9b68f18659
(langdale and later). That commit could be backported alongside this
commit however it adds '/bin' to the PATH by default which may cause
issues (?). As such this commit is modified to add the TARGET_SYS path
into the PATH environment passed into the runCmd call.
---
 meta/lib/oeqa/selftest/cases/minidebuginfo.py | 49 +++
 1 file changed, 49 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/minidebuginfo.py

diff --git a/meta/lib/oeqa/selftest/cases/minidebuginfo.py 
b/meta/lib/oeqa/selftest/cases/minidebuginfo.py
new file mode 100644
index 00..414dad64a3
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/minidebuginfo.py
@@ -0,0 +1,49 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+import os
+import subprocess
+import tempfile
+import shutil
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, runCmd
+
+
+class Minidebuginfo(OESelftestTestCase):
+def test_minidebuginfo(self):
+target_sys = get_bb_var("TARGET_SYS")
+binutils = "binutils-cross-{}".format(get_bb_var("TARGET_ARCH"))
+
+self.write_config("""
+PACKAGE_MINIDEBUGINFO = "1"
+IMAGE_FSTYPES = "tar.bz2"
+""")
+bitbake("core-image-minimal 
{}:do_addto_recipe_sysroot".format(binutils))
+
+deploy_dir = get_bb_var("DEPLOY_DIR_IMAGE")
+native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", binutils)
+readelf = get_bb_var("READELF", "core-image-minimal")
+
+# add usr/bin/${TARGET_SYS} to PATH
+env = os.environ.copy()
+paths = [os.path.join(native_sysroot, "usr", "bin", target_sys)]
+paths += env["PATH"].split(":")
+env["PATH"] = ":".join(paths)
+
+# confirm that executables and shared libraries contain an ELF section
+# ".gnu_debugdata" which stores minidebuginfo.
+with tempfile.TemporaryDirectory(prefix = "unpackfs-") as unpackedfs:
+filename = os.path.join(deploy_dir, 
"core-image-minimal-{}.tar.bz2".format(self.td["MACHINE"]))
+shutil.unpack_archive(filename, unpackedfs)
+
+r = runCmd([readelf, "-W", "-S", os.path.join(unpackedfs, "bin", 
"busybox")],
+native_sysroot = native_sysroot, env = env)
+self.assertIn(".gnu_debugdata", r.output)
+
+r = runCmd([readelf, "-W", "-S", os.path.join(unpackedfs, "lib", 
"libc.so.6")],
+native_sysroot = native_sysroot, env = env)
+self.assertIn(".gnu_debugdata", r.output)
+
---
2.38.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173229): 
https://lists.openembedded.org/g/openembedded-core/message/173229
Mute This Topic: https://lists.openembedded.org/mt/95014775/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH 1/4] oeqa/selftest/lic_checksum: Cleanup changes to emptytest include

[Nathan Rossi]

From: Nathan Rossi 

Config written to the emptytest include file is invalid after the test
has cleaned up its temporary directories resulting in the emptytest
recipe potentially having invalid content when parsed by successive
bitbake runs.

This presents the following error in tests after lic_checksum execution,
e.g. 'oe-selftest -r lic_checksum recipetool'

  ERROR: .../emptytest.bb: Unable to get checksum for emptytest SRC_URI entry 
tmpn_nyosnq: file could not be found

Remove the recipe include content once the bitbake runs are completed in
each lic_checksum test case.

Signed-off-by: Nathan Rossi 
Signed-off-by: Alexandre Belloni 
(cherry picked from commit 087df767a64b271b503d714df3df6d8b3caad1c0)
---
 meta/lib/oeqa/selftest/cases/lic_checksum.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/lic_checksum.py 
b/meta/lib/oeqa/selftest/cases/lic_checksum.py
index 8f1226e6a5..bc0a2b5d8e 100644
--- a/meta/lib/oeqa/selftest/cases/lic_checksum.py
+++ b/meta/lib/oeqa/selftest/cases/lic_checksum.py
@@ -26,6 +26,7 @@ LIC_FILES_CHKSUM = 
"file://%s;md5=d41d8cd98f00b204e9800998ecf8427e"
 SRC_URI = "file://%s;md5=d41d8cd98f00b204e9800998ecf8427e"
 """ % (urllib.parse.quote(lic_path), urllib.parse.quote(lic_path)))
 result = bitbake(bitbake_cmd)
+self.delete_recipeinc('emptytest')
 
 
 # Verify that changing a license file that has an absolute path causes
@@ -51,5 +52,6 @@ SRC_URI = "file://%s;md5=d41d8cd98f00b204e9800998ecf8427e"
 f.write("data")
 
 result = bitbake(bitbake_cmd, ignore_status=True)
+self.delete_recipeinc('emptytest')
 if error_msg not in result.output:
 raise AssertionError(result.output)
---
2.38.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173228): 
https://lists.openembedded.org/g/openembedded-core/message/173228
Mute This Topic: https://lists.openembedded.org/mt/95014773/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 3/3] package: Fix handling of minidebuginfo with newer binutils

[Nathan Rossi]

On Mon, 7 Nov 2022 at 01:55, Alexandre Belloni
 wrote:
>
> Hello,
>
> This series causes the following failure on the autobuilders:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/4354/steps/14/logs/stdio

I looked into these failures, the SRC_URI error appears to be an issue
with the lic_checksum test itself, and it presents without this
series. I have posted a patch to resolve it
https://lists.openembedded.org/g/openembedded-core/message/172945.

The other errors around "test.bb" bash hash changes do not appear to
be errors from the recipetools bitbake execution (note the "2 ERROR
messages"). I suspect they are just captured during the fetch selftest
cases in a common stderr buffer? I was unable to reproduce them
locally when running fetch/recipetool.

Regards,
Nathan



>
> 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add: ERROR
> 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add_bindir: ERROR
> 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add_machine: ERROR
> 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_alternatives: 
> ERROR
> 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_basic: ERROR
> 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_binary: ERROR
> 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_extlayer: ERROR
> 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_func: ERROR
> 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_glob: ERROR
> 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_todir_glob: 
> ERROR
> 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_invalid: ERROR
> 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_orig: ERROR
> 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_patch: ERROR
> 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_postinstall: ERROR
> 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_renamed: ERROR
> 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_script: ERROR
> 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_subdir: ERROR
> 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_todir: ERROR
> 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - 
> recipetool.RecipetoolAppendTests.test_recipetool_appendfile_wildcard: ERROR
>
> On 03/11/2022 07:56:06+, Nathan Rossi wrote:
> > From: Nathan Rossi 
> >
> > Newer versions of binutils (2.38+) have changed how the
> > "--only-keep-debug" of objcopy behaves when stripping non-debug sections
> > from an ELF.
> >
> >   
> > https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=68f543154e92ab0f5d6c569e0fa143f5e8bd2d80
> >
> > This change causes associated sections to be correctly marked as NOBITS
> > with the section contents removed from the output. The side effect is
> > that this causes issues with objcopy's ability to perform symbol and
> > relocation stripping (-S/--strip-all) on the debug split ELF, such that
> > with some object files (e.g. kernel modules) objcopy fails to strip
> > symbols/relocations with an error like the following:
> >
> >   .../.debug/nls_cp950.ko[.rodata]: file truncated
> >
> > Because of this it is now problematic to generate minidebuginfo for
> > these types of ELF objects. However it is not typically useful to inject
> > minidebuginfo into these types of ELFs, and other distributions (e.g.
> > Fedora, referring to find-debuginfo.sh of debugedit) only insert
> > minidebuginfo into executables and shared libraries.
> >
> > This change causes the minide

[OE-core] [PATCH] oeqa/selftest/lic_checksum: Cleanup changes to emptytest include

[Nathan Rossi]

From: Nathan Rossi 

Config written to the emptytest include file is invalid after the test
has cleaned up its temporary directories resulting in the emptytest
recipe potentially having invalid content when parsed by successive
bitbake runs.

This presents the following error in tests after lic_checksum execution,
e.g. 'oe-selftest -r lic_checksum recipetool'

  ERROR: .../emptytest.bb: Unable to get checksum for emptytest SRC_URI entry 
tmpn_nyosnq: file could not be found

Remove the recipe include content once the bitbake runs are completed in
each lic_checksum test case.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/lic_checksum.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/lic_checksum.py 
b/meta/lib/oeqa/selftest/cases/lic_checksum.py
index 5897a396d9..2d0b805b90 100644
--- a/meta/lib/oeqa/selftest/cases/lic_checksum.py
+++ b/meta/lib/oeqa/selftest/cases/lic_checksum.py
@@ -28,6 +28,7 @@ LIC_FILES_CHKSUM = 
"file://%s;md5=d41d8cd98f00b204e9800998ecf8427e"
 SRC_URI = "file://%s;md5=d41d8cd98f00b204e9800998ecf8427e"
 """ % (urllib.parse.quote(lic_path), urllib.parse.quote(lic_path)))
 result = bitbake(bitbake_cmd)
+self.delete_recipeinc('emptytest')
 
 
 # Verify that changing a license file that has an absolute path causes
@@ -53,5 +54,6 @@ SRC_URI = "file://%s;md5=d41d8cd98f00b204e9800998ecf8427e"
 f.write("data")
 
 result = bitbake(bitbake_cmd, ignore_status=True)
+self.delete_recipeinc('emptytest')
 if error_msg not in result.output:
 raise AssertionError(result.output)
---
2.37.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172945): 
https://lists.openembedded.org/g/openembedded-core/message/172945
Mute This Topic: https://lists.openembedded.org/mt/94883635/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 3/3] package: Fix handling of minidebuginfo with newer binutils

[Nathan Rossi]

From: Nathan Rossi 

Newer versions of binutils (2.38+) have changed how the
"--only-keep-debug" of objcopy behaves when stripping non-debug sections
from an ELF.

  
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=68f543154e92ab0f5d6c569e0fa143f5e8bd2d80

This change causes associated sections to be correctly marked as NOBITS
with the section contents removed from the output. The side effect is
that this causes issues with objcopy's ability to perform symbol and
relocation stripping (-S/--strip-all) on the debug split ELF, such that
with some object files (e.g. kernel modules) objcopy fails to strip
symbols/relocations with an error like the following:

  .../.debug/nls_cp950.ko[.rodata]: file truncated

Because of this it is now problematic to generate minidebuginfo for
these types of ELF objects. However it is not typically useful to inject
minidebuginfo into these types of ELFs, and other distributions (e.g.
Fedora, referring to find-debuginfo.sh of debugedit) only insert
minidebuginfo into executables and shared libraries.

This change causes the minidebuginfo injection to only apply to EXEC/DYN
type ELFs, which limits the injection to executables and shared
libraires.

Additionally this change fixes the parsing of the sections from the
"readelf -W -S" output which was not accounting for the section index
column having leading spaces for single digit index values e.g. "[ 1]".

Signed-off-by: Nathan Rossi 
---
 meta/classes-global/package.bbclass | 21 ++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/meta/classes-global/package.bbclass 
b/meta/classes-global/package.bbclass
index 2d985d8aff..7a0a428b30 100644
--- a/meta/classes-global/package.bbclass
+++ b/meta/classes-global/package.bbclass
@@ -490,16 +490,31 @@ def inject_minidebuginfo(file, dvar, dv, d):
 bb.debug(1, 'ELF file {} has no debuginfo, skipping minidebuginfo 
injection'.format(file))
 return
 
+# minidebuginfo does not make sense to apply to ELF objects other than
+# executables and shared libraries, skip applying the minidebuginfo
+# generation for objects like kernel modules.
+for line in subprocess.check_output([readelf, '-h', debugfile], 
universal_newlines=True).splitlines():
+if not line.strip().startswith("Type:"):
+continue
+elftype = line.split(":")[1].strip()
+if not any(elftype.startswith(i) for i in ["EXEC", "DYN"]):
+bb.debug(1, 'ELF file {} is not executable/shared, skipping 
minidebuginfo injection'.format(file))
+return
+break
+
 # Find non-allocated PROGBITS, NOTE, and NOBITS sections in the debuginfo.
 # We will exclude all of these from minidebuginfo to save space.
 remove_section_names = []
 for line in subprocess.check_output([readelf, '-W', '-S', debugfile], 
universal_newlines=True).splitlines():
-fields = line.split()
-if len(fields) < 8:
+# strip the leading "  [ 1]" section index to allow splitting on space
+if ']' not in line:
+continue
+fields = line[line.index(']') + 1:].split()
+if len(fields) < 7:
 continue
 name = fields[0]
 type = fields[1]
-flags = fields[7]
+flags = fields[6]
 # .debug_ sections will be removed by objcopy -S so no need to 
explicitly remove them
 if name.startswith('.debug_'):
 continue
---
2.37.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172617): 
https://lists.openembedded.org/g/openembedded-core/message/172617
Mute This Topic: https://lists.openembedded.org/mt/94752153/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/3] glibc-locale: Do not INHIBIT_DEFAULT_DEPS

[Nathan Rossi]

From: Nathan Rossi 

The glibc-locale recipe already partially depends on the base depends in
order to satisfy the do_package dependency on binutils. However since
commit d6ffd683bf6 NM has defaulted to gcc-nm, meaning do_package
depends on gcc (for minidebuginfo).

Whilst the do_package task could handle having the dependencies
explicitly defined (either in glibc-locale or in package.bbclass),
setting these would require some amount of conditional dependency
configuration (cross/crosssdk/etc.). Since both binutils and gcc are
already dependencies of virtual/libc (although compilerlibs is not),
having glibc-locale not inhibit the default depends simplifies the
handling of this situation for both glibc-locale and package.bbclass.

Signed-off-by: Nathan Rossi 
---
 meta/recipes-core/glibc/glibc-locale.inc | 11 +++
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
b/meta/recipes-core/glibc/glibc-locale.inc
index 7c14abfe99..7f70b3ca4f 100644
--- a/meta/recipes-core/glibc/glibc-locale.inc
+++ b/meta/recipes-core/glibc/glibc-locale.inc
@@ -5,14 +5,9 @@ SUMMARY = "Locale data from glibc"
 BPN = "glibc"
 LOCALEBASEPN = "${MLPREFIX}glibc"
 
-# glibc-collateral.inc inhibits all default deps, but do_package needs objcopy
-# ERROR: objcopy failed with exit code 127 (cmd was 'i586-webos-linux-objcopy' 
--only-keep-debug 'glibc-locale/2.17-r0/package/usr/lib/gconv/IBM1166.so' 
'glibc-locale/2.17-r0/package/usr/lib/gconv/.debug/IBM1166.so')
-# ERROR: Function failed: split_and_strip_files
-BINUTILSDEP = "virtual/${MLPREFIX}${TARGET_PREFIX}binutils:do_populate_sysroot"
-BINUTILSDEP:class-nativesdk = 
"virtual/${TARGET_PREFIX}binutils-crosssdk:do_populate_sysroot"
-do_package[depends] += "${BINUTILSDEP}"
-
-DEPENDS += "virtual/libc"
+# Do not inhibit default deps, do_package requires binutils/gcc for
+# objcopy/gcc-nm and glibc-locale depends on virtual/libc directly.
+INHIBIT_DEFAULT_DEPS = ""
 
 # Binary locales are generated at build time if ENABLE_BINARY_LOCALE_GENERATION
 # is set. The idea is to avoid running localedef on the target (at first boot)
---
2.37.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172616): 
https://lists.openembedded.org/g/openembedded-core/message/172616
Mute This Topic: https://lists.openembedded.org/mt/94752152/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/3] oeqa/selftest/minidebuginfo: Create selftest for minidebuginfo

[Nathan Rossi]

From: Nathan Rossi 

Add a new selftest to validate minidebuginfo support. This selftest
builds a complete target image with PACKAGE_MINIDEBUGINFO enabled. ELFs
included in the image are expected to have minidebuginfo included in the
resulting executables and shared libraries, the self test validates this
by unpacking the image and checking for the associated ".gnu_debugdata"
section on busybox and libc ELFs.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/minidebuginfo.py | 43 +++
 1 file changed, 43 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/minidebuginfo.py

diff --git a/meta/lib/oeqa/selftest/cases/minidebuginfo.py 
b/meta/lib/oeqa/selftest/cases/minidebuginfo.py
new file mode 100644
index 00..7947c3803c
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/minidebuginfo.py
@@ -0,0 +1,43 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+import os
+import subprocess
+import tempfile
+import shutil
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, runCmd
+
+
+class Minidebuginfo(OESelftestTestCase):
+def test_minidebuginfo(self):
+target_sys = get_bb_var("TARGET_SYS")
+binutils = "binutils-cross-{}".format(get_bb_var("TARGET_ARCH"))
+
+self.write_config("""
+PACKAGE_MINIDEBUGINFO = "1"
+IMAGE_FSTYPES = "tar.bz2"
+""")
+bitbake("core-image-minimal 
{}:do_addto_recipe_sysroot".format(binutils))
+
+deploy_dir = get_bb_var("DEPLOY_DIR_IMAGE")
+native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", binutils)
+readelf = get_bb_var("READELF", "core-image-minimal")
+
+# confirm that executables and shared libraries contain an ELF section
+# ".gnu_debugdata" which stores minidebuginfo.
+with tempfile.TemporaryDirectory(prefix = "unpackfs-") as unpackedfs:
+filename = os.path.join(deploy_dir, 
"core-image-minimal-{}.tar.bz2".format(self.td["MACHINE"]))
+shutil.unpack_archive(filename, unpackedfs)
+
+r = runCmd([readelf, "-W", "-S", os.path.join(unpackedfs, "bin", 
"busybox")],
+native_sysroot = native_sysroot, target_sys = target_sys)
+self.assertIn(".gnu_debugdata", r.output)
+
+r = runCmd([readelf, "-W", "-S", os.path.join(unpackedfs, "lib", 
"libc.so.6")],
+native_sysroot = native_sysroot, target_sys = target_sys)
+self.assertIn(".gnu_debugdata", r.output)
+
---
2.37.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172615): 
https://lists.openembedded.org/g/openembedded-core/message/172615
Mute This Topic: https://lists.openembedded.org/mt/94752151/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 2/2] glibc: ptest: Add support for running glibc test suite with ptest

[Nathan Rossi]

OS
> > > > > T_USER=root SSH_HOST_PORT=
> > > > > test-wrapper=/home/lukma/work/yocto/y2038/build/tmp/work/armv7at2hf-neon-poky-linux-gnueabi/glibc-testsuite/2.34-r0/check-test-wrapper
> > > > > user check
> > > > >
> > > > > Or you log to the board via SSH (the 'ssh' mode).
> > > > >
> > > > > >
> > > > > > Has this patchset been run with all the tests in glibc or
> > > > > > just a subset?
> > > > >
> > > > > This patch set allows running all tests [*], or (preferably in
> > > > > my use case) only a time related ones. The latter is to
> > > > > validate Y2038 code, which testing has two issues:
> > > > >
> > > > > 1. It cannot be tested with QEMU user mode [1]. Instead "linux
> > > > > timespaces" were suggested, but it lacks support for
> > > > > CLOCK_REALTIME manipulation for those tests [2].
> >
> > So just a note, the qemu user mode testing is not a complete solution.
> > There are a number of problems with it, but in general it is still
> > quite useful for validating general functionally, especially when
> > implementing a new platform.
>
> +1
>
> > A big one is due to the ability to run
> > tests in parallel with user mode, it easily reduces test run times by
> > an order of magnitude (compared to qemu system, which for some
> > platforms is faster than real hardware).
>
> +1
>
> >
> > The reason why qemu user mode is the default behaviour is because
> > running "bitbake glibc-testsuite -c check" will work without any
> > external dependencies (the remote machine, NFS/sshfs, etc.).
>
> +1
>
> >
> > > > >
> > > > > 2. Testing with SSH is possible, but is not as robust as
> > > > > expected (i.e. imposes SSHD on the target board, and ETH
> > > > > connection).
> > > > > >
> > > > > > I'm always very nervous about having two ways to do similar
> > > > > > things.
> > > > >
> > > > > Technically, I just reuse the glibc-testsuite_2.34.bb to the
> > > > > point where tests are _only_ built. Then I pack those binaries
> > > > > and install on the target's rootfs to be run with ptest
> > > > > framework.
> >
> > So it appears the make variable "run-built-tests" excludes the
> > building of some tests.
>
> As fair as I remember, the 'run-built-tests=no' switch tells the glibc
> build system to not execute built tests.
>
> > So I am not sure if the complete test suite
> > can be built in this way.
>
> I'm pretty sure that all tests required for 'check' are build.

If you look at some of the Makefiles in glibc, it conditionally
includes/excludes some tests from being built based on the value of
run-built-tests. For example:

https://sourceware.org/git/?p=glibc.git;a=blob;f=localedata/Makefile;h=f585e0dd4115a99493e843aeed464090da19665f;hb=HEAD#l141

>
> > Though I can see some value in having the
> > glibc-testsuite recipe being able to only build the tests for
> > development purposes (e.g. TOOLCHAIN_TEST_TARGET = "build"?).
> >
> > When I had implemented the testsuite recipe the glibc testsuite relied
> > quite heavily on the environment it was built and run in, in
> > particular around configuration. That is probably still an issue.
>
> Indeed, some (a few?) tests rely on the build environment in which they
> were build. I do personally, regard this as a some kind of a bug. I do
> believe, that when you build test for a particular version of glibc
> (2.34 in this case), one shall expect that it can be run only by
> linking the proper library.
>
> This is the case for "time" related tests. On my test system with this
> patch applied, all of them are executed with no errors. Those are self
> contained.
>
> >
> > It would be interesting to know how much of the test suite can be
> > run/passes and how that compares to the results of existing
> > system/user emulation with cross execution using the glibc makefile.
>
> I can provide exact numbers when I finish porting more Y2038 related
> patches to newest glibc -master.

Also something to keep in mind is that sometimes the pass/fail of the
test binary is not what is being validated by the test. For example
this test uses mtrace, and compares the results of the output mtrace
file, not the test itself.

https://sourceware.org/git/?p=glibc.git;a=blob;f=localedata/Makefile;h=f585e0dd4115a99493e843aeed464090da19665f;hb=HEAD#l462

Regards,
Nathan

>
> >
> > > > >
> > > > > Such approach has several advantages:
> > > > >
> > > > > - No need for SSH (which can hang, as you need sshfs in user
> > > > > space for the setup)
> >
> > I had not tested with sshfs that likely has quite low performance, the
> > OEQA tests rely on NFS for QEMU system emulation and that is what I
> > had been using for hardware testing.
>
> I had some issues with NFS - either I couldn't place some OE/Yocto
> build directory in it or glibc's build directory was not working with
> it correctly.
>
> That was why I had to use the sshfs instead, which is _really_ slow and
> often breaks.
>
> > However it would be interesting
> > to see if virtiofs could improve the performance for testing with QEMU
> > system emulation.
> >
> > > > >
> > > > > - Run on the real HW (and environment) with the same version of
> > > > > glibc on the real target.
> > > > >
> > > > > - Sources and debugging symbols available on the target, so
> > > > > easier GDB debugging.
> > > > >
> > > > > - Using ptest framework to execute tests
> > > > >
> > > > > - Time related tests are "self contained", so could be executed
> > > > > without built data.
> > > > >
> > > > > At least for Y2038 validation such approach seems more
> > > > > appealing.
> > > >
> > > > Do you have any more feedback regarding this patch?
> > >
> > > We're at feature freeze and this wasn't a planned change so it is
> > > too late for this release cycle, I'm struggling with the things
> > > that were planned.
> > >
> > > I need to spend some time looking at what the other code is doing
> > > and how we're using it on the autobuilder and whether we want to
> > > change to use the ptest approach or not.
> > >
> > > I added Nathan Rossi to cc since he worked on the other code for
> > > this and may have more insight.
> >
> > Thanks for cc'ing, I somehow managed to miss the thread.
> >
> > Regards,
> > Nathan
>
>
> Best regards,
>
> Lukasz Majewski
>
> --
>
> DENX Software Engineering GmbH,  Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lu...@denx.de

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#155402): 
https://lists.openembedded.org/g/openembedded-core/message/155402
Mute This Topic: https://lists.openembedded.org/mt/85087396/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 2/2] glibc: ptest: Add support for running glibc test suite with ptest

[Nathan Rossi]

sets of tests in different packages)
> > > > > > > >
> > > > > > > > That’s seems ok I think the names are quite confusing now
> > > > > > > > not that it was not so much better before but now we have
> > > > > > > > glibc-tests and glibc-testsuites which do same things but in
> > > > > > > > very different way maybe glibc-testsuite should be renamed
> > > > > > > > to something like glibc-tests-crosd or some such
> > > > > > >
> > > > > > > I think that glibc-testsuite_2.34.bb shall be renamed to
> > > > > > > glibc-tests-qemu_2.34.bb as it is more descriptive.
> > > > > >
> > > > > > is it only runnable in qemu ? I thought we could use a read
> > > > > > board as well with IP address
> > > > >
> > > > > It looks like by default the QEMU is used in this recipe.
> > > > >
> > > > > This recipe provides custom check-test-wrapper, which can log
> > > > > into remote board via ssh (when TOOLCHAIN_TEST_TARGET = "ssh").
> > > > >
> > > > > This looks a bit awkward for me, as the goal would be to run built
> > > > > tests on target (exact) HW without the need of SSH.
> > > > >
> > > > > It is also easier to debug the code with the real HW.
> > > >
> > > > I'm a bit worried we're going around in circles with this. We did
> > > > have a different form of tests a while back but I was told those
> > > > didn't work well and they were replaced with the current setup which
> > > > worked with the driver in the glibc source code.
> > >
> > > I've looked into the glibc-testing.inc from SHA1:
> > > 0a42ae7f7ca8fcebe4630b701e50e2352f9b7c3b
> > >
> > > There glibc tests were built and copied on the target via NFS.
> > >
> > > > This was done around
> > > > the time we added support for the other toolchain test suites
> > > > (binutils+gcc).
> > >
> > > I'm not familiar with the "binutils+gcc" issue, which was solved in
> > > the past. Could you share any reference?
> > >
> > > > We ended up with options to use full system qemu or
> > > > user mode qemu emulation depending on the speed of the target (e.g.
> > > > with kvm or not).
> > >
> > > From what I see in the recipe - you either run user mode QEMU by
> > > default (at least when I do run it):
> > >
> > > NOTE: make PARALLELMFLAGS=-j 8 SHELL=/bin/bash -i
> > > QEMU_SYSROOT=/home/lukma/work/yocto/y2038/build/tmp/work/armv7at2hf-neon-poky-linux-gnueabi/glibc-testsuite/2.34-r0/recipe-sysroot
> > > QEMU_OPTIONS=qemu-arm -r 5.1.0  SSH_HOST=localhost SSH_HOS T_USER=root
> > > SSH_HOST_PORT=
> > > test-wrapper=/home/lukma/work/yocto/y2038/build/tmp/work/armv7at2hf-neon-poky-linux-gnueabi/glibc-testsuite/2.34-r0/check-test-wrapper
> > > user check
> > >
> > > Or you log to the board via SSH (the 'ssh' mode).
> > >
> > > >
> > > > Has this patchset been run with all the tests in glibc or just a
> > > > subset?
> > >
> > > This patch set allows running all tests [*], or (preferably in my use
> > > case) only a time related ones. The latter is to validate Y2038 code,
> > > which testing has two issues:
> > >
> > > 1. It cannot be tested with QEMU user mode [1]. Instead "linux
> > > timespaces" were suggested, but it lacks support for CLOCK_REALTIME
> > > manipulation for those tests [2].

So just a note, the qemu user mode testing is not a complete solution.
There are a number of problems with it, but in general it is still
quite useful for validating general functionally, especially when
implementing a new platform. A big one is due to the ability to run
tests in parallel with user mode, it easily reduces test run times by
an order of magnitude (compared to qemu system, which for some
platforms is faster than real hardware).

The reason why qemu user mode is the default behaviour is because
running "bitbake glibc-testsuite -c check" will work without any
external dependencies (the remote machine, NFS/sshfs, etc.).

> > >
> > > 2. Testing with SSH is possible, but is not as robust as expected
> > > (i.e. imposes SSHD on the target board, and ETH connection).
> > >
> > > >
> > > > I'm always very 

[OE-core] [PATCH] gcc: Backport patch to resolve i*86 tune configuration overrides

[Nathan Rossi]

Upstream has merged a change to handle detection of target overrides so
that they are only applied as required. Backport this patch from
upstream and remove the OE specific patch.

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/gcc/gcc-10.2.inc |   2 +-
 .../gcc/0001-lib-Remove-i-86-march-overrides.patch | 113 ---
 ...omic-libgomp-libitc-Fix-bootstrap-PR70454.patch | 208 +
 3 files changed, 209 insertions(+), 114 deletions(-)
 delete mode 100644 
meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch
 create mode 100644 
meta/recipes-devtools/gcc/gcc/0001-libatomic-libgomp-libitc-Fix-bootstrap-PR70454.patch

diff --git a/meta/recipes-devtools/gcc/gcc-10.2.inc 
b/meta/recipes-devtools/gcc/gcc-10.2.inc
index 1e69c91311..c0cd8b31d5 100644
--- a/meta/recipes-devtools/gcc/gcc-10.2.inc
+++ b/meta/recipes-devtools/gcc/gcc-10.2.inc
@@ -69,7 +69,7 @@ SRC_URI = "\

file://0002-aarch64-Introduce-SLS-mitigation-for-RET-and-BR-inst.patch \
file://0003-aarch64-Mitigate-SLS-for-BLR-instruction.patch \
file://0001-aarch64-Fix-up-__aarch64_cas16_acq_rel-fallback.patch \
-   file://0001-lib-Remove-i-86-march-overrides.patch \
+   file://0001-libatomic-libgomp-libitc-Fix-bootstrap-PR70454.patch \
 "
 SRC_URI[sha256sum] = 
"b8dd4368bb9c7f0b98188317ee0254dd8cc99d1e3a18d0ff146c855fe16c1d8c"
 
diff --git 
a/meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch 
b/meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch
deleted file mode 100644
index 3f04e121ba..00
--- a/meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch
+++ /dev/null
@@ -1,113 +0,0 @@
-From 3776789cde95916f95deef48acf9532b0ebf91ae Mon Sep 17 00:00:00 2001
-From: Nathan Rossi 
-Date: Tue, 15 Dec 2020 11:43:36 +1000
-Subject: [PATCH] lib*: Remove i*86 march overrides
-
-OE does not pass the '--with-arch' option to gccs configure, as such
-some gcc-runtime libraries try to override the value of '-march' and
-'-mtune' which OE already provides. This causes conflicts with other
-i*86 instruction/architecture flags (e.g. -msse* and -mfpmath=*).
-Additionally this caused the following libraries to be built with less
-optimized tune configurations.
-
-Upstream suggests that this should be detecting or otherwise checking if
-the target supports the desired functionality before overriding.
-
-https://gcc.gnu.org/legacy-ml/gcc-patches/2016-04/msg01344.html
-
-libatomic also overrides the '-march' specifically for IFUNC objects.
-OE already supplies the correct march flag, so remove setting.
-
-Upstream-Status: Inappropriate [OE Specific]
-Signed-off-by: Nathan Rossi 

- libatomic/Makefile.am   | 1 -
- libatomic/Makefile.in   | 1 -
- libatomic/configure.tgt | 9 -
- libgomp/configure.tgt   | 8 
- libitm/configure.tgt| 9 -
- 5 files changed, 28 deletions(-)
-
-diff --git a/libatomic/Makefile.am b/libatomic/Makefile.am
-index ac1ca64587..5aa16e0699 100644
 a/libatomic/Makefile.am
-+++ b/libatomic/Makefile.am
-@@ -133,7 +133,6 @@ libatomic_la_LIBADD += $(foreach s,$(SIZES),$(addsuffix 
_$(s)_1_.lo,$(SIZEOBJS))
- libatomic_la_LIBADD += $(addsuffix _8_2_.lo,$(SIZEOBJS))
- endif
- if ARCH_I386
--IFUNC_OPTIONS  = -march=i586
- libatomic_la_LIBADD += $(addsuffix _8_1_.lo,$(SIZEOBJS))
- endif
- if ARCH_X86_64
-diff --git a/libatomic/Makefile.in b/libatomic/Makefile.in
-index 97df2d7ff0..82c511d420 100644
 a/libatomic/Makefile.in
-+++ b/libatomic/Makefile.in
-@@ -432,7 +432,6 @@ libatomic_la_LIBADD = $(foreach s,$(SIZES),$(addsuffix \
-   _$(s)_.lo,$(SIZEOBJS))) $(am__append_1) $(am__append_2) \
-   $(am__append_3) $(am__append_4)
- @ARCH_ARM_LINUX_TRUE@@HAVE_IFUNC_TRUE@IFUNC_OPTIONS = -march=armv7-a+fp 
-DHAVE_KERNEL64
--@ARCH_I386_TRUE@@HAVE_IFUNC_TRUE@IFUNC_OPTIONS = -march=i586
- @ARCH_X86_64_TRUE@@HAVE_IFUNC_TRUE@IFUNC_OPTIONS = -mcx16
- libatomic_convenience_la_SOURCES = $(libatomic_la_SOURCES)
- libatomic_convenience_la_LIBADD = $(libatomic_la_LIBADD)
-diff --git a/libatomic/configure.tgt b/libatomic/configure.tgt
-index 5dd0926d20..3cc41773c3 100644
 a/libatomic/configure.tgt
-+++ b/libatomic/configure.tgt
-@@ -82,15 +82,6 @@ case "${target_cpu}" in
-   ;;
- 
-   i[3456]86)
--  case " ${CC} ${CFLAGS} " in
--*" -m64 "*|*" -mx32 "*)
--  ;;
--*)
--  if test -z "$with_arch"; then
--XCFLAGS="${XCFLAGS} -march=i486 -mtune=${target_cpu}"
--XCFLAGS="${XCFLAGS} -fomit-frame-pointer"
--  fi
--  esac
-   ARCH=x86
-   # ??? Detect when -march=i686 is already enabled.
-   try_ifunc=yes
-diff --git a/libgomp/configure.tgt b/libgomp/configure.tgt
-index 4790a31e39..46c4c958e6 100644
 a/libgomp/configure.tgt
-+++ b/libgomp/conf

Re: [OE-core] [PATCH] gcc: Add patch to resolve i*86 tune configuration overrides

[Nathan Rossi]

On Sat, 16 Jan 2021 at 03:40, Khem Raj  wrote:
>
> Nathan
>
> Can you check if your patch is still needed after
>
> https://github.com/kraj/gcc/commit/0411ae7f08e0f5a8b02ff313d26d27a0f6d1bb34

Hi Khem,

I tested this change and it resolves the configuration issue. Did you
want the above change backported to replace the oe specific patch?
Since that oe specific patch has already been backported to a number
of releases.

Thanks,
Nathan

>
> On Tue, Dec 15, 2020 at 12:35 AM Nathan Rossi  wrote:
> >
> > When compiling gcc-runtime for i686 libatomic, libgomp and libitm are
> > compiled with additional '-march' and '-mtune' overrides. This typically
> > does not cause build errors (e.g. core2-32 tune), and results in less
> > optimally tuned outputs. However with specific TUNE_CCARGS (e.g. has
> > '-mfpmath=sse' but not '-msse*'), the build errors with conflicting
> > '-mfpmath' options.
> >
> > This is caused by gcc-runtime specific configure scripts and Makefile
> > options that override by adding additional '-march'/'-mtune' args. OE
> > builds gcc-runtime differently to other build systems, and provides
> > target specific arch/tune options directly. As such, this change adds a
> > patch for gcc to remove the specific cases where the args are added.
> >
> > Signed-off-by: Nathan Rossi 
> > ---
> >  meta/recipes-devtools/gcc/gcc-10.2.inc |   1 +
> >  .../gcc/0001-lib-Remove-i-86-march-overrides.patch | 113 
> > +
> >  2 files changed, 114 insertions(+)
> >  create mode 100644 
> > meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch
> >
> > diff --git a/meta/recipes-devtools/gcc/gcc-10.2.inc 
> > b/meta/recipes-devtools/gcc/gcc-10.2.inc
> > index e88f2ade6d..1e69c91311 100644
> > --- a/meta/recipes-devtools/gcc/gcc-10.2.inc
> > +++ b/meta/recipes-devtools/gcc/gcc-10.2.inc
> > @@ -69,6 +69,7 @@ SRC_URI = "\
> > 
> > file://0002-aarch64-Introduce-SLS-mitigation-for-RET-and-BR-inst.patch \
> > file://0003-aarch64-Mitigate-SLS-for-BLR-instruction.patch \
> > 
> > file://0001-aarch64-Fix-up-__aarch64_cas16_acq_rel-fallback.patch \
> > +   file://0001-lib-Remove-i-86-march-overrides.patch \
> >  "
> >  SRC_URI[sha256sum] = 
> > "b8dd4368bb9c7f0b98188317ee0254dd8cc99d1e3a18d0ff146c855fe16c1d8c"
> >
> > diff --git 
> > a/meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch 
> > b/meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch
> > new file mode 100644
> > index 00..3f04e121ba
> > --- /dev/null
> > +++ 
> > b/meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch
> > @@ -0,0 +1,113 @@
> > +From 3776789cde95916f95deef48acf9532b0ebf91ae Mon Sep 17 00:00:00 2001
> > +From: Nathan Rossi 
> > +Date: Tue, 15 Dec 2020 11:43:36 +1000
> > +Subject: [PATCH] lib*: Remove i*86 march overrides
> > +
> > +OE does not pass the '--with-arch' option to gccs configure, as such
> > +some gcc-runtime libraries try to override the value of '-march' and
> > +'-mtune' which OE already provides. This causes conflicts with other
> > +i*86 instruction/architecture flags (e.g. -msse* and -mfpmath=*).
> > +Additionally this caused the following libraries to be built with less
> > +optimized tune configurations.
> > +
> > +Upstream suggests that this should be detecting or otherwise checking if
> > +the target supports the desired functionality before overriding.
> > +
> > +https://gcc.gnu.org/legacy-ml/gcc-patches/2016-04/msg01344.html
> > +
> > +libatomic also overrides the '-march' specifically for IFUNC objects.
> > +OE already supplies the correct march flag, so remove setting.
> > +
> > +Upstream-Status: Inappropriate [OE Specific]
> > +Signed-off-by: Nathan Rossi 
> > +---
> > + libatomic/Makefile.am   | 1 -
> > + libatomic/Makefile.in   | 1 -
> > + libatomic/configure.tgt | 9 -
> > + libgomp/configure.tgt   | 8 
> > + libitm/configure.tgt| 9 -
> > + 5 files changed, 28 deletions(-)
> > +
> > +diff --git a/libatomic/Makefile.am b/libatomic/Makefile.am
> > +index ac1ca64587..5aa16e0699 100644
> > +--- a/libatomic/Makefile.am
> >  b/libatomic/Makefile.am
> > +@@ -133,7 +133,6 @@ libatomic_la_LIBADD += $(foreach 
> > s,$(SIZES),$(addsuffix _$(s)_1_.lo,$(SIZEOBJS))
> > + libatomi

[OE-core] [PATCH] coreutils: enable xattrs by default for nativesdk

[Nathan Rossi]

When using coreutils nativesdk (e.g. with buildtools-tarball) for
running oe-core builds, a number of recipes/classes/etc. expect xattr
support. This requirement is also expressed by the existing
PACKAGECONFIG_class-native default including xattrs.

Signed-off-by: Nathan Rossi 
---
 meta/recipes-core/coreutils/coreutils_8.32.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-core/coreutils/coreutils_8.32.bb 
b/meta/recipes-core/coreutils/coreutils_8.32.bb
index 4eb357e310..320f93bdc2 100644
--- a/meta/recipes-core/coreutils/coreutils_8.32.bb
+++ b/meta/recipes-core/coreutils/coreutils_8.32.bb
@@ -39,6 +39,9 @@ PACKAGECONFIG_class-target ??= "\
 # The lib/oe/path.py requires xattr
 PACKAGECONFIG_class-native ??= "xattr"
 
+# oe-core builds need xattr support
+PACKAGECONFIG_class-nativesdk ??= "xattr"
+
 # with, without, depends, rdepends
 #
 PACKAGECONFIG[acl] = "--enable-acl,--disable-acl,acl,"
---
2.29.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145643): 
https://lists.openembedded.org/g/openembedded-core/message/145643
Mute This Topic: https://lists.openembedded.org/mt/78972328/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] grep: add nativesdk to BBCLASSEXTEND

[Nathan Rossi]

Signed-off-by: Nathan Rossi 
---
 meta/recipes-extended/grep/grep_3.6.bb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-extended/grep/grep_3.6.bb 
b/meta/recipes-extended/grep/grep_3.6.bb
index edf9b29c8f..cb009b9132 100644
--- a/meta/recipes-extended/grep/grep_3.6.bb
+++ b/meta/recipes-extended/grep/grep_3.6.bb
@@ -41,3 +41,5 @@ ALTERNATIVE_${PN} = "grep egrep fgrep"
 ALTERNATIVE_LINK_NAME[grep] = "${base_bindir}/grep"
 ALTERNATIVE_LINK_NAME[egrep] = "${base_bindir}/egrep"
 ALTERNATIVE_LINK_NAME[fgrep] = "${base_bindir}/fgrep"
+
+BBCLASSEXTEND = "nativesdk"
---
2.29.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145642): 
https://lists.openembedded.org/g/openembedded-core/message/145642
Mute This Topic: https://lists.openembedded.org/mt/78972323/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] sed: add nativesdk to BBCLASSEXTEND

[Nathan Rossi]

Signed-off-by: Nathan Rossi 
---
 meta/recipes-extended/sed/sed_4.8.bb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-extended/sed/sed_4.8.bb 
b/meta/recipes-extended/sed/sed_4.8.bb
index 39e3a61df5..048db47e14 100644
--- a/meta/recipes-extended/sed/sed_4.8.bb
+++ b/meta/recipes-extended/sed/sed_4.8.bb
@@ -63,3 +63,5 @@ do_install_ptest() {
 }
 
 RPROVIDES_${PN} += "${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge', 
'/bin/sed', '', d)}"
+
+BBCLASSEXTEND = "nativesdk"
---
2.29.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145641): 
https://lists.openembedded.org/g/openembedded-core/message/145641
Mute This Topic: https://lists.openembedded.org/mt/78972311/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] which: add nativesdk to BBCLASSEXTEND

[Nathan Rossi]

Signed-off-by: Nathan Rossi 
---
 meta/recipes-extended/which/which_2.21.bb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-extended/which/which_2.21.bb 
b/meta/recipes-extended/which/which_2.21.bb
index fc9185061b..1da69c548e 100644
--- a/meta/recipes-extended/which/which_2.21.bb
+++ b/meta/recipes-extended/which/which_2.21.bb
@@ -33,3 +33,5 @@ ALTERNATIVE_PRIORITY = "100"
 
 ALTERNATIVE_${PN}-doc = "which.1"
 ALTERNATIVE_LINK_NAME[which.1] = "${mandir}/man1/which.1"
+
+BBCLASSEXTEND = "nativesdk"
---
2.29.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145640): 
https://lists.openembedded.org/g/openembedded-core/message/145640
Mute This Topic: https://lists.openembedded.org/mt/78972307/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] gcc: Add patch to resolve i*86 tune configuration overrides

[Nathan Rossi]

When compiling gcc-runtime for i686 libatomic, libgomp and libitm are
compiled with additional '-march' and '-mtune' overrides. This typically
does not cause build errors (e.g. core2-32 tune), and results in less
optimally tuned outputs. However with specific TUNE_CCARGS (e.g. has
'-mfpmath=sse' but not '-msse*'), the build errors with conflicting
'-mfpmath' options.

This is caused by gcc-runtime specific configure scripts and Makefile
options that override by adding additional '-march'/'-mtune' args. OE
builds gcc-runtime differently to other build systems, and provides
target specific arch/tune options directly. As such, this change adds a
patch for gcc to remove the specific cases where the args are added.

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/gcc/gcc-10.2.inc |   1 +
 .../gcc/0001-lib-Remove-i-86-march-overrides.patch | 113 +
 2 files changed, 114 insertions(+)
 create mode 100644 
meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch

diff --git a/meta/recipes-devtools/gcc/gcc-10.2.inc 
b/meta/recipes-devtools/gcc/gcc-10.2.inc
index e88f2ade6d..1e69c91311 100644
--- a/meta/recipes-devtools/gcc/gcc-10.2.inc
+++ b/meta/recipes-devtools/gcc/gcc-10.2.inc
@@ -69,6 +69,7 @@ SRC_URI = "\

file://0002-aarch64-Introduce-SLS-mitigation-for-RET-and-BR-inst.patch \
file://0003-aarch64-Mitigate-SLS-for-BLR-instruction.patch \
file://0001-aarch64-Fix-up-__aarch64_cas16_acq_rel-fallback.patch \
+   file://0001-lib-Remove-i-86-march-overrides.patch \
 "
 SRC_URI[sha256sum] = 
"b8dd4368bb9c7f0b98188317ee0254dd8cc99d1e3a18d0ff146c855fe16c1d8c"
 
diff --git 
a/meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch 
b/meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch
new file mode 100644
index 00..3f04e121ba
--- /dev/null
+++ b/meta/recipes-devtools/gcc/gcc/0001-lib-Remove-i-86-march-overrides.patch
@@ -0,0 +1,113 @@
+From 3776789cde95916f95deef48acf9532b0ebf91ae Mon Sep 17 00:00:00 2001
+From: Nathan Rossi 
+Date: Tue, 15 Dec 2020 11:43:36 +1000
+Subject: [PATCH] lib*: Remove i*86 march overrides
+
+OE does not pass the '--with-arch' option to gccs configure, as such
+some gcc-runtime libraries try to override the value of '-march' and
+'-mtune' which OE already provides. This causes conflicts with other
+i*86 instruction/architecture flags (e.g. -msse* and -mfpmath=*).
+Additionally this caused the following libraries to be built with less
+optimized tune configurations.
+
+Upstream suggests that this should be detecting or otherwise checking if
+the target supports the desired functionality before overriding.
+
+https://gcc.gnu.org/legacy-ml/gcc-patches/2016-04/msg01344.html
+
+libatomic also overrides the '-march' specifically for IFUNC objects.
+OE already supplies the correct march flag, so remove setting.
+
+Upstream-Status: Inappropriate [OE Specific]
+Signed-off-by: Nathan Rossi 
+---
+ libatomic/Makefile.am   | 1 -
+ libatomic/Makefile.in   | 1 -
+ libatomic/configure.tgt | 9 -
+ libgomp/configure.tgt   | 8 
+ libitm/configure.tgt| 9 -
+ 5 files changed, 28 deletions(-)
+
+diff --git a/libatomic/Makefile.am b/libatomic/Makefile.am
+index ac1ca64587..5aa16e0699 100644
+--- a/libatomic/Makefile.am
 b/libatomic/Makefile.am
+@@ -133,7 +133,6 @@ libatomic_la_LIBADD += $(foreach s,$(SIZES),$(addsuffix 
_$(s)_1_.lo,$(SIZEOBJS))
+ libatomic_la_LIBADD += $(addsuffix _8_2_.lo,$(SIZEOBJS))
+ endif
+ if ARCH_I386
+-IFUNC_OPTIONS  = -march=i586
+ libatomic_la_LIBADD += $(addsuffix _8_1_.lo,$(SIZEOBJS))
+ endif
+ if ARCH_X86_64
+diff --git a/libatomic/Makefile.in b/libatomic/Makefile.in
+index 97df2d7ff0..82c511d420 100644
+--- a/libatomic/Makefile.in
 b/libatomic/Makefile.in
+@@ -432,7 +432,6 @@ libatomic_la_LIBADD = $(foreach s,$(SIZES),$(addsuffix \
+   _$(s)_.lo,$(SIZEOBJS))) $(am__append_1) $(am__append_2) \
+   $(am__append_3) $(am__append_4)
+ @ARCH_ARM_LINUX_TRUE@@HAVE_IFUNC_TRUE@IFUNC_OPTIONS = -march=armv7-a+fp 
-DHAVE_KERNEL64
+-@ARCH_I386_TRUE@@HAVE_IFUNC_TRUE@IFUNC_OPTIONS = -march=i586
+ @ARCH_X86_64_TRUE@@HAVE_IFUNC_TRUE@IFUNC_OPTIONS = -mcx16
+ libatomic_convenience_la_SOURCES = $(libatomic_la_SOURCES)
+ libatomic_convenience_la_LIBADD = $(libatomic_la_LIBADD)
+diff --git a/libatomic/configure.tgt b/libatomic/configure.tgt
+index 5dd0926d20..3cc41773c3 100644
+--- a/libatomic/configure.tgt
 b/libatomic/configure.tgt
+@@ -82,15 +82,6 @@ case "${target_cpu}" in
+   ;;
+ 
+   i[3456]86)
+-  case " ${CC} ${CFLAGS} " in
+-*" -m64 "*|*" -mx32 "*)
+-  ;;
+-*)
+-  if test -z "$with_arch"; then
+-XCFLAGS="${XCFLAGS} -march=i486 -mtune=${target_cpu}"
+-  

[OE-core] [PATCH] ncurses: Prevent LDFLAGS being emitted in .pc files

[Nathan Rossi]

By default ncurses includes the values of LDFLAGS in its output
pkgconfig .pc files. This causes issues because OE includes options that
are specific to either the build host, or build configuration. These
options are not expected to be embedded in the pkgconfig output that is
installed.

Specifically this change resolves issues with uninative, where uninative
includes '-Wl,--dynamic-linker=' in LDFLAGS in order to force the
building and execution of native binaries against the dynamic linker
provided by uninative. This path is specific to TMPDIR at the time of
build, such that the installed files (and the associated sstate) have
this path. This prevents the sstate from being portable across build
directories/hosts.

Signed-off-by: Nathan Rossi 
---
 ...ig.in-Do-not-include-LDFLAGS-in-generated.patch | 29 ++
 meta/recipes-core/ncurses/ncurses_6.2.bb   |  1 +
 2 files changed, 30 insertions(+)
 create mode 100644 
meta/recipes-core/ncurses/files/0003-gen-pkgconfig.in-Do-not-include-LDFLAGS-in-generated.patch

diff --git 
a/meta/recipes-core/ncurses/files/0003-gen-pkgconfig.in-Do-not-include-LDFLAGS-in-generated.patch
 
b/meta/recipes-core/ncurses/files/0003-gen-pkgconfig.in-Do-not-include-LDFLAGS-in-generated.patch
new file mode 100644
index 00..1eb17767a0
--- /dev/null
+++ 
b/meta/recipes-core/ncurses/files/0003-gen-pkgconfig.in-Do-not-include-LDFLAGS-in-generated.patch
@@ -0,0 +1,29 @@
+From 3b3e87934bb6d8511261d7c3d6e39b4f71849272 Mon Sep 17 00:00:00 2001
+From: Nathan Rossi 
+Date: Mon, 14 Dec 2020 13:39:02 +1000
+Subject: [PATCH] gen-pkgconfig.in: Do not include LDFLAGS in generated pc
+ files
+
+Including the LDFLAGS in the pkgconfig output is problematic as OE
+includes build host specific paths and options (e.g. uninative and
+'-Wl,--dynamic-linker=').
+
+Upstream-Status: Inappropriate [OE Specific]
+Signed-off-by: Nathan Rossi 
+---
+ misc/gen-pkgconfig.in | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/misc/gen-pkgconfig.in b/misc/gen-pkgconfig.in
+index 8f00b824b9..009d215663 100644
+--- a/misc/gen-pkgconfig.in
 b/misc/gen-pkgconfig.in
+@@ -80,7 +80,7 @@ if [ "$includedir" != "/usr/include" ]; then
+ fi
+ 
+ lib_flags=
+-for opt in -L$libdir @LDFLAGS@ @EXTRA_LDFLAGS@ @LIBS@
++for opt in -L$libdir @LIBS@
+ do
+   case $opt in
+   -l*) # LIBS is handled specially below
diff --git a/meta/recipes-core/ncurses/ncurses_6.2.bb 
b/meta/recipes-core/ncurses/ncurses_6.2.bb
index f3c84c2877..e7d7396a20 100644
--- a/meta/recipes-core/ncurses/ncurses_6.2.bb
+++ b/meta/recipes-core/ncurses/ncurses_6.2.bb
@@ -2,6 +2,7 @@ require ncurses.inc
 
 SRC_URI += "file://0001-tic-hang.patch \
file://0002-configure-reproducible.patch \
+   
file://0003-gen-pkgconfig.in-Do-not-include-LDFLAGS-in-generated.patch \
"
 # commit id corresponds to the revision in package version
 SRCREV = "a669013cd5e9d6434e5301348ea51baf306c93c4"
---
2.29.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145638): 
https://lists.openembedded.org/g/openembedded-core/message/145638
Mute This Topic: https://lists.openembedded.org/mt/78972256/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] qemu.inc: Add seccomp PACKAGECONFIG option

[Nathan Rossi]

Add the seccomp PACKAGECONFIG option to allow building seccomp features
in QEMU. The libseccomp library is available in additional layers (e.g.
meta-security).

Additionally this serves as a way to disable seccomp by default to avoid
the configure of QEMU automatically finding it (via pkg-config) on the
build host when building qemu-system-native and auto enabling the
feature.

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/qemu/qemu.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-devtools/qemu/qemu.inc 
b/meta/recipes-devtools/qemu/qemu.inc
index 274c855d35..b5827dc2ad 100644
--- a/meta/recipes-devtools/qemu/qemu.inc
+++ b/meta/recipes-devtools/qemu/qemu.inc
@@ -188,6 +188,7 @@ PACKAGECONFIG[rbd] = "--enable-rbd,--disable-rbd,ceph,ceph"
 PACKAGECONFIG[vhost] = "--enable-vhost-net,--disable-vhost-net,,"
 PACKAGECONFIG[ust] = 
"--enable-trace-backend=ust,--enable-trace-backend=nop,lttng-ust,"
 PACKAGECONFIG[pie] = "--enable-pie,--disable-pie,,"
+PACKAGECONFIG[seccomp] = "--enable-seccomp,--disable-seccomp,libseccomp"
 
 INSANE_SKIP_${PN} = "arch"
 
---
2.29.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145637): 
https://lists.openembedded.org/g/openembedded-core/message/145637
Mute This Topic: https://lists.openembedded.org/mt/78972214/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] commit breaks menuconfig on upstream kernel "cml1.bbclass: Handle ncurses-native being available via pkg-config"

[Nathan Rossi]

On Fri, 4 Dec 2020 at 14:58, Richard Purdie
 wrote:
>
> On Fri, 2020-12-04 at 10:53 +1000, Nathan Rossi wrote:
> > On Fri, 4 Dec 2020 at 08:31, Andrey Zhizhikin 
> > wrote:
> > > Hello Scott and Nathan,
> > >
> > > On Thu, Dec 3, 2020 at 7:18 PM Scott Branden via
> > > lists.openembedded.org
> > >  wrote:
> > > >
> > > >
> > > > On 2020-12-02 4:19 p.m., Nathan Rossi wrote:
> > > > > On Thu, 3 Dec 2020 at 05:17, Scott Branden <
> > > > > scott.bran...@broadcom.com> wrote:
> > > > > > Hi Nathan,
> > > > > >
> > > > > > Your commit:
> > > > > > "cml1.bbclass: Handle ncurses-native being available via pkg-
> > > > > > config"
> > > > > > https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=master-next&id=ce447d70df386ca55ce1672478b245851556374e
> > > > > >
> > > > > > breaks bitbake menuconfig when using the upstream kernel.
> > > > > Interesting. The purpose of the commit was to actually fix that
> > > > > exact
> > > > > use case since previously the mainline kernel menuconfig was
> > > > > relying
> > > > > on hardcoded paths to the host ncurses libraries.
> > > > >
> > > > > Would you be able to provide the error messages you are getting
> > > > > (and
> > > > > anything else that can help to reproduce the failure), because
> > > > > I am
> > > > > not able to reproduce any failures with a mainline kernel,
> > > > > linux-yocto
> > > > > (with and without the below mention patch) or with other
> > > > > projects that
> > > > > are using cml1 (e.g. u-boot).
> > > > >
> > > > > > It only works with the linux-yocto kernel due to this
> > > > > > workaround which is not upstream.
> > > > > > If you revert this commit in linux-yocto menuconfig will not
> > > > > > work in linux-yocto:
> > > > > > "menuconfig,mconf-cfg: Allow specification of ncurses
> > > > > > location"
> > > > > > https://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto/commit/scripts/kconfig/mconf-cfg.sh?h=v5.8/standard/base&id=1714a5ad9cf61f4d0f4b8432f327cca2998aba77
> > > > > This change should not be required to have menuconfig working
> > > > > when
> > > > > pkg-config is used.
> > > > >
> > > > > > Seems like your commit needs to be reverted or a change made
> > > > > > to work with the upstream kernel.
> > > > > > Or, the linux-yocto change needs to actually be
> > > > > > upstreamed.  I submitted it and the upstream maintainer
> > > > > > questioned why the change is needed:
> > > > > > https://lore.kernel.org/lkml/cak7lnatd0j3c_mfrxaju8-wmdcmrpmrfn7um0yebnfl-_zc...@mail.gmail.com/
> > > > > The problem is if it was accepted, every kernel prior to its
> > > > > inclusion
> > > > > would need to be patched, as well as other projects (u-boot,
> > > > > busybox).
> > > > > This makes supporting menuconfig using that change for kconfig
> > > > > generically problematic. This is why the pkg-config solution is
> > > > > preferable.
> > >
> > > As I've already said before I had similar issues with doing
> > > menuconfig
> > > kernel task. I took a deeper look and actually found out that the
> > > recipe-sysroot-native/usr/lib/pkgconfig/ncursesw.pc in the kernel
> > > recipe build folder contained the absolute path to the ld, which
> > > for
> > > me was taken from the SSTATE_MIRROR produced on the CI system.
> > >
> > > The string inside ncursesw.pc looked like this (note the -Wl,
> > > --dynamic-linker):
> > > Libs:  -L${pcfiledir}/../../../usr/lib -Wl,--enable-new-dtags
> > > -Wl,-rpath-link,${pcfiledir}/../../../usr/lib
> > > -Wl,-rpath-link,${pcfiledir}/../../../lib
> > > -Wl,-rpath,${pcfiledir}/../../../usr/lib
> > > -Wl,-rpath,${pcfiledir}/../../../lib -Wl,-O1
> > > -Wl,--allow-shlib-undefined
> > > -Wl,--dynamic-linker=/teamcity/work/c3acfc3a6f255dcb/build-
> > > output/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
> > > -lncursesw -ltinfo
> > >
> > > I then manually changed it to match

Re: [OE-core] commit breaks menuconfig on upstream kernel "cml1.bbclass: Handle ncurses-native being available via pkg-config"

[Nathan Rossi]

On Fri, 4 Dec 2020 at 08:31, Andrey Zhizhikin  wrote:
>
> Hello Scott and Nathan,
>
> On Thu, Dec 3, 2020 at 7:18 PM Scott Branden via
> lists.openembedded.org
>  wrote:
> >
> >
> >
> > On 2020-12-02 4:19 p.m., Nathan Rossi wrote:
> > > On Thu, 3 Dec 2020 at 05:17, Scott Branden  
> > > wrote:
> > >> Hi Nathan,
> > >>
> > >> Your commit:
> > >> "cml1.bbclass: Handle ncurses-native being available via pkg-config"
> > >> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=master-next&id=ce447d70df386ca55ce1672478b245851556374e
> > >>
> > >> breaks bitbake menuconfig when using the upstream kernel.
> > > Interesting. The purpose of the commit was to actually fix that exact
> > > use case since previously the mainline kernel menuconfig was relying
> > > on hardcoded paths to the host ncurses libraries.
> > >
> > > Would you be able to provide the error messages you are getting (and
> > > anything else that can help to reproduce the failure), because I am
> > > not able to reproduce any failures with a mainline kernel, linux-yocto
> > > (with and without the below mention patch) or with other projects that
> > > are using cml1 (e.g. u-boot).
> > >
> > >> It only works with the linux-yocto kernel due to this workaround which 
> > >> is not upstream.
> > >> If you revert this commit in linux-yocto menuconfig will not work in 
> > >> linux-yocto:
> > >> "menuconfig,mconf-cfg: Allow specification of ncurses location"
> > >> https://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto/commit/scripts/kconfig/mconf-cfg.sh?h=v5.8/standard/base&id=1714a5ad9cf61f4d0f4b8432f327cca2998aba77
> > > This change should not be required to have menuconfig working when
> > > pkg-config is used.
> > >
> > >>
> > >> Seems like your commit needs to be reverted or a change made to work 
> > >> with the upstream kernel.
> > >> Or, the linux-yocto change needs to actually be upstreamed.  I submitted 
> > >> it and the upstream maintainer questioned why the change is needed:
> > >> https://lore.kernel.org/lkml/cak7lnatd0j3c_mfrxaju8-wmdcmrpmrfn7um0yebnfl-_zc...@mail.gmail.com/
> > > The problem is if it was accepted, every kernel prior to its inclusion
> > > would need to be patched, as well as other projects (u-boot, busybox).
> > > This makes supporting menuconfig using that change for kconfig
> > > generically problematic. This is why the pkg-config solution is
> > > preferable.
>
> As I've already said before I had similar issues with doing menuconfig
> kernel task. I took a deeper look and actually found out that the
> recipe-sysroot-native/usr/lib/pkgconfig/ncursesw.pc in the kernel
> recipe build folder contained the absolute path to the ld, which for
> me was taken from the SSTATE_MIRROR produced on the CI system.
>
> The string inside ncursesw.pc looked like this (note the 
> -Wl,--dynamic-linker):
> Libs:  -L${pcfiledir}/../../../usr/lib -Wl,--enable-new-dtags
> -Wl,-rpath-link,${pcfiledir}/../../../usr/lib
> -Wl,-rpath-link,${pcfiledir}/../../../lib
> -Wl,-rpath,${pcfiledir}/../../../usr/lib
> -Wl,-rpath,${pcfiledir}/../../../lib -Wl,-O1
> -Wl,--allow-shlib-undefined
> -Wl,--dynamic-linker=/teamcity/work/c3acfc3a6f255dcb/build-output/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
> -lncursesw -ltinfo
>
> I then manually changed it to match the path on my local PC, and
> menuconfig went totally fine after that.
>
> Nathan,
> I tend to believe this issue is not caused directly by the patch
> committed, but rather by the fact that pkg-conf files do contain
> absolute paths pulled from shared state. At least this is what I've
> observed for my setup.

That is indeed an issue with uninative/sstate, likely because
UNINATIVE_LOADER (which put into BUILD_LDFLAGS) is not covered by the
sstate pack/unpack path rewriting. But uninative already handles
rewriting interp as part of its sstate unpack function, maybe it needs
to be updated to replace instances of LDFLAGS as well? Perhaps Richard
has an opinion on this.

Regards,
Nathan

>
> Scott,
> Can you have a look at the content of your  folder>/recipe-sysroot-native/usr/lib/pkgconfig/ncursesw.pc and check
> if you have the absolute ld path there, which does not match you build
> folder?
>
> > The kernel works with menuconfig without your yocto change today.
> > Only linux-yocto (with the patch mentioned above) works with your yocto 
> > change.
> > >
> > > Regards,
> > > Nathan
> >
> >
> > 
> >
>
>
> --
> Regards,
> Andrey.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145276): 
https://lists.openembedded.org/g/openembedded-core/message/145276
Mute This Topic: https://lists.openembedded.org/mt/78667947/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] commit breaks menuconfig on upstream kernel "cml1.bbclass: Handle ncurses-native being available via pkg-config"

[Nathan Rossi]

On Thu, 3 Dec 2020 at 05:17, Scott Branden  wrote:
>
> Hi Nathan,
>
> Your commit:
> "cml1.bbclass: Handle ncurses-native being available via pkg-config"
> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=master-next&id=ce447d70df386ca55ce1672478b245851556374e
>
> breaks bitbake menuconfig when using the upstream kernel.

Interesting. The purpose of the commit was to actually fix that exact
use case since previously the mainline kernel menuconfig was relying
on hardcoded paths to the host ncurses libraries.

Would you be able to provide the error messages you are getting (and
anything else that can help to reproduce the failure), because I am
not able to reproduce any failures with a mainline kernel, linux-yocto
(with and without the below mention patch) or with other projects that
are using cml1 (e.g. u-boot).

>
> It only works with the linux-yocto kernel due to this workaround which is not 
> upstream.
> If you revert this commit in linux-yocto menuconfig will not work in 
> linux-yocto:
> "menuconfig,mconf-cfg: Allow specification of ncurses location"
> https://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto/commit/scripts/kconfig/mconf-cfg.sh?h=v5.8/standard/base&id=1714a5ad9cf61f4d0f4b8432f327cca2998aba77

This change should not be required to have menuconfig working when
pkg-config is used.

>
>
> Seems like your commit needs to be reverted or a change made to work with the 
> upstream kernel.
> Or, the linux-yocto change needs to actually be upstreamed.  I submitted it 
> and the upstream maintainer questioned why the change is needed:
> https://lore.kernel.org/lkml/cak7lnatd0j3c_mfrxaju8-wmdcmrpmrfn7um0yebnfl-_zc...@mail.gmail.com/

The problem is if it was accepted, every kernel prior to its inclusion
would need to be patched, as well as other projects (u-boot, busybox).
This makes supporting menuconfig using that change for kconfig
generically problematic. This is why the pkg-config solution is
preferable.

Regards,
Nathan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145185): 
https://lists.openembedded.org/g/openembedded-core/message/145185
Mute This Topic: https://lists.openembedded.org/mt/78667947/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] cml1: Set proper LD while during kernel menuconfig

[Nathan Rossi]

On Mon, 16 Nov 2020 at 19:46, Wonmin Jung  wrote:
>
> With 'ld-is-gold' and linux kernel 5.4 or later, menuconfig
> task for kernel recipes will fail with:
>
> $ bitbake -c menuconfig virtual/kernel
> ...
> scripts/kconfig/mconf  Kconfig
> scripts/Kconfig.include:43:  gold linker 'x86_64-poky-linux-ld' not supported
> /OE/build/tmp/work-shared/qemux86-64/kernel-source/scripts/kconfig/Makefile:29:
>  recipe for target 'menuconfig' failed
> make[2]: *** [menuconfig] Error 1
> /OE/build/tmp/work-shared/qemux86-64/kernel-source/Makefile:606:
>  recipe for target 'menuconfig' failed
> make[1]: *** [menuconfig] Error 2
> /OE/build/tmp/work-shared/qemux86-64/kernel-source/Makefile:185:
>  recipe for target '__sub-make' failed
> make: *** [__sub-make] Error 2
> Command failed.
>
> This is because that the KERNEL_LD variable already set in
> kernel-arch.bbclass isn't used by do_menuconfig function of
> cml1.bbclass.
>
> To fix this issue specify LD variable while calling the kernel
> menuconfig command.
>
> Signed-off-by: Wonmin Jung 
> ---
>  meta/classes/cml1.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes/cml1.bbclass b/meta/classes/cml1.bbclass
> index d319d66ab2..02cd058fc5 100644
> --- a/meta/classes/cml1.bbclass
> +++ b/meta/classes/cml1.bbclass
> @@ -48,7 +48,7 @@ python do_menuconfig() {
>  # ensure that environment variables are overwritten with this tasks 'd' 
> values
>  d.appendVar("OE_TERMINAL_EXPORTS", " PKG_CONFIG_DIR PKG_CONFIG_PATH 
> PKG_CONFIG_LIBDIR PKG_CONFIG_SYSROOT_DIR")
>
> -oe_terminal("sh -c \"make %s; if [ \\$? -ne 0 ]; then echo 'Command 
> failed.'; printf 'Press any key to continue... '; read r; fi\"" % 
> d.getVar('KCONFIG_CONFIG_COMMAND'),
> +oe_terminal("sh -c \"make LD=${KERNEL_LD} %s; if [ \\$? -ne 0 ]; then 
> echo 'Command failed.'; printf 'Press any key to continue... '; read r; fi\"" 
> % d.getVar('KCONFIG_CONFIG_COMMAND'),

I don't think this is the right place to be setting this, since cml1
is not only for the kernel. I think a better place to set this would
be in kernel.bbclass, alongside the existing HOSTLDFLAGS setting
through KCONFIG_CONFIG_COMMAND.
(https://git.openembedded.org/openembedded-core/tree/meta/classes/kernel.bbclass#n589)

Regards,
Nathan


>  d.getVar('PN') + ' Configuration', d)
>
>  # FIXME this check can be removed when the minimum bitbake version has 
> been bumped
> --
> 2.29.2
>
>
> 
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#144612): 
https://lists.openembedded.org/g/openembedded-core/message/144612
Mute This Topic: https://lists.openembedded.org/mt/78288650/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 2/2] python3-manifest.json: Updates

[Nathan Rossi]

- Regenerate using create_manifest
- Fix up some indentation compared to generated
- Add "secrets" module into netclient package (introduced in 3.6)
- Move "urllib" python package into core package, it is used more
  commonly than just netclient (e.g. email, xml, mimetypes, pydoc)
- Update compression package dependencies due to some modules moving
  into core package
- Update dependencies due to improvements to get_module_deps handling
  modules of a python package (e.g. ctypes.utils)

Signed-off-by: Nathan Rossi 
---
 .../python/python3/python3-manifest.json   | 66 --
 1 file changed, 48 insertions(+), 18 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index 07b084d48c..69aecb7004 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -285,7 +285,7 @@
 "${libdir}/python${PYTHON_MAJMIN}/operator.py",
 "${libdir}/python${PYTHON_MAJMIN}/optparse.py",
 "${libdir}/python${PYTHON_MAJMIN}/os.py",
-   "${libdir}/python${PYTHON_MAJMIN}/pathlib.py",
+"${libdir}/python${PYTHON_MAJMIN}/pathlib.py",
 "${libdir}/python${PYTHON_MAJMIN}/pkgutil.py",
 "${libdir}/python${PYTHON_MAJMIN}/platform.py",
 "${libdir}/python${PYTHON_MAJMIN}/posixpath.py",
@@ -313,6 +313,8 @@
 "${libdir}/python${PYTHON_MAJMIN}/tokenize.py",
 "${libdir}/python${PYTHON_MAJMIN}/traceback.py",
 "${libdir}/python${PYTHON_MAJMIN}/types.py",
+"${libdir}/python${PYTHON_MAJMIN}/urllib",
+"${libdir}/python${PYTHON_MAJMIN}/urllib/parse.py",
 "${libdir}/python${PYTHON_MAJMIN}/warnings.py",
 "${libdir}/python${PYTHON_MAJMIN}/weakref.py",
 "${prefix}/lib/python${PYTHON_MAJMIN}/config*/*[!.a]"
@@ -359,7 +361,7 @@
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/operator.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/optparse.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/os.*.pyc",
-   "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pathlib.*.pyc",
+"${libdir}/python${PYTHON_MAJMIN}/__pycache__/pathlib.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pkgutil.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/platform.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/posixpath.*.pyc",
@@ -397,7 +399,9 @@
 "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__",
 "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/abc.*.pyc",
 
"${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/machinery.*.pyc",
-"${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/util.*.pyc"
+
"${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/util.*.pyc",
+"${libdir}/python${PYTHON_MAJMIN}/urllib/__pycache__",
+"${libdir}/python${PYTHON_MAJMIN}/urllib/__pycache__/parse.*.pyc"
 ]
 },
 "crypt": {
@@ -427,7 +431,10 @@
 "ctypes": {
 "summary": "Python C types support",
 "rdepends": [
-"core"
+"core",
+"crypt",
+"io",
+"math"
 ],
 "files": [
 "${libdir}/python${PYTHON_MAJMIN}/ctypes",
@@ -537,7 +544,10 @@
 "distutils": {
 "summary": "Python Distribution Utilities",
 "rdepends": [
-"core"
+"compression",
+"core",
+"email",
+"stringold"
 ],
 "files": [
 "${libdir}/python${PYTHON_MAJMIN}/distutils"
@@ -548,7 +558,6 @@
 "summary": "Python framework for running examples in docstrings",
 "rdepends": [
 "asyncio",
-"compression",
 "core",
 "debugger",
 "difflib",
@@ -577,7 +586,9 @@
 "datetime",
 "io",
 "math",
-"netclient"
+"mime",
+"netclient",
+"stringold"
 ],
 "files": [
 

[OE-core] [PATCH v2 1/2] python3: Improve handling of python3 manifest generation

[Nathan Rossi]

Specifically cover detection of modules within a python package that do
not import anything within their __init__.py. This is at least the case
with the xmlrpc package which is only used via its modules xmlrpc.server
and xmlrpc.client. Other important corner cases include ctypes.utils
which depends on some modules not used by ctypes.

This is implemented by generally assuming that importing all the modules
of a package (aka *.py within a package, excluding _*.py) will provide
enough information.

Also due to this change some modules import sysconfig, resulting in
sysconfigdata being imported. Handle the conversion of its path to a
wildcard based on the platform dependent name being replaced.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- Remove duplicate importlib.import_module()
- Only treat imports which are "__init__.py" files as python packages
  for which to import child modules for
- Add additional patch to this series for updating the manifest due to
  the updated dependency logic and other minor fixes
---
 .../python/python3/get_module_deps3.py   | 20 ++--
 .../python/python3/python3-manifest.json |  2 +-
 2 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/get_module_deps3.py 
b/meta/recipes-devtools/python/python3/get_module_deps3.py
index fd12baad84..6806f23172 100644
--- a/meta/recipes-devtools/python/python3/get_module_deps3.py
+++ b/meta/recipes-devtools/python/python3/get_module_deps3.py
@@ -9,6 +9,7 @@
 debug=False
 
 import sys
+import os
 
 # We can get a list of the modules which are currently required to run python
 # so we run python-core and get its modules, we then import what we need
@@ -48,8 +49,19 @@ current_module =  str(sys.argv[1]).rstrip()
 if(debug==True):
 log = open('log_%s' % current_module,'w')
 log.write('Module %s generated the following dependencies:\n' % 
current_module)
-try: 
-importlib.import_module('%s' % current_module)
+try:
+m = importlib.import_module(current_module)
+# handle python packages which may not include all modules in the __init__
+if os.path.basename(m.__file__) == "__init__.py":
+modulepath = os.path.dirname(m.__file__)
+for i in os.listdir(modulepath):
+if i.startswith("_") or not(i.endswith(".py")):
+continue
+submodule = "{}.{}".format(current_module, i[:-3])
+try:
+importlib.import_module(submodule)
+except:
+pass # ignore all import or other exceptions raised during 
import
 except ImportError as e:
 if (debug==True):
 log.write('Module was not found')
@@ -107,6 +119,8 @@ for item in dif:
 dep_path = dep_path.replace(soabi,'*')
 print (dep_path)
 continue
+if "_sysconfigdata" in dep_path:
+dep_path = dep_path.replace(sysconfig._get_sysconfigdata_name(), 
"_sysconfigdata*")
 
 if (debug==True):
 log.write(dep_path+'\n')
@@ -140,6 +154,8 @@ for item in dif:
 log.write(cached)
 cached = fix_path(cached)
 cached = cached.replace(cpython_tag,'*')
+if "_sysconfigdata" in cached:
+cached = cached.replace(sysconfig._get_sysconfigdata_name(), 
"_sysconfigdata*")
 print (cached)
 
 if debug==True:
diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index 3bcc9b8662..07b084d48c 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -324,7 +324,7 @@
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compression.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_markupbase.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sitebuiltins.*.pyc",
-
"${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata.*.pyc",
+
"${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata*.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_weakrefset.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/abc.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/argparse.*.pyc",
---
2.28.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#141351): 
https://lists.openembedded.org/g/openembedded-core/message/141351
Mute This Topic: https://lists.openembedded.org/mt/76103240/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] cmake.bbclass: Rework compiler program variables for allarch

[Nathan Rossi]

CMake projects can specify the NONE project type. Projects that do this
do not use any C or C++ compiler, this currently works fine with caveat
that when changing the machine/arch the compiler is different causing
signature hash differences.

To avoid the signature hash differences clear the associated C/CXX
compiler variables. In order to achieve this with overrides, simplify
the existing construction of the values using a python function and
variable setting and remove the anonymous variable setup.

Signed-off-by: Nathan Rossi 
---
 meta/classes/cmake.bbclass | 36 +---
 1 file changed, 17 insertions(+), 19 deletions(-)

diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index 8243f7ce8c..7c055e8a3d 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -21,23 +21,6 @@ python() {
 d.setVarFlag("do_compile", "progress", r"outof:^\[(\d+)/(\d+)\]\s+")
 else:
 bb.fatal("Unknown CMake Generator %s" % generator)
-
-# C/C++ Compiler (without cpu arch/tune arguments)
-if not d.getVar('OECMAKE_C_COMPILER'):
-cc_list = d.getVar('CC').split()
-if cc_list[0] == 'ccache':
-d.setVar('OECMAKE_C_COMPILER_LAUNCHER', cc_list[0])
-d.setVar('OECMAKE_C_COMPILER', cc_list[1])
-else:
-d.setVar('OECMAKE_C_COMPILER', cc_list[0])
-
-if not d.getVar('OECMAKE_CXX_COMPILER'):
-cxx_list = d.getVar('CXX').split()
-if cxx_list[0] == 'ccache':
-d.setVar('OECMAKE_CXX_COMPILER_LAUNCHER', cxx_list[0])
-d.setVar('OECMAKE_CXX_COMPILER', cxx_list[1])
-else:
-d.setVar('OECMAKE_CXX_COMPILER', cxx_list[0])
 }
 OECMAKE_AR ?= "${AR}"
 
@@ -51,8 +34,23 @@ OECMAKE_CXX_LINK_FLAGS ?= "${HOST_CC_ARCH} 
${TOOLCHAIN_OPTIONS} ${CXXFLAGS} ${LD
 CXXFLAGS += "${HOST_CC_ARCH} ${TOOLCHAIN_OPTIONS}"
 CFLAGS += "${HOST_CC_ARCH} ${TOOLCHAIN_OPTIONS}"
 
-OECMAKE_C_COMPILER_LAUNCHER ?= ""
-OECMAKE_CXX_COMPILER_LAUNCHER ?= ""
+def oecmake_map_compiler(compiler, d):
+args = d.getVar(compiler).split()
+if args[0] == "ccache":
+return args[1], args[0]
+return args[0], ""
+
+# C/C++ Compiler (without cpu arch/tune arguments)
+OECMAKE_C_COMPILER ?= "${@oecmake_map_compiler('CC', d)[0]}"
+OECMAKE_C_COMPILER_LAUNCHER ?= "${@oecmake_map_compiler('CC', d)[1]}"
+OECMAKE_CXX_COMPILER ?= "${@oecmake_map_compiler('CXX', d)[0]}"
+OECMAKE_CXX_COMPILER_LAUNCHER ?= "${@oecmake_map_compiler('CXX', d)[1]}"
+
+# clear compiler vars for allarch to avoid sig hash difference
+OECMAKE_C_COMPILER_allarch = ""
+OECMAKE_C_COMPILER_LAUNCHER_allarch = ""
+OECMAKE_CXX_COMPILER_allarch = ""
+OECMAKE_CXX_COMPILER_LAUNCHER_allarch = ""
 
 OECMAKE_RPATH ?= ""
 OECMAKE_PERLNATIVE_DIR ??= ""
---
2.28.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#141315): 
https://lists.openembedded.org/g/openembedded-core/message/141315
Mute This Topic: https://lists.openembedded.org/mt/76066028/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] python3: Improve handling of python3 manifest generation

[Nathan Rossi]

Specifically cover detection of modules within a python package that do
not import anything within their __init__.py. This is at least the case
with the xmlrpc package which is only used via its modules xmlrpc.server
and xmlrpc.client. Other important corner cases include ctypes.utils
which depends on some modules not used by ctypes.

This is implemented by generally assuming that importing all the modules
of a package (aka *.py within a package, excluding _*.py) will provide
enough information.

Also due to this change some modules import sysconfig, resulting in
sysconfigdata being imported. Handle the conversion of its path to a
wildcard based on the platform dependent name being replaced.

Signed-off-by: Nathan Rossi 
---
 .../python/python3/get_module_deps3.py | 18 --
 .../python/python3/python3-manifest.json   |  2 +-
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/get_module_deps3.py 
b/meta/recipes-devtools/python/python3/get_module_deps3.py
index fd12baad84..59767046f6 100644
--- a/meta/recipes-devtools/python/python3/get_module_deps3.py
+++ b/meta/recipes-devtools/python/python3/get_module_deps3.py
@@ -9,6 +9,7 @@
 debug=False
 
 import sys
+import os
 
 # We can get a list of the modules which are currently required to run python
 # so we run python-core and get its modules, we then import what we need
@@ -48,8 +49,17 @@ current_module =  str(sys.argv[1]).rstrip()
 if(debug==True):
 log = open('log_%s' % current_module,'w')
 log.write('Module %s generated the following dependencies:\n' % 
current_module)
-try: 
-importlib.import_module('%s' % current_module)
+try:
+importlib.import_module(current_module)
+m = importlib.import_module(current_module)
+modulepath = os.path.dirname(m.__file__)
+for i in os.listdir(modulepath):
+if not i.startswith("_") and i.endswith(".py") and i != 
os.path.basename(m.__file__):
+submodule = "{}.{}".format(current_module, i[:-3])
+try:
+importlib.import_module(submodule)
+except:
+pass # ignore all import or other exceptions raised during 
import
 except ImportError as e:
 if (debug==True):
 log.write('Module was not found')
@@ -107,6 +117,8 @@ for item in dif:
 dep_path = dep_path.replace(soabi,'*')
 print (dep_path)
 continue
+if "_sysconfigdata" in dep_path:
+dep_path = dep_path.replace(sysconfig._get_sysconfigdata_name(), 
"_sysconfigdata*")
 
 if (debug==True):
 log.write(dep_path+'\n')
@@ -140,6 +152,8 @@ for item in dif:
 log.write(cached)
 cached = fix_path(cached)
 cached = cached.replace(cpython_tag,'*')
+if "_sysconfigdata" in cached:
+cached = cached.replace(sysconfig._get_sysconfigdata_name(), 
"_sysconfigdata*")
 print (cached)
 
 if debug==True:
diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index 3bcc9b8662..07b084d48c 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -324,7 +324,7 @@
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compression.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_markupbase.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sitebuiltins.*.pyc",
-
"${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata.*.pyc",
+
"${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata*.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_weakrefset.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/abc.*.pyc",
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/argparse.*.pyc",
---
2.28.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#141314): 
https://lists.openembedded.org/g/openembedded-core/message/141314
Mute This Topic: https://lists.openembedded.org/mt/76065990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] libexif: Enable native and nativesdk

[Nathan Rossi]

Signed-off-by: Nathan Rossi 
---
 meta/recipes-support/libexif/libexif_0.6.22.bb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-support/libexif/libexif_0.6.22.bb 
b/meta/recipes-support/libexif/libexif_0.6.22.bb
index a520d5c9f9..2478ba07d8 100644
--- a/meta/recipes-support/libexif/libexif_0.6.22.bb
+++ b/meta/recipes-support/libexif/libexif_0.6.22.bb
@@ -17,3 +17,5 @@ UPSTREAM_CHECK_URI = 
"https://github.com/libexif/libexif/releases/";
 inherit autotools gettext
 
 EXTRA_OECONF += "--disable-docs"
+
+BBCLASSEXTEND = "native nativesdk"
---
2.28.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#141313): 
https://lists.openembedded.org/g/openembedded-core/message/141313
Mute This Topic: https://lists.openembedded.org/mt/76065845/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/2] glibc-testsuite: Exclude this recipe from world builds

[Nathan Rossi]

This recipe is intended to be run manually or via oeqa.

[YOCTO #13737]

Signed-off-by: Nathan Rossi 
---
 meta/recipes-core/glibc/glibc-testsuite_2.31.bb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-core/glibc/glibc-testsuite_2.31.bb 
b/meta/recipes-core/glibc/glibc-testsuite_2.31.bb
index c44f30bf1c..d887aeff79 100644
--- a/meta/recipes-core/glibc/glibc-testsuite_2.31.bb
+++ b/meta/recipes-core/glibc/glibc-testsuite_2.31.bb
@@ -1,5 +1,7 @@
 require glibc_${PV}.bb
 
+EXCLUDE_FROM_WORLD = "1"
+
 # handle PN differences
 FILESEXTRAPATHS_prepend := "${THISDIR}/glibc:"
 
---
2.24.1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] glibc-testsuite: Remove the do_install task

[Nathan Rossi]

This task is not intended to be run as this recipe does not produce any
packages or install any sysroot content. Additionally running this task
behaves differently from a normal glibc build, and can result in issues.

[YOCTO #13737]

Signed-off-by: Nathan Rossi 
---
 meta/recipes-core/glibc/glibc-testsuite_2.31.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-core/glibc/glibc-testsuite_2.31.bb 
b/meta/recipes-core/glibc/glibc-testsuite_2.31.bb
index 657fd4dbc1..c44f30bf1c 100644
--- a/meta/recipes-core/glibc/glibc-testsuite_2.31.bb
+++ b/meta/recipes-core/glibc/glibc-testsuite_2.31.bb
@@ -58,3 +58,4 @@ addtask do_check after do_compile
 
 inherit nopackages
 deltask do_stash_locale
+deltask do_install
---
2.24.1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v4 0/5] Microblaze tune updates

[Nathan Rossi]

On Wed, 5 Feb 2020 at 07:06, Mark Hatle  wrote:
>
> Version 4 of the patch set.
>
> I've collected all of the patches resently sent, and reworked them per
> the comments on the mailing list.
>
> The patches are directly available at:
>
> git://git.openembedded.org/openembedded-core-contrib mgh/xilinx-microblaze
>
> The work was broken up into a few more individual pieces.  This will allow
> backports to Zeus for fixes vs new features.
>
> Mark Hatle (5):
>   microblaze tune: change microblazeeb to microblaze
>   microblaze tune: Allow no version to be set
>   microblaze tune: Enable 64-bit
>   microblaze tune: cleanup +=
>   Microblaze: Adjust Linux items from microblazeeb to microblaze

Looks good. I gave it a quick test, qemu big endian builds and boots
core-image-minimal now.

Thanks,
Nathan
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] microblaze tune: Correct two minor issues with the microblaze tune

[Nathan Rossi]

On Tue, 4 Feb 2020 at 04:55, Mark Hatle  wrote:
>
>
>
> On 1/31/20 10:49 PM, Nathan Rossi wrote:
> > On Sat, 1 Feb 2020 at 08:13, Mark Hatle  
> > wrote:
> >>
> >> From: Mark Hatle 
> >>
> >> TUNE_ARCH - microblazeeb is not a valid architecture, microblaze is big
> >> endian, microblazeel is the little endian version.
> >>
> >> Version arguments:
> >>   If a version feature is not defined, then we don't want to set either
> >>   TUNE_CCARGS or MBPKGARCH_VERSION.
> >>
> >> Signed-off-by: Mark Hatle 
> >> Signed-off-by: Mark Hatle 
> >> ---
> >>  meta/conf/machine/include/microblaze/arch-microblaze.inc  | 2 +-
> >>  .../include/microblaze/feature-microblaze-versions.inc| 4 ++--
> >>  2 files changed, 3 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/meta/conf/machine/include/microblaze/arch-microblaze.inc 
> >> b/meta/conf/machine/include/microblaze/arch-microblaze.inc
> >> index eab7171cb2..265898b6b6 100644
> >> --- a/meta/conf/machine/include/microblaze/arch-microblaze.inc
> >> +++ b/meta/conf/machine/include/microblaze/arch-microblaze.inc
> >> @@ -46,7 +46,7 @@ require 
> >> conf/machine/include/microblaze/feature-microblaze-versions.inc
> >>  require conf/machine/include/microblaze/feature-microblaze-math.inc
> >>
> >>  # Architecture name, either 'microblazeeb' or 'microblazeel' depending on 
> >> endianess
> >
> > Worth updating the comment at the same time.
> >
> >> -TUNE_ARCH = "microblaze${@bb.utils.contains("TUNE_FEATURES", "bigendian", 
> >> "eb", "el", d)}"
> >> +TUNE_ARCH = "microblaze${@bb.utils.contains("TUNE_FEATURES", "bigendian", 
> >> "", "el", d)}"
> >
> > This change will break some things in oe-core (and probably
> > meta-xilinx too). They should be updated at the same time.
>
> Do you have an example of what?  Cause without this, all autoconf invocation 
> of
> big endian failed for me due to invalid architecture.

So core-image-minimal builds were working when I had been testing, but
that was a number of releases ago now.

There are only a few instances in oe-core that check for "microblazeeb".

classes/meson.bbclass:60:elif arch == "microblazeel" or arch ==
"microblazeeb":
Probably just arch.startswith("microblaze") here

classes/kernel-arch.bbclass:37:elif re.match('microblazee[bl]',
a):return 'microblaze'
This one won't break, since "microblaze" is already a valid kernel
arch. But for clean up just needs to be microblazeel.

recipes-extended/ghostscript/ghostscript/microblazeeb/objarch.h

>
> > Might also be worth removing the microblazeeb references in siteinfo,
> > and elf.py. Since they were added to support this.
>
> I looked at these and left them "just in case", as they didn't seem to 
> conflict
> with anything that I saw.

Leaving them is fine, though removal to avoid confusion and for clean
up in general is probably a good idea.

Thanks,
Nathan


>
> --Mark
>
> > Regards,
> > Nathan
> >
> >>
> >>  # Package Architecture formatting
> >>  TUNE_PKGARCH = 
> >> "microblaze${MBPKGARCH_ENDIAN}${MBPKGARCH_VERSION}${MBPKGARCH_TUNE}${MBPKGARCH_MATH}"
> >> diff --git 
> >> a/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc 
> >> b/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc
> >> index 3221e2aab7..003fde3e07 100644
> >> --- a/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc
> >> +++ b/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc
> >> @@ -64,6 +64,6 @@ TUNECONFLICTS[v10.0] = "v8.00 v8.10 v8.20 v8.30 v8.40 
> >> v8.50 v9.0 v9.1 v9.2 v9.3
> >>  TUNECONFLICTS[v11.0] = "v8.00 v8.10 v8.20 v8.30 v8.40 v8.50 v9.0 v9.1 
> >> v9.2 v9.3 v9.4 v9.5 v9.6 v10.0"
> >>
> >>  # Version flags
> >> -TUNE_CCARGS += "-mcpu=${@microblaze_current_version(d, True)}"
> >> -MBPKGARCH_VERSION = "-${@microblaze_current_version(d)}"
> >> +TUNE_CCARGS += "${@'-mcpu=${@microblaze_current_version(d, True)' if 
> >> microblaze_current_version(d, True) != '' else ''}"
> >> +MBPKGARCH_VERSION = "${@'-${@microblaze_current_version(d, True)' if 
> >> microblaze_current_version(d, True) != '' else ''}"
> >>
> >> --
> >> 2.17.1
> >>
> >> --
> >> ___
> >> Openembedded-core mailing list
> >> Openembedded-core@lists.openembedded.org
> >> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 1/2] microblaze tune: Enable 64-bit

[Nathan Rossi]

On Tue, 4 Feb 2020 at 04:36, Mark Hatle  wrote:
>
>
>
> On 1/31/20 10:50 PM, Nathan Rossi wrote:
> > On Sat, 1 Feb 2020 at 09:33, Mark Hatle  
> > wrote:
> >>
> >> From: Mark Hatle 
> >>
> >> 64-bit is not yet available in Linux, but some non-Linux uses exist.
> >>
> >> Signed-off-by: Mark Hatle 
> >> Signed-off-by: Mark Hatle 
> >> ---
> >>  meta/conf/machine/include/microblaze/arch-microblaze.inc | 9 -
> >>  1 file changed, 8 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/meta/conf/machine/include/microblaze/arch-microblaze.inc 
> >> b/meta/conf/machine/include/microblaze/arch-microblaze.inc
> >> index 265898b6b6..fb777d929c 100644
> >> --- a/meta/conf/machine/include/microblaze/arch-microblaze.inc
> >> +++ b/meta/conf/machine/include/microblaze/arch-microblaze.inc
> >> @@ -4,6 +4,9 @@
> >>  TUNEVALID[microblaze] = "MicroBlaze"
> >>  MACHINEOVERRIDES =. "${@bb.utils.contains("TUNE_FEATURES", "microblaze", 
> >> "microblaze:", "", d)}"
> >>
> >> +TUNEVALID[64-bit] = "64-bit MicroBlaze"
> >> +MACHINEOVERRIDES =. "${@bb.utils.contains("TUNE_FEATURES", "64-bit", 
> >> "microblaze64:", "", d)}"
> >
> > This should probably TUNECONFLICTS with all unsupported versions.
>
> I have no information on what versions are unsupported with 64-bit.  If I
> receive that information, I will add it as conflicts.

It looks like it was added in v11.0 (and unsupported in all prior
versions), according to the microblaze reference manual
(https://www.xilinx.com/support/documentation/sw_manuals/xilinx2019_2/ug984-vivado-microblaze-ref.pdf).

Regards,
Nathan

>
> --Mark
>
> > Regards,
> > Nathan
> >
> >
> >> +
> >>  # Endian
> >>  TUNEVALID[bigendian] = "Use Microblaze Big Endian"
> >>  TUNECONFLICTS[bigendian] += "v10.0"
> >> @@ -26,6 +29,7 @@ TUNECONFLICTS[frequency-optimized] += "v8.00 v8.10 v8.20 
> >> v8.30 v8.40 v8.50 v9.0
> >>  TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "barrel-shift", 
> >> "-mxl-barrel-shift", "-mno-xl-barrel-shift", d)}"
> >>  TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "pattern-compare", 
> >> "-mxl-pattern-compare", "-mno-xl-pattern-compare", d)}"
> >>  TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", 
> >> "frequency-optimized", "-mxl-frequency", "", d)}"
> >> +TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "64-bit", "-m64", 
> >> "", d)}"
> >>
> >>  # Disable reorder for v8.30 if pattern-compare is not enabled
> >>  TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "reorder", \
> >> @@ -48,6 +52,9 @@ require 
> >> conf/machine/include/microblaze/feature-microblaze-math.inc
> >>  # Architecture name, either 'microblazeeb' or 'microblazeel' depending on 
> >> endianess
> >>  TUNE_ARCH = "microblaze${@bb.utils.contains("TUNE_FEATURES", "bigendian", 
> >> "", "el", d)}"
> >>
> >> +# Add 64-bit to the PKGARCH if enabled.
> >> +MBPKGARCH_SIZE = "${@bb.utils.contains("TUNE_FEATURES", "64-bit", "64", 
> >> "", d)}"
> >> +
> >>  # Package Architecture formatting
> >> -TUNE_PKGARCH = 
> >> "microblaze${MBPKGARCH_ENDIAN}${MBPKGARCH_VERSION}${MBPKGARCH_TUNE}${MBPKGARCH_MATH}"
> >> +TUNE_PKGARCH = 
> >> "microblaze${MBPKGARCH_SIZE}${MBPKGARCH_ENDIAN}${MBPKGARCH_VERSION}${MBPKGARCH_TUNE}${MBPKGARCH_MATH}"
> >>
> >> --
> >> 2.17.1
> >>
> >> --
> >> ___
> >> Openembedded-core mailing list
> >> Openembedded-core@lists.openembedded.org
> >> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 1/2] microblaze tune: Enable 64-bit

[Nathan Rossi]

On Sat, 1 Feb 2020 at 09:33, Mark Hatle  wrote:
>
> From: Mark Hatle 
>
> 64-bit is not yet available in Linux, but some non-Linux uses exist.
>
> Signed-off-by: Mark Hatle 
> Signed-off-by: Mark Hatle 
> ---
>  meta/conf/machine/include/microblaze/arch-microblaze.inc | 9 -
>  1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/meta/conf/machine/include/microblaze/arch-microblaze.inc 
> b/meta/conf/machine/include/microblaze/arch-microblaze.inc
> index 265898b6b6..fb777d929c 100644
> --- a/meta/conf/machine/include/microblaze/arch-microblaze.inc
> +++ b/meta/conf/machine/include/microblaze/arch-microblaze.inc
> @@ -4,6 +4,9 @@
>  TUNEVALID[microblaze] = "MicroBlaze"
>  MACHINEOVERRIDES =. "${@bb.utils.contains("TUNE_FEATURES", "microblaze", 
> "microblaze:", "", d)}"
>
> +TUNEVALID[64-bit] = "64-bit MicroBlaze"
> +MACHINEOVERRIDES =. "${@bb.utils.contains("TUNE_FEATURES", "64-bit", 
> "microblaze64:", "", d)}"

This should probably TUNECONFLICTS with all unsupported versions.

Regards,
Nathan


> +
>  # Endian
>  TUNEVALID[bigendian] = "Use Microblaze Big Endian"
>  TUNECONFLICTS[bigendian] += "v10.0"
> @@ -26,6 +29,7 @@ TUNECONFLICTS[frequency-optimized] += "v8.00 v8.10 v8.20 
> v8.30 v8.40 v8.50 v9.0
>  TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "barrel-shift", 
> "-mxl-barrel-shift", "-mno-xl-barrel-shift", d)}"
>  TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "pattern-compare", 
> "-mxl-pattern-compare", "-mno-xl-pattern-compare", d)}"
>  TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "frequency-optimized", 
> "-mxl-frequency", "", d)}"
> +TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "64-bit", "-m64", "", 
> d)}"
>
>  # Disable reorder for v8.30 if pattern-compare is not enabled
>  TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "reorder", \
> @@ -48,6 +52,9 @@ require 
> conf/machine/include/microblaze/feature-microblaze-math.inc
>  # Architecture name, either 'microblazeeb' or 'microblazeel' depending on 
> endianess
>  TUNE_ARCH = "microblaze${@bb.utils.contains("TUNE_FEATURES", "bigendian", 
> "", "el", d)}"
>
> +# Add 64-bit to the PKGARCH if enabled.
> +MBPKGARCH_SIZE = "${@bb.utils.contains("TUNE_FEATURES", "64-bit", "64", "", 
> d)}"
> +
>  # Package Architecture formatting
> -TUNE_PKGARCH = 
> "microblaze${MBPKGARCH_ENDIAN}${MBPKGARCH_VERSION}${MBPKGARCH_TUNE}${MBPKGARCH_MATH}"
> +TUNE_PKGARCH = 
> "microblaze${MBPKGARCH_SIZE}${MBPKGARCH_ENDIAN}${MBPKGARCH_VERSION}${MBPKGARCH_TUNE}${MBPKGARCH_MATH}"
>
> --
> 2.17.1
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] microblaze tune: Correct two minor issues with the microblaze tune

[Nathan Rossi]

On Sat, 1 Feb 2020 at 08:13, Mark Hatle  wrote:
>
> From: Mark Hatle 
>
> TUNE_ARCH - microblazeeb is not a valid architecture, microblaze is big
> endian, microblazeel is the little endian version.
>
> Version arguments:
>   If a version feature is not defined, then we don't want to set either
>   TUNE_CCARGS or MBPKGARCH_VERSION.
>
> Signed-off-by: Mark Hatle 
> Signed-off-by: Mark Hatle 
> ---
>  meta/conf/machine/include/microblaze/arch-microblaze.inc  | 2 +-
>  .../include/microblaze/feature-microblaze-versions.inc| 4 ++--
>  2 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/meta/conf/machine/include/microblaze/arch-microblaze.inc 
> b/meta/conf/machine/include/microblaze/arch-microblaze.inc
> index eab7171cb2..265898b6b6 100644
> --- a/meta/conf/machine/include/microblaze/arch-microblaze.inc
> +++ b/meta/conf/machine/include/microblaze/arch-microblaze.inc
> @@ -46,7 +46,7 @@ require 
> conf/machine/include/microblaze/feature-microblaze-versions.inc
>  require conf/machine/include/microblaze/feature-microblaze-math.inc
>
>  # Architecture name, either 'microblazeeb' or 'microblazeel' depending on 
> endianess

Worth updating the comment at the same time.

> -TUNE_ARCH = "microblaze${@bb.utils.contains("TUNE_FEATURES", "bigendian", 
> "eb", "el", d)}"
> +TUNE_ARCH = "microblaze${@bb.utils.contains("TUNE_FEATURES", "bigendian", 
> "", "el", d)}"

This change will break some things in oe-core (and probably
meta-xilinx too). They should be updated at the same time.

Might also be worth removing the microblazeeb references in siteinfo,
and elf.py. Since they were added to support this.

Regards,
Nathan

>
>  # Package Architecture formatting
>  TUNE_PKGARCH = 
> "microblaze${MBPKGARCH_ENDIAN}${MBPKGARCH_VERSION}${MBPKGARCH_TUNE}${MBPKGARCH_MATH}"
> diff --git 
> a/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc 
> b/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc
> index 3221e2aab7..003fde3e07 100644
> --- a/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc
> +++ b/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc
> @@ -64,6 +64,6 @@ TUNECONFLICTS[v10.0] = "v8.00 v8.10 v8.20 v8.30 v8.40 v8.50 
> v9.0 v9.1 v9.2 v9.3
>  TUNECONFLICTS[v11.0] = "v8.00 v8.10 v8.20 v8.30 v8.40 v8.50 v9.0 v9.1 v9.2 
> v9.3 v9.4 v9.5 v9.6 v10.0"
>
>  # Version flags
> -TUNE_CCARGS += "-mcpu=${@microblaze_current_version(d, True)}"
> -MBPKGARCH_VERSION = "-${@microblaze_current_version(d)}"
> +TUNE_CCARGS += "${@'-mcpu=${@microblaze_current_version(d, True)' if 
> microblaze_current_version(d, True) != '' else ''}"
> +MBPKGARCH_VERSION = "${@'-${@microblaze_current_version(d, True)' if 
> microblaze_current_version(d, True) != '' else ''}"
>
> --
> 2.17.1
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] glibc-testing.inc: Remove testglibc script generation

[Nathan Rossi]

Remove the generation of the testglibc script which could be used to run
the glibc test suite with a remote target. The same functionality can
now be achieved with the 'do_check' task of glibc-testsuite or with
oe-selftest (for automation of execution against qemu-user/qemu-system
targets).

Signed-off-by: Nathan Rossi 
---
 meta/recipes-core/glibc/glibc-testing.inc | 95 ---
 meta/recipes-core/glibc/glibc.inc |  1 -
 2 files changed, 96 deletions(-)
 delete mode 100644 meta/recipes-core/glibc/glibc-testing.inc

diff --git a/meta/recipes-core/glibc/glibc-testing.inc 
b/meta/recipes-core/glibc/glibc-testing.inc
deleted file mode 100644
index 0a42ae7f7c..00
--- a/meta/recipes-core/glibc/glibc-testing.inc
+++ /dev/null
@@ -1,95 +0,0 @@
-do_compile_append () {
-   # now generate script to drive testing
-   echo "#!/usr/bin/env sh" >${B}/${HOST_PREFIX}testglibc
-   set >> ${B}/${HOST_PREFIX}testglibc
-   # prune out the unneeded vars
-   sed -i -e "/^BASH/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^USER/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^OPT/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^DIRSTACK/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^EUID/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^FUNCNAME/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^GROUPS/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^HOST/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^HOME/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^IFS/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^LC_ALL/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^LOGNAME/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^MACHTYPE/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^OSTYPE/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^PIPE/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^SHELL/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^'/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^UID/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^TERM/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^PKG_/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^POSIXLY_/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^PPID/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^PS4/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^Q/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^SHLVL/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^STAGING/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^LD_LIBRARY_PATH/d" ${B}/${HOST_PREFIX}testglibc
-   sed -i -e "/^PSEUDO/d" ${B}/${HOST_PREFIX}testglibc
-
-   # point to real sysroot not the toolchain bootstrap sysroot
-   sed -i -e "s/\-tcbootstrap//g" ${B}/${HOST_PREFIX}testglibc
-
-   # use the final cross-gcc to test since some tests need libstdc++
-   sed -i -e "s/^PATH=.*\.gcc-cross-initial\:/PATH=/g" 
${B}/${HOST_PREFIX}testglibc
-
-   # append execution part script
-cat >> ${B}/${HOST_PREFIX}testglibc << STOP
-target="\$1"
-if [ "x\$target" = "x" ]
-then
-   echo "Please specify the target machine and remote user in form of 
user@target"
-   exit 1;
-fi
-ssh \$target ls \$PWD\  2>&1 > /dev/null
-if [ "x\$?" != "x0" ]
-then
-   echo "Failed connecting to \$target it could be because of:"
-   echo "1. You dont have passwordless ssh setup to access \$target"
-   echo "2. NFS share on \$target is not mounted or if mounted then not 
matching the build tree layout."
-   echo "   The tree should be accessible at same location on build host 
and target"
-   echo "   You can add nfs-client to IMAGE_FEATURES to get the nfs client 
on target"
-   echo "3. nfs server on build host is not running."
-   echo "   Please make sure that you have 'no_root_squash' added in 
/etc/exports if you want"
-   echo "   to test as root user on target (usually its recommended to 
create a non"
-   echo "   root user."
-   echo "   As a sanity check make sure that target can read/write to the 
glibc build tree"
-   echo "   Please refer to ${S}/EGLIBC.cross-testing for further 
instructions on setup"
-   exit 1
-fi
-   echo "# we test using cross compiler from real sysroot therefore 
override the" > ${B}/configparms
-   echo "# definitions that come from ${B}/config.make" >> ${B}/configparms
-   fgrep tcbootstrap ${B}/

[OE-core] [PATCH 2/2] gcc: Improve build reproduciblity

[Nathan Rossi]

Prevent the gcc embedded checksum from containing a checksum that was
computed with build specific paths. The checksum-options file included
the value of LINKER/LDFLAGS which contains DEBUG_PREFIX_MAP and
STAGING_DIR_TARGET.

Signed-off-by: Nathan Rossi 
---
This change along with the previous gcc-target/gcc-cross changes to
handle configargs.h are sufficient to have ReproducibleTests of
oe-selftest pass when 'tools-sdk' is in IMAGE_FEATURES.
---
 meta/recipes-devtools/gcc/gcc-cross.inc  | 7 +++
 meta/recipes-devtools/gcc/gcc-target.inc | 7 +++
 2 files changed, 14 insertions(+)

diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc 
b/meta/recipes-devtools/gcc/gcc-cross.inc
index 06ba3ccd15..6784d9673f 100644
--- a/meta/recipes-devtools/gcc/gcc-cross.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross.inc
@@ -68,6 +68,13 @@ do_compile () {
sed -i 's@${STAGING_DIR_TARGET}@/host@g' ${B}/gcc/configargs.h
sed -i 's@${STAGING_DIR_HOST}@/host@g' ${B}/gcc/configargs.h
 
+   # Prevent sysroot/workdir paths from being used in checksum-options.
+   # checksum-options is used to generate a checksum which is embedded into
+   # the output binary.
+   oe_runmake TARGET-gcc=checksum-options all-gcc
+   sed -i 's@${DEBUG_PREFIX_MAP}@@g' ${B}/gcc/checksum-options
+   sed -i 's@${STAGING_DIR_HOST}@/host@g' ${B}/gcc/checksum-options
+
oe_runmake all-host configure-target-libgcc
(cd ${B}/${TARGET_SYS}/libgcc; oe_runmake enable-execute-stack.c 
unwind.h md-unwind-support.h sfp-machine.h gthr-default.h)
# now generate script to drive testing
diff --git a/meta/recipes-devtools/gcc/gcc-target.inc 
b/meta/recipes-devtools/gcc/gcc-target.inc
index 987e88d32c..18d078db0a 100644
--- a/meta/recipes-devtools/gcc/gcc-target.inc
+++ b/meta/recipes-devtools/gcc/gcc-target.inc
@@ -145,6 +145,13 @@ do_compile () {
sed -i 's@${STAGING_DIR_TARGET}@/@g' ${B}/gcc/configargs.h
sed -i 's@${STAGING_DIR_HOST}@/@g' ${B}/gcc/configargs.h
 
+   # Prevent sysroot/workdir paths from being used in checksum-options.
+   # checksum-options is used to generate a checksum which is embedded into
+   # the output binary.
+   oe_runmake TARGET-gcc=checksum-options all-gcc
+   sed -i 's@${DEBUG_PREFIX_MAP}@@g' ${B}/gcc/checksum-options
+   sed -i 's@${STAGING_DIR_TARGET}@/@g' ${B}/gcc/checksum-options
+
oe_runmake all-host
 }
 
---
2.24.0
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] gcc-cross.inc: Remove test runner script generation

[Nathan Rossi]

Remove the generation of the testgcc script which could be used to run
the gcc test suite against a cross compiler with a remote execution
target. The same functionality can now be achieved with the 'do_check'
task of gcc-runtime or with oe-selftest (for automation of execution
against qemu-user/qemu-system targets).

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/gcc/gcc-cross.inc | 90 -
 1 file changed, 90 deletions(-)

diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc 
b/meta/recipes-devtools/gcc/gcc-cross.inc
index 6784d9673f..c68cdd5dc4 100644
--- a/meta/recipes-devtools/gcc/gcc-cross.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross.inc
@@ -77,96 +77,6 @@ do_compile () {
 
oe_runmake all-host configure-target-libgcc
(cd ${B}/${TARGET_SYS}/libgcc; oe_runmake enable-execute-stack.c 
unwind.h md-unwind-support.h sfp-machine.h gthr-default.h)
-   # now generate script to drive testing
-   echo "#!/usr/bin/env sh" >${B}/${TARGET_PREFIX}testgcc
-   set >> ${B}/${TARGET_PREFIX}testgcc
-   # prune out the unneeded vars
-   sed -i -e "/^BASH/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^USER/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^OPT/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^DIRSTACK/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^EUID/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^FUNCNAME/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^GROUPS/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^HOST/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^HOME/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^IFS/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^LC_ALL/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^LOGNAME/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^MACHTYPE/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^OSTYPE/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^PIPE/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^SHELL/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^'/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^UID/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^TERM/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^PKG_/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^POSIXLY_/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^PPID/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^PS4/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^Q/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^SHLVL/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^STAGING/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^LD_LIBRARY_PATH/d" ${B}/${TARGET_PREFIX}testgcc
-   sed -i -e "/^PSEUDO/d" ${B}/${TARGET_PREFIX}testgcc
-
-   # append execution part of the script
-cat >> ${B}/${TARGET_PREFIX}testgcc << STOP
-target="\$1"
-usage () {
-   echo "Usage:"
-   echo "\$0 user@target 'extra options to dejagnu'"
-   echo "\$0 target 'extra options to dejagnu'"
-   echo "\$0 target"
-   echo "e.g. \$0 192.168.7.2 ' dg.exp=visibility-d.c'"
-   echo "will only run visibility-d.c test case"
-   echo "e.g. \$0 192.168.7.2 '/-mthumb dg.exp=visibility-d.c'"
-   echo "will only run visibility-d.c test case in thumb mode"
-   echo "You need to have dejagnu autogen expect installed"
-   echo "on the build host"
-}
-if [ "x\$target" = "x" ]
-then
-   echo "Please specify the target machine and remote user in form of 
user@target\n"
-   usage
-   exit 1;
-fi
-
-shift
-
-echo "\$target" | grep "@" 2>&1 > /dev/null
-if [ "x\$?" = "x0" ]
-then
-   user=\$(echo \$target | cut -d '@' -f 1)
-   target=\$(echo \$target | cut -d '@' -f 2)
-else
-   user=\$USER
-fi
-ssh \$user@\$target date 2>&1 > /dev/null
-if [ "x\$?" != "x0" ]
-then
-   echo "Failed connecting to \$user@\$target it could be because"
-   echo "you don't have passwordless ssh setup to access \$target"
-   echo "or sometimes host key has been changed"
-   echo "in such case do something like below on build host"
-   echo "ssh-keygen -f "~/.ssh/known_hosts" -R \$target"
-   echo "and then try ssh \$user@\$target"
-
-   usage
-   exit 1
-fi
-   echo "lappend boards_dir [pwd]/../../.." > ${B}/sit

[OE-core] [PATCH 1/2] gcc-target.inc: Prevent sysroot from leaking into configargs.h

[Nathan Rossi]

Prevent the full recipe-sysroot path from leaking into configargs.h. The
configargs.h header is intended to be static and unchanged as the
content is used as a means of determining that a gcc plugin is built for
the same gcc. This also effects the output of 'gcc -v'. Due to per
recipe sysroots and staging, the sysroot path would be replaced with the
sysroot local to the recipe thus changing the content of configargs.h.
This change also improves gcc binary reproducibility. The sysroot path
is replaced with the base target root "/".

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/gcc/gcc-target.inc | 8 
 1 file changed, 8 insertions(+)

diff --git a/meta/recipes-devtools/gcc/gcc-target.inc 
b/meta/recipes-devtools/gcc/gcc-target.inc
index bdc6ff658f..987e88d32c 100644
--- a/meta/recipes-devtools/gcc/gcc-target.inc
+++ b/meta/recipes-devtools/gcc/gcc-target.inc
@@ -137,6 +137,14 @@ FILES_${PN}-doc = "\
 "
 
 do_compile () {
+   # Prevent full target sysroot path from being used in configargs.h 
header,
+   # as it will be rewritten when used by other sysroots preventing support
+   # for gcc plugins. Additionally the path is embeddeded into the output
+   # binary, this prevents building a reproducible binary.
+   oe_runmake configure-gcc
+   sed -i 's@${STAGING_DIR_TARGET}@/@g' ${B}/gcc/configargs.h
+   sed -i 's@${STAGING_DIR_HOST}@/@g' ${B}/gcc/configargs.h
+
oe_runmake all-host
 }
 
---
2.24.0
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] gcc-cross.inc: Prevent native sysroot from leaking into configargs.h

[Nathan Rossi]

Prevent the native(sdk) sysroot path from leaking into configargs.h. The
configargs.h header is intended to be static and unchanged as the
content is used as a means of determining that a gcc plugin is built for
the same gcc. This also effects the output of 'gcc --version'. Due to
per recipe sysroots and staging, the sysroot path would be replaced with
the sysroot local to the recipe thus changing the content of
configargs.h.

The sysroot path is replaced with a generic "/host" prefix which
represents the host sysroot (e.g. native or nativesdk).

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/gcc/gcc-cross.inc   | 7 +++
 meta/recipes-devtools/gcc/gcc-runtime.inc | 4 
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc 
b/meta/recipes-devtools/gcc/gcc-cross.inc
index 8855bb1f34..06ba3ccd15 100644
--- a/meta/recipes-devtools/gcc/gcc-cross.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross.inc
@@ -61,6 +61,13 @@ do_compile () {
export CXXFLAGS_FOR_TARGET="${TARGET_CXXFLAGS}"
export LDFLAGS_FOR_TARGET="${TARGET_LDFLAGS}"
 
+   # Prevent native/host sysroot path from being used in configargs.h 
header,
+   # as it will be rewritten when used by other sysroots preventing support
+   # for gcc plugins
+   oe_runmake configure-gcc
+   sed -i 's@${STAGING_DIR_TARGET}@/host@g' ${B}/gcc/configargs.h
+   sed -i 's@${STAGING_DIR_HOST}@/host@g' ${B}/gcc/configargs.h
+
oe_runmake all-host configure-target-libgcc
(cd ${B}/${TARGET_SYS}/libgcc; oe_runmake enable-execute-stack.c 
unwind.h md-unwind-support.h sfp-machine.h gthr-default.h)
# now generate script to drive testing
diff --git a/meta/recipes-devtools/gcc/gcc-runtime.inc 
b/meta/recipes-devtools/gcc/gcc-runtime.inc
index 2da3c02ef0..536b18d97f 100644
--- a/meta/recipes-devtools/gcc/gcc-runtime.inc
+++ b/meta/recipes-devtools/gcc/gcc-runtime.inc
@@ -302,10 +302,6 @@ do_check() {
 
 # HACK: this works around the configure setting CXX with -nostd* args
 sed -i 's/-nostdinc++ -nostdlib++//g' $(find ${B} -name testsuite_flags | 
head -1)
-# HACK: this works around the de-stashing changes to configargs.h, as well 
as recipe-sysroot changing the content
-sed -i '/static const char configuration_arguments/d' ${B}/gcc/configargs.h
-${CC} -v 2>&1 | grep "^Configured with:" | \
-sed 's/Configured with: \(.*\)/static const char 
configuration_arguments[] = "\1";/g' >> ${B}/gcc/configargs.h
 
 if [ "${TOOLCHAIN_TEST_TARGET}" = "user" ]; then
 # qemu user has issues allocating large amounts of memory
---
2.24.0
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/selftest/cases/gcc.py: Split 'gcc' and 'g++' testsuites

[Nathan Rossi]

Split the GccCrossSelfTest testcase into separate testcases for 'gcc'
and 'g++' respectively. In order to split them use the "check-gcc-*"
language make check targets.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/gcc.py | 20 
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/gcc.py 
b/meta/lib/oeqa/selftest/cases/gcc.py
index 5a917b9c44..3efe15228f 100644
--- a/meta/lib/oeqa/selftest/cases/gcc.py
+++ b/meta/lib/oeqa/selftest/cases/gcc.py
@@ -21,8 +21,10 @@ class GccSelfTestBase(OESelftestTestCase, 
OEPTestResultTestCase):
 def run_check(self, *suites, ssh = None):
 targets = set()
 for s in suites:
-if s in ["gcc", "g++"]:
-targets.add("check-gcc")
+if s == "gcc":
+targets.add("check-gcc-c")
+elif s == "g++":
+targets.add("check-gcc-c++")
 else:
 targets.add("check-target-{}".format(s))
 
@@ -77,7 +79,12 @@ class GccSelfTestBase(OESelftestTestCase, 
OEPTestResultTestCase):
 @OETestTag("toolchain-user")
 class GccCrossSelfTest(GccSelfTestBase):
 def test_cross_gcc(self):
-self.run_check("gcc", "g++")
+self.run_check("gcc")
+
+@OETestTag("toolchain-user")
+class GxxCrossSelfTest(GccSelfTestBase):
+def test_cross_gxx(self):
+self.run_check("g++")
 
 @OETestTag("toolchain-user")
 class GccLibAtomicSelfTest(GccSelfTestBase):
@@ -109,7 +116,12 @@ class GccLibItmSelfTest(GccSelfTestBase):
 @OETestTag("toolchain-system")
 class GccCrossSelfTestSystemEmulated(GccSelfTestBase):
 def test_cross_gcc(self):
-self.run_check_emulated("gcc", "g++")
+self.run_check_emulated("gcc")
+
+@OETestTag("toolchain-system")
+class GxxCrossSelfTestSystemEmulated(GccSelfTestBase):
+def test_cross_gxx(self):
+self.run_check_emulated("g++")
 
 @OETestTag("toolchain-system")
 class GccLibAtomicSelfTestSystemEmulated(GccSelfTestBase):
---
2.24.0
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/utils/qemurunner.py: Handle QEMU machines with a single serial

[Nathan Rossi]

Not all QEMU machines are capable of having more than one serial port,
this is due to the machine emulating a physical device/board.

Rework QemuRunner to handle machines that only have 1 serial port, where
the serial port shares output of the kernel log buffer and a login
console. In this case the output is mixed but enables the machine to
boot and have QemuRunner detect the login prompt.

QemuTarget uses SERIAL_CONSOLES to determine the number of available
serial ports.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/targetcontrol.py|  3 ++-
 meta/lib/oeqa/utils/qemurunner.py | 32 +---
 2 files changed, 23 insertions(+), 12 deletions(-)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index 1445e3ecfb..0b2915d5cc 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -143,7 +143,8 @@ class QemuTarget(BaseTarget):
 use_kvm = use_kvm,
 dump_dir = dump_dir,
 dump_host_cmds = d.getVar("testimage_dump_host"),
-logger = logger)
+logger = logger,
+serial_ports = 
len(d.getVar("SERIAL_CONSOLES").split()))
 
 self.target_dumper = TargetDumper(dump_target_cmds, dump_dir, 
self.runner)
 
diff --git a/meta/lib/oeqa/utils/qemurunner.py 
b/meta/lib/oeqa/utils/qemurunner.py
index fe8b77d97a..c025eb09c5 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -31,7 +31,7 @@ re_control_char = re.compile('[%s]' % 
re.escape("".join(control_chars)))
 class QemuRunner:
 
 def __init__(self, machine, rootfs, display, tmpdir, deploy_dir_image, 
logfile, boottime, dump_dir, dump_host_cmds,
- use_kvm, logger, use_slirp=False):
+ use_kvm, logger, use_slirp=False, serial_ports=2):
 
 # Popen object for runqemu
 self.runqemu = None
@@ -55,6 +55,7 @@ class QemuRunner:
 self.thread = None
 self.use_kvm = use_kvm
 self.use_slirp = use_slirp
+self.serial_ports = serial_ports
 self.msg = ''
 
 self.runqemutime = 120
@@ -142,7 +143,8 @@ class QemuRunner:
 
 def launch(self, launch_cmd, get_ip = True, qemuparams = None, 
extra_bootparams = None, env = None):
 try:
-self.threadsock, threadport = self.create_socket()
+if self.serial_ports >= 2:
+self.threadsock, threadport = self.create_socket()
 self.server_socket, self.serverport = self.create_socket()
 except socket.error as msg:
 self.logger.error("Failed to create listening socket: %s" % msg[1])
@@ -160,7 +162,10 @@ class QemuRunner:
 if qemuparams:
 self.qemuparams = self.qemuparams[:-1] + " " + qemuparams + " " + 
'\"'
 
-launch_cmd += ' tcpserial=%s:%s %s' % (threadport, self.serverport, 
self.qemuparams)
+if self.serial_ports >= 2:
+launch_cmd += ' tcpserial=%s:%s %s' % (threadport, 
self.serverport, self.qemuparams)
+else:
+launch_cmd += ' tcpserial=%s %s' % (self.serverport, 
self.qemuparams)
 
 self.origchldhandler = signal.getsignal(signal.SIGCHLD)
 signal.signal(signal.SIGCHLD, self.handleSIGCHLD)
@@ -275,14 +280,15 @@ class QemuRunner:
 self.logger.debug("Target IP: %s" % self.ip)
 self.logger.debug("Server IP: %s" % self.server_ip)
 
-self.thread = LoggingThread(self.log, self.threadsock, self.logger)
-self.thread.start()
-if not self.thread.connection_established.wait(self.boottime):
-self.logger.error("Didn't receive a console connection from qemu. "
- "Here is the qemu command line used:\n%s\nand "
- "output from runqemu:\n%s" % (cmdline, out))
-self.stop_thread()
-return False
+if self.serial_ports >= 2:
+self.thread = LoggingThread(self.log, self.threadsock, self.logger)
+self.thread.start()
+if not self.thread.connection_established.wait(self.boottime):
+self.logger.error("Didn't receive a console connection from 
qemu. "
+ "Here is the qemu command line used:\n%s\nand "
+ "output from runqemu:\n%s" % (cmdline, out))
+self.stop_thread()
+return False
 
 self.logger.debug("Output from runqemu:\n%s", out)
 self.logger.debug("Waiting at most %d seconds for login banner (%s)" %
@@ -310,6 +316,10 @@ class QemuRunner:
 data = data + sock.recv(1024)
 

Re: [OE-core] [PATCH 1/2] tune-riscv: Add support for hard and soft float

[Nathan Rossi]

On Thu, 7 Nov 2019 at 09:34, Adrian Bunk  wrote:
>
> On Wed, Nov 06, 2019 at 10:18:18AM -0800, Alistair Francis wrote:
> >...
> > +TUNE_CCARGS_riscv64 .= "${@bb.utils.contains('TUNE_FEATURES', 'riscv64-f', 
> > ' -mabi=lp64d', ' -mabi=lp64', d)}"
> > +TUNE_CCARGS_riscv32 .= "${@bb.utils.contains('TUNE_FEATURES', 'riscv32-f', 
> > ' -mabi=ilp32f', ' -mabi=ilp32', d)}"
> >...
>
> That looks wrong, what would you put in TUNE_FEATURES
> for -mabi=lp64f when -mabi=lp64d is called riscv64-f?
>
> Also note that this is only the floating point calling convention,
> whether the compiler emits floating point instructions is defined
> by -march.
>
> It would be good if this would start with a plan how to handle
> all possible combination of RISC-V ISA extensions, ideally better
> than the arm mess.

I am keen to see this as well, since I am currently directly
hardcoding -march/-mabi in the machine conf.

It would be nice to see the ISA tunes available in oe-core, even if
that is just the tune features and not predefined tunes (e.g. like
microblaze).

Regards,
Nathan

>
> cu
> Adrian
>
> --
>
>"Is there not promise of rain?" Ling Tan asked suddenly out
> of the darkness. There had been need of rain for many days.
>"Only a promise," Lao Er said.
>Pearl S. Buck - Dragon Seed
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] oeqa/core/utils/concurrencytest.py: Handle exceptions and details

[Nathan Rossi]

Handle the streaming of exception content with details data. The
testtools package allows both 'err' and 'details' kwargs but can only
pass one of them to the parent.

To handle the passing of exception traceback and details data at the
same time, encode the traceback into the details object and remove the
'err' arg from the add* result call. This encodes the traceback similar
to how 'err' is handled without any details object. Decoding is already
done by testtools when the traceback is encoded in the details object.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/core/utils/concurrencytest.py | 31 +
 1 file changed, 18 insertions(+), 13 deletions(-)

diff --git a/meta/lib/oeqa/core/utils/concurrencytest.py 
b/meta/lib/oeqa/core/utils/concurrencytest.py
index 6293cf94ec..0f7b3dcc11 100644
--- a/meta/lib/oeqa/core/utils/concurrencytest.py
+++ b/meta/lib/oeqa/core/utils/concurrencytest.py
@@ -78,29 +78,29 @@ class ProxyTestResult:
 def __init__(self, target):
 self.result = target
 
-def _addResult(self, method, test, *args, **kwargs):
+def _addResult(self, method, test, *args, exception = False, **kwargs):
 return method(test, *args, **kwargs)
 
-def addError(self, test, *args, **kwargs):
-self._addResult(self.result.addError, test, *args, **kwargs)
+def addError(self, test, err = None, **kwargs):
+self._addResult(self.result.addError, test, err, exception = True, 
**kwargs)
 
-def addFailure(self, test, *args, **kwargs):
-self._addResult(self.result.addFailure, test, *args, **kwargs)
+def addFailure(self, test, err = None, **kwargs):
+self._addResult(self.result.addFailure, test, err, exception = True, 
**kwargs)
 
-def addSuccess(self, test, *args, **kwargs):
-self._addResult(self.result.addSuccess, test, *args, **kwargs)
+def addSuccess(self, test, **kwargs):
+self._addResult(self.result.addSuccess, test, **kwargs)
 
-def addExpectedFailure(self, test, *args, **kwargs):
-self._addResult(self.result.addExpectedFailure, test, *args, **kwargs)
+def addExpectedFailure(self, test, err = None, **kwargs):
+self._addResult(self.result.addExpectedFailure, test, err, exception = 
True, **kwargs)
 
-def addUnexpectedSuccess(self, test, *args, **kwargs):
-self._addResult(self.result.addUnexpectedSuccess, test, *args, 
**kwargs)
+def addUnexpectedSuccess(self, test, **kwargs):
+self._addResult(self.result.addUnexpectedSuccess, test, **kwargs)
 
 def __getattr__(self, attr):
 return getattr(self.result, attr)
 
 class ExtraResultsDecoderTestResult(ProxyTestResult):
-def _addResult(self, method, test, *args, **kwargs):
+def _addResult(self, method, test, *args, exception = False, **kwargs):
 if "details" in kwargs and "extraresults" in kwargs["details"]:
 if isinstance(kwargs["details"]["extraresults"], Content):
 kwargs = kwargs.copy()
@@ -114,7 +114,7 @@ class ExtraResultsDecoderTestResult(ProxyTestResult):
 return method(test, *args, **kwargs)
 
 class ExtraResultsEncoderTestResult(ProxyTestResult):
-def _addResult(self, method, test, *args, **kwargs):
+def _addResult(self, method, test, *args, exception = False, **kwargs):
 if hasattr(test, "extraresults"):
 extras = lambda : [json.dumps(test.extraresults).encode()]
 kwargs = kwargs.copy()
@@ -123,6 +123,11 @@ class ExtraResultsEncoderTestResult(ProxyTestResult):
 else:
 kwargs["details"] = kwargs["details"].copy()
 kwargs["details"]["extraresults"] = 
Content(ContentType("application", "json", {'charset': 'utf8'}), extras)
+# if using details, need to encode any exceptions into the details obj,
+# testtools does not handle "err" and "details" together.
+if "details" in kwargs and exception and (len(args) >= 1 and args[0] 
is not None):
+kwargs["details"]["traceback"] = 
testtools.content.TracebackContent(args[0], test)
+args = []
 return method(test, *args, **kwargs)
 
 #
---
2.23.0
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] oeqa/core/case.py: Encode binary data of log

[Nathan Rossi]

Do not decode the log content into a string only to re-encode it as
binary data again. Some logs might un-intentionally contain bytes that
do not decode as utf-8, as such preserve the log file content as it was
on disk.

Handle the decoding on the resulttool side, but also handle the failure
to decode the data.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/core/case.py| 4 ++--
 scripts/lib/resulttool/resultutils.py | 6 +-
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/core/case.py b/meta/lib/oeqa/core/case.py
index 180635ac6c..aae451fef2 100644
--- a/meta/lib/oeqa/core/case.py
+++ b/meta/lib/oeqa/core/case.py
@@ -59,7 +59,7 @@ class OEPTestResultTestCase:
 """
 @staticmethod
 def _compress_log(log):
-logdata = log.encode("utf-8")
+logdata = log.encode("utf-8") if isinstance(log, str) else log
 logdata = zlib.compress(logdata)
 logdata = base64.b64encode(logdata).decode("utf-8")
 return {"compressed" : logdata}
@@ -80,7 +80,7 @@ class OEPTestResultTestCase:
 if log is not None:
 sections[section]["log"] = self._compress_log(log)
 elif logfile is not None:
-with open(logfile, "r") as f:
+with open(logfile, "rb") as f:
 sections[section]["log"] = self._compress_log(f.read())
 
 if duration is not None:
diff --git a/scripts/lib/resulttool/resultutils.py 
b/scripts/lib/resulttool/resultutils.py
index 177fb25f93..7cb85a6aa9 100644
--- a/scripts/lib/resulttool/resultutils.py
+++ b/scripts/lib/resulttool/resultutils.py
@@ -126,7 +126,11 @@ def decode_log(logdata):
 if "compressed" in logdata:
 data = logdata.get("compressed")
 data = base64.b64decode(data.encode("utf-8"))
-return zlib.decompress(data).decode("utf-8")
+data = zlib.decompress(data)
+try:
+return data.decode("utf-8")
+except UnicodeDecodeError:
+return data
 return None
 
 def ptestresult_get_log(results, section):
---
2.23.0
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] meson.bbclass: Handle microblaze* mapping to cpu family

[Nathan Rossi]

Signed-off-by: Nathan Rossi 
---
 meta/classes/meson.bbclass | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/classes/meson.bbclass b/meta/classes/meson.bbclass
index 48ca112d80..efa6234078 100644
--- a/meta/classes/meson.bbclass
+++ b/meta/classes/meson.bbclass
@@ -55,6 +55,8 @@ def meson_cpu_family(var, d):
 return 'mips64'
 elif re.match(r"i[3-6]86", arch):
 return "x86"
+elif arch == "microblazeel" or arch == "microblazeeb":
+return "microblaze"
 else:
 return arch
 
---
2.23.0
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] cmake: 3.15.2 -> 3.15.3

[Nathan Rossi]

Update to fix issues with Boost 1.71.0, see pull request:
  https://gitlab.kitware.com/cmake/cmake/merge_requests/3763

Signed-off-by: Nathan Rossi 
---
 .../cmake/{cmake-native_3.15.2.bb => cmake-native_3.15.3.bb}  | 0
 meta/recipes-devtools/cmake/cmake.inc | 4 ++--
 meta/recipes-devtools/cmake/{cmake_3.15.2.bb => cmake_3.15.3.bb}  | 0
 3 files changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-devtools/cmake/{cmake-native_3.15.2.bb => 
cmake-native_3.15.3.bb} (100%)
 rename meta/recipes-devtools/cmake/{cmake_3.15.2.bb => cmake_3.15.3.bb} (100%)

diff --git a/meta/recipes-devtools/cmake/cmake-native_3.15.2.bb 
b/meta/recipes-devtools/cmake/cmake-native_3.15.3.bb
similarity index 100%
rename from meta/recipes-devtools/cmake/cmake-native_3.15.2.bb
rename to meta/recipes-devtools/cmake/cmake-native_3.15.3.bb
diff --git a/meta/recipes-devtools/cmake/cmake.inc 
b/meta/recipes-devtools/cmake/cmake.inc
index 4cbe26ed60..3711f18db6 100644
--- a/meta/recipes-devtools/cmake/cmake.inc
+++ b/meta/recipes-devtools/cmake/cmake.inc
@@ -18,7 +18,7 @@ SRC_URI = 
"https://cmake.org/files/v${CMAKE_MAJOR_VERSION}/cmake-${PV}.tar.gz \
file://0004-Fail-silently-if-system-Qt-installation-is-broken.patch 
\
 "
 
-SRC_URI[md5sum] = "9ecf167edadb87e2d75cc89fded7aadb"
-SRC_URI[sha256sum] = 
"539088cb29a68e6d6a8fba5c00951e5e5b1a92c68fa38a83e1ed2f355933f768"
+SRC_URI[md5sum] = "bf69e837eeda3dd4e9f52cffcf546a11"
+SRC_URI[sha256sum] = 
"13958243a01365b05652fa01b21d40fa834f70a9e30efa69c02604e64f58b8f5"
 
 UPSTREAM_CHECK_REGEX = "cmake-(?P\d+(\.\d+)+)\.tar"
diff --git a/meta/recipes-devtools/cmake/cmake_3.15.2.bb 
b/meta/recipes-devtools/cmake/cmake_3.15.3.bb
similarity index 100%
rename from meta/recipes-devtools/cmake/cmake_3.15.2.bb
rename to meta/recipes-devtools/cmake/cmake_3.15.3.bb
---
2.23.0
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] glibc-testsuite: Depend on virtual/libc-locale instead of glibc-locale

[Nathan Rossi]

On Thu, 12 Sep 2019 at 02:00, Peter Kjellerstedt
 wrote:
>
> This avoids the following error when using some other toolchain than
> the one provided by OE-Core:
>
>   ERROR: Nothing PROVIDES 'glibc-locale' (but
>   .../meta/recipes-core/glibc/glibc-testsuite_2.30.bb DEPENDS on or
>   otherwise requires it)
>   glibc-locale was skipped: PREFERRED_PROVIDER_virtual/libc-locale set
>   to some-other-glibc-locale, not glibc-locale
>
> Signed-off-by: Peter Kjellerstedt 
> ---
>
> I doubt glibc-testsuite will work when another toolchain is used, but
> with this change it does at least not fail the build anymore.

Just thinking about this a bit more, this recipe really only works
correctly if libc == glibc. So this should instead be a skiprecipe
based on PREFRRED_PROVIDER_virtual/libc and _virtual/libc-locale. I
have sent out a patch that does that, please give it a test and let me
know if that does not resolve the errors you are getting.

Thanks,
Nathan

>
>  meta/recipes-core/glibc/glibc-testsuite_2.30.bb | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/recipes-core/glibc/glibc-testsuite_2.30.bb 
> b/meta/recipes-core/glibc/glibc-testsuite_2.30.bb
> index 64fa8d87df..170e865cb0 100644
> --- a/meta/recipes-core/glibc/glibc-testsuite_2.30.bb
> +++ b/meta/recipes-core/glibc/glibc-testsuite_2.30.bb
> @@ -8,7 +8,7 @@ PROVIDES = ""
>  # setup depends
>  INHIBIT_DEFAULT_DEPS = ""
>
> -DEPENDS += "glibc-locale libgcc gcc-runtime"
> +DEPENDS += "virtual/libc-locale libgcc gcc-runtime"
>
>  # remove the initial depends
>  DEPENDS_remove = "libgcc-initial"
> --
> 2.21.0
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] glibc-testsuite: SkipRecipe if libc is not glibc

[Nathan Rossi]

To prevent issues with parsing or dependencies, limit this recipe to use
only when the libc is glibc (and libc-locale is glibc-locale).

Signed-off-by: Nathan Rossi 
---
 meta/recipes-core/glibc/glibc-testsuite_2.30.bb | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/meta/recipes-core/glibc/glibc-testsuite_2.30.bb 
b/meta/recipes-core/glibc/glibc-testsuite_2.30.bb
index 64fa8d87df..657fd4dbc1 100644
--- a/meta/recipes-core/glibc/glibc-testsuite_2.30.bb
+++ b/meta/recipes-core/glibc/glibc-testsuite_2.30.bb
@@ -8,6 +8,13 @@ PROVIDES = ""
 # setup depends
 INHIBIT_DEFAULT_DEPS = ""
 
+python () {
+libc = d.getVar("PREFERRED_PROVIDER_virtual/libc")
+libclocale = d.getVar("PREFERRED_PROVIDER_virtual/libc-locale")
+if libc != "glibc" or libclocale != "glibc-locale":
+raise bb.parse.SkipRecipe("glibc-testsuite requires that virtual/libc 
is glibc")
+}
+
 DEPENDS += "glibc-locale libgcc gcc-runtime"
 
 # remove the initial depends
---
2.23.0
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 4/4] oeqa/selftest: Rework toolchain tests to use OEPTestResultTestCase

[Nathan Rossi]

Use OEPTestResultTestCase to collect results and add logfile collection
and compression.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- Fixed up missing logpath setting for gcc.py.
---
 meta/lib/oeqa/selftest/cases/binutils.py | 11 ++-
 meta/lib/oeqa/selftest/cases/gcc.py  |  9 +
 meta/lib/oeqa/selftest/cases/glibc.py|  7 ---
 3 files changed, 15 insertions(+), 12 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/binutils.py 
b/meta/lib/oeqa/selftest/cases/binutils.py
index 9bc752040f..821f52f5a8 100644
--- a/meta/lib/oeqa/selftest/cases/binutils.py
+++ b/meta/lib/oeqa/selftest/cases/binutils.py
@@ -4,6 +4,7 @@ import sys
 import re
 import logging
 from oeqa.core.decorator import OETestTag
+from oeqa.core.case import OEPTestResultTestCase
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars
 
@@ -15,7 +16,7 @@ def parse_values(content):
 break
 
 @OETestTag("toolchain-user", "toolchain-system")
-class BinutilsCrossSelfTest(OESelftestTestCase):
+class BinutilsCrossSelfTest(OESelftestTestCase, OEPTestResultTestCase):
 def test_binutils(self):
 self.run_binutils("binutils")
 
@@ -36,14 +37,14 @@ class BinutilsCrossSelfTest(OESelftestTestCase):
 
 bitbake("{0} -c check".format(recipe))
 
-ptestsuite = "binutils-{}".format(suite) if suite != "binutils" else 
suite
-self.extraresults = {"ptestresult.sections" : {ptestsuite : {}}}
-
 sumspath = os.path.join(builddir, suite, "{0}.sum".format(suite))
 if not os.path.exists(sumspath):
 sumspath = os.path.join(builddir, suite, "testsuite", 
"{0}.sum".format(suite))
+logpath = os.path.splitext(sumspath)[0] + ".log"
 
+ptestsuite = "binutils-{}".format(suite) if suite != "binutils" else 
suite
+self.ptest_section(ptestsuite, logfile = logpath)
 with open(sumspath, "r") as f:
 for test, result in parse_values(f):
-self.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
+self.ptest_result(ptestsuite, test, result)
 
diff --git a/meta/lib/oeqa/selftest/cases/gcc.py 
b/meta/lib/oeqa/selftest/cases/gcc.py
index 2c25b5904c..5a917b9c44 100644
--- a/meta/lib/oeqa/selftest/cases/gcc.py
+++ b/meta/lib/oeqa/selftest/cases/gcc.py
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: MIT
 import os
 from oeqa.core.decorator import OETestTag
+from oeqa.core.case import OEPTestResultTestCase
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, 
Command
 
@@ -11,7 +12,7 @@ def parse_values(content):
 yield i[len(v) + 2:].strip(), v
 break
 
-class GccSelfTestBase(OESelftestTestCase):
+class GccSelfTestBase(OESelftestTestCase, OEPTestResultTestCase):
 def check_skip(self, suite):
 targets = get_bb_var("RUNTIMETARGET", "gcc-runtime").split()
 if suite not in targets:
@@ -41,20 +42,20 @@ class GccSelfTestBase(OESelftestTestCase):
 bb_vars = get_bb_vars(["B", "TARGET_SYS"], recipe)
 builddir, target_sys = bb_vars["B"], bb_vars["TARGET_SYS"]
 
-self.extraresults = {"ptestresult.sections" : {}}
 for suite in suites:
 sumspath = os.path.join(builddir, "gcc", "testsuite", suite, 
"{0}.sum".format(suite))
 if not os.path.exists(sumspath): # check in target dirs
 sumspath = os.path.join(builddir, target_sys, suite, 
"testsuite", "{0}.sum".format(suite))
 if not os.path.exists(sumspath): # handle libstdc++-v3 -> libstdc++
 sumspath = os.path.join(builddir, target_sys, suite, 
"testsuite", "{0}.sum".format(suite.split("-")[0]))
+logpath = os.path.splitext(sumspath)[0] + ".log"
 
 ptestsuite = "gcc-{}".format(suite) if suite != "gcc" else suite
 ptestsuite = ptestsuite + "-user" if ssh is None else ptestsuite
-self.extraresults["ptestresult.sections"][ptestsuite] = {}
+self.ptest_section(ptestsuite, logfile = logpath)
 with open(sumspath, "r") as f:
 for test, result in parse_values(f):
-self.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
+self.ptest_result(ptestsuite, test, result)
 
 def run_check_emulated(self, *args, **kwargs):
 # build core-image-minimal with required packages
diff --git a/meta/lib/oeqa/selftest/cases/glibc.py 
b

[OE-core] [PATCH v2 3/4] oeqa/core/case.py: Add OEPTestResultTestCase for ptestresult helpers

[Nathan Rossi]

Add the OEPTestResultTestCase class as a mix-in class to provide helper
functions for interacting with ptestresults within the extraresults
object generated by the test case.

This class also provides default compression of log text and log files.

Also add support to resulttool for decoding/decompressing log files
embedded in the test results.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- Fix up resultutils so that it can decode compressed logs
- Move common log decoding from log to resultutils module, and simplify
  uses of log access
---
 meta/lib/oeqa/core/case.py| 49 +++
 scripts/lib/resulttool/log.py | 43 +++---
 scripts/lib/resulttool/resultutils.py | 45 +++-
 3 files changed, 109 insertions(+), 28 deletions(-)

diff --git a/meta/lib/oeqa/core/case.py b/meta/lib/oeqa/core/case.py
index aca144e9dc..180635ac6c 100644
--- a/meta/lib/oeqa/core/case.py
+++ b/meta/lib/oeqa/core/case.py
@@ -4,6 +4,8 @@
 # SPDX-License-Identifier: MIT
 #
 
+import base64
+import zlib
 import unittest
 
 from oeqa.core.exception import OEQAMissingVariable
@@ -49,3 +51,50 @@ class OETestCase(unittest.TestCase):
 for d in self.decorators:
 d.tearDownDecorator()
 self.tearDownMethod()
+
+class OEPTestResultTestCase:
+"""
+Mix-in class to provide functions to make interacting with extraresults for
+the purposes of storing ptestresult data.
+"""
+@staticmethod
+def _compress_log(log):
+logdata = log.encode("utf-8")
+logdata = zlib.compress(logdata)
+logdata = base64.b64encode(logdata).decode("utf-8")
+return {"compressed" : logdata}
+
+def ptest_rawlog(self, log):
+if not hasattr(self, "extraresults"):
+self.extraresults = {"ptestresult.sections" : {}}
+self.extraresults["ptestresult.rawlogs"] = {"log" : 
self._compress_log(log)}
+
+def ptest_section(self, section, duration = None, log = None, logfile = 
None, exitcode = None):
+if not hasattr(self, "extraresults"):
+self.extraresults = {"ptestresult.sections" : {}}
+
+sections = self.extraresults.get("ptestresult.sections")
+if section not in sections:
+sections[section] = {}
+
+if log is not None:
+sections[section]["log"] = self._compress_log(log)
+elif logfile is not None:
+with open(logfile, "r") as f:
+sections[section]["log"] = self._compress_log(f.read())
+
+if duration is not None:
+sections[section]["duration"] = duration
+if exitcode is not None:
+sections[section]["exitcode"] = exitcode
+
+def ptest_result(self, section, test, result):
+if not hasattr(self, "extraresults"):
+self.extraresults = {"ptestresult.sections" : {}}
+
+sections = self.extraresults.get("ptestresult.sections")
+if section not in sections:
+sections[section] = {}
+resultname = "ptestresult.{}.{}".format(section, test)
+self.extraresults[resultname] = {"status" : result}
+
diff --git a/scripts/lib/resulttool/log.py b/scripts/lib/resulttool/log.py
index 2352c767d9..f1bfd99500 100644
--- a/scripts/lib/resulttool/log.py
+++ b/scripts/lib/resulttool/log.py
@@ -8,12 +8,12 @@ import os
 import resulttool.resultutils as resultutils
 
 def show_ptest(result, ptest, logger):
-if 'ptestresult.sections' in result:
-if ptest in result['ptestresult.sections'] and 'log' in 
result['ptestresult.sections'][ptest]:
-print(result['ptestresult.sections'][ptest]['log'])
-return 0
+logdata = resultutils.ptestresult_get_log(result, ptest)
+if logdata is not None:
+print(logdata)
+return 0
 
-print("ptest '%s' not found" % ptest)
+print("ptest '%s' log not found" % ptest)
 return 1
 
 def show_reproducible(result, reproducible, logger):
@@ -25,7 +25,6 @@ def show_reproducible(result, reproducible, logger):
 print("reproducible '%s' not found" % reproducible)
 return 1
 
-
 def log(args, logger):
 results = resultutils.load_resultsdata(args.source)
 
@@ -35,24 +34,24 @@ def log(args, logger):
 return 1
 
 for _, run_name, _, r in resultutils.test_run_results(results):
-if args.dump_ptest:
-if 'ptestresult.sections' in r:
-for name, ptest in r['ptestresult.sections'].items():
-if 'log' in ptest:
-dest_dir = args.dump_ptest
-   

[OE-core] [PATCH v2 2/4] gcc-cross.inc: Process binaries in build dir to be relocatable

[Nathan Rossi]

Process binaries within the build directory before stashing to be
relocatable with ORIGIN relative rpaths. This corrects issues with
rpaths being invalid when trying to use the binaries from an unstashed
build directory (e.g. gcc-runtime).

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- Nest the stash directory within 'stashed-builddir' to match the
  nesting of the copied build directory, this ensures that the relative
  rpaths are correct when extracting to a similar build directory.
---
 meta/recipes-devtools/gcc/gcc-cross.inc | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc 
b/meta/recipes-devtools/gcc/gcc-cross.inc
index 95af6d89a9..8855bb1f34 100644
--- a/meta/recipes-devtools/gcc/gcc-cross.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross.inc
@@ -206,9 +206,17 @@ do_package_write_ipk[noexec] = "1"
 do_package_write_rpm[noexec] = "1"
 do_package_write_deb[noexec] = "1"
 
-BUILDDIRSTASH = "${WORKDIR}/stashed-builddir"
+inherit chrpath
+
+python gcc_stash_builddir_fixrpaths() {
+# rewrite rpaths, breaking hardlinks as required
+process_dir("/", d.getVar("BUILDDIRSTASH"), d, break_hardlinks = True)
+}
+
+BUILDDIRSTASH = "${WORKDIR}/stashed-builddir/build"
 do_gcc_stash_builddir[dirs] = "${B}"
 do_gcc_stash_builddir[cleandirs] = "${BUILDDIRSTASH}"
+do_gcc_stash_builddir[postfuncs] += "gcc_stash_builddir_fixrpaths"
 do_gcc_stash_builddir () {
dest=${BUILDDIRSTASH}
hardlinkdir . $dest
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 1/4] resulttool: Handle multiple series containing ptestresults

[Nathan Rossi]

Handle multiple results series having ptestresults content. The contents
are merged on a per-result basis where duplicates are ignored (with a
warning message printed). The 'ptestresults.sections' collection is also
merged on a per-suite basis.

Signed-off-by: Nathan Rossi 
---
 scripts/lib/resulttool/report.py | 44 +---
 1 file changed, 32 insertions(+), 12 deletions(-)

diff --git a/scripts/lib/resulttool/report.py b/scripts/lib/resulttool/report.py
index 8b03717d29..883b52517b 100644
--- a/scripts/lib/resulttool/report.py
+++ b/scripts/lib/resulttool/report.py
@@ -32,16 +32,22 @@ class ResultsTextReport(object):
 # Ensure tests without any test results still show up on the report
 for suite in result['ptestresult.sections']:
 if suite not in self.ptests[machine]:
-self.ptests[machine][suite] = {'passed': 0, 'failed': 0, 
'skipped': 0, 'duration' : '-', 'failed_testcases': []}
+self.ptests[machine][suite] = {
+'passed': 0, 'failed': 0, 'skipped': 0, 'duration' 
: '-',
+'failed_testcases': [], "testcases": set(),
+}
 if 'duration' in result['ptestresult.sections'][suite]:
 self.ptests[machine][suite]['duration'] = 
result['ptestresult.sections'][suite]['duration']
 if 'timeout' in result['ptestresult.sections'][suite]:
 self.ptests[machine][suite]['duration'] += " T"
-return
+return True
+
+# process test result
 try:
 _, suite, test = k.split(".", 2)
 except ValueError:
-return
+return True
+
 # Handle 'glib-2.0'
 if 'ptestresult.sections' in result and suite not in 
result['ptestresult.sections']:
 try:
@@ -50,11 +56,23 @@ class ResultsTextReport(object):
 suite = suite + "." + suite1
 except ValueError:
 pass
+
 if suite not in self.ptests[machine]:
-self.ptests[machine][suite] = {'passed': 0, 'failed': 0, 
'skipped': 0, 'duration' : '-', 'failed_testcases': []}
+self.ptests[machine][suite] = {
+'passed': 0, 'failed': 0, 'skipped': 0, 'duration' : '-',
+'failed_testcases': [], "testcases": set(),
+}
+
+# do not process duplicate results
+if test in self.ptests[machine][suite]["testcases"]:
+print("Warning duplicate ptest result '{}.{}' for 
{}".format(suite, test, machine))
+return False
+
 for tk in self.result_types:
 if status in self.result_types[tk]:
 self.ptests[machine][suite][tk] += 1
+self.ptests[machine][suite]["testcases"].add(test)
+return True
 
 def handle_ltptest_result(self, k, status, result, machine):
 if machine not in self.ltptests:
@@ -124,17 +142,20 @@ class ResultsTextReport(object):
 result = testresult.get('result', [])
 for k in result:
 test_status = result[k].get('status', [])
+if k.startswith("ptestresult."):
+if not self.handle_ptest_result(k, test_status, result, 
machine):
+continue
+elif k.startswith("ltpresult."):
+self.handle_ltptest_result(k, test_status, result, machine)
+elif k.startswith("ltpposixresult."):
+self.handle_ltpposixtest_result(k, test_status, result, 
machine)
+
+# process result if it was not skipped by a handler
 for tk in self.result_types:
 if test_status in self.result_types[tk]:
 test_count_report[tk] += 1
 if test_status in self.result_types['failed']:
 test_count_report['failed_testcases'].append(k)
-if k.startswith("ptestresult."):
-self.handle_ptest_result(k, test_status, result, machine)
-if k.startswith("ltpresult."):
-self.handle_ltptest_result(k, test_status, result, machine)
-if k.startswith("ltpposixresult."):
-self.handle_ltpposixtest_result(k, test_status, result, 
machine)
 return test_count_report
 
 def print_test_report(self, template_file_name, test_cou

[OE-core] [PATCH 4/6] gcc-cross.inc: Process binaries in build dir to be relocatable

[Nathan Rossi]

Process binaries within the build directory before stashing to be
relocatable with ORIGIN relative rpaths. This corrects issues with
rpaths being invalid when trying to use the binaries from an unstashed
build directory (e.g. gcc-runtime).

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/gcc/gcc-cross.inc | 8 
 1 file changed, 8 insertions(+)

diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc 
b/meta/recipes-devtools/gcc/gcc-cross.inc
index 95af6d89a9..e20f6cb6c1 100644
--- a/meta/recipes-devtools/gcc/gcc-cross.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross.inc
@@ -206,9 +206,17 @@ do_package_write_ipk[noexec] = "1"
 do_package_write_rpm[noexec] = "1"
 do_package_write_deb[noexec] = "1"
 
+inherit chrpath
+
+python gcc_stash_builddir_fixrpaths() {
+# rewrite rpaths, breaking hardlinks as required
+process_dir("/", d.getVar("BUILDDIRSTASH"), d, break_hardlinks = True)
+}
+
 BUILDDIRSTASH = "${WORKDIR}/stashed-builddir"
 do_gcc_stash_builddir[dirs] = "${B}"
 do_gcc_stash_builddir[cleandirs] = "${BUILDDIRSTASH}"
+do_gcc_stash_builddir[postfuncs] += "gcc_stash_builddir_fixrpaths"
 do_gcc_stash_builddir () {
dest=${BUILDDIRSTASH}
hardlinkdir . $dest
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/6] chrpath.bbclass: Add break_hardlinks kwarg to allow breaking hardlinks

[Nathan Rossi]

Add the break_hardlinks kwarg to break hardlinks when modifying files.
This uses the bb.utils.break_hardlinks function to break hardlinks. The
default is to maintain existing behaviour and leave hardlinks in place.

Signed-off-by: Nathan Rossi 
---
 meta/classes/chrpath.bbclass | 19 +--
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/meta/classes/chrpath.bbclass b/meta/classes/chrpath.bbclass
index ad3c3975a5..2870c10d51 100644
--- a/meta/classes/chrpath.bbclass
+++ b/meta/classes/chrpath.bbclass
@@ -1,7 +1,7 @@
 CHRPATH_BIN ?= "chrpath"
 PREPROCESS_RELOCATE_DIRS ?= ""
 
-def process_file_linux(cmd, fpath, rootdir, baseprefix, tmpdir, d):
+def process_file_linux(cmd, fpath, rootdir, baseprefix, tmpdir, d, 
break_hardlinks = False):
 import subprocess as sub
 
 p = sub.Popen([cmd, '-l', fpath],stdout=sub.PIPE,stderr=sub.PIPE)
@@ -39,6 +39,9 @@ def process_file_linux(cmd, fpath, rootdir, baseprefix, 
tmpdir, d):
 
 # if we have modified some rpaths call chrpath to update the binary
 if modified:
+if break_hardlinks:
+bb.utils.break_hardlinks(fpath)
+
 args = ":".join(new_rpaths)
 #bb.note("Setting rpath for %s to %s" %(fpath, args))
 p = sub.Popen([cmd, '-r', args, fpath],stdout=sub.PIPE,stderr=sub.PIPE)
@@ -46,7 +49,7 @@ def process_file_linux(cmd, fpath, rootdir, baseprefix, 
tmpdir, d):
 if p.returncode != 0:
 bb.fatal("%s: chrpath command failed with exit code %d:\n%s%s" % 
(d.getVar('PN'), p.returncode, out, err))
 
-def process_file_darwin(cmd, fpath, rootdir, baseprefix, tmpdir, d):
+def process_file_darwin(cmd, fpath, rootdir, baseprefix, tmpdir, d, 
break_hardlinks = False):
 import subprocess as sub
 
 p = sub.Popen([d.expand("${HOST_PREFIX}otool"), '-L', 
fpath],stdout=sub.PIPE,stderr=sub.PIPE)
@@ -61,11 +64,14 @@ def process_file_darwin(cmd, fpath, rootdir, baseprefix, 
tmpdir, d):
 if baseprefix not in rpath:
 continue
 
+if break_hardlinks:
+bb.utils.break_hardlinks(fpath)
+
 newpath = "@loader_path/" + os.path.relpath(rpath, 
os.path.dirname(fpath.replace(rootdir, "/")))
 p = sub.Popen([d.expand("${HOST_PREFIX}install_name_tool"), '-change', 
rpath, newpath, fpath],stdout=sub.PIPE,stderr=sub.PIPE)
 out, err = p.communicate()
 
-def process_dir (rootdir, directory, d):
+def process_dir(rootdir, directory, d, break_hardlinks = False):
 import stat
 
 rootdir = os.path.normpath(rootdir)
@@ -95,7 +101,7 @@ def process_dir (rootdir, directory, d):
 continue
 
 if os.path.isdir(fpath):
-process_dir(rootdir, fpath, d)
+process_dir(rootdir, fpath, d, break_hardlinks = break_hardlinks)
 else:
 #bb.note("Testing %s for relocatability" % fpath)
 
@@ -108,8 +114,9 @@ def process_dir (rootdir, directory, d):
 else:
 # Temporarily make the file writeable so we can chrpath it
 os.chmod(fpath, perms|stat.S_IRWXU)
-process_file(cmd, fpath, rootdir, baseprefix, tmpdir, d)
-
+
+process_file(cmd, fpath, rootdir, baseprefix, tmpdir, d, 
break_hardlinks = break_hardlinks)
+
 if perms:
 os.chmod(fpath, perms)
 
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 6/6] oeqa/selftest: Rework toolchain tests to use OEPTestResultTestCase

[Nathan Rossi]

Use OEPTestResultTestCase to collect results and add logfile collection
and compression.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/binutils.py | 11 ++-
 meta/lib/oeqa/selftest/cases/gcc.py  |  8 
 meta/lib/oeqa/selftest/cases/glibc.py|  7 ---
 3 files changed, 14 insertions(+), 12 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/binutils.py 
b/meta/lib/oeqa/selftest/cases/binutils.py
index 9bc752040f..821f52f5a8 100644
--- a/meta/lib/oeqa/selftest/cases/binutils.py
+++ b/meta/lib/oeqa/selftest/cases/binutils.py
@@ -4,6 +4,7 @@ import sys
 import re
 import logging
 from oeqa.core.decorator import OETestTag
+from oeqa.core.case import OEPTestResultTestCase
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars
 
@@ -15,7 +16,7 @@ def parse_values(content):
 break
 
 @OETestTag("toolchain-user", "toolchain-system")
-class BinutilsCrossSelfTest(OESelftestTestCase):
+class BinutilsCrossSelfTest(OESelftestTestCase, OEPTestResultTestCase):
 def test_binutils(self):
 self.run_binutils("binutils")
 
@@ -36,14 +37,14 @@ class BinutilsCrossSelfTest(OESelftestTestCase):
 
 bitbake("{0} -c check".format(recipe))
 
-ptestsuite = "binutils-{}".format(suite) if suite != "binutils" else 
suite
-self.extraresults = {"ptestresult.sections" : {ptestsuite : {}}}
-
 sumspath = os.path.join(builddir, suite, "{0}.sum".format(suite))
 if not os.path.exists(sumspath):
 sumspath = os.path.join(builddir, suite, "testsuite", 
"{0}.sum".format(suite))
+logpath = os.path.splitext(sumspath)[0] + ".log"
 
+ptestsuite = "binutils-{}".format(suite) if suite != "binutils" else 
suite
+self.ptest_section(ptestsuite, logfile = logpath)
 with open(sumspath, "r") as f:
 for test, result in parse_values(f):
-self.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
+self.ptest_result(ptestsuite, test, result)
 
diff --git a/meta/lib/oeqa/selftest/cases/gcc.py 
b/meta/lib/oeqa/selftest/cases/gcc.py
index 2c25b5904c..6daa8d9e15 100644
--- a/meta/lib/oeqa/selftest/cases/gcc.py
+++ b/meta/lib/oeqa/selftest/cases/gcc.py
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: MIT
 import os
 from oeqa.core.decorator import OETestTag
+from oeqa.core.case import OEPTestResultTestCase
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, 
Command
 
@@ -11,7 +12,7 @@ def parse_values(content):
 yield i[len(v) + 2:].strip(), v
 break
 
-class GccSelfTestBase(OESelftestTestCase):
+class GccSelfTestBase(OESelftestTestCase, OEPTestResultTestCase):
 def check_skip(self, suite):
 targets = get_bb_var("RUNTIMETARGET", "gcc-runtime").split()
 if suite not in targets:
@@ -41,7 +42,6 @@ class GccSelfTestBase(OESelftestTestCase):
 bb_vars = get_bb_vars(["B", "TARGET_SYS"], recipe)
 builddir, target_sys = bb_vars["B"], bb_vars["TARGET_SYS"]
 
-self.extraresults = {"ptestresult.sections" : {}}
 for suite in suites:
 sumspath = os.path.join(builddir, "gcc", "testsuite", suite, 
"{0}.sum".format(suite))
 if not os.path.exists(sumspath): # check in target dirs
@@ -51,10 +51,10 @@ class GccSelfTestBase(OESelftestTestCase):
 
 ptestsuite = "gcc-{}".format(suite) if suite != "gcc" else suite
 ptestsuite = ptestsuite + "-user" if ssh is None else ptestsuite
-self.extraresults["ptestresult.sections"][ptestsuite] = {}
+self.ptest_section(ptestsuite, logfile = logpath)
 with open(sumspath, "r") as f:
 for test, result in parse_values(f):
-self.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
+self.ptest_result(ptestsuite, test, result)
 
 def run_check_emulated(self, *args, **kwargs):
 # build core-image-minimal with required packages
diff --git a/meta/lib/oeqa/selftest/cases/glibc.py 
b/meta/lib/oeqa/selftest/cases/glibc.py
index 2e42485dd6..c687f6ef93 100644
--- a/meta/lib/oeqa/selftest/cases/glibc.py
+++ b/meta/lib/oeqa/selftest/cases/glibc.py
@@ -2,6 +2,7 @@
 import os
 import contextlib
 from oeqa.core.decorator import OETestTag
+from oeqa.core.case import OEPTestResultTestCase
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, 
Command
 from oeqa.ut

[OE-core] [PATCH 5/6] oeqa/core/case.py: Add OEPTestResultTestCase for ptestresult helpers

[Nathan Rossi]

Add the OEPTestResultTestCase class as a mix-in class to provide helper
functions for interacting with ptestresults within the extraresults
object generated by the test case.

This class also provides default compression of log text and log files.

Also add support to resulttool for decoding/decompressing log files
embedded in the test results.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/core/case.py| 49 +++
 scripts/lib/resulttool/log.py | 12 ++-
 2 files changed, 60 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/core/case.py b/meta/lib/oeqa/core/case.py
index aca144e9dc..180635ac6c 100644
--- a/meta/lib/oeqa/core/case.py
+++ b/meta/lib/oeqa/core/case.py
@@ -4,6 +4,8 @@
 # SPDX-License-Identifier: MIT
 #
 
+import base64
+import zlib
 import unittest
 
 from oeqa.core.exception import OEQAMissingVariable
@@ -49,3 +51,50 @@ class OETestCase(unittest.TestCase):
 for d in self.decorators:
 d.tearDownDecorator()
 self.tearDownMethod()
+
+class OEPTestResultTestCase:
+"""
+Mix-in class to provide functions to make interacting with extraresults for
+the purposes of storing ptestresult data.
+"""
+@staticmethod
+def _compress_log(log):
+logdata = log.encode("utf-8")
+logdata = zlib.compress(logdata)
+logdata = base64.b64encode(logdata).decode("utf-8")
+return {"compressed" : logdata}
+
+def ptest_rawlog(self, log):
+if not hasattr(self, "extraresults"):
+self.extraresults = {"ptestresult.sections" : {}}
+self.extraresults["ptestresult.rawlogs"] = {"log" : 
self._compress_log(log)}
+
+def ptest_section(self, section, duration = None, log = None, logfile = 
None, exitcode = None):
+if not hasattr(self, "extraresults"):
+self.extraresults = {"ptestresult.sections" : {}}
+
+sections = self.extraresults.get("ptestresult.sections")
+if section not in sections:
+sections[section] = {}
+
+if log is not None:
+sections[section]["log"] = self._compress_log(log)
+elif logfile is not None:
+with open(logfile, "r") as f:
+sections[section]["log"] = self._compress_log(f.read())
+
+if duration is not None:
+sections[section]["duration"] = duration
+if exitcode is not None:
+sections[section]["exitcode"] = exitcode
+
+def ptest_result(self, section, test, result):
+if not hasattr(self, "extraresults"):
+self.extraresults = {"ptestresult.sections" : {}}
+
+sections = self.extraresults.get("ptestresult.sections")
+if section not in sections:
+sections[section] = {}
+resultname = "ptestresult.{}.{}".format(section, test)
+self.extraresults[resultname] = {"status" : result}
+
diff --git a/scripts/lib/resulttool/log.py b/scripts/lib/resulttool/log.py
index 2352c767d9..99663772a3 100644
--- a/scripts/lib/resulttool/log.py
+++ b/scripts/lib/resulttool/log.py
@@ -5,6 +5,8 @@
 # SPDX-License-Identifier: GPL-2.0-only
 #
 import os
+import base64
+import zlib
 import resulttool.resultutils as resultutils
 
 def show_ptest(result, ptest, logger):
@@ -25,6 +27,9 @@ def show_reproducible(result, reproducible, logger):
 print("reproducible '%s' not found" % reproducible)
 return 1
 
+def decode_compressed_log(logdata):
+data = base64.b64decode(logdata.encode("utf-8"))
+return zlib.decompress(data).decode("utf-8")
 
 def log(args, logger):
 results = resultutils.load_resultsdata(args.source)
@@ -39,6 +44,7 @@ def log(args, logger):
 if 'ptestresult.sections' in r:
 for name, ptest in r['ptestresult.sections'].items():
 if 'log' in ptest:
+logdata = ptest['log']
 dest_dir = args.dump_ptest
 if args.prepend_run:
 dest_dir = os.path.join(dest_dir, run_name)
@@ -48,7 +54,11 @@ def log(args, logger):
 dest = os.path.join(dest_dir, '%s.log' % name)
 print(dest)
 with open(dest, 'w') as f:
-f.write(ptest['log'])
+if isinstance(logdata, str):
+f.write(logdata)
+elif isinstance(logdata, dict):
+if "compressed" in logdata:
+
f.write(decode_compressed_log(logdata.get("compressed")))
 
 if args.raw_ptest:
 if 'ptestresult.rawlogs' in r:
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/6] oeqa/selftest/context.py: For -t/-T use append argparse action

[Nathan Rossi]

Use the 'append' action of argparse instead of nargs. This changes the
behaviour of the option from "-t foo bar -r" to "-t foo -t bar -r".

Additionally rename the long form options to be consistent with
behaviour, such that they specifying a single tag at a time.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/context.py | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/meta/lib/oeqa/selftest/context.py 
b/meta/lib/oeqa/selftest/context.py
index 3126ada716..c4eb5d614e 100644
--- a/meta/lib/oeqa/selftest/context.py
+++ b/meta/lib/oeqa/selftest/context.py
@@ -78,12 +78,12 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
 parser.add_argument('--machine', required=False, choices=['random', 
'all'],
 help='Run tests on different machines 
(random/all).')
 
-parser.add_argument('-t', '--select-tags', dest="select_tags",
-nargs='*', default=None,
-help='Filter all (unhidden) tests to any that match any of the 
specified tags.')
-parser.add_argument('-T', '--exclude-tags', dest="exclude_tags",
-nargs='*', default=None,
-help='Exclude all (unhidden) tests that match any of the 
specified tags. (exclude applies before select)')
+parser.add_argument('-t', '--select-tag', dest="select_tags",
+action='append', default=None,
+help='Filter all (unhidden) tests to any that match any of the 
specified tag(s).')
+parser.add_argument('-T', '--exclude-tag', dest="exclude_tags",
+action='append', default=None,
+help='Exclude all (unhidden) tests that match any of the 
specified tag(s). (exclude applies before select)')
 
 parser.set_defaults(func=self.run)
 
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/6] resulttool: Handle multiple series containing ptestresults

[Nathan Rossi]

Handle multiple results series having ptestresults content. The contents
are merged on a per-result basis where duplicates are ignored (with a
warning message printed). The 'ptestresults.sections' collection is also
merged on a per-suite basis.

Signed-off-by: Nathan Rossi 
---
 scripts/lib/resulttool/report.py | 44 +---
 1 file changed, 32 insertions(+), 12 deletions(-)

diff --git a/scripts/lib/resulttool/report.py b/scripts/lib/resulttool/report.py
index 8b03717d29..883b52517b 100644
--- a/scripts/lib/resulttool/report.py
+++ b/scripts/lib/resulttool/report.py
@@ -32,16 +32,22 @@ class ResultsTextReport(object):
 # Ensure tests without any test results still show up on the report
 for suite in result['ptestresult.sections']:
 if suite not in self.ptests[machine]:
-self.ptests[machine][suite] = {'passed': 0, 'failed': 0, 
'skipped': 0, 'duration' : '-', 'failed_testcases': []}
+self.ptests[machine][suite] = {
+'passed': 0, 'failed': 0, 'skipped': 0, 'duration' 
: '-',
+'failed_testcases': [], "testcases": set(),
+}
 if 'duration' in result['ptestresult.sections'][suite]:
 self.ptests[machine][suite]['duration'] = 
result['ptestresult.sections'][suite]['duration']
 if 'timeout' in result['ptestresult.sections'][suite]:
 self.ptests[machine][suite]['duration'] += " T"
-return
+return True
+
+# process test result
 try:
 _, suite, test = k.split(".", 2)
 except ValueError:
-return
+return True
+
 # Handle 'glib-2.0'
 if 'ptestresult.sections' in result and suite not in 
result['ptestresult.sections']:
 try:
@@ -50,11 +56,23 @@ class ResultsTextReport(object):
 suite = suite + "." + suite1
 except ValueError:
 pass
+
 if suite not in self.ptests[machine]:
-self.ptests[machine][suite] = {'passed': 0, 'failed': 0, 
'skipped': 0, 'duration' : '-', 'failed_testcases': []}
+self.ptests[machine][suite] = {
+'passed': 0, 'failed': 0, 'skipped': 0, 'duration' : '-',
+'failed_testcases': [], "testcases": set(),
+}
+
+# do not process duplicate results
+if test in self.ptests[machine][suite]["testcases"]:
+print("Warning duplicate ptest result '{}.{}' for 
{}".format(suite, test, machine))
+return False
+
 for tk in self.result_types:
 if status in self.result_types[tk]:
 self.ptests[machine][suite][tk] += 1
+self.ptests[machine][suite]["testcases"].add(test)
+return True
 
 def handle_ltptest_result(self, k, status, result, machine):
 if machine not in self.ltptests:
@@ -124,17 +142,20 @@ class ResultsTextReport(object):
 result = testresult.get('result', [])
 for k in result:
 test_status = result[k].get('status', [])
+if k.startswith("ptestresult."):
+if not self.handle_ptest_result(k, test_status, result, 
machine):
+continue
+elif k.startswith("ltpresult."):
+self.handle_ltptest_result(k, test_status, result, machine)
+elif k.startswith("ltpposixresult."):
+self.handle_ltpposixtest_result(k, test_status, result, 
machine)
+
+# process result if it was not skipped by a handler
 for tk in self.result_types:
 if test_status in self.result_types[tk]:
 test_count_report[tk] += 1
 if test_status in self.result_types['failed']:
 test_count_report['failed_testcases'].append(k)
-if k.startswith("ptestresult."):
-self.handle_ptest_result(k, test_status, result, machine)
-if k.startswith("ltpresult."):
-self.handle_ltptest_result(k, test_status, result, machine)
-if k.startswith("ltpposixresult."):
-self.handle_ltpposixtest_result(k, test_status, result, 
machine)
 return test_count_report
 
 def print_test_report(self, template_file_name, test_cou

[OE-core] [PATCH 7/7] oeqa/selftest/cases/glibc.py: Rework and tag with toolchain-user/system

[Nathan Rossi]

Rework the glibc execution into a common base class. Additionally tag
the tests with "toolchain-user" and "toolchain-system".

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/glibc.py | 44 +++
 1 file changed, 24 insertions(+), 20 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/glibc.py 
b/meta/lib/oeqa/selftest/cases/glibc.py
index e13de87014..ea0f79e144 100644
--- a/meta/lib/oeqa/selftest/cases/glibc.py
+++ b/meta/lib/oeqa/selftest/cases/glibc.py
@@ -13,12 +13,8 @@ def parse_values(content):
 yield i[len(v) + 2:].strip(), v
 break
 
-@OETestTag("machine")
-class GlibcSelfTest(OESelftestTestCase):
-def test_glibc(self):
-self.glibc_run_check()
-
-def glibc_run_check(self, ssh = None):
+class GlibcSelfTestBase(OESelftestTestCase):
+def run_check(self, ssh = None):
 # configure ssh target
 features = []
 if ssh is not None:
@@ -40,25 +36,23 @@ class GlibcSelfTest(OESelftestTestCase):
 for test, result in parse_values(f):
 self.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
 
-class GlibcSelfTestSystemEmulated(GlibcSelfTest):
-default_installed_packages = [
-"glibc-charmaps",
-"libgcc",
-"libstdc++",
-"libatomic",
-"libgomp",
-# "python3",
-# "python3-pexpect",
-"nfs-utils",
-]
-
-def glibc_run_check(self):
+def run_check_emulated(self):
 with contextlib.ExitStack() as s:
 # use the base work dir, as the nfs mount, since the recipe 
directory may not exist
 tmpdir = get_bb_var("BASE_WORKDIR")
 nfsport, mountport = s.enter_context(unfs_server(tmpdir))
 
 # build core-image-minimal with required packages
+default_installed_packages = [
+"glibc-charmaps",
+"libgcc",
+"libstdc++",
+"libatomic",
+"libgomp",
+# "python3",
+# "python3-pexpect",
+"nfs-utils",
+]
 features = []
 features.append('IMAGE_FEATURES += "ssh-server-openssh"')
 features.append('CORE_IMAGE_EXTRA_INSTALL += "{0}"'.format(" 
".join(self.default_installed_packages)))
@@ -80,5 +74,15 @@ class GlibcSelfTestSystemEmulated(GlibcSelfTest):
 if status != 0:
 raise Exception("Failed to setup NFS mount on target 
({})".format(repr(output)))
 
-super().glibc_run_check(ssh = qemu.ip)
+self.run_check(ssh = qemu.ip)
+
+@OETestTag("machine", "toolchain-user")
+class GlibcSelfTest(GlibcSelfTestBase):
+def test_glibc(self):
+self.run_check()
+
+@OETestTag("machine", "toolchain-system")
+class GlibcSelfTestSystemEmulated(GlibcSelfTestBase):
+def test_glibc(self):
+self.run_check_emulated()
 
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/7] oeqa/selftest/context.py: Change -t/-T args to be optional

[Nathan Rossi]

Change the -t/-T args to be optional so that they can be used together
with the existing -r/-a/... args to run a more flexible filtering of
test tags.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/context.py | 30 +++---
 1 file changed, 19 insertions(+), 11 deletions(-)

diff --git a/meta/lib/oeqa/selftest/context.py 
b/meta/lib/oeqa/selftest/context.py
index 47de08e3f2..3126ada716 100644
--- a/meta/lib/oeqa/selftest/context.py
+++ b/meta/lib/oeqa/selftest/context.py
@@ -61,12 +61,6 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
 group.add_argument('-r', '--run-tests', required=False, action='store',
 nargs='+', dest="run_tests", default=None,
 help='Select what tests to run (modules, classes or test 
methods). Format should be: ..')
-group.add_argument('-t', '--run-only-tags', action='store',
-nargs='+', dest="run_only_tags", default=None,
-help='Run all (unhidden) tests which match any of the 
specified tags.')
-group.add_argument('-T', '--run-exclude-tags', action='store',
-nargs='+', dest="run_exclude_tags", default=None,
-help='Run all (unhidden) tests excluding any that match any of 
the specified tags.')
 
 group.add_argument('-m', '--list-modules', required=False,
 action="store_true", default=False,
@@ -83,7 +77,14 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
 
 parser.add_argument('--machine', required=False, choices=['random', 
'all'],
 help='Run tests on different machines 
(random/all).')
-
+
+parser.add_argument('-t', '--select-tags', dest="select_tags",
+nargs='*', default=None,
+help='Filter all (unhidden) tests to any that match any of the 
specified tags.')
+parser.add_argument('-T', '--exclude-tags', dest="exclude_tags",
+nargs='*', default=None,
+help='Exclude all (unhidden) tests that match any of the 
specified tags. (exclude applies before select)')
+
 parser.set_defaults(func=self.run)
 
 def _get_available_machines(self):
@@ -155,10 +156,17 @@ class 
OESelftestTestContextExecutor(OETestContextExecutor):
 copyfile(self.tc_kwargs['init']['config_paths']['bblayers'], 
 self.tc_kwargs['init']['config_paths']['bblayers_backup'])
 
-if args.run_only_tags:
-self.tc_kwargs['load']['tags_filter'] = lambda tags: not tags or 
not any(tag in args.run_only_tags for tag in tags)
-if args.run_exclude_tags:
-self.tc_kwargs['load']['tags_filter'] = lambda tags: any(tag in 
args.run_exclude_tags for tag in tags)
+def tag_filter(tags):
+if args.exclude_tags:
+if any(tag in args.exclude_tags for tag in tags):
+return True
+if args.select_tags:
+if not tags or not any(tag in args.select_tags for tag in 
tags):
+return True
+return False
+
+if args.select_tags or args.exclude_tags:
+self.tc_kwargs['load']['tags_filter'] = tag_filter
 
 self.tc_kwargs['run']['skips'] = args.skips
 self.tc_kwargs['run']['processes'] = args.processes
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/7] oeqa/core: Implement proper extra result collection and serialization

[Nathan Rossi]

Implement handling of extra result (e.g. ptestresult) collection with
the addition of a "extraresults" extraction function in OETestResult. In
order to be able to serialize and deserialize the extraresults data,
allow OETestResult add* calls to take a details kwarg. The subunit
module can handle cross-process transfer of binary data for the details
kwarg. With a TestResult proxy class to sit inbetween to encode and
decode to and from json.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/core/runner.py| 41 +--
 meta/lib/oeqa/core/utils/concurrencytest.py | 61 -
 2 files changed, 96 insertions(+), 6 deletions(-)

diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py
index 63a8bae93f..f656e1a9c5 100644
--- a/meta/lib/oeqa/core/runner.py
+++ b/meta/lib/oeqa/core/runner.py
@@ -43,6 +43,7 @@ class OETestResult(_TestResult):
 self.starttime = {}
 self.endtime = {}
 self.progressinfo = {}
+self.extraresults = {}
 
 # Inject into tc so that TestDepends decorator can see results
 tc.results = self
@@ -129,19 +130,51 @@ class OETestResult(_TestResult):
 
 return 'UNKNOWN', None
 
-def addSuccess(self, test):
+def extractExtraResults(self, test, details = None):
+extraresults = None
+if details is not None and "extraresults" in details:
+extraresults = details.get("extraresults", {})
+elif hasattr(test, "extraresults"):
+extraresults = test.extraresults
+
+if extraresults is not None:
+for k, v in extraresults.items():
+# handle updating already existing entries (e.g. 
ptestresults.sections)
+if k in self.extraresults:
+self.extraresults[k].update(v)
+else:
+self.extraresults[k] = v
+
+def addError(self, test, *args, details = None):
+self.extractExtraResults(test, details = details)
+return super(OETestResult, self).addError(test, *args)
+
+def addFailure(self, test, *args, details = None):
+self.extractExtraResults(test, details = details)
+return super(OETestResult, self).addFailure(test, *args)
+
+def addSuccess(self, test, details = None):
 #Added so we can keep track of successes too
 self.successes.append((test, None))
-super(OETestResult, self).addSuccess(test)
+self.extractExtraResults(test, details = details)
+return super(OETestResult, self).addSuccess(test)
+
+def addExpectedFailure(self, test, *args, details = None):
+self.extractExtraResults(test, details = details)
+return super(OETestResult, self).addExpectedFailure(test, *args)
+
+def addUnexpectedSuccess(self, test, details = None):
+self.extractExtraResults(test, details = details)
+return super(OETestResult, self).addUnexpectedSuccess(test)
 
 def logDetails(self, json_file_dir=None, configuration=None, 
result_id=None,
 dump_streams=False):
 self.tc.logger.info("RESULTS:")
 
-result = {}
+result = self.extraresults
 logs = {}
 if hasattr(self.tc, "extraresults"):
-result = self.tc.extraresults
+result.update(self.tc.extraresults)
 
 for case_name in self.tc._registry['cases']:
 case = self.tc._registry['cases'][case_name]
diff --git a/meta/lib/oeqa/core/utils/concurrencytest.py 
b/meta/lib/oeqa/core/utils/concurrencytest.py
index 6bf7718863..fa6fa34b0e 100644
--- a/meta/lib/oeqa/core/utils/concurrencytest.py
+++ b/meta/lib/oeqa/core/utils/concurrencytest.py
@@ -21,6 +21,7 @@ import testtools
 import threading
 import time
 import io
+import json
 import subunit
 
 from queue import Queue
@@ -28,6 +29,8 @@ from itertools import cycle
 from subunit import ProtocolTestCase, TestProtocolClient
 from subunit.test_results import AutoTimingTestResultDecorator
 from testtools import ThreadsafeForwardingResult, iterate_tests
+from testtools.content import Content
+from testtools.content_type import ContentType
 from oeqa.utils.commands import get_test_layer
 
 import bb.utils
@@ -70,6 +73,58 @@ class 
BBThreadsafeForwardingResult(ThreadsafeForwardingResult):
 self.semaphore.release()
 super(BBThreadsafeForwardingResult, 
self)._add_result_with_semaphore(method, test, *args, **kwargs)
 
+class ProxyTestResult:
+# a very basic TestResult proxy, in order to modify add* calls
+def __init__(self, target):
+self.result = target
+
+def _addResult(self, method, test, *args, **kwargs):
+return method(test, *args, **kwargs)
+
+def addError(self, test, *args, **kwargs):
+self._addResult(self.result.addError, test, *args, **kwargs)
+
+def addFailure(self, test, *args, **kwargs):
+self._addRes

[OE-core] [PATCH 6/7] oeqa/selftest/cases/gcc.py: Split into classes for parallelism

[Nathan Rossi]

Split the gcc selftest cases into multiple classes one for each test.
This is done in order to make it easy to execute multiple gcc tests in
parallel when using oe-selftest with the '-j' arg.

Additionally tag the user tests with "toolchain-user" and the system
emulation (qemu system) tests with "toolchain-system".

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/gcc.py | 99 ++---
 1 file changed, 70 insertions(+), 29 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/gcc.py 
b/meta/lib/oeqa/selftest/cases/gcc.py
index d0f0ca06e7..6eafa41720 100644
--- a/meta/lib/oeqa/selftest/cases/gcc.py
+++ b/meta/lib/oeqa/selftest/cases/gcc.py
@@ -11,34 +11,13 @@ def parse_values(content):
 yield i[len(v) + 2:].strip(), v
 break
 
-@OETestTag("machine")
-class GccSelfTest(OESelftestTestCase):
-def gcc_runtime_check_skip(self, suite):
+class GccSelfTestBase(OESelftestTestCase):
+def check_skip(self, suite):
 targets = get_bb_var("RUNTIMETARGET", "gcc-runtime").split()
 if suite not in targets:
 self.skipTest("Target does not use {0}".format(suite))
 
-def test_cross_gcc(self):
-self.gcc_run_check("gcc", "g++")
-
-def test_libatomic(self):
-self.gcc_run_check("libatomic")
-
-def test_libgomp(self):
-self.gcc_run_check("libgomp")
-
-def test_libstdcxx(self):
-self.gcc_run_check("libstdc++-v3")
-
-def test_libssp(self):
-self.gcc_runtime_check_skip("libssp")
-self.gcc_run_check("libssp")
-
-def test_libitm(self):
-self.gcc_runtime_check_skip("libitm")
-self.gcc_run_check("libitm")
-
-def gcc_run_check(self, *suites, ssh = None):
+def run_check(self, *suites, ssh = None):
 targets = set()
 for s in suites:
 if s in ["gcc", "g++"]:
@@ -77,11 +56,9 @@ class GccSelfTest(OESelftestTestCase):
 for test, result in parse_values(f):
 self.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
 
-class GccSelfTestSystemEmulated(GccSelfTest):
-default_installed_packages = ["libgcc", "libstdc++", "libatomic", 
"libgomp"]
-
-def gcc_run_check(self, *args, **kwargs):
+def run_check_emulated(self, *args, **kwargs):
 # build core-image-minimal with required packages
+default_installed_packages = ["libgcc", "libstdc++", "libatomic", 
"libgomp"]
 features = []
 features.append('IMAGE_FEATURES += "ssh-server-openssh"')
 features.append('CORE_IMAGE_EXTRA_INSTALL += "{0}"'.format(" 
".join(self.default_installed_packages)))
@@ -94,5 +71,69 @@ class GccSelfTestSystemEmulated(GccSelfTest):
 status, _ = qemu.run("uname")
 self.assertEqual(status, 0)
 
-return super().gcc_run_check(*args, **kwargs, ssh = qemu.ip)
+return self.run_check(*args, **kwargs, ssh = qemu.ip)
+
+@OETestTag("machine", "toolchain-user")
+class GccCrossSelfTest(GccSelfTestBase):
+def test_cross_gcc(self):
+self.run_check("gcc", "g++")
+
+@OETestTag("machine", "toolchain-user")
+class GccLibAtomicSelfTest(GccSelfTestBase):
+def test_libatomic(self):
+self.run_check("libatomic")
+
+@OETestTag("machine", "toolchain-user")
+class GccLibGompSelfTest(GccSelfTestBase):
+def test_libgomp(self):
+self.run_check("libgomp")
+
+@OETestTag("machine", "toolchain-user")
+class GccLibStdCxxSelfTest(GccSelfTestBase):
+def test_libstdcxx(self):
+self.run_check("libstdc++-v3")
+
+@OETestTag("machine", "toolchain-user")
+class GccLibSspSelfTest(GccSelfTestBase):
+def test_libssp(self):
+self.check_skip("libssp")
+self.run_check("libssp")
+
+@OETestTag("machine", "toolchain-user")
+class GccLibItmSelfTest(GccSelfTestBase):
+def test_libitm(self):
+self.check_skip("libitm")
+self.run_check("libitm")
+
+@OETestTag("machine", "toolchain-system")
+class GccCrossSelfTestSystemEmulated(GccSelfTestBase):
+def test_cross_gcc(self):
+self.run_check_emulated("gcc", "g++")
+
+@OETestTag("machine", "toolchain-system")
+class GccLibAtomicSelfTestSystemEmulated(GccSelfTestBase):
+def test_libatomic(self):
+self.run_check_emulated("libatomic")
+
+@OETestTag("machine", "toolcha

[OE-core] [PATCH 5/7] oeqa/core/decorator: Fix super class modifying subclass tags

[Nathan Rossi]

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/core/decorator/__init__.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/core/decorator/__init__.py 
b/meta/lib/oeqa/core/decorator/__init__.py
index 1a5ac40134..1a82518ab6 100644
--- a/meta/lib/oeqa/core/decorator/__init__.py
+++ b/meta/lib/oeqa/core/decorator/__init__.py
@@ -70,7 +70,8 @@ def OETestTag(*tags):
 expandedtags += strToList(tag)
 def decorator(item):
 if hasattr(item, "__oeqa_testtags"):
-item.__oeqa_testtags += expandedtags
+# do not append, create a new list (to handle classes with 
inheritance)
+item.__oeqa_testtags = list(item.__oeqa_testtags) + expandedtags
 else:
 item.__oeqa_testtags = expandedtags
 return item
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/7] oeqa/selftest: Use extraresults on self instead of self.tc

[Nathan Rossi]

In order to take advantage of multiprocess execution of tests the
extraresults must be passed through the TestResult. With changes to how
oeqa/core handles test cases the extraresults attribute of the testcase
is passed to the TestResult, with passing across process boundaries
handled automatically.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/binutils.py | 13 ++---
 meta/lib/oeqa/selftest/cases/gcc.py  | 14 +++---
 meta/lib/oeqa/selftest/cases/glibc.py| 13 ++---
 meta/lib/oeqa/selftest/cases/reproducible.py | 13 +
 4 files changed, 12 insertions(+), 41 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/binutils.py 
b/meta/lib/oeqa/selftest/cases/binutils.py
index 7f887959e4..759cd9b8ff 100644
--- a/meta/lib/oeqa/selftest/cases/binutils.py
+++ b/meta/lib/oeqa/selftest/cases/binutils.py
@@ -16,15 +16,6 @@ def parse_values(content):
 
 @OETestTag("machine")
 class BinutilsCrossSelfTest(OESelftestTestCase):
-@classmethod
-def setUpClass(cls):
-super().setUpClass()
-if not hasattr(cls.tc, "extraresults"):
-cls.tc.extraresults = {}
-
-if "ptestresult.sections" not in cls.tc.extraresults:
-cls.tc.extraresults["ptestresult.sections"] = {}
-
 def test_binutils(self):
 self.run_binutils("binutils")
 
@@ -46,7 +37,7 @@ class BinutilsCrossSelfTest(OESelftestTestCase):
 bitbake("{0} -c check".format(recipe))
 
 ptestsuite = "binutils-{}".format(suite) if suite != "binutils" else 
suite
-self.tc.extraresults["ptestresult.sections"][ptestsuite] = {}
+self.extraresults = {"ptestresult.sections" : {ptestsuite : {}}}
 
 sumspath = os.path.join(builddir, suite, "{0}.sum".format(suite))
 if not os.path.exists(sumspath):
@@ -54,5 +45,5 @@ class BinutilsCrossSelfTest(OESelftestTestCase):
 
 with open(sumspath, "r") as f:
 for test, result in parse_values(f):
-self.tc.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
+self.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
 
diff --git a/meta/lib/oeqa/selftest/cases/gcc.py 
b/meta/lib/oeqa/selftest/cases/gcc.py
index ef85df5e36..d0f0ca06e7 100644
--- a/meta/lib/oeqa/selftest/cases/gcc.py
+++ b/meta/lib/oeqa/selftest/cases/gcc.py
@@ -13,15 +13,6 @@ def parse_values(content):
 
 @OETestTag("machine")
 class GccSelfTest(OESelftestTestCase):
-@classmethod
-def setUpClass(cls):
-super().setUpClass()
-if not hasattr(cls.tc, "extraresults"):
-cls.tc.extraresults = {}
-
-if "ptestresult.sections" not in cls.tc.extraresults:
-cls.tc.extraresults["ptestresult.sections"] = {}
-
 def gcc_runtime_check_skip(self, suite):
 targets = get_bb_var("RUNTIMETARGET", "gcc-runtime").split()
 if suite not in targets:
@@ -71,6 +62,7 @@ class GccSelfTest(OESelftestTestCase):
 bb_vars = get_bb_vars(["B", "TARGET_SYS"], recipe)
 builddir, target_sys = bb_vars["B"], bb_vars["TARGET_SYS"]
 
+self.extraresults = {"ptestresult.sections" : {}}
 for suite in suites:
 sumspath = os.path.join(builddir, "gcc", "testsuite", suite, 
"{0}.sum".format(suite))
 if not os.path.exists(sumspath): # check in target dirs
@@ -80,10 +72,10 @@ class GccSelfTest(OESelftestTestCase):
 
 ptestsuite = "gcc-{}".format(suite) if suite != "gcc" else suite
 ptestsuite = ptestsuite + "-user" if ssh is None else ptestsuite
-self.tc.extraresults["ptestresult.sections"][ptestsuite] = {}
+self.extraresults["ptestresult.sections"][ptestsuite] = {}
 with open(sumspath, "r") as f:
 for test, result in parse_values(f):
-
self.tc.extraresults["ptestresult.{}.{}".format(ptestsuite, test)] = {"status" 
: result}
+self.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
 
 class GccSelfTestSystemEmulated(GccSelfTest):
 default_installed_packages = ["libgcc", "libstdc++", "libatomic", 
"libgomp"]
diff --git a/meta/lib/oeqa/selftest/cases/glibc.py 
b/meta/lib/oeqa/selftest/cases/glibc.py
index 4d0c13131b..e13de87014 100644
--- a/meta/lib/oeqa/selftest/cases/glibc.py
+++ b/meta/lib/oeqa/selftest/cases/glibc.py
@@ -15,15 +15,6 @@ def parse_values(content):
 
 @OETestTag("machine")
 class GlibcSelfTest(OESelftestTestCase):
-   

[OE-core] [PATCH 1/7] oeqa/core/runner.py: Fix OETestTag listing

[Nathan Rossi]

Use the __oeqa_testtags attribute added by OETestTag and display no, one
or more tags separated by a comma. Also change the formatting of the
output so that the list of tests is formatted as " ()" and
remove the table header for "id" (it is no longer available).

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/core/runner.py | 19 ---
 1 file changed, 8 insertions(+), 11 deletions(-)

diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py
index 930620ea19..63a8bae93f 100644
--- a/meta/lib/oeqa/core/runner.py
+++ b/meta/lib/oeqa/core/runner.py
@@ -205,23 +205,20 @@ class OETestRunner(_TestRunner):
 self._walked_cases = self._walked_cases + 1
 
 def _list_tests_name(self, suite):
-from oeqa.core.decorator.oetag import OETestTag
-
 self._walked_cases = 0
 
 def _list_cases(logger, case):
-oetag = None
-
-if hasattr(case, 'decorators'):
-for d in case.decorators:
-if isinstance(d, OETestTag):
-oetag = d.oetag
-
-logger.info("%s\t\t%s" % (oetag, case.id()))
+oetags = []
+if hasattr(case, '__oeqa_testtags'):
+oetags = getattr(case, '__oeqa_testtags')
+if oetags:
+logger.info("%s (%s)" % (case.id(), ",".join(oetags)))
+else:
+logger.info("%s" % (case.id()))
 
 self.tc.logger.info("Listing all available tests:")
 self._walked_cases = 0
-self.tc.logger.info("id\ttag\t\ttest")
+self.tc.logger.info("test (tags)")
 self.tc.logger.info("-" * 80)
 self._walk_suite(suite, _list_cases)
 self.tc.logger.info("-" * 80)
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] archiver.bbclass: Do not archive srpm's if there are no package tasks

[Nathan Rossi]

Do not attempt to archive recipes which have no packaging tasks (e.g.
inherits nopackages).

Signed-off-by: Nathan Rossi 
---
 meta/classes/archiver.bbclass | 14 --
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index af9f010dfc..13b05bb5f2 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -78,6 +78,9 @@ python () {
 bb.debug(1, 'archiver: %s is excluded, covered by gcc-source' % pn)
 return
 
+def hasTask(task):
+return bool(d.getVarFlag(task, "task", False)) and not 
bool(d.getVarFlag(task, "noexec", False))
+
 ar_src = d.getVarFlag('ARCHIVER_MODE', 'src')
 ar_dumpdata = d.getVarFlag('ARCHIVER_MODE', 'dumpdata')
 ar_recipe = d.getVarFlag('ARCHIVER_MODE', 'recipe')
@@ -98,9 +101,6 @@ python () {
 
 # There is a corner case with "gcc-source-${PV}" recipes, they don't 
have
 # the "do_configure" task, so we need to use "do_preconfigure"
-def hasTask(task):
-return bool(d.getVarFlag(task, "task", False)) and not 
bool(d.getVarFlag(task, "noexec", False))
-
 if hasTask("do_preconfigure"):
 d.appendVarFlag('do_ar_configured', 'depends', ' 
%s:do_preconfigure' % pn)
 elif hasTask("do_configure"):
@@ -118,7 +118,11 @@ python () {
 
 # Output the SRPM package
 if d.getVarFlag('ARCHIVER_MODE', 'srpm') == "1" and d.getVar('PACKAGES'):
-if "package_rpm" in d.getVar('PACKAGE_CLASSES'):
+if "package_rpm" not in d.getVar('PACKAGE_CLASSES'):
+bb.fatal("ARCHIVER_MODE[srpm] needs package_rpm in 
PACKAGE_CLASSES")
+
+# Some recipes do not have any packaging tasks
+if hasTask("do_package_write_rpm"):
 d.appendVarFlag('do_deploy_archives', 'depends', ' 
%s:do_package_write_rpm' % pn)
 d.appendVarFlag('do_package_write_rpm', 'dirs', ' 
${ARCHIVER_RPMTOPDIR}')
 d.appendVarFlag('do_package_write_rpm', 'sstate-inputdirs', ' 
${ARCHIVER_RPMTOPDIR}')
@@ -133,8 +137,6 @@ python () {
 d.appendVarFlag('do_package_write_rpm', 'depends', ' 
%s:do_ar_patched' % pn)
 elif ar_src == "configured":
 d.appendVarFlag('do_package_write_rpm', 'depends', ' 
%s:do_ar_configured' % pn)
-else:
-bb.fatal("ARCHIVER_MODE[srpm] needs package_rpm in 
PACKAGE_CLASSES")
 }
 
 # Take all the sources for a recipe and puts them in WORKDIR/archiver-work/.
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] security_flags.inc: Add overrides for glibc-testsuite to match glibc

[Nathan Rossi]

Ensure glibc-testsuite has the same flags as glibc.

Signed-off-by: Nathan Rossi 
---
 meta/conf/distro/include/security_flags.inc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/conf/distro/include/security_flags.inc 
b/meta/conf/distro/include/security_flags.inc
index 758bd89c3e..aaf04e9e59 100644
--- a/meta/conf/distro/include/security_flags.inc
+++ b/meta/conf/distro/include/security_flags.inc
@@ -36,6 +36,7 @@ SECURITY_CFLAGS_remove_powerpc = "${SECURITY_PIE_CFLAGS}"
 SECURITY_CFLAGS_pn-libgcc_powerpc = ""
 
 SECURITY_CFLAGS_pn-glibc = ""
+SECURITY_CFLAGS_pn-glibc-testsuite = ""
 SECURITY_CFLAGS_pn-gcc-runtime = ""
 SECURITY_CFLAGS_pn-grub = ""
 SECURITY_CFLAGS_pn-grub-efi = ""
@@ -59,6 +60,7 @@ TARGET_LDFLAGS_append_class-target = " ${SECURITY_LDFLAGS}"
 
 SECURITY_STACK_PROTECTOR_pn-gcc-runtime = ""
 SECURITY_STACK_PROTECTOR_pn-glibc = ""
+SECURITY_STACK_PROTECTOR_pn-glibc-testsuite = ""
 # All xorg module drivers need to be linked this way as well and are
 # handled in recipes-graphics/xorg-driver/xorg-driver-common.inc
 SECURITY_LDFLAGS_pn-xserver-xorg = "${SECURITY_X_LDFLAGS}"
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] Clean up remnants of glibc-initial

[Nathan Rossi]

Remove remnants of the glibc-initial recipe.

Signed-off-by: Nathan Rossi 
---
 meta/classes/staging.bbclass| 2 +-
 meta/conf/distro/include/maintainers.inc| 1 -
 meta/conf/distro/include/security_flags.inc | 2 --
 meta/conf/distro/include/tcmode-default.inc | 4 
 meta/recipes-core/glibc/glibc-package.inc   | 9 +
 5 files changed, 2 insertions(+), 16 deletions(-)

diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
index cdc342dbb9..55a9b52ed2 100644
--- a/meta/classes/staging.bbclass
+++ b/meta/classes/staging.bbclass
@@ -197,7 +197,7 @@ def staging_populate_sysroot_dir(targetsysroot, 
nativesysroot, native, d):
 for pkgarch in pkgarchs:
 for manifest in 
glob.glob(d.expand("${SSTATE_MANIFESTS}/manifest-%s-*.populate_sysroot" % 
pkgarch)):
 if manifest.endswith("-initial.populate_sysroot"):
-# skip glibc-initial and libgcc-initial due to file overlap
+# skip libgcc-initial due to file overlap
 continue
 if not native and (manifest.endswith("-native.populate_sysroot") 
or "nativesdk-" in manifest):
 continue
diff --git a/meta/conf/distro/include/maintainers.inc 
b/meta/conf/distro/include/maintainers.inc
index aaae55ac86..566641a895 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -217,7 +217,6 @@ RECIPE_MAINTAINER_pn-glew = "Anuj Mittal 
"
 RECIPE_MAINTAINER_pn-glib-2.0 = "Anuj Mittal "
 RECIPE_MAINTAINER_pn-glib-networking = "Anuj Mittal "
 RECIPE_MAINTAINER_pn-glibc = "Khem Raj "
-RECIPE_MAINTAINER_pn-glibc-initial = "Khem Raj "
 RECIPE_MAINTAINER_pn-glibc-locale = "Khem Raj "
 RECIPE_MAINTAINER_pn-glibc-mtrace = "Khem Raj "
 RECIPE_MAINTAINER_pn-glibc-scripts = "Khem Raj "
diff --git a/meta/conf/distro/include/security_flags.inc 
b/meta/conf/distro/include/security_flags.inc
index 620978a8ed..758bd89c3e 100644
--- a/meta/conf/distro/include/security_flags.inc
+++ b/meta/conf/distro/include/security_flags.inc
@@ -36,7 +36,6 @@ SECURITY_CFLAGS_remove_powerpc = "${SECURITY_PIE_CFLAGS}"
 SECURITY_CFLAGS_pn-libgcc_powerpc = ""
 
 SECURITY_CFLAGS_pn-glibc = ""
-SECURITY_CFLAGS_pn-glibc-initial = ""
 SECURITY_CFLAGS_pn-gcc-runtime = ""
 SECURITY_CFLAGS_pn-grub = ""
 SECURITY_CFLAGS_pn-grub-efi = ""
@@ -60,7 +59,6 @@ TARGET_LDFLAGS_append_class-target = " ${SECURITY_LDFLAGS}"
 
 SECURITY_STACK_PROTECTOR_pn-gcc-runtime = ""
 SECURITY_STACK_PROTECTOR_pn-glibc = ""
-SECURITY_STACK_PROTECTOR_pn-glibc-initial = ""
 # All xorg module drivers need to be linked this way as well and are
 # handled in recipes-graphics/xorg-driver/xorg-driver-common.inc
 SECURITY_LDFLAGS_pn-xserver-xorg = "${SECURITY_X_LDFLAGS}"
diff --git a/meta/conf/distro/include/tcmode-default.inc 
b/meta/conf/distro/include/tcmode-default.inc
index 5c28386c4f..d8f434609b 100644
--- a/meta/conf/distro/include/tcmode-default.inc
+++ b/meta/conf/distro/include/tcmode-default.inc
@@ -14,8 +14,6 @@ PREFERRED_PROVIDER_virtual/${SDK_PREFIX}g++ = 
"gcc-crosssdk-${SDK_SYS}"
 PREFERRED_PROVIDER_virtual/${SDK_PREFIX}compilerlibs = "nativesdk-gcc-runtime"
 
 # Default libc config
-PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}libc-initial = "${TCLIBC}-initial"
-PREFERRED_PROVIDER_virtual/nativesdk-${SDK_PREFIX}libc-initial ?= 
"nativesdk-glibc-initial"
 PREFERRED_PROVIDER_virtual/gettext ??= "gettext"
 
 GCCVERSION ?= "9.%"
@@ -62,8 +60,6 @@ PREFERRED_VERSION_glibc-locale ?= 
"${GLIBCVERSION}"
 PREFERRED_VERSION_glibc-mtrace ?= "${GLIBCVERSION}"
 PREFERRED_VERSION_glibc-scripts?= "${GLIBCVERSION}"
 PREFERRED_VERSION_nativesdk-glibc  ?= "${GLIBCVERSION}"
-PREFERRED_VERSION_glibc-initial?= "${GLIBCVERSION}"
-PREFERRED_VERSION_nativesdk-glibc-initial  ?= "${GLIBCVERSION}"
 PREFERRED_VERSION_cross-localedef-native   ?= "${GLIBCVERSION}"
 
 PREFERRED_VERSION_qemu ?= "${QEMUVERSION}"
diff --git a/meta/recipes-core/glibc/glibc-package.inc 
b/meta/recipes-core/glibc/glibc-package.inc
index f796876a87..9b1e7b7903 100644
--- a/meta/recipes-core/glibc/glibc-package.inc
+++ b/meta/recipes-core/glibc/glibc-package.inc
@@ -88,14 +88,7 @@ do_install_append () {
rm -f ${D}${sysconfdir}/ld.so.conf
rm -f ${D}${base_sbindir}/ldconfig
# This directory will be empty now so remove it too.
-   # But check whether it exists first, since it won't for 
glibc-initial.
-   if [ -d ${D}${sysconfdir} ]; then
-   rmdir ${D}${sysconfdir}
-   fi
-   fi
-
-   if echo ${PN}|grep -q "glibc-initial"; then
-   return
+   rmdir ${D}${sysconfdir}
fi
 
install -d ${D}${sysconfdir}/init.d
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] oe-selftest: Implement console 'keepalive' output

[Nathan Rossi]

Similar to bitbake, implement a 'keepalive' output to the console to
ensure CI systems do not kill the process. The default timeout for
bitbake is 5000s.

Signed-off-by: Nathan Rossi 
---
 scripts/lib/scriptutils.py | 43 +--
 scripts/oe-selftest|  2 +-
 2 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/scripts/lib/scriptutils.py b/scripts/lib/scriptutils.py
index e7e7021c24..c573dc7f67 100644
--- a/scripts/lib/scriptutils.py
+++ b/scripts/lib/scriptutils.py
@@ -16,12 +16,51 @@ import string
 import subprocess
 import sys
 import tempfile
+import threading
 import importlib
 from importlib import machinery
 
-def logger_create(name, stream=None):
+class KeepAliveStreamHandler(logging.StreamHandler):
+def __init__(self, keepalive=True, **kwargs):
+super().__init__(**kwargs)
+if keepalive is True:
+keepalive = 5000 # default timeout
+self._timeout = threading.Condition()
+self._stop = False
+
+# background thread waits on condition, if the condition does not
+# happen emit a keep alive message
+def thread():
+while not self._stop:
+with self._timeout:
+if not self._timeout.wait(keepalive):
+self.emit(logging.LogRecord("keepalive", logging.INFO,
+None, None, "Keepalive message", None, None))
+
+self._thread = threading.Thread(target = thread, daemon = True)
+self._thread.start()
+
+def close(self):
+# mark the thread to stop and notify it
+self._stop = True
+with self._timeout:
+self._timeout.notify()
+# wait for it to join
+self._thread.join()
+super().close()
+
+def emit(self, record):
+super().emit(record)
+# trigger timer reset
+with self._timeout:
+self._timeout.notify()
+
+def logger_create(name, stream=None, keepalive=None):
 logger = logging.getLogger(name)
-loggerhandler = logging.StreamHandler(stream=stream)
+if keepalive is not None:
+loggerhandler = KeepAliveStreamHandler(stream=stream, 
keepalive=keepalive)
+else:
+loggerhandler = logging.StreamHandler(stream=stream)
 loggerhandler.setFormatter(logging.Formatter("%(levelname)s: %(message)s"))
 logger.addHandler(loggerhandler)
 logger.setLevel(logging.INFO)
diff --git a/scripts/oe-selftest b/scripts/oe-selftest
index 57662b2f75..18ac0f5869 100755
--- a/scripts/oe-selftest
+++ b/scripts/oe-selftest
@@ -33,7 +33,7 @@ scriptpath.add_bitbake_lib_path()
 from oeqa.utils import load_test_components
 from oeqa.core.exception import OEQAPreRun
 
-logger = scriptutils.logger_create('oe-selftest', stream=sys.stdout)
+logger = scriptutils.logger_create('oe-selftest', stream=sys.stdout, 
keepalive=True)
 
 def main():
 description = "Script that runs unit tests against bitbake and other Yocto 
related tools. The goal is to validate tools functionality and metadata 
integrity. Refer to https://wiki.yoctoproject.org/wiki/Oe-selftest for more 
information."
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/4] oeqa/selftest: Toolchain tests suffix "-user" for qemu usermode results

[Nathan Rossi]

Suffix the ptestresults suite with "-user" for tests that are executing
against usermode qemu.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/gcc.py   | 1 +
 meta/lib/oeqa/selftest/cases/glibc.py | 5 +++--
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/gcc.py 
b/meta/lib/oeqa/selftest/cases/gcc.py
index 4f7ffdcb2a..ef85df5e36 100644
--- a/meta/lib/oeqa/selftest/cases/gcc.py
+++ b/meta/lib/oeqa/selftest/cases/gcc.py
@@ -79,6 +79,7 @@ class GccSelfTest(OESelftestTestCase):
 sumspath = os.path.join(builddir, target_sys, suite, 
"testsuite", "{0}.sum".format(suite.split("-")[0]))
 
 ptestsuite = "gcc-{}".format(suite) if suite != "gcc" else suite
+ptestsuite = ptestsuite + "-user" if ssh is None else ptestsuite
 self.tc.extraresults["ptestresult.sections"][ptestsuite] = {}
 with open(sumspath, "r") as f:
 for test, result in parse_values(f):
diff --git a/meta/lib/oeqa/selftest/cases/glibc.py 
b/meta/lib/oeqa/selftest/cases/glibc.py
index 7992ea6bf2..4d0c13131b 100644
--- a/meta/lib/oeqa/selftest/cases/glibc.py
+++ b/meta/lib/oeqa/selftest/cases/glibc.py
@@ -43,10 +43,11 @@ class GlibcSelfTest(OESelftestTestCase):
 
 builddir = get_bb_var("B", "glibc-testsuite")
 
-self.tc.extraresults["ptestresult.sections"]["glibc"] = {}
+ptestsuite = "glibc-user" if ssh is None else "glibc"
+self.tc.extraresults["ptestresult.sections"][ptestsuite] = {}
 with open(os.path.join(builddir, "tests.sum"), "r") as f:
 for test, result in parse_values(f):
-self.tc.extraresults["ptestresult.glibc.{}".format(test)] = 
{"status" : result}
+self.tc.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
 
 class GlibcSelfTestSystemEmulated(GlibcSelfTest):
 default_installed_packages = [
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/4] maintainers.inc: Add entries for toolchain related recipes

[Nathan Rossi]

Add entries for the added toolchain related recipes including
binutils-cross-testsuite and glibc-testsuite setting the maintainer as
Khem to match existing definitions. Also and entry for dejagnu setting
Nathan Rossi as the maintainer.

Signed-off-by: Nathan Rossi 
---
 meta/conf/distro/include/maintainers.inc | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/conf/distro/include/maintainers.inc 
b/meta/conf/distro/include/maintainers.inc
index 0757230b1a..aaae55ac86 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -66,6 +66,7 @@ RECIPE_MAINTAINER_pn-bind = "Armin Kuster 
"
 RECIPE_MAINTAINER_pn-binutils = "Khem Raj "
 RECIPE_MAINTAINER_pn-binutils-cross-${TARGET_ARCH} = "Khem Raj 
"
 RECIPE_MAINTAINER_pn-binutils-cross-canadian-${TRANSLATED_TARGET_ARCH} = "Khem 
Raj "
+RECIPE_MAINTAINER_pn-binutils-cross-testsuite = "Khem Raj "
 RECIPE_MAINTAINER_pn-binutils-crosssdk-${SDK_SYS} = "Khem Raj 
"
 RECIPE_MAINTAINER_pn-bison = "Chen Qi "
 RECIPE_MAINTAINER_pn-bjam-native = "Alexander Kanavin "
@@ -142,6 +143,7 @@ RECIPE_MAINTAINER_pn-dbus-glib = "Chen Qi 
"
 RECIPE_MAINTAINER_pn-dbus-test = "Chen Qi "
 RECIPE_MAINTAINER_pn-dbus-wait = "Chen Qi "
 RECIPE_MAINTAINER_pn-debianutils = "Yi Zhao "
+RECIPE_MAINTAINER_pn-dejagnu = "Nathan Rossi "
 RECIPE_MAINTAINER_pn-depmodwrapper-cross = "Mark Hatle 
"
 RECIPE_MAINTAINER_pn-desktop-file-utils = "Alexander Kanavin 
"
 RECIPE_MAINTAINER_pn-dhcp = "Hongxu Jia "
@@ -219,6 +221,7 @@ RECIPE_MAINTAINER_pn-glibc-initial = "Khem Raj 
"
 RECIPE_MAINTAINER_pn-glibc-locale = "Khem Raj "
 RECIPE_MAINTAINER_pn-glibc-mtrace = "Khem Raj "
 RECIPE_MAINTAINER_pn-glibc-scripts = "Khem Raj "
+RECIPE_MAINTAINER_pn-glibc-testsuite = "Khem Raj "
 RECIPE_MAINTAINER_pn-glide = "Otavio Salvador 
"
 RECIPE_MAINTAINER_pn-gmp = "Khem Raj "
 RECIPE_MAINTAINER_pn-gnome-desktop-testing = "Ross Burton 
"
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/4] oeqa/selftest: For toolchain tests do not fail if a test failed

[Nathan Rossi]

Do not fail the running selftest test case if the test suite has a
failed test case. Currently toolchain tests suites (binutils, gcc,
glibc) fail but this does not indicate failure to execute the tests.

Also remove the logging of each test that failed.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/binutils.py | 6 --
 meta/lib/oeqa/selftest/cases/gcc.py  | 6 --
 meta/lib/oeqa/selftest/cases/glibc.py| 5 -
 3 files changed, 17 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/binutils.py 
b/meta/lib/oeqa/selftest/cases/binutils.py
index 4edee09390..7f887959e4 100644
--- a/meta/lib/oeqa/selftest/cases/binutils.py
+++ b/meta/lib/oeqa/selftest/cases/binutils.py
@@ -52,13 +52,7 @@ class BinutilsCrossSelfTest(OESelftestTestCase):
 if not os.path.exists(sumspath):
 sumspath = os.path.join(builddir, suite, "testsuite", 
"{0}.sum".format(suite))
 
-failed = 0
 with open(sumspath, "r") as f:
 for test, result in parse_values(f):
 self.tc.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
-if result == "FAIL":
-self.logger.info("failed: '{}'".format(test))
-failed += 1
-
-self.assertEqual(failed, 0)
 
diff --git a/meta/lib/oeqa/selftest/cases/gcc.py 
b/meta/lib/oeqa/selftest/cases/gcc.py
index 0ad6e21d6b..4f7ffdcb2a 100644
--- a/meta/lib/oeqa/selftest/cases/gcc.py
+++ b/meta/lib/oeqa/selftest/cases/gcc.py
@@ -71,7 +71,6 @@ class GccSelfTest(OESelftestTestCase):
 bb_vars = get_bb_vars(["B", "TARGET_SYS"], recipe)
 builddir, target_sys = bb_vars["B"], bb_vars["TARGET_SYS"]
 
-failed = 0
 for suite in suites:
 sumspath = os.path.join(builddir, "gcc", "testsuite", suite, 
"{0}.sum".format(suite))
 if not os.path.exists(sumspath): # check in target dirs
@@ -84,11 +83,6 @@ class GccSelfTest(OESelftestTestCase):
 with open(sumspath, "r") as f:
 for test, result in parse_values(f):
 
self.tc.extraresults["ptestresult.{}.{}".format(ptestsuite, test)] = {"status" 
: result}
-if result == "FAIL":
-self.logger.info("failed: '{}'".format(test))
-failed += 1
-
-self.assertEqual(failed, 0)
 
 class GccSelfTestSystemEmulated(GccSelfTest):
 default_installed_packages = ["libgcc", "libstdc++", "libatomic", 
"libgomp"]
diff --git a/meta/lib/oeqa/selftest/cases/glibc.py 
b/meta/lib/oeqa/selftest/cases/glibc.py
index 6a34da39d9..7992ea6bf2 100644
--- a/meta/lib/oeqa/selftest/cases/glibc.py
+++ b/meta/lib/oeqa/selftest/cases/glibc.py
@@ -43,15 +43,10 @@ class GlibcSelfTest(OESelftestTestCase):
 
 builddir = get_bb_var("B", "glibc-testsuite")
 
-failed = 0
 self.tc.extraresults["ptestresult.sections"]["glibc"] = {}
 with open(os.path.join(builddir, "tests.sum"), "r") as f:
 for test, result in parse_values(f):
 self.tc.extraresults["ptestresult.glibc.{}".format(test)] = 
{"status" : result}
-if result == "FAIL":
-self.logger.info("failed: '{}'".format(test))
-failed += 1
-self.assertEqual(failed, 0)
 
 class GlibcSelfTestSystemEmulated(GlibcSelfTest):
 default_installed_packages = [
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/4] oeqa/selftest/cases/glibc.py: Don't install python3 and pexpect

[Nathan Rossi]

When running the system emulation test case, do not include python3 or
pexpect in the image. The test cases that use these also need gdb (with
python configured).

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/glibc.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/glibc.py 
b/meta/lib/oeqa/selftest/cases/glibc.py
index f836367094..6a34da39d9 100644
--- a/meta/lib/oeqa/selftest/cases/glibc.py
+++ b/meta/lib/oeqa/selftest/cases/glibc.py
@@ -60,8 +60,8 @@ class GlibcSelfTestSystemEmulated(GlibcSelfTest):
 "libstdc++",
 "libatomic",
 "libgomp",
-"python3",
-"python3-pexpect",
+# "python3",
+# "python3-pexpect",
 "nfs-utils",
 ]
 
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 07/12] oeqa/core/tests: Fix test_data module tests

[Nathan Rossi]

These two tests relied on the context containing the results information.
This was moved into the OETestResults class.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- New
---
 meta/lib/oeqa/core/tests/test_data.py | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/core/tests/test_data.py 
b/meta/lib/oeqa/core/tests/test_data.py
index 50811bb381..ac74098b78 100755
--- a/meta/lib/oeqa/core/tests/test_data.py
+++ b/meta/lib/oeqa/core/tests/test_data.py
@@ -22,8 +22,9 @@ class TestData(TestBase):
 expectedException = "oeqa.core.exception.OEQAMissingVariable"
 
 tc = self._testLoader(modules=self.modules)
-self.assertEqual(False, tc.runTests().wasSuccessful())
-for test, data in tc.errors:
+results = tc.runTests()
+self.assertFalse(results.wasSuccessful())
+for test, data in results.errors:
 expect = False
 if expectedException in data:
 expect = True
@@ -35,8 +36,9 @@ class TestData(TestBase):
 d = {'IMAGE' : 'core-image-sato', 'ARCH' : 'arm'}
 
 tc = self._testLoader(d=d, modules=self.modules)
-self.assertEqual(False, tc.runTests().wasSuccessful())
-for test, data in tc.failures:
+results = tc.runTests()
+self.assertFalse(results.wasSuccessful())
+for test, data in results.failures:
 expect = False
 if expectedError in data:
 expect = True
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 09/12] oeqa/selftest: Add test run filtering based on test tags

[Nathan Rossi]

Add '--run-only-tags' for running tests which match any of the provided
tags, and '--run-exclude-tags' for running all tests except those that
have any of the provided tags.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- New
---
 meta/lib/oeqa/selftest/context.py | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/meta/lib/oeqa/selftest/context.py 
b/meta/lib/oeqa/selftest/context.py
index d279994ddf..47de08e3f2 100644
--- a/meta/lib/oeqa/selftest/context.py
+++ b/meta/lib/oeqa/selftest/context.py
@@ -61,6 +61,12 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
 group.add_argument('-r', '--run-tests', required=False, action='store',
 nargs='+', dest="run_tests", default=None,
 help='Select what tests to run (modules, classes or test 
methods). Format should be: ..')
+group.add_argument('-t', '--run-only-tags', action='store',
+nargs='+', dest="run_only_tags", default=None,
+help='Run all (unhidden) tests which match any of the 
specified tags.')
+group.add_argument('-T', '--run-exclude-tags', action='store',
+nargs='+', dest="run_exclude_tags", default=None,
+help='Run all (unhidden) tests excluding any that match any of 
the specified tags.')
 
 group.add_argument('-m', '--list-modules', required=False,
 action="store_true", default=False,
@@ -149,6 +155,11 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
 copyfile(self.tc_kwargs['init']['config_paths']['bblayers'], 
 self.tc_kwargs['init']['config_paths']['bblayers_backup'])
 
+if args.run_only_tags:
+self.tc_kwargs['load']['tags_filter'] = lambda tags: not tags or 
not any(tag in args.run_only_tags for tag in tags)
+if args.run_exclude_tags:
+self.tc_kwargs['load']['tags_filter'] = lambda tags: any(tag in 
args.run_exclude_tags for tag in tags)
+
 self.tc_kwargs['run']['skips'] = args.skips
 self.tc_kwargs['run']['processes'] = args.processes
 
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 12/12] oeqa/selftest/glibc: Create selftest case for glibc test suite

[Nathan Rossi]

Create a oeqa selftest test case to execute the glibc test suite and
report the results. The results are populated into the extraresults
variable of the test case which are written to testresults.json for
resulttool to analyse.

An additional subclass is created to separate the execution with qemu
linux-user and qemu system. The GlibcSelfTestSystemEmulated test case
handles setup of the target image, setup of and NFS server as well as
execution with runqemu.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- Populate test results as ptestresults
- Rename BUILD_TEST_* to TOOLCHAIN_TEST_*
- Mark tests with 'machine' tag
---
 meta/lib/oeqa/selftest/cases/glibc.py | 97 +++
 1 file changed, 97 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/glibc.py

diff --git a/meta/lib/oeqa/selftest/cases/glibc.py 
b/meta/lib/oeqa/selftest/cases/glibc.py
new file mode 100644
index 00..f836367094
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/glibc.py
@@ -0,0 +1,97 @@
+# SPDX-License-Identifier: MIT
+import os
+import contextlib
+from oeqa.core.decorator import OETestTag
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, 
Command
+from oeqa.utils.nfs import unfs_server
+
+def parse_values(content):
+for i in content:
+for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", 
"UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]:
+if i.startswith(v + ": "):
+yield i[len(v) + 2:].strip(), v
+break
+
+@OETestTag("machine")
+class GlibcSelfTest(OESelftestTestCase):
+@classmethod
+def setUpClass(cls):
+super().setUpClass()
+if not hasattr(cls.tc, "extraresults"):
+cls.tc.extraresults = {}
+
+if "ptestresult.sections" not in cls.tc.extraresults:
+cls.tc.extraresults["ptestresult.sections"] = {}
+
+def test_glibc(self):
+self.glibc_run_check()
+
+def glibc_run_check(self, ssh = None):
+# configure ssh target
+features = []
+if ssh is not None:
+features.append('TOOLCHAIN_TEST_TARGET = "ssh"')
+features.append('TOOLCHAIN_TEST_HOST = "{0}"'.format(ssh))
+features.append('TOOLCHAIN_TEST_HOST_USER = "root"')
+features.append('TOOLCHAIN_TEST_HOST_PORT = "22"')
+# force single threaded test execution
+features.append('EGLIBCPARALLELISM_task-check_pn-glibc-testsuite = 
"PARALLELMFLAGS="-j1""')
+self.write_config("\n".join(features))
+
+bitbake("glibc-testsuite -c check")
+
+builddir = get_bb_var("B", "glibc-testsuite")
+
+failed = 0
+self.tc.extraresults["ptestresult.sections"]["glibc"] = {}
+with open(os.path.join(builddir, "tests.sum"), "r") as f:
+for test, result in parse_values(f):
+self.tc.extraresults["ptestresult.glibc.{}".format(test)] = 
{"status" : result}
+if result == "FAIL":
+self.logger.info("failed: '{}'".format(test))
+failed += 1
+self.assertEqual(failed, 0)
+
+class GlibcSelfTestSystemEmulated(GlibcSelfTest):
+default_installed_packages = [
+"glibc-charmaps",
+"libgcc",
+"libstdc++",
+"libatomic",
+"libgomp",
+"python3",
+"python3-pexpect",
+"nfs-utils",
+]
+
+def glibc_run_check(self):
+with contextlib.ExitStack() as s:
+# use the base work dir, as the nfs mount, since the recipe 
directory may not exist
+tmpdir = get_bb_var("BASE_WORKDIR")
+nfsport, mountport = s.enter_context(unfs_server(tmpdir))
+
+# build core-image-minimal with required packages
+features = []
+features.append('IMAGE_FEATURES += "ssh-server-openssh"')
+features.append('CORE_IMAGE_EXTRA_INSTALL += "{0}"'.format(" 
".join(self.default_installed_packages)))
+self.write_config("\n".join(features))
+bitbake("core-image-minimal")
+
+# start runqemu
+qemu = s.enter_context(runqemu("core-image-minimal", runqemuparams 
= "nographic"))
+
+# validate that SSH is working
+status, _ = qemu.run("uname")
+self.assertEqual(status, 0)
+

[OE-core] [PATCH v2 11/12] oeqa/selftest/gcc: Create selftest case for gcc test suite

[Nathan Rossi]

Create a oeqa selftest test case to execute the gcc test suites and
report the results. The results are populated into the extraresults
variable of the test case which are written to testresults.json for
resulttool to analyse.

An additional subclass is created to separate the execution with qemu
linux-user and qemu system. The GccSelfTestSystemEmulated test case
handles setup of the target image as well as execution with runqemu.

Signed-off-by: Nathan Rossi 
---
Changed in v2:
- Change to using only gcc-runtime recipe and associated check targets
- Remove gxx/g++ testcase, this is part of the "check-gcc" target
- Populate test results as ptestresults
- Rename BUILD_TEST_* to TOOLCHAIN_TEST_*
- Mark tests with 'machine' tag
---
 meta/lib/oeqa/selftest/cases/gcc.py | 111 
 1 file changed, 111 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/gcc.py

diff --git a/meta/lib/oeqa/selftest/cases/gcc.py 
b/meta/lib/oeqa/selftest/cases/gcc.py
new file mode 100644
index 00..0ad6e21d6b
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/gcc.py
@@ -0,0 +1,111 @@
+# SPDX-License-Identifier: MIT
+import os
+from oeqa.core.decorator import OETestTag
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, 
Command
+
+def parse_values(content):
+for i in content:
+for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", 
"UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]:
+if i.startswith(v + ": "):
+yield i[len(v) + 2:].strip(), v
+break
+
+@OETestTag("machine")
+class GccSelfTest(OESelftestTestCase):
+@classmethod
+def setUpClass(cls):
+super().setUpClass()
+if not hasattr(cls.tc, "extraresults"):
+cls.tc.extraresults = {}
+
+if "ptestresult.sections" not in cls.tc.extraresults:
+cls.tc.extraresults["ptestresult.sections"] = {}
+
+def gcc_runtime_check_skip(self, suite):
+targets = get_bb_var("RUNTIMETARGET", "gcc-runtime").split()
+if suite not in targets:
+self.skipTest("Target does not use {0}".format(suite))
+
+def test_cross_gcc(self):
+self.gcc_run_check("gcc", "g++")
+
+def test_libatomic(self):
+self.gcc_run_check("libatomic")
+
+def test_libgomp(self):
+self.gcc_run_check("libgomp")
+
+def test_libstdcxx(self):
+self.gcc_run_check("libstdc++-v3")
+
+def test_libssp(self):
+self.gcc_runtime_check_skip("libssp")
+self.gcc_run_check("libssp")
+
+def test_libitm(self):
+self.gcc_runtime_check_skip("libitm")
+self.gcc_run_check("libitm")
+
+def gcc_run_check(self, *suites, ssh = None):
+targets = set()
+for s in suites:
+if s in ["gcc", "g++"]:
+targets.add("check-gcc")
+else:
+targets.add("check-target-{}".format(s))
+
+# configure ssh target
+features = []
+features.append('MAKE_CHECK_TARGETS = "{0}"'.format(" ".join(targets)))
+if ssh is not None:
+features.append('TOOLCHAIN_TEST_TARGET = "ssh"')
+features.append('TOOLCHAIN_TEST_HOST = "{0}"'.format(ssh))
+features.append('TOOLCHAIN_TEST_HOST_USER = "root"')
+features.append('TOOLCHAIN_TEST_HOST_PORT = "22"')
+self.write_config("\n".join(features))
+
+recipe = "gcc-runtime"
+bitbake("{} -c check".format(recipe))
+
+bb_vars = get_bb_vars(["B", "TARGET_SYS"], recipe)
+builddir, target_sys = bb_vars["B"], bb_vars["TARGET_SYS"]
+
+failed = 0
+for suite in suites:
+sumspath = os.path.join(builddir, "gcc", "testsuite", suite, 
"{0}.sum".format(suite))
+if not os.path.exists(sumspath): # check in target dirs
+sumspath = os.path.join(builddir, target_sys, suite, 
"testsuite", "{0}.sum".format(suite))
+if not os.path.exists(sumspath): # handle libstdc++-v3 -> libstdc++
+sumspath = os.path.join(builddir, target_sys, suite, 
"testsuite", "{0}.sum".format(suite.split("-")[0]))
+
+ptestsuite = "gcc-{}".format(suite) if suite != "gcc" else suite
+self.tc.extraresults["ptestresult.sections"][ptestsuite] = {}
+wit

[OE-core] [PATCH v2 10/12] oeqa/selftest/binutils: Create selftest case for binutils test suite

[Nathan Rossi]

Create a oeqa selftest test case to execute the binutils test suites and
report the results. The results are populated into the extraresults
variable of the test case which are written to testresults.json for
resulttool to analyse.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- Update to use binutils-cross-testsuite and CHECK_TARGETS
- Drop gold and libiberty test cases (and result parsing code)
- Populate test results as ptestresults
- Mark tests with 'machine' tag
---
 meta/lib/oeqa/selftest/cases/binutils.py | 64 
 1 file changed, 64 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/binutils.py

diff --git a/meta/lib/oeqa/selftest/cases/binutils.py 
b/meta/lib/oeqa/selftest/cases/binutils.py
new file mode 100644
index 00..4edee09390
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/binutils.py
@@ -0,0 +1,64 @@
+# SPDX-License-Identifier: MIT
+import os
+import sys
+import re
+import logging
+from oeqa.core.decorator import OETestTag
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars
+
+def parse_values(content):
+for i in content:
+for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", 
"UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]:
+if i.startswith(v + ": "):
+yield i[len(v) + 2:].strip(), v
+break
+
+@OETestTag("machine")
+class BinutilsCrossSelfTest(OESelftestTestCase):
+@classmethod
+def setUpClass(cls):
+super().setUpClass()
+if not hasattr(cls.tc, "extraresults"):
+cls.tc.extraresults = {}
+
+if "ptestresult.sections" not in cls.tc.extraresults:
+cls.tc.extraresults["ptestresult.sections"] = {}
+
+def test_binutils(self):
+self.run_binutils("binutils")
+
+def test_gas(self):
+self.run_binutils("gas")
+
+def test_ld(self):
+self.run_binutils("ld")
+
+def run_binutils(self, suite):
+features = []
+features.append('CHECK_TARGETS = "{0}"'.format(suite))
+self.write_config("\n".join(features))
+
+recipe = "binutils-cross-testsuite"
+bb_vars = get_bb_vars(["B", "TARGET_SYS", "T"], recipe)
+builddir, target_sys, tdir = bb_vars["B"], bb_vars["TARGET_SYS"], 
bb_vars["T"]
+
+bitbake("{0} -c check".format(recipe))
+
+ptestsuite = "binutils-{}".format(suite) if suite != "binutils" else 
suite
+self.tc.extraresults["ptestresult.sections"][ptestsuite] = {}
+
+sumspath = os.path.join(builddir, suite, "{0}.sum".format(suite))
+if not os.path.exists(sumspath):
+sumspath = os.path.join(builddir, suite, "testsuite", 
"{0}.sum".format(suite))
+
+failed = 0
+with open(sumspath, "r") as f:
+for test, result in parse_values(f):
+self.tc.extraresults["ptestresult.{}.{}".format(ptestsuite, 
test)] = {"status" : result}
+if result == "FAIL":
+self.logger.info("failed: '{}'".format(test))
+failed += 1
+
+self.assertEqual(failed, 0)
+
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 08/12] oeqa/core: Rework OETestTag and remove unused OETestFilter

[Nathan Rossi]

Rework OETestTag so that it does not rely on the existing decorator code
base and instead inserts the tags into an attribute on the decorated
target (e.g. class/type or method). This allows the use of OETestTag on
classes and method.

In order to filter tagged tests rework the loaders filtering code,
removing the generic-ness (with validation and attributes/etc.) and
replace it with a "tags_filter" parameter which is a function that
filters a test based on the tags it has. This allows the loader user to
filter on tags in more specific ways (e.g. include all untagged tests
and any tests tagged with foo). Plumb all this through the context code
and testing code.

Update the associated tests to pass correctly with the changes.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- New
---
 meta/lib/oeqa/core/context.py   |  4 +-
 meta/lib/oeqa/core/decorator/__init__.py| 20 +---
 meta/lib/oeqa/core/decorator/oetag.py   | 27 --
 meta/lib/oeqa/core/loader.py| 61 +++
 meta/lib/oeqa/core/tests/cases/data.py  |  2 +-
 meta/lib/oeqa/core/tests/cases/oetag.py | 21 +++-
 meta/lib/oeqa/core/tests/common.py  |  4 +-
 meta/lib/oeqa/core/tests/test_decorators.py | 77 +++--
 meta/lib/oeqa/core/tests/test_loader.py | 25 --
 9 files changed, 103 insertions(+), 138 deletions(-)
 delete mode 100644 meta/lib/oeqa/core/decorator/oetag.py

diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
index 68819cc338..14fc6a54f4 100644
--- a/meta/lib/oeqa/core/context.py
+++ b/meta/lib/oeqa/core/context.py
@@ -64,12 +64,12 @@ class OETestContext(object):
 setattr(tclass, 'setUpHooker', skipfuncgen('Skip by the 
command line argument "%s"' % skip))
 
 def loadTests(self, module_paths, modules=[], tests=[],
-modules_manifest="", modules_required=[], filters={}):
+modules_manifest="", modules_required=[], **kwargs):
 if modules_manifest:
 modules = self._read_modules_from_manifest(modules_manifest)
 
 self.loader = self.loaderClass(self, module_paths, modules, tests,
-modules_required, filters)
+modules_required, **kwargs)
 self.suites = self.loader.discover()
 
 def runTests(self, processes=None, skips=[]):
diff --git a/meta/lib/oeqa/core/decorator/__init__.py 
b/meta/lib/oeqa/core/decorator/__init__.py
index 923b218266..1a5ac40134 100644
--- a/meta/lib/oeqa/core/decorator/__init__.py
+++ b/meta/lib/oeqa/core/decorator/__init__.py
@@ -6,6 +6,7 @@
 
 from functools import wraps
 from abc import abstractmethod, ABCMeta
+from oeqa.core.utils.misc import strToList
 
 decoratorClasses = set()
 
@@ -63,12 +64,15 @@ class OETestDiscover(OETestDecorator):
 def discover(registry):
 return registry['cases']
 
-class OETestFilter(OETestDecorator):
+def OETestTag(*tags):
+expandedtags = []
+for tag in tags:
+expandedtags += strToList(tag)
+def decorator(item):
+if hasattr(item, "__oeqa_testtags"):
+item.__oeqa_testtags += expandedtags
+else:
+item.__oeqa_testtags = expandedtags
+return item
+return decorator
 
-# OETestLoader call it while loading the tests
-# in loadTestsFromTestCase method, it needs to
-# return a bool, True if needs to be filtered.
-# This method must consume the filter used.
-@abstractmethod
-def filtrate(self, filters):
-return False
diff --git a/meta/lib/oeqa/core/decorator/oetag.py 
b/meta/lib/oeqa/core/decorator/oetag.py
deleted file mode 100644
index 8c31138dac..00
--- a/meta/lib/oeqa/core/decorator/oetag.py
+++ /dev/null
@@ -1,27 +0,0 @@
-#
-# Copyright (C) 2016 Intel Corporation
-#
-# SPDX-License-Identifier: MIT
-#
-
-from . import OETestFilter, registerDecorator
-from oeqa.core.utils.misc import strToList
-
-def _tagFilter(tags, filters):
-return False if set(tags) & set(filters) else True
-
-@registerDecorator
-class OETestTag(OETestFilter):
-attrs = ('oetag',)
-
-def bind(self, registry, case):
-super(OETestTag, self).bind(registry, case)
-self.oetag = strToList(self.oetag, 'oetag')
-
-def filtrate(self, filters):
-if filters.get('oetag'):
-filterx = strToList(filters['oetag'], 'oetag')
-del filters['oetag']
-if _tagFilter(self.oetag, filterx):
-return True
-return False
diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
index 7fea0585c7..0d7970d49e 100644
--- a/meta/lib/oeqa/core/loader.py
+++ b/meta/lib/oeqa/core/loader.py
@@ -16,7 +16,7 @@ from oeqa.core.utils.test import getSuiteModules, getCaseID
 from oeqa.core.exception import OEQATestNotFound
 from oeqa.core.case impor

[OE-core] [PATCH v2 00/12] Add gnu testsuite execution for OEQA

[Nathan Rossi]

This series adds support to execute the gnu test suites for binutils,
gcc and glibc. With the intention for enabling automated test running of
these test suites within the OEQA framework such that they can be
executed by the Yocto Autobuilder.

The test suites covered need significant resources or build artifacts
such that running them on the target is undesirable which rules out the
use of ptest. Because of this the test suites can be run on the build
host and call out to the target for execution (via ssh or using qemu
usermode).

The following implementation adds a do_check task to gcc-runtime, and
creates recipes for binutils and glibc in order to execute the test
suite on the build host. For binutils the reason for a separate recipe
is due to test suite dependence on target libraries (libgcc, glibc) and
the issues associated with a -cross recipe depending on target
libraries. For glibc the reason for a recipe is due to the dependency
chain (libgcc -> glibc -> libgcc-initial) and the test suite dependence
on libgcc.

Target execution is another important issue specifically since target
execution is slow to extremely slow depending on the physical or
emulated target performance. In order to provide faster execution
performance qemu linux-user is implemented alongside qemu system
emulation (via ssh+nfs). In initial testing qemu linux user vs qemu
system emulation provided performance gains of between 10x to 60x whilst
initially having small pass/fail differences. Further work as covered in
this series reduces the pass/fail differences for gcc/gcc-runtime close
to 0, which can be further reduced by marking known failures with test
result filtering.

However glibc is more strict with its expectations of CPU implementation
as well as syscall behaviour. Additionally glibc expects to be able to
execute OS tools such as 'sh' and 'echo' of which is not easy to provide
in the recipe-sysroot (bindir is not populated into the sysroot). As
such correct test results for glibc rely on execution with qemu system
emulation or on a physical target. This series however still includes
qemu user execution for glibc which can be useful (especially on slow
targets) assuming the known failing tests are excluded.

This series also introduces some OEQA test cases which cover running the
test suites. The test cases are split into binutils, gcc and glibc.
Individual test cases provide execution of the sub-suites of tests
within each target. For example binutils has binutils, gas and ld and
suites which can each be executed independently.

The test cases populate the individual test suite test cases into
testresults similar to ptestresults so that resulttool can analyse them.
Filtering of the failed test cases where expected failures occur is not
included in this series.

The OEQA test cases implement execution on qemu linux user by default.
Subclasses implement qemu system emulation setup and configuration for
running the tests.

Changes in v2:
- Dropped merged patches from series:
  - scripts/lib/resulttool/report.py: Add more result types
  - oeqa/utils/nfs: Add unfs_server function to setup a userspace NFS server
  - binutils: Fix mips patch which changes default emulation
- Added dejagnu recipe to oe-core, imported from meta-oe
- Changes from binutils-cross to binutils-cross-testsuite recipe
- Dropped gold and libiberty test suite execution
- Changes from gcc-cross to only gcc-runtime for gcc/g++ tests
- BUILD_TEST_* to TOOLCHAIN_TEST_*
- New changes to oeqa/core for tagging tests, and patches to fix up
  broken tests in the oeqa/core testsuite
- New changes to oeqa/selftest for '--run-only-tags' and
  '--run-exclude-tags'
- Changes to new selftest tests to populate results like ptestresults
- Mark new selftest tests with 'machine' tag
---

Nathan Rossi (12):
  dejagnu: Add dejagnu for binutils/gcc test suites
  binutils-cross-testsuite: Create recipe for test suite execution
  gcc-runtime: Add do_check task for executing gcc test suites
  gcc-common.inc: Process staging fixme with correct target/native
sysroot
  glibc-testsuite: Create a recipe to implement glibc test suite
  oeqa/core/tests: Skip test_fail_duplicated_module
  oeqa/core/tests: Fix test_data module tests
  oeqa/core: Rework OETestTag and remove unused OETestFilter
  oeqa/selftest: Add test run filtering based on test tags
  oeqa/selftest/binutils: Create selftest case for binutils test suite
  oeqa/selftest/gcc: Create selftest case for gcc test suite
  oeqa/selftest/glibc: Create selftest case for glibc test suite

 meta/lib/oeqa/core/context.py |   4 +-
 meta/lib/oeqa/core/decorator/__init__.py  |  20 ++--
 meta/lib/oeqa/core/decorator/oetag.py |  27 -
 meta/lib/oeqa/core/loader.py  |  61 +++---
 meta/lib/oeqa/core/tests/cases/data.py|   2 +-
 meta/lib/oeqa/core/tests/cases/oetag.py   |  21 +++-
 meta/lib/oeqa/core/tests/common.py   

[OE-core] [PATCH v2 04/12] gcc-common.inc: Process staging fixme with correct target/native sysroot

[Nathan Rossi]

Correct the 'staging_processfixme' call so that target sysroot and
native sysroot paths are corrected when extracting the stashed build
directory. This is required for 'make check' to work correctly due paths
used in configuration and scripts which point at the native sysroot.

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/gcc/gcc-common.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/gcc/gcc-common.inc 
b/meta/recipes-devtools/gcc/gcc-common.inc
index 89273a7745..44cba287f3 100644
--- a/meta/recipes-devtools/gcc/gcc-common.inc
+++ b/meta/recipes-devtools/gcc/gcc-common.inc
@@ -17,7 +17,7 @@ python extract_stashed_builddir () {
 src = 
d.expand("${COMPONENTS_DIR}/${BUILD_ARCH}/gcc-stashed-builddir-${TARGET_SYS}")
 dest = d.getVar("B")
 oe.path.copyhardlinktree(src, dest)
-staging_processfixme([src + "/fixmepath"], dest, dest, dest, d)
+staging_processfixme([src + "/fixmepath"], dest, 
d.getVar("RECIPE_SYSROOT"), d.getVar("RECIPE_SYSROOT_NATIVE"), d)
 }
 
 def get_gcc_float_setting(bb, d):
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 05/12] glibc-testsuite: Create a recipe to implement glibc test suite

[Nathan Rossi]

A recipe needs to be created for the test suite due to the dependency
chain between libgcc -> glibc -> libgcc-initial, and the requirements of
the test suite to have libgcc for compilation and execution.

The glibc test suite does not use dejagnu like the gcc test suites do.
Instead a test wrapper script is used along with the assumed dependency
of having the same filesystem available on build host and target. For
qemu linux-user the same filesystem is inherently available, for remote
targets NFS is used. Separate test wrapper scripts are created for qemu
linux-user or ssh targets, with the same TOOLCHAIN_TEST_* variables used for
configuration.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- Renamed BUILD_TEST_* to TOOLCHAIN_TEST_*
---
 meta/recipes-core/glibc/glibc-testsuite_2.30.bb  | 51 +
 meta/recipes-core/glibc/glibc/check-test-wrapper | 71 
 2 files changed, 122 insertions(+)
 create mode 100644 meta/recipes-core/glibc/glibc-testsuite_2.30.bb
 create mode 100644 meta/recipes-core/glibc/glibc/check-test-wrapper

diff --git a/meta/recipes-core/glibc/glibc-testsuite_2.30.bb 
b/meta/recipes-core/glibc/glibc-testsuite_2.30.bb
new file mode 100644
index 00..88764d9e2b
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc-testsuite_2.30.bb
@@ -0,0 +1,51 @@
+require glibc_${PV}.bb
+
+# handle PN differences
+FILESEXTRAPATHS_prepend := "${THISDIR}/glibc:"
+
+# strip provides
+PROVIDES = ""
+# setup depends
+INHIBIT_DEFAULT_DEPS = ""
+
+DEPENDS += "glibc-locale libgcc gcc-runtime"
+
+# remove the initial depends
+DEPENDS_remove = "libgcc-initial"
+
+inherit qemu
+
+SRC_URI += "file://check-test-wrapper"
+
+DEPENDS += "${@'qemu-native' if d.getVar('TOOLCHAIN_TEST_TARGET') == 'user' 
else ''}"
+
+TOOLCHAIN_TEST_TARGET ??= "user"
+TOOLCHAIN_TEST_HOST ??= "localhost"
+TOOLCHAIN_TEST_HOST_USER ??= "root"
+TOOLCHAIN_TEST_HOST_PORT ??= ""
+
+do_check[dirs] += "${B}"
+do_check[nostamp] = "1"
+do_check () {
+chmod 0755 ${WORKDIR}/check-test-wrapper
+
+# clean out previous test results
+oe_runmake tests-clean
+# makefiles don't clean entirely (and also sometimes fails due to too many 
args)
+find ${B} -type f -name "*.out" -delete
+find ${B} -type f -name "*.test-result" -delete
+find ${B}/catgets -name "*.cat" -delete
+find ${B}/conform -name "symlist-*" -delete
+[ ! -e ${B}/timezone/testdata ] || rm -rf ${B}/timezone/testdata
+
+oe_runmake -i \
+QEMU_SYSROOT="${RECIPE_SYSROOT}" \
+QEMU_OPTIONS="${@qemu_target_binary(d)} ${QEMU_OPTIONS}" \
+SSH_HOST="${TOOLCHAIN_TEST_HOST}" \
+SSH_HOST_USER="${TOOLCHAIN_TEST_HOST_USER}" \
+SSH_HOST_PORT="${TOOLCHAIN_TEST_HOST_PORT}" \
+test-wrapper="${WORKDIR}/check-test-wrapper ${TOOLCHAIN_TEST_TARGET}" \
+check
+}
+addtask do_check after do_compile
+
diff --git a/meta/recipes-core/glibc/glibc/check-test-wrapper 
b/meta/recipes-core/glibc/glibc/check-test-wrapper
new file mode 100644
index 00..f8e04e02d2
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc/check-test-wrapper
@@ -0,0 +1,71 @@
+#!/usr/bin/env python3
+import sys
+import os
+import subprocess
+
+env = os.environ.copy()
+args = sys.argv[1:]
+targettype = args.pop(0)
+
+if targettype == "user":
+qemuargs = os.environ.get("QEMU_OPTIONS", "").split()
+if not os.path.exists(qemuargs[0]):
+# ensure qemu args has a valid absolute path
+for i in os.environ.get("PATH", "").split(":"):
+if os.path.exists(os.path.join(i, qemuargs[0])):
+qemuargs[0] = os.path.join(i, qemuargs[0])
+break
+sysroot = os.environ.get("QEMU_SYSROOT", None)
+if not sysroot:
+sys.exit(-1)
+libpaths = [sysroot + "/usr/lib", sysroot + "/lib"]
+
+if args[0] == "env":
+args.pop(0)
+if len(args) == 0:
+args = ["env"]
+else:
+# process options
+while args[0].startswith("-"):
+opt = args.pop(0).lstrip("-")
+if "i" in opt:
+env.clear()
+# process environment vars
+while "=" in args[0]:
+key, val = args.pop(0).split("=", 1)
+if key == "LD_LIBRARY_PATH":
+libpaths += val.split(":")
+else:
+env[key] = val
+if args[0] == "cp":
+# ignore copies, the filesystem is the same
+sys.exit(0)
+
+qemuargs += ["-L", sysroo

[OE-core] [PATCH v2 03/12] gcc-runtime: Add do_check task for executing gcc test suites

[Nathan Rossi]

Add a do_check task to implement execution of the gcc component test
suites. The component test suites require execution of compiled programs
on the target.

The implementation provided allows for execution testing against a host
via SSH or within the local build environment using qemu linux-user
execution. The selection of execution is done via the
TOOLCHAIN_TEST_TARGET variable, and configuration of the remote host is
done with the TOOLCHAIN_TEST_HOST, TOOLCHAIN_TEST_HOST_USER and
TOOLCHAIN_TEST_HOST_PORT variables.

By default the do_check task will execute all check targets, this can be
changed by setting MAKE_CHECK_TARGETS to the desired test suite target
(e.g. check-gcc or check-target-libatomic).

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- Dropped gcc-cross changes, all execution from gcc-runtime. Resolves
  potential issues with -cross depending on target libraries/etc.
- Due to merge, running gcc and g++ test suites separately is no longer
  supported through MAKE_CHECK_TARGETS.
- Changed BUILD_TEST_ to TOOLCHAIN_TEST_
- Added workaround for issue with configargs.h being modified by
  builddir extraction which causes issues with plugins.
---
 meta/recipes-devtools/gcc/gcc-runtime.inc   |  42 +++
 meta/recipes-devtools/gcc/gcc-testsuite.inc | 106 
 2 files changed, 148 insertions(+)
 create mode 100644 meta/recipes-devtools/gcc/gcc-testsuite.inc

diff --git a/meta/recipes-devtools/gcc/gcc-runtime.inc 
b/meta/recipes-devtools/gcc/gcc-runtime.inc
index 22c1d78dd1..2da3c02ef0 100644
--- a/meta/recipes-devtools/gcc/gcc-runtime.inc
+++ b/meta/recipes-devtools/gcc/gcc-runtime.inc
@@ -277,3 +277,45 @@ FILES_libitm-dev = "\
 SUMMARY_libitm-dev = "GNU transactional memory support library - development 
files"
 FILES_libitm-staticdev = "${libdir}/libitm.a"
 SUMMARY_libitm-staticdev = "GNU transactional memory support library - static 
development files"
+
+require gcc-testsuite.inc
+
+EXTRA_OEMAKE_prepend_task-check = "${PARALLEL_MAKE} "
+
+MAKE_CHECK_TARGETS ??= "check-gcc ${@" ".join("check-target-" + i for i in 
d.getVar("RUNTIMETARGET").split())}"
+# prettyprinters and xmethods require gdb tooling
+MAKE_CHECK_IGNORE ??= "prettyprinters.exp xmethods.exp"
+MAKE_CHECK_RUNTESTFLAGS ??= "${MAKE_CHECK_BOARDARGS} --ignore 
'${MAKE_CHECK_IGNORE}'"
+
+# specific host and target dependencies required for test suite running
+do_check[depends] += "dejagnu-native:do_populate_sysroot 
expect-native:do_populate_sysroot"
+do_check[depends] += "virtual/libc:do_populate_sysroot"
+# only depend on qemu if targeting linux user execution
+do_check[depends] += "${@'qemu-native:do_populate_sysroot' if "user" in 
d.getVar('TOOLCHAIN_TEST_TARGET') else ''}"
+# extend the recipe sysroot to include the built libraries (for qemu usermode)
+do_check[prefuncs] += "extend_recipe_sysroot"
+do_check[prefuncs] += "check_prepare"
+do_check[dirs] = "${WORKDIR}/dejagnu ${B}"
+do_check[nostamp] = "1"
+do_check() {
+export DEJAGNU="${WORKDIR}/dejagnu/site.exp"
+
+# HACK: this works around the configure setting CXX with -nostd* args
+sed -i 's/-nostdinc++ -nostdlib++//g' $(find ${B} -name testsuite_flags | 
head -1)
+# HACK: this works around the de-stashing changes to configargs.h, as well 
as recipe-sysroot changing the content
+sed -i '/static const char configuration_arguments/d' ${B}/gcc/configargs.h
+${CC} -v 2>&1 | grep "^Configured with:" | \
+sed 's/Configured with: \(.*\)/static const char 
configuration_arguments[] = "\1";/g' >> ${B}/gcc/configargs.h
+
+if [ "${TOOLCHAIN_TEST_TARGET}" = "user" ]; then
+# qemu user has issues allocating large amounts of memory
+export G_SLICE=always-malloc
+# no test should need more that 10G of memory, this prevents tests 
like pthread7-rope from leaking memory
+ulimit -m 4194304
+ulimit -v 10485760
+fi
+
+oe_runmake -i ${MAKE_CHECK_TARGETS} 
RUNTESTFLAGS="${MAKE_CHECK_RUNTESTFLAGS}"
+}
+addtask check after do_compile do_populate_sysroot
+
diff --git a/meta/recipes-devtools/gcc/gcc-testsuite.inc 
b/meta/recipes-devtools/gcc/gcc-testsuite.inc
new file mode 100644
index 00..b383a358d8
--- /dev/null
+++ b/meta/recipes-devtools/gcc/gcc-testsuite.inc
@@ -0,0 +1,106 @@
+inherit qemu
+
+TOOLCHAIN_TEST_TARGET ??= "user"
+TOOLCHAIN_TEST_HOST ??= "localhost"
+TOOLCHAIN_TEST_HOST_USER ??= "root"
+TOOLCHAIN_TEST_HOST_PORT ??= ""
+
+MAKE_CHECK_BOARDFLAGS ??= ""
+MAKE_CHECK_BOARDARGS ??= 
"--target_board=${TOOLCHAIN_TEST_TARGET}${MAKE_CHECK_BOARDFLAGS}"
+
+python () {
+# Provide the t

[OE-core] [PATCH v2 06/12] oeqa/core/tests: Skip test_fail_duplicated_module

[Nathan Rossi]

At some point the modules in tests/cases/loader/invalid/ were removed,
this test relied on having tests/cases/loader/invalid/oetag.py to
trigger a ImportError in the loader module. Skip this test.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- New
---
 meta/lib/oeqa/core/tests/test_loader.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/lib/oeqa/core/tests/test_loader.py 
b/meta/lib/oeqa/core/tests/test_loader.py
index 519ba96091..e73c91b141 100755
--- a/meta/lib/oeqa/core/tests/test_loader.py
+++ b/meta/lib/oeqa/core/tests/test_loader.py
@@ -40,6 +40,7 @@ class TestLoader(TestBase):
 result = True if expect in str(e) else False
 self.assertTrue(result, msg=msg)
 
+@unittest.skip("invalid directory is missing oetag.py")
 def test_fail_duplicated_module(self):
 cases_path = self.cases_path
 invalid_path = os.path.join(cases_path, 'loader', 'invalid')
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 01/12] dejagnu: Add dejagnu for binutils/gcc test suites

[Nathan Rossi]

The binutils and gcc test suites use dejagnu for execution. Copy the
dejagnu recipe from meta-oe, and update it to 1.6.2.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- New
---
 meta/recipes-devtools/dejagnu/dejagnu_1.6.2.bb | 15 +++
 1 file changed, 15 insertions(+)
 create mode 100644 meta/recipes-devtools/dejagnu/dejagnu_1.6.2.bb

diff --git a/meta/recipes-devtools/dejagnu/dejagnu_1.6.2.bb 
b/meta/recipes-devtools/dejagnu/dejagnu_1.6.2.bb
new file mode 100644
index 00..7dc4e417f6
--- /dev/null
+++ b/meta/recipes-devtools/dejagnu/dejagnu_1.6.2.bb
@@ -0,0 +1,15 @@
+SUMMARY = "GNU unit testing framework, written in Expect and Tcl"
+LICENSE = "GPLv2"
+LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
+SECTION = "devel"
+
+DEPENDS += "expect-native"
+
+inherit autotools
+
+SRC_URI = "${GNU_MIRROR}/${BPN}/${BP}.tar.gz"
+
+SRC_URI[md5sum] = "e1b07516533f351b3aba3423fafeffd6"
+SRC_URI[sha256sum] = 
"0d0671e1b45189c5fc8ade4b3b01635fb9eeab45cf54f57db23e4c4c1a17d261"
+
+BBCLASSEXTEND = "native"
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 02/12] binutils-cross-testsuite: Create recipe for test suite execution

[Nathan Rossi]

Create the do_check task in a new recipe 'binutils-cross-testsuite'.
This recipe is built within a target recipe (not -cross) to ensure
correct testing against target specific libraries/etc. The do_check task
is used to execute the binutils test suite for the cross target
binutils. By default this executes tests for binutils, gas and ld. This
can however be changed by setting CHECK_TARGETS to the desired test
suite target (e.g. 'gas').

The binutils test suites do not require any target execution, as such
the check task can be run without QEMU or a target device. However
since the binutils tests do rely on a C compiler there is dependence on
both gcc and libc in order to run the tests.

Signed-off-by: Nathan Rossi 
---
Changes in v2:
- Moved implementation of do_check from binutils-cross to
  binutils-cross-testsuite. This is to fix issues with target dependency
  from a cross recipe.
- Execute dejagnu directly instead of running via make
- Drop support for executing the gold and libiberty testsuites, the gold
  suite was not executing correctly and the -cross variant of libiberty
  is not used by target or -native
- No longer dependent on build directory tools, using as,ld,* from
  sysroot
- (mips64) Hack/workaround gas defaulting to n32 despite ld defaulting
  to 64
---
 .../binutils/binutils-cross-testsuite_2.32.bb  | 83 ++
 1 file changed, 83 insertions(+)
 create mode 100644 
meta/recipes-devtools/binutils/binutils-cross-testsuite_2.32.bb

diff --git a/meta/recipes-devtools/binutils/binutils-cross-testsuite_2.32.bb 
b/meta/recipes-devtools/binutils/binutils-cross-testsuite_2.32.bb
new file mode 100644
index 00..e62e64e8ae
--- /dev/null
+++ b/meta/recipes-devtools/binutils/binutils-cross-testsuite_2.32.bb
@@ -0,0 +1,83 @@
+require binutils.inc
+require binutils-${PV}.inc
+
+BPN = "binutils"
+
+DEPENDS += "dejagnu-native expect-native"
+DEPENDS += "binutils-native"
+
+deltask do_compile
+deltask do_install
+
+do_configure[dirs] += "${B}/ld ${B}/bfd"
+do_configure() {
+# create config.h, oe enables initfini-array by default
+echo "#define HAVE_INITFINI_ARRAY" > ${B}/ld/config.h
+# use the bfd_stdint.h from binutils-native, this is the same of the one
+# generated by binutils-cross
+cp ${RECIPE_SYSROOT_NATIVE}/usr/include/bfd_stdint.h ${B}/bfd/
+}
+
+# target depends
+DEPENDS += "virtual/${MLPREFIX}${TARGET_PREFIX}binutils"
+DEPENDS += "virtual/${MLPREFIX}${TARGET_PREFIX}gcc"
+DEPENDS += "virtual/${MLPREFIX}${TARGET_PREFIX}compilerlibs"
+DEPENDS += "virtual/${MLPREFIX}libc"
+
+python check_prepare() {
+def suffix_sys(sys):
+if sys.endswith("-linux"):
+return sys + "-gnu"
+return sys
+
+def generate_site_exp(d, suite):
+content = []
+content.append('set srcdir "{0}/{1}"'.format(d.getVar("S"), suite))
+content.append('set objdir "{0}/{1}"'.format(d.getVar("B"), suite))
+content.append('set build_alias "{0}"'.format(d.getVar("BUILD_SYS")))
+content.append('set build_triplet {0}'.format(d.getVar("BUILD_SYS")))
+# use BUILD here since HOST=TARGET
+content.append('set host_alias "{0}"'.format(d.getVar("BUILD_SYS")))
+content.append('set host_triplet {0}'.format(d.getVar("BUILD_SYS")))
+content.append('set target_alias "{0}"'.format(d.getVar("TARGET_SYS")))
+content.append('set target_triplet 
{0}'.format(suffix_sys(d.getVar("TARGET_SYS"
+content.append("set development true")
+content.append("set experimental false")
+
+content.append(d.expand('set CXXFILT "${TARGET_PREFIX}c++filt"'))
+content.append(d.expand('set CC "${TARGET_PREFIX}gcc 
--sysroot=${STAGING_DIR_TARGET} ${TUNE_CCARGS}"'))
+content.append(d.expand('set CXX "${TARGET_PREFIX}g++ 
--sysroot=${STAGING_DIR_TARGET} ${TUNE_CCARGS}"'))
+content.append(d.expand('set CFLAGS_FOR_TARGET 
"--sysroot=${STAGING_DIR_TARGET} ${TUNE_CCARGS}"'))
+
+if suite == "ld" and d.getVar("TUNE_ARCH") == "mips64":
+# oe patches binutils to have the default mips64 abi as 64bit, but
+# skips gas causing issues with the ld test suite (which uses gas)
+content.append('set ASFLAGS "-64"')
+
+return "\n".join(content)
+
+for i in ["binutils", "gas", "ld"]:
+builddir = os.path.join(d.getVar("B"), i)
+if not os.path.isdir(builddir):
+os.makedirs(buil

[OE-core] [PATCH 9/11] oeqa/utils/nfs: Add unfs_server function to setup a userspace NFS server

[Nathan Rossi]

Add a nfs module into oeqa utils. This module provides unfs_server which
allows a test case to build unfs3-native and setup the unfs server on a
target directory of the host. This directory is then shared and can be
mounted by the host or a target device attached to the host (e.g. qemu
via tap or slirp). The nfs server is setup over UDP and automatically
assigns user privileged ports. The function provides the UDP ports for
the server as part of a returned python contextmanager which handles
cleanup of the server process on completion or exception.

Also add a 'udp' arg to get_free_port to get a free UDP port.

Note: unfs3 still requires the host to have rpcbind or portmap running.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/utils/network.py |  4 ++--
 meta/lib/oeqa/utils/nfs.py | 39 +++
 2 files changed, 41 insertions(+), 2 deletions(-)
 create mode 100644 meta/lib/oeqa/utils/nfs.py

diff --git a/meta/lib/oeqa/utils/network.py b/meta/lib/oeqa/utils/network.py
index 59cbbc4f1b..59d01723a1 100644
--- a/meta/lib/oeqa/utils/network.py
+++ b/meta/lib/oeqa/utils/network.py
@@ -4,8 +4,8 @@
 
 import socket
 
-def get_free_port():
-s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+def get_free_port(udp = False):
+s = socket.socket(socket.AF_INET, socket.SOCK_STREAM if not udp else 
socket.SOCK_DGRAM)
 s.bind(('', 0))
 addr = s.getsockname()
 s.close()
diff --git a/meta/lib/oeqa/utils/nfs.py b/meta/lib/oeqa/utils/nfs.py
new file mode 100644
index 00..a37686c914
--- /dev/null
+++ b/meta/lib/oeqa/utils/nfs.py
@@ -0,0 +1,39 @@
+# SPDX-License-Identifier: MIT
+import os
+import sys
+import tempfile
+import contextlib
+import socket
+from oeqa.utils.commands import bitbake, get_bb_var, Command
+from oeqa.utils.network import get_free_port
+
+@contextlib.contextmanager
+def unfs_server(directory, logger = None):
+unfs_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "unfs3-native")
+if not os.path.exists(os.path.join(unfs_sysroot, "usr", "bin", "unfsd")):
+# build native tool
+bitbake("unfs3-native -c addto_recipe_sysroot")
+
+exports = None
+cmd = None
+try:
+# create the exports file
+with tempfile.NamedTemporaryFile(delete = False) as exports:
+exports.write("{0} 
(rw,no_root_squash,no_all_squash,insecure)\n".format(directory).encode())
+
+# find some ports for the server
+nfsport, mountport = get_free_port(udp = True), get_free_port(udp = 
True)
+
+nenv = dict(os.environ)
+nenv['PATH'] = 
"{0}/sbin:{0}/usr/sbin:{0}/usr/bin:".format(unfs_sysroot) + nenv.get('PATH', '')
+cmd = Command(["unfsd", "-d", "-p", "-N", "-e", exports.name, "-n", 
str(nfsport), "-m", str(mountport)],
+bg = True, env = nenv, output_log = logger)
+cmd.run()
+yield nfsport, mountport
+finally:
+if cmd is not None:
+cmd.stop()
+if exports is not None:
+# clean up exports file
+os.unlink(exports.name)
+
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 10/11] oeqa/selftest/glibc: Create selftest case for glibc test suite

[Nathan Rossi]

Create a oeqa selftest test case to execute the glibc test suite and
report the results. The results are populated into the extraresults
variable of the test case which are written to testresults.json for
resulttool to analyse.

An additional subclass is created to separate the execution with qemu
linux-user and qemu system. The GlibcSelfTestSystemEmulated test case
handles setup of the target image, setup of and NFS server as well as
execution with runqemu.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/glibc.py | 91 +++
 1 file changed, 91 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/glibc.py

diff --git a/meta/lib/oeqa/selftest/cases/glibc.py 
b/meta/lib/oeqa/selftest/cases/glibc.py
new file mode 100644
index 00..5252ebb1b5
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/glibc.py
@@ -0,0 +1,91 @@
+# SPDX-License-Identifier: MIT
+import os
+import contextlib
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, 
Command
+from oeqa.utils.nfs import unfs_server
+
+def parse_values(content):
+for i in content:
+for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", 
"UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]:
+if i.startswith(v + ": "):
+yield i[len(v) + 2:].strip(), v
+break
+
+class GlibcSelfTest(OESelftestTestCase):
+@classmethod
+def setUpClass(cls):
+super().setUpClass()
+if not hasattr(cls.tc, "extraresults"):
+cls.tc.extraresults = {}
+
+def test_glibc(self):
+self.glibc_run_check()
+
+def glibc_run_check(self, ssh = None):
+# configure ssh target
+features = []
+if ssh is not None:
+features.append('BUILD_TEST_TARGET = "ssh"')
+features.append('BUILD_TEST_HOST = "{0}"'.format(ssh))
+features.append('BUILD_TEST_HOST_USER = "root"')
+features.append('BUILD_TEST_HOST_PORT = "22"')
+# force single threaded test execution
+features.append('EGLIBCPARALLELISM_task-check_pn-glibc-testsuite = 
"PARALLELMFLAGS="-j1""')
+self.write_config("\n".join(features))
+
+bitbake("glibc-testsuite -c check")
+
+builddir = get_bb_var("B", "glibc-testsuite")
+
+failed = 0
+with open(os.path.join(builddir, "tests.sum"), "r") as f:
+for test, result in parse_values(f):
+self.tc.extraresults["{}.{}".format(type(self).__name__, 
test)] = {"status" : result}
+if result == "FAIL":
+self.logger.info("failed: '{}'".format(test))
+failed += 1
+self.assertEqual(failed, 0)
+
+class GlibcSelfTestSystemEmulated(GlibcSelfTest):
+default_installed_packages = [
+"glibc-charmaps",
+"libgcc",
+"libstdc++",
+"libatomic",
+"libgomp",
+"python3",
+"python3-pexpect",
+"nfs-utils",
+]
+
+def glibc_run_check(self):
+with contextlib.ExitStack() as s:
+# use the base work dir, as the nfs mount, since the recipe 
directory may not exist
+tmpdir = get_bb_var("BASE_WORKDIR")
+nfsport, mountport = s.enter_context(unfs_server(tmpdir))
+
+# build core-image-minimal with required packages
+features = []
+features.append('IMAGE_FEATURES += "ssh-server-openssh"')
+features.append('CORE_IMAGE_EXTRA_INSTALL += "{0}"'.format(" 
".join(self.default_installed_packages)))
+self.write_config("\n".join(features))
+bitbake("core-image-minimal")
+
+# start runqemu
+qemu = s.enter_context(runqemu("core-image-minimal", runqemuparams 
= "nographic"))
+
+# validate that SSH is working
+status, _ = qemu.run("uname")
+self.assertEqual(status, 0)
+
+# setup nfs mount
+if qemu.run("mkdir -p \"{0}\"".format(tmpdir))[0] != 0:
+raise Exception("Failed to setup NFS mount directory on 
target")
+mountcmd = "mount -o noac,nfsvers=3,port={0},udp,mountport={1} 
\"{2}:{3}\" \"{3}\"".format(nfsport, mountport, qemu.server_ip, tmpdir)
+status, output = qemu.run(mountcmd)
+if status != 0:
+raise Exception("Failed to setup NFS mount on target 
({})".format(repr(output)))
+
+super().glibc_run_check(ssh = qemu.ip)
+
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 11/11] scripts/lib/resulttool/report.py: Add more result types

[Nathan Rossi]

Add additional result types into the dictionary to handle dejagnu style
test results. These include PASS, FAIL, XPASS, XFAIL, UNSUPPORTED,
UNTESTED, UNRESOLVED and ERROR.

Signed-off-by: Nathan Rossi 
---
 scripts/lib/resulttool/report.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/scripts/lib/resulttool/report.py b/scripts/lib/resulttool/report.py
index f706280aa7..8b03717d29 100644
--- a/scripts/lib/resulttool/report.py
+++ b/scripts/lib/resulttool/report.py
@@ -19,9 +19,9 @@ class ResultsTextReport(object):
 self.ptests = {}
 self.ltptests = {}
 self.ltpposixtests = {}
-self.result_types = {'passed': ['PASSED', 'passed'],
- 'failed': ['FAILED', 'failed', 'ERROR', 'error', 
'UNKNOWN'],
- 'skipped': ['SKIPPED', 'skipped']}
+self.result_types = {'passed': ['PASSED', 'passed', 'PASS', 'XFAIL'],
+ 'failed': ['FAILED', 'failed', 'FAIL', 'ERROR', 
'error', 'UNKNOWN', 'XPASS'],
+ 'skipped': ['SKIPPED', 'skipped', 'UNSUPPORTED', 
'UNTESTED', 'UNRESOLVED']}
 
 
 def handle_ptest_result(self, k, status, result, machine):
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 6/11] binutils: Fix mips patch which changes default emulation

[Nathan Rossi]

The patch incorrectly removes 'mips_elf32_ntrad_le_vec' from the
'targ_selvecs' replacing it with duplicate entries for
'mips_elf32_ntrad_be_vec'. Correct this so that the default binutils can
still handle 'mips_elf32_ntrad_le_vec' binaries.

Signed-off-by: Nathan Rossi 
---
 .../0010-Change-default-emulation-for-mips64-linux.patch | 9 +++--
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git 
a/meta/recipes-devtools/binutils/binutils/0010-Change-default-emulation-for-mips64-linux.patch
 
b/meta/recipes-devtools/binutils/binutils/0010-Change-default-emulation-for-mips64-linux.patch
index ba5e4c2ce5..5f4ac72f48 100644
--- 
a/meta/recipes-devtools/binutils/binutils/0010-Change-default-emulation-for-mips64-linux.patch
+++ 
b/meta/recipes-devtools/binutils/binutils/0010-Change-default-emulation-for-mips64-linux.patch
@@ -1,4 +1,4 @@
-From d540e95d05cd7c4b8924ac7b257c14ae0105d0ab Mon Sep 17 00:00:00 2001
+From 958a49749b772660d3bafb80748829cba6bed065 Mon Sep 17 00:00:00 2001
 From: Khem Raj 
 Date: Mon, 2 Mar 2015 01:44:14 +
 Subject: [PATCH 10/15] Change default emulation for mips64*-*-linux
@@ -14,7 +14,7 @@ Signed-off-by: Khem Raj 
  2 files changed, 8 insertions(+), 8 deletions(-)
 
 diff --git a/bfd/config.bfd b/bfd/config.bfd
-index 0e1ddb659c..cc65547588 100644
+index 0e1ddb659c..d4f50f0a8d 100644
 --- a/bfd/config.bfd
 +++ b/bfd/config.bfd
 @@ -919,12 +919,12 @@ case "${targ}" in
@@ -30,7 +30,7 @@ index 0e1ddb659c..cc65547588 100644
 -targ_defvec=mips_elf32_ntrad_be_vec
 -targ_selvecs="mips_elf32_ntrad_le_vec mips_elf32_trad_be_vec 
mips_elf32_trad_le_vec mips_elf64_trad_be_vec mips_elf64_trad_le_vec"
 +targ_defvec=mips_elf64_trad_be_vec
-+targ_selvecs="mips_elf32_ntrad_be_vec mips_elf32_ntrad_be_vec 
mips_elf32_trad_be_vec mips_elf32_trad_le_vec mips_elf64_trad_le_vec"
++targ_selvecs="mips_elf32_ntrad_be_vec mips_elf32_ntrad_le_vec 
mips_elf32_trad_be_vec mips_elf32_trad_le_vec mips_elf64_trad_le_vec"
  ;;
mips*el-*-linux*)
  targ_defvec=mips_elf32_trad_le_vec
@@ -54,6 +54,3 @@ index beba17ef51..917be6f8eb 100644
targ_extra_libpath=$targ_extra_emuls ;;
  mips*el-*-linux-*)targ_emul=elf32ltsmip
targ_extra_emuls="elf32btsmip elf32ltsmipn32 
elf64ltsmip elf32btsmipn32 elf64btsmip"
--- 
-2.20.1
-
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 8/11] oeqa/selftest/gcc: Create selftest case for gcc test suite

[Nathan Rossi]

Create a oeqa selftest test case to execute the gcc test suites and
report the results. The results are populated into the extraresults
variable of the test case which are written to testresults.json for
resulttool to analyse.

An additional subclass is created to separate the execution with qemu
linux-user and qemu system. The GccSelfTestSystemEmulated test case
handles setup of the target image as well as execution with runqemu.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/gcc.py | 107 
 1 file changed, 107 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/gcc.py

diff --git a/meta/lib/oeqa/selftest/cases/gcc.py 
b/meta/lib/oeqa/selftest/cases/gcc.py
new file mode 100644
index 00..c36637d066
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/gcc.py
@@ -0,0 +1,107 @@
+# SPDX-License-Identifier: MIT
+import os
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, 
Command
+
+def parse_values(content):
+for i in content:
+for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", 
"UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]:
+if i.startswith(v + ": "):
+yield i[len(v) + 2:].strip(), v
+break
+
+class GccSelfTest(OESelftestTestCase):
+@classmethod
+def setUpClass(cls):
+super().setUpClass()
+if not hasattr(cls.tc, "extraresults"):
+cls.tc.extraresults = {}
+
+def gcc_runtime_check_skip(self, suite):
+targets = get_bb_var("RUNTIMETARGET", "gcc-runtime").split()
+if suite not in targets:
+self.skipTest("Target does not use {0}".format(suite))
+
+def test_cross_gcc(self):
+self.gcc_cross_run_check("gcc")
+
+def test_cross_gxx(self):
+self.gcc_cross_run_check("g++")
+
+def test_gcc_runtime_libatomic(self):
+self.gcc_runtime_run_check("libatomic")
+
+def test_gcc_runtime_libgomp(self):
+self.gcc_runtime_run_check("libgomp")
+
+def test_gcc_runtime_libstdcxx(self):
+self.gcc_runtime_run_check("libstdc++-v3")
+
+def test_gcc_runtime_libssp(self):
+self.gcc_runtime_check_skip("libssp")
+self.gcc_runtime_run_check("libssp")
+
+def test_gcc_runtime_libitm(self):
+self.gcc_runtime_check_skip("libitm")
+self.gcc_runtime_run_check("libitm")
+
+def gcc_cross_run_check(self, suite):
+return 
self.gcc_run_check("gcc-cross-{0}".format(get_bb_var("TUNE_ARCH")), suite)
+
+def gcc_runtime_run_check(self, suite):
+return self.gcc_run_check("gcc-runtime", suite, target_prefix = 
"check-target-")
+
+def gcc_run_check(self, recipe, suite, target_prefix = "check-", ssh = 
None):
+target = target_prefix + suite.replace("gcc", "gcc").replace("g++", 
"c++")
+
+# configure ssh target
+features = []
+features.append('MAKE_CHECK_TARGETS = "{0}"'.format(target))
+if ssh is not None:
+features.append('BUILD_TEST_TARGET = "ssh"')
+features.append('BUILD_TEST_HOST = "{0}"'.format(ssh))
+features.append('BUILD_TEST_HOST_USER = "root"')
+features.append('BUILD_TEST_HOST_PORT = "22"')
+self.write_config("\n".join(features))
+
+bitbake("{0} -c check".format(recipe))
+
+bb_vars = get_bb_vars(["TUNE_ARCH", "B", "TARGET_SYS"], recipe)
+tune_arch, builddir, target_sys = bb_vars["TUNE_ARCH"], bb_vars["B"], 
bb_vars["TARGET_SYS"]
+
+sumspath = os.path.join(builddir, "gcc", "testsuite", suite, 
"{0}.sum".format(suite))
+if not os.path.exists(sumspath): # check in target dirs
+sumspath = os.path.join(builddir, target_sys, suite, "testsuite", 
"{0}.sum".format(suite))
+if not os.path.exists(sumspath): # handle libstdc++-v3 -> libstdc++
+sumspath = os.path.join(builddir, target_sys, suite, "testsuite", 
"{0}.sum".format(suite.split("-")[0]))
+
+failed = 0
+with open(sumspath, "r") as f:
+for test, result in parse_values(f):
+self.tc.extraresults["{}.{}.{}".format(type(self).__name__, 
suite, test)] = {"status" : result}
+if result == "FAIL":
+self.logger.info("failed: '{}'".format(

[OE-core] [PATCH 7/11] oeqa/selftest/binutils: Create selftest case for binutils test suite

[Nathan Rossi]

Create a oeqa selftest test case to execute the binutils test suites and
report the results. The results are populated into the extraresults
variable of the test case which are written to testresults.json for
resulttool to analyse.

Signed-off-by: Nathan Rossi 
---
 meta/lib/oeqa/selftest/cases/binutils.py | 95 
 1 file changed, 95 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/binutils.py

diff --git a/meta/lib/oeqa/selftest/cases/binutils.py 
b/meta/lib/oeqa/selftest/cases/binutils.py
new file mode 100644
index 00..5cb7e106ac
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/binutils.py
@@ -0,0 +1,95 @@
+# SPDX-License-Identifier: MIT
+import os
+import sys
+import re
+import logging
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars
+
+def parse_values(content, gold = False):
+suffix = ": " if not gold else " "
+for i in content:
+for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", 
"UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]:
+if i.startswith(v + suffix):
+name = i[len(v) + len(suffix):].strip()
+if v == "FAIL" and gold: # clean off exit status on gold
+name = name.split(" (exit status:")[0]
+yield name, v
+break
+
+class BinutilsCrossSelfTest(OESelftestTestCase):
+@classmethod
+def setUpClass(cls):
+super().setUpClass()
+if not hasattr(cls.tc, "extraresults"):
+cls.tc.extraresults = {}
+
+def test_binutils(self):
+self.run_binutils("binutils")
+
+def test_gas(self):
+self.run_binutils("gas")
+
+def test_ld(self):
+self.run_binutils("ld")
+
+def test_gold(self):
+self.run_binutils("gold")
+
+def test_libiberty(self):
+self.run_binutils("libiberty")
+
+def run_binutils(self, suite):
+features = []
+features.append('MAKE_CHECK_TARGETS = "check-{0}"'.format(suite))
+self.write_config("\n".join(features))
+
+tune_arch = get_bb_var("TUNE_ARCH")
+recipe = "binutils-cross-{0}".format(tune_arch)
+bb_vars = get_bb_vars(["B", "TARGET_SYS", "T"], recipe)
+builddir, target_sys, tdir = bb_vars["B"], bb_vars["TARGET_SYS"], 
bb_vars["T"]
+
+bitbake("{0} -c check".format(recipe))
+
+failed = 0
+def add_result(test, result):
+nonlocal failed
+self.tc.extraresults["binutils.{}.{}".format(suite, test)] = 
{"status" : result}
+if result == "FAIL":
+self.logger.info("failed: '{}'".format(test))
+failed += 1
+
+if suite in ["binutils", "gas", "ld"]:
+sumspath = os.path.join(builddir, suite, "{0}.sum".format(suite))
+if not os.path.exists(sumspath):
+sumspath = os.path.join(builddir, suite, "testsuite", 
"{0}.sum".format(suite))
+with open(sumspath, "r") as f:
+for test, result in parse_values(f):
+add_result(test, result)
+elif suite in ["gold"]:
+# gold tests are not dejagnu, so no sums file
+logspath = os.path.join(builddir, suite, "testsuite")
+if os.path.exists(logspath):
+for t in os.listdir(logspath):
+if not t.endswith(".log") or t == "test-suite.log":
+continue
+with open(os.path.join(logspath, t), "r") as f:
+for test, result in parse_values(f, gold = True):
+add_result(test, result)
+else:
+self.skipTest("Target does not use {0}".format(suite))
+elif suite in ["libiberty"]:
+# libiberty tests are not dejagnu, no sums or log files
+logpath = os.path.join(tdir, "log.do_check")
+if os.path.exists(logpath):
+with open(logpath, "r") as f:
+logdata = f.read()
+m = re.search(r"entering 
directory\s+'[^\r\n]+?libiberty/testsuite'.*?$(.*?)" +
+"^[^\r\n]+?leaving 
directory\s+'[^\r\n]+?libiberty/testsuite'.*?$",
+logdata, re.DOTALL | re.MULTILINE | re.IGNORECASE)
+if m is not None:
+for test, result in parse_values(m.group(1).splitlines()):
+add_result(test, result)
+
+self.assertEqual(failed, 0)
+
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/11] gcc-runtime: Add do_check task for executing gcc-runtime test suites

[Nathan Rossi]

Add a do_check task to implement execution of the gcc-runtime component
test suites. The component test suites require execution of compiled
programs on the target, this recipe reuses the same setup as gcc-cross
for setup of the target (either as ssh or qemu linux-user).

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/gcc/gcc-runtime.inc | 37 +++
 1 file changed, 37 insertions(+)

diff --git a/meta/recipes-devtools/gcc/gcc-runtime.inc 
b/meta/recipes-devtools/gcc/gcc-runtime.inc
index 22c1d78dd1..ce0b5e482d 100644
--- a/meta/recipes-devtools/gcc/gcc-runtime.inc
+++ b/meta/recipes-devtools/gcc/gcc-runtime.inc
@@ -277,3 +277,40 @@ FILES_libitm-dev = "\
 SUMMARY_libitm-dev = "GNU transactional memory support library - development 
files"
 FILES_libitm-staticdev = "${libdir}/libitm.a"
 SUMMARY_libitm-staticdev = "GNU transactional memory support library - static 
development files"
+
+require gcc-testsuite.inc
+
+EXTRA_OEMAKE_prepend_task-check = "${PARALLEL_MAKE} "
+
+MAKE_CHECK_TARGETS ??= "${@" ".join("check-target-" + i for i in 
d.getVar("RUNTIMETARGET").split())}"
+MAKE_CHECK_IGNORE ??= "prettyprinters.exp xmethods.exp"
+MAKE_CHECK_RUNTESTFLAGS ??= "${MAKE_CHECK_BOARDARGS} --ignore 
'${MAKE_CHECK_IGNORE}'"
+
+# specific host and target dependencies required for test suite running
+do_check[depends] += "dejagnu-native:do_populate_sysroot 
expect-native:do_populate_sysroot"
+do_check[depends] += "virtual/libc:do_populate_sysroot"
+# only depend on qemu if targeting linux user execution
+do_check[depends] += "${@'qemu-native:do_populate_sysroot' if "user" in 
d.getVar('BUILD_TEST_TARGET') else ''}"
+# extend the recipe sysroot to include the built libraries (for qemu usermode)
+do_check[prefuncs] += "extend_recipe_sysroot"
+do_check[prefuncs] += "check_prepare"
+do_check[dirs] = "${WORKDIR}/dejagnu ${B}"
+do_check[nostamp] = "1"
+do_check() {
+export DEJAGNU="${WORKDIR}/dejagnu/site.exp"
+
+# HACK: this works around the configure setting CXX with -nostd* args
+sed -i 's/-nostdinc++ -nostdlib++//g' $(find ${B} -name testsuite_flags | 
head -1)
+
+if [ "${BUILD_TEST_TARGET}" = "user" ]; then
+# qemu user has issues allocating large amounts of memory
+export G_SLICE=always-malloc
+# no test should need more that 10G of memory, this prevents tests 
like pthread7-rope from leaking memory
+ulimit -m 4194304
+ulimit -v 10485760
+fi
+
+oe_runmake -i ${MAKE_CHECK_TARGETS} 
RUNTESTFLAGS="${MAKE_CHECK_RUNTESTFLAGS}"
+}
+addtask check after do_compile do_populate_sysroot
+
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/11] binutils: Add do_check task for executing binutils test suite

[Nathan Rossi]

Create the do_check task to the binutils-cross include. This task can be
used to execute the binutils test suite for the cross target binutils.
By default this executes all the check targets of the binutils Makefile,
this can however be changed by setting MAKE_CHECK_TARGETS to the desired
test suite target (e.g. check-gas).

The binutils test suites do not require any target execution, as such
the check target can be run without QEMU or a target device. However
since the binutils tests do rely on a C compiler there is dependence on
both gcc and libc in order to run the tests.

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/binutils/binutils-cross.inc | 28 +++
 1 file changed, 28 insertions(+)

diff --git a/meta/recipes-devtools/binutils/binutils-cross.inc 
b/meta/recipes-devtools/binutils/binutils-cross.inc
index 02ec891606..76eb453f0e 100644
--- a/meta/recipes-devtools/binutils/binutils-cross.inc
+++ b/meta/recipes-devtools/binutils/binutils-cross.inc
@@ -36,3 +36,31 @@ do_install () {
rmdir ${D}${STAGING_DIR_NATIVE}${prefix_native}/${libdir}64 || :
rmdir ${D}${STAGING_DIR_NATIVE}${prefix_native}/${prefix} || :
 }
+
+EXTRA_OEMAKE_prepend_task-check = "${PARALLEL_MAKE} "
+MAKE_CHECK_TARGETS ??= "check-binutils check-gas check-gold check-ld 
check-libiberty"
+
+python () {
+# crosssdk deps have different virtual targets
+if bb.data.inherits_class('crosssdk', d):
+d.appendVarFlag("do_check", "depends", " 
virtual/${TARGET_PREFIX}gcc-crosssdk:do_populate_sysroot")
+d.appendVarFlag("do_check", "depends", " 
virtual/nativesdk-${TARGET_PREFIX}compilerlibs:do_populate_sysroot")
+else:
+d.appendVarFlag("do_check", "depends", " 
virtual/${TARGET_PREFIX}gcc:do_populate_sysroot")
+d.appendVarFlag("do_check", "depends", " 
virtual/${TARGET_PREFIX}compilerlibs:do_populate_sysroot")
+}
+
+do_check[depends] += "dejagnu-native:do_populate_sysroot 
expect-native:do_populate_sysroot"
+do_check[depends] += "virtual/libc:do_populate_sysroot"
+do_check[dirs] = "${B}"
+do_check[nostamp] = "1"
+do_check() {
+# need to inject CC and CXX as the target CC and CXX with sysroot
+oe_runmake -i ${MAKE_CHECK_TARGETS} \
+RUNTESTFLAGS=" \
+CC='${TARGET_PREFIX}gcc --sysroot=${STAGING_DIR_TARGET} 
${TUNE_CCARGS}' \
+CXX='${TARGET_PREFIX}g++ --sysroot=${STAGING_DIR_TARGET} 
${TUNE_CCARGS}' \
+"
+}
+addtask check after do_compile
+
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/11] gcc-common.inc: Process staging fixme with correct target/native sysroot

[Nathan Rossi]

Correct the 'staging_processfixme' call so that target sysroot and
native sysroot paths are corrected when extracting the stashed build
directory. This is required for 'make check' to work correctly due paths
used in configuration and scripts which point at the native sysroot.

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/gcc/gcc-common.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/gcc/gcc-common.inc 
b/meta/recipes-devtools/gcc/gcc-common.inc
index 96334e54b4..a8f4b60c32 100644
--- a/meta/recipes-devtools/gcc/gcc-common.inc
+++ b/meta/recipes-devtools/gcc/gcc-common.inc
@@ -17,7 +17,7 @@ python extract_stashed_builddir () {
 src = 
d.expand("${COMPONENTS_DIR}/${BUILD_ARCH}/gcc-stashed-builddir-${TARGET_SYS}")
 dest = d.getVar("B")
 oe.path.copyhardlinktree(src, dest)
-staging_processfixme([src + "/fixmepath"], dest, dest, dest, d)
+staging_processfixme([src + "/fixmepath"], dest, 
d.getVar("RECIPE_SYSROOT"), d.getVar("RECIPE_SYSROOT_NATIVE"), d)
 }
 
 def get_gcc_float_setting(bb, d):
---
2.23.0.rc1
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/11] gcc-cross: Add do_check task for executing gcc test suite

[Nathan Rossi]

Add a do_check task and supporting configuration to implement execution
of the gcc compiler test suite. The test suite requires execution of
compiled programs.

The implementation provided allows for execution testing against a host
via SSH or within the local build environment using qemu linux-user
execution. The selection of execution is done via the BUILD_TEST_TARGET
variable, and configuration of the remote host is done with the
BUILD_TEST_HOST, BUILD_TEST_HOST_USER and BUILD_TEST_HOST_PORT
variables.

By default the do_check task will execute all check targets, this can be
changed by setting MAKE_CHECK_TARGETS to the desired test suite target
(e.g. check-gcc or check-g++).

Signed-off-by: Nathan Rossi 
---
 meta/recipes-devtools/gcc/gcc-cross.inc |  41 +++
 meta/recipes-devtools/gcc/gcc-testsuite.inc | 106 
 2 files changed, 147 insertions(+)
 create mode 100644 meta/recipes-devtools/gcc/gcc-testsuite.inc

diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc 
b/meta/recipes-devtools/gcc/gcc-cross.inc
index 6222c2e8c9..f9534340d2 100644
--- a/meta/recipes-devtools/gcc/gcc-cross.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross.inc
@@ -221,3 +221,44 @@ python do_gcc_stash_builddir_setscene () {
 sstate_setscene(d)
 }
 addtask do_gcc_stash_builddir_setscene
+
+require gcc-testsuite.inc
+
+check_prepare_sysroot () {
+if [ ! -s ${RECIPE_SYSROOT}${target_includedir}/limits.h ]; then
+# this file was created by the configure task, but is replaced by the
+# libc version when populating the sysroot for the do_check task
+rm ${RECIPE_SYSROOT}${target_includedir}/limits.h
+fi
+}
+
+EXTRA_OEMAKE_prepend_task-check = "${PARALLEL_MAKE} "
+
+MAKE_CHECK_TARGETS ??= "check-gcc check-g++ check-lto"
+
+python () {
+# crosssdk deps have different virtual targets
+if bb.data.inherits_class('crosssdk', d):
+d.appendVarFlag("do_check", "depends", " 
virtual/nativesdk-${TARGET_PREFIX}compilerlibs:do_populate_sysroot")
+else:
+d.appendVarFlag("do_check", "depends", " 
virtual/${TARGET_PREFIX}compilerlibs:do_populate_sysroot")
+}
+
+# specific host and target dependencies required for test suite running
+do_check[depends] += "dejagnu-native:do_populate_sysroot 
expect-native:do_populate_sysroot"
+do_check[depends] += "virtual/libc:do_populate_sysroot"
+# only depend on qemu if targeting linux user execution
+do_check[depends] += "${@'qemu-native:do_populate_sysroot' if "user" in 
d.getVar('BUILD_TEST_TARGET') else ''}"
+# check_prepare_sysroot is before extend in order to perform the limits.h 
removal
+do_check[prefuncs] += "check_prepare_sysroot"
+do_check[prefuncs] += "extend_recipe_sysroot"
+do_check[prefuncs] += "check_prepare"
+do_check[dirs] = "${WORKDIR}/dejagnu ${B}"
+do_check[nostamp] = "1"
+do_check() {
+export DEJAGNU="${WORKDIR}/dejagnu/site.exp"
+
+oe_runmake -i -C ${B}/gcc ${MAKE_CHECK_TARGETS} 
RUNTESTFLAGS="${MAKE_CHECK_BOARDARGS}"
+}
+addtask check after do_compile do_populate_sysroot
+
diff --git a/meta/recipes-devtools/gcc/gcc-testsuite.inc 
b/meta/recipes-devtools/gcc/gcc-testsuite.inc
new file mode 100644
index 00..f9924f2d86
--- /dev/null
+++ b/meta/recipes-devtools/gcc/gcc-testsuite.inc
@@ -0,0 +1,106 @@
+inherit qemu
+
+BUILD_TEST_TARGET ??= "user"
+BUILD_TEST_HOST ??= "localhost"
+BUILD_TEST_HOST_USER ??= "root"
+BUILD_TEST_HOST_PORT ??= ""
+
+MAKE_CHECK_BOARDFLAGS ??= ""
+MAKE_CHECK_BOARDARGS ??= 
"--target_board=${BUILD_TEST_TARGET}${MAKE_CHECK_BOARDFLAGS}"
+
+python () {
+# Provide the targets compiler args via targets options. This allows 
dejagnu to
+# correctly mark incompatible tests as UNSUPPORTED (e.g. needs soft-float
+# but running on hard-float target).
+#
+# These options are called "multilib_flags" within the gcc test suite. Most
+# architectures handle these options in a sensible way such that tests that
+# are incompatible with the provided multilib are marked as UNSUPPORTED.
+#
+# Note: multilib flags are added to the compile command after the args
+# provided by any test (through dg-options), CFLAGS_FOR_TARGET is always
+# added to the compile command before any other args but is not interpted
+# as options like multilib flags.
+#
+# i686, x86-64 and aarch64 are special, since most toolchains built for
+# these targets don't do multilib the tests do not get correctly marked as
+# UNSUPPORTED. More importantly the test suite itself does not handle
+# overriding the multilib flags where it could (like other archs do). As
+# such do not pass the target compiler args for these targets.
+