Re: [OE-core] [PATCH][RESEND 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts

2019-10-13 Thread Richard Purdie
On Sat, 2019-10-12 at 13:03 +, Martin Jansa wrote:
> The following changes since commit
> 59938780e7e776d87146002ea939b185f8704408:
> 
>   build-appliance-image: Update to master head revision (2019-10-09
> 22:28:44 +0100)
> 
> are available in the Git repository at:
> 
>   git://git.openembedded.org/openembedded-core-contrib
> jansa/artifacts
>   
> http://cgit.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifacts
> 
> Martin Jansa (6):
>   image-artifact-names: introduce new bbclass and move some variables
> into it
>   bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in
> the _LINK_NAME variables and change it to hard link
>   kernel-artifact-names.bbclass: use PR instead of PKGR in
> KERNEL_ARTIFACT_NAME
>   kernel.bbclass: imageType without {}
>   kernel.bbclass: drop unnecessary package_get_auto_pr for do_install
>   *-artifact-names: include version only in the artifact links

I tried this on the autobuilder and it seems to break several things:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/423

Looks like do_bootimg fails, as well as the testimage oeqa code being
unable to find the testdata json files.

There could well be further issues beyond that but its hard to tell.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/2] core-image-sato-sdk-ptest: Remove valgrind ptests for riscv

2019-10-13 Thread Richard Purdie
On Sat, 2019-10-12 at 08:48 -0700, Khem Raj wrote:
> ping

It merged but I tweaked the patch as I mentioned:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=f2df235b49ad920911d152f75cfe2b1e2726e9d4

Its not rebasing out but you can drop.

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/5] grub-efi: replace anonymous function with static configuration

2019-10-13 Thread Richard Purdie
On Sun, 2019-09-29 at 13:15 +0300, dbarysh...@gmail.com wrote:
> From: Dmitry Eremin-Solenikov 
> 
> Replace anonymous function setting GRUB_* variables with static
> configuration, since grub-efi.bbclass will use fixed names for grub
> bootloader.
> 
> Signed-off-by: Dmitry Eremin-Solenikov <
> dmitry_eremin-soleni...@mentor.com>
> ---
>  meta/recipes-bsp/grub/grub-efi_2.04.bb | 40 --
> 
>  1 file changed, 18 insertions(+), 22 deletions(-)

This series failed in testing. There were two selftest failures:

2019-10-12 16:27:33,173 - oe-selftest - INFO - RESULTS - 
efibootpartition.GenericEFITest.test_boot_efi: ERROR (2101.45s)
2019-10-12 16:27:33,173 - oe-selftest - INFO - RESULTS - 
distrodata.Distrodata.test_maintainers: FAILED (106.07s)

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/430
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/430
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/430
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/426

Also shim appears to have build issues during deployment:

https://autobuilder.yoctoproject.org/typhoon/#/builders/52/builds/1114
https://autobuilder.yoctoproject.org/typhoon/#/builders/108/builds/23

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/2] u-boot: Bump from 2019.07 to 2019.10

2019-10-13 Thread Richard Purdie
On Tue, 2019-10-08 at 14:45 -0700, Alistair Francis wrote:
> Signed-off-by: Alistair Francis 
> ---
>  .../u-boot/files/0001-CVE-2019-13103.patch| 69 ---
> 
>  .../u-boot/files/0002-CVE-2019-13104.patch| 49 -
>  .../u-boot/files/0003-CVE-2019-13105.patch| 37 --
>  .../u-boot/files/0004-CVE-2019-13106.patch| 56 ---
>  .../0005-CVE-2019-14192-14193-14199.patch | 43 
>  ...-14197-14200-14201-14202-14203-14204.patch | 44 
>  .../files/0007-CVE-2019-14194-14198.patch | 42 ---
>  .../u-boot/files/0008-CVE-2019-14195.patch| 42 ---
>  .../u-boot/files/0009-CVE-2019-14196.patch| 48 -
>  meta/recipes-bsp/u-boot/u-boot-common.inc | 13 +---
>  ..._2019.07.bb => u-boot-fw-utils_2019.10.bb} |  0
>  ...ols_2019.07.bb => u-boot-tools_2019.10.bb} |  0
>  .../{u-boot_2019.07.bb => u-boot_2019.10.bb}  |  0

I suspect this breaks on musl:

https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/1135
https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/1137

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] libxvmc:upgrade 1.0.11 -> 1.0.12

2019-10-12 Thread Richard Purdie
On Mon, 2019-10-07 at 14:00 +0800, Zang Ruochen wrote:
> Signed-off-by: Zang Ruochen 
> ---
>  .../xorg-lib/{libxvmc_1.0.11.bb =>
> libxvmc_1.0.12.bb} | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>  rename meta/recipes-graphics/xorg-lib/{libxvmc_1.0.11.bb =>
> libxvmc_1.0.12.bb} (76%)
> 
> diff --git a/meta/recipes-graphics/xorg-lib/libxvmc_1.0.11.bb
> b/meta/recipes-graphics/xorg-lib/libxvmc_1.0.12.bb
> similarity index 76%
> rename from meta/recipes-graphics/xorg-lib/libxvmc_1.0.11.bb
> rename to meta/recipes-graphics/xorg-lib/libxvmc_1.0.12.bb
> index d95f809..29ed0c4 100644
> --- a/meta/recipes-graphics/xorg-lib/libxvmc_1.0.11.bb
> +++ b/meta/recipes-graphics/xorg-lib/libxvmc_1.0.12.bb
> @@ -15,5 +15,5 @@ PE = "1"
>  
>  XORG_PN = "libXvMC"
>  
> -SRC_URI[md5sum] = "707175185a2e0490b8173686c657324f"
> -SRC_URI[sha256sum] =
> "4a2e34d444a683a7c010b01b23cefe2b8043a063ce4dc6a9b855836b5262622d"
> +SRC_URI[md5sum] = "3569ff7f3e26864d986d6a21147eaa58"
> +SRC_URI[sha256sum] =
> "6b3da7977b3f7eaf4f0ac6470ab1e562298d82c4e79077765787963ab7966dcd"

NOTE: recipe xf86-video-intel-2_2.99.917+gitAUTOINC+33ee0c3b21-r0: task 
do_prepare_recipe_sysroot: Started
ERROR: xf86-video-intel-2_2.99.917+gitAUTOINC+33ee0c3b21-r0 
do_prepare_recipe_sysroot: The file /usr/include/X11/extensions/vldXvMC.h is 
installed by both xorgproto and libxvmc, aborting
ERROR: Logfile of failure stored in: 
/home/pokybuild/yocto-worker/genericx86-alt/build/build/tmp/work/core2-32-poky-linux/xf86-video-intel/2_2.99.917+gitAUTOINC+33ee0c3b21-r0/temp/log.do_prepare_recipe_sysroot.44681
NOTE: recipe xf86-video-intel-2_2.99.917+gitAUTOINC+33ee0c3b21-r0: task 
do_prepare_recipe_sysroot: Failed
ERROR: Task 
(/home/pokybuild/yocto-worker/genericx86-alt/build/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb:do_prepare_recipe_sysroot)
 failed with exit code '1'

I think this may need an upgrade of xorgproto at the same time.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 00/23] Update BSD license

2019-10-12 Thread Richard Purdie
On Mon, 2019-10-07 at 13:08 +, Christophe PRIOUZEAU wrote:
> Some recipes use BSD as license while SPDX license reference : BSD-0-
> Clause,
> BSD-1-Clause, BSD-2-Clause, BSD-3-Clause, BSD-4-Clause.
> The goal of this request are to indicate the SPDX license euqivalent
> for recipes
> which indicate "BSD" as license.
> 
> The following changes since commit
> 108ce1447ca5dcb838ef67ef6b9a9d25e993da1d:

Thanks for this. In future if there are more of these I'd like to have
one tweak, specifically the commit message being "Clarify BSD license
variant" rather than "update license" which is a little less specific.

For this series I've just tweaked the commit messages to save reposting
them.

It is good to have these clarified!

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH v2] uninative: Update to 2.7 release

2019-10-11 Thread Richard Purdie
On Fri, 2019-10-11 at 10:26 +0200, Stefan Agner wrote:
> We run several build in parallel using the same downloads folder, it
> seems to me that
> there is a race condition. The history shows the warning several
> times already with
> exit code 127, but in those instances the bulid succeeded 
> 
> What I don't understand is how the update to 2.7 release could affect
> the behavior
> such that it suddenly leads to issues. Any idea?

We use the bitbake fetcher for uninative so there should be locking,
the same as any other download.

Are these builds in parallel on different machines with a shared
network filesystem for downloads or the same physical machine?

Are you using any special MIRRORS/PREMIRRORS?

FWIW we run many builds on the autobuilder and don't see this. I also
haven't seen any other reports of it.

I'm therefore at a loss as to why the locking would seem to be breaking
for you :(

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH] readline-native: Fix builds on tumbleweed

2019-10-09 Thread Richard Purdie
OpenSuse's libreadline has extra symbol information which upsets our uninative
loader as our libreadline is missing symbols with the appropriate versions.

The simplest solution is to add the version information as they're harmless.

Signed-off-by: Richard Purdie 
---
 .../recipes-core/readline/readline-8.0/rl-native.map | 12 
 meta/recipes-core/readline/readline.inc  |  5 +
 2 files changed, 17 insertions(+)
 create mode 100644 meta/recipes-core/readline/readline-8.0/rl-native.map

diff --git a/meta/recipes-core/readline/readline-8.0/rl-native.map 
b/meta/recipes-core/readline/readline-8.0/rl-native.map
new file mode 100644
index 000..5e7d49cdd22
--- /dev/null
+++ b/meta/recipes-core/readline/readline-8.0/rl-native.map
@@ -0,0 +1,12 @@
+READLINE_6.3 {
+rl_change_environment;
+rl_clear_history;
+rl_executing_key;
+rl_executing_keyseq;
+rl_filename_stat_hook;
+rl_history_substr_search_backward;
+rl_history_substr_search_forward;
+rl_input_available_hook;
+rl_print_last_kbd_macro;
+rl_signal_event_hook;
+};
diff --git a/meta/recipes-core/readline/readline.inc 
b/meta/recipes-core/readline/readline.inc
index e9665228dc2..07f54a76f18 100644
--- a/meta/recipes-core/readline/readline.inc
+++ b/meta/recipes-core/readline/readline.inc
@@ -43,3 +43,8 @@ do_install_append () {
 BBCLASSEXTEND = "native nativesdk"
 
 CONFFILES_${PN} += "${sysconfdir}/inputrc"
+
+# OpenSuse injects versions into libreadline leading to conficits between our 
native one and theirs
+# see their spec file for where this is injected. Extra versioning is harmless 
so we just do the same.
+SRC_URI_append_class-native = " file://rl-native.map"
+LDFLAGS_append_class-native = " -Wl,--version-script=${WORKDIR}/rl-native.map"
\ No newline at end of file
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 2/2] image-buildinfo.bbclass: Introduce IMAGE_BUILDINFO_GITDESCRIBE

2019-10-09 Thread Richard Purdie
On Wed, 2019-10-09 at 18:05 +0100, Andrei Gherzan wrote:
> This knob can switch in between having full revisions and having git
> describe references in the build info file. By default it is set to 0
> to
> maintain the old behaviour.
> 
> Signed-off-by: Andrei Gherzan 
> ---
>  meta/classes/image-buildinfo.bbclass | 14 --
>  1 file changed, 12 insertions(+), 2 deletions(-)

I'm not sure we want to even encourage this. We probably do want to
print/store the full hashes as output from this class as they may be
used by CI or in bug reports. The could be unique at the time of
printing but clash later. The output is also dependent on the fetched
tree.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH] scripts/gen-lockedsig-cache: Don't list paths which don't exist

2019-10-09 Thread Richard Purdie
This avoids failures seen on the autobuilder when generating eSDKs
and release sstate copies.

Signed-off-by: Richard Purdie 
---
 scripts/gen-lockedsig-cache | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/gen-lockedsig-cache b/scripts/gen-lockedsig-cache
index 48cb67112fd..9bfae9d8323 100755
--- a/scripts/gen-lockedsig-cache
+++ b/scripts/gen-lockedsig-cache
@@ -24,6 +24,8 @@ def extract_sha(filename):
 # a map from hash to list of file with that hash
 def map_sha_to_files(dir_, prefix, sha_map):
 sstate_prefix_path = dir_ + '/' + prefix + '/'
+if not os.path.exists(sstate_prefix_path):
+return
 sstate_files = os.listdir(sstate_prefix_path)
 for f in sstate_files:
 try:
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] perl: Handle PACKAGES_DYNAMIC for perl-native

2019-10-07 Thread Richard Purdie
On Sun, 2019-10-06 at 08:47 -0700, Khem Raj wrote:
> A perl module recipe extending to provide native version causes target
> perl dependencies to be pulled into native build if the module recipe
> has RDEPENDS_${PN} = "perl-module-" e.g. libxml-sax-base-perl
> recipe.
> 
> The reason is that native bbclass empties out PACKAGES_DYNAMIC and
> perl's PACKAGES_DYNAMIC_class-target is greedy enough to usurp native
> modules as well.
> 
> Eventually we end up with errors like when sstate is used across
> machines
> 
> * ERROR: libxml-sax-base-perl-native different signature for task 
> do_populate_sysroot.sigdata between qemux86copy and qemuarm
> 
> Therefore, to fix this native case needs to handled specially when
> re-assigning module dependencies in split_perl_packages(), where the
> modules are named correctly for native case and have a single dependency
> on perl-native, secondly, PACKAGES_DYNAMIC for target case needs to be
> reined in to spare, -native modules, thirdly, let perl-native take over
> the case for providing native modules
> 
> This will fix several sstate signature errors like above with external
> perl modules providing native variants and having runtime dependencies on
> modules which are provided by perl proper
> 
> Signed-off-by: Khem Raj 
> ---
>  meta/recipes-devtools/perl/perl_5.30.0.bb | 13 +
>  1 file changed, 9 insertions(+), 4 deletions(-)
> 
> diff --git a/meta/recipes-devtools/perl/perl_5.30.0.bb 
> b/meta/recipes-devtools/perl/perl_5.30.0.bb
> index a221bce52b..9614477982 100644
> --- a/meta/recipes-devtools/perl/perl_5.30.0.bb
> +++ b/meta/recipes-devtools/perl/perl_5.30.0.bb
> @@ -265,13 +265,18 @@ python split_perl_packages () {
>  # Read the pre-generated dependency file, and use it to set module 
> dependecies
>  for line in open(d.expand("${WORKDIR}") + 
> '/perl-rdepends.txt').readlines():
>  splitline = line.split()
> -module = splitline[0].replace("RDEPENDS_perl", "RDEPENDS_${PN}")
> -depends = splitline[2].strip('"').replace("perl-module", 
> "${PN}-module")
> +if bb.data.inherits_class('native', d):
> +module = splitline[0] + '-native'
> +depends = "perl-native"
> +else:
> +module = splitline[0].replace("RDEPENDS_perl", "RDEPENDS_${PN}")
> +depends = splitline[2].strip('"').replace("perl-module", 
> "${PN}-module")
>  d.appendVar(d.expand(module), " " + depends)
>  }
>  
> -PACKAGES_DYNAMIC_class-target += "^perl-module-.*"
> -PACKAGES_DYNAMIC_class-nativesdk += "^nativesdk-perl-module-.*"
> +PACKAGES_DYNAMIC_class-native_forcevariable = "^perl-module-.*-native$"
> +PACKAGES_DYNAMIC_class-target = "^perl-module-.*(? +PACKAGES_DYNAMIC_class-nativesdk = "^nativesdk-perl-module-.*"
>  
>  RDEPENDS_${PN}-misc += "perl perl-modules"
>  RDEPENDS_${PN}-pod += "perl"

We should never be using _forcevariable in public repos, let alone OE-
Core. Its a tool of last resort. I guess this is because native.bbclass
is clearing it but we need to find a better way.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] Fix #13560 - Partition numbering is broken for MBR primary partition #4

2019-10-03 Thread Richard Purdie
On Thu, 2019-10-03 at 18:36 +0300, Michael Cooper wrote:
> Bug: When wks describes extra partitions that aren't in the partition
> table (e.g. boot loader) and exactly four primary MBR partitions, the
> last partition gets added to fstab as partition #5 instead of #4.
> 
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13560
> 
> Signed-off-by: Michael Cooper 
> ---
>  scripts/lib/wic/plugins/imager/direct.py | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)

Thanks, I've queued this.

For reference, your patch arrived line wrapped. It was simple enough I
could edit that out by hand (been doing this too long, clearly!).

I also tweaked the shortlog to match our guidelines and tweaked the bug
number reference to the standard [YOCTO #13560]. I just mention these
so you can sort next time, thanks for the patch!

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [master][PATCH] gen-lockedsig-cache: Replace glob lookup with hash to filename lookup

2019-10-03 Thread Richard Purdie
On Wed, 2019-10-02 at 16:25 +0100, richard.pur...@linuxfoundation.org
wrote:
> I've queued such a patch in master-next and am testing, its an
> improvement but I'm worried its not resolving the problem.
> 
> Options are to prune the sstate cache, or to teach the code to
> generate
> the full filenames (would mean refactoring)...
> 
> We could give you access to the autobuilder to help reproduce the
> problem but I think its clear where the delays are...

We've cleaned out the sstate on the autobuilder to try and reduce the
impact of this issue as this patch, whilst helping, didn't solve the
problem.

I'm wondering about a different approach.


Just to elaborate on what teaching the code would entail, as well as
using the 'hashfn' data as part of the hash validate function in from
runqueue.py:

sq_data['hashfn'][tid] = self.rqdata.dataCaches[mc].hashfn[taskfn]

we could inject that data into BB_TASKDEPDATA. If we did that, the
oecore code could probably create the full sstate tasknames for the
tasks it needs using logic extracted out of sstate_checkhashes() from
sstate.bbclass.

This would mean the script wouldn't have to guess at filenames, it
could access them directly.

Worth some further thought...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts

2019-10-03 Thread Richard Purdie
On Thu, 2019-10-03 at 10:27 +0200, Martin Jansa wrote:
> Any feedback on this or does it have to wait for 3.1?

This is the kind of change we need to make early in the release since
it means the release scripts, release engineering and potentially QA
all need to update. It was late enough in M3 and we had enough other
issues that I didn't really want to go here then. Please do remind me
early in 3.1 to get this sorted.

> At least the last patch is a fix for regression already included in
> 3.0.

Thanks, it applied with offset fuzz so I'll queue it.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] testimage.bbclass: Add kernel provider and version to testresult

2019-10-03 Thread Richard Purdie
On Thu, 2019-10-03 at 07:35 +0800, Yeoh Ee Peng wrote:
> In running QA testing, we sometime need to select custom  provider
> for
> virtual/kernel and version. To track the selected virtual/kernel
> provider
> and version used during test, we need to add these information to
> testresult.
> 
> This patch add the virtual/kernel and version into the testresult
> configuration section.
> 
> Example of virtual/kernel and version added to testresult
> configuration:
>- "KERNEL_PROVIDER_VERSION": "linux-intel_4.19"
> 
> Signed-off-by: Yeoh Ee Peng 
> ---
>  meta/classes/testimage.bbclass | 20 ++--
>  1 file changed, 18 insertions(+), 2 deletions(-)

My reply last time this was posted:

http://lists.openembedded.org/pipermail/openembedded-core/2019-September/287138.html

still applies.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [master][PATCH] gen-lockedsig-cache: Replace glob lookup with hash to filename lookup

2019-10-02 Thread richard . purdie
On Wed, 2019-10-02 at 10:56 -0400, Konrad Scherer wrote:
> On 10/2/19 6:17 AM, Richard Purdie wrote:
> > My benchmark before was seeing it spend over 30 minutes in bitbake
> > core-image-minimal:do_populate_sdk_ext on an otherwise idle
> > autobuilder
> > cluster/NAS (35 minutes from a clean tmpdir).
> > 
> > With the patch applied and my above tweak, I saw:
> > 
> > real6m58.120s
> > 
> > and I'd note this was with a full build running on the other
> > workers so
> > the NAS was under load. I could try and get an exact time for the
> > above
> > but didn't really see the point in spending another 30 minutes on
> > it.
> > 
> > This is the time for the whole SDK image, not just the time this
> > script
> > takes but its enough for me to say its a vast improvement! :)
> 
> Great! It was frustrating to develop the code without a good
> reproducer.

Unfortunately I think we may be celebrating prematurely. On an
otherwise unloaded system this looked ok however in real world builds
the sdk creation is still excruciatingly slow.

We don't even need to use the script to understand why:

pokybuild@debian9-ty-2:~$ time ls 
/srv/autobuilder/autobuilder.yoctoproject.org/pub/sstate/c6

real9m3.140s
user0m4.432s
sys 0m5.164s

So we'd expect the script to take 2*9*255 minutes :(

(214054 files in there incidentally, I got 1m49 the second try)

I've cc'd our sysadmin.

> > Konrad: Mind if I squash in the above tweaks?
> 
> Not at all. Sorry that you had to spend time debugging my code. That
> sha extraction code should have been more robust.
> 
> > Also, the new code is a bit too chatty and leads to a lot of log
> > output, we'll need to tweak that too.
> 
> Please do.

I've queued such a patch in master-next and am testing, its an
improvement but I'm worried its not resolving the problem.

Options are to prune the sstate cache, or to teach the code to generate
the full filenames (would mean refactoring)...

We could give you access to the autobuilder to help reproduce the
problem but I think its clear where the delays are...

Cheers,

Richard



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/2] sanity.conf: Bump minimum bitbake version

2019-10-02 Thread Richard Purdie
We need SignatureGeneratorUniHashMixIn from newer bitbake so bump the minimum
version.

Signed-off-by: Richard Purdie 
---
 meta/conf/sanity.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/sanity.conf b/meta/conf/sanity.conf
index 92e1886990f..8b2f6553949 100644
--- a/meta/conf/sanity.conf
+++ b/meta/conf/sanity.conf
@@ -3,7 +3,7 @@
 # See sanity.bbclass
 #
 # Expert users can confirm their sanity with "touch conf/sanity.conf"
-BB_MIN_VERSION = "1.43.1"
+BB_MIN_VERSION = "1.43.2"
 
 SANITY_ABIFILE = "${TMPDIR}/abi_version"
 
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/2] base: Improve module import error message

2019-10-02 Thread Richard Purdie
Turn:

ERROR: Unable to parse Var 
Traceback (most recent call last):
  File "Var ", line 1, in 
  File "/media/build1/poky/meta/classes/base.bbclass", line 35, in 
oe_import(d=):
 for toimport in oe.data.typed_value("OE_IMPORTS", d):
>imported = __import__(toimport)
 inject(toimport.split(".", 1)[0], imported)
  File "/media/build1/poky/meta/lib/oe/sstatesig.py", line 267, in :

>class SignatureGeneratorOEEquivHash(SignatureGeneratorOEBasicHashMixIn, 
bb.siggen.SignatureGeneratorUniHashMixIn, 
bb.siggen.SignatureGeneratorBasicHash):
 name = "OEEquivHash"
bb.data_smart.ExpansionError: Failure expanding variable OE_IMPORTED[:=], 
expression was ${@oe_import(d)} which triggered exception AttributeError: 
module 'bb.siggen' has no attribute 'SignatureGeneratorUniHashMixIn'

into:
ERROR: Error importing OE modules: module 'bb.siggen' has no attribute 
'SignatureGeneratorUniHashMixIn'

which can then trigger a version mismatch error message.

Signed-off-by: Richard Purdie 
---
 meta/classes/base.bbclass | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 0c8a4b28629..d3184ecf7bb 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -32,9 +32,11 @@ def oe_import(d):
 
 import oe.data
 for toimport in oe.data.typed_value("OE_IMPORTS", d):
-imported = __import__(toimport)
-inject(toimport.split(".", 1)[0], imported)
-
+try:
+imported = __import__(toimport)
+inject(toimport.split(".", 1)[0], imported)
+except AttributeError as e:
+bb.error("Error importing OE modules: %s" % str(e))
 return ""
 
 # We need the oe module name space early (before INHERITs get added)
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] layer.conf: Add xserver-nodm-init to SIGGEN_EXCLUDERECIPES_ABISAFE

2019-10-02 Thread Richard Purdie
On Mon, 2019-09-30 at 14:37 -0700, Khem Raj wrote:
> Found signature differences when same machine is used as a copy
> see
> 
> ERROR: xscreensaver different signature for task
> do_package_write_ipk.sigdata between qemux86copy and qemux86
> Hash for dependent task x11-common/xserver-nodm-
> init_3.0.bb:do_packagedata changed from
> de0944d4fcaeed0efdb143a18cc406bd043469ae291de1704a999bc878a7691c to
> ba7bdaf35860ba5bf5a5f4ce06379a77c88eb9806e09a1fc5373933888a46507
> 
> Signed-off-by: Khem Raj 
> ---
>  meta/conf/layer.conf | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
> index 27893b633e..a13c8dc9b2 100644
> --- a/meta/conf/layer.conf
> +++ b/meta/conf/layer.conf
> @@ -44,6 +44,7 @@ SIGGEN_EXCLUDERECIPES_ABISAFE += " \
>opkg-utils \
>gstreamer1.0-meta-base \
>ca-certificates \
> +  xserver-nodm-init \
>  "
>  
>  SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS += " \

This seems odd, why is xscreensaver depending on xserver-nodm-init?

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [kernel-cache][PATCH] bug 13333 Add SPDX license headers to all source files for yocto-kernel-cache

2019-10-02 Thread Richard Purdie
diff --git a/staging/small-ck-stage.scc b/staging/small-ck-stage.scc
index 0508194..328de6e 100644
--- a/staging/small-ck-stage.scc
+++ b/staging/small-ck-stage.scc
@@ -1 +1,2 @@
-include small/small-ck.scc
+# SPDX-License-Identifier: MIT
+include small/small-ck.scc
\ No newline at end of file

It looks very like your patch has stripped all the newlines off the
last lines in the files. I suspect we probably don't want to do that.

Also, please follow the section in the README, "Submitting Patches" as
the mailing list is incorrect.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [master][PATCH] gen-lockedsig-cache: Replace glob lookup with hash to filename lookup

2019-10-02 Thread Richard Purdie
On Fri, 2019-09-27 at 14:56 -0400, Konrad Scherer wrote:
> From: Konrad Scherer 
> 
> Using the glob function to map signatures to sstate files is very
> slow
> when the sstate is large and accessed over nfs. The lookup now only
> loads the necessary prefixes and doesn't use glob as all.
> 
> Unfortunately I don't have access to the systems where the
> performance
> isse was noticed and on my test system the glob is fast enough that
> the performance numbers aren't useful. I could verify that file list
> returned by the new code is the same.
> 
> [YOCTO #13539]
> 
> Signed-off-by: Konrad Scherer 
> ---
>  meta/lib/oe/copy_buildsystem.py |  3 ++-
>  scripts/gen-lockedsig-cache | 44 +
> 
>  2 files changed, 41 insertions(+), 6 deletions(-)

Thanks for this, its hugely appreciated!

I aimed to do some profile measurements to show the difference in speed
but when testing it failed with:

Filtering out gdb-cross-x86_64:do_patch
Filtering out gdb-cross-x86_64:do_prepare_recipe_sysroot
Filtering out gdb-cross-x86_64:do_unpack
Gathering file list
Traceback (most recent call last):
  File 
"/home/pokybuild/yocto-worker/qa-extras/build/scripts/gen-lockedsig-cache", 
line 77, in 
sstate_content_cache[prefix] = build_sha_cache(prefix)
  File 
"/home/pokybuild/yocto-worker/qa-extras/build/scripts/gen-lockedsig-cache", 
line 39, in build_sha_cache
map_sha_to_files(sstate_dir, prefix, sha_map)
  File 
"/home/pokybuild/yocto-worker/qa-extras/build/scripts/gen-lockedsig-cache", 
line 29, in map_sha_to_files
sha = extract_sha(f)
  File 
"/home/pokybuild/yocto-worker/qa-extras/build/scripts/gen-lockedsig-cache", 
line 21, in extract_sha
return filename.split(':')[7].split('_')[0]
IndexError: list index out of range

and then when I fixed that by ignoring files which don't match the pattern:

Gathering file list
Traceback (most recent call last):
  File 
"/home/pokybuild/yocto-worker/qa-extras/build/scripts/gen-lockedsig-cache", 
line 80, in 
sstate_content_cache[prefix] = build_sha_cache(prefix)
  File 
"/home/pokybuild/yocto-worker/qa-extras/build/scripts/gen-lockedsig-cache", 
line 45, in build_sha_cache
map_sha_to_files(native_sstate_dir, prefix, sha_map)
  File 
"/home/pokybuild/yocto-worker/qa-extras/build/scripts/gen-lockedsig-cache", 
line 27, in map_sha_to_files
sstate_files = os.listdir(sstate_prefix_path)
FileNotFoundError: [Errno 2] No such file or directory: 
'/srv/autobuilder/autobuilder.yoctoproject.org/pub/sstateuniversal/b6/'


I therefore added:

diff --git a/scripts/gen-lockedsig-cache b/scripts/gen-lockedsig-cache
index ae5e09d89f..48cb67112f 100755
--- a/scripts/gen-lockedsig-cache
+++ b/scripts/gen-lockedsig-cache
@@ -26,10 +26,13 @@ def map_sha_to_files(dir_, prefix, sha_map):
 sstate_prefix_path = dir_ + '/' + prefix + '/'
 sstate_files = os.listdir(sstate_prefix_path)
 for f in sstate_files:
-sha = extract_sha(f)
-if sha not in sha_map:
-sha_map[sha] = []
-sha_map[sha].append(sstate_prefix_path + f)
+try:
+sha = extract_sha(f)
+if sha not in sha_map:
+sha_map[sha] = []
+sha_map[sha].append(sstate_prefix_path + f)
+except IndexError:
+continue
 
 # given a prefix build a map of hash to list of files
 def build_sha_cache(prefix):
@@ -38,7 +41,7 @@ def build_sha_cache(prefix):
 sstate_dir = sys.argv[2]
 map_sha_to_files(sstate_dir, prefix, sha_map)
 
-native_sstate_dir = sys.argv[2] + sys.argv[4]
+native_sstate_dir = sys.argv[2] + '/' + sys.argv[4]
 map_sha_to_files(native_sstate_dir, prefix, sha_map)
 
 return sha_map


My benchmark before was seeing it spend over 30 minutes in bitbake
core-image-minimal:do_populate_sdk_ext on an otherwise idle autobuilder
cluster/NAS (35 minutes from a clean tmpdir).

With the patch applied and my above tweak, I saw:

real6m58.120s

and I'd note this was with a full build running on the other workers so
the NAS was under load. I could try and get an exact time for the above
but didn't really see the point in spending another 30 minutes on it.

This is the time for the whole SDK image, not just the time this script
takes but its enough for me to say its a vast improvement! :)

Konrad: Mind if I squash in the above tweaks?

Also, the new code is a bit too chatty and leads to a lot of log
output, we'll need to tweak that too.

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH v4] rpm: make rpm work in toolchain.

2019-10-02 Thread Richard Purdie
On Wed, 2019-09-25 at 07:45 +0800, Zheng Ruoqin wrote:
> We need to configure rpm to use package architecture from yocto build
> system.
> 
> Install rpmrc and rpm/platform to ${SDKTARGETSYSROOT} because config
> file in host-sysroot as /opt/poky/2.7+snapshot/sysroots/x86_64-
> pokysdk-linux will be covered by another ARCH which result in
> previous config settings inefficacy.
> 
> To resolve it, put config file in target-sysroot like
> /opt/poky/2.7+snapshot/sysroots/core2-64-poky-linux. As each ARCH has
> its own target-sysroot, config file will not be covered.
> 
> Signed-off-by: Zheng Ruoqin 
> ---
>  meta/recipes-devtools/rpm/files/rpm-setup.py | 27
> 
>  meta/recipes-devtools/rpm/rpm_4.14.2.1.bb| 19 ++
>  2 files changed, 46 insertions(+)
>  create mode 100644 meta/recipes-devtools/rpm/files/rpm-setup.py
> 
> diff --git a/meta/recipes-devtools/rpm/files/rpm-setup.py
> b/meta/recipes-devtools/rpm/files/rpm-setup.py
> new file mode 100644
> index 00..b3e8a1198c
> --- /dev/null
> +++ b/meta/recipes-devtools/rpm/files/rpm-setup.py
> @@ -0,0 +1,27 @@
> +#!/usr/bin/env python3
> +
> +import os
> +import sys
> +import shutil
> +
> +try:
> +native_sysroot = os.environ['OECORE_NATIVE_SYSROOT']
> +sdktarget_sysroot = os.environ['SDKTARGETSYSROOT']
> +except KeyError:
> +print("Not in environment setup, bailing")
> +sys.exit(1)
> +
> +target_etc_dir = os.path.join(sdktarget_sysroot, 'etc/rpm')
> +
> +if not os.path.exists(target_etc_dir):
> +os.makedirs(target_etc_dir)
> +
> +template_file = os.path.join(native_sysroot, 'usr/share/rpm/rpmrc')
> +cross_file = os.path.join(sdktarget_sysroot, 'etc/rpmrc')
> +shutil.copy(template_file, cross_file)
> +
> +template_file = os.path.join(native_sysroot,
> 'usr/share/rpm/platform')
> +cross_file = os.path.join(sdktarget_sysroot, 'etc/rpm/platform')
> +shutil.copy(template_file, cross_file)
> +
> +
> diff --git a/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
> b/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
> index c37330eb4c..e1d1951d74 100644
> --- a/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
> +++ b/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
> @@ -44,6 +44,9 @@ SRC_URI = "git://github.com/rpm-software-
> management/rpm;branch=rpm-4.14.x \
> file://0001-mono-find-provides-requires-do-not-use-
> monodis-from-.patch \
> "
>  
> +SRC_URI_append_class-nativesdk = "file://rpm-setup.py \
> + "
> +
>  PE = "1"
>  SRCREV = "4a9440006398646583f0d9ae1837dad2875013aa"
>  
> @@ -113,6 +116,20 @@ do_install_append_class-nativesdk() {
>  done
>  
>  rm -rf ${D}/var
> +install -d ${D}${datadir}/rpm
> +
> +cat >${D}/${datadir}/rpm/rpmrc < +arch_compat: ${MACHINE_ARCH}: all any noarch ${PACKAGE_EXTRA_ARCHS}
> +EOF
> +
> +# Arch Info should be fixed as '-' is instead of '_'.
> +sed -i 's/-/_/' ${D}${datadir}/rpm/rpmrc
> +cat >${D}/${datadir}/rpm/platform < +${MACHINE_ARCH}-pc-linux
> +EOF

This is heading in the right direction but this patch still makes the
nativesdk-rpm recipe machine specific and we can't do that.

Cheers,

Richard



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] Yocto Project Status WW40’19

2019-10-01 Thread richard . purdie
On Tue, 2019-10-01 at 21:16 +0200, Alexander Kanavin wrote:
> On Tue, 1 Oct 2019 at 16:54, Stephen K Jolley <
> sjolley.yp...@gmail.com> wrote:
> > A significant performance problem has been found on the autobuilder
> > where some builds are scaling in time badly as the sstate cache
> > grows, taking 12 hours or more in some cases. Unfortunately nobody
> > seems motivated to help work on this kind of issue.
> > 
> 
> Wait a moment, I believe there is a patch for this issue?
> http://lists.openembedded.org/pipermail/openembedded-core/2019-September/287445.html

Sorry, that was copied form last week and should have been updated but
I became distracted in completing the blockers list.

There is a patch and the plan is to test that on the autobuilder to
compare performance, I appreciate the patch!

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 3/4] lib/sstatesig: Fix class inheritance problems

2019-09-27 Thread Richard Purdie
The locked sigs class needs to be inherited after the hashequiv mixin so
that get_unihash can correctly wrap the underlying hashequiv function.

To do this turn the locked sigs class into a second mixin, then the order
can be correctly handled. Tweak the get/set_taskdata to match.

Signed-off-by: Richard Purdie 
---
 meta/lib/oe/sstatesig.py | 15 ---
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index 43eb6034e6a..c566ce5a0cb 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -90,8 +90,7 @@ class 
SignatureGeneratorOEBasic(bb.siggen.SignatureGeneratorBasic):
 def rundep_check(self, fn, recipename, task, dep, depname, dataCache = 
None):
 return sstate_rundepfilter(self, fn, recipename, task, dep, depname, 
dataCache)
 
-class SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
-name = "OEBasicHash"
+class SignatureGeneratorOEBasicHashMixIn(object):
 def init_rundepcheck(self, data):
 self.abisaferecipes = (data.getVar("SIGGEN_EXCLUDERECIPES_ABISAFE") or 
"").split()
 self.saferecipedeps = (data.getVar("SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS") 
or "").split()
@@ -129,12 +128,11 @@ class 
SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
 return sstate_rundepfilter(self, fn, recipename, task, dep, depname, 
dataCache)
 
 def get_taskdata(self):
-data = super(bb.siggen.SignatureGeneratorBasicHash, 
self).get_taskdata()
-return (data, self.lockedpnmap, self.lockedhashfn, self.lockedhashes)
+return (self.lockedpnmap, self.lockedhashfn, self.lockedhashes) + 
super().get_taskdata()
 
 def set_taskdata(self, data):
-coredata, self.lockedpnmap, self.lockedhashfn, self.lockedhashes = data
-super(bb.siggen.SignatureGeneratorBasicHash, 
self).set_taskdata(coredata)
+self.lockedpnmap, self.lockedhashfn, self.lockedhashes = data[:3]
+super().set_taskdata(data[3:])
 
 def dump_sigs(self, dataCache, options):
 sigfile = os.getcwd() + "/locked-sigs.inc"
@@ -263,7 +261,10 @@ class 
SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
 if error_msgs:
 bb.fatal("\n".join(error_msgs))
 
-class SignatureGeneratorOEEquivHash(bb.siggen.SignatureGeneratorUniHashMixIn, 
SignatureGeneratorOEBasicHash):
+class SignatureGeneratorOEBasicHash(SignatureGeneratorOEBasicHashMixIn, 
bb.siggen.SignatureGeneratorBasicHash):
+name = "OEBasicHash"
+
+class SignatureGeneratorOEEquivHash(SignatureGeneratorOEBasicHashMixIn, 
bb.siggen.SignatureGeneratorUniHashMixIn, 
bb.siggen.SignatureGeneratorBasicHash):
 name = "OEEquivHash"
 
 def init_rundepcheck(self, data):
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/4] sstatesig: Fix hash equivlanency locked signature issues

2019-09-27 Thread Richard Purdie
Using locked signatures with the hash equivalency server ran into
problems. We need to:

a) Ensure the lockedhashes data object is passed from the core to
   any individual tasks using the get/set_taskdata methods

b) Return a locked singature instead of a unihash

c) Write the unihash being used to locked signature lists rather than
   the calculated taskhash

d) Skip warnings of hash mismatch if the hash is a unihash

These changes fix esdk builds (which use locked sigs) when a hash equivalence
server is in use.

Signed-off-by: Richard Purdie 
---
 meta/lib/oe/sstatesig.py | 14 ++
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index 50d80bf51a1..43eb6034e6a 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -130,10 +130,10 @@ class 
SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
 
 def get_taskdata(self):
 data = super(bb.siggen.SignatureGeneratorBasicHash, 
self).get_taskdata()
-return (data, self.lockedpnmap, self.lockedhashfn)
+return (data, self.lockedpnmap, self.lockedhashfn, self.lockedhashes)
 
 def set_taskdata(self, data):
-coredata, self.lockedpnmap, self.lockedhashfn = data
+coredata, self.lockedpnmap, self.lockedhashfn, self.lockedhashes = data
 super(bb.siggen.SignatureGeneratorBasicHash, 
self).set_taskdata(coredata)
 
 def dump_sigs(self, dataCache, options):
@@ -171,10 +171,11 @@ class 
SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
 h_locked = self.lockedsigs[recipename][task][0]
 var = self.lockedsigs[recipename][task][1]
 self.lockedhashes[tid] = h_locked
+unihash = super().get_unihash(tid)
 self.taskhash[tid] = h_locked
 #bb.warn("Using %s %s %s" % (recipename, task, h))
 
-if h != h_locked:
+if h != h_locked and h_locked != unihash:
 self.mismatch_msgs.append('The %s:%s sig is computed to be 
%s, but the sig is locked to %s in %s'
   % (recipename, task, h, h_locked, 
var))
 
@@ -182,6 +183,11 @@ class 
SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
 #bb.warn("%s %s %s" % (recipename, task, h))
 return h
 
+def get_unihash(self, tid):
+if tid in self.lockedhashes:
+return self.lockedhashes[tid]
+return super().get_unihash(tid)
+
 def dump_sigtask(self, fn, task, stampbase, runtime):
 tid = fn + ":" + task
 if tid in self.lockedhashes:
@@ -211,7 +217,7 @@ class 
SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
 (_, _, task, fn) = bb.runqueue.split_tid_mcfn(tid)
 if tid not in self.taskhash:
 continue
-f.write("" + self.lockedpnmap[fn] + ":" + task + ":" + 
self.taskhash[tid] + " \\\n")
+f.write("" + self.lockedpnmap[fn] + ":" + task + ":" + 
self.get_unihash(tid) + " \\\n")
 f.write('"\n')
 f.write('SIGGEN_LOCKEDSIGS_TYPES_%s = "%s"' % (self.machine, " 
".join(l)))
 
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 4/4] populate_sdk_ext: Fix for hash equiv

2019-09-27 Thread Richard Purdie
Write out the hash equiv cache file into any eSDK so that it doesn't rely
on having to call the hash server for the basic data requests.

Signed-off-by: Richard Purdie 
---
 meta/classes/populate_sdk_ext.bbclass | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/meta/classes/populate_sdk_ext.bbclass 
b/meta/classes/populate_sdk_ext.bbclass
index 086f55df0c1..9fda1c9e78e 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -379,6 +379,11 @@ python copy_buildsystem () {
 f.write('require conf/locked-sigs.inc\n')
 f.write('require conf/unlocked-sigs.inc\n')
 
+if os.path.exists(builddir + '/cache/bb_unihashes.dat'):
+bb.parse.siggen.save_unitaskhashes()
+bb.utils.mkdirhier(os.path.join(baseoutpath, 'cache'))
+shutil.copyfile(builddir + '/cache/bb_unihashes.dat', baseoutpath + 
'/cache/bb_unihashes.dat')
+
 # Write a templateconf.cfg
 with open(baseoutpath + '/conf/templateconf.cfg', 'w') as f:
 f.write('meta/conf\n')
@@ -440,6 +445,11 @@ python copy_buildsystem () {
 else:
 tasklistfn = None
 
+if os.path.exists(builddir + '/cache/bb_unihashes.dat'):
+bb.parse.siggen.save_unitaskhashes()
+bb.utils.mkdirhier(os.path.join(baseoutpath, 'cache'))
+shutil.copyfile(builddir + '/cache/bb_unihashes.dat', baseoutpath + 
'/cache/bb_unihashes.dat')
+
 # Add packagedata if enabled
 if d.getVar('SDK_INCLUDE_PKGDATA') == '1':
 lockedsigs_base = d.getVar('WORKDIR') + '/locked-sigs-base.inc'
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/4] oeqa/selftest/signing: Fix for hash equivlance server

2019-09-27 Thread Richard Purdie
There were two issues with the test one is that an equivalent hash
could come from the server meaning the signature didn't change when it
should. A uuid string is injected to ensure this does not happen.

If there were multiple warnings the test would also fail as only the
first is prefixed with WARNING. Tweak the string to avoid that failure
mode.

Signed-off-by: Richard Purdie 
---
 meta/lib/oeqa/selftest/cases/signing.py | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/signing.py 
b/meta/lib/oeqa/selftest/cases/signing.py
index b390f37d8e6..5c4e01b2c36 100644
--- a/meta/lib/oeqa/selftest/cases/signing.py
+++ b/meta/lib/oeqa/selftest/cases/signing.py
@@ -180,6 +180,8 @@ class LockedSignatures(OESelftestTestCase):
 AutomatedBy: Daniel Istrate 
 """
 
+import uuid
+
 test_recipe = 'ed'
 locked_sigs_file = 'locked-sigs.inc'
 
@@ -197,9 +199,10 @@ class LockedSignatures(OESelftestTestCase):
 bitbake(test_recipe)
 
 # Make a change that should cause the locked task signature to change
+# Use uuid so hash equivalance server isn't triggered
 recipe_append_file = test_recipe + '_' + get_bb_var('PV', test_recipe) 
+ '.bbappend'
 recipe_append_path = os.path.join(self.testlayer_path, 'recipes-test', 
test_recipe, recipe_append_file)
-feature = 'SUMMARY += "test locked signature"\n'
+feature = 'SUMMARY_${PN} = "test locked signature%s"\n' % uuid.uuid4()
 
 os.mkdir(os.path.join(self.testlayer_path, 'recipes-test', 
test_recipe))
 write_file(recipe_append_path, feature)
@@ -210,7 +213,7 @@ class LockedSignatures(OESelftestTestCase):
 ret = bitbake(test_recipe)
 
 # Verify you get the warning and that the real task *isn't* run (i.e. 
the locked signature has worked)
-patt = r'WARNING: The %s:do_package sig is computed to be \S+, but the 
sig is locked to \S+ in SIGGEN_LOCKEDSIGS\S+' % test_recipe
+patt = r'The %s:do_package sig is computed to be \S+, but the sig is 
locked to \S+ in SIGGEN_LOCKEDSIGS\S+' % test_recipe
 found_warn = re.search(patt, ret.output)
 
 self.assertIsNotNone(found_warn, "Didn't find the expected warning 
message. Output: %s" % ret.output)
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] classes/image-live.bbclass: Don't hardcode cpio.gz

2019-09-26 Thread richard . purdie
On Thu, 2019-09-26 at 19:09 +0200, Böszörményi Zoltán wrote:
> 2019. 09. 26. 19:02 keltezéssel, Böszörményi Zoltán via Openembedded-
> core írta:
> > 2019. 09. 26. 18:50 keltezéssel, Böszörményi Zoltán via
> > Openembedded-core írta:
> > > 2019. 09. 26. 17:45 keltezéssel, Richard Purdie írta:
> > > > 
> > > > I'm a little worried that INITRAMFS_FSTYPES can contain
> > > > multiple values
> > > > by the sounds of its name...
> > > 
> > >  From the looks of the current value, it's already contains
> > > multiple values
> > > delimited by that dot. "cpio" + "gz".
> > > 
> > > Also, according to meta/conf/documentation.conf, line 228:
> > > 
> > > INITRAMFS_FSTYPES[doc] = "Defines the format for the output image
> > > of an initial RAM disk 
> > > (initramfs), which is used during boot."
> > 
> > Also, image-live.bbclass uses this variable this way:
> > 
> > INITRD_LIVE ?= "${DEPLOY_DIR_IMAGE}/${INITRD_IMAGE_LIVE}-
> > ${MACHINE}.${INITRAMFS_FSTYPES}"
> > 
> > The initrd/initramfs file name would definitely look strange if
> > this variable could contain multiple space delimited settings.
> 
> Also, openembedded-core/scripts/lib/wic/plugins/source/isoimage-
> isohybrid.py
> has that _build_initramfs_path function from line 143. The usage of
> the same variable in this python function also points back to the
> documentation line, i.e. it's a filename suffix and not multiple
> settings delimited by space.

This does look ok given the above. Thanks for confirming.

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [bitbake-devel] Keeping multiconfig config files in sync

2019-09-26 Thread Richard Purdie
Hi Chris,

On Tue, 2019-09-24 at 18:21 +, chris.laplante--- via bitbake-devel
wrote:
> Thanks to Richard and others recent hard work, multiconfig is poised
> to become much more practical in YP 3.0.
>  
> One thing I’m not clear on, however, is how it will work in a team
> environment. If I have recipes with multiconfig dependencies, then I
> must ensure that anyone else using those recipes has the exact same
> set of multiconfig config files defined in their
> build/conf/multiconfig/ directory. Or at least, compatible
> multiconfigs with matching names.
>  
> Since these files are under build/conf/, it’s not immediately clear
> how one might version control these. Unless I’m mistaken and there is
> a better way, as a workaround I will probably put the multiconfigs in
> a layer and instruct developers to symlink build/conf/multiconfig to
> there.
>  
> Any insight?

This has bugged me since we first created multiconfig. It was always
intended we'd have configs you could reference. It turned out to be
hard to code and we (well, I) decided to "come back to it". That hasn't
happened yet.

I'd welcome proposals on how it could work (its harder than it first
appears). It'd be good to check we do have a bug open for it too.

So yes, its an issue, I know about it, I don't have a good fix but
would like to see some kind of improvement.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] lighttpd: remove fam as a PACKAGECONFIG option

2019-09-26 Thread Richard Purdie
On Mon, 2019-09-23 at 21:16 +0100, Ross Burton wrote:
> On 23/09/2019 19:01, Trevor Gamblin wrote:
> > From: Trevor Gamblin
> > 
> > lighttpd builds fail if "fam" (and therefore gamin) is enabled.
> > 
> > In conf/local.conf:
> > 
> >  CORE_IMAGE_EXTRA_INSTALL += "lighttpd"
> >  PACKAGECONFIG_append_pn-lighttpd = " fam"
> > 
> > bitbake error:
> > 
> >  ERROR: Nothing PROVIDES 'gamin' (but /yow-lpggp31/tgamblin/oe-
> > core.git/meta/recipes-extended/lighttpd/lighttpd_1.4.54.bb DEPENDS
> > on or otherwise requires it)
> >  NOTE: Runtime target 'lighttpd' is unbuildable, removing...
> >  Missing or unbuildable dependency chain was: ['lighttpd',
> > 'gamin']
> >  ERROR: Required build target 'core-image-minimal' has no
> > buildable providers.
> >  Missing or unbuildable dependency chain was: ['core-image-
> > minimal', 'lighttpd', 'gamin']
> > 
> > Since gamin doesn't appear to have a recipe and hasn't been
> > maintained for several years, this should be removed from the
> > list of lighttpd PACKAGECONFIG options.
> 
> Non-default PACKAGECONFIGs are allowed to depend on recipes that are
> not 
> in core, plenty of recipes do this.  Otherwise there'd be no way to
> turn 
> on options that have dependencies in other layers.
> 
> However, the fact that Gamin is very much dead is a good reason to 
> remove the PACKAGECONFIG and simply hardcode --without-fam for good
> measure.

Agreed, can we have a v2 with --without-fam hardcoded in EXTRA_OECONF
please?

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 0/1] Add new mirror archiver mode

2019-09-26 Thread Richard Purdie
On Tue, 2019-09-24 at 10:06 +0100, Paul Barker wrote:
> This patch allows the creation of a full source mirror using the
> archiver
> bbclass. Using the archiver allows us to make use of copyleft license
> filtering, recipe type filtering and other features that are not
> available if
> we just copy the contents of the downloads directory itself to create
> a
> mirror. We also get the advantage of sstate caching and we don't have
> to
> throw away the entire downloads directory before building to keep
> sources for
> other builds out of our mirror.
> 
> The resulting mirror may be published and used as an entry in
> PREMIRRORS or
> MIRRORS for a subsequent build. The functionality is explained in
> more detail
> in the patch commit message and the additions to the documentation
> comments.
> 
> We've been using this patch in our Oryx Linux distro for a few months
> now to
> confirm that it works as expected. All feedback is welcome,
> especially from
> people who have worked on the archiver bbclass before.
> 
> Mirror creation was tested with the following additions to
> local.conf:
> 
> INHERIT += "archiver"
> BB_GENERATE_MIRROR_TARBALLS = "1"
> ARCHIVER_MODE[src] = "mirror"
> ARCHIVER_MODE[mirror] = "combined"
> ARCHIVER_MIRROR_EXCLUDE = "file://"
> COPYLEFT_LICENSE_INCLUDE = "*"
> 
> The resulting mirror directory was moved out of the tmpdir and a
> rebuild was
> tested with the following additions to local.conf:
> 
> BB_NO_NETWORK = "1"
> INHERIT += "own-mirrors"
> SOURCE_MIRROR_URL = "file://${TOPDIR}/mirror"
> 
> Paul Barker (1):
>   archiver.bbclass: Add new mirror archiver mode
> 
>  meta/classes/archiver.bbclass | 136 +---
> --
>  1 file changed, 117 insertions(+), 19 deletions(-)

This is coming in a bit late to be in 3.0 as we're way past feature
freeze. Have you given thought to regression testing? I'd also really
like to see some tests. There are some already:

meta/lib/oeqa/selftest/cases/archiver.py 

and whilst I'd not going to claim they're wonderful, it would be good
to extend as we add new functionality.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] classes/image-live.bbclass: Don't hardcode cpio.gz

2019-09-26 Thread Richard Purdie
On Thu, 2019-09-26 at 11:05 +0200, Böszörményi Zoltán via Openembedded-
core wrote:
> There's INITRAMFS_FSTYPES that can be set differently.
> 
> Signed-off-by: Böszörményi Zoltán 
> ---
> 
> With the hardcoded initrd filename suffix but INITRAMFS_FSTYPES
> set to cpio.lzma, this error occurs:
> 
> ERROR: sicom-pos-image-1.0-r0 do_bootimg:
> .../deploy/glibc/images/intel-core2-32/core-image-minimal-initramfs-
> intel-core2-32.cpio.lzma is invalid. initrd image creation failed.
> ERROR: sicom-pos-image-1.0-r0 do_bootimg: Function failed:
> build_hddimg (log file is located at .../tmp-sicom-
> glibc/work/intel_core2_32-sicom-linux/sicom-pos-image/1.0-
> r0/temp/log.do_bootimg.32210)
> ERROR: Logfile of failure stored in: .../tmp-sicom-
> glibc/work/intel_core2_32-sicom-linux/sicom-pos-image/1.0-
> r0/temp/log.do_bootimg.32210
> ERROR: Task (.../layers/meta-sicom/images/sicom-pos-
> image.bb:do_bootimg) failed with exit code '1'
> 
>  meta/classes/image-live.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image-live.bbclass b/meta/classes/image-
> live.bbclass
> index af71be5093..54058b350d 100644
> --- a/meta/classes/image-live.bbclass
> +++ b/meta/classes/image-live.bbclass
> @@ -37,7 +37,7 @@ do_bootimg[depends] += "dosfstools-
> native:do_populate_sysroot \
>  LABELS_LIVE ?= "boot install"
>  ROOT_LIVE ?= "root=/dev/ram0"
>  INITRD_IMAGE_LIVE ?= "${MLPREFIX}core-image-minimal-initramfs"
> -INITRD_LIVE ?= "${DEPLOY_DIR_IMAGE}/${INITRD_IMAGE_LIVE}-
> ${MACHINE}.cpio.gz"
> +INITRD_LIVE ?= "${DEPLOY_DIR_IMAGE}/${INITRD_IMAGE_LIVE}-
> ${MACHINE}.${INITRAMFS_FSTYPES}"
>  
>  LIVE_ROOTFS_TYPE ?= "ext4"
>  ROOTFS ?= "${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.${LIVE_ROOTFS_TYPE}"

I'm a little worried that INITRAMFS_FSTYPES can contain multiple values
by the sounds of its name...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] testimage.bbclass: Add kernel provider and version to testresult

2019-09-20 Thread Richard Purdie
On Tue, 2019-09-17 at 09:27 +0800, Yeoh Ee Peng wrote:
> In running QA testing, we sometime need to select custom  provider
> for
> virtual/kernel and version. To track the selected virtual/kernel
> provider
> and version used during test, we need to add these information to
> testresult.
> 
> This patch add the virtual/kernel and version into the testresult
> configuration section.
> 
> Example of virtual/kernel and version added to testresult
> configuration:
>- "KERNEL_PROVIDER_VERSION": "linux-intel_4.19"
> 
> Signed-off-by: Yeoh Ee Peng 
> ---
>  meta/classes/testimage.bbclass | 20 ++--
>  1 file changed, 18 insertions(+), 2 deletions(-)

This fixes a very specific use case but we don't really want to add
extra variables to this each time someone has a new use case. In other
words this isn't going to scale.

The idea originally is that "DISTRO" should give a pretty good idea of
what was being tested. Another idea would be to allow the variables to
be configurable but I'm not sure we want to support adding arbitrary
variables...

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH] oeqa/concurrenttest: Use ionice to delete build directories

2019-09-19 Thread Richard Purdie
Autobuilder type infrastructure can benefit from deletion of certain files as
background IO due to the way Linux filesystem priority works.

We have problems where build directories as part of oe-selftest being
delete starves the running tasks of IO to the point builds take much
longer to compelte.

Having this option of running the deletion at "idle" helps a lot with
that. Use the new option added to bb.utils.prunedir().

Signed-off-by: Richard Purdie 
---
 meta/lib/oeqa/core/utils/concurrencytest.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/core/utils/concurrencytest.py 
b/meta/lib/oeqa/core/utils/concurrencytest.py
index fa6fa34b0e1..6293cf94ec5 100644
--- a/meta/lib/oeqa/core/utils/concurrencytest.py
+++ b/meta/lib/oeqa/core/utils/concurrencytest.py
@@ -216,7 +216,7 @@ def removebuilddir(d):
 while delay and os.path.exists(d + "/bitbake.lock"):
 time.sleep(1)
 delay = delay - 1
-bb.utils.prunedir(d)
+bb.utils.prunedir(d, ionice=True)
 
 def fork_for_tests(concurrency_num, suite):
 result = []
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH V5] weston-init: Add possibility to run weston as non-root user

2019-09-19 Thread Richard Purdie
On Mon, 2019-09-16 at 15:35 -0700, Khem Raj wrote:
> These changes are from meta-96boards primarily
> Launch the session via a udev rule based on what kind of display
> device
> is available
> 
> delete weston-conf and move the fuctionality into weston-init other
> layers are doing same
> 
> weston-init installs machine specific weston.ini therefore mark is
> machine specific now
> 
> Signed-off-by: Khem Raj 
> Cc: Otavio Salvador 
> Signed-off-by: Ross Burton 
> ---
> v2: Drop duplicate isntall rule and use systemd_system_unitdir
> v3: Use systemd_system_unitdir in FILES section too
> v4: Move weston-conf logic into weston-init and delete it
> v5: Mark machine specific

You didn't actually retest this and v5 still fails the selftest. I've
squashed the fix needed into the patch.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/2] systemd: Handle slow to boot mips hwdb update timeouts

2019-09-18 Thread Richard Purdie
This is a temporary workaround to avoid autobuilder failures until
https://github.com/systemd/systemd/issues/13581 is resolved.

Its being done globally even though its a mips problem for simplicity,
it doesn't hurt anything else to have a longer timeout.

Signed-off-by: Richard Purdie 
---
 meta/recipes-core/systemd/systemd_243.bb | 5 +
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-core/systemd/systemd_243.bb 
b/meta/recipes-core/systemd/systemd_243.bb
index 3ae1516dfb5..f0e8c569b81 100644
--- a/meta/recipes-core/systemd/systemd_243.bb
+++ b/meta/recipes-core/systemd/systemd_243.bb
@@ -298,6 +298,11 @@ do_install() {
install -Dm 0644 ${WORKDIR}/99-default.preset 
${D}${systemd_unitdir}/system-preset/99-default.preset
 }
 
+do_install_append () {
+   # Mips qemu is extremely slow, allow more time for the hwdb update
+   # This is a workaround until 
https://github.com/systemd/systemd/issues/13581 is resolved
+   sed -i -e s#TimeoutSec=90s#TimeoutSec=180s# 
${D}${systemd_unitdir}/system/systemd-hwdb-update.service
+}
 
 python populate_packages_prepend (){
 systemdlibdir = d.getVar("rootlibdir")
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/2] meta-extsdk: Either an sstate task is a proper task or it isn't

2019-09-18 Thread Richard Purdie
Ensure the task is properly regsistered as an sstate task as this
"half way" state confuses new code in bitbake and it isn't supported.

Signed-off-by: Richard Purdie 
---
 meta/recipes-core/meta/meta-extsdk-toolchain.bb | 5 +
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-core/meta/meta-extsdk-toolchain.bb 
b/meta/recipes-core/meta/meta-extsdk-toolchain.bb
index 235d6ecc0f4..b4009ceaa18 100644
--- a/meta/recipes-core/meta/meta-extsdk-toolchain.bb
+++ b/meta/recipes-core/meta/meta-extsdk-toolchain.bb
@@ -24,3 +24,8 @@ python do_locked_sigs() {
 sigfile = os.path.join(outdir, 'locked-sigs-extsdk-toolchain.inc')
 oe.copy_buildsystem.generate_locked_sigs(sigfile, d)
 }
+
+python do_locked_sigs_setscene () {
+sstate_setscene(d)
+}
+addtask do_locked_sigs_setscene
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH v3 1/9] uefi.conf: add config file holding configuration for UEFI applications

2019-09-18 Thread richard . purdie
On Wed, 2019-09-18 at 12:04 +0300, Dmitry Eremin-Solenikov wrote:
> ср, 18 сент. 2019 г. в 01:16, Richard Purdie
> :
> > On Tue, 2019-09-17 at 18:36 +0300, dbarysh...@gmail.com wrote:
> > > 
> > Now I understand more about how this configuration file is being
> > used,
> > should it be called image-uefi.conf ?
> 
> Fine, I will rename the conf file
> 
> > I feel really strongly that we do not want an uefi.bbclass, its
> > simply
> > not warranted and will just continue to expand the current mess of
> > classes. If all we need it for is some functions, those functions
> > should be added elsewhere.
> 
> As those EFI_PROVIDER bootloader classes are called only form
> live-vm-common, maybe I should just add them to live-vm-common and
> make individual classes _append those functions?

That sounds much better.

> > I'm also on the lookout for tests of these kinds of codepaths. Code
> > is
> > much more likely to be accepted if tests are added for it. I'm not
> > quite sure what would make most sense here in this case buts its a
> > general point I will be pushing for going forward.
> 
> What kind of tests would you like? This code already exists and is
> called as a part of any live image generation.

I think these patches are ok, you hinted this was part of a larger set
of changes and I'm worried about the fit/signed image/uboot codepaths
though which are in a similar area to some of this.

Its something to be mindful of as if changes are made to something
which isn't currently tested I will be asking for tests.

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH v3 1/9] uefi.conf: add config file holding configuration for UEFI applications

2019-09-17 Thread Richard Purdie
On Tue, 2019-09-17 at 18:36 +0300, dbarysh...@gmail.com wrote:
> From: Dmitry Eremin-Solenikov 
> 
> Create new config file defining common variables for all UEFI-related
> packages (bootloaders, test applications, etc).
> 
> Signed-off-by: Dmitry Eremin-Solenikov <
> dmitry_eremin-soleni...@mentor.com>
> ---
>  meta/conf/uefi.conf | 16 
>  1 file changed, 16 insertions(+)
>  create mode 100644 meta/conf/uefi.conf

This is heading in the right direction however if we're going to try
and clean things up I've concluded we need to do it properly and get it
right.

Now I understand more about how this configuration file is being used,
should it be called image-uefi.conf ?

I feel really strongly that we do not want an uefi.bbclass, its simply
not warranted and will just continue to expand the current mess of
classes. If all we need it for is some functions, those functions
should be added elsewhere.

I hadn't realised they were shell functions earlier, sorry about that.
One option could be to convert them to python since they could be
written in python easily. You could call a python function as a prefunc
or postfunc of another shell or python function.

Other options could be to make them shell commands (via the script
directory) or create an image-functions.bbclass which contains such
shared functions generically. As long as things are namespaced (which
they already are), that should be fine. I would ask that documentation
comments be added to them to say what the functions do.

I'm also on the lookout for tests of these kinds of codepaths. Code is
much more likely to be accepted if tests are added for it. I'm not
quite sure what would make most sense here in this case buts its a
general point I will be pushing for going forward.

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH v2 1/9] uefi.bbclass: add bbclass holding configuration for UEFI applications

2019-09-17 Thread richard . purdie
On Tue, 2019-09-17 at 12:33 +0300, Dmitry Eremin-Solenikov wrote:
> вт, 17 сент. 2019 г. в 01:17, Richard Purdie
> :
> > On Fri, 2019-09-13 at 18:44 +0300, dbarysh...@gmail.com wrote:
> > > From: Dmitry Eremin-Solenikov  > > >
> > > 
> > > Create new bbclass defining common variables for all UEFI-related
> > > packages (bootloaders, test applications, etc).
> > > 
> > > Signed-off-by: Dmitry Eremin-Solenikov <
> > > dmitry_eremin-soleni...@mentor.com>
> > > ---
> > >  meta/classes/uefi.bbclass | 26 ++
> > >  1 file changed, 26 insertions(+)
> > >  create mode 100644 meta/classes/uefi.bbclass
> > 
> > I really want to get away from the proliferation of bbclass files
> > we
> > have. Wouldn't this make more sense as a .conf file?
> 
> Moving configuration to .conf file might make sense. I even can
> implement anonymous function as per-arch override. However there are
> still functions like efi_populate_common() and
> efi_iso_populate_common(), which can not be moved to .conf file.
> 
> Would you prefer separate uefi.conf file and uefi-bootloader.bbclass
> files

Would functions like this not be better suited to a lib/oe/efi.py ?

Its really easy to just put everything in a bbclass but I'm not
convinced its scaling well...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH] runqemu: Mention snapshot in the help output

2019-09-17 Thread Richard Purdie
This is a useful option but not documented in the help text.

Signed-off-by: Richard Purdie 
---
 scripts/runqemu | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/runqemu b/scripts/runqemu
index 68ba7dcfb94..1a5aca98ac7 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -73,6 +73,7 @@ of the following environment variables (in any order):
 serial - enable a serial console on /dev/ttyS0
 serialstdio - enable a serial console on the console (regardless of 
graphics mode)
 slirp - enable user networking, no root privileges is required
+snapshot - don't write changes to back to images
 kvm - enable KVM when running x86/x86_64 (VT-capable CPU required)
 kvm-vhost - enable KVM with vhost when running x86/x86_64 (VT-capable CPU 
required)
 publicvnc - enable a VNC server open to all hosts
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 3/4] gtk+3: Set depends to the virtual needed not explicitly on Mesa

2019-09-16 Thread Richard Purdie
On Fri, 2019-09-13 at 15:36 -0400, Andrew F. Davis via Openembedded-core wrote:
> The dependency is for EGL and GLES2 libraries. On some systems these
> are not provided by Mesa, list what is actually needed so the system
> can choose the correct provider.
> 
> Signed-off-by: Andrew F. Davis 
> ---
>  meta/recipes-gnome/gtk+/gtk+3.inc | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-gnome/gtk+/gtk+3.inc 
> b/meta/recipes-gnome/gtk+/gtk+3.inc
> index 77b6c31536..7ec40dcbf5 100644
> --- a/meta/recipes-gnome/gtk+/gtk+3.inc
> +++ b/meta/recipes-gnome/gtk+/gtk+3.inc
> @@ -52,7 +52,7 @@ PACKAGECONFIG[x11] = 
> "--enable-x11-backend,--disable-x11-backend,at-spi2-atk fon
>  # this is provided by oe-core patch that removes epoxy/gl dependency from a 
> X11 build
>  PACKAGECONFIG[opengl] = "--enable-opengl,--disable-opengl,libepoxy"
>  PACKAGECONFIG[glx] = "--enable-glx,--disable-glx,,libgl"
> -PACKAGECONFIG[wayland] = 
> "--enable-wayland-backend,--disable-wayland-backend,wayland wayland-protocols 
> libxkbcommon virtual/mesa wayland-native"
> +PACKAGECONFIG[wayland] = 
> "--enable-wayland-backend,--disable-wayland-backend,wayland wayland-protocols 
> libxkbcommon virtual/egl virtual/gles2 wayland-native"
>  PACKAGECONFIG[cups] = "--enable-cups,--disable-cups,cups"
>  
>  prepare_gtk_scripts() {

This breaks things:

https://autobuilder.yoctoproject.org/typhoon/#/builders/57/builds/1037

step1b: ERROR: Nothing PROVIDES 'virtual/gles2' (but 
/home/pokybuild/yocto-worker/qemux86-64-x32/build/meta/recipes-gnome/gtk+/gtk+3_3.24.8.bb
 DEPENDS on or otherwise requires it). Close matches:

Cheers,

Richard



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH V3] weston-init: Add possibility to run weston as non-root user

2019-09-16 Thread Richard Purdie
On Fri, 2019-09-13 at 06:41 -0700, Khem Raj wrote:
> On Fri, Sep 13, 2019 at 5:27 AM Ross Burton 
> wrote:
> > On 11/09/2019 05:29, Khem Raj wrote:
> > >   .../wayland/weston-init/weston.ini| 74
> > > +++
> > 
> > core-image-weston now fails:
> > 
> >file /etc/xdg/weston/weston.ini conflicts between attempted
> > installs
> > of weston-init-1.0-r0.noarch and weston-conf-1.0-r0.qemux86_64
> > 
> 
> on x86 qemu right. Sent a V4. give it a try

Fails oe-selftest:

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/383/steps/8/logs/step2d

oe-selftest -r statetests.SStateTests.test_sstate_sametune_samesigs

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH v2 1/9] uefi.bbclass: add bbclass holding configuration for UEFI applications

2019-09-16 Thread Richard Purdie
On Fri, 2019-09-13 at 18:44 +0300, dbarysh...@gmail.com wrote:
> From: Dmitry Eremin-Solenikov 
> 
> Create new bbclass defining common variables for all UEFI-related
> packages (bootloaders, test applications, etc).
> 
> Signed-off-by: Dmitry Eremin-Solenikov <
> dmitry_eremin-soleni...@mentor.com>
> ---
>  meta/classes/uefi.bbclass | 26 ++
>  1 file changed, 26 insertions(+)
>  create mode 100644 meta/classes/uefi.bbclass

I really want to get away from the proliferation of bbclass files we
have. Wouldn't this make more sense as a .conf file? 

The anonymous function could be done with overrides, or an appropriate
function from lib/oe/.

I appreciate we don't use conf files so much but we need to start
somewhere and I think this is a good candidate for it.

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] dnf: make dnf work in toolchain

2019-09-16 Thread Richard Purdie
On Mon, 2019-09-16 at 14:36 +0800, Zheng Ruoqin wrote:
> We need to configure dnf to use package architecture from yocto build
> system.
> 
> Install etc/dnf/vars to ${SDKTARGETSYSROOT} because config file in
> host-sysroot as /opt/poky/2.7+snapshot/sysroots/x86_64-pokysdk-linux
> will be covered by another ARCH which result in previous config
> settings inefficacy.
> 
> To resolve it, put config file in target-sysroot like
> /opt/poky/2.7+snapshot/sysroots/core2-64-poky-linux. As each ARCH has
> its own target-sysroot, config file will not be covered.

I'm afraid we can't have target dependencies within nativesdk packages.

This is shown by the fact this patch (and the similar rpm one) cause
selftest failures in the signature tests:

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/381/steps/8/logs/step2d

2019-09-16 00:24:54,670 - oe-selftest - INFO - RESULTS - 
sstatetests.SStateTests.test_sstate_allarch_samesigs: FAILED (111.59s)
2019-09-16 00:24:54,670 - oe-selftest - INFO - RESULTS - 
sstatetests.SStateTests.test_sstate_nativesdk_samesigs_multilib: FAILED 
(144.14s)
2019-09-16 00:24:54,670 - oe-selftest - INFO - RESULTS - 
sstatetests.SStateTests.test_sstate_sametune_samesigs: FAILED (161.53s)

you can run these with commands like:

os-selftest -r sstatetests.SStateTests.test_sstate_allarch_samesigs

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH v3] systemd: upgrade to 243

2019-09-15 Thread Richard Purdie
On Sat, 2019-09-14 at 08:00 -0700, akuster808 wrote:
> 
> On 9/13/19 4:26 PM, Scott Murray wrote:
> > PATCH REBASED:
> > ==
> > 0001-binfmt-Don-t-install-dependency-links-at-install-tim.patch
> > 0001-do-not-disable-buffer-in-writing-files.patch
> > 0002-use-lnr-wrapper-instead-of-looking-for-relative-opti.patch
> > 0004-add-fallback-parse_printf_format-implementation.patch
> > 0004-rules-whitelist-hd-devices.patch
> > 0005-rules-watch-metadata-changes-in-ide-devices.patch
> > 0005-src-basic-missing.h-check-for-missing-strndupa.patch
> > 0006-Include-netinet-if_ether.h.patch
> > 0007-don-t-fail-if-GLOB_BRACE-and-GLOB_ALTDIRFUNC-is-not.patch
> > 0017-Do-not-disable-buffering-when-writing-to-oom_score_a.patch
> > 
> > PATCH DROPPED:
> > ==
> > 0001-Replace-the-legacy-ULONG_LONG_MAX-with-the-C99-ULLON.patch
> > 0001-src-udev-udev-event.c-must-include-sys-wait.h.patch
> > 0023-socket-util.h-include-string.h.patch
> > 0025-fs-utilh-add-missing-sys-stat-include.patch
> > 
> > PATCH ADDED:
> > 
> > 0002-src-login-brightness.c-include-sys-wait.h.patch
> > 0003-src-basic-copy.c-include-signal.h.patch
> > 0004-src-shared-cpu-set-util.h-add-__cpu_mask-definition.patch
> > 
> > Also applied libc-glibc over-ride to pkg_postinst and pkg_prerm
> > function
> > definitions, as musl does not provide nsswitch.conf.
> 
> This update did not introduce any new issues. We had hoped it might
> address the mips-alt systmed timeouts we are seeing but it did not.
> 
> Hopefully this is a data point for Richard when he gets back.

Its a good data point thanks!

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] libevent: don't treat test stats line as pass/fail in ptest

2019-09-09 Thread Richard Purdie
On Mon, 2019-09-09 at 11:16 -0400, Trevor Gamblin wrote:
> From: Trevor Gamblin 
> 
> The libevent "regress" test outputs its own pass/fail results,
> e.g. "2/300 TESTS FAILED. (31 skipped)", which will be
> miscounted as an extra test fail in the ptest log. Fixed this
> to ignore the libevent results line when counting actual
> pass/fail results.
> 
> Also removed the for loop in run-ptest and targeted only the
> libevent "regress" test, as the other tests being run were
> related to performance and did not provide a relevant pass/fail
> output.

Can you put a comment into the runner script about why we're not
running these other tests please? This is so that someone looking at
the script in the future can immediately see we're not running them
intentionally.

Thanks,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] python3native, pythonnative: Separate definition and export of PYTHON_LIBRARY and PYTHON_INCLUDE_DIR

2019-09-09 Thread Richard Purdie
On Sun, 2019-09-08 at 20:16 -0700, Khem Raj wrote:
> This helps recipes where they need to explicitly pass the variable
> and
> does not entertain the ones from environment
> 
> Signed-off-by: Khem Raj 
> ---
>  meta/classes/python3native.bbclass | 6 --
>  meta/classes/pythonnative.bbclass  | 6 --
>  2 files changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/meta/classes/python3native.bbclass
> b/meta/classes/python3native.bbclass
> index d98fb4c758..bed04bd941 100644
> --- a/meta/classes/python3native.bbclass
> +++ b/meta/classes/python3native.bbclass
> @@ -14,8 +14,8 @@ export STAGING_LIBDIR
>  # find_package(PythonLibs REQUIRED)
>  # which ends up using libs/includes from build host
>  # Therefore pre-empt that effort
> -export
> PYTHON_LIBRARY="${STAGING_LIBDIR}/lib${PYTHON_DIR}${PYTHON_ABI}.so"
> -export
> PYTHON_INCLUDE_DIR="${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI}"
> +PYTHON_LIBRARY="${STAGING_LIBDIR}/lib${PYTHON_DIR}${PYTHON_ABI}.so"
> +PYTHON_INCLUDE_DIR="${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI}"
>  
>  export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
>  
> @@ -24,3 +24,5 @@ export PYTHONNOUSERSITE = "1"
>  
>  # autoconf macros will use their internal default preference
> otherwise
>  export PYTHON
> +export PYTHON_LIBRARY
> +export PYTHON_INCLUDE_DIR

I'm confused as this makes no difference to bitbake and is equivalent.
exported variables are always set in the datastore...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH v2] libevent: add granularity to ptest log

2019-09-07 Thread Richard Purdie
On Fri, 2019-09-06 at 14:51 -0400, Trevor Gamblin wrote:
> From: Trevor Gamblin 
> 
> The libevent ptest used to report only a global pass or a fail
> result.
> Count individual PASS, FAIL, SKIP results. The SKIP results now
> include tests that are disabled in the libevent code.
> 
> libevent's ptest output did not comply with the automake-style output
> "result: testname", and reported a FAIL status at the end of the test
> run if any of the libevent tests failed. This patch makes the log
> consistent with the automake style:
> 
> PASS: http/cancel_by_host_no_ns
> PASS: http/cancel_inactive_server
> PASS: http/cancel_by_host_no_ns_inactive_server
> SKIPPED: http/cancel_by_host_server_timeout
> SKIPPED: http/cancel_server_timeout
> 
> and provides a summary as follows:
> 
> === Test Summary ===
> TOTAL: 316
> PASSED: 300
> FAILED: 0
> SKIPPED: 16
> DURATION: 87
> END: /usr/lib/libevent/ptest
> 
> Signed-off-by: Trevor Gamblin 

Thanks for this, it is a definite improvement on what was there before.
Results from the autobuilder run:

https://autobuilder.yocto.io/pub/non-release/20190907-4/testresults/testresult-report.txt

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH V3] dnf.py: installroot support usrmerge

2019-09-06 Thread richard . purdie
On Thu, 2019-09-05 at 14:25 -0400, Randy MacLeod wrote:
> On 9/4/19 4:36 AM, Changqing Li wrote:
> > I didn't met failure in test_parselog during test,  the whole 
> > testimage is passed.
> > 
> > Could you help to send me the link of the failure test if the new 
> > version still
> > 
> > met test_parselog failure? Thanks.
> 
> Sandy,
> 
> Richard shouldn't have to do that since many people have such
> requests and
> 
> if he helped everyone he would (and has in the past) spend/t too
> much 
> time helping people.
> 
> We should be able to setup the autobuilder locally to reproduce the
> problem.
> 
> 
> Also during the YP bug review call, someone suggested that you check
> if the
> 
> dnf install tests are running in a deterministic order. If they are
> not, 
> then that
> 
> might explain why the AB sees the problem but you do not.

It wasn't clear if the issue was fixed in the revised patch or not so I
did include this in the last round of tests and those passed. I
therefore suspect whatever the issue was did get fixed in the v3.

Of course a different patch failed (mdadm) so I've replied about that
one separately (I've confirmed which patch it was manually, it did
reproduce).

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/2] mdadm: fix do_package failed when changed local.conf but not cleaned

2019-09-06 Thread Richard Purdie
On Fri, 2019-09-06 at 11:37 +0800, changqing...@windriver.com wrote:
> From: Changqing Li 
> 
> reproduce steps:
> 1. add DISTRO_FEATURE_append = 'usrmerge' in local.conf
> 2. bitbake mdadm --success
> 3. remove DISTRO_FEATURE_append = 'usrmerge' from local.conf
> 4. bitbake mdadm  -- failed when do_package
> 
> it is not proper to change source Makefile during do_install by sed,
> change to add patch for it.
> 
> [YOCTO #13493]
> 
> Signed-off-by: Changqing Li 

This somehow seems to break:

oe-selftest -r devtool.DevtoolModifyTests.test_devtool_buildclean
and
oe-selftest -r devtool.DevtoolModifyTests.test_devtool_modify

https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/694

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] gzip: add nativesdk support

2019-09-06 Thread richard . purdie
On Fri, 2019-09-06 at 19:00 +0300, Ruslan Bilovol wrote:
> On 9/4/19 2:01 PM, Ross Burton wrote:
> > On 04/09/2019 11:41, Denys Zagorui -X (dzagorui - GLOBALLOGIC INC
> > at 
> > Cisco) via Openembedded-core wrote:
> > > Signed-off-by: Denys Zagorui 
> > 
> > Can you check your mailer configuration, as the From: is the
> > mailing 
> > list, not your email address.  This means the patch doesn't get
> > your 
> > authorship.
> 
> This is OE mailing list specific issue, we can't fix it
> unfortunately.

Well, its also related to the way your company email is setup.

> This was discussed multiple times before:
> http://lists.openembedded.org/pipermail/openembedded-core/2019-March/280394.html
> 
> Richard, do you have any updates on this issue?

We're transitioning to groups.io soon for mailing lists. That is
proving quite tricky to do but work is in progress on that and it
should happen in a week or so...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] libevent: add granularity to ptest

2019-09-06 Thread richard . purdie
On Fri, 2019-09-06 at 11:10 -0400, Trevor Gamblin wrote:
> On 9/6/19 10:26 AM, Richard Purdie wrote:
> > On Fri, 2019-09-06 at 10:04 -0400, Trevor Gamblin wrote:
> > > From: Trevor Gamblin 
> > > 
> > > The libevent ptest used to report only a global pass or a fail
> > > result.
> > > Count individual PASS, FAIL, SKIP results. The SKIP results now
> > > include tests that are disabled in the libevent code.
> > > 
> > > Signed-off-by: Trevor Gamblin 
> > > ---
> > >  .../libevent/libevent/run-ptest   | 34 -
> > > 
> > > --
> > >  1 file changed, 22 insertions(+), 12 deletions(-)
> > > 
> > > diff --git a/meta/recipes-support/libevent/libevent/run-ptest
> > > b/meta/recipes-support/libevent/libevent/run-ptest
> > > index 0241851c70..b7d945246f 100644
> > > --- a/meta/recipes-support/libevent/libevent/run-ptest
> > > +++ b/meta/recipes-support/libevent/libevent/run-ptest
> > > @@ -1,18 +1,28 @@
> > >  #!/bin/sh
> > >  
> > > -fail=0
> > > +# run-ptest - 'ptest' test infrastructure shell script that
> > > +#   wraps the libevent test scripts 
> > > +#
> > > +# Trevor Gamblin 
> > > +###
> > > +LIBEVENTLIB=/usr/lib/libevent
> > > +LOG="${LIBEVENTLIB}/ptest/libevent_ptest_$(date +%Y%m%d-
> > > %H%M%S).log"
> > > +
> > > +cd ${LIBEVENTLIB}/ptest 
> > > +
> > >  for test in ./test/*
> > >  do
> > > - $test
> > > - if [ $? -ne 0 ]
> > > - then
> > > - fail=1
> > > - fi
> > > +$test 2>&1|tee -a ${LOG}
> > >  done
> > >  
> > > -if [ $fail -eq 0 ]
> > > -then
> > > - echo "PASS: libevent"
> > > -else
> > > - echo "FAIL: libevent"
> > > -fi
> > > +passed=`grep OK ${LOG}|wc -l`
> > > +failed=`grep FAILED ${LOG}|wc -l`
> > > +skipped=`grep -E 'DISABLED|SKIPPED' ${LOG}|wc -l`
> > > +all=$((passed + failed + skipped))
> > > +
> > > +(   echo "=== Test Summary ==="
> > > +echo "TOTAL: ${all}"
> > > +echo "PASSED: ${passed}"
> > > +echo "FAILED: ${failed}"
> > > +echo "SKIPPED: ${skipped}"
> > > +) | tee -a ${LOG}
> > 
> > Does this work correctly such that ptest-runner picks up the
> > individual
> > test pass/fail results? I thought the tests would need individual
> > pass/fail lines for that to work?
> > 
> > Cheers,
> > 
> > Richard
> > 
> > 
> 
> They do have individual pass/fail results in the ptest-runner output,
> but those don't match up with the pass/fail usually provided by
> ptest, so the log is parsed for OK/FAILED/SKIPPED/DISABLED and
> translated. Example output:
> write_cb: write 12
> write_cb: write -1
> === Test Summary ===
> TOTAL: 316
> PASSED: 300
> FAILED: 0
> SKIPPED: 16
> DURATION: 87
> END: /usr/lib/libevent/ptest
> 2019-09-06T14:50
> STOP: ptest-runner
> Compare with the last log from autobuilder (
> https://autobuilder.yocto.io/pub/non-release/20190829-10/testresults/qemux86-64-ptest/libevent.log
> ):
> write_cb: write 12
> write_cb: write -1
> FAIL: libevent
> DURATION: 116
> I'm working on a v2 to address multilib - I'll add the example output
> to the commit as well.

You're missing my point. There are log parsers such as:

http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/lib/oeqa/utils/logparser.py

The output has to follow the format in:

https://wiki.yoctoproject.org/wiki/Ptest

to quote it:

"""
One major point of ptest is to consolidate the output format of all
tests into a single common format. The format selected is the automake
"simple test" format:

result: testname

Where "result" is one of PASS, FAIL or SKIP and "testname" can be any
identifying string (the same format used by Automake.) 
"""

which is why you'll see other run-ptest scripts using sed to add in
PASS/FAIL/SKIP strings.

I think the PASSED/FAILED counts are generated by ptest-runner itself.

Cheers,

Richard



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] libevent: add granularity to ptest

2019-09-06 Thread Richard Purdie
On Fri, 2019-09-06 at 10:04 -0400, Trevor Gamblin wrote:
> From: Trevor Gamblin 
> 
> The libevent ptest used to report only a global pass or a fail
> result.
> Count individual PASS, FAIL, SKIP results. The SKIP results now
> include tests that are disabled in the libevent code.
> 
> Signed-off-by: Trevor Gamblin 
> ---
>  .../libevent/libevent/run-ptest   | 34 -
> --
>  1 file changed, 22 insertions(+), 12 deletions(-)
> 
> diff --git a/meta/recipes-support/libevent/libevent/run-ptest
> b/meta/recipes-support/libevent/libevent/run-ptest
> index 0241851c70..b7d945246f 100644
> --- a/meta/recipes-support/libevent/libevent/run-ptest
> +++ b/meta/recipes-support/libevent/libevent/run-ptest
> @@ -1,18 +1,28 @@
>  #!/bin/sh
>  
> -fail=0
> +# run-ptest - 'ptest' test infrastructure shell script that
> +#   wraps the libevent test scripts 
> +#
> +# Trevor Gamblin 
> +###
> +LIBEVENTLIB=/usr/lib/libevent
> +LOG="${LIBEVENTLIB}/ptest/libevent_ptest_$(date +%Y%m%d-%H%M%S).log"
> +
> +cd ${LIBEVENTLIB}/ptest 
> +
>  for test in ./test/*
>  do
> - $test
> - if [ $? -ne 0 ]
> - then
> - fail=1
> - fi
> +$test 2>&1|tee -a ${LOG}
>  done
>  
> -if [ $fail -eq 0 ]
> -then
> - echo "PASS: libevent"
> -else
> - echo "FAIL: libevent"
> -fi
> +passed=`grep OK ${LOG}|wc -l`
> +failed=`grep FAILED ${LOG}|wc -l`
> +skipped=`grep -E 'DISABLED|SKIPPED' ${LOG}|wc -l`
> +all=$((passed + failed + skipped))
> +
> +(   echo "=== Test Summary ==="
> +echo "TOTAL: ${all}"
> +echo "PASSED: ${passed}"
> +echo "FAILED: ${failed}"
> +echo "SKIPPED: ${skipped}"
> +) | tee -a ${LOG}

Does this work correctly such that ptest-runner picks up the individual
test pass/fail results? I thought the tests would need individual
pass/fail lines for that to work?

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 2/2] gcc-cross: Fix header file corruption problems

2019-09-06 Thread richard . purdie
On Fri, 2019-09-06 at 07:15 -0700, Khem Raj wrote:
> 
> 
> On Fri, Sep 6, 2019 at 12:24 AM Richard Purdie <
> richard.pur...@linuxfoundation.org> wrote:
> > gcc's makefile can move files, replacing with the contents
> > "timestamp". This
> > corrupts the headers and breaks things like the gcc testsuite.
> > 
> > Add in a fix to ensure the headers are not corrupted through their
> > hardlink copies.
> > 
> > Signed-off-by: Richard Purdie 
> > ---
> >  meta/recipes-devtools/gcc/gcc-cross.inc | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc
> > b/meta/recipes-devtools/gcc/gcc-cross.inc
> > index e417b898734..95af6d89a9d 100644
> > --- a/meta/recipes-devtools/gcc/gcc-cross.inc
> > +++ b/meta/recipes-devtools/gcc/gcc-cross.inc
> > @@ -212,6 +212,9 @@ do_gcc_stash_builddir[cleandirs] =
> > "${BUILDDIRSTASH}"
> >  do_gcc_stash_builddir () {
> > dest=${BUILDDIRSTASH}
> > hardlinkdir . $dest
> > +   # Makefile does move-if-change which can end up with
> > 'timestamp' as file contents so break links to those files
> > +   rm $dest/gcc/include/*.h
> > +   cp gcc/include/*.h $dest/gcc/include/
> >  }
> 
> I think we moved them outside gcc dir isn’t it 

Not sure I follow? I think the patch is correct?

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH] pango: 1.44.6 upgrade

2019-09-06 Thread Richard Purdie
From: Ross Burton 

Signed-off-by: Ross Burton 
---
 .../pango/{pango_1.44.5.bb => pango_1.44.6.bb}| 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-graphics/pango/{pango_1.44.5.bb => pango_1.44.6.bb} (91%)

diff --git a/meta/recipes-graphics/pango/pango_1.44.5.bb 
b/meta/recipes-graphics/pango/pango_1.44.6.bb
similarity index 91%
rename from meta/recipes-graphics/pango/pango_1.44.5.bb
rename to meta/recipes-graphics/pango/pango_1.44.6.bb
index a143723c7c6..abe90c28d78 100644
--- a/meta/recipes-graphics/pango/pango_1.44.5.bb
+++ b/meta/recipes-graphics/pango/pango_1.44.6.bb
@@ -16,8 +16,8 @@ GNOMEBASEBUILDCLASS = "meson"
 inherit gnomebase gtk-doc ptest-gnome upstream-version-is-even 
gobject-introspection
 
 SRC_URI += "file://run-ptest"
-SRC_URI[archive.md5sum] = "b6bf689e3ce4f46b0fd887b64c850ea1"
-SRC_URI[archive.sha256sum] = 
"8527dfcbeedb4390149b6f94620c0fa64e26046ab85042c2a7556438847d7fc1"
+SRC_URI[archive.md5sum] = "db0a3243ba33e02aaa775412f8e5f412"
+SRC_URI[archive.sha256sum] = 
"3e1e41ba838737e200611ff001e3b304c2ca4cdbba63d200a20db0b0ddc0f86c"
 
 DEPENDS = "glib-2.0 glib-2.0-native fontconfig freetype virtual/libiconv cairo 
harfbuzz fribidi"
 
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/2] systemtap: Test latest and greatest for 5.2 kernel

2019-09-06 Thread richard . purdie
On Fri, 2019-09-06 at 08:24 -0400, Bruce Ashfield wrote:
> On Fri, Sep 6, 2019 at 3:24 AM Richard Purdie
>  wrote:
> > Systemtap has issues with the 5.2 kernel which are fixed in master,
> > we helped
> > debug and submitted some of the patches. Update to a git version
> > which includes
> > all the fixes.
> 
> And I can confirm that I can now run the stap test on all the arches
> with 5.2.

Cool. I also got a few more systemtap patches accepted upstream so we
can clean up the patches there a bit.

> 
> I'll send the rest of my 5.2 queue in a few hours (the removal of
> 5.0,
> and setting 5.2 as default where needed). I just want to run a few
> more tests before sending.

I actually merged my patch to poky which changed the default FWIW since
it made it though the autobuilder...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH][V2] sdkext: use simpler kernel module for devtool test

2019-09-06 Thread richard . purdie
On Thu, 2019-09-05 at 17:10 -0700, Michael Halstead wrote:
> On 9/5/19 2:27 PM, richard.pur...@linuxfoundation.org wrote:
> > On Thu, 2019-09-05 at 13:54 -0400, Mark Asselstine wrote:
> > > Richard, are you thinking we need to figure out the
> > > git.yoctoproject.org repo 
> > > before we can move ahead with merging this?
> > I would like to fix that, yes. I could ask Michael for help but I
> > do
> > have access to sort it myself too.
> > 
> > I'm also trying to sort out a few other issues before -next can
> > merge.
> > 
> > Since I'm failing to get to it,
> > 
> > Michael,
> > 
> > Could you clone:
> > 
> > https://github.com/masselstine/kernel-module-hello-world.git 
> > 
> > onto git.yp.org as a new repo and give write access to it for Mark,
> > myself, Bruce and Ross please?
> 
> This repo is ready at
> g...@push.yoctoproject.org:kernel-module-hello-world for the four of
> you.
> 
> It is available read-only for anyone at,
> 
> git://git.yoctoproject.org/kernel-module-hello-world
> 
> and
> 
> https://git.yoctoproject.org/git/pokykernel-module-hello-world

Thanks Michael.

Mark: I tweaked your patch and its now merged (avoiding the typo
above!)

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] resulttool: Add reproducible log extraction

2019-09-06 Thread Richard Purdie
On Thu, 2019-09-05 at 08:54 -0500, Joshua Watt wrote:
> Adds an argument to the log subcommand to extract the raw logs from
> the
> reproducible selftest.
> 
> To prevent ambiguity, the "--raw" argument has been renamed
> "--raw-ptest", although the old "--raw" argument is kept around for
> compatibility.
> 
> [YOCTO #13324]
> 
> Signed-off-by: Joshua Watt 
> ---
>  scripts/lib/resulttool/log.py | 33 ++---
>  1 file changed, 30 insertions(+), 3 deletions(-)
> 
> diff --git a/scripts/lib/resulttool/log.py
> b/scripts/lib/resulttool/log.py
> index 25c3396717e..2352c767d91 100644
> --- a/scripts/lib/resulttool/log.py
> +++ b/scripts/lib/resulttool/log.py
> @@ -16,6 +16,16 @@ def show_ptest(result, ptest, logger):
>  print("ptest '%s' not found" % ptest)
>  return 1
>  
> +def show_reproducible(result, reproducible, logger):
> +try:
> +print(result['reproducible'][reproducible]['diffoscope.text'
> ])
> +return 0
> +
> +except KeyError:
> +print("reproducible '%s' not found" % reproducible)
> +return 1
> +
> +
>  def log(args, logger):
>  results = resultutils.load_resultsdata(args.source)
>  
> @@ -40,17 +50,28 @@ def log(args, logger):
>  with open(dest, 'w') as f:
>  f.write(ptest['log'])
> 

You might want to consider this in autobuilder context:

http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder-helper/tree/scripts/generate-testresult-index.py#n121

Since the indexer could extract reproducible logs as well as ptest
ones.

Nathan: We probably need to think about the new testsuite pieces in
this context too...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH] systemtap: Drop patches merged upstream

2019-09-06 Thread Richard Purdie
Several of our patches were merged upstream just beyond our current version.
Update to that version and drop them.

Signed-off-by: Richard Purdie 
---
 .../configure-allow-to-disable-libvirt.patch  | 39 ---
 .../systemtap/systemtap/monitor-option.patch  | 37 --
 .../systemtap/systemtap/no-msgfmt-check.patch | 33 
 .../systemtap/systemtap/x32_abi_time.patch| 34 
 .../systemtap/systemtap_git.inc   |  6 +--
 5 files changed, 1 insertion(+), 148 deletions(-)
 delete mode 100644 
meta/recipes-kernel/systemtap/systemtap/configure-allow-to-disable-libvirt.patch
 delete mode 100644 meta/recipes-kernel/systemtap/systemtap/monitor-option.patch
 delete mode 100644 
meta/recipes-kernel/systemtap/systemtap/no-msgfmt-check.patch
 delete mode 100644 meta/recipes-kernel/systemtap/systemtap/x32_abi_time.patch

diff --git 
a/meta/recipes-kernel/systemtap/systemtap/configure-allow-to-disable-libvirt.patch
 
b/meta/recipes-kernel/systemtap/systemtap/configure-allow-to-disable-libvirt.patch
deleted file mode 100644
index b4f2fbc0664..000
--- 
a/meta/recipes-kernel/systemtap/systemtap/configure-allow-to-disable-libvirt.patch
+++ /dev/null
@@ -1,39 +0,0 @@
-From 5eb10d90af9178edb65e6091ae939d1b5b19bb78 Mon Sep 17 00:00:00 2001
-From: Wenzong Fan 
-Date: Tue, 23 Sep 2014 04:47:10 -0400
-Subject: [PATCH] systemtap: allow to disable libvirt
-
-Upstream-Status: Pending
-
-Signed-off-by: Wenzong Fan 

- configure.ac |   13 +
- 1 file changed, 9 insertions(+), 4 deletions(-)
-
-diff --git a/configure.ac b/configure.ac
-index a631ae7..cb4885b 100644
 a/configure.ac
-+++ b/configure.ac
-@@ -525,10 +525,15 @@ dnl Check for the libvirt and libxml2 devel packages
- 
- dnl We require libvirt >= 1.0.2 because stapvirt relies on the
- dnl virDomainOpenChannel function, which was implemented in 1.0.2.
--PKG_CHECK_MODULES([libvirt], [libvirt >= 1.0.2], [
--   have_libvirt=yes
--   AC_DEFINE([HAVE_LIBVIRT],[1],[Define to 1 if libvirt development libraries 
are installed])
--   ], [have_libvirt=no])
-+AC_ARG_ENABLE([libvirt],
-+  AS_HELP_STRING([--disable-libvirt], [Do not use libvirt even if present]))
-+
-+if test "$enable_libvirt" != no; then
-+  PKG_CHECK_MODULES([libvirt], [libvirt >= 1.0.2], [
-+ have_libvirt=yes
-+ AC_DEFINE([HAVE_LIBVIRT],[1],[Define to 1 if libvirt development 
libraries are installed])
-+ ], [have_libvirt=no])
-+fi
- AM_CONDITIONAL([HAVE_LIBVIRT], [test "${have_libvirt}" = "yes"])
- PKG_CHECK_MODULES([libxml2], [libxml-2.0], [
-have_libxml2=yes
--- 
-1.7.9.5
-
diff --git a/meta/recipes-kernel/systemtap/systemtap/monitor-option.patch 
b/meta/recipes-kernel/systemtap/systemtap/monitor-option.patch
deleted file mode 100644
index 9313a5aba3e..000
--- a/meta/recipes-kernel/systemtap/systemtap/monitor-option.patch
+++ /dev/null
@@ -1,37 +0,0 @@
-From 93fc4744fedf6fc593ee656968da97f7b1862ada Mon Sep 17 00:00:00 2001
-From: Ross Burton 
-Date: Tue, 4 Oct 2016 16:37:53 +0100
-Subject: [PATCH 4/6] systemtap: rationalise dependencies
-
-Add an option to explicitly disable the monitor (and therefore the dependency 
on
-json-c and ncurses).
-
-Upstream-Status: Pending
-Signed-off-by: Ross Burton 
-

- configure.ac | 5 -
- 1 file changed, 4 insertions(+), 1 deletion(-)
-
-Index: git/configure.ac
-===
 git.orig/configure.ac
-+++ git/configure.ac
-@@ -766,13 +766,16 @@ dnl We want either (or both) python prob
- AM_CONDITIONAL([HAVE_PYTHON_PROBES],
-   [test "x$have_python2_support" = "xyes" -o "x$have_python3_support" = 
"xyes"])
- 
-+AC_ARG_ENABLE([monitor], AS_HELP_STRING([--disable-monitor],[Disable 
monitor]))
-+if test "$enable_monitor" != "no"; then
- dnl Check for presence of json-c and ncurses for use in monitor mode
- PKG_CHECK_MODULES([jsonc], [json-c >= 0.11], [have_jsonc=yes], 
[have_jsonc=no])
- PKG_CHECK_MODULES([ncurses], [ncurses], [have_ncurses=yes], [have_ncurses=no])
--AM_CONDITIONAL([HAVE_MONITOR_LIBS], [test "${have_jsonc}" == "yes" -a 
"${have_ncurses}" == "yes"])
- if test "${have_jsonc}" == "yes" -a "${have_ncurses}" == yes; then
-   AC_DEFINE([HAVE_MONITOR_LIBS],[1],[Define to 1 if json-c and ncurses 
libraries are installed])
- fi
-+fi
-+AM_CONDITIONAL([HAVE_MONITOR_LIBS], [test "${have_jsonc}" == "yes" -a 
"${have_ncurses}" == "yes" -a "$enable_monitor" != "no"])
- 
- AC_CACHE_CHECK([for assembler .section "?" flags support], stap_cv_sectionq, [
- old_CFLAGS="$CFLAGS"
diff --git a/meta/recipes-kernel/systemtap/systemtap/no-msgfmt-check.patch 
b/meta/recipes-kernel/systemtap/systemtap/no-msgfmt-check.patch
deleted file mode 100644
index 2c860b19e51..000

[OE-core] [PATCH 2/2] gcc-cross: Fix header file corruption problems

2019-09-06 Thread Richard Purdie
gcc's makefile can move files, replacing with the contents "timestamp". This
corrupts the headers and breaks things like the gcc testsuite.

Add in a fix to ensure the headers are not corrupted through their hardlink 
copies.

Signed-off-by: Richard Purdie 
---
 meta/recipes-devtools/gcc/gcc-cross.inc | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc 
b/meta/recipes-devtools/gcc/gcc-cross.inc
index e417b898734..95af6d89a9d 100644
--- a/meta/recipes-devtools/gcc/gcc-cross.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross.inc
@@ -212,6 +212,9 @@ do_gcc_stash_builddir[cleandirs] = "${BUILDDIRSTASH}"
 do_gcc_stash_builddir () {
dest=${BUILDDIRSTASH}
hardlinkdir . $dest
+   # Makefile does move-if-change which can end up with 'timestamp' as 
file contents so break links to those files
+   rm $dest/gcc/include/*.h
+   cp gcc/include/*.h $dest/gcc/include/
 }
 addtask do_gcc_stash_builddir after do_compile before do_install
 SSTATETASKS += "do_gcc_stash_builddir"
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/2] systemtap: Test latest and greatest for 5.2 kernel

2019-09-06 Thread Richard Purdie
Systemtap has issues with the 5.2 kernel which are fixed in master, we helped
debug and submitted some of the patches. Update to a git version which includes
all the fixes.

Signed-off-by: Richard Purdie 
---
 meta/recipes-kernel/systemtap/systemtap_git.inc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-kernel/systemtap/systemtap_git.inc 
b/meta/recipes-kernel/systemtap/systemtap_git.inc
index c5348b3204b..f16f851e610 100644
--- a/meta/recipes-kernel/systemtap/systemtap_git.inc
+++ b/meta/recipes-kernel/systemtap/systemtap_git.inc
@@ -1,7 +1,7 @@
 LICENSE = "GPLv2"
 LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
-SRCREV = "984d6d1696ed06626b07cb65ab55d6ae0ece1131"
-PV = "4.1"
+SRCREV = "a25199c9580b9b66424c494d3bf39c4d1b7e1f7b"
+PV = "4.1+git${SRCPV}"
 
 SRC_URI = "git://sourceware.org/git/systemtap.git \
file://configure-allow-to-disable-libvirt.patch \
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH][V2] sdkext: use simpler kernel module for devtool test

2019-09-05 Thread richard . purdie
On Thu, 2019-09-05 at 13:54 -0400, Mark Asselstine wrote:
> On Thursday, August 22, 2019 11:56:16 A.M. EDT Mark Asselstine wrote:
> > The current devtool test for the building of an out-of-tree kernel
> > module uses something which requires several "high order" kconfigs
> > to
> > be set. This results in the test failing, not for expected reasons,
> > but rather because it depends on specific kernel configuration.
> > 
> > You will get error messages such as
> > 
> >   ERROR: "video_ioctl2"
> >  
> > [.../1.0-r5/testsdkext/workspace/sources/v4l2loopback-
> > driver/v4l2loopback.k
> > o] undefined!
> >   ERROR: "video_unregister_device"
> >  
> > [.../1.0-r5/testsdkext/workspace/sources/v4l2loopback-
> > driver/v4l2loopback.k
> > o] undefined!
> > 
> > Using a simpler hello-world kernel module example will only require
> > that CONFIG_MODULE is enabled, thus avoiding a false positive.
> > 
> > Signed-off-by: Mark Asselstine 
> > ---
> > 
> > **V2**
> >  - also updated the CROPS manual test json file
> >(I have no idea how to run this so untested but this is a
> > straight
> > substition)
> >  - continue to use github until we can move the sample to somewhere
> >like git.yoctoproject.org
> 
> Richard, are you thinking we need to figure out the
> git.yoctoproject.org repo 
> before we can move ahead with merging this?

I would like to fix that, yes. I could ask Michael for help but I do
have access to sort it myself too.

I'm also trying to sort out a few other issues before -next can merge.

Since I'm failing to get to it,

Michael,

Could you clone:

https://github.com/masselstine/kernel-module-hello-world.git 

onto git.yp.org as a new repo and give write access to it for Mark,
myself, Bruce and Ross please?

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [OE-Core][PATCH] diffutils: Add valgrind to support ptest

2019-09-05 Thread Richard Purdie
On Thu, 2019-09-05 at 11:38 -0400, Peiran Hong wrote:
> From: Peiran Hong 
> 
> The test "strip-trailing-cr" for diffutils package is skipped since
> it requires
> valgrind to run. This commit adds valgrind to the run-time dependency
> of
> the recipe for diffutils.
> 
> Signed-off-by: Peiran Hong 
> ---
>  meta/recipes-extended/diffutils/diffutils_3.7.bb | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-extended/diffutils/diffutils_3.7.bb
> b/meta/recipes-extended/diffutils/diffutils_3.7.bb
> index 7daeee3513..2d763633b6 100644
> --- a/meta/recipes-extended/diffutils/diffutils_3.7.bb
> +++ b/meta/recipes-extended/diffutils/diffutils_3.7.bb
> @@ -17,7 +17,9 @@ acpaths = "-I ./m4"
>  
>  inherit ptest
>  
> -RDEPENDS_${PN}-ptest += "make"
> +# Include valgrind since the test "strip-trailing-cr" requires it to
> run, although
> +# valgrind is a heavy package for diffutils
> +RDEPENDS_${PN}-ptest += "make valgrind"

This effectively means "bitbake diffutils" now requires valgrind to
build before it. I'm not sure its worth doing that for a single test?

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 6/6] xz: Remove GPLv3 license checksum

2019-09-04 Thread richard . purdie
On Wed, 2019-09-04 at 16:18 -0400, Mark Hatle wrote:
> On 9/4/19 3:53 PM, Adrian Bunk wrote:
> > I am getting more and more confused about both the patch and the 
> > semantics of LICENSE.
> > 
> > The status quo in the recipe is:
> > 
> > <--  snip  ->
> > 
> > # The source includes bits of PD, GPLv2, GPLv3, LGPLv2.1+, but the
> > only file
> > # which is GPLv3 is an m4 macro which isn't shipped in any of our
> > packages,
> > # and the LGPL bits are under lib/, which appears to be used for
> > libgnu, which
> > # appears to be used for DOS builds. So we're left with GPLv2+ and
> > PD.
> > LICENSE = "GPLv2+ & GPL-3.0-with-autoconf-exception & LGPLv2.1+ &
> > PD"
> > LICENSE_${PN} = "GPLv2+"
> > LICENSE_${PN}-dev = "GPLv2+"
> > LICENSE_${PN}-staticdev = "GPLv2+"
> > LICENSE_${PN}-doc = "GPLv2+"
> > LICENSE_${PN}-dbg = "GPLv2+"
> > LICENSE_${PN}-locale = "GPLv2+"
> > LICENSE_liblzma = "PD"
> > 
> > LIC_FILES_CHKSUM =
> > "file://COPYING;md5=97d554a32881fee0aa283d96e47cb24a \
> > file://COPYING.GPLv2;md5=b234ee4d69f5fce4486a80
> > fdaf4a4263 \
> > file://COPYING.GPLv3;md5=d32239bcb673463ab874e8
> > 0d47fae504 \
> > file://COPYING.LGPLv2.1;md5=4fbd65380cdd2559510
> > 79008b364516c \
> > file://lib/getopt.c;endline=23;md5=2069b0ee7105
> > 72c03bb3114e4532cd84 \
> > "
> > 
> > <--  snip  -->
> > 
> > My confusion about the patch is that it removes COPYING.GPLv3 from 
> > LIC_FILES_CHKSUM but keeps GPL-3.0-with-autoconf-exception in
> > LICENSE.
> > 
> > My confusion about the semantics of LICENSE is that I fail to find
> > a 
> > clear statement in the documentation that the legal meaning of
> > LICENSE 
> > in OE is what Mark claims it would be. Is this just Marks personal 
> > opinion on what should be done, or is this undocumented tribal 
> > knowledge, or is the exact semantics of LICENSE documented
> > somewhere in a language that lawyers understand?
> > 
> > My guess for the latter would be "undocumented tribal knowledge",
> > and clarification is required what is actually correct or incorrect
> > here. And I think this is also what Mark was asking for.
> 
> It -was- documented at one time, but I suspect that documentation was
> revised
> and the language was lost (or it never made it into a final version
> of the docs.)
> 
> This is why I was suggesting the TSC weigh in, and just clarify that:
> 
> LICENSE = is the _source code_ license, and only includes items that
> are or
> could be included in the making of the binaries.  This does NOT
> include
> autoconf, automake, makefiles, etc that are only used during the
> build process,
> but not in the sources used to build the outputs.
> 
> LICENSE_ = is the binary license for a specific package.  It
> defaults to
> the same as the LICENSE.

Historically, most m4 files don't include their license. This meant
that we haven't accounted for them in LICENSE. Where they are present,
we've tended to turn a blind eye to them as they don't cover the
binaries produced which is the piece of key importance to most people,
this recipe was one of the ones which attempted to clarify the
situation.

I suspect we do need to include such things in LICENSE since for
example these licenses could have an impact on the license of output
from the source archiver for example which might include build scripts.

The result is rather ugly but licensing in general is :(

It also gets tricky as we delete many autoconf files and regenerate
them. You'd hope that was under the same license but...

Cheers,

Richard




-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 6/6] xz: Remove GPLv3 license checksum

2019-09-04 Thread Richard Purdie
On Wed, 2019-09-04 at 08:07 -0400, Mark Hatle wrote:
> On 9/3/19 1:59 PM, Wes Lindauer wrote:
> > Mark,
> > 
> > In reference to "It typically does NOT include the license of
> > things used to
> > build the software (such as makefiles, autoconf fragments, etc)".
> > Since the only file that is licensed under GPLv3 is a M4 macro,
> > does that mean
> > the current patch is still valid? Shouldn't the GPLv3 license be
> > removed from
> > this recipe?
> 
> Unless the M4 file is generating/injecting code into the build(very
> few I've
> seen do this), then I would say it's not under GPLv3 at all.  (And I
> wouldn't
> have included GPLv3 in the LICENSE statement.)
> 
> But we need more consensus then just me saying so.
> 
> This may be a good question for the OE-TSC to ensure that we have
> clarification
> on this issue, and it's not just me saying I think one way or
> another.

Not sure it needs to go to the TSC, we just need a patch which clearly
says why the LICENSE statement is incorrect. I don't think the original
patch in the series was clear about why GPLv3 didn't apply but if the
commit message is improved, its probably fine.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/2] python3: Expose PYTHON_BINABI in global config metadata

2019-09-04 Thread Richard Purdie
On Wed, 2019-09-04 at 11:10 -0700, Khem Raj wrote:
> packages can use
> 
> find_package(PythonInterp REQUIRED)
> find_package(PythonLibs REQUIRED)
> 
> while we control PYTHON pointing to native py3 the libs and include
> directories will then point to build host version, which can result
> in
> unexpected combination and if we are lucky we get errors if its quite
> different e.g. py2 libs/includes and py3 executable
> 
> This variable can be then used to export PYTHON_LIBRARY and
> PYTHON_INCLUDE_DIR so that above find_packages can work correctly
> 
> Signed-off-by: Khem Raj 
> ---
>  meta/conf/distro/include/tcmode-default.inc   | 3 +++
>  meta/recipes-devtools/python/python3_3.7.4.bb | 1 -
>  2 files changed, 3 insertions(+), 1 deletion(-)

Putting this into the global namespace seems like a really bad idea.
Can we not use a class like Alex mentions? I thought we already had
one?

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH V3] dnf.py: installroot support usrmerge

2019-09-04 Thread Richard Purdie
On Wed, 2019-09-04 at 16:04 +0800, Changqing Li wrote:
> Ping
> Test with/without  usrmege feature passed.
> With usrmerge:
> NOTE: test_dnf_installroot (dnf.DnfRepoTest)
> DEBUG: Checking if 'DISTRO_FEATURES' value contains 'usrmerge' to
> skip the test
> NOTE:  ... skipped 'Test run when not enable usrmerge'
> Test run when not enable usrmerge
> NOTE: test_dnf_installroot_usrmerge (dnf.DnfRepoTest)
> DEBUG: Checking if 'DISTRO_FEATURES' value contains 'usrmerge' to run
> the test
> DEBUG: [Running]$ ssh -l root -o UserKnownHostsFile=/dev/null -o
> StrictHostKeyChecking=no -o LogLevel=ERROR 192.168.7.16 export
> PATH=/usr/sbin:/sbin:/usr/bin:/bin; mkdir -p
> /home/root/chroot/test/etc
> 
> 
> Without usrmerge:
> NOTE: test_dnf_installroot (dnf.DnfRepoTest)
> DEBUG: Checking if 'DISTRO_FEATURES' value contains 'usrmerge' to
> skip the test
> DEBUG: [Running]$ ssh -l root -o UserKnownHostsFile=/dev/null -o
> StrictHostKeyChecking=no -o LogLevel=ERROR 192.168.7.10 export
> PATH=/usr/sbin:/sbin:/usr/bin:/bin; mkdir -p
> /home/root/chroot/test/etc
> DEBUG: time: 1567404974.8490767, endtime: 1567406474.8424258
> DEBUG: [Command returned '0' after 0.13 seconds]
> DEBUG: Command: mkdir -p /home/root/chroot/test/etc
> Output:  
> ...
> NOTE:  ... ok
> NOTE: test_dnf_installroot_usrmerge (dnf.DnfRepoTest)
> DEBUG: Checking if 'DISTRO_FEATURES' value contains 'usrmerge' to run
> the test
> NOTE:  ... skipped 'Test run when enable usrmege'
> Test run when enable usrmege

In a previous version of this patch, it introduced a failure in the
parselogs test as an error was generated into the logs. I haven't
tested the new version of the patch, was that issue addressed?

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] Summary of M3 status

2019-09-04 Thread richard . purdie
On Tue, 2019-09-03 at 19:59 -0700, akuster808 wrote:
> 
> On 9/3/19 5:39 AM, Bruce Ashfield wrote:
> > On Tue, Sep 3, 2019 at 4:10 AM Richard Purdie
> >  wrote:
> > > I'm going to summarise the current state as it will help people
> > > see the
> > > set of issues we have. We have three issues blocking the current
> > > patch
> > > queue and kernel uprev, spread over four machines and five
> > > autobuilder
> > > targets:
> > > 
> > > qemumips64:
> > > 
> > >   
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/987
> > > 
> > >   RESULTS - parselogs.ParseLogsTest.test_parselogs: FAILED
> > > (1.97s)
> > > 
> > >   Central error: [1.689184] kprobes: failed to populate
> > > blacklist: -22
> > >   [happens 5.2 kernel only]
> > I'll have a look at the kprobes log. Could just be something we
> > need
> > to whitelist, but I'll root cause it.
> > 
> > > qemumips:
> > > 
> > >   qemu machine locks up:
> > > 
> > >   
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/982
> > > 
> > >   (happens with 5.0 and 5.2 kernels and qemu 4.0 and 4.1 and is
> > > in
> > >master. Not sure of the original cause)
> > > 
> > > qemuarm64:
> > > 
> > >   Central error: [1.816386] kprobes: failed to populate
> > > blacklist: -22
> > > 
> > >   RESULTS - parselogs.ParseLogsTest.test_parselogs: FAILED
> > > (1.11s)
> > > 
> > >   
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/97/builds/329
> > >   
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/985
> > >   [happens 5.2 kernel only]
> > > 
> > > qemuarm:
> > > 
> > >   
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/38/builds/990
> > >   
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/989
> > > 
> > >   See compile failure for stap.StapTest.test_stap below.
> > >   [happens 5.2 kernel only]
> > And I can look at this one as well.
> > 
> > If anyone gets to them first, let me know!
> 
> There is a patch under test in master-next. I just noticed it. Guess
> it
> means some got to it before either of us.

It means we had autobuilder time so I tried the obvious fix. It didn't
help and the problem remains, help very much appreciated.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] Yocto Project Status WW36’19

2019-09-03 Thread Richard Purdie
Current Dev Position: YP 2.8 M4 Feature FreezeNext Deadline: YP 3.0
Final Release 25th Oct
SWAT Team Rotation:SWAT lead is currently: ArminSWAT team rotation:
Armin -> Paul on Sept. 6, 2019SWAT team rotation: Paul -> Ross on Sept.
13, 2019
https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team
Next Team Meetings:Bug Triage meeting Thursday Sept 5th at 7:30am PDT (
https://zoom.us/j/454367603)Monthly Project Meeting Tuesday Sept. 3rd
at 8am PDT (https://zoom.us/j/990892712) Twitch - Next event is Tuesday
Sept. 10th at 8am PDT (https://www.twitch.tv/yocto_project)
Key Status/Updates:We’re now in feature freeze for 3.0 and working on
bug fixing for final releaseSeveral key 3.0 changes have now merged:5.2
linux-libc-kernel-headers5.2 kernel recipesRemoval of LSB and poky-lsb
and replacement with alt config testingPatches have significantly
reduced the dependency on python2Reproducible builds are now being
tested for core-image-minimalThere are the following changes still
under discussion/testing:Systemd vs. sysvinit defaults (maybe for the
poky alternative configuration tests)Making 5.2 the default kernel for
qemu (has regressions, see below)Hashserv performance issue
resolutionFinal configuration of hash equivalencytoolchain testsuite
patch seriesThe sstate hash equivalency continues to have challenges:we
need to rework the client/server communication to scale to the
autobuilderthere are some bugs in the code the autobuilder continues to
exposewe have cases where its not 100% clear what the correct behaviour
should betest cases are proving to be problematic to write in a
maintainable wayPatch merging is roughly up to date againWe have
several significant problematic bugs that are being worked on:5.2 stap
build issue on qemuarm5.2 kprobes error message (qemuarm64,
qemumips64)qemumips machine hangIf anyone has any status items for the
project they’d like to add to the weekly reports, please email
Richard+Stephen.
Planned Releases for YP 3.0 {2.8}:M3 Release 6th SeptM4 Cutoff 30th
Sept - this will be YP 3.0.YP 3.0 {2.8 (M4)} Final Release 25th Oct
Planned upcoming dot releases:YP 2.7.2 (Warrior) is planned for after
YP 3.0 release.YP 2.6.4 (Thud) is planned for after YP 2.7.2  release.
Tracking Metrics:WDD 2413 (last week 2485) (
https://wiki.yoctoproject.org/charts/combo.html)Poky Patch
Metrics  Total patches found: 1470 (last week 1474)Patches in the
Pending State: 611 (42%) [last week 613 (41%)]
Key Status Links for YP:
https://wiki.yoctoproject.org/wiki/Yocto_Project_v2.8_Statushttps://wiki.yoctoproject.org/wiki/Yocto_2.8_Schedulehttps://wiki.yoctoproject.org/wiki/Yocto_2.8_Features
The Yocto Project’s technical governance is through its Technical
Steering Committee, more information is available at:
https://wiki.yoctoproject.org/wiki/TSC
The Status reports are now stored on the wiki at: 
https://wiki.yoctoproject.org/wiki/Weekly_Status
[If anyone has suggestions for other information you’d like to see on
this weekly status update, let us know!]


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] lttng: babeltrace: Fix negative clock offset

2019-09-03 Thread richard . purdie
On Tue, 2019-09-03 at 16:53 -0400, Jonathan Rajotte-Julien wrote:
> Hi,
> 
> This patch breaks the API of stable-1.5. It is clearly mentioned in
> the commit message:
> 
>  It introduces API-breaking changes in the C and Python APIs, since
> we
>  need to be able to return negative time values, which were
> previously
>  used as errors (-1ULL).
> 
> You can have it in your layer but it should never be upstreamed to
> yocto/oe in any case for the stable 1.5 branch of babeltrace.
> 
> The change you are referencing
> (61cf588beae752e5ddfc60b6b5310f769ac9e852) is queued to be in
> Babeltrace 2.0.0.
> 
> Richard: as part of the lttng/babeltrace team I advice against using
> this patch in OE.

Thanks, it was in master-next but its now been removed locally and will
propagate. I appreciate the advice.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] Summary of M3 status

2019-09-03 Thread Richard Purdie
I'm going to summarise the current state as it will help people see the
set of issues we have. We have three issues blocking the current patch
queue and kernel uprev, spread over four machines and five autobuilder
targets:

qemumips64:

  https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/987

  RESULTS - parselogs.ParseLogsTest.test_parselogs: FAILED (1.97s)

  Central error: [1.689184] kprobes: failed to populate blacklist: -22
  [happens 5.2 kernel only]

qemumips:

  qemu machine locks up:

  https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/982

  (happens with 5.0 and 5.2 kernels and qemu 4.0 and 4.1 and is in 
   master. Not sure of the original cause)

qemuarm64:

  Central error: [1.816386] kprobes: failed to populate blacklist: -22

  RESULTS - parselogs.ParseLogsTest.test_parselogs: FAILED (1.11s)

  https://autobuilder.yoctoproject.org/typhoon/#/builders/97/builds/329
  https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/985
  [happens 5.2 kernel only]

qemuarm:

  https://autobuilder.yoctoproject.org/typhoon/#/builders/38/builds/990
  https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/989

  See compile failure for stap.StapTest.test_stap below.
  [happens 5.2 kernel only]

The qemumips issue is of particular concern.

Also needed to sort out M3 are:

 * toolchain tests nrossi is working on
 * resolution to the hashserv scaling problem
 * bug fixing for the hash equiv code

Cheers,

Richard

stap.StapTest.test_stap failure:


WARNING: Kernel function symbol table missing [man warning::symbols]
In file included from /usr/share/systemtap/runtime/linux/runtime.h:214,
 from /usr/share/systemtap/runtime/runtime.h:26,
 from /tmp/stapgYOLtD/stap_19208_src.c:26:
/usr/share/systemtap/runtime/linux/access_process_vm.h: In function 
'__access_process_vm_':
/usr/share/systemtap/runtime/linux/access_process_vm.h:28:8: error: implicit 
declaration of function 'get_task_mm'; did you mean 'get_task_comm'? 
[-Werror=implicit-function-declaration]
   28 |   mm = get_task_mm (tsk);
  |^~~
  |get_task_comm
/usr/share/systemtap/runtime/linux/access_process_vm.h:28:6: error: assignment 
to 'struct mm_struct *' from 'int' makes pointer from integer without a cast 
[-Werror=int-conversion]
   28 |   mm = get_task_mm (tsk);
  |  ^
/usr/share/systemtap/runtime/linux/access_process_vm.h:51:61: error: passing 
argument 6 of 'get_user_pages_remote' makes pointer from integer without a cast 
[-Werror=int-conversion]
   51 |   ret = get_user_pages_remote (tsk, mm, addr, 1, write, 1, , 
);
  | ^
  | |
  | int
In file included from ./include/linux/kallsyms.h:12,
 from ./include/linux/ftrace.h:11,
 from ./include/linux/kprobes.h:29,
 from /usr/share/systemtap/runtime/linux/runtime.h:21,
 from /usr/share/systemtap/runtime/runtime.h:26,
 from /tmp/stapgYOLtD/stap_19208_src.c:26:
./include/linux/mm.h:1592:46: note: expected 'struct page **' but argument is 
of type 'int'
 1592 |unsigned int gup_flags, struct page **pages,
  |~~^
In file included from /usr/share/systemtap/runtime/linux/runtime.h:214,
 from /usr/share/systemtap/runtime/runtime.h:26,
 from /tmp/stapgYOLtD/stap_19208_src.c:26:
/usr/share/systemtap/runtime/linux/access_process_vm.h:51:64: error: passing 
argument 7 of 'get_user_pages_remote' from incompatible pointer type 
[-Werror=incompatible-pointer-types]
   51 |   ret = get_user_pages_remote (tsk, mm, addr, 1, write, 1, , 
);
  |^
  ||
  |struct 
page **
In file included from ./include/linux/kallsyms.h:12,
 from ./include/linux/ftrace.h:11,
 from ./include/linux/kprobes.h:29,
 from /usr/share/systemtap/runtime/linux/runtime.h:21,
 from /usr/share/systemtap/runtime/runtime.h:26,
 from /tmp/stapgYOLtD/stap_19208_src.c:26:
./include/linux/mm.h:1593:32: note: expected 'struct vm_area_struct **' but 
argument is of type 'struct page **'
 1593 |struct vm_area_struct **vmas, int *locked);
  |^~~~
In file included from /usr/share/systemtap/runtime/linux/runtime.h:214,
 from /usr/share/systemtap/runtime/runtime.h:26,
 from /tmp/stapgYOLtD/stap_19208_src.c:26:
/usr/share/systemtap/runtime/linux/access_process_vm.h:51:71: error: passing 
argument 8 of 

Re: [OE-core] [PATCH] gcc-cross: Clean up fixed-includes

2019-08-30 Thread richard . purdie
On Fri, 2019-08-30 at 09:51 -0700, Khem Raj wrote:
> On Fri, Aug 30, 2019 at 8:54 AM Richard Purdie
>  wrote:
> > We had interesting failures where building gcc-cross-powerpc with
> > 5.0 kernel
> > headers, then building eudev after moving to 5.2 headers failed.
> > 
> > gcc-cross doesn't rebuild when linux-libc-headers changes due to
> > its
> > listing in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS. This shouldn't matter
> > but
> > fixincludes as adding asm-generic/socket.h to its filtered list
> > which
> > was then replacing the real header with an older version. This
> > mismatch
> > lead to build failures.
> > 
> > We trust the Linux kernel headers to be ANSI safe so lets just
> > clear out
> > any headers and trust the originals to be correct.
> > 
> > Signed-off-by: Richard Purdie 
> > ---
> >  meta/recipes-devtools/gcc/gcc-cross.inc | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc
> > b/meta/recipes-devtools/gcc/gcc-cross.inc
> > index 6222c2e8c91..e417b898734 100644
> > --- a/meta/recipes-devtools/gcc/gcc-cross.inc
> > +++ b/meta/recipes-devtools/gcc/gcc-cross.inc
> > @@ -196,6 +196,8 @@ do_install () {
> > # We use libiberty from binutils
> > find ${D}${exec_prefix}/lib -name libiberty.a | xargs rm -f
> > find ${D}${exec_prefix}/lib -name libiberty.h | xargs rm -f
> > +
> > +   find ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include-fixed
> > -type f -not -name "README" -not -name limits.h -not -name
> > syslimits.h | xargs rm -f
> 
> Whats different for limits.h and syslimits.h as compared to standard
> headers from libc, Maybe we should just get that sorted as well while
> here

limits.h looks to be gcc specific additions/fallback. syslimits.h is
just a passthrough. I decided to go with less risk in the patch and
leave those as is but am open to more investigation.

I suspect limits.h is there for pre glibc bootstrap?

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] base.bbclass: add dependency on pseudo from do_prepare_recipe_sysroot

2019-08-30 Thread Richard Purdie
On Fri, 2019-08-16 at 11:13 +0200, Mattias Hansson wrote:
> do_prepare_recipe_sysroot may perform groupadd, which requires pseudo.
> However, do_prepare_recipe_sysroot does not depend on pseudo explicitly,
> which sometimes causes a build error when building a recipe that adds
> groups.
> 
> This issue only occurs when executing do_prepare_recipe_sysroot for a
> recipe that adds groups before finishing a task that depends on pseudo
> for a recipe that doesn't add groups.
> 
> Signed-off-by: Mattias Hansson 
> ---
>  meta/classes/base.bbclass | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index 0c8a4b2862..0576b110c9 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -480,6 +480,7 @@ python () {
>  # If we're building a target package we need to use fakeroot (pseudo)
>  # in order to capture permissions, owners, groups and special files
>  if not bb.data.inherits_class('native', d) and not 
> bb.data.inherits_class('cross', d):
> +d.setVarFlag('do_prepare_recipe_sysroot', 'fakeroot', '1')
>  d.setVarFlag('do_unpack', 'umask', '022')
>  d.setVarFlag('do_configure', 'umask', '022')
>  d.setVarFlag('do_compile', 'umask', '022')

This basically forces all target recipes prepare-recipe sysroot to run
under pseudo "just in case", with all the performance overhead that
entails. prepare_recipe_sysroot does a lot of file accesses so this is
significant. It will also increase the pseudo database sizes
everywhere.

We'll need to find a better way to handle this I'm afraid.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] Automatic prebuilt management

2019-08-30 Thread Richard Purdie
On Fri, 2019-08-30 at 18:27 +0200, Loic Poulain wrote:
> HI Andre,
> 
> On Fri, 23 Aug 2019 at 20:11, Andre McCurdy 
> wrote:
> > On Mon, Aug 5, 2019 at 9:06 AM Loic Poulain <
> > loic.poul...@linaro.org> wrote:
> > > Say a company works with an OE internal tree, with open-source
> > and closed-source packages. The company wants to release the tree
> > so that its customers can fully customize the images/distro. The
> > closed sources being only available internally, the company has to
> > generate and handle prebuilt binaries for proprietary packages.
> > Ideally, with a unified public/internal OE tree (same recipes for
> > internal/public build).
> > >
> > > This question is actually similar to an older thread:
> > > https://marc.info/?l=openembedded-core=146779329804683
> > 
> > That very old thread was also referenced recently in a follow up
> > where
> > I shared a later solution, see:
> > 
> >   
> > http://lists.openembedded.org/pipermail/openembedded-core/2019-July/284896.html
> 
> Thanks, I missed that one.
> 
> > 
> > > I started to work on this and added a 'generate-prebuilt' class
> > which generates a tarball of ${D} in deploy/prebuilts after
> > do_install task. Symmetrically, It's also possible to create an
> > 'install-prebuilt' class which bypasses do_fetch, do_unpack, ...,
> > do_install with noexec flag and instead uncompresses a previously
> > generated prebuilt tarball into ${D} and continue normally. But I
> > would like something smarter, e.g. first trying to check if the
> > SRC_URI is available, if not switching on using the prebuilt
> > package if available (e.g. in a DIR_PREBUILT)...
> > >
> > > Before going further is there already an existing solution for
> > that? do you have any recommendation on the easier/best way to
> > achieve this?
> > 
> > I did initially try the approach of having a single recipe which
> > can
> > automatically support both building from source and extracting a
> > prebuilts tar file, but that (for me anyway) turned out to be a
> > dead
> > end. Building from source requires build dependencies and config
> > options but extracting a prebuilt tar file does not, so the two end
> > up
> > sharing very little... so 90% of the recipe ends up being
> > conditional
> > on which mode it's running in. The solution I ended up with (see
> > link
> > above) was for the class which creates the prebuilt tar file to
> > also
> > create a dedicated mini recipe to extract it. 
> > From my experience however an equally hard part of the problem is
> > the
> > distribution of prebuilts. It starts off easy (you share a tar file
> > via email or an ftp site and the receiver manually copies to their
> > downloads directory...) but that doesn't scale if you need to make
> > regular updates.
> 
> Very good point. maybe having binaries also in a repo could help a
> bit (e.g. using git LFS).
>  
> > My solution is discussed a little more in the thread
> > linked to above.
> 
> Your solution is interesting and I'll probably jump on the thread
> with more questions.
> Having an upstream solution part of oe-core would be nice though.

We'd be happy to have one, it will take someone to propose something
suitably general purpose that others can use along with tests.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 0/5] kernel-yocto: misc build / config changes

2019-08-30 Thread richard . purdie
On Fri, 2019-08-30 at 10:50 -0400, Bruce Ashfield wrote:
> On Fri, Aug 30, 2019 at 10:39 AM 
> wrote:
> > On Thu, 2019-08-29 at 11:32 -0400, Bruce Ashfield wrote:
> > > On Thu, Aug 29, 2019 at 11:31 AM Bruce Ashfield
> > >  wrote:
> > > > On Thu, Aug 29, 2019 at 11:28 AM Jonathan Rajotte-Julien
> > > >  wrote:
> > > > > > FWIW: lttng-ust builds fine for me with the 5.2 kernel +
> > > > > > 5.2
> > > > > > headers,
> > > > > > it is one of the things I build and test as part of
> > > > > > core-image-kernel-dev.
> > > > > 
> > > > > Even on ppc?
> > > > > 
> > > > 
> > > > yup. I build all arches for core-image-kernel-dev, which
> > > > includes
> > > > all of lttng.
> > > 
> > > but I've recently switched to RP's master-next branch, so
> > > hopefully a
> > > difference in the configs will pop out.
> > 
> > I'll post here about what I'm finding. eudev fails since
> > SO_SNDTIMEO
> > isn't defined. This comes from asm-generic/socket.h which
> > asm/socket.h
> > include.
> > 
> > richard@jet:work/ppce300c3-poky-linux/eudev/3.2.8-
> > r0/build/src/shared$ /media/build1/poky/build/tmp/work/ppce300c3-
> > poky-linux/eudev/3.2.8-r0/recipe-sysroot-native/usr/bin/powerpc-
> > poky-linux/powerpc-poky-linux-gcc -m32 -mhard-float -mcpu=e300c3
> > -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-
> > security -Werror=format-security --
> > sysroot=/media/build1/poky/build/tmp/work/ppce300c3-poky-
> > linux/eudev/3.2.8-r0/recipe-sysroot -DHAVE_CONFIG_H -I.
> > -I../../../eudev-3.2.8/src/shared -I../.. -DUDEV_ROOT_RUN=\"/run\"
> > -include ../../config.h -O2 -pipe -g -feliminate-unused-debug-types 
> > -fmacro-prefix-map=/media/build1/poky/build/tmp/work/ppce300c3-
> > poky-linux/eudev/3.2.8-r0=/usr/src/debug/eudev/3.2.8-r0 -fdebug-
> > prefix-map=/media/build1/poky/build/tmp/work/ppce300c3-poky-
> > linux/eudev/3.2.8-r0=/usr/src/debug/eudev/3.2.8-r0 -fdebug-prefix-
> > map=/media/build1/poky/build/tmp/work/ppce300c3-poky-
> > linux/eudev/3.2.8-r0/recipe-sysroot= -fdebug-prefix-
> > map=/media/build1/poky/build/tmp/work/ppce300c3-poky-
> > linux/eudev/3.2.8-r0/recipe-sysroot-native= -c ../../../eudev-
> > 3.2.8/src/shared/log.c  -fPIC -DPIC -o .libs/log.o
> > ../../../eudev-3.2.8/src/shared/log.c: In function
> > 'create_log_socket':
> > ../../../eudev-3.2.8/src/shared/log.c:128:43: error: 'SO_SNDTIMEO'
> > undeclared (first use in this function); did you mean 'SO_TXTIME'?
> >   128 | (void) setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, ,
> > sizeof(tv));
> >   |   ^~~
> >   |   SO_TXTIME
> > 
> > The file:
> > ppce300c3-poky-linux/eudev/3.2.8-r0/recipe-sysroot/usr/include/asm-
> > generic/socket.h
> > isn't included.
> > 
> > if I put:
> > 
> > #include 
> > 
> > at the top of log.c it will build, everything else doesn't.
> > 
> > Debugging with gcc -e shows its including a different 'interesting'
> > file.
> > 
> > My build has a
> > build/tmp/work/ppce300c3-poky-linux/eudev/3.2.8-r0/recipe-sysroot-
> > native/usr/lib/powerpc-poky-linux/gcc/powerpc-poky-
> > linux/9.2.0/include-fixed/asm-generic/socket.h
> > which looks totally wrong when I compare it to the real asm-
> > generic/socket.h.
> 
> This is similar to what I was seeing on some early qemuppc builds as
> well. fixincludes was tossing out the original file and putting in a
> broken variant. I had to hide the "fixed" files to build. But then it
> stopped happening, so I moved on.
> 
> I'm not familiar with how/when fixincludes fires, so I have no idea
> about any possible races, etc, that could be happening.
> 
> I'll do some more research down that route to see if anything pops
> up.

For the achieves, I've posted a patch which should fix this. Its due to
headers leaking into includes-fixed in gcc which has its dependency on
linux-libc-headers marked as "safe" for various reasons.

We don't need gcc to do this on linux so we can remove them. It
explains Bruce's previous fun here too.

We'll try a new build and see how things look with that fix.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH] gcc-cross: Clean up fixed-includes

2019-08-30 Thread Richard Purdie
We had interesting failures where building gcc-cross-powerpc with 5.0 kernel
headers, then building eudev after moving to 5.2 headers failed.

gcc-cross doesn't rebuild when linux-libc-headers changes due to its
listing in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS. This shouldn't matter but
fixincludes as adding asm-generic/socket.h to its filtered list which
was then replacing the real header with an older version. This mismatch
lead to build failures.

We trust the Linux kernel headers to be ANSI safe so lets just clear out
any headers and trust the originals to be correct.

Signed-off-by: Richard Purdie 
---
 meta/recipes-devtools/gcc/gcc-cross.inc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc 
b/meta/recipes-devtools/gcc/gcc-cross.inc
index 6222c2e8c91..e417b898734 100644
--- a/meta/recipes-devtools/gcc/gcc-cross.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross.inc
@@ -196,6 +196,8 @@ do_install () {
# We use libiberty from binutils
find ${D}${exec_prefix}/lib -name libiberty.a | xargs rm -f
find ${D}${exec_prefix}/lib -name libiberty.h | xargs rm -f
+
+   find ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include-fixed -type f -not 
-name "README" -not -name limits.h -not -name syslimits.h | xargs rm -f
 }
 
 do_package[noexec] = "1"
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 0/5] kernel-yocto: misc build / config changes

2019-08-30 Thread richard . purdie
On Thu, 2019-08-29 at 11:32 -0400, Bruce Ashfield wrote:
> On Thu, Aug 29, 2019 at 11:31 AM Bruce Ashfield
>  wrote:
> > On Thu, Aug 29, 2019 at 11:28 AM Jonathan Rajotte-Julien
> >  wrote:
> > > > FWIW: lttng-ust builds fine for me with the 5.2 kernel + 5.2
> > > > headers,
> > > > it is one of the things I build and test as part of
> > > > core-image-kernel-dev.
> > > 
> > > Even on ppc?
> > > 
> > 
> > yup. I build all arches for core-image-kernel-dev, which includes
> > all of lttng.
> 
> but I've recently switched to RP's master-next branch, so hopefully a
> difference in the configs will pop out.

I'll post here about what I'm finding. eudev fails since SO_SNDTIMEO
isn't defined. This comes from asm-generic/socket.h which asm/socket.h
include.

richard@jet:work/ppce300c3-poky-linux/eudev/3.2.8-r0/build/src/shared$ 
/media/build1/poky/build/tmp/work/ppce300c3-poky-linux/eudev/3.2.8-r0/recipe-sysroot-native/usr/bin/powerpc-poky-linux/powerpc-poky-linux-gcc
 -m32 -mhard-float -mcpu=e300c3 -fstack-protector-strong -D_FORTIFY_SOURCE=2 
-Wformat -Wformat-security -Werror=format-security 
--sysroot=/media/build1/poky/build/tmp/work/ppce300c3-poky-linux/eudev/3.2.8-r0/recipe-sysroot
 -DHAVE_CONFIG_H -I. -I../../../eudev-3.2.8/src/shared -I../.. 
-DUDEV_ROOT_RUN=\"/run\" -include ../../config.h -O2 -pipe -g 
-feliminate-unused-debug-types 
-fmacro-prefix-map=/media/build1/poky/build/tmp/work/ppce300c3-poky-linux/eudev/3.2.8-r0=/usr/src/debug/eudev/3.2.8-r0
 
-fdebug-prefix-map=/media/build1/poky/build/tmp/work/ppce300c3-poky-linux/eudev/3.2.8-r0=/usr/src/debug/eudev/3.2.8-r0
 
-fdebug-prefix-map=/media/build1/poky/build/tmp/work/ppce300c3-poky-linux/eudev/3.2.8-r0/recipe-sysroot=
 -fdebug-prefix-map=/media/build1/poky/build/tmp/work/ppce
 300c3-poky-linux/eudev/3.2.8-r0/recipe-sysroot-native= -c 
../../../eudev-3.2.8/src/shared/log.c  -fPIC -DPIC -o .libs/log.o
../../../eudev-3.2.8/src/shared/log.c: In function 'create_log_socket':
../../../eudev-3.2.8/src/shared/log.c:128:43: error: 'SO_SNDTIMEO' undeclared 
(first use in this function); did you mean 'SO_TXTIME'?
  128 | (void) setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, , sizeof(tv));
  |   ^~~
  |   SO_TXTIME

The file:
ppce300c3-poky-linux/eudev/3.2.8-r0/recipe-sysroot/usr/include/asm-generic/socket.h
isn't included.

if I put:

#include 

at the top of log.c it will build, everything else doesn't.

Debugging with gcc -e shows its including a different 'interesting' file.

My build has a 
build/tmp/work/ppce300c3-poky-linux/eudev/3.2.8-r0/recipe-sysroot-native/usr/lib/powerpc-poky-linux/gcc/powerpc-poky-linux/9.2.0/include-fixed/asm-generic/socket.h
which looks totally wrong when I compare it to the real asm-generic/socket.h.

Why this is in the toolchain I don't know and I'll have to investigate further.

I worry it depends on which MACHINE you have set when you build 
gcc-cross-powerpc.

So we have some progress/a lead but more to be done.

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] Remove LSB support

2019-08-30 Thread Richard Purdie
On Fri, 2019-08-30 at 11:04 +0800, Robert Yang wrote:
> Hi Adrian,
> 
> On 8/26/19 1:21 AM, Adrian Bunk wrote:
> > The only part with some (marginal) usage is lsb_release,
> > which is split from the lsb package into an own lsb-release
> > package.
> > 
> > Signed-off-by: Adrian Bunk 
> > ---
> > https://github.com/AdrianBunk/meta-lsb contains a preliminary
> 
> Do you have a plan to move it to git.yoctoproject.org or
> git.openembedded.org, 
> please?

I'm happy to have it on git.yoctoproject.org but we need a
maintainer...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH] ca-certificates: Fix reproducibilty and multilib issue

2019-08-29 Thread Richard Purdie
This command was dependent on the order of files on the disk and for multilib 
builds
could result in:

Error: Transaction check error:
  file /etc/ca-certificates.conf conflicts between attempted installs of 
ca-certificates-20190110-r0.core2_32 and 
lib64-ca-certificates-20190110-r0.x86_64

Sorting the file makes things deterministic.

Signed-off-by: Richard Purdie 
---
 .../recipes-support/ca-certificates/ca-certificates_20190110.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-support/ca-certificates/ca-certificates_20190110.bb 
b/meta/recipes-support/ca-certificates/ca-certificates_20190110.bb
index efd9eaa71e4..7d87da10bb5 100644
--- a/meta/recipes-support/ca-certificates/ca-certificates_20190110.bb
+++ b/meta/recipes-support/ca-certificates/ca-certificates_20190110.bb
@@ -53,7 +53,7 @@ do_install () {
 echo "# Lines starting with ! will remove certificate on next update"
 echo "#"
 find ${D}${datadir}/ca-certificates -type f -name '*.crt' | \
-sed 's,^${D}${datadir}/ca-certificates/,,'
+sed 's,^${D}${datadir}/ca-certificates/,,' | sort
 } >${D}${sysconfdir}/ca-certificates.conf
 }
 
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/2] uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend

2019-08-29 Thread richard . purdie
On Thu, 2019-08-29 at 11:05 +0200, Daniel Klauer wrote:
> > That commit message isn't correct.
> > 
> > It removes cleandirs for the prefunc at the start of the prefunc and
> > cleandirs for the main function at the start of that main function.
> 
> Oh ok, so the prefuncs/postfuncs can have their own [cleandirs], that's
> interesting.
> 
> How about the following (trying to find a good way to explain it):

Yes, that sounds fine. The bit in brackets probably isn't needed.

Cheers,

Richard

> 
> -
> When inherited by the u-boot recipe (UBOOT_PN), uboot-sign.bbclass adds
> a concat_dtb step, which places additional files into ${DEPLOYDIR}
> before do_deploy.
> 
> The use of prefuncs for this prevents us from adding
>   do_deploy[cleandirs] = "${DEPLOYDIR}"
> because that would remove the files produced by the prefunc.
> (Each of prefuncs/task/postfuncs functions has their own cleandirs, and
> each function's cleandirs are removed at the start of that function.)
> 
> Thus, it seems good to make concat_dtb a part of do_deploy, such that it
> runs after removal of do_deploy's cleandirs. As before, care is taken to
> not interfere with the kernel's do_deploy definition, since concat_dtb
> was only needed for u-boot.
> -
> 

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 0/5] kernel-yocto: misc build / config changes

2019-08-29 Thread richard . purdie
On Wed, 2019-08-14 at 11:31 -0400, bruce.ashfi...@gmail.com wrote:
> From: Bruce Ashfield 
> 
> Hi all,
> 
> This pull request is a collection of smaller fixes that I've been
> collecting
> while I work through the 5.2/5.3 kernel intro, new libc-headers, etc.
> There's
> no reason to make them wait while I finish that work, so here they
> are in a
> single branch.
> 
> There's nothing major here, just a some legwork changes, licensing
> update
> and a short term workaround for embedded host paths in the kernel
> binaries.

There appear to be a few issues with the new headers. ppc looks to have
particular issues:

https://autobuilder.yoctoproject.org/typhoon/#/builders/70/builds/976

eudev:

util.lo ../../../eudev-3.2.8/src/shared/sysctl-util.c
| ../../../eudev-3.2.8/src/shared/log.c: In function 'create_log_socket':
| ../../../eudev-3.2.8/src/shared/log.c:123:43: error: 'SO_SNDTIMEO' undeclared 
(first use in this function); did you mean 'SO_TXTIME'?
|   123 | (void) setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, , 
sizeof(tv));
|   |   ^~~
|   |   SO_TXTIME
| ../../../eudev-3.2.8/src/shared/log.c:123:43: note: each undeclared 
identifier is reported only once for each function it appears in

neard:

../neard-0.16/plugins/p2p.c: In function 'p2p_connect_blocking':
../neard-0.16/plugins/p2p.c:324:33: error: 'SO_RCVTIMEO' undeclared (first use 
in this function); did you mean 'SO_RCVTIMEO_OLD'?
  324 |  if (setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, (char *),
  | ^~~
  | SO_RCVTIMEO_OLD
../neard-0.16/plugins/p2p.c:324:33: note: each undeclared identifier is 
reported only once for each function it appears in
../neard-0.16/plugins/p2p.c:328:33: error: 'SO_SNDTIMEO' undeclared (first use 
in this function); did you mean 'SO_TXTIME'?
  328 |  if (setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, (char *),
  | ^~~
  | SO_TXTIME
../neard-0.16/plugins/p2p.c: In function 'p2p_connect':
../neard-0.16/plugins/p2p.c:555:33: error: 'SO_RCVTIMEO' undeclared (first use 
in this function); did you mean 'SO_RCVTIMEO_OLD'?
  555 |  if (setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, (char *),
  | ^~~
  | SO_RCVTIMEO_OLD
../neard-0.16/plugins/p2p.c:559:33: error: 'SO_SNDTIMEO' undeclared (first use 
in this function); did you mean 'SO_TXTIME'?
  559 |  if (setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, (char *),
  | ^~~
  | SO_TXTIME
Makefile:1973: recipe for target 'plugins/src_neard-p2p.o' failed
make[1]: *** [plugins/src_neard-p2p.o] Error 1

and errors from lttng-ust (cc'd Jonathan).

qemuarm is erroring due to /dev/fb0 disappearing, which is probably kernel?

https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/975
https://autobuilder.yoctoproject.org/typhoon/#/builders/38/builds/976
https://autobuilder.yoctoproject.org/typhoon/#/builders/47/builds/986

and we also have an unintended side effect multilib build failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/995

which might be due to changed dependencies, I need to check into that one 
further.

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] common-licenses: update BSD-2-CLAUSE license text

2019-08-29 Thread Richard Purdie
On Wed, 2019-08-28 at 11:13 -0700, Andre McCurdy wrote:
> On Wed, Aug 28, 2019 at 3:05 AM Christophe PRIOUZEAU
>  wrote:
> > Using the generic BSD-2-CLAUSE license as specified on
> > https://opensource.org/licenses/BSD-2-Clause
> 
> Fixing these kind of issues has been discussed before. I think the
> consensus was that a wider ranging cleanup is needed. The thread
> starts here:
> 
>   
> http://lists.openembedded.org/pipermail/openembedded-core/2018-June/270475.html

I agree there is probably more cleanup needed but this patch seems like
a step in the right direction...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] serial-getty@.service: Allow device to fast fail if it does not exist

2019-08-28 Thread Richard Purdie
On Wed, 2019-08-28 at 15:24 -0500, Jason Wessel wrote:
> On 8/27/19 7:15 PM, richard.pur...@linuxfoundation.org wrote:
> > On Tue, 2019-08-27 at 19:03 -0500, Jason Wessel wrote:
> > > On 8/27/19 5:58 PM, Richard Purdie wrote:
> > > > Hi Jason,
> > > > Somehow this change is responsible for this build failure:
> > > > 
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/976
> > > > 
> > > > (steps 5c and 7c so failure during testimage).
> > > > 
> > > > I have bisected it to this change, I haven't looked into why.
> > > 
> > > Thanks for tracking it down.   I am sure how to try an duplicate
> > > this.  I clicked around to try and find out a bit about what it
> > > is
> > > running for these phases of the build but it is not very obvious.
> > > 
> > > Is there a local.conf I can try along with what ever commands it
> > > ran?
> > 
> > The configuration its using is shown in
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/978/steps/8/logs/stdio
> > for each step.
> > 
> 
> I tried the configuration on my system but it seems to work fine, see
> below.   Is there a way to get complete log file of the boot on the
> broken host?  I am also curious if it fails every time or not.
> 
> /home/pokybuild/yocto-worker/qa-
> extras2/build/build/tmp/work/qemux86_64-poky-linux/core-image-
> sato/1.0-r0/testimage/qemu_boot_log.2019082722
> 
> I am not sure it will tell us anything further or not, but it
> certainly looks like the system didn't boot correctly in the first
> place.

I've rerun it as 
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/983

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] serial-getty@.service: Allow device to fast fail if it does not exist

2019-08-28 Thread richard . purdie
On Wed, 2019-08-28 at 15:24 -0500, Jason Wessel wrote:
> On 8/27/19 7:15 PM, richard.pur...@linuxfoundation.org wrote:
> > On Tue, 2019-08-27 at 19:03 -0500, Jason Wessel wrote:
> > > On 8/27/19 5:58 PM, Richard Purdie wrote:
> > > > Hi Jason,
> > > > Somehow this change is responsible for this build failure:
> > > > 
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/976
> > > > 
> > > > (steps 5c and 7c so failure during testimage).
> > > > 
> > > > I have bisected it to this change, I haven't looked into why.
> > > 
> > > Thanks for tracking it down.   I am sure how to try an duplicate
> > > this.  I clicked around to try and find out a bit about what it
> > > is
> > > running for these phases of the build but it is not very obvious.
> > > 
> > > Is there a local.conf I can try along with what ever commands it
> > > ran?
> > 
> > The configuration its using is shown in
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/978/steps/8/logs/stdio
> > for each step.
> > 
> 
> I tried the configuration on my system but it seems to work fine, see
> below.   Is there a way to get complete log file of the boot on the
> broken host?  I am also curious if it fails every time or not.
> 
> /home/pokybuild/yocto-worker/qa-
> extras2/build/build/tmp/work/qemux86_64-poky-linux/core-image-
> sato/1.0-r0/testimage/qemu_boot_log.2019082722

If we'd got this straight after the build then yes but its been
recycled now.

It did seem to fail repeatedly when I tested it. I'd probably have to
add the patch back and retest again to get the log.

> I am not sure it will tell us anything further or not, but it
> certainly looks like the system didn't boot correctly in the first
> place.

It does seem like a boot issue somehow...

The test case one the autobuilder seems simple enough, systemd boot
alone with no sysvinit.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 4/8] kernel-devsrc: remove python2 dependency

2019-08-28 Thread richard . purdie
On Wed, 2019-08-28 at 23:05 +0300, Adrian Bunk wrote:
> On Wed, Aug 28, 2019 at 03:28:38PM -0400, bruce.ashfi...@gmail.com
> wrote:
> > From: Bruce Ashfield 
> > 
> > Witht the approaching EOL of python2, the kernel packages need to
> > be updated to depend on python3.
> > ...
> 
> Yocto 3.0 ships and supports Python 2.7, so there is no urgent need
> for 
> last-minute Python 2 -> 3 changes.

Removal of 2.X dependencies where we can was an objective of 3.0 and
2.X is EOL and will expire in the lifetime of 3.0. I'd say this is
therefore quite in keeping with that...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] glibc: Fix locale DEPENDS

2019-08-28 Thread Richard Purdie
On Tue, 2019-08-27 at 07:45 -0500, Joshua Watt wrote:
> gettext is required to generate the glibc locales in do_compile. If not
> present, glibc will skip the generation which isn't reproducible.
> 
> Signed-off-by: Joshua Watt 
> ---
>  meta/recipes-core/glibc/glibc.inc | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-core/glibc/glibc.inc 
> b/meta/recipes-core/glibc/glibc.inc
> index 252fd56c13c..f1a6ae2a245 100644
> --- a/meta/recipes-core/glibc/glibc.inc
> +++ b/meta/recipes-core/glibc/glibc.inc
> @@ -6,7 +6,7 @@ DEPENDS = "virtual/${TARGET_PREFIX}gcc libgcc-initial 
> linux-libc-headers"
>  
>  PROVIDES = "virtual/libc"
>  PROVIDES += "virtual/libintl virtual/libiconv"
> -inherit autotools texinfo distro_features_check systemd
> +inherit autotools texinfo distro_features_check systemd gettext

I suspect this may not do what you expect.

At least as I read the class and recipes, glibc sets
INHIBIT_DEFAULT_DEPS which means no gettext-native dependency is added,
instead, configure has --disable-nls added.

Is that what we want?

I'm curious to understand how glibc locales are ever generated
correctly and what we're aiming to add here (a gettext-native
dependency?)

FWIW gettext-native is a very heavy thing to add in as a dependency
from a build time perspective.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH V3] nfs-utils: decrease RLIMIT_NOFILE to 4k for systemd

2019-08-28 Thread richard . purdie
On Wed, 2019-08-28 at 09:29 +0800, Kang Kai wrote:
> On 2019/8/28 上午7:29, richard.pur...@linuxfoundation.org wrote:
> > On Tue, 2019-08-27 at 17:43 +0800, Kang Kai wrote:
> > > Hi Richard,
> > > 
> > > This patch could fix the test_image failure with systemd. Would
> > > like
> > > to
> > > try systemd as default init manager on yocto build again
> > > to check whether any more blocking issues?
> > 
> > There is at least one issue:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/67/builds/967
> > 
> > Please monitor this build which should show up any other issues:
> 
> Got it. Thanks.
> 
> 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/85/builds/645
> > 
> > (master-next was looking green other than this change).
> 
> It fails with libedit-native do fetch errors. I'll check it but it
> seems not systemd related at first sight.

Agreed, that is a different issue. For the systemd vs sysvinit we have:

qemux86-lsb parselogs failure:
https://autobuilder.yoctoproject.org/typhoon/#/builders/67/builds/967

qemuarm-lsb parselogs failure:
https://autobuilder.yoctoproject.org/typhoon/#/builders/38/builds/970

qemumips runtime systemd test failure:
https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/962

qemumips64 runtime systemd test failure:
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/967

oe-selftest failures:
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/658
2019-08-28 00:55:15,114 - oe-selftest - INFO - RESULTS -
runqemu.RunqemuTests.test_boot_machine_slirp_qcow2: ERROR (1025.19s)
2019-08-28 00:55:15,114 - oe-selftest - INFO - RESULTS -
runqemu.RunqemuTests.test_boot_recipe_image_vdi: ERROR (1010.73s)
2019-08-28 00:55:15,114 - oe-selftest - INFO - RESULTS -
runqemu.RunqemuTests.test_boot_recipe_image_vmdk: ERROR (1028.30s)
2019-08-28 00:55:15,114 - oe-selftest - INFO - RESULTS -
wic.Wic.test_iso_image: FAILED (63.97s)

and it looks like those runqemu failures happen on
qemux86-64:
https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/969
qemumips:
https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/962
qemux86:
https://autobuilder.yoctoproject.org/typhoon/#/builders/59/builds/961
qemuppc:
https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/961
qemumips64:
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/967
qemuarm:
https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/969

so it looks like 3-4 different issues.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/11] binutils: Add do_check task for executing binutils test suite

2019-08-28 Thread richard . purdie
On Wed, 2019-08-28 at 05:06 +, Nathan Rossi wrote:
> Create the do_check task to the binutils-cross include. This task can
> be
> used to execute the binutils test suite for the cross target
> binutils.
> By default this executes all the check targets of the binutils
> Makefile,
> this can however be changed by setting MAKE_CHECK_TARGETS to the
> desired
> test suite target (e.g. check-gas).
> 
> The binutils test suites do not require any target execution, as such
> the check target can be run without QEMU or a target device. However
> since the binutils tests do rely on a C compiler there is dependence
> on
> both gcc and libc in order to run the tests.
> 
> Signed-off-by: Nathan Rossi 

Looking at this a bit further we have a problem with gcc-cross and
binutils-cross.

The trouble is we extracted all "target" dependencies away from the
cross recipes. This means we build *-cross once per target arch but the
runtime pieces are build once per target tune.

The tests are very much target tune specific so shouldn't be in the
cross recipes.

This is going to cause worlds of pain when we switch between machines
that use the same arch but different tunes and the sysroot has files
from a previous different tune.

I suspect this will manifest itself should we run the signature checks
but there real underlying problem here people will run into that we'll
have to solve before this can merge.

There is a secondary issue later in the series with adding the test
results to "selftest".

On the autobuilder we run "selftest" once and assume its machine
independent. These results clearly aren't. This gives problems for both
running the tests and processing the results.

Equally, "cross" toolchain tests aren't really "runtime" but are
machine/target specific. FWIW, injecting the test results the way you
are doing is good and the right thing to do but I think we may need a
different class of test, or we inject them as a special case of runtime
test, or runtime ptest (a cross runtime ptest result?).

I'm happy to try and help figure out how to address some of this if I
can help somehow. I am conscious the release deadlines are pressing too
:(.

Cheers,

Richard









-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/11] binutils: Add do_check task for executing binutils test suite

2019-08-28 Thread richard . purdie
On Wed, 2019-08-28 at 05:06 +, Nathan Rossi wrote:
> Create the do_check task to the binutils-cross include. This task can be
> used to execute the binutils test suite for the cross target binutils.
> By default this executes all the check targets of the binutils Makefile,
> this can however be changed by setting MAKE_CHECK_TARGETS to the desired
> test suite target (e.g. check-gas).
> 
> The binutils test suites do not require any target execution, as such
> the check target can be run without QEMU or a target device. However
> since the binutils tests do rely on a C compiler there is dependence on
> both gcc and libc in order to run the tests.
> 
> Signed-off-by: Nathan Rossi 
> ---
>  meta/recipes-devtools/binutils/binutils-cross.inc | 28 
> +++
>  1 file changed, 28 insertions(+)
> 
> diff --git a/meta/recipes-devtools/binutils/binutils-cross.inc 
> b/meta/recipes-devtools/binutils/binutils-cross.inc
> index 02ec891606..76eb453f0e 100644
> --- a/meta/recipes-devtools/binutils/binutils-cross.inc
> +++ b/meta/recipes-devtools/binutils/binutils-cross.inc
> @@ -36,3 +36,31 @@ do_install () {
>   rmdir ${D}${STAGING_DIR_NATIVE}${prefix_native}/${libdir}64 || :
>   rmdir ${D}${STAGING_DIR_NATIVE}${prefix_native}/${prefix} || :
>  }
> +
> +EXTRA_OEMAKE_prepend_task-check = "${PARALLEL_MAKE} "
> +MAKE_CHECK_TARGETS ??= "check-binutils check-gas check-gold check-ld 
> check-libiberty"
> +
> +python () {
> +# crosssdk deps have different virtual targets
> +if bb.data.inherits_class('crosssdk', d):
> +d.appendVarFlag("do_check", "depends", " 
> virtual/${TARGET_PREFIX}gcc-crosssdk:do_populate_sysroot")
> +d.appendVarFlag("do_check", "depends", " 
> virtual/nativesdk-${TARGET_PREFIX}compilerlibs:do_populate_sysroot")
> +else:
> +d.appendVarFlag("do_check", "depends", " 
> virtual/${TARGET_PREFIX}gcc:do_populate_sysroot")
> +d.appendVarFlag("do_check", "depends", " 
> virtual/${TARGET_PREFIX}compilerlibs:do_populate_sysroot")
> +}

I'm torn here on whether we should do:

do_check[depends] += "${BINUTILS_TARGETDEPS}"
BINUTILS_TARGETDEPS = "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot 
virtual/${TARGET_PREFIX}compilerlibs:do_populate_sysroot"
BINUTILS_TARGETDEPS_class-crosssdk = 
"virtual/${TARGET_PREFIX}gcc-crosssdk:do_populate_sysroot 
virtual/nativesdk-${TARGET_PREFIX}compilerlibs:do_populate_sysroot"

instead of the above.

Cheers,

Richard


> +do_check[depends] += "dejagnu-native:do_populate_sysroot 
> expect-native:do_populate_sysroot"
> +do_check[depends] += "virtual/libc:do_populate_sysroot"
> +do_check[dirs] = "${B}"
> +do_check[nostamp] = "1"
> +do_check() {
> +# need to inject CC and CXX as the target CC and CXX with sysroot
> +oe_runmake -i ${MAKE_CHECK_TARGETS} \
> +RUNTESTFLAGS=" \
> +CC='${TARGET_PREFIX}gcc --sysroot=${STAGING_DIR_TARGET} 
> ${TUNE_CCARGS}' \
> +CXX='${TARGET_PREFIX}g++ --sysroot=${STAGING_DIR_TARGET} 
> ${TUNE_CCARGS}' \
> +"
> +}
> +addtask check after do_compile
> +
> ---
> 2.23.0.rc1
> 

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/2] uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend

2019-08-28 Thread Richard Purdie
On Wed, 2019-08-28 at 13:41 +0200, Daniel Klauer wrote:
> bitbake removes cleandirs once per prefunc and then again for the actual
> task.

That commit message isn't correct.

It removes cleandirs for the prefunc at the start of the prefunc and
cleandirs for the main function at the start of that main function.

The code is in lib/bb/build.py:

for func in (prefuncs or '').split():
exec_func(func, localdata)
exec_func(task, localdata)
for func in (postfuncs or '').split():
exec_func(func, localdata)

with the cleandirs happening in exec_func() for each function.

This does mean the cleandirs happens after the prefunc is called but
not as described above.

>  By moving the concat_dtb step here from prefunc to main task we can
> add
> do_deploy[cleandirs] = "${DEPLOYDIR}"
> to deploy.bbclass without losing the files produced by concat_dtb.

I think you also need to state here that this is only expected to be
run for the uboot recipe and not the kernel.

The patch itself is probably ok but I'd ask the commit message be
corrected please.

Cheers,

Richard

> It looks like using do_deploy(_append) was the original goal anyways.
> 
> I tested this with poky, putting the following in local.conf:
>   UBOOT_SIGN_ENABLE = "1"
>   KERNEL_CLASSES = " kernel-fitimage "
>   KERNEL_IMAGETYPE = "fitImage"
> and then doing
>   bitbake core-image-minimal
>   bitbake u-boot
> It builds successfully, the kernel and uboot recipes' temp/run.do_deploy
> scripts look good (kernel's do_deploy is not broken, and uboot's do_deploy
> still calls concat_dtb).
> 
> Signed-off-by: Daniel Klauer 
> ---
>  meta/classes/uboot-sign.bbclass | 11 ++-
>  1 file changed, 6 insertions(+), 5 deletions(-)
> 
> diff --git a/meta/classes/uboot-sign.bbclass b/meta/classes/uboot-sign.bbclass
> index 982ed46d01..713196df41 100644
> --- a/meta/classes/uboot-sign.bbclass
> +++ b/meta/classes/uboot-sign.bbclass
> @@ -117,15 +117,16 @@ do_install_append() {
>   fi
>  }
>  
> +do_deploy_prepend_pn-${UBOOT_PN}() {
> + if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ]; then
> + concat_dtb
> + fi
> +}
> +
>  python () {
>  if d.getVar('UBOOT_SIGN_ENABLE') == '1' and d.getVar('PN') == 
> d.getVar('UBOOT_PN') and d.getVar('UBOOT_DTB_BINARY'):
>  kernel_pn = d.getVar('PREFERRED_PROVIDER_virtual/kernel')
>  
>  # Make "bitbake u-boot -cdeploy" deploys the signed u-boot.dtb
>  d.appendVarFlag('do_deploy', 'depends', ' %s:do_deploy' % kernel_pn)
> -
> -# kernerl's do_deploy is a litle special, so we can't use
> -# do_deploy_append, otherwise it would override
> -# kernel_do_deploy.
> -d.appendVarFlag('do_deploy', 'prefuncs', ' concat_dtb')
>  



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] serial-getty@.service: Allow device to fast fail if it does not exist

2019-08-27 Thread richard . purdie
On Tue, 2019-08-27 at 19:03 -0500, Jason Wessel wrote:
> 
> On 8/27/19 5:58 PM, Richard Purdie wrote:
> > Hi Jason,
> > Somehow this change is responsible for this build failure:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/976
> > 
> > (steps 5c and 7c so failure during testimage).
> > 
> > I have bisected it to this change, I haven't looked into why.
> 
> Thanks for tracking it down.   I am sure how to try an duplicate
> this.  I clicked around to try and find out a bit about what it is
> running for these phases of the build but it is not very obvious.
> 
> Is there a local.conf I can try along with what ever commands it ran?

The configuration its using is shown in 
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/978/steps/8/logs/stdio
for each step.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH V3] nfs-utils: decrease RLIMIT_NOFILE to 4k for systemd

2019-08-27 Thread richard . purdie
On Tue, 2019-08-27 at 17:43 +0800, Kang Kai wrote:
> Hi Richard,
> 
> This patch could fix the test_image failure with systemd. Would like
> to 
> try systemd as default init manager on yocto build again
> to check whether any more blocking issues?


There is at least one issue:

https://autobuilder.yoctoproject.org/typhoon/#/builders/67/builds/967

Please monitor this build which should show up any other issues: 

https://autobuilder.yoctoproject.org/typhoon/#/builders/85/builds/645

(master-next was looking green other than this change).

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] serial-getty@.service: Allow device to fast fail if it does not exist

2019-08-27 Thread Richard Purdie
On Tue, 2019-08-20 at 17:27 -0700, Jason Wessel wrote:
> Some BSPs use a USB serial port which may or may not actually be
> plugged all the time.  It is quite useful to have a USB serial port
> have a getty running but it does not make sense to wait for it for 90
> seconds before completing the system startup if it might never get
> plugged in.  The typical example is that a USB serial device might
> only need to be plugged in when debugging, upgrading, or initially
> configuring a device.
> 
> This change is somewhat subtle.  Systemd uses the "BindsTo" directive
> to ensure existence of the device in order to start the service as
> well as to terminate the service if the device goes away.  The
> "After"
> directive makes that same relationship stronger, and has the
> undesired
> side effect that systemd will wait until its internal time out value
> for the device to come on line before executing a fail operation or
> letting other tasks and groups continue.  This is certainly the kind
> of behavior we want for a disk, but not for serial ports in general.
> 
> The kernel module loader and device detection will have run a long
> time before the getty startup.  By the time the getty startup occurs
> the system has all the serial devices its going to get.
> 
> If you want to observe the problem with qemu, it is easy to
> replicate.
> Simply add the following line to your local.conf for a x86-64 qemu
> build.
> 
> SERIAL_CONSOLES="115200;ttyS0 115200;ttyUSB0"
> 
> Login right after the system boots and observe:
> 
>root@qemux86-64:~# systemctl list-jobs |cat
>JOB UNIT TYPE  STATE
>  1 multi-user.targetstart waiting
> 69 serial-getty@ttyUSB0.service start waiting
> 64 getty.target start waiting
> 71 dev-ttyUSB0.device   start running
> 62 systemd-update-utmp-runlevel.service start waiting
> 
>5 jobs listed.
> 
> You can see above that the dev-ttyUSB0.device will block for 1min 30
> seconds.  While that might not be a problem for this reference build.
> It is certainly a problem for images that have software watchdogs
> that
> verify the system booted up all the way to systemd completion in less
> than 90 seconds.
> 
> This other nice effect of this change is that the fast fail device
> extend to additional serial ports that may not exist on ARM BSPs or
> that might be configured in or out by the dtb files on different
> boards.
> 
> Signed-off-by: Jason Wessel 
> ---
>  .../systemd/systemd-serialgetty/serial-getty@.service   | 2
> +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Hi Jason,

Somehow this change is responsible for this build failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/976

(steps 5c and 7c so failure during testimage).

I have bisected it to this change, I haven't looked into why.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] openssl: Fix documentation DEPENDS

2019-08-27 Thread Richard Purdie
On Tue, 2019-08-27 at 18:16 +0200, Alexander Kanavin wrote:
> On Tue, 27 Aug 2019 at 18:12, Joshua Watt 
> wrote:
> > It's not populated just once. Any task can add things to the RSS as
> > the 
> > build progresses. However, the RSS is only cleaned out at one
> > specific 
> > point early in the build, which I can't recall ATM.
> 
> Can you point me to a specific example of tasks adding things to RSS
> please? I'm fairly sure I haven't seen this happen yet, so would
> like  to know the mechanism and use cases.

It definitely does happen. An easy example is quilt-native for
do_patch. Another would be file-native or rpm-native for do_package.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] ltp: cve/meltdown.c: Fix kernel symbol finding

2019-08-27 Thread Richard Purdie
On Tue, 2019-08-27 at 12:38 +0800, zhe...@windriver.com wrote:
> From: He Zhe 
> 
> Backport a patch to fix the following error.
> safe_file_ops.c:219: BROK: Expected 3 conversions got 2 at
> meltdown.c:272
> 
> Signed-off-by: He Zhe 
> ---
>  ...-cve-meltdown.c-Fix-kernel-symbol-finding.patch | 83
> ++
>  meta/recipes-extended/ltp/ltp_20190517.bb  |  1 +
>  2 files changed, 84 insertions(+)
>  create mode 100644 meta/recipes-extended/ltp/ltp/0001-cve-
> meltdown.c-Fix-kernel-symbol-finding.patch

Patch doesn't apply, how was this tested?

https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/981/steps/8/logs/step1b

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 09/13] glibc: drop obsolete packaging of glibc libnsl libs

2019-08-27 Thread Richard Purdie
On Tue, 2019-08-27 at 16:56 +0300, Adrian Bunk wrote:
> On Fri, Aug 23, 2019 at 01:51:40PM -0700, Andre McCurdy wrote:
> > Packaging rules were left behind when libnsl was removed:
> > ...
> 
> How has this change been tested?
> 
> ERROR: glibc-2.30-r0 do_package: QA Issue: glibc: Files/directories
> were installed but not shipped in any package:
>   /lib/libnsl-2.30.so
>   /lib/libnsl.so.1
> Please set FILES such that these items are packaged. Alternatively if
> they are unneeded, avoid installing them or delete them within
> do_install.
> glibc: 2 installed and not shipped files. [installed-vs-shipped]
> ERROR: glibc-2.30-r0 do_package: Fatal QA errors found, failing task.

Showed up on the autobuilder too:

https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/981/steps/8/logs/step1b

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] Yocto Project Status WW35’19

2019-08-27 Thread Richard Purdie
Current Dev Position: YP 2.8 M4 Feature Freeze
Next Deadline: YP 3.0 Final Release 25th Oct

SWAT Team Rotation:
SWAT lead is currently: AnujSWAT team rotation: Anuj -> Armin on Aug.
30, 2019SWAT team rotation: Armin -> Paulj on Sept. 6, 2019
https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team
Next Team Meetings:
Bug Triage meeting Thursday August 29th at 7:30am PDT (
https://zoom.us/j/454367603)Monthly Project Meeting Tuesday Sept. 3rd
at 8am PDT (https://zoom.us/j/990892712) Twitch - Next event is Tuesday
Sept. 10th at 8am PDT (https://www.twitch.tv/yocto_project)
Key Status/Updates:
We’re now in feature freeze for 3.0 and working on bug fixing for final
releaseThere are the following planned features which are still under
consideration for 3.0:Systemd vs. sysvinit defaults (maybe for the poky
alternative configuration tests)Removal of LSB and poky-lsb and
replacement with alt config testingFinal configuration of hash
equivalencyThe sstate hash equivalency continues to have challenges:we
need to rework the client/server communication to scale to the
autobuilderthere are some bugs in the code the autobuilder continues to
exposewe have cases where its not 100% clear what the correct behaviour
should betest cases are proving to be problematic to write in a
maintainable wayPatch merging has been delayed due to vacations (RP and
Ross) so the final feature patches aren’t resolved yet.If anyone has
any status items for the project they’d like to add to the weekly
reports, please email Richard+Stephen.
Planned Releases for YP 3.0 {2.8}:
M3 Release 6th SeptM4 Cutoff 30th Sept - this will be YP 3.0.YP 3.0
{2.8 (M4)} Final Release 25th Oct
Planned upcoming dot releases:
YP 2.7.2 (Warrior) is planned for after YP 3.0 release.YP 2.6.4 (Thud)
is planned for after YP 2.7.2  release.
Tracking Metrics:
WDD 2485 (last week 2495) (
https://wiki.yoctoproject.org/charts/combo.html)Poky Patch
Metrics  Total patches found: 1474 (last week 1491)Patches in the
Pending State: 613 (42%) [last week 616 (41%)]
Key Status Links for YP:
https://wiki.yoctoproject.org/wiki/Yocto_Project_v2.8_Status
https://wiki.yoctoproject.org/wiki/Yocto_2.8_Schedule
https://wiki.yoctoproject.org/wiki/Yocto_2.8_Features

The Yocto Project’s technical governance is through its Technical
Steering Committee, more information is available at:
https://wiki.yoctoproject.org/wiki/TSC

The Status reports are now stored on the wiki at: 
https://wiki.yoctoproject.org/wiki/Weekly_Status

[If anyone has suggestions for other information you’d like to see on
this weekly status update, let us know!]
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCHv2] libxcrypt:upgrade 4.4.6 -> 4.4.7

2019-08-27 Thread Richard Purdie
On Tue, 2019-08-27 at 16:21 +0300, Adrian Bunk wrote:
> On Mon, Aug 26, 2019 at 01:32:38PM +0800, Zang Ruochen wrote:
> > -License-Update: Many individual files are under other
> > licenses,update their information.
> > 
> > Signed-off-by: Zang Ruochen 
> > ---
> >  .../libxcrypt/{libxcrypt_4.4.6.bb =>
> > libxcrypt_4.4.7.bb}| 6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> >  rename meta/recipes-core/libxcrypt/{libxcrypt_4.4.6.bb =>
> > libxcrypt_4.4.7.bb} (86%)
> > ...
> 
> You also have to rename libxcrypt-compat_4.4.6.bb

Right, this just failed on the autobuilder and makes me wonder how it
was tested as it doesn't parse :/.

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/1] gettext: avoid useless RPATH QA issue

2019-08-27 Thread Richard Purdie
On Tue, 2019-08-27 at 10:22 +0800, Chen Qi wrote:
> We are getting useless rpath QA error when enabling libunistring
> and msgcat-curses PACKAGECONFIG. Use chrpath to delete the redundant
> RPATH in binaries.
> 
> Signed-off-by: Chen Qi 
> ---
>  meta/recipes-core/gettext/gettext_0.19.8.1.bb | 10 ++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/meta/recipes-core/gettext/gettext_0.19.8.1.bb
> b/meta/recipes-core/gettext/gettext_0.19.8.1.bb
> index 30121ad..4ce47a6 100644
> --- a/meta/recipes-core/gettext/gettext_0.19.8.1.bb
> +++ b/meta/recipes-core/gettext/gettext_0.19.8.1.bb
> @@ -118,6 +118,14 @@ FILES_gettext-runtime-doc =
> "${mandir}/man1/gettext.* \
>  
>  do_install_append() {
>  rm -f ${D}${libdir}/preloadable_libintl.so
> +# remove useless rpath to avoid QA issue
> +useless_rpath_list="${D}${libdir}/gettext/urlget
> ${D}${libdir}/gettext/cldr-plurals \
> +${D}${libdir}/gettext/hostname
> ${D}${bindir}/recode-sr-latin"
> +for f in $useless_rpath_list; do
> + if [ -e $f ]; then
> + chrpath -d $f
> + fi
> +done
>  }
>  
>  do_install_append_class-native () {
> @@ -163,6 +171,8 @@ do_install_ptest() {
>  find ${D}${PTEST_PATH}/ -name "*.o" -exec rm {} \;
>  chmod 0755 ${D}${PTEST_PATH}/tests/lang-vala
> ${D}${PTEST_PATH}/tests/plural-1 ${D}${PTEST_PATH}/tests/xgettext-
> tcl-4 \
> ${D}${PTEST_PATH}/tests/xgettext-vala-
> 1  ${D}${PTEST_PATH}/tests/xgettext-po-2
> +# avoid useless rpath
> +[ -e ${D}${PTEST_PATH}/src/cldr-plurals ] && chrpath -d
> ${D}${PTEST_PATH}/src/cldr-plurals
>  fi
>  }

In general we try and fix the reason they're getting included in the
first place as it means the compiler flags are incorrect. Any idea why
that is happening and if we can fix it?

Cheers,

Richard



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


  1   2   3   4   5   6   7   8   9   10   >