Re: [OE-core] Always install initramfs-framework-base in case of linux-yocto & INITRAMFS_IMAGE_BUNDLE=1

[Alexey Brodkin]

Hi Andrea,

> -Original Message-
> From: Andrea Adami 
> Sent: Thursday, February 7, 2019 5:26 PM
> To: Alexey Brodkin 
> Cc: Leon Woestenberg ; Patches and discussions about the 
> oe-core layer
> 
> Subject: Re: [OE-core] Always install initramfs-framework-base in case of 
> linux-yocto &
> INITRAMFS_IMAGE_BUNDLE=1
> 
> On Wed, Feb 6, 2019 at 4:54 PM Alexey Brodkin
>  wrote:
> >
> > Hi Leon,
> >
> > > -Original Message-
> > > From: Leon Woestenberg 
> > > Sent: Friday, February 1, 2019 12:18 AM
> > > To: Alexey Brodkin 
> > > Cc: Patches and discussions about the oe-core layer 
> > > 
> > > Subject: Re: [OE-core] Always install initramfs-framework-base in case of 
> > > linux-yocto &
> > > INITRAMFS_IMAGE_BUNDLE=1
> > >
> > > On Tue, Jan 29, 2019 at 9:01 PM Alexey Brodkin
> > >  wrote:
> > > > --->8--
> > > > /dev/console is missing or not a character device!
> > > > Please ensure your rootfs is properly configured
> > > > --->8--
> > >
> > > I thought /dev/console could also be created by the kernel itself onto
> > > a "temporary filesystem for device nodes" (DEVTMPFS).
> > > I am not sure, but here is a similar case:
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.busybox.net_pipermail_buildroot_2015-
> > >
> 2DMarch_123309.html=DwIBaQ=DPL6_X_6JkXFx7AXWqB0tg=lqdeeSSEes0GFDDl656eViXO7breS55ytWkhpk5R81I=
> > > 9xab_6OSIxhk6sCiqSFEwTUASBIaA_-C_wGiSMfJn3o=SJ9X7QVWrd1gCdyBsL9fdY841I4qG2y47VH3Sw-IRF4=
> > >
> > > Is your (other) kernel configured with CONFIG_DEVTMPFS=y?
> >
> > Indeed it is.
> >
> > The problem is devtmpfs is auto-mounted in prepare_namespace() which
> > is not executed in case of initramfs because in case of initramfs we
> > expect its "/init" will do everything itself, see [1].
> 
> Hi,
> I jump in because I was at the same point years ago [1]: the issue
> here is basically the linux-yocto specific patch adding the check for
> console.
> It was simply reverted in our case...

See this is fixed in linux-yocto now by [1].

[1] 
http://git.yoctoproject.org/cgit.cgi/linux-yocto/commit/?id=11e0e616ed095bb8012e1b4a231254c9656a0193=master

-Alexey
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] mesa-lima: Add support for mesa for Mail-4xx GPUs

[Alistair Francis]

Add support for the open source reverse engineered drivers for the
Mali-4xx GPUs.

Signed-off-by: Alistair Francis 
---
 .../recipes-graphics/mesa/mesa-lima_18.3.0.bb | 23 +++
 1 file changed, 23 insertions(+)
 create mode 100644 meta/recipes-graphics/mesa/mesa-lima_18.3.0.bb

diff --git a/meta/recipes-graphics/mesa/mesa-lima_18.3.0.bb 
b/meta/recipes-graphics/mesa/mesa-lima_18.3.0.bb
new file mode 100644
index 00..0f26ede2c4
--- /dev/null
+++ b/meta/recipes-graphics/mesa/mesa-lima_18.3.0.bb
@@ -0,0 +1,23 @@
+require ${BPN}.inc
+
+BRANCH = "lima-18.3"
+SRCREV = "8e713e4781982c700bfc7b09279146189d525a86"
+SRC_URI = "git://gitlab.freedesktop.org/lima/mesa.git;branch=${BRANCH} \
+"
+
+S = "${WORKDIR}/git"
+
+DEPENDS += "python3-native python3-mako python3-mako-native gettext-native 
libdrm"
+DEPENDS += "${@bb.utils.contains('PACKAGECONFIG', 'x11', 'xrandr', '', d)}"
+
+PACKAGECONFIG[sun4i] = ""
+GALLIUMDRIVERS_append ="${@bb.utils.contains('PACKAGECONFIG', 'sun4i', 
',lima,sun4i', '', d)}"
+
+#because we cannot rely on the fact that all apps will use pkgconfig,
+#make eglplatform.h independent of MESA_EGL_NO_X11_HEADER
+do_install_append() {
+if ${@bb.utils.contains('PACKAGECONFIG', 'egl', 'true', 'false', d)}; then
+sed -i -e 's/^#if defined(MESA_EGL_NO_X11_HEADERS)$/#if 
defined(MESA_EGL_NO_X11_HEADERS) || ${@bb.utils.contains('PACKAGECONFIG', 
'x11', '0', '1', d)}/' ${D}${includedir}/EGL/eglplatform.h
+fi
+}
+
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [OE-Core][PATCH v2 3/3] devtool: provide support for devtool menuconfig command.

[Chandana Kalluri]

Ping

-Original Message-
From: Sai Hari Chandana Kalluri  
Sent: Friday, January 25, 2019 11:53 AM
To: openembedded-core@lists.openembedded.org
Cc: Alejandro Enedino Hernandez Samaniego ; Chandana 
Kalluri 
Subject: [OE-Core][PATCH v2 3/3] devtool: provide support for devtool 
menuconfig command.

All packages that support the menuconfig task will be able to run devtool 
menuconfig command. This would allow the user to modify the current configure 
options and create a config fragment which can be added to a recipe using 
devtool finish.

1. The patch checks if devtool menuconfig command is called for a valid package.
2. It checks for oe-local-files dir within source and creates one if needed, 
this directory is needed to store the final generated config fragment so that 
devtool finish can update the recipe.
3. Menuconfig command is called for users to make necessary changes. After 
saving the changes, diffconfig command is run to generate the fragment.

Syntax:
devtool menuconfig 
 Ex: devtool menuconfig linux-yocto

The config fragment is saved as devtool-fragment.cfg within oe-local-files dir.

Ex: 
/sources/linux-yocto/oe-local-files/devtool-fragment.cfg

Run devtool finish to update the recipe by appending the config fragment to 
SRC_URI and place a copy of the fragment within the layer where the recipe 
resides.
Ex: devtool finish linux-yocto meta

[YOCTO #10416]

Signed-off-by: Sai Hari Chandana Kalluri 
Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 scripts/lib/devtool/menuconfig.py | 80 +++
 1 file changed, 80 insertions(+)
 create mode 100644 scripts/lib/devtool/menuconfig.py

diff --git a/scripts/lib/devtool/menuconfig.py 
b/scripts/lib/devtool/menuconfig.py
new file mode 100644
index 000..38133db
--- /dev/null
+++ b/scripts/lib/devtool/menuconfig.py
@@ -0,0 +1,80 @@
+# OpenEmbedded Development tool - menuconfig command plugin # # 
+Copyright (C) 2018 Xilinx # Written by: Chandana Kalluri 
+ # # This program is free software; you can 
+redistribute it and/or modify # it under the terms of the GNU General 
+Public License version 2 as # published by the Free Software 
+Foundation.
+#
+# This program is distributed in the hope that it will be useful, # but 
+WITHOUT ANY WARRANTY; without even the implied warranty of # 
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the # GNU 
+General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License 
+along # with this program; if not, write to the Free Software 
+Foundation, Inc., # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+"""Devtool menuconfig plugin"""
+
+import os
+import bb
+import logging
+import argparse
+import re
+import glob
+from devtool import setup_tinfoil, parse_recipe, DevtoolError, 
+standard, exec_build_env_command
+
+logger = logging.getLogger('devtool')
+
+def menuconfig(args, config, basepath, workspace):
+"""Entry point for the devtool 'menuconfig' subcommand"""
+
+rd = "" 
+kconfigpath = ""
+pn_src = ""
+localfilesdir = ""
+workspace_dir = ""
+tinfoil = setup_tinfoil(basepath=basepath)
+try:
+  rd = parse_recipe(config, tinfoil, args.component, appends=True, 
filter_workspace=False)
+  if not rd:
+ return 1
+
+  pn =  rd.getVar('PN', True)
+  if pn not in workspace:
+ raise DevtoolError("Run devtool modify before calling 
+ menuconfig for %s" %pn)
+
+  if not rd.getVarFlag('do_menuconfig','task'):
+ raise DevtoolError("This package does not support menuconfig 
+ option")
+
+  workspace_dir = os.path.join(basepath,'workspace/sources')
+  kconfigpath = rd.getVar('B')
+  pn_src = os.path.join(workspace_dir,pn)
+
+  #add check to see if oe_local_files exists or not
+  localfilesdir = os.path.join(pn_src,'oe-local-files') 
+  if not os.path.exists(localfilesdir):
+  bb.utils.mkdirhier(localfilesdir)
+  #Add gitignore to ensure source tree is clean
+  gitignorefile = os.path.join(localfilesdir,'.gitignore')
+  with open(gitignorefile, 'w') as f:
+  f.write('# Ignore local files, by default. Remove this file 
if you want to commit the directory to Git\n')
+  f.write('*\n')
+
+finally:
+  tinfoil.shutdown()
+
+logger.info('Launching menuconfig')
+exec_build_env_command(config.init_path, basepath, 'bitbake -c menuconfig 
%s' % pn, watch=True) 
+fragment = os.path.join(localfilesdir, 'devtool-fragment.cfg')
+res = standard._create_kconfig_diff(pn_src,rd,fragment)
+
+return 0
+
+def register_commands(subparsers, context):
+"""register devtool subcommands from this plugin"""
+parser_menuconfig = subparsers.add_parser('menuconfig',help='allows 
altering the system component configuration', description='launches the make 
menuconfig command, allows user to make changes to 

[OE-core] [OE-Core][PATCH] wpa-supplicant: update to 2.7

[Changhyeok Bae]

CVE patches is already applied in v2.7

Signed-off-by: Changhyeok Bae 
---
 .../key-replay-cve-multiple1.patch| 191 -
 .../key-replay-cve-multiple2.patch| 267 --
 .../key-replay-cve-multiple3.patch| 201 -
 .../key-replay-cve-multiple4.patch|  96 ---
 .../key-replay-cve-multiple5.patch|  81 --
 .../key-replay-cve-multiple6.patch| 149 --
 .../key-replay-cve-multiple7.patch|  60 
 .../key-replay-cve-multiple8.patch|  99 ---
 .../wpa_supplicant-CVE-2018-14526.patch   |  44 ---
 ...upplicant_2.6.bb => wpa-supplicant_2.7.bb} |  19 +-
 10 files changed, 5 insertions(+), 1202 deletions(-)
 delete mode 100644 
meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/key-replay-cve-multiple1.patch
 delete mode 100644 
meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/key-replay-cve-multiple2.patch
 delete mode 100644 
meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/key-replay-cve-multiple3.patch
 delete mode 100644 
meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/key-replay-cve-multiple4.patch
 delete mode 100644 
meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/key-replay-cve-multiple5.patch
 delete mode 100644 
meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/key-replay-cve-multiple6.patch
 delete mode 100644 
meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/key-replay-cve-multiple7.patch
 delete mode 100644 
meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/key-replay-cve-multiple8.patch
 delete mode 100644 
meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/wpa_supplicant-CVE-2018-14526.patch
 rename meta/recipes-connectivity/wpa-supplicant/{wpa-supplicant_2.6.bb => 
wpa-supplicant_2.7.bb} (81%)

diff --git 
a/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/key-replay-cve-multiple1.patch
 
b/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/key-replay-cve-multiple1.patch
deleted file mode 100644
index d4d49e7fca..00
--- 
a/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/key-replay-cve-multiple1.patch
+++ /dev/null
@@ -1,191 +0,0 @@
-The WPA2 four-way handshake protocol is vulnerable to replay attacks which can
-result in unauthenticated clients gaining access to the network.
-
-Backport a number of patches from upstream to fix this.
-
-CVE: CVE-2017-13077
-CVE: CVE-2017-13078
-CVE: CVE-2017-13079
-CVE: CVE-2017-13080
-CVE: CVE-2017-13081
-CVE: CVE-2017-13082
-CVE: CVE-2017-13086
-CVE: CVE-2017-13087
-CVE: CVE-2017-13088
-
-Upstream-Status: Backport
-Signed-off-by: Ross Burton 
-
-From cf4cab804c7afd5c45505528a8d16e46163243a2 Mon Sep 17 00:00:00 2001
-From: Mathy Vanhoef 
-Date: Fri, 14 Jul 2017 15:15:35 +0200
-Subject: [PATCH 1/8] hostapd: Avoid key reinstallation in FT handshake
-
-Do not reinstall TK to the driver during Reassociation Response frame
-processing if the first attempt of setting the TK succeeded. This avoids
-issues related to clearing the TX/RX PN that could result in reusing
-same PN values for transmitted frames (e.g., due to CCM nonce reuse and
-also hitting replay protection on the receiver) and accepting replayed
-frames on RX side.
-
-This issue was introduced by the commit
-0e84c25434e6a1f283c7b4e62e483729085b78d2 ('FT: Fix PTK configuration in
-authenticator') which allowed wpa_ft_install_ptk() to be called multiple
-times with the same PTK. While the second configuration attempt is
-needed with some drivers, it must be done only if the first attempt
-failed.
-
-Signed-off-by: Mathy Vanhoef 

- src/ap/ieee802_11.c  | 16 +---
- src/ap/wpa_auth.c| 11 +++
- src/ap/wpa_auth.h|  3 ++-
- src/ap/wpa_auth_ft.c | 10 ++
- src/ap/wpa_auth_i.h  |  1 +
- 5 files changed, 37 insertions(+), 4 deletions(-)
-
-diff --git a/src/ap/ieee802_11.c b/src/ap/ieee802_11.c
-index 4e04169..333035f 100644
 a/src/ap/ieee802_11.c
-+++ b/src/ap/ieee802_11.c
-@@ -1841,6 +1841,7 @@ static int add_associated_sta(struct hostapd_data *hapd,
- {
-   struct ieee80211_ht_capabilities ht_cap;
-   struct ieee80211_vht_capabilities vht_cap;
-+  int set = 1;
- 
-   /*
-* Remove the STA entry to ensure the STA PS state gets cleared and
-@@ -1848,9 +1849,18 @@ static int add_associated_sta(struct hostapd_data *hapd,
-* FT-over-the-DS, where a station re-associates back to the same AP but
-* skips the authentication flow, or if working with a driver that
-* does not support full AP client state.
-+   *
-+   * Skip this if the STA has already completed FT reassociation and the
-+   * TK has been configured since the TX/RX PN must not be reset to 0 for
-+   * the same key.
-*/
--  if (!sta->added_unassoc)
-+  if (!sta->added_unassoc &&
-+  (!(sta->flags & WLAN_STA_AUTHORIZED) ||
-+   !wpa_auth_sta_ft_tk_already_set(sta->wpa_sm))) {
-   

[OE-core] [OE-Core][PATCH] maintainers.inc: replace Changhyeok Bae's @lge email address with a personal one

[Changhyeok Bae]

As I will leaving LGE, this address will no longer be valid,
so swap it for my person one for now.

Signed-off-by: Changhyeok Bae 
---
 meta/conf/distro/include/maintainers.inc | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/meta/conf/distro/include/maintainers.inc 
b/meta/conf/distro/include/maintainers.inc
index 8d6eb1afc0..3890f2a65a 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -99,7 +99,7 @@ RECIPE_MAINTAINER_pn-clutter-gtk-1.0 = "Ross Burton 
"
 RECIPE_MAINTAINER_pn-cmake = "Pascal Bach "
 RECIPE_MAINTAINER_pn-cmake-native = "Pascal Bach "
 RECIPE_MAINTAINER_pn-cogl-1.0 = "Ross Burton "
-RECIPE_MAINTAINER_pn-connman = "Changhyeok Bae "
+RECIPE_MAINTAINER_pn-connman = "Changhyeok Bae "
 RECIPE_MAINTAINER_pn-connman-conf = "Ross Burton "
 RECIPE_MAINTAINER_pn-connman-gnome = "Ross Burton "
 RECIPE_MAINTAINER_pn-console-tools = "Chen Qi "
@@ -168,7 +168,7 @@ RECIPE_MAINTAINER_pn-elfutils = "Hongxu Jia 
"
 RECIPE_MAINTAINER_pn-enchant = "Anuj Mittal "
 RECIPE_MAINTAINER_pn-encodings = "Armin Kuster "
 RECIPE_MAINTAINER_pn-epiphany = "Alexander Kanavin "
-RECIPE_MAINTAINER_pn-ethtool = "Changhyeok Bae "
+RECIPE_MAINTAINER_pn-ethtool = "Changhyeok Bae "
 RECIPE_MAINTAINER_pn-eudev = "Anuj Mittal "
 RECIPE_MAINTAINER_pn-expat = "Yi Zhao "
 RECIPE_MAINTAINER_pn-expect = "Alexander Kanavin "
@@ -280,12 +280,12 @@ RECIPE_MAINTAINER_pn-initramfs-live-install-efi-testfs = 
"Anuj Mittal http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/3] python3: upgrade to 3.7.2

[Khem Raj]

On Thu, Feb 7, 2019 at 6:42 AM Alexander Kanavin  wrote:
>
> Thanks - the libnewt-python failure I could not reproduce (and we
> haven't seen it on Yocto AB - it is a oecore recipe).
>
> For python-gevent, I just sent a patch to oe-devel list.
>

https://errors.yoctoproject.org/Errors/Details/221647/

> Alex
>
> On Wed, 6 Feb 2019 at 19:03, Khem Raj  wrote:
> >
> > Alex
> >
> > Here are couple of failures I see
> >
> > https://errors.yoctoproject.org/Errors/Details/221405/
> > https://errors.yoctoproject.org/Errors/Details/221458/
> >
> > On Wed, Feb 6, 2019 at 8:26 AM Alexander Kanavin  
> > wrote:
> > >
> > > I took the same approach as the recent perl upgrade: write recipe from 
> > > scratch,
> > > taking the pieces from the old recipe only when they were proven to be 
> > > necessary.
> > >
> > > The pgo, manifest and ptest features are all preserved.
> > >
> > > New features:
> > >
> > > - native and target recipes are now unified into one recipe
> > >
> > > - check_build_completeness.py runs right after do_compile() and verifies 
> > > that
> > > all optional modules have been built (a notorious source of regressions)
> > >
> > > - a new approach to sysconfig.py and distutils/sysconfig.py returning 
> > > values
> > > appropriate for native or target builds: we copy the configuration file 
> > > to a
> > > separate folder, add that folder to sys.path (through environment variable
> > > that differs between native and target builds), and point python to the 
> > > file
> > > through another environment variable.
> > >
> > > Signed-off-by: Alexander Kanavin 
> > > ---
> > >  meta/classes/python3-dir.bbclass  |2 +-
> > >  meta/classes/python3native.bbclass|2 +
> > >  ...ib-termcap-to-linker-flags-to-avoid-.patch |   25 +
> > >  ...lib-as-location-for-site-packages-an.patch |  196 +++
> > >  ...hell-version-of-python-config-that-w.patch |   35 +
> > >  ...-qemu-wrapper-when-gathering-profile.patch |   25 +
> > >  ...fig-append-STAGING_LIBDIR-python-sys.patch |   42 +
> > >  ...tutils-prefix-is-inside-staging-area.patch |   54 +
> > >  .../python3/avoid_warning_about_tkinter.patch |   36 +
> > >  .../python-sanity/python3/cgi_py.patch|   32 +
> > >  .../python3/check_build_completeness.py   |   17 +
> > >  .../python-sanity/python3/create_manifest3.py |  433 ++
> > >  .../python-sanity/python3/get_module_deps3.py |  146 ++
> > >  .../python-sanity/python3/python-config.patch |   46 +
> > >  .../python3/python3-manifest.json | 1228 +
> > >  .../python-sanity/python3/run-ptest   |3 +
> > >  .../python-sanity/python3_3.7.2.bb|  284 
> > >  17 files changed, 2605 insertions(+), 1 deletion(-)
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/0001-Do-not-hardcode-lib-as-location-for-site-packages-an.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/cgi_py.patch
> > >  create mode 100755 
> > > meta/recipes-devtools/python-sanity/python3/check_build_completeness.py
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/create_manifest3.py
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/get_module_deps3.py
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/python-config.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/python3-manifest.json
> > >  create mode 100644 meta/recipes-devtools/python-sanity/python3/run-ptest
> > >  create mode 100644 meta/recipes-devtools/python-sanity/python3_3.7.2.bb
> > >
> > > diff --git a/meta/classes/python3-dir.bbclass 
> > > b/meta/classes/python3-dir.bbclass
> > > index 06bb046d9c2..7dd130bad99 100644
> > > --- a/meta/classes/python3-dir.bbclass
> > > +++ b/meta/classes/python3-dir.bbclass
> > > @@ -1,4 +1,4 @@
> > > -PYTHON_BASEVERSION = "3.5"
> > > +PYTHON_BASEVERSION = "3.7"
> > >  PYTHON_ABI = "m"
> > >  PYTHON_DIR = "python${PYTHON_BASEVERSION}"
> > >  PYTHON_PN = "python3"
> > > diff --git a/meta/classes/python3native.bbclass 
> > > b/meta/classes/python3native.bbclass
> > > index da12a714703..a3acaf61bbd 100644
> 

[OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

[Khem Raj]

This has been a constant source of trouble for build failures due to 
host-user-contaminated QA
errors of sort

ERROR: QA Issue: glibc-locale: 
/glibc-binary-localedata-ca-es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY
 is owned by uid 3004, which is the same as the user running bitbake. This may 
be due to host contamination [host-user-contaminated]

So far we have tried to mould cp command into not carrying the build
user permissions into install area but it is never entirely fixed since
the issue keeps popping up in various scenes

This patch replaces use of cp with install utility and specifies install
mode for files explcitly

Signed-off-by: Khem Raj 
---
 meta/recipes-core/glibc/glibc-locale.inc | 41 
 1 file changed, 20 insertions(+), 21 deletions(-)

diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
b/meta/recipes-core/glibc/glibc-locale.inc
index 6384f9cbf1..9cce61bf0b 100644
--- a/meta/recipes-core/glibc/glibc-locale.inc
+++ b/meta/recipes-core/glibc/glibc-locale.inc
@@ -71,28 +71,27 @@ FILES_localedef = "${bindir}/localedef"
 
 LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
 
-do_install () {
-   mkdir -p ${D}${bindir} ${D}${datadir}
-   if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
-   cp -R --no-dereference --preserve=mode,links 
${LOCALETREESRC}/${bindir}/* ${D}${bindir}
-   fi
-   if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
-   mkdir -p ${D}${localedir}
-   cp -R --no-dereference --preserve=mode,links 
${LOCALETREESRC}/${localedir}/* ${D}${localedir}
-   fi
-   if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
-   mkdir -p ${D}${libdir}
-   if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
-   cp -R --no-dereference --preserve=mode,links 
${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
-   fi
-   if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
-   cp -R --no-dereference --preserve=mode,links 
${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
-   fi
-   fi
-   if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
-   cp -R --no-dereference --preserve=mode,links 
${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
+copy_locale_files() {
+   local dir=$1 mode=$2
+
+   [ -e "${LOCALETREESRC}$dir" ] || return 0
+
+   for d in . $(find "${LOCALETREESRC}$dir" -type d -printf '%P '); do
+   install -d ${D}$dir/$d
+   find "${LOCALETREESRC}$dir/$d" -maxdepth 1 -type f \
+   -exec install -m $mode -t "${D}$dir/$d" {} \;
+   done
+}
+
+do_install() {
+   copy_locale_files ${bindir} 0755
+   copy_locale_files ${localedir} 0644
+   if [ ${PACKAGE_NO_GCONV} -eq 0 ]; then
+   copy_locale_files ${libdir}/gconv 0755
+   copy_locale_files ${datadir}/i18n 0644
fi
-   cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/SUPPORTED 
${WORKDIR}
+   copy_locale_files ${datadir}/locale 0644
+   install -m 0644 ${LOCALETREESRC}/SUPPORTED ${WORKDIR}/SUPPORTED
 }
 
 inherit libc-package
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/3] libva: upgrade 2.3.0 -> 2.4.0

[Richard Purdie]

On Thu, 2019-02-07 at 13:57 +0800, Anuj Mittal wrote:
> For changes, see:
> 
> https://github.com/intel/libva/releases
> 
> Signed-off-by: Anuj Mittal 
> ---
>  .../recipes-graphics/libva/{libva_2.3.0.bb => libva_2.4.0.bb} | 4
> ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>  rename meta/recipes-graphics/libva/{libva_2.3.0.bb =>
> libva_2.4.0.bb} (92%)

This seemed to cause:

https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/266

Cheers,

Richard



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/3] opkg: add --ignore-recommends flag

[Scott Rifenbark]

On Thu, Feb 7, 2019 at 9:44 AM Alejandro Del Castillo <
alejandro.delcasti...@ni.com> wrote:

>
>
> On 2/7/19 11:36 AM, Scott Rifenbark wrote:
> > This change looks like it impacts documentation (i.e.
> >
> https://yoctoproject.org/docs/2.7/ref-manual/ref-manual.html#var-BAD_RECOMMENDATIONS.
>
> > When will this change go into effect?
>
> I believe it doesn't. It is refactoring the opkg implementation to
> leverage a new opkg feature (--add-ignore-recommends), which is a more
> robust implementation. Should be transparent to users.
>

Thank you for the clarification... I thought it might be a user option.


> > On Thu, Feb 7, 2019 at 7:58 AM Alejandro del Castillo
> > mailto:alejandro.delcasti...@ni.com>>
> wrote:
> >
> > To be used for BAD_RECOMMENDATIONS feature.
> >
> > Signed-off-by: Alejandro del Castillo  > >
> > ---
> >   ...pkg-add-add-ignore-recommends-option.patch | 259
> ++
> >   meta/recipes-devtools/opkg/opkg_0.4.0.bb
> > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__opkg-5F0.4.0.bb=DwMFaQ=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA=wNcrL2akRn6jfxhHaKavUrJB_C9JAMXtynjLd8ZzgXQ=8hd_9vew95YIf_VIv2QHNu2EFnc3G2WZ5KA2Upnv5j8=JvgzuAyAoVrkeSZAdWZUuIsFWQ8okApbCk1iiF13CDc=>
>
> >  |   1 +
> >   2 files changed, 260 insertions(+)
> >   create mode 100644
> >
>  
> meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
> >
> > diff --git
> >
>  
> a/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
> >
>  
> b/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
> > new file mode 100644
> > index 00..47d1b3c37e
> > --- /dev/null
> > +++
> >
>  
> b/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
> > @@ -0,0 +1,259 @@
> > +From 64aa98646a17c299bf37af2975b98daf5d7d30b4 Mon Sep 17 00:00:00
> 2001
> > +From: Alejandro del Castillo  > >
> > +Date: Thu, 31 Jan 2019 18:16:08 -0600
> > +Subject: [PATCH] libopkg: add --add-ignore-recommends option
> > +
> > +Add option to ignore specific recommended packages. On the libsolv
> > +backed, this feature will only work on libsolv version > 0.7.2 [1].
> > +
> > +[1]
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openSUSE_libsolv_issues_254=DwIBaQ=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA=wNcrL2akRn6jfxhHaKavUrJB_C9JAMXtynjLd8ZzgXQ=GObNHzFJpWpf_PripIrf-K2RhsktYdAUEieAJexXOKw=3G-meChUqClFggFPqsrAxIZBfLnRKIHm62Uuy1X6nQQ=
> > +
> > +Signed-off-by: Alejandro del Castillo  > >
> > +
> > +Upstream-Status: Accepted
> > +---
> > + libopkg/opkg_conf.c   |  2 +
> > + libopkg/opkg_conf.h   |  1 +
> > + .../solvers/internal/pkg_depends_internal.c   |  3 +-
> > + libopkg/solvers/libsolv/opkg_solver_libsolv.c | 21 ++-
> > + man/opkg.1.in
> > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__opkg.1.in=DwMFaQ=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA=wNcrL2akRn6jfxhHaKavUrJB_C9JAMXtynjLd8ZzgXQ=8hd_9vew95YIf_VIv2QHNu2EFnc3G2WZ5KA2Upnv5j8=8yIWRVT6JV0YnkHzVgeo8pAkiFPM4xz8y9APvxobmhY=>
>
> > |  3 +
> > + src/opkg.c|  6 ++
> > + tests/Makefile|  1 +
> > + tests/core/43_add_ignore_recommends.py| 62
> +++
> > + 8 files changed, 97 insertions(+), 2 deletions(-)
> > + create mode 100755 tests/core/43_add_ignore_recommends.py
> > +
> > +diff --git a/libopkg/opkg_conf.c b/libopkg/opkg_conf.c
> > +index 06880a1..f2330cd 100644
> > +--- a/libopkg/opkg_conf.c
> >  b/libopkg/opkg_conf.c
> > +@@ -597,6 +597,7 @@ int opkg_conf_init(void)
> > + pkg_dest_list_init(_config->tmp_dest_list);
> > + nv_pair_list_init(_config->arch_list);
> > + str_list_init(_config->exclude_list);
> > ++str_list_init(_config->ignore_recommends_list);
> > +
> > + return 0;
> > + }
> > +@@ -938,6 +939,7 @@ void opkg_conf_deinit(void)
> > + pkg_dest_list_deinit(_config->pkg_dest_list);
> > + nv_pair_list_deinit(_config->arch_list);
> > + str_list_deinit(_config->exclude_list);
> > ++str_list_deinit(_config->ignore_recommends_list);
> > +
> > + if (opkg_config->verbosity >= DEBUG) {
> > + hash_print_stats(_config->pkg_hash);
> > +diff --git a/libopkg/opkg_conf.h b/libopkg/opkg_conf.h
> > +index eb56a29..316c500 100644
> > +--- a/libopkg/opkg_conf.h
> >  b/libopkg/opkg_conf.h
> > +@@ -61,6 +61,7 @@ typedef struct opkg_conf {
> > + pkg_dest_list_t tmp_dest_list;
> > + nv_pair_list_t arch_list;
> > + 

Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

[Joshua Watt]

On Thu, 2019-02-07 at 17:21 +, Richard Purdie wrote:
> On Thu, 2019-02-07 at 10:59 -0600, Joshua Watt wrote:
> > On Thu, Feb 7, 2019 at 10:44 AM Joshua Watt 
> > wrote:
> > > Martin,
> > > 
> > > Do you by any chance build in Docker (or some other transient
> > > container)? We recently found an issue with building in docker
> > > where when the primary container command would exit, causing the
> > > container to terminate. When the container terminates, the kernel
> > > unceremoniously sends SIGKILL to all its processes. This plays
> > > *very* poorly with pseudo because pseudo keeps it's entire
> > > database
> > > in a in-memory sqlite database and only flushes it out to disk on
> > > a
> > > periodic interval, or when it get SIGTERM (maybe SIGQUIT?) or
> > > exits
> > > normally. By default the pseudo daemon will also hang around for
> > > 20(?) seconds waiting for a new connection from a pseudo client,
> > > meaning that when our docker container was exiting, there were
> > > potentially several pseudo dameons sitting around waiting for a
> > > client with unflushed databases in memory that suddenly got a
> > > SIGKILL and all their data was lost. This was particularly bad if
> > > you aborted a build while it was running, or some build failure
> > > occurred. Since 
> >  the pseudo database was lost, the only way to correct these errors
> > was to clean and rebuild.
> > > I think pseudo could handle this better and keep the database on
> > > disk instead of in memory, but in the short term you can hack
> > > bitbake to pass the -S flag when invoking the pseudo client which
> > > will make it tell the dameon to shutdown (and thus flush the
> > > database) if it is the last connection.
> > 
> > FWIW, this patch makes it much easier to do that
> > http://lists.openembedded.org/pipermail/bitbake-devel/2019-February/019813.html
> > Just add
> >  FAKEROOTCMD += "-S"
> > in local.conf
> 
> Should we be doing that by default? Is there a downside?

pseudo won't stick around waiting for connections from other clients.
I'm not sure if this would impact performance at all, as I'm not sure
how often a client is able to connect during the "grace period" to
prevent a new server needing to be spun up. I would image this
potentially cuts down on the spin up time for back-to-back fakeroot
tasks in the same recipe?

> 
> Keeping the database flushed to disk would likely be slow but this
> seems reasonable.

I think there are 3 options in increasing order of
complexity/risk/safety

 1. Add -S to FAKEROOTCMD by default
 2. Make pseudo flush the in memory cache every time the last client
disconnects
 3. Remove the in-memory cache from pseudo completely and make it write
to disk each time. Even if we removed all disk flushes from the sqlite
database to keep sync()'s down, it would no worse that what happens
today in any case I can think of (specifically, a power loss will cause
lost data), and in many cases it would be better (e.g. SIGKILL and
crashes in pseudo wouldn't cause data loss).

> 
> Cheers,
> 
> Richard
> 
> 
-- 
Joshua Watt 

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH V2] glibc-locale: Rewrite do_install using install utility instead of cp

[Peter Kjellerstedt]

> -Original Message-
> From: openembedded-core-boun...@lists.openembedded.org  core-boun...@lists.openembedded.org> On Behalf Of Khem Raj
> Sent: den 7 februari 2019 19:04
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH V2] glibc-locale: Rewrite do_install using
> install utility instead of cp
> 
> This has been a constant source of trouble for build failures due to
> host-user-contaminated QA
> errors of sort
> 
> ERROR: QA Issue: glibc-locale: /glibc-binary-localedata-ca-
> es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY is owned by uid
> 3004, which is the same as the user running bitbake. This may be due to
> host contamination [host-user-contaminated]
> 
> So far we have tried to mould cp command into not carrying the build
> user permissions into install area but it is never entirely fixed since
> the issue keeps popping up in various scenes
> 
> This patch replaces use of cp with install utility and specifies
> install
> mode for files explcitly
> 
> Signed-off-by: Khem Raj 
> ---
> v2: Turn the common install loop into a shell function ( Thanks Peter
> Kjellerstedt)
>  meta/recipes-core/glibc/glibc-locale.inc | 43 
>  1 file changed, 21 insertions(+), 22 deletions(-)
> 
> diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
> b/meta/recipes-core/glibc/glibc-locale.inc
> index 6384f9cbf1..c51fa482a9 100644
> --- a/meta/recipes-core/glibc/glibc-locale.inc
> +++ b/meta/recipes-core/glibc/glibc-locale.inc
> @@ -71,28 +71,27 @@ FILES_localedef = "${bindir}/localedef"
> 
>  LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
> 
> -do_install () {
> - mkdir -p ${D}${bindir} ${D}${datadir}
> - if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
> - fi
> - if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
> - mkdir -p ${D}${localedir}
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
> - fi
> - if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
> - mkdir -p ${D}${libdir}
> - if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
> - fi
> - if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
> - fi
> - fi
> - if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
> - fi
> - cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/SUPPORTED 
> ${WORKDIR}
> +copy_locale_files() {
> +local dir=$1 mode=$2
> +
> +[ -e "${LOCALETREESRC}$dir" ] || return 0
> +
> +for d in . $(find "${LOCALETREESRC}$dir" -type d -printf '%P '); do
> +install -d ${D}$dir/$d
> +find "${LOCALETREESRC}$dir/$d" -maxdepth 1 -type f \
> + -exec install -m $mode -t "${D}$dir/$d" {} \;
> +done
> +}
> +
> +do_install() {
> +copy_locale_files ${bindir} 0755
> +copy_locale_files ${localedir} 0644
> +if [ ${PACKAGE_NO_GCONV} -eq 0 ]; then
> +copy_locale_files ${libdir}/gconv 0755
> +copy_locale_files ${datadir}/i18n 0644
> +fi
> +copy_locale_files ${datadir}/locale 0644
> +install -m 0644 ${LOCALETREESRC}/SUPPORTED ${WORKDIR}/SUPPORTED
>  }
> 
>  inherit libc-package
> --
> 2.20.1

Except for that you turned all the tabs into spaces, it looks fine.

//Peter

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH V2] glibc-locale: Rewrite do_install using install utility instead of cp

[Khem Raj]

This has been a constant source of trouble for build failures due to 
host-user-contaminated QA
errors of sort

ERROR: QA Issue: glibc-locale: 
/glibc-binary-localedata-ca-es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY
 is owned by uid 3004, which is the same as the user running bitbake. This may 
be due to host contamination [host-user-contaminated]

So far we have tried to mould cp command into not carrying the build
user permissions into install area but it is never entirely fixed since
the issue keeps popping up in various scenes

This patch replaces use of cp with install utility and specifies install
mode for files explcitly

Signed-off-by: Khem Raj 
---
v2: Turn the common install loop into a shell function ( Thanks Peter
Kjellerstedt)
 meta/recipes-core/glibc/glibc-locale.inc | 43 
 1 file changed, 21 insertions(+), 22 deletions(-)

diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
b/meta/recipes-core/glibc/glibc-locale.inc
index 6384f9cbf1..c51fa482a9 100644
--- a/meta/recipes-core/glibc/glibc-locale.inc
+++ b/meta/recipes-core/glibc/glibc-locale.inc
@@ -71,28 +71,27 @@ FILES_localedef = "${bindir}/localedef"
 
 LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
 
-do_install () {
-   mkdir -p ${D}${bindir} ${D}${datadir}
-   if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
-   cp -R --no-dereference --preserve=mode,links 
${LOCALETREESRC}/${bindir}/* ${D}${bindir}
-   fi
-   if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
-   mkdir -p ${D}${localedir}
-   cp -R --no-dereference --preserve=mode,links 
${LOCALETREESRC}/${localedir}/* ${D}${localedir}
-   fi
-   if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
-   mkdir -p ${D}${libdir}
-   if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
-   cp -R --no-dereference --preserve=mode,links 
${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
-   fi
-   if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
-   cp -R --no-dereference --preserve=mode,links 
${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
-   fi
-   fi
-   if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
-   cp -R --no-dereference --preserve=mode,links 
${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
-   fi
-   cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/SUPPORTED 
${WORKDIR}
+copy_locale_files() {
+local dir=$1 mode=$2
+
+[ -e "${LOCALETREESRC}$dir" ] || return 0
+
+for d in . $(find "${LOCALETREESRC}$dir" -type d -printf '%P '); do
+install -d ${D}$dir/$d
+find "${LOCALETREESRC}$dir/$d" -maxdepth 1 -type f \
+ -exec install -m $mode -t "${D}$dir/$d" {} \;
+done
+}
+
+do_install() {
+copy_locale_files ${bindir} 0755
+copy_locale_files ${localedir} 0644
+if [ ${PACKAGE_NO_GCONV} -eq 0 ]; then
+copy_locale_files ${libdir}/gconv 0755
+copy_locale_files ${datadir}/i18n 0644
+fi
+copy_locale_files ${datadir}/locale 0644
+install -m 0644 ${LOCALETREESRC}/SUPPORTED ${WORKDIR}/SUPPORTED
 }
 
 inherit libc-package
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/3] python3: upgrade to 3.7.2

[Khem Raj]

On Thu, Feb 7, 2019 at 6:42 AM Alexander Kanavin  wrote:
>
> Thanks - the libnewt-python failure I could not reproduce (and we
> haven't seen it on Yocto AB - it is a oecore recipe).
>
> For python-gevent, I just sent a patch to oe-devel list.
>

Thanks Alex, really appreciated

> Alex
>
> On Wed, 6 Feb 2019 at 19:03, Khem Raj  wrote:
> >
> > Alex
> >
> > Here are couple of failures I see
> >
> > https://errors.yoctoproject.org/Errors/Details/221405/
> > https://errors.yoctoproject.org/Errors/Details/221458/
> >
> > On Wed, Feb 6, 2019 at 8:26 AM Alexander Kanavin  
> > wrote:
> > >
> > > I took the same approach as the recent perl upgrade: write recipe from 
> > > scratch,
> > > taking the pieces from the old recipe only when they were proven to be 
> > > necessary.
> > >
> > > The pgo, manifest and ptest features are all preserved.
> > >
> > > New features:
> > >
> > > - native and target recipes are now unified into one recipe
> > >
> > > - check_build_completeness.py runs right after do_compile() and verifies 
> > > that
> > > all optional modules have been built (a notorious source of regressions)
> > >
> > > - a new approach to sysconfig.py and distutils/sysconfig.py returning 
> > > values
> > > appropriate for native or target builds: we copy the configuration file 
> > > to a
> > > separate folder, add that folder to sys.path (through environment variable
> > > that differs between native and target builds), and point python to the 
> > > file
> > > through another environment variable.
> > >
> > > Signed-off-by: Alexander Kanavin 
> > > ---
> > >  meta/classes/python3-dir.bbclass  |2 +-
> > >  meta/classes/python3native.bbclass|2 +
> > >  ...ib-termcap-to-linker-flags-to-avoid-.patch |   25 +
> > >  ...lib-as-location-for-site-packages-an.patch |  196 +++
> > >  ...hell-version-of-python-config-that-w.patch |   35 +
> > >  ...-qemu-wrapper-when-gathering-profile.patch |   25 +
> > >  ...fig-append-STAGING_LIBDIR-python-sys.patch |   42 +
> > >  ...tutils-prefix-is-inside-staging-area.patch |   54 +
> > >  .../python3/avoid_warning_about_tkinter.patch |   36 +
> > >  .../python-sanity/python3/cgi_py.patch|   32 +
> > >  .../python3/check_build_completeness.py   |   17 +
> > >  .../python-sanity/python3/create_manifest3.py |  433 ++
> > >  .../python-sanity/python3/get_module_deps3.py |  146 ++
> > >  .../python-sanity/python3/python-config.patch |   46 +
> > >  .../python3/python3-manifest.json | 1228 +
> > >  .../python-sanity/python3/run-ptest   |3 +
> > >  .../python-sanity/python3_3.7.2.bb|  284 
> > >  17 files changed, 2605 insertions(+), 1 deletion(-)
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/0001-Do-not-hardcode-lib-as-location-for-site-packages-an.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/cgi_py.patch
> > >  create mode 100755 
> > > meta/recipes-devtools/python-sanity/python3/check_build_completeness.py
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/create_manifest3.py
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/get_module_deps3.py
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/python-config.patch
> > >  create mode 100644 
> > > meta/recipes-devtools/python-sanity/python3/python3-manifest.json
> > >  create mode 100644 meta/recipes-devtools/python-sanity/python3/run-ptest
> > >  create mode 100644 meta/recipes-devtools/python-sanity/python3_3.7.2.bb
> > >
> > > diff --git a/meta/classes/python3-dir.bbclass 
> > > b/meta/classes/python3-dir.bbclass
> > > index 06bb046d9c2..7dd130bad99 100644
> > > --- a/meta/classes/python3-dir.bbclass
> > > +++ b/meta/classes/python3-dir.bbclass
> > > @@ -1,4 +1,4 @@
> > > -PYTHON_BASEVERSION = "3.5"
> > > +PYTHON_BASEVERSION = "3.7"
> > >  PYTHON_ABI = "m"
> > >  PYTHON_DIR = "python${PYTHON_BASEVERSION}"
> > >  PYTHON_PN = "python3"
> > > diff --git a/meta/classes/python3native.bbclass 
> > > b/meta/classes/python3native.bbclass
> > > index da12a714703..a3acaf61bbd 100644
> > > --- 

Re: [OE-core] [PATCH 1/3] opkg: add --ignore-recommends flag

[Alejandro Del Castillo]

On 2/7/19 11:36 AM, Scott Rifenbark wrote:
> This change looks like it impacts documentation (i.e. 
> https://yoctoproject.org/docs/2.7/ref-manual/ref-manual.html#var-BAD_RECOMMENDATIONS.
>   
> When will this change go into effect? 

I believe it doesn't. It is refactoring the opkg implementation to 
leverage a new opkg feature (--add-ignore-recommends), which is a more 
robust implementation. Should be transparent to users.

> On Thu, Feb 7, 2019 at 7:58 AM Alejandro del Castillo 
> mailto:alejandro.delcasti...@ni.com>> wrote:
> 
> To be used for BAD_RECOMMENDATIONS feature.
> 
> Signed-off-by: Alejandro del Castillo  >
> ---
>   ...pkg-add-add-ignore-recommends-option.patch | 259 ++
>   meta/recipes-devtools/opkg/opkg_0.4.0.bb
> 
> 
>  
>      |   1 +
>   2 files changed, 260 insertions(+)
>   create mode 100644
> 
> meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
> 
> diff --git
> 
> a/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
> 
> b/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
> new file mode 100644
> index 00..47d1b3c37e
> --- /dev/null
> +++
> 
> b/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
> @@ -0,0 +1,259 @@
> +From 64aa98646a17c299bf37af2975b98daf5d7d30b4 Mon Sep 17 00:00:00 2001
> +From: Alejandro del Castillo  >
> +Date: Thu, 31 Jan 2019 18:16:08 -0600
> +Subject: [PATCH] libopkg: add --add-ignore-recommends option
> +
> +Add option to ignore specific recommended packages. On the libsolv
> +backed, this feature will only work on libsolv version > 0.7.2 [1].
> +
> +[1]
> 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openSUSE_libsolv_issues_254=DwIBaQ=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA=wNcrL2akRn6jfxhHaKavUrJB_C9JAMXtynjLd8ZzgXQ=GObNHzFJpWpf_PripIrf-K2RhsktYdAUEieAJexXOKw=3G-meChUqClFggFPqsrAxIZBfLnRKIHm62Uuy1X6nQQ=
> +
> +Signed-off-by: Alejandro del Castillo  >
> +
> +Upstream-Status: Accepted
> +---
> + libopkg/opkg_conf.c                           |  2 +
> + libopkg/opkg_conf.h                           |  1 +
> + .../solvers/internal/pkg_depends_internal.c   |  3 +-
> + libopkg/solvers/libsolv/opkg_solver_libsolv.c | 21 ++-
> + man/opkg.1.in
> 
> 
>  
>                                 |  3 +
> + src/opkg.c                                    |  6 ++
> + tests/Makefile                                |  1 +
> + tests/core/43_add_ignore_recommends.py        | 62 +++
> + 8 files changed, 97 insertions(+), 2 deletions(-)
> + create mode 100755 tests/core/43_add_ignore_recommends.py
> +
> +diff --git a/libopkg/opkg_conf.c b/libopkg/opkg_conf.c
> +index 06880a1..f2330cd 100644
> +--- a/libopkg/opkg_conf.c
>  b/libopkg/opkg_conf.c
> +@@ -597,6 +597,7 @@ int opkg_conf_init(void)
> +     pkg_dest_list_init(_config->tmp_dest_list);
> +     nv_pair_list_init(_config->arch_list);
> +     str_list_init(_config->exclude_list);
> ++    str_list_init(_config->ignore_recommends_list);
> +
> +     return 0;
> + }
> +@@ -938,6 +939,7 @@ void opkg_conf_deinit(void)
> +     pkg_dest_list_deinit(_config->pkg_dest_list);
> +     nv_pair_list_deinit(_config->arch_list);
> +     str_list_deinit(_config->exclude_list);
> ++    str_list_deinit(_config->ignore_recommends_list);
> +
> +     if (opkg_config->verbosity >= DEBUG) {
> +         hash_print_stats(_config->pkg_hash);
> +diff --git a/libopkg/opkg_conf.h b/libopkg/opkg_conf.h
> +index eb56a29..316c500 100644
> +--- a/libopkg/opkg_conf.h
>  b/libopkg/opkg_conf.h
> +@@ -61,6 +61,7 @@ typedef struct opkg_conf {
> +     pkg_dest_list_t tmp_dest_list;
> +     nv_pair_list_t arch_list;
> +     str_list_t exclude_list;
> ++    str_list_t ignore_recommends_list;
> +
> +     int restrict_to_default_dest;
> +     pkg_dest_t *default_dest;
> +diff --git a/libopkg/solvers/internal/pkg_depends_internal.c
> b/libopkg/solvers/internal/pkg_depends_internal.c
> +index cd56d84..5deee70 100644
> +--- 

Re: [OE-core] [PATCH 1/3] opkg: add --ignore-recommends flag

[Scott Rifenbark]

This change looks like it impacts documentation (i.e.
https://yoctoproject.org/docs/2.7/ref-manual/ref-manual.html#var-BAD_RECOMMENDATIONS).
When will this change go into effect?

Scott

On Thu, Feb 7, 2019 at 7:58 AM Alejandro del Castillo <
alejandro.delcasti...@ni.com> wrote:

> To be used for BAD_RECOMMENDATIONS feature.
>
> Signed-off-by: Alejandro del Castillo 
> ---
>  ...pkg-add-add-ignore-recommends-option.patch | 259 ++
>  meta/recipes-devtools/opkg/opkg_0.4.0.bb  |   1 +
>  2 files changed, 260 insertions(+)
>  create mode 100644
> meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
>
> diff --git
> a/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
> b/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
> new file mode 100644
> index 00..47d1b3c37e
> --- /dev/null
> +++
> b/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
> @@ -0,0 +1,259 @@
> +From 64aa98646a17c299bf37af2975b98daf5d7d30b4 Mon Sep 17 00:00:00 2001
> +From: Alejandro del Castillo 
> +Date: Thu, 31 Jan 2019 18:16:08 -0600
> +Subject: [PATCH] libopkg: add --add-ignore-recommends option
> +
> +Add option to ignore specific recommended packages. On the libsolv
> +backed, this feature will only work on libsolv version > 0.7.2 [1].
> +
> +[1]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openSUSE_libsolv_issues_254=DwIBaQ=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA=wNcrL2akRn6jfxhHaKavUrJB_C9JAMXtynjLd8ZzgXQ=GObNHzFJpWpf_PripIrf-K2RhsktYdAUEieAJexXOKw=3G-meChUqClFggFPqsrAxIZBfLnRKIHm62Uuy1X6nQQ=
> +
> +Signed-off-by: Alejandro del Castillo 
> +
> +Upstream-Status: Accepted
> +---
> + libopkg/opkg_conf.c   |  2 +
> + libopkg/opkg_conf.h   |  1 +
> + .../solvers/internal/pkg_depends_internal.c   |  3 +-
> + libopkg/solvers/libsolv/opkg_solver_libsolv.c | 21 ++-
> + man/opkg.1.in |  3 +
> + src/opkg.c|  6 ++
> + tests/Makefile|  1 +
> + tests/core/43_add_ignore_recommends.py| 62 +++
> + 8 files changed, 97 insertions(+), 2 deletions(-)
> + create mode 100755 tests/core/43_add_ignore_recommends.py
> +
> +diff --git a/libopkg/opkg_conf.c b/libopkg/opkg_conf.c
> +index 06880a1..f2330cd 100644
> +--- a/libopkg/opkg_conf.c
>  b/libopkg/opkg_conf.c
> +@@ -597,6 +597,7 @@ int opkg_conf_init(void)
> + pkg_dest_list_init(_config->tmp_dest_list);
> + nv_pair_list_init(_config->arch_list);
> + str_list_init(_config->exclude_list);
> ++str_list_init(_config->ignore_recommends_list);
> +
> + return 0;
> + }
> +@@ -938,6 +939,7 @@ void opkg_conf_deinit(void)
> + pkg_dest_list_deinit(_config->pkg_dest_list);
> + nv_pair_list_deinit(_config->arch_list);
> + str_list_deinit(_config->exclude_list);
> ++str_list_deinit(_config->ignore_recommends_list);
> +
> + if (opkg_config->verbosity >= DEBUG) {
> + hash_print_stats(_config->pkg_hash);
> +diff --git a/libopkg/opkg_conf.h b/libopkg/opkg_conf.h
> +index eb56a29..316c500 100644
> +--- a/libopkg/opkg_conf.h
>  b/libopkg/opkg_conf.h
> +@@ -61,6 +61,7 @@ typedef struct opkg_conf {
> + pkg_dest_list_t tmp_dest_list;
> + nv_pair_list_t arch_list;
> + str_list_t exclude_list;
> ++str_list_t ignore_recommends_list;
> +
> + int restrict_to_default_dest;
> + pkg_dest_t *default_dest;
> +diff --git a/libopkg/solvers/internal/pkg_depends_internal.c
> b/libopkg/solvers/internal/pkg_depends_internal.c
> +index cd56d84..5deee70 100644
> +--- a/libopkg/solvers/internal/pkg_depends_internal.c
>  b/libopkg/solvers/internal/pkg_depends_internal.c
> +@@ -228,7 +228,8 @@ int pkg_hash_fetch_unsatisfied_dependencies(pkg_t
> *pkg,
> + || compound_depend->type == SUGGEST)
> + && (satisfying_pkg->state_want == SW_DEINSTALL
> + || satisfying_pkg->state_want == SW_PURGE
> +-|| opkg_config->no_install_recommends);
> ++|| opkg_config->no_install_recommends
> ++||
> str_list_contains(_config->ignore_recommends_list,
> satisfying_pkg->name));
> + if (ignore) {
> + opkg_msg(NOTICE,
> +  "%s: ignoring recommendation for "
> +diff --git a/libopkg/solvers/libsolv/opkg_solver_libsolv.c
> b/libopkg/solvers/libsolv/opkg_solver_libsolv.c
> +index 2b27e3a..403e07b 100644
> +--- a/libopkg/solvers/libsolv/opkg_solver_libsolv.c
>  b/libopkg/solvers/libsolv/opkg_solver_libsolv.c
> +@@ -484,6 +484,7 @@ static void pkg2solvable(pkg_t *pkg, Solvable
> *solvable_out)
> + static void populate_installed_repo(libsolv_solver_t *libsolv_solver)
> + {
> + int i;
> ++Id what;
> +
> + 

Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

[Richard Purdie]

On Thu, 2019-02-07 at 10:59 -0600, Joshua Watt wrote:
> On Thu, Feb 7, 2019 at 10:44 AM Joshua Watt 
> wrote:
> > Martin,
> > 
> > Do you by any chance build in Docker (or some other transient
> > container)? We recently found an issue with building in docker
> > where when the primary container command would exit, causing the
> > container to terminate. When the container terminates, the kernel
> > unceremoniously sends SIGKILL to all its processes. This plays
> > *very* poorly with pseudo because pseudo keeps it's entire database
> > in a in-memory sqlite database and only flushes it out to disk on a
> > periodic interval, or when it get SIGTERM (maybe SIGQUIT?) or exits
> > normally. By default the pseudo daemon will also hang around for
> > 20(?) seconds waiting for a new connection from a pseudo client,
> > meaning that when our docker container was exiting, there were
> > potentially several pseudo dameons sitting around waiting for a
> > client with unflushed databases in memory that suddenly got a
> > SIGKILL and all their data was lost. This was particularly bad if
> > you aborted a build while it was running, or some build failure
> > occurred. Since 
>  the pseudo database was lost, the only way to correct these errors
> was to clean and rebuild.
> > I think pseudo could handle this better and keep the database on
> > disk instead of in memory, but in the short term you can hack
> > bitbake to pass the -S flag when invoking the pseudo client which
> > will make it tell the dameon to shutdown (and thus flush the
> > database) if it is the last connection.
> 
> FWIW, this patch makes it much easier to do that
> http://lists.openembedded.org/pipermail/bitbake-devel/2019-February/019813.html
> Just add
>  FAKEROOTCMD += "-S"
> in local.conf

Should we be doing that by default? Is there a downside?

Keeping the database flushed to disk would likely be slow but this
seems reasonable.

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

[Martin Jansa]

On Thu, Feb 07, 2019 at 10:59:18AM -0600, Joshua Watt wrote:
> On Thu, Feb 7, 2019 at 10:44 AM Joshua Watt  wrote:
> >
> > Martin,
> >
> > Do you by any chance build in Docker (or some other transient container)? 
> > We recently found an issue with building in docker where when the primary 
> > container command would exit, causing the container to terminate. When the 
> > container terminates, the kernel unceremoniously sends SIGKILL to all its 
> > processes. This plays *very* poorly with pseudo because pseudo keeps it's 
> > entire database in a in-memory sqlite database and only flushes it out to 
> > disk on a periodic interval, or when it get SIGTERM (maybe SIGQUIT?) or 
> > exits normally. By default the pseudo daemon will also hang around for 
> > 20(?) seconds waiting for a new connection from a pseudo client, meaning 
> > that when our docker container was exiting, there were potentially several 
> > pseudo dameons sitting around waiting for a client with unflushed databases 
> > in memory that suddenly got a SIGKILL and all their data was lost. This was 
> > particularly bad if you aborted a build while it was running, or some build 
> > failure occurred. Since the pseudo database was lost, the only way to 
> > correct these errors was to clean and rebuild.
> >
> > I think pseudo could handle this better and keep the database on disk 
> > instead of in memory, but in the short term you can hack bitbake to pass 
> > the -S flag when invoking the pseudo client which will make it tell the 
> > dameon to shutdown (and thus flush the database) if it is the last 
> > connection.
> 
> FWIW, this patch makes it much easier to do that
> http://lists.openembedded.org/pipermail/bitbake-devel/2019-February/019813.html
> Just add
>  FAKEROOTCMD += "-S"
> in local.conf

Hi Joshua,

most of the build failures caused by this weren't from the builds in
Docker (or any other container), just plain Ubuntu (various versions
14.04 - 18.04) with just the prerequisites installed.

When I get access to some bigger build farm (probably in few months) I
can run that for loop reproducer from the ticket again with
FAKEROOTCMD += "-S"
to see if it makes any difference.

Regards,

> > On Thu, 2019-02-07 at 17:17 +0100, Martin Jansa wrote:
> >
> > There are other rare reproducers which don't involve cp/install (e.g. older 
> > qmake was triggering it quite often as well), I've mentioned some in:
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12434
> >
> > glibc-locale is the most common one for sure as pretty much everybody is 
> > building that one, I don't have objection converting this from cp to 
> > install to prevent this random error (very rare random failure doesn't 
> > really help with #12434 reproducer and whoever work on it, can either 
> > revert this change or try to reproduce with something else) - I've included 
> > few hundreds of INSANE_SKIPs to ignore _random_ host-user-contaminated QAs 
> > in LG layers to prevent our builds failing randomly and the one for 
> > glibc-locale is the longest part:
> > https://github.com/webosose/meta-webosose/blob/master/meta-webos/recipes-core/glibc/glibc-locale_%25.bbappend
> >
> > Regards,
> >
> > On Thu, Feb 7, 2019 at 3:50 PM Khem Raj  wrote:
> >
> > On Thu, Feb 7, 2019 at 1:00 AM Richard Purdie
> >  wrote:
> > >
> > > On Wed, 2019-02-06 at 16:35 -0800, Khem Raj wrote:
> > > > This has been a constant source of trouble for build failures due to 
> > > > host-user-contaminated QA
> > > > errors of sort
> > > >
> > > > ERROR: QA Issue: glibc-locale: 
> > > > /glibc-binary-localedata-ca-es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY
> > > >  is owned by uid 3004, which is the same as the user running bitbake. 
> > > > This may be due to host contamination [host-user-contaminated]
> > > >
> > > > So far we have tried to mould cp command into not carrying the build
> > > > user permissions into install area but it is never entirely fixed since
> > > > the issue keeps popping up in various scenes
> > > >
> > > > This patch replaces use of cp with install utility and specifies install
> > > > mode for files explcitly
> > > >
> > > > Signed-off-by: Khem Raj 
> > > > ---
> > > >  meta/recipes-core/glibc/glibc-locale.inc | 44 ++--
> > > >  1 file changed, 25 insertions(+), 19 deletions(-)
> > > >
> > > > diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
> > > > b/meta/recipes-core/glibc/glibc-locale.inc
> > > > index 6384f9cbf1..9b256a5108 100644
> > > > --- a/meta/recipes-core/glibc/glibc-locale.inc
> > > > +++ b/meta/recipes-core/glibc/glibc-locale.inc
> > > > @@ -72,27 +72,33 @@ FILES_localedef = "${bindir}/localedef"
> > > >  LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
> > > >
> > > >  do_install () {
> > > > - mkdir -p ${D}${bindir} ${D}${datadir}
> > > > - if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> > > > - cp -R --no-dereference --preserve=mode,links 
> > > > 

Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

[Joshua Watt]

On Thu, Feb 7, 2019 at 10:44 AM Joshua Watt  wrote:
>
> Martin,
>
> Do you by any chance build in Docker (or some other transient container)? We 
> recently found an issue with building in docker where when the primary 
> container command would exit, causing the container to terminate. When the 
> container terminates, the kernel unceremoniously sends SIGKILL to all its 
> processes. This plays *very* poorly with pseudo because pseudo keeps it's 
> entire database in a in-memory sqlite database and only flushes it out to 
> disk on a periodic interval, or when it get SIGTERM (maybe SIGQUIT?) or exits 
> normally. By default the pseudo daemon will also hang around for 20(?) 
> seconds waiting for a new connection from a pseudo client, meaning that when 
> our docker container was exiting, there were potentially several pseudo 
> dameons sitting around waiting for a client with unflushed databases in 
> memory that suddenly got a SIGKILL and all their data was lost. This was 
> particularly bad if you aborted a build while it was running, or some build 
> failure occurred. Since 
 the pseudo database was lost, the only way to correct these errors was to 
clean and rebuild.
>
> I think pseudo could handle this better and keep the database on disk instead 
> of in memory, but in the short term you can hack bitbake to pass the -S flag 
> when invoking the pseudo client which will make it tell the dameon to 
> shutdown (and thus flush the database) if it is the last connection.

FWIW, this patch makes it much easier to do that
http://lists.openembedded.org/pipermail/bitbake-devel/2019-February/019813.html
Just add
 FAKEROOTCMD += "-S"
in local.conf

>
>
> On Thu, 2019-02-07 at 17:17 +0100, Martin Jansa wrote:
>
> There are other rare reproducers which don't involve cp/install (e.g. older 
> qmake was triggering it quite often as well), I've mentioned some in:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12434
>
> glibc-locale is the most common one for sure as pretty much everybody is 
> building that one, I don't have objection converting this from cp to install 
> to prevent this random error (very rare random failure doesn't really help 
> with #12434 reproducer and whoever work on it, can either revert this change 
> or try to reproduce with something else) - I've included few hundreds of 
> INSANE_SKIPs to ignore _random_ host-user-contaminated QAs in LG layers to 
> prevent our builds failing randomly and the one for glibc-locale is the 
> longest part:
> https://github.com/webosose/meta-webosose/blob/master/meta-webos/recipes-core/glibc/glibc-locale_%25.bbappend
>
> Regards,
>
> On Thu, Feb 7, 2019 at 3:50 PM Khem Raj  wrote:
>
> On Thu, Feb 7, 2019 at 1:00 AM Richard Purdie
>  wrote:
> >
> > On Wed, 2019-02-06 at 16:35 -0800, Khem Raj wrote:
> > > This has been a constant source of trouble for build failures due to 
> > > host-user-contaminated QA
> > > errors of sort
> > >
> > > ERROR: QA Issue: glibc-locale: 
> > > /glibc-binary-localedata-ca-es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY
> > >  is owned by uid 3004, which is the same as the user running bitbake. 
> > > This may be due to host contamination [host-user-contaminated]
> > >
> > > So far we have tried to mould cp command into not carrying the build
> > > user permissions into install area but it is never entirely fixed since
> > > the issue keeps popping up in various scenes
> > >
> > > This patch replaces use of cp with install utility and specifies install
> > > mode for files explcitly
> > >
> > > Signed-off-by: Khem Raj 
> > > ---
> > >  meta/recipes-core/glibc/glibc-locale.inc | 44 ++--
> > >  1 file changed, 25 insertions(+), 19 deletions(-)
> > >
> > > diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
> > > b/meta/recipes-core/glibc/glibc-locale.inc
> > > index 6384f9cbf1..9b256a5108 100644
> > > --- a/meta/recipes-core/glibc/glibc-locale.inc
> > > +++ b/meta/recipes-core/glibc/glibc-locale.inc
> > > @@ -72,27 +72,33 @@ FILES_localedef = "${bindir}/localedef"
> > >  LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
> > >
> > >  do_install () {
> > > - mkdir -p ${D}${bindir} ${D}${datadir}
> > > - if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> > > - cp -R --no-dereference --preserve=mode,links 
> > > ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
> > > - fi
> > > - if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
> > > - mkdir -p ${D}${localedir}
> > > - cp -R --no-dereference --preserve=mode,links 
> > > ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
> > > - fi
> > > +install -d ${D}${bindir}
> > > +find "${LOCALETREESRC}/${bindir}" -maxdepth 1 -type f \
> > > +-exec install -m 0755 -t "${D}${bindir}" {} \;
> > > +
> > > +for d in . $(find "${LOCALETREESRC}/${localedir}" -type d 
> > > -printf '%P ') ; do
> > > +install -d "${D}${localedir}/$d"
> > > +  

[OE-core] [PATCH] python3: tweaks to achieve 100% pass rate in ptest

[Alexander Kanavin]

Signed-off-by: Alexander Kanavin 
---
 .../python-sanity/python3/python3-manifest.json   | 4 ++--
 meta/recipes-devtools/python-sanity/python3_3.7.2.bb  | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-devtools/python-sanity/python3/python3-manifest.json 
b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json
index 24f9805fbd2..0a4ab2cbe43 100644
--- a/meta/recipes-devtools/python-sanity/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json
@@ -212,6 +212,7 @@
 "files": [
 "${bindir}/python*[!-config]",
 "${includedir}/python${PYTHON_BINABI}/pyconfig*.h",
+"${prefix}/lib/python${PYTHON_MAJMIN}/config*/*[!.a]",
 "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
 "${libdir}/python${PYTHON_MAJMIN}/UserList.py",
 "${libdir}/python${PYTHON_MAJMIN}/UserString.py",
@@ -491,8 +492,7 @@
 "${libdir}/*.la",
 "${libdir}/*.o",
 "${libdir}/lib*${SOLIBSDEV}",
-"${libdir}/pkgconfig",
-"${prefix}/lib/python${PYTHON_MAJMIN}/config*/"
+"${libdir}/pkgconfig"
 ],
 "rdepends": [
 "core"
diff --git a/meta/recipes-devtools/python-sanity/python3_3.7.2.bb 
b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb
index e05ab42b0ca..8f9d7dc8fea 100644
--- a/meta/recipes-devtools/python-sanity/python3_3.7.2.bb
+++ b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb
@@ -281,7 +281,7 @@ FILES_${PN}-misc = "${libdir}/python${PYTHON_MAJMIN} 
${libdir}/python${PYTHON_MA
 PACKAGES += "${PN}-man"
 FILES_${PN}-man = "${datadir}/man"
 
-RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2"
+RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2 libgcc"
 RDEPENDS_${PN}-tkinter += "${@bb.utils.contains('PACKAGECONFIG', 'tk', 'tk', 
'', d)}"
 RDEPENDS_${PN}-dev = ""
 
-- 
2.17.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

[Joshua Watt]

Martin,
Do you by any chance build in Docker (or some other transient
container)? We recently found an issue with building in docker where
when the primary container command would exit, causing the container to
terminate. When the container terminates, the kernel unceremoniously
sends SIGKILL to all its processes. This plays *very* poorly with
pseudo because pseudo keeps it's entire database in a in-memory sqlite
database and only flushes it out to disk on a periodic interval, or
when it get SIGTERM (maybe SIGQUIT?) or exits normally. By default the
pseudo daemon will also hang around for 20(?) seconds waiting for a new
connection from a pseudo client, meaning that when our docker container
was exiting, there were potentially several pseudo dameons sitting
around waiting for a client with unflushed databases in memory that
suddenly got a SIGKILL and all their data was lost. This was
particularly bad if you aborted a build while it was running, or some
build failure occurred. Since the pseudo database was lost, the only
way to correct these errors was to clean and rebuild.
I think pseudo could handle this better and keep the database on disk
instead of in memory, but in the short term you can hack bitbake to
pass the -S flag when invoking the pseudo client which will make it
tell the dameon to shutdown (and thus flush the database) if it is the
last connection.

On Thu, 2019-02-07 at 17:17 +0100, Martin Jansa wrote:
> There are other rare reproducers which don't involve cp/install (e.g.
> older qmake was triggering it quite often as well), I've mentioned
> some in:https://bugzilla.yoctoproject.org/show_bug.cgi?id=12434
> 
> glibc-locale is the most common one for sure as pretty much everybody
> is building that one, I don't have objection converting this from cp
> to install to prevent this random error (very rare random failure
> doesn't really help with #12434 reproducer and whoever work on it,
> can either revert this change or try to reproduce with something
> else) - I've included few hundreds of INSANE_SKIPs to ignore _random_
> host-user-contaminated QAs in LG layers to prevent our builds failing
> randomly and the one for glibc-locale is the longest part:
> https://github.com/webosose/meta-webosose/blob/master/meta-webos/recipes-core/glibc/glibc-locale_%25.bbappend
> 
> Regards,
> 
> On Thu, Feb 7, 2019 at 3:50 PM Khem Raj  wrote:
> > On Thu, Feb 7, 2019 at 1:00 AM Richard Purdie
> > 
> >  wrote:
> > 
> > >
> > 
> > > On Wed, 2019-02-06 at 16:35 -0800, Khem Raj wrote:
> > 
> > > > This has been a constant source of trouble for build failures
> > due to host-user-contaminated QA
> > 
> > > > errors of sort
> > 
> > > >
> > 
> > > > ERROR: QA Issue: glibc-locale: 
> > /glibc-binary-localedata-ca-es+valencia/usr/lib/locale/ca_ES@valencia
> > /LC_MONETARY is owned by uid 3004, which is the same as the user
> > running bitbake. This may be due to host contamination [host-user-
> > contaminated]
> > 
> > > >
> > 
> > > > So far we have tried to mould cp command into not carrying the
> > build
> > 
> > > > user permissions into install area but it is never entirely
> > fixed since
> > 
> > > > the issue keeps popping up in various scenes
> > 
> > > >
> > 
> > > > This patch replaces use of cp with install utility and
> > specifies install
> > 
> > > > mode for files explcitly
> > 
> > > >
> > 
> > > > Signed-off-by: Khem Raj 
> > 
> > > > ---
> > 
> > > >  meta/recipes-core/glibc/glibc-locale.inc | 44 ++
> > --
> > 
> > > >  1 file changed, 25 insertions(+), 19 deletions(-)
> > 
> > > >
> > 
> > > > diff --git a/meta/recipes-core/glibc/glibc-locale.inc
> > b/meta/recipes-core/glibc/glibc-locale.inc
> > 
> > > > index 6384f9cbf1..9b256a5108 100644
> > 
> > > > --- a/meta/recipes-core/glibc/glibc-locale.inc
> > 
> > > > +++ b/meta/recipes-core/glibc/glibc-locale.inc
> > 
> > > > @@ -72,27 +72,33 @@ FILES_localedef = "${bindir}/localedef"
> > 
> > > >  LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-
> > stash-locale"
> > 
> > > >
> > 
> > > >  do_install () {
> > 
> > > > - mkdir -p ${D}${bindir} ${D}${datadir}
> > 
> > > > - if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> > 
> > > > - cp -R --no-dereference --preserve=mode,links
> > ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
> > 
> > > > - fi
> > 
> > > > - if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
> > 
> > > > - mkdir -p ${D}${localedir}
> > 
> > > > - cp -R --no-dereference --preserve=mode,links
> > ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
> > 
> > > > - fi
> > 
> > > > +install -d ${D}${bindir}
> > 
> > > > +find "${LOCALETREESRC}/${bindir}" -maxdepth 1 -type f
> > \
> > 
> > > > +-exec install -m 0755 -t "${D}${bindir}" {} \;
> > 
> > > > +
> > 
> > > > +for d in . $(find "${LOCALETREESRC}/${localedir}"
> > -type d -printf '%P ') ; do
> > 
> > > > +install -d "${D}${localedir}/$d"
> > 
> > > > +   

Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

[Martin Jansa]

There are other rare reproducers which don't involve cp/install (e.g. older
qmake was triggering it quite often as well), I've mentioned some in:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12434

glibc-locale is the most common one for sure as pretty much everybody is
building that one, I don't have objection converting this from cp to
install to prevent this random error (very rare random failure doesn't
really help with #12434 reproducer and whoever work on it, can either
revert this change or try to reproduce with something else) - I've included
few hundreds of INSANE_SKIPs to ignore _random_ host-user-contaminated QAs
in LG layers to prevent our builds failing randomly and the one for
glibc-locale is the longest part:
https://github.com/webosose/meta-webosose/blob/master/meta-webos/recipes-core/glibc/glibc-locale_%25.bbappend

Regards,

On Thu, Feb 7, 2019 at 3:50 PM Khem Raj  wrote:

> On Thu, Feb 7, 2019 at 1:00 AM Richard Purdie
>  wrote:
> >
> > On Wed, 2019-02-06 at 16:35 -0800, Khem Raj wrote:
> > > This has been a constant source of trouble for build failures due to
> host-user-contaminated QA
> > > errors of sort
> > >
> > > ERROR: QA Issue: glibc-locale:
> /glibc-binary-localedata-ca-es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY
> is owned by uid 3004, which is the same as the user running bitbake. This
> may be due to host contamination [host-user-contaminated]
> > >
> > > So far we have tried to mould cp command into not carrying the build
> > > user permissions into install area but it is never entirely fixed since
> > > the issue keeps popping up in various scenes
> > >
> > > This patch replaces use of cp with install utility and specifies
> install
> > > mode for files explcitly
> > >
> > > Signed-off-by: Khem Raj 
> > > ---
> > >  meta/recipes-core/glibc/glibc-locale.inc | 44 ++--
> > >  1 file changed, 25 insertions(+), 19 deletions(-)
> > >
> > > diff --git a/meta/recipes-core/glibc/glibc-locale.inc
> b/meta/recipes-core/glibc/glibc-locale.inc
> > > index 6384f9cbf1..9b256a5108 100644
> > > --- a/meta/recipes-core/glibc/glibc-locale.inc
> > > +++ b/meta/recipes-core/glibc/glibc-locale.inc
> > > @@ -72,27 +72,33 @@ FILES_localedef = "${bindir}/localedef"
> > >  LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
> > >
> > >  do_install () {
> > > - mkdir -p ${D}${bindir} ${D}${datadir}
> > > - if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> > > - cp -R --no-dereference --preserve=mode,links
> ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
> > > - fi
> > > - if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
> > > - mkdir -p ${D}${localedir}
> > > - cp -R --no-dereference --preserve=mode,links
> ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
> > > - fi
> > > +install -d ${D}${bindir}
> > > +find "${LOCALETREESRC}/${bindir}" -maxdepth 1 -type f \
> > > +-exec install -m 0755 -t "${D}${bindir}" {} \;
> > > +
> > > +for d in . $(find "${LOCALETREESRC}/${localedir}" -type d
> -printf '%P ') ; do
> > > +install -d "${D}${localedir}/$d"
> > > +find "${LOCALETREESRC}/${localedir}/$d" -maxdepth 1
> -type f \
> > > +-exec install -m 0644 -t "${D}${localedir}/$d" {} \;
> > > +done
> > >   if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
> > > - mkdir -p ${D}${libdir}
> > > - if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
> > > - cp -R --no-dereference --preserve=mode,links
> ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
> > > - fi
> > > - if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
> > > - cp -R --no-dereference --preserve=mode,links
> ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
> > > - fi
> > > - fi
> > > - if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
> > > - cp -R --no-dereference --preserve=mode,links
> ${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
> > > +for d in . $(find "${LOCALETREESRC}/${libdir}/gconv"
> -type d -printf '%P ') ; do
> > > +install -d "${D}${libdir}/gconv/$d"
> > > +find "${LOCALETREESRC}/${libdir}/gconv/$d"
> -maxdepth 1 -type f \
> > > +-exec install -m 0755 -t
> "${D}${libdir}/gconv/$d" {} \;
> > > +done
> > > +for d in . $(find "${LOCALETREESRC}/${datadir}/i18n"
> -type d -printf '%P ') ; do
> > > +install -d "${D}${datadir}/i18n/$d"
> > > +find "${LOCALETREESRC}/${datadir}/i18n/$d"
> -maxdepth 1 -type f \
> > > +-exec install -m 0644 -t
> "${D}${datadir}/i18n/$d" {} \;
> > > +done
> > >   fi
> > > - cp -R --no-dereference --preserve=mode,links
> ${LOCALETREESRC}/SUPPORTED ${WORKDIR}
> > > +for d in . 

[OE-core] [PATCH 1/3] opkg: add --ignore-recommends flag

[Alejandro del Castillo]

To be used for BAD_RECOMMENDATIONS feature.

Signed-off-by: Alejandro del Castillo 
---
 ...pkg-add-add-ignore-recommends-option.patch | 259 ++
 meta/recipes-devtools/opkg/opkg_0.4.0.bb  |   1 +
 2 files changed, 260 insertions(+)
 create mode 100644 
meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch

diff --git 
a/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
 
b/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
new file mode 100644
index 00..47d1b3c37e
--- /dev/null
+++ 
b/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
@@ -0,0 +1,259 @@
+From 64aa98646a17c299bf37af2975b98daf5d7d30b4 Mon Sep 17 00:00:00 2001
+From: Alejandro del Castillo 
+Date: Thu, 31 Jan 2019 18:16:08 -0600
+Subject: [PATCH] libopkg: add --add-ignore-recommends option
+
+Add option to ignore specific recommended packages. On the libsolv
+backed, this feature will only work on libsolv version > 0.7.2 [1].
+
+[1] 
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openSUSE_libsolv_issues_254=DwIBaQ=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA=wNcrL2akRn6jfxhHaKavUrJB_C9JAMXtynjLd8ZzgXQ=GObNHzFJpWpf_PripIrf-K2RhsktYdAUEieAJexXOKw=3G-meChUqClFggFPqsrAxIZBfLnRKIHm62Uuy1X6nQQ=
+
+Signed-off-by: Alejandro del Castillo 
+
+Upstream-Status: Accepted
+---
+ libopkg/opkg_conf.c   |  2 +
+ libopkg/opkg_conf.h   |  1 +
+ .../solvers/internal/pkg_depends_internal.c   |  3 +-
+ libopkg/solvers/libsolv/opkg_solver_libsolv.c | 21 ++-
+ man/opkg.1.in |  3 +
+ src/opkg.c|  6 ++
+ tests/Makefile|  1 +
+ tests/core/43_add_ignore_recommends.py| 62 +++
+ 8 files changed, 97 insertions(+), 2 deletions(-)
+ create mode 100755 tests/core/43_add_ignore_recommends.py
+
+diff --git a/libopkg/opkg_conf.c b/libopkg/opkg_conf.c
+index 06880a1..f2330cd 100644
+--- a/libopkg/opkg_conf.c
 b/libopkg/opkg_conf.c
+@@ -597,6 +597,7 @@ int opkg_conf_init(void)
+ pkg_dest_list_init(_config->tmp_dest_list);
+ nv_pair_list_init(_config->arch_list);
+ str_list_init(_config->exclude_list);
++str_list_init(_config->ignore_recommends_list);
+ 
+ return 0;
+ }
+@@ -938,6 +939,7 @@ void opkg_conf_deinit(void)
+ pkg_dest_list_deinit(_config->pkg_dest_list);
+ nv_pair_list_deinit(_config->arch_list);
+ str_list_deinit(_config->exclude_list);
++str_list_deinit(_config->ignore_recommends_list);
+ 
+ if (opkg_config->verbosity >= DEBUG) {
+ hash_print_stats(_config->pkg_hash);
+diff --git a/libopkg/opkg_conf.h b/libopkg/opkg_conf.h
+index eb56a29..316c500 100644
+--- a/libopkg/opkg_conf.h
 b/libopkg/opkg_conf.h
+@@ -61,6 +61,7 @@ typedef struct opkg_conf {
+ pkg_dest_list_t tmp_dest_list;
+ nv_pair_list_t arch_list;
+ str_list_t exclude_list;
++str_list_t ignore_recommends_list;
+ 
+ int restrict_to_default_dest;
+ pkg_dest_t *default_dest;
+diff --git a/libopkg/solvers/internal/pkg_depends_internal.c 
b/libopkg/solvers/internal/pkg_depends_internal.c
+index cd56d84..5deee70 100644
+--- a/libopkg/solvers/internal/pkg_depends_internal.c
 b/libopkg/solvers/internal/pkg_depends_internal.c
+@@ -228,7 +228,8 @@ int pkg_hash_fetch_unsatisfied_dependencies(pkg_t *pkg,
+ || compound_depend->type == SUGGEST)
+ && (satisfying_pkg->state_want == SW_DEINSTALL
+ || satisfying_pkg->state_want == SW_PURGE
+-|| opkg_config->no_install_recommends);
++|| opkg_config->no_install_recommends
++|| 
str_list_contains(_config->ignore_recommends_list, satisfying_pkg->name));
+ if (ignore) {
+ opkg_msg(NOTICE,
+  "%s: ignoring recommendation for "
+diff --git a/libopkg/solvers/libsolv/opkg_solver_libsolv.c 
b/libopkg/solvers/libsolv/opkg_solver_libsolv.c
+index 2b27e3a..403e07b 100644
+--- a/libopkg/solvers/libsolv/opkg_solver_libsolv.c
 b/libopkg/solvers/libsolv/opkg_solver_libsolv.c
+@@ -484,6 +484,7 @@ static void pkg2solvable(pkg_t *pkg, Solvable 
*solvable_out)
+ static void populate_installed_repo(libsolv_solver_t *libsolv_solver)
+ {
+ int i;
++Id what;
+ 
+ pkg_vec_t *installed_pkgs = pkg_vec_alloc();
+ 
+@@ -507,6 +508,15 @@ static void populate_installed_repo(libsolv_solver_t 
*libsolv_solver)
+ /* set solvable attributes */
+ pkg2solvable(pkg, solvable);
+ 
++/* if the package is in ignore-recommends-list, disfavor installation 
*/
++if (str_list_contains(_config->ignore_recommends_list, 
pkg->name)) {
++opkg_message(NOTICE, "Disfavor package: %s\n",
++ 

[OE-core] [PATCH 2/3] libsolv: fix segfault when using SOLVER_DISFAVOR

[Alejandro del Castillo]

Opkg --add-ignore-recommends use the flag SOLVER_DISFAVOR. If a package
is set to be ignored as a recommendee, but there are not other
recommends on the package graph, libsolv segfaults.

Signed-off-by: Alejandro del Castillo 
---
 ...y-disfavor-recommends-if-there-are-a.patch | 33 +++
 .../recipes-extended/libsolv/libsolv_0.7.3.bb |  4 ++-
 2 files changed, 36 insertions(+), 1 deletion(-)
 create mode 100644 
meta/recipes-extended/libsolv/libsolv/0001-solver_solve-only-disfavor-recommends-if-there-are-a.patch

diff --git 
a/meta/recipes-extended/libsolv/libsolv/0001-solver_solve-only-disfavor-recommends-if-there-are-a.patch
 
b/meta/recipes-extended/libsolv/libsolv/0001-solver_solve-only-disfavor-recommends-if-there-are-a.patch
new file mode 100644
index 00..139613a0ad
--- /dev/null
+++ 
b/meta/recipes-extended/libsolv/libsolv/0001-solver_solve-only-disfavor-recommends-if-there-are-a.patch
@@ -0,0 +1,33 @@
+From 19d7cc87adba92d31d5fafdf7db00920d24a96a6 Mon Sep 17 00:00:00 2001
+From: Alejandro del Castillo 
+Date: Wed, 6 Feb 2019 13:24:04 -0600
+Subject: [PATCH] solver_solve: only disfavor recommends if there are any
+
+In a repo that have pkg 'a' and 'b' available, and 'b' is disfavored,
+but 'a' doesn't recommend 'b', libsolv segfaults on
+solver_addrecommendsrules, since solv->recommendsruleq is null. Only
+call solver_addrecommendsrules if there are recommends rules.
+
+Signed-off-by: Alejandro del Castillo 
+
+Upstream-Status: Accepted
+---
+ src/solver.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/src/solver.c b/src/solver.c
+index a80090d19..ad78327a8 100644
+--- a/src/solver.c
 b/src/solver.c
+@@ -3920,7 +3920,7 @@ solver_solve(Solver *solv, Queue *job)
+   else
+ solv->yumobsrules = solv->yumobsrules_end = solv->nrules;
+ 
+-  if (solv->havedisfavored && solv->strongrecommends)
++  if (solv->havedisfavored && solv->strongrecommends && solv->recommendsruleq)
+ solver_addrecommendsrules(solv);
+   else
+ solv->recommendsrules = solv->recommendsrules_end = solv->nrules;
+-- 
+2.20.1
+
diff --git a/meta/recipes-extended/libsolv/libsolv_0.7.3.bb 
b/meta/recipes-extended/libsolv/libsolv_0.7.3.bb
index 44757ac54f..3fe6f2c20a 100644
--- a/meta/recipes-extended/libsolv/libsolv_0.7.3.bb
+++ b/meta/recipes-extended/libsolv/libsolv_0.7.3.bb
@@ -7,7 +7,9 @@ LIC_FILES_CHKSUM = 
"file://LICENSE.BSD;md5=62272bd11c97396d4aaf1c41bc11f7d8"
 
 DEPENDS = "expat zlib"
 
-SRC_URI = "git://github.com/openSUSE/libsolv.git"
+SRC_URI = "git://github.com/openSUSE/libsolv.git \
+   
file://0001-solver_solve-only-disfavor-recommends-if-there-are-a.patch \
+"
 
 SRCREV = "dc7d0f1c3113f2c8217563166906bef3eb5d1ee1"
 UPSTREAM_CHECK_GITTAGREGEX = "(?P\d+(\.\d+)+)"
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] use new opkg --add-ignore-recommends for BAD_RECOMMENDATIONS feature

[Alejandro del Castillo]

- Add new opkg --add-ignore-recommends cli flag
- Fix libsolv segfaul triggered by --add-ignore-recommends
- Replace BAD_RECOMMENDATIONS backend in package_manager.py

Alejandro del Castillo (3):
  opkg: add --ignore-recommends flag
  libsolv: fix segfault when using SOLVER_DISFAVOR
  OpkgPM: use --add-ignore-recommends to process BAD_RECOMMENDATIONS

 meta/lib/oe/package_manager.py|  41 +--
 meta/lib/oe/rootfs.py |   2 -
 ...pkg-add-add-ignore-recommends-option.patch | 259 ++
 meta/recipes-devtools/opkg/opkg_0.4.0.bb  |   1 +
 ...y-disfavor-recommends-if-there-are-a.patch |  33 +++
 .../recipes-extended/libsolv/libsolv_0.7.3.bb |   4 +-
 6 files changed, 298 insertions(+), 42 deletions(-)
 create mode 100644 
meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
 create mode 100644 
meta/recipes-extended/libsolv/libsolv/0001-solver_solve-only-disfavor-recommends-if-there-are-a.patch

-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] OpkgPM: use --add-ignore-recommends to process BAD_RECOMMENDATIONS

[Alejandro del Castillo]

Currently, BAD_RECOMMENDATIONS on the opkg backed relies on editing the
opkg status file (it sets BAD_RECOMMENDATIONS pkg want state to
deinstalled and pinned). This is brittle, and not consistent across the
different solver backends. Use new --add-ignore-recommends flag instead.

Signed-off-by: Alejandro del Castillo 
---
 meta/lib/oe/package_manager.py | 41 ++
 meta/lib/oe/rootfs.py  |  2 --
 2 files changed, 2 insertions(+), 41 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index f26f597d03..7e7d7db083 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -1339,6 +1339,8 @@ class OpkgPM(OpkgDpkgPM):
 cmd = "%s %s" % (self.opkg_cmd, self.opkg_args)
 for exclude in (self.d.getVar("PACKAGE_EXCLUDE") or "").split():
 cmd += " --add-exclude %s" % exclude
+for bad_recommendation in (self.d.getVar("BAD_RECOMMENDATIONS") or 
"").split():
+cmd += " --add-ignore-recommends %s" % bad_recommendation
 cmd += " install "
 cmd += " ".join(pkgs)
 
@@ -1407,45 +1409,6 @@ class OpkgPM(OpkgDpkgPM):
 def list_installed(self):
 return OpkgPkgsList(self.d, self.target_rootfs, 
self.config_file).list_pkgs()
 
-def handle_bad_recommendations(self):
-bad_recommendations = self.d.getVar("BAD_RECOMMENDATIONS") or ""
-if bad_recommendations.strip() == "":
-return
-
-status_file = os.path.join(self.opkg_dir, "status")
-
-# If status file existed, it means the bad recommendations has already
-# been handled
-if os.path.exists(status_file):
-return
-
-cmd = "%s %s info " % (self.opkg_cmd, self.opkg_args)
-
-with open(status_file, "w+") as status:
-for pkg in bad_recommendations.split():
-pkg_info = cmd + pkg
-
-try:
-output = subprocess.check_output(pkg_info.split(), 
stderr=subprocess.STDOUT).strip().decode("utf-8")
-except subprocess.CalledProcessError as e:
-bb.fatal("Cannot get package info. Command '%s' "
- "returned %d:\n%s" % (pkg_info, e.returncode, 
e.output.decode("utf-8")))
-
-if output == "":
-bb.note("Ignored bad recommendation: '%s' is "
-"not a package" % pkg)
-continue
-
-for line in output.split('\n'):
-if line.startswith("Status:"):
-status.write("Status: deinstall hold not-installed\n")
-else:
-status.write(line + "\n")
-
-# Append a blank line after each package entry to ensure that 
it
-# is separated from the following entry
-status.write("\n")
-
 def dummy_install(self, pkgs):
 """
 The following function dummy installs pkgs and returns the log of 
output.
diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index 551dcfc75f..cf5cd996a2 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -891,8 +891,6 @@ class OpkgRootfs(DpkgOpkgRootfs):
 
 self.pm.update()
 
-self.pm.handle_bad_recommendations()
-
 if self.progress_reporter:
 self.progress_reporter.next_stage()
 
-- 
2.20.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] wpa_supplicant: Changed systemd template units

[Joshua DeWeese]

From: Joshua DeWeese 

I goofed up the scissor line on the last attempt. Not sure how much it matters,
but here it is correct this time.

Here it is, updated to work with wpa-supplicant_2.6.bb.

-- >8 --
https://www.freedesktop.org/software/systemd/man/systemd.unit.html#WantedBy=

When building root filesystems with any of the wpa_supplicant systemd
template service files enabled (current default is to have them disabled) the
systemd-native-fake script would not process the line:

Alias=multi-user.target.wants/wpa_supplicant@%i.service

appropriately due the the use of "%i."

According to the systemd documentation "WantedBy=foo.service in a service
bar.service is mostly equivalent to Alias=foo.service.wants/bar.service in
the same file." However, this is not really the intended purpose of install
Aliases.

All lines of the form:

Alias=multi-user.target.wants/*%i.service

Were replaced with the following lines:

WantedBy=multi-user.target

Signed-off-by: Joshua DeWeese 
---
 ...place-systemd-install-Alias-with-WantedBy.patch | 52 ++
 .../wpa-supplicant/wpa-supplicant_2.6.bb   |  1 +
 2 files changed, 53 insertions(+)
 create mode 100644 
meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/0001-replace-systemd-install-Alias-with-WantedBy.patch

diff --git 
a/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/0001-replace-systemd-install-Alias-with-WantedBy.patch
 
b/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/0001-replace-systemd-install-Alias-with-WantedBy.patch
new file mode 100644
index 00..a476cf040e
--- /dev/null
+++ 
b/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/0001-replace-systemd-install-Alias-with-WantedBy.patch
@@ -0,0 +1,52 @@
+From 94c401733a5a3d294cc412671166e6adfb409f53 Mon Sep 17 00:00:00 2001
+From: Joshua DeWeese 
+Date: Wed, 30 Jan 2019 16:19:47 -0500
+Subject: [PATCH] replace systemd install Alias with WantedBy
+
+According to the systemd documentation "WantedBy=foo.service in a
+service bar.service is mostly equivalent to
+Alias=foo.service.wants/bar.service in the same file." However,
+this is not really the intended purpose of install Aliases.
+
+Upstream-Status: Submitted [hos...@lists.infradead.org]
+
+Signed-off-by: Joshua DeWeese 
+---
+ wpa_supplicant/systemd/wpa_supplicant-nl80211.service.arg.in | 2 +-
+ wpa_supplicant/systemd/wpa_supplicant-wired.service.arg.in   | 2 +-
+ wpa_supplicant/systemd/wpa_supplicant.service.arg.in | 2 +-
+ 3 files changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/wpa_supplicant/systemd/wpa_supplicant-nl80211.service.arg.in 
b/wpa_supplicant/systemd/wpa_supplicant-nl80211.service.arg.in
+index 03ac507..da69a87 100644
+--- a/wpa_supplicant/systemd/wpa_supplicant-nl80211.service.arg.in
 b/wpa_supplicant/systemd/wpa_supplicant-nl80211.service.arg.in
+@@ -12,4 +12,4 @@ Type=simple
+ ExecStart=@BINDIR@/wpa_supplicant 
-c/etc/wpa_supplicant/wpa_supplicant-nl80211-%I.conf -Dnl80211 -i%I
+ 
+ [Install]
+-Alias=multi-user.target.wants/wpa_supplicant-nl80211@%i.service
++WantedBy=multi-user.target
+diff --git a/wpa_supplicant/systemd/wpa_supplicant-wired.service.arg.in 
b/wpa_supplicant/systemd/wpa_supplicant-wired.service.arg.in
+index c8a744d..ca3054b 100644
+--- a/wpa_supplicant/systemd/wpa_supplicant-wired.service.arg.in
 b/wpa_supplicant/systemd/wpa_supplicant-wired.service.arg.in
+@@ -12,4 +12,4 @@ Type=simple
+ ExecStart=@BINDIR@/wpa_supplicant 
-c/etc/wpa_supplicant/wpa_supplicant-wired-%I.conf -Dwired -i%I
+ 
+ [Install]
+-Alias=multi-user.target.wants/wpa_supplicant-wired@%i.service
++WantedBy=multi-user.target
+diff --git a/wpa_supplicant/systemd/wpa_supplicant.service.arg.in 
b/wpa_supplicant/systemd/wpa_supplicant.service.arg.in
+index 7788b38..55d2b9c 100644
+--- a/wpa_supplicant/systemd/wpa_supplicant.service.arg.in
 b/wpa_supplicant/systemd/wpa_supplicant.service.arg.in
+@@ -12,4 +12,4 @@ Type=simple
+ ExecStart=@BINDIR@/wpa_supplicant 
-c/etc/wpa_supplicant/wpa_supplicant-%I.conf -i%I
+ 
+ [Install]
+-Alias=multi-user.target.wants/wpa_supplicant@%i.service
++WantedBy=multi-user.target
+-- 
+2.7.4
+
diff --git a/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_2.6.bb 
b/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_2.6.bb
index aa4c4c2da0..c92ed4ab93 100644
--- a/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_2.6.bb
+++ b/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_2.6.bb
@@ -33,6 +33,7 @@ SRC_URI = "http://w1.fi/releases/wpa_supplicant-${PV}.tar.gz  
\
file://key-replay-cve-multiple7.patch \
file://key-replay-cve-multiple8.patch \
file://wpa_supplicant-CVE-2018-14526.patch \
+   file://0001-replace-systemd-install-Alias-with-WantedBy.patch \
   "
 SRC_URI[md5sum] = "091569eb4440b7d7f2b4276dbfc03c3c"
 SRC_URI[sha256sum] = 
"b4936d34c4e6cdd44954beba74296d964bc2c9668ecaa5255e499636fe2b1450"
-- 
2.11.0

-- 
___

Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

[Khem Raj]

On Thu, Feb 7, 2019 at 3:44 AM Peter Kjellerstedt
 wrote:
>
> > -Original Message-
> > From: openembedded-core-boun...@lists.openembedded.org  > core-boun...@lists.openembedded.org> On Behalf Of Khem Raj
> > Sent: den 7 februari 2019 01:36
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [PATCH] glibc-locale: Rewrite do_install using
> > install utility instead of cp
> >
> > This has been a constant source of trouble for build failures due to
> > host-user-contaminated QA errors of sort
> >
> > ERROR: QA Issue: glibc-locale: /glibc-binary-localedata-ca-
> > es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY is owned by uid
> > 3004, which is the same as the user running bitbake. This may be due to
> > host contamination [host-user-contaminated]
> >
> > So far we have tried to mould cp command into not carrying the build
> > user permissions into install area but it is never entirely fixed since
> > the issue keeps popping up in various scenes
> >
> > This patch replaces use of cp with install utility and specifies
> > install mode for files explcitly
> >
> > Signed-off-by: Khem Raj 
> > ---
> >  meta/recipes-core/glibc/glibc-locale.inc | 44 ++--
> >  1 file changed, 25 insertions(+), 19 deletions(-)
> >
> > diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
> > b/meta/recipes-core/glibc/glibc-locale.inc
> > index 6384f9cbf1..9b256a5108 100644
> > --- a/meta/recipes-core/glibc/glibc-locale.inc
> > +++ b/meta/recipes-core/glibc/glibc-locale.inc
> > @@ -72,27 +72,33 @@ FILES_localedef = "${bindir}/localedef"
> >  LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
> >
> >  do_install () {
> > - mkdir -p ${D}${bindir} ${D}${datadir}
> > - if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
> > - fi
> > - if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
> > - mkdir -p ${D}${localedir}
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
> > - fi
> > +install -d ${D}${bindir}
> > +find "${LOCALETREESRC}/${bindir}" -maxdepth 1 -type f \
> > +-exec install -m 0755 -t "${D}${bindir}" {} \;
> > +
> > +for d in . $(find "${LOCALETREESRC}/${localedir}" -type d -printf 
> > '%P ') ; do
> > +install -d "${D}${localedir}/$d"
> > +find "${LOCALETREESRC}/${localedir}/$d" -maxdepth 1 -type 
> > f \
> > +-exec install -m 0644 -t "${D}${localedir}/$d" {} \;
> > +done
> >   if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
> > - mkdir -p ${D}${libdir}
> > - if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
> > - fi
> > - if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
> > - fi
> > - fi
> > - if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
> > +for d in . $(find "${LOCALETREESRC}/${libdir}/gconv" -type 
> > d -printf '%P ') ; do
> > +install -d "${D}${libdir}/gconv/$d"
> > +find "${LOCALETREESRC}/${libdir}/gconv/$d" 
> > -maxdepth 1 -type f \
> > +-exec install -m 0755 -t "${D}${libdir}/gconv/$d" 
> > {} \;
> > +done
> > +for d in . $(find "${LOCALETREESRC}/${datadir}/i18n" -type 
> > d -printf '%P ') ; do
> > +install -d "${D}${datadir}/i18n/$d"
> > +find "${LOCALETREESRC}/${datadir}/i18n/$d" 
> > -maxdepth 1 -type f \
> > +-exec install -m 0644 -t "${D}${datadir}/i18n/$d" 
> > {} \;
> > +done
> >   fi
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/SUPPORTED ${WORKDIR}
> > +for d in . $(find "${LOCALETREESRC}/${datadir}/locale" -type d 
> > -printf '%P ') ; do
> > +install -d "${D}${datadir}/locale/$d"
> > +find "${LOCALETREESRC}/${datadir}/locale/$d" -maxdepth 1 
> > -type f \
> > +-exec install -m 0644 -t "${D}${datadir}/locale/$d" {} \;
> > +done
> > + install -m 0644 ${LOCALETREESRC}/SUPPORTED ${WORKDIR}/SUPPORTED
> >  }
> >
> >  inherit libc-package
> > --
> > 2.20.1
>
> May I suggest using a help function to simplify the code a lot:
>
> copy_locale_files() {
> local dir=$1 mode=$2
>
> [ -e "${LOCALETREESRC}$dir" ] || return 0
>
> for d in . $(find "${LOCALETREESRC}$dir" -type d -printf '%P 

Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

[Khem Raj]

On Thu, Feb 7, 2019 at 1:00 AM Richard Purdie
 wrote:
>
> On Wed, 2019-02-06 at 16:35 -0800, Khem Raj wrote:
> > This has been a constant source of trouble for build failures due to 
> > host-user-contaminated QA
> > errors of sort
> >
> > ERROR: QA Issue: glibc-locale: 
> > /glibc-binary-localedata-ca-es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY
> >  is owned by uid 3004, which is the same as the user running bitbake. This 
> > may be due to host contamination [host-user-contaminated]
> >
> > So far we have tried to mould cp command into not carrying the build
> > user permissions into install area but it is never entirely fixed since
> > the issue keeps popping up in various scenes
> >
> > This patch replaces use of cp with install utility and specifies install
> > mode for files explcitly
> >
> > Signed-off-by: Khem Raj 
> > ---
> >  meta/recipes-core/glibc/glibc-locale.inc | 44 ++--
> >  1 file changed, 25 insertions(+), 19 deletions(-)
> >
> > diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
> > b/meta/recipes-core/glibc/glibc-locale.inc
> > index 6384f9cbf1..9b256a5108 100644
> > --- a/meta/recipes-core/glibc/glibc-locale.inc
> > +++ b/meta/recipes-core/glibc/glibc-locale.inc
> > @@ -72,27 +72,33 @@ FILES_localedef = "${bindir}/localedef"
> >  LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
> >
> >  do_install () {
> > - mkdir -p ${D}${bindir} ${D}${datadir}
> > - if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
> > - fi
> > - if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
> > - mkdir -p ${D}${localedir}
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
> > - fi
> > +install -d ${D}${bindir}
> > +find "${LOCALETREESRC}/${bindir}" -maxdepth 1 -type f \
> > +-exec install -m 0755 -t "${D}${bindir}" {} \;
> > +
> > +for d in . $(find "${LOCALETREESRC}/${localedir}" -type d -printf 
> > '%P ') ; do
> > +install -d "${D}${localedir}/$d"
> > +find "${LOCALETREESRC}/${localedir}/$d" -maxdepth 1 -type 
> > f \
> > +-exec install -m 0644 -t "${D}${localedir}/$d" {} \;
> > +done
> >   if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
> > - mkdir -p ${D}${libdir}
> > - if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
> > - fi
> > - if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
> > - fi
> > - fi
> > - if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
> > +for d in . $(find "${LOCALETREESRC}/${libdir}/gconv" -type 
> > d -printf '%P ') ; do
> > +install -d "${D}${libdir}/gconv/$d"
> > +find "${LOCALETREESRC}/${libdir}/gconv/$d" 
> > -maxdepth 1 -type f \
> > +-exec install -m 0755 -t "${D}${libdir}/gconv/$d" 
> > {} \;
> > +done
> > +for d in . $(find "${LOCALETREESRC}/${datadir}/i18n" -type 
> > d -printf '%P ') ; do
> > +install -d "${D}${datadir}/i18n/$d"
> > +find "${LOCALETREESRC}/${datadir}/i18n/$d" 
> > -maxdepth 1 -type f \
> > +-exec install -m 0644 -t "${D}${datadir}/i18n/$d" 
> > {} \;
> > +done
> >   fi
> > - cp -R --no-dereference --preserve=mode,links 
> > ${LOCALETREESRC}/SUPPORTED ${WORKDIR}
> > +for d in . $(find "${LOCALETREESRC}/${datadir}/locale" -type d 
> > -printf '%P ') ; do
> > +install -d "${D}${datadir}/locale/$d"
> > +find "${LOCALETREESRC}/${datadir}/locale/$d" -maxdepth 1 
> > -type f \
> > +-exec install -m 0644 -t "${D}${datadir}/locale/$d" {} \;
> > +done
> > + install -m 0644 ${LOCALETREESRC}/SUPPORTED ${WORKDIR}/SUPPORTED
> >  }
> >
> >  inherit libc-package
>
> The trouble is this is a workaround. The cp commands should work and
> there is some underlying issue in pseudo causing this. We really need
> to figure out what that is since this will likely just mean we see the
> problem somewhere else :(

so far this is the only place where it shows in world builds that
meta-oe does which includes many layers.

>
> I still suspect some kind of inode number reuse problem which these cp
> commands trigger...

cp and install are not same when it comes to how they operate
underneath. install is better when it comes 

[OE-core] (no subject)

[Joshua DeWeese]

Here it is, updated to work with wpa-supplicant_2.6.bb.

-- >8 --
>From 12d39342b00a1ad5536b105e23b9ea732bba5489 Mon Sep 17 00:00:00 2001
From: Joshua DeWeese 
Date: Tue, 5 Feb 2019 10:08:37 -0500
Subject: [PATCH] wpa_supplicant: Changed systemd template units

https://www.freedesktop.org/software/systemd/man/systemd.unit.html#WantedBy=

When building root filesystems with any of the wpa_supplicant systemd
template service files enabled (current default is to have them disabled) the
systemd-native-fake script would not process the line:

Alias=multi-user.target.wants/wpa_supplicant@%i.service

appropriately due the the use of "%i."

According to the systemd documentation "WantedBy=foo.service in a service
bar.service is mostly equivalent to Alias=foo.service.wants/bar.service in
the same file." However, this is not really the intended purpose of install
Aliases.

All lines of the form:

Alias=multi-user.target.wants/*%i.service

Were replaced with the following lines:

WantedBy=multi-user.target

Signed-off-by: Joshua DeWeese 
---
 ...place-systemd-install-Alias-with-WantedBy.patch | 52 ++
 .../wpa-supplicant/wpa-supplicant_2.6.bb   |  1 +
 2 files changed, 53 insertions(+)
 create mode 100644 
meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/0001-replace-systemd-install-Alias-with-WantedBy.patch

diff --git 
a/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/0001-replace-systemd-install-Alias-with-WantedBy.patch
 
b/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/0001-replace-systemd-install-Alias-with-WantedBy.patch
new file mode 100644
index 00..a476cf040e
--- /dev/null
+++ 
b/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant/0001-replace-systemd-install-Alias-with-WantedBy.patch
@@ -0,0 +1,52 @@
+From 94c401733a5a3d294cc412671166e6adfb409f53 Mon Sep 17 00:00:00 2001
+From: Joshua DeWeese 
+Date: Wed, 30 Jan 2019 16:19:47 -0500
+Subject: [PATCH] replace systemd install Alias with WantedBy
+
+According to the systemd documentation "WantedBy=foo.service in a
+service bar.service is mostly equivalent to
+Alias=foo.service.wants/bar.service in the same file." However,
+this is not really the intended purpose of install Aliases.
+
+Upstream-Status: Submitted [hos...@lists.infradead.org]
+
+Signed-off-by: Joshua DeWeese 
+---
+ wpa_supplicant/systemd/wpa_supplicant-nl80211.service.arg.in | 2 +-
+ wpa_supplicant/systemd/wpa_supplicant-wired.service.arg.in   | 2 +-
+ wpa_supplicant/systemd/wpa_supplicant.service.arg.in | 2 +-
+ 3 files changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/wpa_supplicant/systemd/wpa_supplicant-nl80211.service.arg.in 
b/wpa_supplicant/systemd/wpa_supplicant-nl80211.service.arg.in
+index 03ac507..da69a87 100644
+--- a/wpa_supplicant/systemd/wpa_supplicant-nl80211.service.arg.in
 b/wpa_supplicant/systemd/wpa_supplicant-nl80211.service.arg.in
+@@ -12,4 +12,4 @@ Type=simple
+ ExecStart=@BINDIR@/wpa_supplicant 
-c/etc/wpa_supplicant/wpa_supplicant-nl80211-%I.conf -Dnl80211 -i%I
+ 
+ [Install]
+-Alias=multi-user.target.wants/wpa_supplicant-nl80211@%i.service
++WantedBy=multi-user.target
+diff --git a/wpa_supplicant/systemd/wpa_supplicant-wired.service.arg.in 
b/wpa_supplicant/systemd/wpa_supplicant-wired.service.arg.in
+index c8a744d..ca3054b 100644
+--- a/wpa_supplicant/systemd/wpa_supplicant-wired.service.arg.in
 b/wpa_supplicant/systemd/wpa_supplicant-wired.service.arg.in
+@@ -12,4 +12,4 @@ Type=simple
+ ExecStart=@BINDIR@/wpa_supplicant 
-c/etc/wpa_supplicant/wpa_supplicant-wired-%I.conf -Dwired -i%I
+ 
+ [Install]
+-Alias=multi-user.target.wants/wpa_supplicant-wired@%i.service
++WantedBy=multi-user.target
+diff --git a/wpa_supplicant/systemd/wpa_supplicant.service.arg.in 
b/wpa_supplicant/systemd/wpa_supplicant.service.arg.in
+index 7788b38..55d2b9c 100644
+--- a/wpa_supplicant/systemd/wpa_supplicant.service.arg.in
 b/wpa_supplicant/systemd/wpa_supplicant.service.arg.in
+@@ -12,4 +12,4 @@ Type=simple
+ ExecStart=@BINDIR@/wpa_supplicant 
-c/etc/wpa_supplicant/wpa_supplicant-%I.conf -i%I
+ 
+ [Install]
+-Alias=multi-user.target.wants/wpa_supplicant@%i.service
++WantedBy=multi-user.target
+-- 
+2.7.4
+
diff --git a/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_2.6.bb 
b/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_2.6.bb
index aa4c4c2da0..c92ed4ab93 100644
--- a/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_2.6.bb
+++ b/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_2.6.bb
@@ -33,6 +33,7 @@ SRC_URI = "http://w1.fi/releases/wpa_supplicant-${PV}.tar.gz  
\
file://key-replay-cve-multiple7.patch \
file://key-replay-cve-multiple8.patch \
file://wpa_supplicant-CVE-2018-14526.patch \
+   file://0001-replace-systemd-install-Alias-with-WantedBy.patch \
   "
 SRC_URI[md5sum] = "091569eb4440b7d7f2b4276dbfc03c3c"
 SRC_URI[sha256sum] = 
"b4936d34c4e6cdd44954beba74296d964bc2c9668ecaa5255e499636fe2b1450"
-- 
2.11.0

-- 

Re: [OE-core] [PATCH 1/3] python3: upgrade to 3.7.2

[Alexander Kanavin]

Thanks - the libnewt-python failure I could not reproduce (and we
haven't seen it on Yocto AB - it is a oecore recipe).

For python-gevent, I just sent a patch to oe-devel list.

Alex

On Wed, 6 Feb 2019 at 19:03, Khem Raj  wrote:
>
> Alex
>
> Here are couple of failures I see
>
> https://errors.yoctoproject.org/Errors/Details/221405/
> https://errors.yoctoproject.org/Errors/Details/221458/
>
> On Wed, Feb 6, 2019 at 8:26 AM Alexander Kanavin  
> wrote:
> >
> > I took the same approach as the recent perl upgrade: write recipe from 
> > scratch,
> > taking the pieces from the old recipe only when they were proven to be 
> > necessary.
> >
> > The pgo, manifest and ptest features are all preserved.
> >
> > New features:
> >
> > - native and target recipes are now unified into one recipe
> >
> > - check_build_completeness.py runs right after do_compile() and verifies 
> > that
> > all optional modules have been built (a notorious source of regressions)
> >
> > - a new approach to sysconfig.py and distutils/sysconfig.py returning values
> > appropriate for native or target builds: we copy the configuration file to a
> > separate folder, add that folder to sys.path (through environment variable
> > that differs between native and target builds), and point python to the file
> > through another environment variable.
> >
> > Signed-off-by: Alexander Kanavin 
> > ---
> >  meta/classes/python3-dir.bbclass  |2 +-
> >  meta/classes/python3native.bbclass|2 +
> >  ...ib-termcap-to-linker-flags-to-avoid-.patch |   25 +
> >  ...lib-as-location-for-site-packages-an.patch |  196 +++
> >  ...hell-version-of-python-config-that-w.patch |   35 +
> >  ...-qemu-wrapper-when-gathering-profile.patch |   25 +
> >  ...fig-append-STAGING_LIBDIR-python-sys.patch |   42 +
> >  ...tutils-prefix-is-inside-staging-area.patch |   54 +
> >  .../python3/avoid_warning_about_tkinter.patch |   36 +
> >  .../python-sanity/python3/cgi_py.patch|   32 +
> >  .../python3/check_build_completeness.py   |   17 +
> >  .../python-sanity/python3/create_manifest3.py |  433 ++
> >  .../python-sanity/python3/get_module_deps3.py |  146 ++
> >  .../python-sanity/python3/python-config.patch |   46 +
> >  .../python3/python3-manifest.json | 1228 +
> >  .../python-sanity/python3/run-ptest   |3 +
> >  .../python-sanity/python3_3.7.2.bb|  284 
> >  17 files changed, 2605 insertions(+), 1 deletion(-)
> >  create mode 100644 
> > meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch
> >  create mode 100644 
> > meta/recipes-devtools/python-sanity/python3/0001-Do-not-hardcode-lib-as-location-for-site-packages-an.patch
> >  create mode 100644 
> > meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
> >  create mode 100644 
> > meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch
> >  create mode 100644 
> > meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
> >  create mode 100644 
> > meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch
> >  create mode 100644 
> > meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch
> >  create mode 100644 meta/recipes-devtools/python-sanity/python3/cgi_py.patch
> >  create mode 100755 
> > meta/recipes-devtools/python-sanity/python3/check_build_completeness.py
> >  create mode 100644 
> > meta/recipes-devtools/python-sanity/python3/create_manifest3.py
> >  create mode 100644 
> > meta/recipes-devtools/python-sanity/python3/get_module_deps3.py
> >  create mode 100644 
> > meta/recipes-devtools/python-sanity/python3/python-config.patch
> >  create mode 100644 
> > meta/recipes-devtools/python-sanity/python3/python3-manifest.json
> >  create mode 100644 meta/recipes-devtools/python-sanity/python3/run-ptest
> >  create mode 100644 meta/recipes-devtools/python-sanity/python3_3.7.2.bb
> >
> > diff --git a/meta/classes/python3-dir.bbclass 
> > b/meta/classes/python3-dir.bbclass
> > index 06bb046d9c2..7dd130bad99 100644
> > --- a/meta/classes/python3-dir.bbclass
> > +++ b/meta/classes/python3-dir.bbclass
> > @@ -1,4 +1,4 @@
> > -PYTHON_BASEVERSION = "3.5"
> > +PYTHON_BASEVERSION = "3.7"
> >  PYTHON_ABI = "m"
> >  PYTHON_DIR = "python${PYTHON_BASEVERSION}"
> >  PYTHON_PN = "python3"
> > diff --git a/meta/classes/python3native.bbclass 
> > b/meta/classes/python3native.bbclass
> > index da12a714703..a3acaf61bbd 100644
> > --- a/meta/classes/python3native.bbclass
> > +++ b/meta/classes/python3native.bbclass
> > @@ -9,6 +9,8 @@ DEPENDS_append = " python3-native "
> >  export STAGING_INCDIR
> >  export STAGING_LIBDIR
> >
> > +export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
> > +
> >  # suppress host user's site-packages dirs.
> >  export PYTHONNOUSERSITE = 

Re: [OE-core] Always install initramfs-framework-base in case of linux-yocto & INITRAMFS_IMAGE_BUNDLE=1

[Andrea Adami]

On Wed, Feb 6, 2019 at 4:54 PM Alexey Brodkin
 wrote:
>
> Hi Leon,
>
> > -Original Message-
> > From: Leon Woestenberg 
> > Sent: Friday, February 1, 2019 12:18 AM
> > To: Alexey Brodkin 
> > Cc: Patches and discussions about the oe-core layer 
> > 
> > Subject: Re: [OE-core] Always install initramfs-framework-base in case of 
> > linux-yocto &
> > INITRAMFS_IMAGE_BUNDLE=1
> >
> > On Tue, Jan 29, 2019 at 9:01 PM Alexey Brodkin
> >  wrote:
> > > --->8--
> > > /dev/console is missing or not a character device!
> > > Please ensure your rootfs is properly configured
> > > --->8--
> >
> > I thought /dev/console could also be created by the kernel itself onto
> > a "temporary filesystem for device nodes" (DEVTMPFS).
> > I am not sure, but here is a similar case:
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.busybox.net_pipermail_buildroot_2015-
> > 2DMarch_123309.html=DwIBaQ=DPL6_X_6JkXFx7AXWqB0tg=lqdeeSSEes0GFDDl656eViXO7breS55ytWkhpk5R81I=
> > 9xab_6OSIxhk6sCiqSFEwTUASBIaA_-C_wGiSMfJn3o=SJ9X7QVWrd1gCdyBsL9fdY841I4qG2y47VH3Sw-IRF4=
> >
> > Is your (other) kernel configured with CONFIG_DEVTMPFS=y?
>
> Indeed it is.
>
> The problem is devtmpfs is auto-mounted in prepare_namespace() which
> is not executed in case of initramfs because in case of initramfs we
> expect its "/init" will do everything itself, see [1].

Hi,
I jump in because I was at the same point years ago [1]: the issue
here is basically the linux-yocto specific patch adding the check for
console.
It was simply reverted in our case...

Cheers
Andrea

[1]
http://cgit.openembedded.org/meta-openembedded/commit/meta-initramfs/recipes-kernel?id=7183bacd5c866d433467a750e485e6c01af4aafc


>
> That said enabling CONFIG_DEVTMPFS_MOUNT doesn't help with initramfs too.
> So I don't see any other solution except disabling this check in case
> of enabled CONFIG_BLK_DEV_INITRD, see [2].
>
> [1] 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/early-userspace/README#n140
> [2] 
> https://lists.yoctoproject.org/pipermail/linux-yocto/2019-February/007552.html
>
> -Alexey
>
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] python3: forward-port a few patches from 3.5.6

[Alexander Kanavin]

There were a few other patches where it was difficult to decide if the patch
is still relevant, and how to test that it works correctly; please add those
as-needed by testing the new python.

Signed-off-by: Alexander Kanavin 
---
 ...fig-append-STAGING_LIBDIR-python-sys.patch |   2 +-
 ...asename-to-replace-CC-for-checking-c.patch | 120 +++
 ...runtime-test-to-get-float-byte-order.patch | 199 ++
 ...ssing-libraries-to-Extension-for-mul.patch |  80 +++
 ...tutils-prefix-is-inside-staging-area.patch |   2 +-
 .../python-sanity/python3_3.7.2.bb|   3 +
 6 files changed, 404 insertions(+), 2 deletions(-)
 create mode 100644 
meta/recipes-devtools/python-sanity/python3/0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch
 create mode 100644 
meta/recipes-devtools/python-sanity/python3/0002-Don-t-do-runtime-test-to-get-float-byte-order.patch
 create mode 100644 
meta/recipes-devtools/python-sanity/python3/0003-setup.py-pass-missing-libraries-to-Extension-for-mul.patch

diff --git 
a/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
 
b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
index 56f7f713112..8083345a4e0 100644
--- 
a/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
+++ 
b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
@@ -1,4 +1,4 @@
-From 5b2885fd3eaf05b4397385195872d4ec8240a47c Mon Sep 17 00:00:00 2001
+From 4865615a2bc2b78c739e4c33f536712c7f9af061 Mon Sep 17 00:00:00 2001
 From: Alexander Kanavin 
 Date: Thu, 31 Jan 2019 16:46:30 +0100
 Subject: [PATCH] distutils/sysconfig: append
diff --git 
a/meta/recipes-devtools/python-sanity/python3/0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch
 
b/meta/recipes-devtools/python-sanity/python3/0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch
new file mode 100644
index 000..5735954628e
--- /dev/null
+++ 
b/meta/recipes-devtools/python-sanity/python3/0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch
@@ -0,0 +1,120 @@
+From 2645317fef09afe31b01bb2c1d4fe5b9afdbb11a Mon Sep 17 00:00:00 2001
+From: Changqing Li 
+Date: Mon, 22 Oct 2018 15:19:51 +0800
+Subject: [PATCH] python3: use cc_basename to replace CC for checking compiler
+
+When working path contains "clang"/"gcc"/"icc", it might be part of $CC
+because of the "--sysroot" parameter. That could cause judgement error
+about clang/gcc/icc compilers. e.g.
+When "icc" is containded in working path, below errors are reported when
+compiling python3:
+x86_64-wrs-linux-gcc: error: strict: No such file or directory
+x86_64-wrs-linux-gcc: error: unrecognized command line option '-fp-model'
+
+Here use cc_basename to replace CC for checking compiler to avoid such
+kind of issue.
+
+Upstream-Status: Pending
+
+Signed-off-by: Li Zhou 
+
+patch originally from Li Zhou, I just rework it to new version
+
+Signed-off-by: Changqing Li 
+---
+ configure.ac | 19 ++-
+ 1 file changed, 10 insertions(+), 9 deletions(-)
+
+diff --git a/configure.ac b/configure.ac
+index a7de901..4a3681f 100644
+--- a/configure.ac
 b/configure.ac
+@@ -54,6 +54,7 @@ AC_CONFIG_HEADER(pyconfig.h)
+ AC_CANONICAL_HOST
+ AC_SUBST(build)
+ AC_SUBST(host)
++LT_INIT
+ 
+ # pybuilddir.txt will be created by --generate-posix-vars in the Makefile
+ rm -f pybuilddir.txt
+@@ -695,7 +696,7 @@ AC_MSG_RESULT($with_cxx_main)
+ preset_cxx="$CXX"
+ if test -z "$CXX"
+ then
+-case "$CC" in
++case "$cc_basename" in
+ gcc)AC_PATH_TOOL(CXX, [g++], [g++], [notfound]) ;;
+ cc) AC_PATH_TOOL(CXX, [c++], [c++], [notfound]) ;;
+ clang|*/clang) AC_PATH_TOOL(CXX, [clang++], [clang++], 
[notfound]) ;;
+@@ -979,7 +980,7 @@ rmdir CaseSensitiveTestDir
+ 
+ case $ac_sys_system in
+ hp*|HP*)
+-case $CC in
++case $cc_basename in
+ cc|*/cc) CC="$CC -Ae";;
+ esac;;
+ esac
+@@ -1336,7 +1337,7 @@ else
+ fi],
+ [AC_MSG_RESULT(no)])
+ if test "$Py_LTO" = 'true' ; then
+-  case $CC in
++  case $cc_basename in
+ *clang*)
+   AC_SUBST(LLVM_AR)
+   AC_PATH_TARGET_TOOL(LLVM_AR, llvm-ar, '', ${llvm_path})
+@@ -1426,7 +1427,7 @@ then
+   fi
+ fi
+ LLVM_PROF_ERR=no
+-case $CC in
++case $cc_basename in
+   *clang*)
+ # Any changes made here should be reflected in the GCC+Darwin case below
+ PGO_PROF_GEN_FLAG="-fprofile-instr-generate"
+@@ -1500,7 +1501,7 @@ then
+WRAP="-fwrapv"
+ fi
+ 
+-case $CC in
++case $cc_basename in
+ *clang*)
+ cc_is_clang=1
+ ;;
+@@ -1623,7 +1624,7 @@ yes)
+ 
+ # ICC doesn't recognize the option, but only emits a warning
+ ## XXX does it emit an unused result warning and can it be disabled?
+-case "$CC" in
++case "$cc_basename" in
+ *icc*)

Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

[Peter Kjellerstedt]

> -Original Message-
> From: openembedded-core-boun...@lists.openembedded.org  core-boun...@lists.openembedded.org> On Behalf Of Khem Raj
> Sent: den 7 februari 2019 01:36
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH] glibc-locale: Rewrite do_install using
> install utility instead of cp
> 
> This has been a constant source of trouble for build failures due to
> host-user-contaminated QA errors of sort
> 
> ERROR: QA Issue: glibc-locale: /glibc-binary-localedata-ca-
> es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY is owned by uid
> 3004, which is the same as the user running bitbake. This may be due to
> host contamination [host-user-contaminated]
> 
> So far we have tried to mould cp command into not carrying the build
> user permissions into install area but it is never entirely fixed since
> the issue keeps popping up in various scenes
> 
> This patch replaces use of cp with install utility and specifies 
> install mode for files explcitly
> 
> Signed-off-by: Khem Raj 
> ---
>  meta/recipes-core/glibc/glibc-locale.inc | 44 ++--
>  1 file changed, 25 insertions(+), 19 deletions(-)
> 
> diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
> b/meta/recipes-core/glibc/glibc-locale.inc
> index 6384f9cbf1..9b256a5108 100644
> --- a/meta/recipes-core/glibc/glibc-locale.inc
> +++ b/meta/recipes-core/glibc/glibc-locale.inc
> @@ -72,27 +72,33 @@ FILES_localedef = "${bindir}/localedef"
>  LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
> 
>  do_install () {
> - mkdir -p ${D}${bindir} ${D}${datadir}
> - if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
> - fi
> - if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
> - mkdir -p ${D}${localedir}
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
> - fi
> +install -d ${D}${bindir}
> +find "${LOCALETREESRC}/${bindir}" -maxdepth 1 -type f \
> +-exec install -m 0755 -t "${D}${bindir}" {} \;
> +
> +for d in . $(find "${LOCALETREESRC}/${localedir}" -type d -printf 
> '%P ') ; do
> +install -d "${D}${localedir}/$d"
> +find "${LOCALETREESRC}/${localedir}/$d" -maxdepth 1 -type f \
> +-exec install -m 0644 -t "${D}${localedir}/$d" {} \;
> +done
>   if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
> - mkdir -p ${D}${libdir}
> - if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
> - fi
> - if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
> - fi
> - fi
> - if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
> +for d in . $(find "${LOCALETREESRC}/${libdir}/gconv" -type d 
> -printf '%P ') ; do
> +install -d "${D}${libdir}/gconv/$d"
> +find "${LOCALETREESRC}/${libdir}/gconv/$d" -maxdepth 
> 1 -type f \
> +-exec install -m 0755 -t "${D}${libdir}/gconv/$d" {} 
> \;
> +done
> +for d in . $(find "${LOCALETREESRC}/${datadir}/i18n" -type d 
> -printf '%P ') ; do
> +install -d "${D}${datadir}/i18n/$d"
> +find "${LOCALETREESRC}/${datadir}/i18n/$d" -maxdepth 
> 1 -type f \
> +-exec install -m 0644 -t "${D}${datadir}/i18n/$d" {} 
> \;
> +done
>   fi
> - cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/SUPPORTED 
> ${WORKDIR}
> +for d in . $(find "${LOCALETREESRC}/${datadir}/locale" -type d 
> -printf '%P ') ; do
> +install -d "${D}${datadir}/locale/$d"
> +find "${LOCALETREESRC}/${datadir}/locale/$d" -maxdepth 1 
> -type f \
> +-exec install -m 0644 -t "${D}${datadir}/locale/$d" {} \;
> +done
> + install -m 0644 ${LOCALETREESRC}/SUPPORTED ${WORKDIR}/SUPPORTED
>  }
> 
>  inherit libc-package
> --
> 2.20.1

May I suggest using a help function to simplify the code a lot:

copy_locale_files() {
local dir=$1 mode=$2

[ -e "${LOCALETREESRC}$dir" ] || return 0

for d in . $(find "${LOCALETREESRC}$dir" -type d -printf '%P '); do
install -d ${D}$dir/$d
find "${LOCALETREESRC}$dir/$d" -maxdepth 1 -type f \
 -exec install -m $mode -t "${D}$dir/$d" {} \;
done
}

do_install() {
copy_locale_files ${bindir} 0755

Re: [OE-core] [PATCH V3 1/2] multilib_header_wrapper.h: Use #pragma once

[Alex Kiernan]

On Thu, Feb 7, 2019 at 6:27 AM Khem Raj  wrote:
>
> Please try the patch I posted for openssl10 and report back
> https://patchwork.openembedded.org/patch/158573/
>

LGTM.

-- 
Alex Kiernan
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

[Richard Purdie]

On Wed, 2019-02-06 at 16:35 -0800, Khem Raj wrote:
> This has been a constant source of trouble for build failures due to 
> host-user-contaminated QA
> errors of sort
> 
> ERROR: QA Issue: glibc-locale: 
> /glibc-binary-localedata-ca-es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY
>  is owned by uid 3004, which is the same as the user running bitbake. This 
> may be due to host contamination [host-user-contaminated]
> 
> So far we have tried to mould cp command into not carrying the build
> user permissions into install area but it is never entirely fixed since
> the issue keeps popping up in various scenes
> 
> This patch replaces use of cp with install utility and specifies install
> mode for files explcitly
> 
> Signed-off-by: Khem Raj 
> ---
>  meta/recipes-core/glibc/glibc-locale.inc | 44 ++--
>  1 file changed, 25 insertions(+), 19 deletions(-)
> 
> diff --git a/meta/recipes-core/glibc/glibc-locale.inc 
> b/meta/recipes-core/glibc/glibc-locale.inc
> index 6384f9cbf1..9b256a5108 100644
> --- a/meta/recipes-core/glibc/glibc-locale.inc
> +++ b/meta/recipes-core/glibc/glibc-locale.inc
> @@ -72,27 +72,33 @@ FILES_localedef = "${bindir}/localedef"
>  LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
>  
>  do_install () {
> - mkdir -p ${D}${bindir} ${D}${datadir}
> - if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
> - fi
> - if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
> - mkdir -p ${D}${localedir}
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
> - fi
> +install -d ${D}${bindir}
> +find "${LOCALETREESRC}/${bindir}" -maxdepth 1 -type f \
> +-exec install -m 0755 -t "${D}${bindir}" {} \;
> +
> +for d in . $(find "${LOCALETREESRC}/${localedir}" -type d -printf 
> '%P ') ; do
> +install -d "${D}${localedir}/$d"
> +find "${LOCALETREESRC}/${localedir}/$d" -maxdepth 1 -type f \
> +-exec install -m 0644 -t "${D}${localedir}/$d" {} \;
> +done
>   if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
> - mkdir -p ${D}${libdir}
> - if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
> - fi
> - if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
> - fi
> - fi
> - if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
> - cp -R --no-dereference --preserve=mode,links 
> ${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
> +for d in . $(find "${LOCALETREESRC}/${libdir}/gconv" -type d 
> -printf '%P ') ; do
> +install -d "${D}${libdir}/gconv/$d"
> +find "${LOCALETREESRC}/${libdir}/gconv/$d" -maxdepth 
> 1 -type f \
> +-exec install -m 0755 -t "${D}${libdir}/gconv/$d" {} 
> \;
> +done
> +for d in . $(find "${LOCALETREESRC}/${datadir}/i18n" -type d 
> -printf '%P ') ; do
> +install -d "${D}${datadir}/i18n/$d"
> +find "${LOCALETREESRC}/${datadir}/i18n/$d" -maxdepth 
> 1 -type f \
> +-exec install -m 0644 -t "${D}${datadir}/i18n/$d" {} 
> \;
> +done
>   fi
> - cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/SUPPORTED 
> ${WORKDIR}
> +for d in . $(find "${LOCALETREESRC}/${datadir}/locale" -type d 
> -printf '%P ') ; do
> +install -d "${D}${datadir}/locale/$d"
> +find "${LOCALETREESRC}/${datadir}/locale/$d" -maxdepth 1 
> -type f \
> +-exec install -m 0644 -t "${D}${datadir}/locale/$d" {} \;
> +done
> + install -m 0644 ${LOCALETREESRC}/SUPPORTED ${WORKDIR}/SUPPORTED
>  }
>  
>  inherit libc-package

The trouble is this is a workaround. The cp commands should work and
there is some underlying issue in pseudo causing this. We really need
to figure out what that is since this will likely just mean we see the
problem somewhere else :(

I still suspect some kind of inode number reuse problem which these cp
commands trigger...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core