Re: [OE-core] [RFC PATCH 1/2] cross-localedef-native: Add hardlink resolver from util-linux
I think it’s ok to have it in Glibc/localedef not all distros have this version of util Linux yet and perhaps we should start creating a buildtools tarballs which is a prerequisite for building OE and collect these dependencies in there instead Of sticking them in unusual places On Fri, Aug 16, 2019 at 7:59 PM Jason Wessel wrote: > On 8/16/19 4:50 PM, richard.pur...@linuxfoundation.org wrote: > > On Fri, 2019-08-16 at 14:29 -0700, Khem Raj wrote: > >> On Fri, Aug 16, 2019 at 2:25 PM Richard Purdie > >> wrote: > >>> This was discussed on IRC. We really really want to avoid that as a > >>> dependency if we care about build performance. Its really easy to > >>> just > >>> add that in but the implications are significant :( > >>> > >> A captured version of this utility would also mean that we have to > >> keep this in sync for bug fixes and security fixes etc. which could > >> also be considered as expense, so how much do we lose if util-linux- > >> native is brought early into dependency chain? > > The tool is being used for a specific purpose at build time. Its not > > exposed on target and its use is extremely constrained to a specific > > set of files where its behaviour should be deterministic. > > > > As such, yes there are questions about bug fixes and about security but > > I don't think its going to be a serious burden. We'd need to carry this > > until such times as the host systems we support have hardlink which is > > probably a few years out but will arrive. > > > > The impact of having util-linux-native in the dependency graph at this > > point is significant. I say that as someone who has done a lot of work > > in trimming down the times of our bootstrap process. This is exactly > > the kind of dependency I've removed and avoided. > > > > glibc is special in that its the worst choke point on the dependency > > graph. Nothing to do with target binaries happens until it is ready and > > then gcc-runtime can build. Whilst the locales are split from glibc, > > they are still connected in the task graph for packaging purposes so > > this is important. > > > > Its been pointed out there are other places where util-linux-native has > > crept in, in particular python3-native. My preference would be to > > minimise these dependencies rather than use them as a way to justify > > adding it here too. > > > So what is the consensus here? Can we require the host system to have > hardlink? It is available as a stand alone binary on Ubuntu 14.04 and up > and on Fedora systems, but isn't installed by default, and we'd have to > check the version because the arguments sadly are slightly different. We > could certainly make util-linux a dependency, but that is why this patch > existed, in order to avoid the dependency. > > I did find a python version at > https://raw.githubusercontent.com/akaihola/hardlinkpy/master/hardlink.py. > I made a one line modification to it and it works, but it isn't nearly as > efficient. The proof is below, using the same locale-tree each time. It > is important to look at both the real and user time. The difference gets > more significant when you run it under pseudo, where we expect it to run > 100% of the time. It is roughly 10x slower to do this in python. > > > time recipe-sysroot-native/usr/bin/cross-localedef-hardlink -c -v > locale-tree > Directories: 982 > Objects:7441 > Regular files: 6441 > Comparisons:3331 > Linked: 3109 > Saved: 754675712 > > real0m0.641s > user0m0.120s > sys 0m0.518s > > time python3 hardlink.py -v 0 -c locale-tree > > > Hard linking Statistics: > Directories : 982 > Regular files : 6441 > Comparisons : 31884 > Hardlinked this run : 3109 > Total hardlinks : 3109 > Bytes saved this run : 744089843 (709.619 mebibytes) > Total bytes saved : 744089843 (709.619 mebibytes) > Total run time: 2.124422073364258 seconds > > real0m2.180s > user0m1.220s > sys 0m0.953s > > > Now lets run it under pseudo > > time recipe-sysroot-native/usr/bin/cross-localedef-hardlink -c -v > locale-tree > Directories: 982 > Objects:7441 > Regular files: 6441 > Comparisons:3331 > Linked: 3109 > Saved: 754675712 > > real0m2.505s > user0m0.373s > sys 0m1.286s > > time python3 hardlink.py -v 0 -c locale-tree > > > Hard linking Statistics: > Directories : 982 > Regular files : 6441 > Comparisons : 31884 > Hardlinked this run : 3109 > Total hardlinks : 3109 > Bytes saved this run : 744089843 (709.619 mebibytes) > Total bytes saved : 744089843 (709.619 mebibytes) > Total run time: 12.971824884414673 seconds > > real0m13.046s > user0m3.633s > sys 0m5.192s > > > -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org
Re: [OE-core] [RFC PATCH 1/2] cross-localedef-native: Add hardlink resolver from util-linux
On 8/16/19 4:50 PM, richard.pur...@linuxfoundation.org wrote: On Fri, 2019-08-16 at 14:29 -0700, Khem Raj wrote: On Fri, Aug 16, 2019 at 2:25 PM Richard Purdie wrote: This was discussed on IRC. We really really want to avoid that as a dependency if we care about build performance. Its really easy to just add that in but the implications are significant :( A captured version of this utility would also mean that we have to keep this in sync for bug fixes and security fixes etc. which could also be considered as expense, so how much do we lose if util-linux- native is brought early into dependency chain? The tool is being used for a specific purpose at build time. Its not exposed on target and its use is extremely constrained to a specific set of files where its behaviour should be deterministic. As such, yes there are questions about bug fixes and about security but I don't think its going to be a serious burden. We'd need to carry this until such times as the host systems we support have hardlink which is probably a few years out but will arrive. The impact of having util-linux-native in the dependency graph at this point is significant. I say that as someone who has done a lot of work in trimming down the times of our bootstrap process. This is exactly the kind of dependency I've removed and avoided. glibc is special in that its the worst choke point on the dependency graph. Nothing to do with target binaries happens until it is ready and then gcc-runtime can build. Whilst the locales are split from glibc, they are still connected in the task graph for packaging purposes so this is important. Its been pointed out there are other places where util-linux-native has crept in, in particular python3-native. My preference would be to minimise these dependencies rather than use them as a way to justify adding it here too. So what is the consensus here? Can we require the host system to have hardlink? It is available as a stand alone binary on Ubuntu 14.04 and up and on Fedora systems, but isn't installed by default, and we'd have to check the version because the arguments sadly are slightly different. We could certainly make util-linux a dependency, but that is why this patch existed, in order to avoid the dependency. I did find a python version at https://raw.githubusercontent.com/akaihola/hardlinkpy/master/hardlink.py. I made a one line modification to it and it works, but it isn't nearly as efficient. The proof is below, using the same locale-tree each time. It is important to look at both the real and user time. The difference gets more significant when you run it under pseudo, where we expect it to run 100% of the time. It is roughly 10x slower to do this in python. time recipe-sysroot-native/usr/bin/cross-localedef-hardlink -c -v locale-tree Directories: 982 Objects: 7441 Regular files: 6441 Comparisons: 3331 Linked: 3109 Saved: 754675712 real 0m0.641s user 0m0.120s sys 0m0.518s time python3 hardlink.py -v 0 -c locale-tree Hard linking Statistics: Directories : 982 Regular files : 6441 Comparisons : 31884 Hardlinked this run : 3109 Total hardlinks : 3109 Bytes saved this run : 744089843 (709.619 mebibytes) Total bytes saved : 744089843 (709.619 mebibytes) Total run time : 2.124422073364258 seconds real 0m2.180s user 0m1.220s sys 0m0.953s Now lets run it under pseudo time recipe-sysroot-native/usr/bin/cross-localedef-hardlink -c -v locale-tree Directories: 982 Objects: 7441 Regular files: 6441 Comparisons: 3331 Linked: 3109 Saved: 754675712 real 0m2.505s user 0m0.373s sys 0m1.286s time python3 hardlink.py -v 0 -c locale-tree Hard linking Statistics: Directories : 982 Regular files : 6441 Comparisons : 31884 Hardlinked this run : 3109 Total hardlinks : 3109 Bytes saved this run : 744089843 (709.619 mebibytes) Total bytes saved : 744089843 (709.619 mebibytes) Total run time : 12.971824884414673 seconds real 0m13.046s user 0m3.633s sys 0m5.192s -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [RFC PATCH 1/2] cross-localedef-native: Add hardlink resolver from util-linux
On Fri, 2019-08-16 at 14:29 -0700, Khem Raj wrote: > On Fri, Aug 16, 2019 at 2:25 PM Richard Purdie > wrote: > > This was discussed on IRC. We really really want to avoid that as a > > dependency if we care about build performance. Its really easy to > > just > > add that in but the implications are significant :( > > > > A captured version of this utility would also mean that we have to > keep this in sync for bug fixes and security fixes etc. which could > also be considered as expense, so how much do we lose if util-linux- > native is brought early into dependency chain? The tool is being used for a specific purpose at build time. Its not exposed on target and its use is extremely constrained to a specific set of files where its behaviour should be deterministic. As such, yes there are questions about bug fixes and about security but I don't think its going to be a serious burden. We'd need to carry this until such times as the host systems we support have hardlink which is probably a few years out but will arrive. The impact of having util-linux-native in the dependency graph at this point is significant. I say that as someone who has done a lot of work in trimming down the times of our bootstrap process. This is exactly the kind of dependency I've removed and avoided. glibc is special in that its the worst choke point on the dependency graph. Nothing to do with target binaries happens until it is ready and then gcc-runtime can build. Whilst the locales are split from glibc, they are still connected in the task graph for packaging purposes so this is important. Its been pointed out there are other places where util-linux-native has crept in, in particular python3-native. My preference would be to minimise these dependencies rather than use them as a way to justify adding it here too. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [RFC PATCH 1/2] cross-localedef-native: Add hardlink resolver from util-linux
On 8/16/19 4:29 PM, Khem Raj wrote: > On Fri, Aug 16, 2019 at 2:25 PM Richard Purdie > wrote: >> >> On Fri, 2019-08-16 at 14:22 -0700, Khem Raj wrote: >>> On Fri, Aug 16, 2019 at 2:06 PM Jason Wessel < >>> jason.wes...@windriver.com> wrote: The hard link resolver that is built into localedef cannot be run in parallel. It will search sibling directories (which are be processed in parallel) and perform a creation of a .tmp file and remove the original and move the .tmp file in. The problem is that if a probe occurs a hard link can be requested to the file that is being removed. This will lead to a stray copy or potentially, on a loaded system cause race condition which pseudo cannot deal with, where it is left with a hard link request to a file that no longer exists. In this situation psuedo will inherit the permissions of what ever the target inode had to offer. In short, there are two problems: 1) You will be left with stray copies when using the hard link resolution that is built in while running in parallel with localedef. 2) When running under pseudo the possibility exists for uid/gid leakage when the source file is removed before the hard link can be completed. The solution is to call localedef with --no-hard-links and separately process the hardlinks at a later point. To do this requires the inclusion of the hardlink utility found in modern versions of util-linux. Most host systems do not have this, so it will be included with the cross-localedef binary. >>> >>> can we depend on util-linux-native for this ? >> >> This was discussed on IRC. We really really want to avoid that as a >> dependency if we care about build performance. Its really easy to just >> add that in but the implications are significant :( >> > > A captured version of this utility would also mean that we have to > keep this in sync > for bug fixes and security fixes etc. which could also be considered > as expense, so > how much do we lose if util-linux-native is brought early into dependency > chain? security should not be a significant concern here as the utility is only local (and used by) the localdef build (consider it an internal to the recipe) tool. So I'm not concerned about that. As for the bug fixes, this utility is quite stable and the last changes were simply to move it from a standalone utility into util-linux. So again, I find it unlikely but again since it's an internal tool unless it impacted the cross localedef functionality it's unlikely to be a problem. Longer term, as hardlink becomes standard on distributions then the need for the local tool should go away, and it can be removed at that time. But currently it's only common on Red Hat lineage machines, while Debian and Ubuntu either have their own versions, or it's "optional" and not part of the regular distribution (even though it comes from util-linux). --Mark >> Cheers, >> >> Richard >> >> >> -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [RFC PATCH 1/2] cross-localedef-native: Add hardlink resolver from util-linux
On Fri, Aug 16, 2019 at 2:25 PM Richard Purdie wrote: > > On Fri, 2019-08-16 at 14:22 -0700, Khem Raj wrote: > > On Fri, Aug 16, 2019 at 2:06 PM Jason Wessel < > > jason.wes...@windriver.com> wrote: > > > The hard link resolver that is built into localedef cannot be run > > > in > > > parallel. It will search sibling directories (which are be > > > processed > > > in parallel) and perform a creation of a .tmp file and remove the > > > original and move the .tmp file in. The problem is that if a probe > > > occurs a hard link can be requested to the file that is being > > > removed. > > > This will lead to a stray copy or potentially, on a loaded system > > > cause race condition which pseudo cannot deal with, where it is > > > left > > > with a hard link request to a file that no longer exists. In this > > > situation psuedo will inherit the permissions of what ever the > > > target > > > inode had to offer. > > > > > > In short, there are two problems: > > > > > > 1) You will be left with stray copies when using the hard link > > > resolution that is built in while running in parallel with > > > localedef. > > > > > > 2) When running under pseudo the possibility exists for uid/gid > > > leakage when the source file is removed before the hard link > > > can > > > be completed. > > > > > > The solution is to call localedef with --no-hard-links and > > > separately > > > process the hardlinks at a later point. To do this requires the > > > inclusion of the hardlink utility found in modern versions of > > > util-linux. Most host systems do not have this, so it will be > > > included with the cross-localedef binary. > > > > > > > can we depend on util-linux-native for this ? > > This was discussed on IRC. We really really want to avoid that as a > dependency if we care about build performance. Its really easy to just > add that in but the implications are significant :( > A captured version of this utility would also mean that we have to keep this in sync for bug fixes and security fixes etc. which could also be considered as expense, so how much do we lose if util-linux-native is brought early into dependency chain? > Cheers, > > Richard > > > -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [RFC PATCH 1/2] cross-localedef-native: Add hardlink resolver from util-linux
On Fri, 2019-08-16 at 14:22 -0700, Khem Raj wrote: > On Fri, Aug 16, 2019 at 2:06 PM Jason Wessel < > jason.wes...@windriver.com> wrote: > > The hard link resolver that is built into localedef cannot be run > > in > > parallel. It will search sibling directories (which are be > > processed > > in parallel) and perform a creation of a .tmp file and remove the > > original and move the .tmp file in. The problem is that if a probe > > occurs a hard link can be requested to the file that is being > > removed. > > This will lead to a stray copy or potentially, on a loaded system > > cause race condition which pseudo cannot deal with, where it is > > left > > with a hard link request to a file that no longer exists. In this > > situation psuedo will inherit the permissions of what ever the > > target > > inode had to offer. > > > > In short, there are two problems: > > > > 1) You will be left with stray copies when using the hard link > > resolution that is built in while running in parallel with > > localedef. > > > > 2) When running under pseudo the possibility exists for uid/gid > > leakage when the source file is removed before the hard link > > can > > be completed. > > > > The solution is to call localedef with --no-hard-links and > > separately > > process the hardlinks at a later point. To do this requires the > > inclusion of the hardlink utility found in modern versions of > > util-linux. Most host systems do not have this, so it will be > > included with the cross-localedef binary. > > > > can we depend on util-linux-native for this ? This was discussed on IRC. We really really want to avoid that as a dependency if we care about build performance. Its really easy to just add that in but the implications are significant :( Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [RFC PATCH 1/2] cross-localedef-native: Add hardlink resolver from util-linux
On Fri, Aug 16, 2019 at 2:06 PM Jason Wessel wrote:
>
> The hard link resolver that is built into localedef cannot be run in
> parallel. It will search sibling directories (which are be processed
> in parallel) and perform a creation of a .tmp file and remove the
> original and move the .tmp file in. The problem is that if a probe
> occurs a hard link can be requested to the file that is being removed.
> This will lead to a stray copy or potentially, on a loaded system
> cause race condition which pseudo cannot deal with, where it is left
> with a hard link request to a file that no longer exists. In this
> situation psuedo will inherit the permissions of what ever the target
> inode had to offer.
>
> In short, there are two problems:
>
> 1) You will be left with stray copies when using the hard link
> resolution that is built in while running in parallel with
> localedef.
>
> 2) When running under pseudo the possibility exists for uid/gid
> leakage when the source file is removed before the hard link can
> be completed.
>
> The solution is to call localedef with --no-hard-links and separately
> process the hardlinks at a later point. To do this requires the
> inclusion of the hardlink utility found in modern versions of
> util-linux. Most host systems do not have this, so it will be
> included with the cross-localedef binary.
>
can we depend on util-linux-native for this ?
> There are two patches here. The first imports the raw version of
> hardlink.c and a couple of header files directly from util-linux. The
> second patch applies the fix-ups to make it compile, along with a
> change to recipe to package the new binary.
>
> [YOCTO #11299]
> [YOCTO #12434]
>
> Signed-off-by: Jason Wessel
> ---
> .../glibc/cross-localedef-native_2.30.bb |3 +
> .../glibc/add-cross-localedef-hardlink.patch | 1115 +
> ...le-separate-from-util-linux-hardlink.patch | 226
> 3 files changed, 1344 insertions(+)
> create mode 100644
> meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch
> create mode 100644
> meta/recipes-core/glibc/glibc/allow-compile-separate-from-util-linux-hardlink.patch
>
> diff --git a/meta/recipes-core/glibc/cross-localedef-native_2.30.bb
> b/meta/recipes-core/glibc/cross-localedef-native_2.30.bb
> index e4923c73d5..29cd43e89a 100644
> --- a/meta/recipes-core/glibc/cross-localedef-native_2.30.bb
> +++ b/meta/recipes-core/glibc/cross-localedef-native_2.30.bb
> @@ -30,6 +30,8 @@ SRC_URI = "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \
>
> file://0022-eglibc-Forward-port-cross-locale-generation-support.patch \
> file://0023-Define-DUMMY_LOCALE_T-if-not-defined.patch \
>
> file://0024-localedef-add-to-archive-uses-a-hard-coded-locale-pa.patch \
> + file://add-cross-localedef-hardlink.patch \
> + file://allow-compile-separate-from-util-linux-hardlink.patch \
> "
> # Makes for a rather long rev (22 characters), but...
> #
> @@ -43,4 +45,5 @@ CFLAGS += "-fgnu89-inline -std=gnu99 -DIS_IN\(x\)='0'"
> do_install() {
> install -d ${D}${bindir}
> install -m 0755 ${B}/localedef ${D}${bindir}/cross-localedef
> + install -m 0755 ${B}/cross-localedef-hardlink
> ${D}${bindir}/cross-localedef-hardlink
> }
> diff --git a/meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch
> b/meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch
> new file mode 100644
> index 00..8471121949
> --- /dev/null
> +++ b/meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch
> @@ -0,0 +1,1115 @@
> +---
> + locale/programs/c.h| 407 ++
> + locale/programs/cross-localedef-hardlink.c | 528
> +
> + locale/programs/xalloc.h | 129 +++
> + localedef/Makefile.in |8
> + 4 files changed, 1071 insertions(+), 1 deletion(-)
> +
> +--- /dev/null
> b/locale/programs/cross-localedef-hardlink.c
> +@@ -0,0 +1,528 @@
> ++/*
> ++ * hardlink - consolidate duplicate files via hardlinks
> ++ *
> ++ * Copyright (C) 2018 Red Hat, Inc. All rights reserved.
> ++ * Written by Jakub Jelinek
> ++ *
> ++ * Copyright (C) 2019 Karel Zak
> ++ *
> ++ * This program is free software; you can redistribute it and/or modify
> ++ * it under the terms of the GNU General Public License as published by
> ++ * the Free Software Foundation; either version 2 of the License, or
> ++ * (at your option) any later version.
> ++ *
> ++ * This program is distributed in the hope that it would be useful,
> ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
> ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> ++ * GNU General Public License for more details.
> ++ *
> ++ * You should have received a copy of the GNU General Public License along
> ++ * with this program; if not, write to the Free Software Foundation, Inc.,
> ++ *
[OE-core] [RFC PATCH 2/2] libc-package.bbclass: Split locale hard link processing into two parts
The locale-processing in cross-localedef was proven to be unsafe to
run in parallel due to the way it tried to make hard links to files
that could disappear before the link operation was completed.
To avoid corruption of the pseudo database, and create a
deterministically generated link tree, the operation of the locale
generation and the hard link resolution have been split into two
separate parts. A side effect of this change is that the do_package()
rule for glibc-locale will be slightly smaller because some of the
hard links were missed in the past, particularly if you had a lot of
cpus free to perform the work.
Before the patch:
% du -sk locale-tree
312524 locale-tree
With the patch:
% du -sk locale-tree
290772 locale-tree
A number of comparisons were performed such as diffing the output,
cross checking the link references using tar's listing output, and
comparing against the results with out a parallel build to ensure the
locale-tree is correct in its final form.
[YOCTO #11299]
[YOCTO #12434]
Signed-off-by: Jason Wessel
---
meta/classes/libc-package.bbclass | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/meta/classes/libc-package.bbclass
b/meta/classes/libc-package.bbclass
index 8859dad566..a66e540884 100644
--- a/meta/classes/libc-package.bbclass
+++ b/meta/classes/libc-package.bbclass
@@ -82,6 +82,9 @@ do_collect_bins_from_locale_tree() {
parent=$(dirname ${localedir})
mkdir -p ${PKGD}/$parent
tar -cf - -C $treedir/$parent -p $(basename ${localedir}) | tar -xf -
-C ${PKGD}$parent
+
+ # Finalize tree by chaning all duplicate files into hard links
+ cross-localedef-hardlink -c -v ${WORKDIR}/locale-tree
}
inherit qemu
@@ -265,7 +268,7 @@ python package_do_split_gconvs () {
bb.error("locale_arch_options not found for target_arch=" +
target_arch)
bb.fatal("unknown arch:" + target_arch + " for
locale_arch_options")
-localedef_opts += " --force --no-archive --prefix=%s \
+localedef_opts += " --force --no-hard-links --no-archive
--prefix=%s \
--inputfile=%s/%s/i18n/locales/%s --charmap=%s %s/%s" \
% (treedir, treedir, datadir, locale, encoding, outputpath,
name)
@@ -273,7 +276,7 @@ python package_do_split_gconvs () {
(path, i18npath, gconvpath, localedef_opts)
else: # earlier slower qemu way
qemu = qemu_target_binary(d)
-localedef_opts = "--force --no-archive --prefix=%s \
+localedef_opts = "--force --no-hard-links --no-archive --prefix=%s
\
--inputfile=%s/i18n/locales/%s --charmap=%s %s" \
% (treedir, datadir, locale, encoding, name)
--
2.21.0
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [RFC PATCH 1/2] cross-localedef-native: Add hardlink resolver from util-linux
The hard link resolver that is built into localedef cannot be run in
parallel. It will search sibling directories (which are be processed
in parallel) and perform a creation of a .tmp file and remove the
original and move the .tmp file in. The problem is that if a probe
occurs a hard link can be requested to the file that is being removed.
This will lead to a stray copy or potentially, on a loaded system
cause race condition which pseudo cannot deal with, where it is left
with a hard link request to a file that no longer exists. In this
situation psuedo will inherit the permissions of what ever the target
inode had to offer.
In short, there are two problems:
1) You will be left with stray copies when using the hard link
resolution that is built in while running in parallel with
localedef.
2) When running under pseudo the possibility exists for uid/gid
leakage when the source file is removed before the hard link can
be completed.
The solution is to call localedef with --no-hard-links and separately
process the hardlinks at a later point. To do this requires the
inclusion of the hardlink utility found in modern versions of
util-linux. Most host systems do not have this, so it will be
included with the cross-localedef binary.
There are two patches here. The first imports the raw version of
hardlink.c and a couple of header files directly from util-linux. The
second patch applies the fix-ups to make it compile, along with a
change to recipe to package the new binary.
[YOCTO #11299]
[YOCTO #12434]
Signed-off-by: Jason Wessel
---
.../glibc/cross-localedef-native_2.30.bb |3 +
.../glibc/add-cross-localedef-hardlink.patch | 1115 +
...le-separate-from-util-linux-hardlink.patch | 226
3 files changed, 1344 insertions(+)
create mode 100644
meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch
create mode 100644
meta/recipes-core/glibc/glibc/allow-compile-separate-from-util-linux-hardlink.patch
diff --git a/meta/recipes-core/glibc/cross-localedef-native_2.30.bb
b/meta/recipes-core/glibc/cross-localedef-native_2.30.bb
index e4923c73d5..29cd43e89a 100644
--- a/meta/recipes-core/glibc/cross-localedef-native_2.30.bb
+++ b/meta/recipes-core/glibc/cross-localedef-native_2.30.bb
@@ -30,6 +30,8 @@ SRC_URI = "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \
file://0022-eglibc-Forward-port-cross-locale-generation-support.patch \
file://0023-Define-DUMMY_LOCALE_T-if-not-defined.patch \
file://0024-localedef-add-to-archive-uses-a-hard-coded-locale-pa.patch \
+ file://add-cross-localedef-hardlink.patch \
+ file://allow-compile-separate-from-util-linux-hardlink.patch \
"
# Makes for a rather long rev (22 characters), but...
#
@@ -43,4 +45,5 @@ CFLAGS += "-fgnu89-inline -std=gnu99 -DIS_IN\(x\)='0'"
do_install() {
install -d ${D}${bindir}
install -m 0755 ${B}/localedef ${D}${bindir}/cross-localedef
+ install -m 0755 ${B}/cross-localedef-hardlink
${D}${bindir}/cross-localedef-hardlink
}
diff --git a/meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch
b/meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch
new file mode 100644
index 00..8471121949
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc/add-cross-localedef-hardlink.patch
@@ -0,0 +1,1115 @@
+---
+ locale/programs/c.h| 407 ++
+ locale/programs/cross-localedef-hardlink.c | 528
+
+ locale/programs/xalloc.h | 129 +++
+ localedef/Makefile.in |8
+ 4 files changed, 1071 insertions(+), 1 deletion(-)
+
+--- /dev/null
b/locale/programs/cross-localedef-hardlink.c
+@@ -0,0 +1,528 @@
++/*
++ * hardlink - consolidate duplicate files via hardlinks
++ *
++ * Copyright (C) 2018 Red Hat, Inc. All rights reserved.
++ * Written by Jakub Jelinek
++ *
++ * Copyright (C) 2019 Karel Zak
++ *
++ * This program is free software; you can redistribute it and/or modify
++ * it under the terms of the GNU General Public License as published by
++ * the Free Software Foundation; either version 2 of the License, or
++ * (at your option) any later version.
++ *
++ * This program is distributed in the hope that it would be useful,
++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
++ * GNU General Public License for more details.
++ *
++ * You should have received a copy of the GNU General Public License along
++ * with this program; if not, write to the Free Software Foundation, Inc.,
++ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
++ */
++#include
++#include
++#include
++#include
++#include
++#include
++#include
++#include
++#include
++#include
++#include
++#ifdef HAVE_PCRE
++# define PCRE2_CODE_UNIT_WIDTH 8
++# include
++#endif
++
++#include "c.h"
++#include
Re: [OE-core] [PATCH 6/6] xz: Remove GPLv3 license checksum
On Fri, Aug 16, 2019 at 12:46 PM Wes Lindauer wrote:
>
> Although xz has some files that are GPLv3 licensed, none of them get
> packaged up, and therefore none of it ends up in the final rootfs. Since
> there is no GPLv3 code in the final image, we don't want to include it
> when we collect licenses, as that would give the incorrect impression
> that the image contains GPLv3 code.
We will be distributing this in src packages though. Maybe these files
should be deleted before the build even starts.
>
> Also fixes the endline for getopt.c which had a couple of non-license
> text lines at the bottom.
> ---
> meta/recipes-extended/xz/xz_5.2.4.bb | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/meta/recipes-extended/xz/xz_5.2.4.bb
> b/meta/recipes-extended/xz/xz_5.2.4.bb
> index 791746e1cf..e6725e5f12 100644
> --- a/meta/recipes-extended/xz/xz_5.2.4.bb
> +++ b/meta/recipes-extended/xz/xz_5.2.4.bb
> @@ -17,9 +17,8 @@ LICENSE_liblzma = "PD"
>
> LIC_FILES_CHKSUM = "file://COPYING;md5=97d554a32881fee0aa283d96e47cb24a \
>
> file://COPYING.GPLv2;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
> -
> file://COPYING.GPLv3;md5=d32239bcb673463ab874e80d47fae504 \
>
> file://COPYING.LGPLv2.1;md5=4fbd65380cdd255951079008b364516c \
> -
> file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \
> +
> file://lib/getopt.c;endline=21;md5=32575ca1a42c5fa04aac6f08566ddf48 \
> "
>
> SRC_URI = "http://tukaani.org/xz/xz-${PV}.tar.gz;
> --
> 2.14.5
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] ✗ patchtest: failure for "iw: Fix license field to BSD-2..." and 5 more
== Series Details == Series: "iw: Fix license field to BSD-2..." and 5 more Revision: 1 URL : https://patchwork.openembedded.org/series/19323/ State : failure == Summary == Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures: * Patch[1/6] iw: Fix license field to BSD-2-Clause Issue Patch is missing Signed-off-by [test_signed_off_by_presence] Suggested fixSign off the patch (either manually or with "git commit --amend -s") If you believe any of these test results are incorrect, please reply to the mailing list (openembedded-core@lists.openembedded.org) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...). --- Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 6/6] xz: Remove GPLv3 license checksum
Although xz has some files that are GPLv3 licensed, none of them get
packaged up, and therefore none of it ends up in the final rootfs. Since
there is no GPLv3 code in the final image, we don't want to include it
when we collect licenses, as that would give the incorrect impression
that the image contains GPLv3 code.
Also fixes the endline for getopt.c which had a couple of non-license
text lines at the bottom.
---
meta/recipes-extended/xz/xz_5.2.4.bb | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/meta/recipes-extended/xz/xz_5.2.4.bb
b/meta/recipes-extended/xz/xz_5.2.4.bb
index 791746e1cf..e6725e5f12 100644
--- a/meta/recipes-extended/xz/xz_5.2.4.bb
+++ b/meta/recipes-extended/xz/xz_5.2.4.bb
@@ -17,9 +17,8 @@ LICENSE_liblzma = "PD"
LIC_FILES_CHKSUM = "file://COPYING;md5=97d554a32881fee0aa283d96e47cb24a \
file://COPYING.GPLv2;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
-file://COPYING.GPLv3;md5=d32239bcb673463ab874e80d47fae504 \
file://COPYING.LGPLv2.1;md5=4fbd65380cdd255951079008b364516c \
-
file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \
+
file://lib/getopt.c;endline=21;md5=32575ca1a42c5fa04aac6f08566ddf48 \
"
SRC_URI = "http://tukaani.org/xz/xz-${PV}.tar.gz;
--
2.14.5
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 3/6] shadow: Fix BSD license file checksum
BSD license files must include the copyright notice. --- meta/recipes-extended/shadow/shadow.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-extended/shadow/shadow.inc b/meta/recipes-extended/shadow/shadow.inc index 7f82d20826..6d9e300739 100644 --- a/meta/recipes-extended/shadow/shadow.inc +++ b/meta/recipes-extended/shadow/shadow.inc @@ -4,7 +4,7 @@ BUGTRACKER = "https://alioth.debian.org/tracker/?group_id=30580; SECTION = "base/utils" LICENSE = "BSD | Artistic-1.0" LIC_FILES_CHKSUM = "file://COPYING;md5=ed80ff1c2b40843cf5768e5229cf16e5 \ - file://src/passwd.c;beginline=8;endline=30;md5=d83888ea14ae61951982d77125947661" + file://src/passwd.c;beginline=2;endline=30;md5=5720ff729a6ff39ecc9f64555d75f4af" DEPENDS = "virtual/crypt" -- 2.14.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 5/6] libunwind: Fix MIT license file checksum
MIT license files must include the copyright notice. --- meta/recipes-support/libunwind/libunwind.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-support/libunwind/libunwind.inc b/meta/recipes-support/libunwind/libunwind.inc index 36851d07ed..5726589661 100644 --- a/meta/recipes-support/libunwind/libunwind.inc +++ b/meta/recipes-support/libunwind/libunwind.inc @@ -2,7 +2,7 @@ SUMMARY = "Library for obtaining the call-chain of a program" DESCRIPTION = "a portable and efficient C programming interface (API) to determine the call-chain of a program" HOMEPAGE = "http://www.nongnu.org/libunwind; LICENSE = "MIT" -LIC_FILES_CHKSUM = "file://COPYING;beginline=3;md5=3fced11d6df719b47505837a51c16ae5" +LIC_FILES_CHKSUM = "file://COPYING;md5=2d80c8ed4062b8339b715f90fa68cc9f" DEPENDS += "libatomic-ops" inherit autotools multilib_header -- 2.14.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 4/6] sudo: Fix BSD license file checksum
BSD license files must include the copyright notice. --- meta/recipes-extended/sudo/sudo.inc | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/meta/recipes-extended/sudo/sudo.inc b/meta/recipes-extended/sudo/sudo.inc index 90f2039bb8..15075bcefd 100644 --- a/meta/recipes-extended/sudo/sudo.inc +++ b/meta/recipes-extended/sudo/sudo.inc @@ -6,12 +6,12 @@ SECTION = "admin" LICENSE = "ISC & BSD & Zlib" LIC_FILES_CHKSUM = "file://doc/LICENSE;md5=6c76b73603ac7763ab0516ebfbe67b42 \ file://plugins/sudoers/redblack.c;beginline=1;endline=46;md5=4a162fc04b86b03f5632180fe6076cda \ - file://lib/util/reallocarray.c;beginline=3;endline=16;md5=85b0905b795d4d58bf2e00635649eec6 \ + file://lib/util/reallocarray.c;beginline=3;endline=15;md5=b47f1f85a12f05a0744cd8b1b6f41a0d \ file://lib/util/fnmatch.c;beginline=3;endline=27;md5=67f83ee9bd456557397082f8f1be0efd \ - file://lib/util/getcwd.c;beginline=5;endline=27;md5=449af4cc57fc7d46f42090608ba3e681 \ - file://lib/util/glob.c;beginline=6;endline=31;md5=5872733146b9eb0deb79e1f664815b85 \ - file://lib/util/snprintf.c;beginline=6;endline=34;md5=c82c1b3a5c32e08545c9ec5d71e41e50 \ - file://include/sudo_queue.h;beginline=5;endline=27;md5=449af4cc57fc7d46f42090608ba3e681 \ + file://lib/util/getcwd.c;beginline=2;endline=27;md5=09068a19b4f6b6f0a0958655bfe98b63 \ + file://lib/util/glob.c;beginline=2;endline=31;md5=1f2f771c35fb0658d567a7824007e56d \ + file://lib/util/snprintf.c;beginline=3;endline=33;md5=63e48e1b992bce749a19dd9b2256e9a0 \ + file://include/sudo_queue.h;beginline=2;endline=27;md5=082b138b72ba3e568a13a25c3bf254dc \ file://lib/util/inet_pton.c;beginline=3;endline=17;md5=3970ab0518ab79cbd0bafb697f10b33a \ file://lib/util/arc4random.c;beginline=3;endline=20;md5=15bdc89c1b003fa4d7353e6296ebfd68 \ file://lib/util/arc4random_uniform.c;beginline=3;endline=17;md5=31e630ac814d692fd0ab7a942659b46f \ -- 2.14.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/6] openssh: Update LICENSE field with missing values
The LICENSE file states that some code is licensed under BSD, some under ISC, and some under MIT. The LICENSE field should reflect all of these. --- meta/recipes-connectivity/openssh/openssh_8.0p1.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-connectivity/openssh/openssh_8.0p1.bb b/meta/recipes-connectivity/openssh/openssh_8.0p1.bb index 35cdf71eb3..01eaecd4ec 100644 --- a/meta/recipes-connectivity/openssh/openssh_8.0p1.bb +++ b/meta/recipes-connectivity/openssh/openssh_8.0p1.bb @@ -5,7 +5,7 @@ Ssh (Secure Shell) is a program for logging into a remote machine \ and for executing commands on a remote machine." HOMEPAGE = "http://www.openssh.com/; SECTION = "console/network" -LICENSE = "BSD" +LICENSE = "BSD & ISC & MIT" LIC_FILES_CHKSUM = "file://LICENCE;md5=429658c6612f3a9b1293782366ab29d8" DEPENDS = "zlib openssl virtual/crypt" -- 2.14.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/6] iw: Fix license field to BSD-2-Clause
Using just "BSD" license implies BSD-3-Clause and this recipe appears to be closer to a BSD-2-Clause. --- meta/recipes-connectivity/iw/iw_5.0.1.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-connectivity/iw/iw_5.0.1.bb b/meta/recipes-connectivity/iw/iw_5.0.1.bb index e127963513..87b21e668d 100644 --- a/meta/recipes-connectivity/iw/iw_5.0.1.bb +++ b/meta/recipes-connectivity/iw/iw_5.0.1.bb @@ -4,7 +4,7 @@ wireless devices. It supports almost all new drivers that have been added \ to the kernel recently. " HOMEPAGE = "http://wireless.kernel.org/en/users/Documentation/iw; SECTION = "base" -LICENSE = "BSD" +LICENSE = "BSD-2-Clause" LIC_FILES_CHKSUM = "file://COPYING;md5=878618a5c4af25e9b93ef0be1a93f774" DEPENDS = "libnl" -- 2.14.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] ✗ patchtest: failure for "gcc: Search in OE specific tar..." and 1 more
== Series Details == Series: "gcc: Search in OE specific tar..." and 1 more Revision: 1 URL : https://patchwork.openembedded.org/series/19322/ State : failure == Summary == Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures: * Issue Series does not apply on top of target branch [test_series_merge_on_head] Suggested fixRebase your series on top of targeted branch Targeted branch master (currently at 6b36db8365) If you believe any of these test results are incorrect, please reply to the mailing list (openembedded-core@lists.openembedded.org) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...). --- Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [warrior][PATCH] bitbake.conf: add git-lfs to HOSTTOOLS_NONFATAL
On Thu, Aug 15, 2019 at 8:45 PM akuster808 wrote: > On 8/15/19 2:31 PM, Andre McCurdy wrote: > > On Thu, Aug 15, 2019 at 2:09 PM Richard Purdie > > wrote: > >> On Thu, 2019-08-15 at 09:23 +0200, Martin Jansa wrote: > >>> NAK > >>> > >>> Yes, the first part was merged in warrior and is correct. > >>> > >>> But this second part isn't good (you don't want git-lfs to sometimes > >>> work and sometimes fail) and that's why it was rejected for master > >>> and _shouldn't_ be merged to warrior. If you have recipes which need > >>> git-lfs, then add it to normal HOSTTOOLS in your builds to make sure > >>> it's always present when needed. > >> I don't like the patch but it did merge to master. > >> > >> Building a git-lfs-native is a nightmare due to all its dependencies (I > >> think its go based?) and there wasn't really any other way to sort it. > > Upstream provides generic binaries for at least x86, x86-64 and ARM64: > > > > https://github.com/git-lfs/git-lfs/releases > > > > https://github.com/git-lfs/git-lfs/releases/download/v2.8.0/git-lfs-linux-386-v2.8.0.tar.gz > > > > https://github.com/git-lfs/git-lfs/releases/download/v2.8.0/git-lfs-linux-amd64-v2.8.0.tar.gz > > > > https://github.com/git-lfs/git-lfs/releases/download/v2.8.0/git-lfs-linux-arm64-v2.8.0.tar.gz > > > > Would just downloading and installing the official upstream binaries be so > > bad? > > making a big change like this seems to be a no-go for a stable branch. Right. This probably isn't suitable for the stable branch. It would obviously be implemented first in master anyway. Point was just that if we made use of the upstream binaries (which are completely distro independent as far as I can tell?) then a git-lfs-native recipe would transform from "a nightmare" to almost trivial. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively
Thanks Martin to fix a bunch of QA issues in meta-openembedded
surfaced after this change in core, there still are some left if
someone wants to take a look at it
https://errors.yoctoproject.org/Errors/Build/87032/
On Tue, Aug 13, 2019 at 11:24 AM Alexander Kanavin
wrote:
>
> Recursive RDEPENDS resolution requires that all of the dependent
> recipes' packaging has completed. There is no mechanism to ensure that
> and therefore races were observed.
>
> This change effectively requires recipes to list their runtime file
> dependencies explicitly rather than have them pulled indirectly.
> This may require a bit of fixing in layers, but should result
> in a better definition of runtime file dependencies.
>
> Signed-off-by: Alexander Kanavin
> ---
> meta/classes/insane.bbclass | 18 --
> 1 file changed, 18 deletions(-)
>
> diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
> index 35c4fdb4913..9b886d13805 100644
> --- a/meta/classes/insane.bbclass
> +++ b/meta/classes/insane.bbclass
> @@ -722,25 +722,7 @@ def package_qa_check_rdepends(pkg, pkgdest, skip,
> taskdeps, packages, d):
> filerdepends[subkey] = key[13:]
>
> if filerdepends:
> -next = rdepends
> done = rdepends[:]
> -# Find all the rdepends on the dependency chain
> -while next:
> -new = []
> -for rdep in next:
> -rdep_data = oe.packagedata.read_subpkgdata(rdep, d)
> -sub_rdeps = rdep_data.get("RDEPENDS_" + rdep)
> -if not sub_rdeps:
> -continue
> -for sub_rdep in bb.utils.explode_deps(sub_rdeps):
> -if sub_rdep in done:
> -continue
> -if oe.packagedata.has_subpkgdata(sub_rdep, d):
> -# It's a new rdep
> -done.append(sub_rdep)
> -new.append(sub_rdep)
> -next = new
> -
> # Add the rprovides of itself
> if pkg not in done:
> done.insert(0, pkg)
> --
> 2.17.1
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH V3 2/2] libffi: Upgrade to 3.3-rc0
libffi 3.1 release has been a bit aged and new architectures, compilers
have since been come on stage to compile it, we have been carrying
patches, but its better to use the latest 3.3 rc0 which has lot of these
issues handled and is in good shape.
Use 3.2.1+3.3-rc0 for PV to keep room for upgrade path without PE bump
fix the multilib header conflict
file /usr/include/ffi.h from install of libffi-dev-3.2.1+3.3+rc0-r0.core2_32
conflicts with file from package lib64-libffi-dev-3.2.1+3.3+rc0-r0.x86_64
Signed-off-by: Khem Raj
---
v2: Set PV = 3.2.1+3.3-rc0
v3: fix ffi.h multilib header conflict
.../libffi/0001-New-RISC-V-port-281.patch | 827 --
.../0001-libffi-Support-musl-x32-build.patch | 30 -
...e-compiler-internal-define-for-linux.patch | 32 -
...-mips-fix-MIPS-softfloat-build-issue.patch | 177
.../libffi/libffi/not-win32.patch | 10 +-
...ibffi_3.2.1.bb => libffi_3.2.1+3.3-rc0.bb} | 19 +-
6 files changed, 14 insertions(+), 1081 deletions(-)
delete mode 100644
meta/recipes-support/libffi/libffi/0001-New-RISC-V-port-281.patch
delete mode 100644
meta/recipes-support/libffi/libffi/0001-libffi-Support-musl-x32-build.patch
delete mode 100644
meta/recipes-support/libffi/libffi/0001-mips-Use-compiler-internal-define-for-linux.patch
delete mode 100644
meta/recipes-support/libffi/libffi/0001-mips-fix-MIPS-softfloat-build-issue.patch
rename meta/recipes-support/libffi/{libffi_3.2.1.bb =>
libffi_3.2.1+3.3-rc0.bb} (71%)
diff --git a/meta/recipes-support/libffi/libffi/0001-New-RISC-V-port-281.patch
b/meta/recipes-support/libffi/libffi/0001-New-RISC-V-port-281.patch
deleted file mode 100644
index 589c4d3c44..00
--- a/meta/recipes-support/libffi/libffi/0001-New-RISC-V-port-281.patch
+++ /dev/null
@@ -1,827 +0,0 @@
-From 8ac73103bf12ce4f776940cb17f3ced15a362f23 Mon Sep 17 00:00:00 2001
-From: Stef O'Rear
-Date: Sun, 11 Mar 2018 05:55:15 -0700
-Subject: [PATCH] New RISC-V port (#281)
-
-* Add RISC-V support
-
-This patch adds support for the RISC-V architecture (https://riscv.org).
-
-This patch has been tested using QEMU user-mode emulation and GCC 7.2.0
-in the following configurations:
-
-* -march=rv32imac -mabi=ilp32
-* -march=rv32g -mabi=ilp32d
-* -march=rv64imac -mabi=lp64
-* -march=rv64g -mabi=lp64d
-
-The ABI currently can be found at
-https://github.com/riscv/riscv-elf-psabi-doc/blob/master/riscv-elf.md .
-
-* Add RISC-V to README
-
-* RISC-V: fix configure.host
-
-Upstream-Status: Backport
[https://github.com/libffi/libffi/commit/3840d49aaa831d649b1597518a2903dfed0d57f3]
-Signed-off-by: Alistair Francis
- Makefile.am | 4 +
- configure.ac | 5 +
- src/riscv/ffi.c | 445 ++
- src/riscv/ffitarget.h | 68 +++
- src/riscv/sysv.S | 214
- 5 files changed, 736 insertions(+)
- create mode 100644 src/riscv/ffi.c
- create mode 100644 src/riscv/ffitarget.h
- create mode 100644 src/riscv/sysv.S
-
-diff --git a/Makefile.am b/Makefile.am
-index 0e40451..3837650 100644
a/Makefile.am
-+++ b/Makefile.am
-@@ -32,6 +32,7 @@ EXTRA_DIST = LICENSE ChangeLog.v1 ChangeLog.libgcj
\
-src/powerpc/asm.h src/powerpc/aix.S src/powerpc/darwin.S \
-src/powerpc/aix_closure.S src/powerpc/darwin_closure.S \
-src/powerpc/ffi_darwin.c src/powerpc/ffitarget.h \
-+ src/riscv/ffi.c src/riscv/ffitarget.h src/riscv/sysv.S \
-src/s390/ffi.c src/s390/sysv.S src/s390/ffitarget.h\
-src/sh/ffi.c src/sh/sysv.S src/sh/ffitarget.h src/sh64/ffi.c \
-src/sh64/sysv.S src/sh64/ffitarget.h src/sparc/v8.S\
-@@ -122,6 +123,9 @@ endif
- if MIPS
- nodist_libffi_la_SOURCES += src/mips/ffi.c src/mips/o32.S src/mips/n32.S
- endif
-+if RISCV
-+nodist_libffi_la_SOURCES += src/riscv/ffi.c src/riscv/sysv.S
-+endif
- if BFIN
- nodist_libffi_la_SOURCES += src/bfin/ffi.c src/bfin/sysv.S
- endif
-diff --git a/configure.ac b/configure.ac
-index ce30853..33375aa 100644
a/configure.ac
-+++ b/configure.ac
-@@ -226,6 +226,10 @@ case "$host" in
- TARGET=MIPS; TARGETDIR=mips
- ;;
-
-+ riscv*-*-*)
-+ TARGET=RISCV; TARGETDIR=riscv
-+ ;;
-+
- nios2*-linux*)
- TARGET=NIOS2; TARGETDIR=nios2
- ;;
-@@ -298,6 +302,7 @@ if test $TARGETDIR = unknown; then
- fi
-
- AM_CONDITIONAL(MIPS, test x$TARGET = xMIPS)
-+AM_CONDITIONAL(RISCV, test x$TARGET = xRISCV)
- AM_CONDITIONAL(BFIN, test x$TARGET = xBFIN)
- AM_CONDITIONAL(SPARC, test x$TARGET = xSPARC)
- AM_CONDITIONAL(X86, test x$TARGET = xX86)
-diff --git a/src/riscv/ffi.c b/src/riscv/ffi.c
-new file mode 100644
-index 000..b744fdd
/dev/null
-+++ b/src/riscv/ffi.c
-@@ -0,0 +1,445 @@
-+/* ---
-+ ffi.c - Copyright (c) 2015 Michael Knyszek
-+ 2015 Andrew Waterman
-+ 2018
[OE-core] [PATCH 1/2] gcc: Search in OE specific target gcclibdir
We put gcclibir to be /usr/lib//... and not default usr/lib/gcc/, therefore make the include search path also look into this directory, this should help in finding gcc headers like omp.h Signed-off-by: Khem Raj Cc: Martin Jansa --- .../0021-Ensure-target-gcc-headers-can-be-included.patch | 8 .../0019-Ensure-target-gcc-headers-can-be-included.patch | 6 +++--- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/meta/recipes-devtools/gcc/gcc-8.3/0021-Ensure-target-gcc-headers-can-be-included.patch b/meta/recipes-devtools/gcc/gcc-8.3/0021-Ensure-target-gcc-headers-can-be-included.patch index 60d72699d6..58b4d1aa68 100644 --- a/meta/recipes-devtools/gcc/gcc-8.3/0021-Ensure-target-gcc-headers-can-be-included.patch +++ b/meta/recipes-devtools/gcc/gcc-8.3/0021-Ensure-target-gcc-headers-can-be-included.patch @@ -1,4 +1,4 @@ -From 476eda9054df443d094273c8b61fce63d940adfc Mon Sep 17 00:00:00 2001 +From 94cc995cee76a9d59204a900fcaa2afb2eaa75c2 Mon Sep 17 00:00:00 2001 From: Khem Raj Date: Fri, 20 Feb 2015 10:25:11 + Subject: [PATCH 21/40] Ensure target gcc headers can be included @@ -23,14 +23,14 @@ Signed-off-by: Khem Raj 4 files changed, 15 insertions(+), 7 deletions(-) diff --git a/gcc/Makefile.in b/gcc/Makefile.in -index 71a8275c39f..5ae693fb06c 100644 +index 71a8275c39f..aa4adc3db24 100644 --- a/gcc/Makefile.in +++ b/gcc/Makefile.in @@ -614,6 +614,7 @@ libexecdir = @libexecdir@ # Directory in which the compiler finds libraries etc. libsubdir = $(libdir)/gcc/$(real_target_noncanonical)/$(version)$(accel_dir_suffix) -+libsubdir_target = gcc/$(target_noncanonical)/$(version) ++libsubdir_target = $(target_noncanonical)/$(version) # Directory in which the compiler finds executables libexecsubdir = $(libexecdir)/gcc/$(real_target_noncanonical)/$(version)$(accel_dir_suffix) # Directory in which all plugin resources are installed @@ -94,5 +94,5 @@ index 570cdc00034..3fb64d453f1 100644 #undef MD_EXEC_PREFIX #undef MD_STARTFILE_PREFIX -- -2.21.0 +2.22.1 diff --git a/meta/recipes-devtools/gcc/gcc-9.2/0019-Ensure-target-gcc-headers-can-be-included.patch b/meta/recipes-devtools/gcc/gcc-9.2/0019-Ensure-target-gcc-headers-can-be-included.patch index 4809bdf4bd..af0ea5c60a 100644 --- a/meta/recipes-devtools/gcc/gcc-9.2/0019-Ensure-target-gcc-headers-can-be-included.patch +++ b/meta/recipes-devtools/gcc/gcc-9.2/0019-Ensure-target-gcc-headers-can-be-included.patch @@ -1,4 +1,4 @@ -From 5225980db3e03b2fe6597393d7177d0cafd1b1d9 Mon Sep 17 00:00:00 2001 +From d27ba49e2e5c608c43265462d6831363cc7f565b Mon Sep 17 00:00:00 2001 From: Khem Raj Date: Fri, 20 Feb 2015 10:25:11 + Subject: [PATCH 19/36] Ensure target gcc headers can be included @@ -23,14 +23,14 @@ Signed-off-by: Khem Raj 4 files changed, 15 insertions(+), 7 deletions(-) diff --git a/gcc/Makefile.in b/gcc/Makefile.in -index 0064a282488..21472745c2c 100644 +index 0064a282488..98d1488171f 100644 --- a/gcc/Makefile.in +++ b/gcc/Makefile.in @@ -618,6 +618,7 @@ libexecdir = @libexecdir@ # Directory in which the compiler finds libraries etc. libsubdir = $(libdir)/gcc/$(real_target_noncanonical)/$(version)$(accel_dir_suffix) -+libsubdir_target = gcc/$(target_noncanonical)/$(version) ++libsubdir_target = $(target_noncanonical)/$(version) # Directory in which the compiler finds executables libexecsubdir = $(libexecdir)/gcc/$(real_target_noncanonical)/$(version)$(accel_dir_suffix) # Directory in which all plugin resources are installed -- 2.22.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] ✗ patchtest: failure for YOCTO #12937 - Consistent naming scheme for deployed artifacts
== Series Details == Series: YOCTO #12937 - Consistent naming scheme for deployed artifacts Revision: 1 URL : https://patchwork.openembedded.org/series/19321/ State : failure == Summary == Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures: * Patch[RFC, 2/6] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link Issue Commit shortlog is too long [test_shortlog_length] Suggested fixEdit shortlog so that it is 90 characters or less (currently 119 characters) If you believe any of these test results are incorrect, please reply to the mailing list (openembedded-core@lists.openembedded.org) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...). --- Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)
On Fri, Aug 16, 2019 at 04:54:48PM +0100, richard.pur...@linuxfoundation.org wrote: > On Fri, 2019-08-16 at 17:00 +0200, Martin Jansa wrote: > > > Will try to bump BB_NUMBER_THREADS from 8 to 72. > > > > I've tried to remove icecc.bbclass inherit (because it does things > > while parsing and RP probably doesn't have it inherited), but that > > didn't save much time. > > > > All 3 tests were with bitbake > > 18d4a31fdcec1f0e5d2199d6142f0ce833fca1a7 > > 94m19.081s 8 BB_NUMBER_THREADS and icecc > > 82m59.574s 8 BB_NUMBER_THREADS no icecc > > 68m3.556s72 BB_NUMBER_THREADS no icecc > > > > Still don't know how to get to sub 10min world runs RP was seeing, > > but at least it's as slow as it was before runqeueu changes (or even > > a bit faster in lastest master). > > Just thinking out loud, other things which can influence timings: > > Is SSTATE_DIR on NFS or local disk? SSTATE_DIR is empty directory on local disk /dev/mapper/vg00-jenkins on /jenkins type ext4 (rw,noatime,nobarrier,commit=6000,stripe=128,data=ordered) > Are sstate mirrors configured? None, normally I have SSTATE_MIRRORS over sshfs in this case, but I've removed it before any performance testing. > Is there an existing build or not, if so, how much is valid? Nothing, I remove whole TMPDIR and cache before each run. Then let it recreate the cache before starting the profile: bitbake -p; time bitbake world -P -n > Underlying filesystem of the build? ext4, everything is pretty much generic Ubuntu 18.04 There is plenty of ram, I'll try to test this from tmpfs as well. > Your build seems especially slow at executing through the tasks which > is effectively a test on how fast the system can fork() and return in > some ways. It would be interesting to block dry-run on the server side, > skip the fork and see how the numbers compare. As discussed on IRC, it's slower than yours (8 minutes from 68), but the most significant chunk of time is lost somewhere else. > My build manages some parts of the tasklist faster than others, perhaps > because there are more "free" tasks to execute at some points in the > task graph than others. > > Also, I have some patches which improve performance a bit which I'm > still testing. Thanks for all the work on this! Cheers, -- Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com signature.asc Description: Digital signature -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [RFC][PATCH 6/6] *-artifact-names: include version only in the artifact links
* drop ${PKGE}-${PKGV}-${PR} from kernel artifacts names (this is the
latest build) and add it only in hardlinks created in do_deploy_links
so that we can use PKGR there again (because these links are generally
used only by human operators and they don't have their own TASKHASH or
the IMAGE_VERSION_SUFFIX might be set to some release name which they
do understand
* this allows to drop package_get_auto_pr from kernel do_deploy as well,
leaving only 2 EXTENDPRAUTO bumps for each kernel build (do_package
and do_deploy_links, unfortunatelly these will still have different
value, so if you're looking for the exact kernel image in deploy
directory based on kernel image package version seen on the device the
EXTENDPRAUTO part of PR will be different).
[YOCTO #12937]
Signed-off-by: Martin Jansa
---
meta/classes/image-artifact-names.bbclass | 2 +-
meta/classes/kernel-artifact-names.bbclass | 7 +--
meta/classes/kernel.bbclass| 1 -
3 files changed, 2 insertions(+), 8 deletions(-)
diff --git a/meta/classes/image-artifact-names.bbclass
b/meta/classes/image-artifact-names.bbclass
index d5ba035f5a..b23cef22ca 100644
--- a/meta/classes/image-artifact-names.bbclass
+++ b/meta/classes/image-artifact-names.bbclass
@@ -3,7 +3,7 @@
##
IMAGE_BASENAME = "${PN}"
-IMAGE_VERSION_SUFFIX = "-${DATETIME}"
+IMAGE_VERSION_SUFFIX = "${PKGE}-${PKGV}-${PKGR}-${DATETIME}"
IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME"
IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}"
IMAGE_LINK_NAME = "${IMAGE_NAME}${IMAGE_VERSION_SUFFIX}"
diff --git a/meta/classes/kernel-artifact-names.bbclass
b/meta/classes/kernel-artifact-names.bbclass
index 41ef6e884d..92e08297cc 100644
--- a/meta/classes/kernel-artifact-names.bbclass
+++ b/meta/classes/kernel-artifact-names.bbclass
@@ -6,12 +6,7 @@
inherit image-artifact-names
-# Intentionally use PR instead of PKGR, because EXTENDPRAUTO included
-# in PKGR will have different value for do_install/do_deploy/do_deploy_links
-# tasks with different TASKHASH, causing multiple EXTENDPRAUTO increments for
-# each kernel build and more importantly preventing do_deploy_links to
-# reference artifacts created do_deploy task
-KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PR}-${MACHINE}"
+KERNEL_ARTIFACT_NAME ?= "${MACHINE}"
KERNEL_ARTIFACT_LINK_NAME ?= "${KERNEL_ARTIFACT_NAME}${IMAGE_VERSION_SUFFIX}"
KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}"
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index e41adcbebe..3b8bfd2991 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -700,7 +700,6 @@ kernel_do_deploy() {
}
do_deploy[cleandirs] = "${DEPLOYDIR}"
do_deploy[dirs] = "${DEPLOYDIR} ${B}"
-do_deploy[prefuncs] += "package_get_auto_pr"
addtask deploy after do_populate_sysroot do_packagedata
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [RFC][PATCH 5/6] kernel.bbclass: drop unnecessary package_get_auto_pr for do_install
* do_install doesn't use whole "version" as do_deploy does, e.g.
${PKGE}-${PKGV}-${PKGR}-${MACHINE}
it installs only the files with only ${KERNEL_VERSION} in filename or
path, so it doesn't need expanded AUTOINC value in PKGV nor
EXPANDPRAUTO in PKGR like do_deploy does
* it was introduced in
http://git.openembedded.org/openembedded-core/commit/?id=1392f959cb8cd50b5a4492899e54f3ed68ef56d7
but it's not clear why it was needed back then, but doesn't seem to be
useful at all currently, only causes multiple EXTENDPRAUTO bumps every
time different linux-yocto is being built.
* There are currently 4 EXTENDPRAUTO bumps during each build as shown in
prserv:
$ sqlite3 cache/prserv.sqlite3
SQLite version 3.27.2 2019-02-25 16:06:06
Enter ".help" for usage hints.
sqlite> select * from PRMAIN_nohist where version like 'linux-yocto-dev%';
linux-yocto-dev-4.19+gitAUTOINC+57b791cb9f_122d468967-r0|qemux86|20601304a6e4fa0b7ac13fa1262040c976c862d177077799dc15492215fa51df|0
linux-yocto-dev-4.19+gitAUTOINC+57b791cb9f_122d468967-r0|qemux86|2820d331b7eba5165943bc016a1c274d42e7605e24244873b15cc1c9c6f657e2|1
linux-yocto-dev-4.19+gitAUTOINC+57b791cb9f_122d468967-r0|qemux86|4f29da98c268aa5bf1c4767bb2bb157fc6077b1d76dfd434028b18bf3252e0c0|2
linux-yocto-dev-4.19+gitAUTOINC+57b791cb9f_122d468967-r0|qemux86|23d8d17b23bc6db1dd7f0f30086f0ec6ade2b2180e787a78d89b6e43b8c4fad6|3
linux-yocto-dev-5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0|qemux86|2a23d8783f794b3e79b438889ec60661ca635f9ec09d0519176a31d832377f1c|0
linux-yocto-dev-5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0|qemux86|dafc2a636e7e18357b7efbf99981af45234105c3f46b056edfd2142d5a5d4993|1
linux-yocto-dev-5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0|qemux86|09798369f303700fb8d42550d959e310a05fb4573b71646df51acc00d3a6fe89|2
linux-yocto-dev-5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0|qemux86|dab07eb869034df28be59ae13914989ab88bdca5a9a9362ca96b4eb38180afd7|3
the TASKHASHes correspond to do_install, do_package, do_deploy,
do_deploy_links tasks:
$ ls
tmp-glibc/stamps/qemux86-webos-linux/linux-yocto-dev/5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0.do_*sigdata*{2a23d8783f794b3e79b438889ec60661ca635f9ec09d0519176a31d832377f1c,dafc2a636e7e18357b7efbf99981af45234105c3f46b056edfd2142d5a5d4993,09798369f303700fb8d42550d959e310a05fb4573b71646df51acc00d3a6fe89,dab07eb869034df28be59ae13914989ab88bdca5a9a9362ca96b4eb38180afd7}*
tmp-glibc/stamps/qemux86-webos-linux/linux-yocto-dev/5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0.do_install.sigdata.2a23d8783f794b3e79b438889ec60661ca635f9ec09d0519176a31d832377f1c
tmp-glibc/stamps/qemux86-webos-linux/linux-yocto-dev/5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0.do_package.sigdata.dafc2a636e7e18357b7efbf99981af45234105c3f46b056edfd2142d5a5d4993
tmp-glibc/stamps/qemux86-webos-linux/linux-yocto-dev/5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0.do_deploy.sigdata.09798369f303700fb8d42550d959e310a05fb4573b71646df51acc00d3a6fe89
tmp-glibc/stamps/qemux86-webos-linux/linux-yocto-dev/5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0.do_deploy_links.sigdata.dab07eb869034df28be59ae13914989ab88bdca5a9a9362ca96b4eb38180afd7
[YOCTO #12937]
Signed-off-by: Martin Jansa
---
meta/classes/kernel.bbclass | 1 -
1 file changed, 1 deletion(-)
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index bda3af2950..e41adcbebe 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -383,7 +383,6 @@ kernel_do_install() {
install -d ${D}${sysconfdir}/modules-load.d
install -d ${D}${sysconfdir}/modprobe.d
}
-do_install[prefuncs] += "package_get_auto_pr"
# Must be ran no earlier than after do_kernel_checkout or else Makefile won't
be in ${S}/Makefile
do_kernel_version_sanity_check() {
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [RFC][PATCH 3/6] kernel-artifact-names.bbclass: use PR instead of PKGR in KERNEL_ARTIFACT_NAME
* otherwise PKGR seen in do_install, do_deploy and do_deploy_links will
have different value in each of them (PRSERV will return different
value of EXTENDPRAUTO because TASKHASH is different for each of these
tasks and also cause unnecessary multiple EXTENDPRAUTO increments for
each build).
[YOCTO #12937]
Signed-off-by: Martin Jansa
---
meta/classes/kernel-artifact-names.bbclass | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/meta/classes/kernel-artifact-names.bbclass
b/meta/classes/kernel-artifact-names.bbclass
index 529e0c565e..41ef6e884d 100644
--- a/meta/classes/kernel-artifact-names.bbclass
+++ b/meta/classes/kernel-artifact-names.bbclass
@@ -6,7 +6,12 @@
inherit image-artifact-names
-KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}"
+# Intentionally use PR instead of PKGR, because EXTENDPRAUTO included
+# in PKGR will have different value for do_install/do_deploy/do_deploy_links
+# tasks with different TASKHASH, causing multiple EXTENDPRAUTO increments for
+# each kernel build and more importantly preventing do_deploy_links to
+# reference artifacts created do_deploy task
+KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PR}-${MACHINE}"
KERNEL_ARTIFACT_LINK_NAME ?= "${KERNEL_ARTIFACT_NAME}${IMAGE_VERSION_SUFFIX}"
KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [RFC][PATCH 4/6] kernel.bbclass: imageType without {}
* just to make sure it looks like bash variable not bitbake variable in
run.do_* scripts
[YOCTO #12937]
Signed-off-by: Martin Jansa
---
meta/classes/kernel.bbclass | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 0c82946fcc..bda3af2950 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -371,9 +371,9 @@ kernel_do_install() {
install -d ${D}/${KERNEL_IMAGEDEST}
install -d ${D}/boot
for imageType in ${KERNEL_IMAGETYPES} ; do
- install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType}
${D}/${KERNEL_IMAGEDEST}/${imageType}-${KERNEL_VERSION}
+ install -m 0644 ${KERNEL_OUTPUT_DIR}/$imageType
${D}/${KERNEL_IMAGEDEST}/$imageType-${KERNEL_VERSION}
if [ "${KERNEL_PACKAGE_NAME}" = "kernel" ]; then
- ln -sf ${imageType}-${KERNEL_VERSION}
${D}/${KERNEL_IMAGEDEST}/${imageType}
+ ln -sf $imageType-${KERNEL_VERSION}
${D}/${KERNEL_IMAGEDEST}/$imageType
fi
done
install -m 0644 System.map ${D}/boot/System.map-${KERNEL_VERSION}
@@ -681,8 +681,8 @@ kernel_do_deploy() {
fi
for imageType in ${KERNEL_IMAGETYPES} ; do
- install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType}
$deployDir/${imageType}-${KERNEL_IMAGE_NAME}.bin
- ln -sf ${imageType}-${KERNEL_IMAGE_NAME}.bin
$deployDir/${imageType}
+ install -m 0644 ${KERNEL_OUTPUT_DIR}/$imageType
$deployDir/$imageType-${KERNEL_IMAGE_NAME}.bin
+ ln -sf $imageType-${KERNEL_IMAGE_NAME}.bin $deployDir/$imageType
done
if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e
'^CONFIG_MODULES=y$' .config); then
@@ -695,7 +695,7 @@ kernel_do_deploy() {
if [ "$imageType" = "fitImage" ] ; then
continue
fi
- install -m 0644
${KERNEL_OUTPUT_DIR}/${imageType}.initramfs
$deployDir/${imageType}-${INITRAMFS_NAME}.bin
+ install -m 0644
${KERNEL_OUTPUT_DIR}/$imageType.initramfs
$deployDir/$imageType-${INITRAMFS_NAME}.bin
done
fi
}
@@ -713,7 +713,7 @@ kernel_do_deploy_links() {
fi
for imageType in ${KERNEL_IMAGETYPES} ; do
- ln -vf $deployDir/${imageType}-${KERNEL_IMAGE_NAME}.bin
$deployDir/${imageType}-${KERNEL_IMAGE_LINK_NAME}.bin
+ ln -vf $deployDir/$imageType-${KERNEL_IMAGE_NAME}.bin
$deployDir/$imageType-${KERNEL_IMAGE_LINK_NAME}.bin
done
if [ ${MODULE_TARBALL_DEPLOY} = "1" -a -f
$deployDir/modules-${MODULE_TARBALL_NAME}.tgz ] ; then
@@ -725,7 +725,7 @@ kernel_do_deploy_links() {
if [ "$imageType" = "fitImage" ] ; then
continue
fi
- ln -vf ${imageType}-${INITRAMFS_NAME}.bin
${imageType}-${INITRAMFS_LINK_NAME}.bin
+ ln -vf $imageType-${INITRAMFS_NAME}.bin
$imageType-${INITRAMFS_LINK_NAME}.bin
done
fi
}
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [RFC][PATCH 2/6] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link
* just RFC, the part for images isn't finished yet and there is
still some issue with DATETIME when kernel artifacts are used
from sstate, this is just to validate the idea behind
[YOCTO #12937] before finishing the implementation (it's already
finished and used by various LGE builds, but having simpler
way of doing it directly in oe-core mighe be useful for others).
* move IMAGE_VERSION_SUFFIX from _NAME variables to _LINK_NAME
that way e.g. kernel.do_deploy can be reused from sstate to
provide "version-less" artifacts and then very fast
do_deploy_links task just adds links with consistent suffixes
(by default the version from the recipe but could be easily set
to e.g. some release name when building some products).
* create hard links instead of symlinks, so that whatever version
the filename says is really there
* some IMAGE_FSTYPES might need the "version-less" IMAGE_NAME file
to be removed first or they might either append or update the
content of the image instead of creating new image file from
scratch - I have seen this only with one proprietary format we
generate with our own tool, so hopefully this isn't very common
* this is basically the mechanism are using in webOS with
WEBOS_IMAGE_NAME_SUFFIX which is for official builds set from
jenkins job and then all artifacts (images as well as corresponding
kernel files) have the same version string)
* without this, you can still easily set the variables to contain
the version from jenkins job (excluded from sstate signature like
DATETIME currently is to prevent rebuilding it everytime even when
the content didn't change) but then when kernel is reused from sstate
you can have version 1.0 used on kernel artifacts and 2.0 on image
artifacts.
* if you don't exclude the version string with vardepsexclude, then
you get the right version in the filenames but for cost of
re-executing do_deploy every single time, which with rm_work will
cause all kernel tasks to be re-executed (together with everything
which depends on it like external modules etc).
* the implementation "from outside" is a bit tricky as shown in webOS
OSE, because first you need to reverse the meaning of IMAGE_NAME
and IMAGE_LINK_NAME like here, but also replace all symlinks with
hardlinks and then adjust all recipes/bbclasses to depend on our
do_deploy_fixup task instead of the original do_deploy
see the variable modifications:
https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/conf/distro/include/webos.inc#L65
and then various bbclasses to hook do_webos_deploy_fixup task creating
the hardlinks for possible artifacts:
https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/webos_deploy.bbclass
https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/kernel.bbclass
https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/image.bbclass
so hopefully with all these changes in oe-core other project can
achieve the same just by setting one variable IMAGE_VERSION_SUFFIX
[YOCTO #12937]
Signed-off-by: Martin Jansa
---
meta/classes/cve-check.bbclass | 4 +-
meta/classes/image-artifact-names.bbclass | 4 +-
meta/classes/image.bbclass | 10 ++---
meta/classes/kernel-artifact-names.bbclass | 4 +-
meta/classes/kernel-devicetree.bbclass | 21 +--
meta/classes/kernel.bbclass| 43 --
meta/classes/qemuboot.bbclass | 2 +-
meta/classes/rootfs-postcommands.bbclass | 4 +-
8 files changed, 63 insertions(+), 29 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index c00d2910be..19ef51eb05 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -100,10 +100,10 @@ python cve_check_write_rootfs_manifest () {
if manifest_name and os.path.exists(manifest_name):
manifest_link = os.path.join(deploy_dir, "%s.cve" % link_name)
-# If we already have another manifest, update symlinks
+# If we already have another manifest, update hardlinks
if os.path.exists(os.path.realpath(manifest_link)):
os.remove(manifest_link)
-os.symlink(os.path.basename(manifest_name), manifest_link)
+os.link(manifest_name, manifest_link)
bb.plain("Image CVE report stored in: %s" % manifest_name)
}
diff --git a/meta/classes/image-artifact-names.bbclass
b/meta/classes/image-artifact-names.bbclass
index 5ab8f1b7aa..d5ba035f5a 100644
--- a/meta/classes/image-artifact-names.bbclass
+++ b/meta/classes/image-artifact-names.bbclass
@@ -5,8 +5,8 @@
IMAGE_BASENAME = "${PN}"
IMAGE_VERSION_SUFFIX = "-${DATETIME}"
IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME"
-IMAGE_NAME =
[OE-core] [RFC][PATCH 1/6] image-artifact-names: introduce new bbclass and move some variables into it
* similar to kernel-artifact-names for other recipes/bbclasses which
need to use some deployed artifacts
* bitbake.conf: move IMAGE_BASENAME, IMAGE_VERSION_SUFFIX, IMAGE_NAME,
IMAGE_LINK_NAME variables
* image_types.bbclass: move IMAGE_NAME_SUFFIX variable
* currently IMAGE_NAME_SUFFIX is used only by image.bbclass,
image_types.bbclass and
meta/recipes-core/images/build-appliance-image_15.0.0.bb
but if it's needed by some recipe which isn't itself an image, then
it's useful in bitbake.conf, e.g. we have a recipe for creating
VirtualBox appliances which combines .wic.vmdk with .ovf file to
create .zip with appliance, but for that we need the filename of
.wic.vmdk which now contains IMAGE_NAME_SUFFIX
https://github.com/webOS-ports/meta-webos-ports/blob/4980ce52a43ac6897657602810313af359f0b839/meta-luneos/recipes-core/images/luneos-emulator-appliance.inc#L24
* we were hardcoding .rootfs suffix where needed, but for quite long
time it's configurable with IMAGE_NAME_SUFFIX since:
commit 380ee36811939d947024bf78de907e3c071b834f
Author: Patrick Ohly
Date: Mon Mar 7 18:07:52 2016 +0100
image creation: allow overriding .rootfs suffix
and might not match with hardcoded .rootfs, so make it easier to
use IMAGE_NAME_SUFFIX where needed even without inheritting whole
image_types.bbclass
[YOCTO #12937]
Signed-off-by: Martin Jansa
---
meta/classes/buildhistory.bbclass | 2 ++
meta/classes/image-artifact-names.bbclass | 15 +++
meta/classes/image-live.bbclass| 2 +-
meta/classes/image_types.bbclass | 9 ++---
meta/classes/kernel-artifact-names.bbclass | 8
meta/classes/qemuboot.bbclass | 2 ++
meta/classes/rootfs-postcommands.bbclass | 2 ++
meta/classes/testimage.bbclass | 2 ++
meta/conf/bitbake.conf | 5 -
9 files changed, 34 insertions(+), 13 deletions(-)
create mode 100644 meta/classes/image-artifact-names.bbclass
diff --git a/meta/classes/buildhistory.bbclass
b/meta/classes/buildhistory.bbclass
index f986f7c794..fb244bc04e 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -7,6 +7,8 @@
# Copyright (C) 2007-2011 Koen Kooi
#
+inherit image-artifact-names
+
BUILDHISTORY_FEATURES ?= "image package sdk"
BUILDHISTORY_DIR ?= "${TOPDIR}/buildhistory"
BUILDHISTORY_DIR_IMAGE =
"${BUILDHISTORY_DIR}/images/${MACHINE_ARCH}/${TCLIBC}/${IMAGE_BASENAME}"
diff --git a/meta/classes/image-artifact-names.bbclass
b/meta/classes/image-artifact-names.bbclass
new file mode 100644
index 00..5ab8f1b7aa
--- /dev/null
+++ b/meta/classes/image-artifact-names.bbclass
@@ -0,0 +1,15 @@
+##
+# Specific image creation and rootfs population info.
+##
+
+IMAGE_BASENAME = "${PN}"
+IMAGE_VERSION_SUFFIX = "-${DATETIME}"
+IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME"
+IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
+IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
+
+# IMAGE_NAME is the base name for everything produced when building images.
+# The actual image that contains the rootfs has an additional suffix (.rootfs
+# by default) followed by additional suffices which describe the format (.ext4,
+# .ext4.xz, etc.).
+IMAGE_NAME_SUFFIX ??= ".rootfs"
diff --git a/meta/classes/image-live.bbclass b/meta/classes/image-live.bbclass
index af71be5093..d44573efe8 100644
--- a/meta/classes/image-live.bbclass
+++ b/meta/classes/image-live.bbclass
@@ -22,7 +22,7 @@
# ${HDDIMG_ID} - FAT image volume-id
# ${ROOTFS} - indicates a filesystem image to include as the root filesystem
(optional)
-inherit live-vm-common
+inherit live-vm-common image-artifact-names
do_bootimg[depends] += "dosfstools-native:do_populate_sysroot \
mtools-native:do_populate_sysroot \
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 2eeffbb366..a6d42c03e4 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -1,9 +1,3 @@
-# IMAGE_NAME is the base name for everything produced when building images.
-# The actual image that contains the rootfs has an additional suffix (.rootfs
-# by default) followed by additional suffices which describe the format (.ext4,
-# .ext4.xz, etc.).
-IMAGE_NAME_SUFFIX ??= ".rootfs"
-
# The default aligment of the size of the rootfs is set to 1KiB. In case
# you're using the SD card emulation of a QEMU system simulator you may
# set this value to 2048 (2MiB alignment).
@@ -229,7 +223,8 @@ IMAGE_CMD_f2fs () {
EXTRA_IMAGECMD = ""
-inherit siteinfo kernel-arch
+inherit siteinfo kernel-arch image-artifact-names
+
JFFS2_ENDIANNESS ?= "${@oe.utils.conditional('SITEINFO_ENDIANNESS', 'le',
'-l', '-b', d)}"
JFFS2_ERASEBLOCK ?= "0x4"
EXTRA_IMAGECMD_jffs2 ?= "--pad ${JFFS2_ENDIANNESS}
[OE-core] [RFC][PATCH 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts
Let me explain a bit what these changes do for us in LGE. We have jenkins jobs for CI as well for official releases. All built artifacts are moved from jenkins builder to fileserver after the build. Each jobs have some identifier which is then included in the filenames of all relevant build artifacts, e.g. CI jobs will add e.g. lgsvl-verf-12 to show where it was created and what build created it (12 is BUILD_NUMBER from jenkins, verf is type of build, lgsvl is location). To do this you can already use IMAGE_VERSION_SUFFIX variable and add this as a suffix to current artifact names. But that has some bad limitations. A) If you keep IMAGE_VERSION_SUFFIX in do_deploy signatures, then the artifacts will be rebuilt even when the deploy sstate archive in cache is identical except tha filename. We had this for a while, but all CI jobs were slow, because of rebuilding kernel every single time. B) If you vardepexclude IMAGE_VERSION_SUFFIX from the tasks which use it, you get faster CI builds, but with inconsistent artifact names when kernel deploy sstate is reused, e.g. image will have lgsvl-verf-12 but all kernel artifacts will end with lgsvl-verf-11, when the kernel do_deploy was reused from previously built sstate-cache It gets even worse with B) when you have some other tooling (like runtime testing farm) which is tasked to flash image and kernel from lgsvl-verf-12 and it fails to find kernel image to flash, because it's named differently. C) Using version-less artifacts and just storing them in different directories might work better, but then it would make sense to include IMAGE_VERSION_SUFFIX in DEPLOY_DIR_IMAGE and remove it from the actual files inside (with version-less symlinks pointing to them). But this is a bit problematic when the individual images are usually downloaded by BFUs over http (and they end with various identically named files for which they don't remember from where they came). So this was the motivation why we have this in webOS. The difference for you (most people shouldn't even notice): 1) hard links instead of symlinks in DEPLOY_DIR_IMAGE, because now the version is in the *_LINK_* variables and you don't want symlink with version release-1 pointing to file created with release-2 build. 2) do_deploy, do_rootfs can still be reused from sstate, it will restore the version-less artifact and then just quicky add another hardlink with new filename (do_deploy_links task). 3) we're using this for couple years (badly hacked into OE, because we didn't want to overlay all relevant .bbclasses, but still needed to inject do_deploy_links task dependency in multiple places) and the only issue I've noticed was with one our proprietary IMAGE_FSTYPE format which was appending to image file in DEPLOY_DIR_IMAGE if it existed before, instead of overwritting it from scratch - to fix that I've just changed fstype function to remove the file before creating it again. 4) Examples: All show "ls -laih DEPLOY_DIR_IMAGE | sort" to show the symlinks and hardlinks TMPDIR is removed between each example, except the e) and f), but sstate-cache is kept in all cases and reused where possible MODULE_TARBALL_DEPLOY = "1" is added to local.conf to have more than just 1 kernel image as artifact from kernel (e.g. rpi MACHINEs have a lot of them with all the dtds). a) Current master with default values: $ ls -laih qemux86-64-master-default/ | sort 47054849 drwxr-xr-x 13 bitbake bitbake 4.0K Aug 16 16:37 .. 50735788 -rw-r--r-- 1 bitbake bitbake 612K Aug 15 17:16 grub-efi-bootx64.efi 50735796 drwxr-xr-x 2 bitbake bitbake 4.0K Aug 16 14:23 . 50741892 -rwxr-xr-x 1 bitbake bitbake 95K Aug 15 17:16 systemd-bootx64.efi 52067796 -rw-r--r-- 1 bitbake bitbake 7.6M Aug 16 14:22 bzImage--5.0.19+git2+c2e34d9ab2_00638cdd8f-r0.17-qemux86-64-20190816141126.bin 52067797 lrwxrwxrwx 1 bitbake bitbake 78 Aug 16 14:22 bzImage-qemux86-64.bin -> bzImage--5.0.19+git2+c2e34d9ab2_00638cdd8f-r0.17-qemux86-64-20190816141126.bin 52067798 lrwxrwxrwx 1 bitbake bitbake 78 Aug 16 14:22 bzImage -> bzImage--5.0.19+git2+c2e34d9ab2_00638cdd8f-r0.17-qemux86-64-20190816141126.bin 52067799 -rw-r--r-- 1 bitbake bitbake 7.0M Aug 16 14:22 modules--5.0.19+git2+c2e34d9ab2_00638cdd8f-r0.17-qemux86-64-20190816141126.tgz 52068116 lrwxrwxrwx 1 bitbake bitbake 78 Aug 16 14:22 modules-qemux86-64.tgz -> modules--5.0.19+git2+c2e34d9ab2_00638cdd8f-r0.17-qemux86-64-20190816141126.tgz 52068127 -rw-r--r-- 1 bitbake bitbake 1.3K Aug 16 14:23 core-image-minimal-qemux86-64-20190816141126.qemuboot.conf 52068128 lrwxrwxrwx 1 bitbake bitbake 58 Aug 16 14:23 core-image-minimal-qemux86-64.qemuboot.conf -> core-image-minimal-qemux86-64-20190816141126.qemuboot.conf 52068136 -rw-r--r-- 1 bitbake bitbake 162K Aug 16 14:23 core-image-minimal-qemux86-64-20190816141126.testdata.json 52068137 lrwxrwxrwx 1 bitbake bitbake 58 Aug 16 14:23 core-image-minimal-qemux86-64.testdata.json ->
Re: [OE-core] [PATCH 3/3] qemuriscv64: Specify the firmware as a bios instead of kernel
On Thu, Aug 15, 2019 at 11:40 PM Jacob Kroon wrote:
>
> Hi,
>
> On 8/15/19 11:31 PM, Alistair Francis wrote:
> > Signed-off-by: Alistair Francis
> > ---
> > meta/conf/machine/include/riscv/qemuriscv.inc | 2 +-
> > meta/conf/machine/qemuriscv64.conf| 2 --
> > 2 files changed, 1 insertion(+), 3 deletions(-)
> >
> > diff --git a/meta/conf/machine/include/riscv/qemuriscv.inc
> > b/meta/conf/machine/include/riscv/qemuriscv.inc
> > index 797a27660d..2ebd0a01f2 100644
> > --- a/meta/conf/machine/include/riscv/qemuriscv.inc
> > +++ b/meta/conf/machine/include/riscv/qemuriscv.inc
> > @@ -28,7 +28,7 @@ UBOOT_ENTRYPOINT_riscv64 = "0x8020"
> > QB_KERNEL_CMDLINE_APPEND = "earlycon=sbi"
> > QB_MEM = "-m 512"
> > QB_MACHINE = "-machine virt"
> > -QB_DEFAULT_KERNEL = "fw_jump.elf"
> > +QB_DEFAULT_BIOS = "fw_jump.elf"
> > QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
> > QB_NETWORK_DEVICE = "-device virtio-net-device,netdev=net0,mac=@MAC@"
> > QB_ROOTFS_OPT = "-drive id=disk0,file=@ROOTFS@,if=none,format=raw -device
> > virtio-blk-device,drive=disk0"
> > diff --git a/meta/conf/machine/qemuriscv64.conf
> > b/meta/conf/machine/qemuriscv64.conf
> > index dba8f09e93..99b48b309b 100644
> > --- a/meta/conf/machine/qemuriscv64.conf
> > +++ b/meta/conf/machine/qemuriscv64.conf
> > @@ -4,8 +4,6 @@
> >
> > require conf/machine/include/riscv/qemuriscv.inc
> >
> > -QB_OPT_APPEND += "-show-cursor -monitor null -device
> > loader,file=${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE},addr=0x8020"
> > -
>
> Was it intentional to remove these lines aswell ? I can't tell from the
> commit message.
This was intentional.
We used to use:
-kernel: Specify the firmware
-device (above): Specify the kernel
Now we can use:
-bios: Specify the firmware
-kernel: Specify the kernel
So this line isn't required.
Alistair
>
> /Jacob
>
> > EXTRA_IMAGEDEPENDS += "u-boot"
> > UBOOT_MACHINE = "qemu-riscv64_defconfig"
> > UBOOT_ELF = "u-boot"
> >
>
>
>
> [mikrodidakt]
> Jacob Kroon • +46 46325040
>
> mikrodidakt.se • Skiffervägen 48, SE-224 78 LUND, Sweden
>
> Consultans since 1980 • SW, HW, Embedded Systems, Linux
>
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] package: Fix race between do_package and do_packagedata
do_package has PKGDESTWORK as a cleandir and do_packagedata has it as an
sstate-input
dir. This means do_package wipes out the directory at both do_package and
do_package_setscene.
do_package_setscene and do_packagedata_setscene can run in parallel when
installing from
sstate which means they can wipe out parts of each other leading to interesting
build failures.
We therefore have to add in a hardlink copy so that the directories can work
independently
of each other.
Signed-off-by: Richard Purdie
---
meta/classes/package.bbclass | 10 +++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index e67bb5bd97d..114d6559f5e 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -2252,14 +2252,18 @@ python do_package_setscene () {
}
addtask do_package_setscene
-do_packagedata () {
- :
+# Copy from PKGDESTWORK to tempdirectory as tempdirectory can be cleaned at
both
+# do_package_setscene and do_packagedata_setscene leading to races
+python do_packagedata () {
+src = d.expand("${PKGDESTWORK}")
+dest = d.expand("${WORKDIR}/pkgdata-pdata-input")
+oe.path.copyhardlinktree(src, dest)
}
addtask packagedata before do_build after do_package
SSTATETASKS += "do_packagedata"
-do_packagedata[sstate-inputdirs] = "${PKGDESTWORK}"
+do_packagedata[sstate-inputdirs] = "${WORKDIR}/pkgdata-pdata-input"
do_packagedata[sstate-outputdirs] = "${PKGDATA_DIR}"
do_packagedata[stamp-extra-info] = "${MACHINE_ARCH}"
--
2.20.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] yocto-check-layer: Ensure we use OEBasicHash as the signature handler
The layer checks are designed to work with OEBasicHash so ensure that handler
is in use rather than the new hash equivalency one as an example.
Signed-off-by: Richard Purdie
---
scripts/lib/checklayer/__init__.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/lib/checklayer/__init__.py
b/scripts/lib/checklayer/__init__.py
index 8244cf07208..5aeec2f00f7 100644
--- a/scripts/lib/checklayer/__init__.py
+++ b/scripts/lib/checklayer/__init__.py
@@ -245,7 +245,7 @@ def get_signatures(builddir, failsafe=False, machine=None):
sigs = {}
tune2tasks = {}
-cmd = ''
+cmd = 'BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE BB_SIGNATURE_HANDLER"
BB_SIGNATURE_HANDLER="OEBasicHash" '
if machine:
cmd += 'MACHINE=%s ' % machine
cmd += 'bitbake '
--
2.20.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)
On Fri, 2019-08-16 at 17:00 +0200, Martin Jansa wrote: > > Will try to bump BB_NUMBER_THREADS from 8 to 72. > > I've tried to remove icecc.bbclass inherit (because it does things > while parsing and RP probably doesn't have it inherited), but that > didn't save much time. > > All 3 tests were with bitbake > 18d4a31fdcec1f0e5d2199d6142f0ce833fca1a7 > 94m19.081s 8 BB_NUMBER_THREADS and icecc > 82m59.574s 8 BB_NUMBER_THREADS no icecc > 68m3.556s72 BB_NUMBER_THREADS no icecc > > Still don't know how to get to sub 10min world runs RP was seeing, > but at least it's as slow as it was before runqeueu changes (or even > a bit faster in lastest master). Just thinking out loud, other things which can influence timings: Is SSTATE_DIR on NFS or local disk? Are sstate mirrors configured? Is there an existing build or not, if so, how much is valid? Underlying filesystem of the build? Your build seems especially slow at executing through the tasks which is effectively a test on how fast the system can fork() and return in some ways. It would be interesting to block dry-run on the server side, skip the fork and see how the numbers compare. My build manages some parts of the tasklist faster than others, perhaps because there are more "free" tasks to execute at some points in the task graph than others. Also, I have some patches which improve performance a bit which I'm still testing. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH V2] gcc-runtime: Move content from gcclibdir into libdir
On Fri, Aug 16, 2019 at 5:00 AM Martin Jansa wrote:
>
> Hi,
>
> I have an app which includes omp.h from gomp, it used to find it without
> adding any -I for that (with just -fopenmp to enable openmp).
>
> Now the header file is included in RSS:
> lib32-recipe-sysroot/usr/lib/arm-oemllib32-linux-gnueabi/9.2.0/include/omp.h
> but no longer found in default search dirs.
>
> Is this expected or should gcc be adjusted to search for it automatically?
>
It should have adjusted automatically but these headers are searhed
relative to its own install and not sysroot reelative
so it might have to be addressed. Does it work with normal (
non-multilib ) case ?
> Looking at the default search paths I see:
>
> ignoring nonexistent directory
> "BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/../../../../../arm-oemllib32-linux-gnueabi/include"
> ignoring nonexistent directory "/not/exist/usr/include/c++/9.2.0"
> ignoring nonexistent directory
> "/not/exist/usr/include/c++/9.2.0/arm-oemllib32-linux-gnueabi"
> ignoring nonexistent directory "/not/exist/usr/include/c++/9.2.0/backward"
> ignoring duplicate directory
> "BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/../../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/include"
> ignoring nonexistent directory
> "/not/exist/usr/lib/gcc/arm-oemllib32-linux-gnueabi/9.2.0/include"
> ignoring nonexistent directory "/not/exist/usr/local/include"
> ignoring duplicate directory
> "BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/../../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/include-fixed"
> ignoring nonexistent directory
> "BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/../../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/../../../../../arm-oemllib32-linux-gnueabi/include"
> ignoring nonexistent directory "/not/exist/usr/include"
> #include "..." search starts here:
> #include <...> search starts here:
>
> BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/include
>
> BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/include-fixed
> End of search list.
>
> On Tue, Aug 13, 2019 at 5:29 PM Khem Raj wrote:
>>
>> OE does not use the traditional /usr/lib/gcc prefix to store gcc-runtime
>> it basically is moved into libdir, however some newer files were
>> installed by newer versions of gcc especially libgomp ( omp.h openacc.h )
>> into gcclibdir, so we have content in both directories, this confuses
>> other tools which are trying to guess the gcc installation and its
>> runtime location, since now we have two directories, the tools either
>> choose one or other and we get inconsistent behavior, e.g. clang for
>> aarch64 uses /usr/lib but same clang for riscv64 chose /usr/lib/gcc
>>
>> This change ensures that OE ends up with single valid location for gcc
>> runtime files
>>
>> Move more common bits into common inc file
>>
>> Signed-off-by: Khem Raj
>> ---
>> v2: Divert packaging to use new path in whole recipe
>>
>> meta/recipes-devtools/gcc/gcc-runtime.inc| 18 +++---
>> meta/recipes-devtools/gcc/gcc-runtime_8.3.bb | 10 --
>> meta/recipes-devtools/gcc/gcc-runtime_9.1.bb | 10 --
>> 3 files changed, 15 insertions(+), 23 deletions(-)
>>
>> diff --git a/meta/recipes-devtools/gcc/gcc-runtime.inc
>> b/meta/recipes-devtools/gcc/gcc-runtime.inc
>> index a5c2600d7f..22c1d78dd1 100644
>> --- a/meta/recipes-devtools/gcc/gcc-runtime.inc
>> +++ b/meta/recipes-devtools/gcc/gcc-runtime.inc
>> @@ -17,6 +17,12 @@ EXTRA_OECONF_PATHS = "\
>> EXTRA_OECONF_append_linuxstdbase = " --enable-clocale=gnu"
>> EXTRA_OECONF_append = " --cache-file=${B}/config.cache"
>>
>> +# Disable ifuncs for libatomic on arm conflicts -march/-mcpu
>> +EXTRA_OECONF_append_arm = " libat_cv_have_ifunc=no "
>> +
>> +# Building with thumb enabled on armv6t fails
>> +ARM_INSTRUCTION_SET_armv6 = "arm"
>> +
>> RUNTIMELIBITM = "libitm"
>> RUNTIMELIBITM_arc = ""
>> RUNTIMELIBITM_mipsarch = ""
>> @@ -77,6 +83,11 @@ do_install () {
>> cd ${B}/${TARGET_SYS}/$d/
>> oe_runmake 'DESTDIR=${D}'
>> MULTIBUILDTOP=${B}/${TARGET_SYS}/$d/ install
>>
Re: [OE-core] [bitbake-devel] RFC: exposing information about the SRC_URI(s)/branch via buildhistory (or similar mechanism)
> > I think this supports my point about being more interested in patches
> > allowing people to extend/customise buildhistory than just adding X.
> >
> > Whilst we want to have good defaults, there are always going to be
> > niche cases for people wanting to extend it...
>
>
> Agreed. Then we can implement our BRANCH scheme without polluting the core
> code with it.
Richard, et al. -
I have created the cpl/buildhistory branch on poky-contrib to share a draft of
my ideas for making buildhistory more modular and extensible. Would appreciate
any comments.
The first change was to break up buildhistory.bbclass. Previously it was ~1000
lines, one of the biggest bbclasses in meta/classes. Now, buildhistory just
contains common functionality and plumbing for the buildhistory features. Each
feature listed in BUILDHISTORY_FEATURES causes a class to be automatically
inherited, with a naming convention of "buildhistory-{feature}". I created
buildhistory-package, buildhistory-task, buildhistory-sdk, and
buildhistory-image.
Users could extend buildhistory by inheriting from buildhistory directly (e.g.
if they have a completely new type of buildhistory to implement), or they can
inherit from a feature class. For example,
buildhistory-my-custom-packaging.bbclass. Then, they would use something like
the following:
BUILDHISTORY_FEATURES_remove = "package"
BUILDHISTORY_FEATURES_append = " my-custom-packaging"
Alternatively, they could just add "buildhistory-my-custom-packaging" to
INHERIT.
Some features need to do additional things in the buildhistory event handler,
e.g. "task" writes task signatures during commit. For this, I created
BUILDHISTORY_COMMIT_PREFUNCS, which nicely decouples "task" from the base
buildhistory class.
The only other big change (for now) on this branch is to abstract the
buildhistory-packaging feature a bit. I do this by introducing several "hooks":
BUILDHISTORY_PACKAGE_HOOKS - Python functions or BB functions run at the
end of buildhistory_emit_pkghistory. Currently used to invoke
buildhistory_list_pkg_files.
BUILDHISTORY_PACKAGE_RECIPEHISTORY_HOOKS - Python functions invoked in
write_recipehistory, providing an opportunity to write additional lines to the
per-recipe "latest" file.
BUILDHISTORY_PACKAGE_PACKAGEHISTORY_HOOKS - Python functions invoked in
write_packagehistory providing an opportunity to write additional lines to the
per-package "latest" file.
BUILDHISTORY_PACKAGE_SRCREV_HOOKS - Python functions invoked in
write_latest_srcrev, called once per srcrev-supporting SRC_URI.
BUILDHISTORY_PACKAGE_TAGSRCREV_HOOKS - Python functions invoked in
write_latest_srcrev, called once per tag srcrev SRC_URI.
Additionally, I have re-implemented the latest_srcrev functionality in terms of
BUILDHISTORY_PACKAGE_SRCREV_HOOKS and BUILDHISTORY_PACKAGE_TAGSRCREV_HOOKS.
An example usage of BUILDHISTORY_PACKAGE_SRCREV_HOOKS could be the following:
def branch_hook(d, name, srcrev, ud, total_count, f):
if total_count == 1:
f.write('BRANCH = "{0}"\n'.format(ud.branches[name]))
else:
f.write('BRANCH_{0} = "{1}"\n'.format(name, ud.branches[name]))
BUILDHISTORY_PACKAGE_SRCREV_HOOKS_append = " branch_hook"
Things I'm currently unsure about:
1. Variable naming - I thought about calling them _PREFUNCS or _POSTFUNCS
instead of _HOOKS, or something like that. But the problem is that some of them
can only be Python functions, not BB functions. Which leads me to:
2. Most hooks are intended to be Python functions that are found via
looking in globals(). If someone has a better solution I'd be interested. But I
also don't think this one is necessarily bad.
3. I'm trying to err on the side of not adding too many hooks for the first
release. Hopefully the set I have chosen to add is seen as useful.
4. I wonder if I should create a "buildhistory" directory under "classes"
into which I would move the feature classes. To avoid polluting the "classes"
directory.
Additional stuff to explore:
1. Adding a buildhistory-logs feature class, which archives the
log.do_{TASK} for a user-configurable selection of tasks. Default would be
compile, configure, and install.
2. Adding other extension points where it makes sense.
Thanks,
Chris
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH V2] libffi: Upgrade to 3.3-rc0
On Fri, Aug 16, 2019 at 7:48 AM Richard Purdie wrote: > > On Wed, 2019-08-14 at 16:00 -0700, Khem Raj wrote: > > libffi 3.1 release has been a bit aged and new architectures, > > compilers > > have since been come on stage to compile it, we have been carrying > > patches, but its better to use the latest 3.3 rc0 which has lot of > > these > > issues handled and is in good shape. > > > > Use 3.2.1+3.3-rc0 for PV to keep room for upgrade path without PE > > bump > > > > Signed-off-by: Khem Raj > > --- > > v2: Set PV = 3.2.1+3.3-rc0 > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/960 > > Its a multilib file conflict issue, see step5c: > > Running transaction test > Error: Transaction check error: > file /usr/include/ffi.h from install of > libffi-dev-3.2.1+3.3+rc0-r0.core2_32 conflicts with file from package > lib64-libffi-dev-3.2.1+3.3+rc0-r0.x86_64 > Will fix and send patch soon. hmmm, I wonder why it did not happen in the past. > Cheers, > > Richard > -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)
> Will try to bump BB_NUMBER_THREADS from 8 to 72.
I've tried to remove icecc.bbclass inherit (because it does things while
parsing and RP probably doesn't have it inherited), but that didn't save
much time.
All 3 tests were with bitbake 18d4a31fdcec1f0e5d2199d6142f0ce833fca1a7
94m19.081s 8 BB_NUMBER_THREADS and icecc
82m59.574s 8 BB_NUMBER_THREADS no icecc
68m3.556s72 BB_NUMBER_THREADS no icecc
Still don't know how to get to sub 10min world runs RP was seeing, but at
least it's as slow as it was before runqeueu changes (or even a bit faster
in lastest master).
On Fri, Aug 16, 2019 at 12:24 PM Martin Jansa
wrote:
> On Thu, Aug 15, 2019 at 05:05:48PM +0200, Martin Jansa wrote:
> > On Tue, Aug 13, 2019 at 10:04:08AM +0100, Richard Purdie wrote:
> > > On Mon, 2019-08-12 at 20:26 +, Peter Kjellerstedt wrote:
> > > > Comparing that build to a corresponding do-nothing build with Thud,
> > > > the time difference matches those three minutes where I have no idea
> > > > what bitbake is doing now that it didn’t need to do before…
> > > >
> > > > Hopefully these time degradations can be solved, because the current
> > > > state of bitbake is barely usable. We also need to look into possible
> > > > ways to improve the cooker output when it is running the setscene
> > > > tasks so it makes some kind of sense again.
> > >
> > > We talked on irc and you pointed at the commit things started to go
> > > wrong. Just to summarise things for the benefit of the list, this is
> > > some quick testing I did:
> > >
> > > "bitbake -p; time bitbake core-image-minimal -n"
> > >
> > > 30.0s 6c7c0cefd34067311144a1d4c01986fe0a4aef26
> > > 30.6s a0d941c787cf3ef030d190903279d311bc05d752
> > > 40.3s 7df31ff36892c2f9c65326b06b4c70
> > > 42.2s a0542ed3ff700eca35f9195f743c9e28bcd50f3e
> > > 45.4s 9983b07fffd19082abded7c3f15cc77d306dd69c
> > > 76.9s master-next
> >
> > I know I'm late to the party, but it took really long for the test to
> finish..
> >
> > I'm not using poky, so reproduce this testing on our builds I've used
> > bitbake revisions matching with poky revision RP was testing:
> >
> > + oe-core 18cdc087fd5da30e2b31f3d4e81b153cd36ca844 for older bitbake,
> because current master isn't compatible with old bitbake
> (bb.data_smart.ExpansionError: Failure expanding variable OE_IMPORTED[:=],
> expression was ${@oe_import(d)} which triggered exception AttributeError:
> module 'bb.siggen' has no attribute 'SignatureGeneratorUniHashMixIn')
> > and oe-core 23db236a054ee7a989cdbbcb42ad5c6eefd4a6ae for newer bitbake
> (to possibly eliminate impact of metadata changes)
> >
> > tested on 72core (Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz) with 380GB
> RAM
> >
> > nothing in TMPDIR/SSTATE_DIR, no SSTATE_MIRRO, no Hash Equivalence
> Server configured
> >
> > on a build with few more layers:
> > Parsing of 3238 .bb files complete (0 cached, 3238 parsed). 7632
> targets, 1927 skipped, 54 masked, 0 errors.
> >
> > but first doing just core-image-minimal like RP did:
> >
> > time pokybitbake
> oe-core
> > 2m8.191s 6c7c0cefd34067311144a1d4c01986fe0a4aef26
> 18d4a31fdcec1f0e5d2199d6142f0ce833fca1a7
> 18cdc087fd5da30e2b31f3d4e81b153cd36ca844
> > N/A a0d941c787cf3ef030d190903279d311bc05d752doesn't
> exist in poky/poky-contrib
> 18cdc087fd5da30e2b31f3d4e81b153cd36ca844
> > 2m17.053s 7df31ff36892c2f9c65326b06b4c70
> 1f630fdf0260db08541d3ca9f25f852931c19905
> 18cdc087fd5da30e2b31f3d4e81b153cd36ca844
> > 2m18.515s a0542ed3ff700eca35f9195f743c9e28bcd50f3e
> f43778c2d19e70d4befd483b06aaf247fc65c799
> 23db236a054ee7a989cdbbcb42ad5c6eefd4a6ae
> > 2m22.220s 9983b07fffd19082abded7c3f15cc77d306dd69c
> 8c26b451f22193ef1c544e2017cc84515566c1b8
> > 2m38.185s master-next
> fbcb89dc3dbabfc80310e9a4ebe72d919300a32e
> > cache.py:446: ResourceWarning: unclosed family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('127.0.0.1', 41117)>
> > value = pickled.load()
> > started showing with this revision
> > 2m17.991s master-next + fixups
> f7cd14f90463957b3e4be6d3876def98b78f1424
> > 2m9.651s master-next + "Holding off tasks %s" out
> >
> > now world
> > 253m36.637s 7df31ff36892c2f9c65326b06b4c70
> 1f630fdf0260db08541d3ca9f25f852931c19905
> 18cdc087fd5da30e2b31f3d4e81b153cd36ca844
> > 174m13.324s a0542ed3ff700eca35f9195f743c9e28bcd50f3e
> f43778c2d19e70d4befd483b06aaf247fc65c799
> 23db236a054ee7a989cdbbcb42ad5c6eefd4a6ae
> >this is time when killed at "NOTE: Executing Tasks" without showing
> any progress on the tasks (unlike other tests) and only one bitbake process
> is running
> > 633m27.475s master-next
> fbcb89dc3dbabfc80310e9a4ebe72d919300a32e
> >this is time when killed at (1417 from 71749) - running very slowly
> unlike other bitbake revisions where the task number changes relatively
> quickly - about 10 tasks per second
> > 89m13.992smaster-next + "Holding off tasks
Re: [OE-core] [PATCH V2] libffi: Upgrade to 3.3-rc0
On Wed, 2019-08-14 at 16:00 -0700, Khem Raj wrote: > libffi 3.1 release has been a bit aged and new architectures, > compilers > have since been come on stage to compile it, we have been carrying > patches, but its better to use the latest 3.3 rc0 which has lot of > these > issues handled and is in good shape. > > Use 3.2.1+3.3-rc0 for PV to keep room for upgrade path without PE > bump > > Signed-off-by: Khem Raj > --- > v2: Set PV = 3.2.1+3.3-rc0 > https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/960 Its a multilib file conflict issue, see step5c: Running transaction test Error: Transaction check error: file /usr/include/ffi.h from install of libffi-dev-3.2.1+3.3+rc0-r0.core2_32 conflicts with file from package lib64-libffi-dev-3.2.1+3.3+rc0-r0.x86_64 Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] ofono: update to 1.30
Removed upstreamed patches. Signed-off-by: Oleksandr Kravchuk --- ...-t-overwrite-src_ofonod_DEPENDENCIES.patch | 50 --- ...Add-check-for-explicit_bzero-support.patch | 27 -- .../0001-build-Fix-a-race-condition.patch | 28 --- ...in-Quiet-ld-errors-with-external-ell.patch | 36 - meta/recipes-connectivity/ofono/ofono_1.29.bb | 13 - meta/recipes-connectivity/ofono/ofono_1.30.bb | 9 6 files changed, 9 insertions(+), 154 deletions(-) delete mode 100644 meta/recipes-connectivity/ofono/ofono/0001-Makefile.am-Don-t-overwrite-src_ofonod_DEPENDENCIES.patch delete mode 100644 meta/recipes-connectivity/ofono/ofono/0001-build-Add-check-for-explicit_bzero-support.patch delete mode 100644 meta/recipes-connectivity/ofono/ofono/0001-build-Fix-a-race-condition.patch delete mode 100644 meta/recipes-connectivity/ofono/ofono/0001-main-Quiet-ld-errors-with-external-ell.patch delete mode 100644 meta/recipes-connectivity/ofono/ofono_1.29.bb create mode 100644 meta/recipes-connectivity/ofono/ofono_1.30.bb diff --git a/meta/recipes-connectivity/ofono/ofono/0001-Makefile.am-Don-t-overwrite-src_ofonod_DEPENDENCIES.patch b/meta/recipes-connectivity/ofono/ofono/0001-Makefile.am-Don-t-overwrite-src_ofonod_DEPENDENCIES.patch deleted file mode 100644 index 77073ddb1c..00 --- a/meta/recipes-connectivity/ofono/ofono/0001-Makefile.am-Don-t-overwrite-src_ofonod_DEPENDENCIES.patch +++ /dev/null @@ -1,50 +0,0 @@ -From eeaf8dac80f15a2b7c18d1d4ee63106973a6a873 Mon Sep 17 00:00:00 2001 -From: Adrian Bunk -Date: Tue, 21 May 2019 14:13:35 +0300 -Subject: Makefile.am: Don't overwrite src_ofonod_DEPENDENCIES - -src_ofonod_DEPENDENCIES did overwrite the dependencies -automatically generated from src_ofonod_LDADD with -incomplete dependencies. - -This fixes the following build error: -... - CCLD src/ofonod -libtool: error: cannot find the library 'gdbus/libgdbus-internal.la' or unhandled argument 'gdbus/libgdbus-internal.la' -make: *** [Makefile:3448: src/ofonod] Error 1 - -Upstream-Status: Submitted -Signed-off-by: Adrian Bunk - Makefile.am | 4 - 1 file changed, 4 deletions(-) - -diff --git a/Makefile.am b/Makefile.am -index d2e310d5..5edc4488 100644 a/Makefile.am -+++ b/Makefile.am -@@ -6,12 +6,10 @@ noinst_LTLIBRARIES = - if EXTERNAL_ELL - ell_cflags = @ELL_CFLAGS@ - ell_ldadd = @ELL_LIBS@ --ell_dependencies = - ell_built_sources = - else - ell_cflags = - ell_ldadd = ell/libell-internal.la --ell_dependencies = $(ell_ldadd) - ell_built_sources = ell/internal ell/ell.h - - noinst_LTLIBRARIES += ell/libell-internal.la -@@ -729,8 +727,6 @@ src_ofonod_LDADD = gdbus/libgdbus-internal.la $(builtin_libadd) $(ell_ldadd) \ - src_ofonod_LDFLAGS = -Wl,--export-dynamic \ - -Wl,--version-script=$(srcdir)/src/ofono.ver - --src_ofonod_DEPENDENCIES = $(ell_dependencies) -- - BUILT_SOURCES = $(local_headers) $(ell_built_sources) src/builtin.h - - CLEANFILES = $(BUILT_SOURCES) $(rules_DATA) --- -2.20.1 - diff --git a/meta/recipes-connectivity/ofono/ofono/0001-build-Add-check-for-explicit_bzero-support.patch b/meta/recipes-connectivity/ofono/ofono/0001-build-Add-check-for-explicit_bzero-support.patch deleted file mode 100644 index 4676bb7668..00 --- a/meta/recipes-connectivity/ofono/ofono/0001-build-Add-check-for-explicit_bzero-support.patch +++ /dev/null @@ -1,27 +0,0 @@ -From dff595671bf5a46f7c9231966a6cd7d2a0183076 Mon Sep 17 00:00:00 2001 -From: Marcel Holtmann -Date: Wed, 15 May 2019 09:39:44 +0200 -Subject: build: Add check for explicit_bzero support - -Upstream-Status: Backport -Signed-off-by: Adrian Bunk - configure.ac | 2 ++ - 1 file changed, 2 insertions(+) - -diff --git a/configure.ac b/configure.ac -index bacb7ac5..cad35c5d 100644 a/configure.ac -+++ b/configure.ac -@@ -54,6 +54,8 @@ AC_ARG_ENABLE(pie, AC_HELP_STRING([--enable-pie], - fi - ]) - -+AC_CHECK_FUNCS(explicit_bzero) -+ - AC_CHECK_FUNC(signalfd, dummy=yes, - AC_MSG_ERROR(signalfd support is required)) - --- -2.20.1 - diff --git a/meta/recipes-connectivity/ofono/ofono/0001-build-Fix-a-race-condition.patch b/meta/recipes-connectivity/ofono/ofono/0001-build-Fix-a-race-condition.patch deleted file mode 100644 index 2e2aa88265..00 --- a/meta/recipes-connectivity/ofono/ofono/0001-build-Fix-a-race-condition.patch +++ /dev/null @@ -1,28 +0,0 @@ -From 3e43e3342f3fada1624a28500a5337019fed7d89 Mon Sep 17 00:00:00 2001 -From: Adrian Bunk -Date: Thu, 9 May 2019 12:19:51 +0300 -Subject: build: Fix a race condition - -ell/ell.h was written before ell/ existed - -Upstream-Status: Backport -Signed-off-by: Adrian Bunk - Makefile.am | 1 + - 1 file changed, 1 insertion(+) - -diff --git a/Makefile.am b/Makefile.am -index a569c4a3..d2e310d5 100644 a/Makefile.am -+++ b/Makefile.am -@@ -1107,6 +1107,7 @@ ell/internal: Makefile - done > $@ - - ell/ell.h: Makefile -+ $(AM_V_at)$(MKDIR_P)
Re: [OE-core] [PATCH V2] gcc-runtime: Move content from gcclibdir into libdir
Hi,
I have an app which includes omp.h from gomp, it used to find it without
adding any -I for that (with just -fopenmp to enable openmp).
Now the header file is included in RSS:
lib32-recipe-sysroot/usr/lib/arm-oemllib32-linux-gnueabi/9.2.0/include/omp.h
but no longer found in default search dirs.
Is this expected or should gcc be adjusted to search for it automatically?
Looking at the default search paths I see:
ignoring nonexistent directory
"BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/../../../../../arm-oemllib32-linux-gnueabi/include"
ignoring nonexistent directory "/not/exist/usr/include/c++/9.2.0"
ignoring nonexistent directory
"/not/exist/usr/include/c++/9.2.0/arm-oemllib32-linux-gnueabi"
ignoring nonexistent directory "/not/exist/usr/include/c++/9.2.0/backward"
ignoring duplicate directory
"BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/../../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/include"
ignoring nonexistent directory
"/not/exist/usr/lib/gcc/arm-oemllib32-linux-gnueabi/9.2.0/include"
ignoring nonexistent directory "/not/exist/usr/local/include"
ignoring duplicate directory
"BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/../../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/include-fixed"
ignoring nonexistent directory
"BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/../../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/../../../../../arm-oemllib32-linux-gnueabi/include"
ignoring nonexistent directory "/not/exist/usr/include"
#include "..." search starts here:
#include <...> search starts here:
BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/include
BUILD/work/machine-oemllib32-linux-gnueabi/lib32-component/2.0.0-76-r7/recipe-sysroot-native/usr/bin/arm-oemllib32-linux-gnueabi/../../lib/arm-oemllib32-linux-gnueabi/gcc/arm-oemllib32-linux-gnueabi/9.2.0/include-fixed
End of search list.
On Tue, Aug 13, 2019 at 5:29 PM Khem Raj wrote:
> OE does not use the traditional /usr/lib/gcc prefix to store gcc-runtime
> it basically is moved into libdir, however some newer files were
> installed by newer versions of gcc especially libgomp ( omp.h openacc.h )
> into gcclibdir, so we have content in both directories, this confuses
> other tools which are trying to guess the gcc installation and its
> runtime location, since now we have two directories, the tools either
> choose one or other and we get inconsistent behavior, e.g. clang for
> aarch64 uses /usr/lib but same clang for riscv64 chose /usr/lib/gcc
>
> This change ensures that OE ends up with single valid location for gcc
> runtime files
>
> Move more common bits into common inc file
>
> Signed-off-by: Khem Raj
> ---
> v2: Divert packaging to use new path in whole recipe
>
> meta/recipes-devtools/gcc/gcc-runtime.inc| 18 +++---
> meta/recipes-devtools/gcc/gcc-runtime_8.3.bb | 10 --
> meta/recipes-devtools/gcc/gcc-runtime_9.1.bb | 10 --
> 3 files changed, 15 insertions(+), 23 deletions(-)
>
> diff --git a/meta/recipes-devtools/gcc/gcc-runtime.inc
> b/meta/recipes-devtools/gcc/gcc-runtime.inc
> index a5c2600d7f..22c1d78dd1 100644
> --- a/meta/recipes-devtools/gcc/gcc-runtime.inc
> +++ b/meta/recipes-devtools/gcc/gcc-runtime.inc
> @@ -17,6 +17,12 @@ EXTRA_OECONF_PATHS = "\
> EXTRA_OECONF_append_linuxstdbase = " --enable-clocale=gnu"
> EXTRA_OECONF_append = " --cache-file=${B}/config.cache"
>
> +# Disable ifuncs for libatomic on arm conflicts -march/-mcpu
> +EXTRA_OECONF_append_arm = " libat_cv_have_ifunc=no "
> +
> +# Building with thumb enabled on armv6t fails
> +ARM_INSTRUCTION_SET_armv6 = "arm"
> +
> RUNTIMELIBITM = "libitm"
> RUNTIMELIBITM_arc = ""
> RUNTIMELIBITM_mipsarch = ""
> @@ -77,6 +83,11 @@ do_install () {
> cd ${B}/${TARGET_SYS}/$d/
> oe_runmake 'DESTDIR=${D}'
> MULTIBUILDTOP=${B}/${TARGET_SYS}/$d/ install
> done
> + if [ -d ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include ]; then
> + install -d ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> + mv ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include/*
> ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> + rmdir --ignore-fail-on-non-empty -p
> ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include
> + fi
>
Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)
On Thu, Aug 15, 2019 at 05:05:48PM +0200, Martin Jansa wrote:
> On Tue, Aug 13, 2019 at 10:04:08AM +0100, Richard Purdie wrote:
> > On Mon, 2019-08-12 at 20:26 +, Peter Kjellerstedt wrote:
> > > Comparing that build to a corresponding do-nothing build with Thud,
> > > the time difference matches those three minutes where I have no idea
> > > what bitbake is doing now that it didn’t need to do before…
> > >
> > > Hopefully these time degradations can be solved, because the current
> > > state of bitbake is barely usable. We also need to look into possible
> > > ways to improve the cooker output when it is running the setscene
> > > tasks so it makes some kind of sense again.
> >
> > We talked on irc and you pointed at the commit things started to go
> > wrong. Just to summarise things for the benefit of the list, this is
> > some quick testing I did:
> >
> > "bitbake -p; time bitbake core-image-minimal -n"
> >
> > 30.0s 6c7c0cefd34067311144a1d4c01986fe0a4aef26
> > 30.6s a0d941c787cf3ef030d190903279d311bc05d752
> > 40.3s 7df31ff36892c2f9c65326b06b4c70
> > 42.2s a0542ed3ff700eca35f9195f743c9e28bcd50f3e
> > 45.4s 9983b07fffd19082abded7c3f15cc77d306dd69c
> > 76.9s master-next
>
> I know I'm late to the party, but it took really long for the test to finish..
>
> I'm not using poky, so reproduce this testing on our builds I've used
> bitbake revisions matching with poky revision RP was testing:
>
> + oe-core 18cdc087fd5da30e2b31f3d4e81b153cd36ca844 for older bitbake, because
> current master isn't compatible with old bitbake
> (bb.data_smart.ExpansionError: Failure expanding variable OE_IMPORTED[:=],
> expression was ${@oe_import(d)} which triggered exception AttributeError:
> module 'bb.siggen' has no attribute 'SignatureGeneratorUniHashMixIn')
> and oe-core 23db236a054ee7a989cdbbcb42ad5c6eefd4a6ae for newer bitbake (to
> possibly eliminate impact of metadata changes)
>
> tested on 72core (Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz) with 380GB RAM
>
> nothing in TMPDIR/SSTATE_DIR, no SSTATE_MIRRO, no Hash Equivalence Server
> configured
>
> on a build with few more layers:
> Parsing of 3238 .bb files complete (0 cached, 3238 parsed). 7632 targets,
> 1927 skipped, 54 masked, 0 errors.
>
> but first doing just core-image-minimal like RP did:
>
> time pokybitbake
> oe-core
> 2m8.191s 6c7c0cefd34067311144a1d4c01986fe0a4aef26
> 18d4a31fdcec1f0e5d2199d6142f0ce833fca1a7
> 18cdc087fd5da30e2b31f3d4e81b153cd36ca844
> N/A a0d941c787cf3ef030d190903279d311bc05d752doesn't exist
> in poky/poky-contrib 18cdc087fd5da30e2b31f3d4e81b153cd36ca844
> 2m17.053s 7df31ff36892c2f9c65326b06b4c70
> 1f630fdf0260db08541d3ca9f25f852931c19905
> 18cdc087fd5da30e2b31f3d4e81b153cd36ca844
> 2m18.515s a0542ed3ff700eca35f9195f743c9e28bcd50f3e
> f43778c2d19e70d4befd483b06aaf247fc65c799
> 23db236a054ee7a989cdbbcb42ad5c6eefd4a6ae
> 2m22.220s 9983b07fffd19082abded7c3f15cc77d306dd69c
> 8c26b451f22193ef1c544e2017cc84515566c1b8
> 2m38.185s master-next
> fbcb89dc3dbabfc80310e9a4ebe72d919300a32e
> cache.py:446: ResourceWarning: unclosed family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('127.0.0.1', 41117)>
> value = pickled.load()
> started showing with this revision
> 2m17.991s master-next + fixups
> f7cd14f90463957b3e4be6d3876def98b78f1424
> 2m9.651s master-next + "Holding off tasks %s" out
>
> now world
> 253m36.637s 7df31ff36892c2f9c65326b06b4c70
> 1f630fdf0260db08541d3ca9f25f852931c19905
> 18cdc087fd5da30e2b31f3d4e81b153cd36ca844
> 174m13.324s a0542ed3ff700eca35f9195f743c9e28bcd50f3e
> f43778c2d19e70d4befd483b06aaf247fc65c799
> 23db236a054ee7a989cdbbcb42ad5c6eefd4a6ae
>this is time when killed at "NOTE: Executing Tasks" without showing any
> progress on the tasks (unlike other tests) and only one bitbake process is
> running
> 633m27.475s master-next
> fbcb89dc3dbabfc80310e9a4ebe72d919300a32e
>this is time when killed at (1417 from 71749) - running very slowly unlike
> other bitbake revisions where the task number changes relatively quickly -
> about 10 tasks per second
> 89m13.992smaster-next + "Holding off tasks %s" out
> 89m59.201smaster-next updated today
> 85fe627fdb6510f0942917964386fad9d8c479c8
>
> Interestingly old 1f630fdf0260db08541d3ca9f25f852931c19905 is 3 times
> slower than recent master-next.
>
> I'll send -P output next.
So 1f630fdf0260db08541d3ca9f25f852931c19905 was already "bad", because
this commit introduced that
logger.debug(2, "Holding off tasks %s" % pprint.pformat(self.holdoff_tasks))
Here is -P output from it:
5min (til "Executing Tasks" message):
[OE-core] ✗ patchtest: failure for dpkg: Use less as pager (rev3)
== Series Details == Series: dpkg: Use less as pager (rev3) Revision: 3 URL : https://patchwork.openembedded.org/series/18240/ State : failure == Summary == Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures: * Issue Series does not apply on top of target branch [test_series_merge_on_head] Suggested fixRebase your series on top of targeted branch Targeted branch warrior (currently at 952bfcc3f4) If you believe any of these test results are incorrect, please reply to the mailing list (openembedded-core@lists.openembedded.org) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...). --- Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [warrior][PATCH] dpkg: Use less as pager
From: Ricardo Ribalda Delgado
Debian traditionaly uses /usr/bin/pager as the system pager, which is a
link to the user preferred pager. This is a Debianism.
Without this patch:
root@qt5122:~# dpkg -l
sh: pager: command not found
dpkg-query: error: showing package list on pager subprocess returned error exit
status 127
Signed-off-by: Ricardo Ribalda Delgado
Signed-off-by: Richard Purdie
Signed-off-by: Richard Leitner
---
meta/recipes-devtools/dpkg/dpkg/pager.patch | 21 +
meta/recipes-devtools/dpkg/dpkg_1.19.4.bb | 1 +
2 files changed, 22 insertions(+)
create mode 100644 meta/recipes-devtools/dpkg/dpkg/pager.patch
diff --git a/meta/recipes-devtools/dpkg/dpkg/pager.patch
b/meta/recipes-devtools/dpkg/dpkg/pager.patch
new file mode 100644
index 00..e56b9d28af
--- /dev/null
+++ b/meta/recipes-devtools/dpkg/dpkg/pager.patch
@@ -0,0 +1,21 @@
+pager: Use less instead of pager
+
+pager is a Debianism. Istead use directly pager.
+
+Upstream-Status: Inappropriate [OE-Core integration specific]
+
+Suggested-by: Burton, Ross
+Signed-off-by: Ricardo Ribalda
+diff --git a/lib/dpkg/dpkg.h b/lib/dpkg/dpkg.h
+index 2bb067a..6cbce80 100644
+--- a/lib/dpkg/dpkg.h
b/lib/dpkg/dpkg.h
+@@ -95,7 +95,7 @@ DPKG_BEGIN_DECLS
+ #define MAXUPDATES 250
+
+ #define DEFAULTSHELL"sh"
+-#define DEFAULTPAGER"pager"
++#define DEFAULTPAGER"less"
+
+ #define MD5HASHLEN 32
+ #define MAXTRIGDIRECTIVE 256
diff --git a/meta/recipes-devtools/dpkg/dpkg_1.19.4.bb
b/meta/recipes-devtools/dpkg/dpkg_1.19.4.bb
index 77fbfb5c91..b83868fec0 100644
--- a/meta/recipes-devtools/dpkg/dpkg_1.19.4.bb
+++ b/meta/recipes-devtools/dpkg/dpkg_1.19.4.bb
@@ -12,6 +12,7 @@ SRC_URI = "${DEBIAN_MIRROR}/main/d/${BPN}/${BPN}_${PV}.tar.xz
\
file://0006-add-musleabi-to-known-target-tripets.patch \
file://0007-dpkg-deb-build.c-Remove-usage-of-clamp-mtime-in-tar.patch \
file://0001-dpkg-Support-muslx32-build.patch \
+ file://pager.patch \
"
SRC_URI_append_class-native = " \
file://tweak-options-require-tar-1.27.patch \
--
2.21.0
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] base.bbclass: add dependency on pseudo from do_prepare_recipe_sysroot
do_prepare_recipe_sysroot may perform groupadd, which requires pseudo.
However, do_prepare_recipe_sysroot does not depend on pseudo explicitly,
which sometimes causes a build error when building a recipe that adds
groups.
This issue only occurs when executing do_prepare_recipe_sysroot for a
recipe that adds groups before finishing a task that depends on pseudo
for a recipe that doesn't add groups.
Signed-off-by: Mattias Hansson
---
meta/classes/base.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 0c8a4b2862..0576b110c9 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -480,6 +480,7 @@ python () {
# If we're building a target package we need to use fakeroot (pseudo)
# in order to capture permissions, owners, groups and special files
if not bb.data.inherits_class('native', d) and not
bb.data.inherits_class('cross', d):
+d.setVarFlag('do_prepare_recipe_sysroot', 'fakeroot', '1')
d.setVarFlag('do_unpack', 'umask', '022')
d.setVarFlag('do_configure', 'umask', '022')
d.setVarFlag('do_compile', 'umask', '022')
--
2.11.0
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] screen: add /etc/screenrc as global config file
Signed-off-by: Yi Zhao
---
meta/recipes-extended/screen/screen_4.6.2.bb | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-extended/screen/screen_4.6.2.bb
b/meta/recipes-extended/screen/screen_4.6.2.bb
index 24ec751..21b476d 100644
--- a/meta/recipes-extended/screen/screen_4.6.2.bb
+++ b/meta/recipes-extended/screen/screen_4.6.2.bb
@@ -35,10 +35,11 @@ inherit autotools texinfo
PACKAGECONFIG ??= ""
PACKAGECONFIG[utempter] =
"ac_cv_header_utempter_h=yes,ac_cv_header_utempter_h=no,libutempter,"
-EXTRA_OECONF = "--with-pty-mode=0620 --with-pty-group=5 \
+EXTRA_OECONF = "--with-pty-mode=0620 --with-pty-group=5
--with-sys-screenrc=${sysconfdir}/screenrc \
${@bb.utils.contains('DISTRO_FEATURES', 'pam', '--enable-pam',
'--disable-pam', d)}"
do_install_append () {
+ install -D -m 644 ${S}/etc/etcscreenrc ${D}/${sysconfdir}/screenrc
if [ "${@bb.utils.filter('DISTRO_FEATURES', 'pam', d)}" ]; then
install -D -m 644 ${WORKDIR}/screen.pam
${D}/${sysconfdir}/pam.d/screen
fi
--
2.7.4
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] LF Community Bridge Mentorship
hi there. just a reminder! cheers On Tue, Jul 30, 2019 at 4:46 PM Nicolas Dechesne wrote: > > Dear all, > > The Community Bridge platform was launched earlier this year by the > Linux Foundation. It’s a set of services available to LF projects to > help them build stronger ecosystems, and foster developers > relationship within their respective communities. See [1] and [2]. The > Yocto Project has joined the Community Bridge Funding and Mentorship > programs. > > If you have any specific questions about these programs, please do not > hesitate to reach out to me. > > Today, we would like to talk more about the mentorship program. We > have tried several initiatives in the past about interns and > mentorship, and we would like to do that again using the new LF > platform. A mentorship program for the Yocto Project is a good > opportunity for our overall community to reach out to new developers, > improve our developers diversity. It will help increase the pool of > available developers with appropriate skills about the Yocto Project > and Open Embedded technologies and as a consequence create a more > sustainable ecosystem around the Yocto Project. It also offers > existing senior developers the chance to become mentors, and share > their knowledge! > > We would like to request feedback about how to best define the Yocto > Project Mentorship program. > > The structure of the mentorship is flexible and we can adjust as > needed. Our initial thought is to set 12-week project, which is what > the Linux Kernel Mentorship program is using, see [4]. The overall > process involves: > > 1. Finding / defining work items for potential mentees. > 2. Get appropriate funding for mentees’ stipends > 3. Recruit mentors within our community > 4. Review and select mentees’ applications > > There has already been some work on the potential work items in the > form of the newcomer bugs, some of which could be turned into work > suitable for a mentorship programme. > > === Call for ACTION === > > * We need your help for any of the items above, and especially #1 to > get started. Whether you are a long time contributor to the project or > you recently joined our mailing list, your feedback will be greatly > appreciated. > * If you are interested to participate as a mentor, please let me know! > * You don’t need to be a project expert to be a mentor, you need to > know how the project works and be willing to help guide someone > through their task. > > There is no specific timeline set yet, it will most likely depend on > how much feedback we receive about this initiative, but most likely > the internships will have to be tied to the Yocto Project release > timeline. > > If you have any questions or any suggestions, please reach out to me! > cheers, > Nico > > [1] > https://www.linuxfoundation.org/press-release/2019/03/the-linux-foundation-launches-new-communitybridge-platform-to-help-sustain-open-source-communities/ > [2] https://communitybridge.org/ > [3] https://people.communitybridge.org/ > [4] https://wiki.linuxfoundation.org/lkmp -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] Bug report: util-linux: circular dependencies occurs when enable systemd in PACKAGECONFIG
Hi,
Some DISTRO_FEATURES related to systemd have been enabled in local.conf
DISTRO_FEATURES_append = " systemd"
DISTRO_FEATURES_BACKFILL_CONSIDERED += "sysvinit"
VIRTUAL-RUNTIME_init_manager = "systemd"
VIRTUAL-RUNTIME_initscripts = "systemd-compat-units"
KERNEL_FEATURES_append = " cfg/systemd.scc"
1.
meta/recipes-core/util-linux/util-linux.inc has below codes:
(
PACKAGECONFIG[systemd] = "--with-systemd
--with-systemdsystemunitdir=${systemd_system_unitdir}, --without-systemd
--without-systemdsystemunitdir,systemd"
BBCLASSEXTEND = "native nativesdk"
)
2.
Now I want to enable systemd feature, then I add below code to
util-linux.inc:
(
PACKAGECONFIG_class-target += " systemd"
)
3.
Then bitbake util-linux
You will below errors:(I just cut out a bit)
These are usually caused by circular dependencies and any circular
dependency chains found will be printed below. Increase the debug level
to see a list of unbuildable tasks.
Identifying dependency loops (this may take a short while)...
ERROR:
Dependency loop #1 found:
Task
/buildarea/raid0/hsong/proj/lykerneldevsrc/layers/oe-core/meta/recipes-core/systemd/systemd_242.bb:do_package
(dependent Tasks ['base-files_3.0.14.bb:do_packagedata',
'kmod_git.bb:do_packagedata', 'glibc_2.30.bb:do_packagedata',
'util-linux_2.34.bb:do_packagedata', 'libcap_2.27.bb:do_packagedata',
'opkg-utils_0.4.1.bb:do_packagedata', 'libpam_1.3.1.bb:do_packagedata',
'base-passwd_3.5.29.bb:do_packagedata',
'pseudo_git.bb:do_populate_sysroot', 'systemd_242.bb:do_install',
'rpm_4.14.2.1.bb:do_populate_sysroot',
'dwarfsrcfiles.bb:do_populate_sysroot', 'shadow_4.6.bb:do_packagedata',
'bash-completion_2.9.bb:do_packagedata',
'shadow-sysroot_4.6.bb:do_packagedata', 'xz_5.2.4.bb:do_p
4. I tried some other code:
PACKAGECONFIG[systemd] = "--with-systemd
--with-systemdsystemunitdir=${systemd_system_unitdir}, --without-systemd
--without-systemdsystemunitdir,systemd"
>> PACKAGECONFIG[systemd] = "--with-systemd
--with-systemdsystemunitdir=${systemd_system_unitdir}, --without-systemd
--without-systemdsystemunitdir"
or PACKAGECONFIG_append += " systemd"
There are still other errors.
Best Regard,
--Hongzhi
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] core-image-sato-sdk: test image with 512M memory
On 2019/8/12 下午4:57, Kang Kai wrote: On 2019/7/27 下午4:42, Kang Kai wrote: On 2019/7/27 上午5:40, richard.pur...@linuxfoundation.org wrote: On Fri, 2019-07-26 at 05:23 -0400, kai.k...@windriver.com wrote: From: Kai Kang When run do_testimage for core-image-sato-sdk, it fails to pass test case: RESULTS - systemd.SystemdBasicTests.test_systemd_failed: FAILED (0.43s) It is OOM issue and daemon rpc.statd is killed: [ 531.306146] Out of memory: Kill process 193 (rpc.statd) score 200 or sacrifice child Increase the memory of qemu to 512M to avoid such OOM issue. Signed-off-by: Kai Kang --- meta/recipes-sato/images/core-image-sato-sdk.bb | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/recipes-sato/images/core-image-sato-sdk.bb b/meta/recipes-sato/images/core-image-sato-sdk.bb index d7cc52b52b..f7963d018e 100644 --- a/meta/recipes-sato/images/core-image-sato-sdk.bb +++ b/meta/recipes-sato/images/core-image-sato-sdk.bb @@ -9,3 +9,4 @@ IMAGE_FEATURES += "dev-pkgs tools-sdk \ IMAGE_INSTALL += "kernel-devsrc" +TEST_QEMUPARAMS = "-m 512" Any idea what is using so much memory in the image when this happens? Its rather sad that we can't have NFS+systemd with 256MB memory... It caused by stap test case. I minimized the test cases to TEST_SUITES = "ping date ssh systemd stap kernelmodule gcc " which could reproduce the error. And it PASSes testimage that remove stap test from default TEST_SUITES: TEST_SUITES_remove = 'stap' But I can't reproduce the OOM failure to run stap tests manually. Hi Richard, The root cause of test case stap fails is available memory is too low. Compare systemd with sysvinit, memory status are listed: systemd: root@qemux86:~# free -h total used free shared buff/cache available Mem: 239Mi 120Mi 23Mi 8.0Mi 94Mi 100Mi Swap: 0B 0B 0B sysvinit: root@qemux86:~# free -h total used free shared buff/cache available Mem: 239Mi 45Mi 111Mi 0.0Ki 82Mi 184Mi Swap: 0B 0B 0B With systemd, processes of rpc.statd and rpc.mountd take about less than 80M memories. And I compared with Debian 10, they take similar size of memories. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 464 rpcuser 20 0 56316 51336 2192 S 0.0 20.9 0:00.09 rpc.statd 186 root 20 0 30136 27024 2280 S 0.0 11.0 0:00.02 rpc.mountd Compare to sysvinit, they take only about 2M and 448K: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 567 rpcuser 20 0 3444 2372 1792 S 0.0 1.0 0:00.00 rpc.statd 677 root 20 0 3700 448 0 S 0.0 0.2 0:00.00 rpc.mountd I didn't figure out the differences between these 2 ways to start the commands: ExecStart=/usr/sbin/rpc.statd -F $STATD_OPTS Vs. test -x "$NFS_STATD" || NFS_STATD=/usr/sbin/rpc.statd start-stop-daemon --start --exec "$NFS_STATD" --pidfile "$STATD_PID" Hi Richard, Any more comment here please? Regards, Kai Regards, Kai Regards, Kai Cheers, Richard -- Kai Kang -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 3/3] qemuriscv64: Specify the firmware as a bios instead of kernel
Hi,
On 8/15/19 11:31 PM, Alistair Francis wrote:
> Signed-off-by: Alistair Francis
> ---
> meta/conf/machine/include/riscv/qemuriscv.inc | 2 +-
> meta/conf/machine/qemuriscv64.conf| 2 --
> 2 files changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/meta/conf/machine/include/riscv/qemuriscv.inc
> b/meta/conf/machine/include/riscv/qemuriscv.inc
> index 797a27660d..2ebd0a01f2 100644
> --- a/meta/conf/machine/include/riscv/qemuriscv.inc
> +++ b/meta/conf/machine/include/riscv/qemuriscv.inc
> @@ -28,7 +28,7 @@ UBOOT_ENTRYPOINT_riscv64 = "0x8020"
> QB_KERNEL_CMDLINE_APPEND = "earlycon=sbi"
> QB_MEM = "-m 512"
> QB_MACHINE = "-machine virt"
> -QB_DEFAULT_KERNEL = "fw_jump.elf"
> +QB_DEFAULT_BIOS = "fw_jump.elf"
> QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
> QB_NETWORK_DEVICE = "-device virtio-net-device,netdev=net0,mac=@MAC@"
> QB_ROOTFS_OPT = "-drive id=disk0,file=@ROOTFS@,if=none,format=raw -device
> virtio-blk-device,drive=disk0"
> diff --git a/meta/conf/machine/qemuriscv64.conf
> b/meta/conf/machine/qemuriscv64.conf
> index dba8f09e93..99b48b309b 100644
> --- a/meta/conf/machine/qemuriscv64.conf
> +++ b/meta/conf/machine/qemuriscv64.conf
> @@ -4,8 +4,6 @@
>
> require conf/machine/include/riscv/qemuriscv.inc
>
> -QB_OPT_APPEND += "-show-cursor -monitor null -device
> loader,file=${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE},addr=0x8020"
> -
Was it intentional to remove these lines aswell ? I can't tell from the
commit message.
/Jacob
> EXTRA_IMAGEDEPENDS += "u-boot"
> UBOOT_MACHINE = "qemu-riscv64_defconfig"
> UBOOT_ELF = "u-boot"
>
[mikrodidakt]
Jacob Kroon • +46 46325040
mikrodidakt.se • Skiffervägen 48, SE-224 78 LUND, Sweden
Consultans since 1980 • SW, HW, Embedded Systems, Linux
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
