Re: [OE-core] [PATCH 1/2 v5] resultstool: enable merge, store, report and regression analysis
Hi RP, I had executed the runtime/ptest using the latest master with the latest changes to understand the new improvement. For now, the resulttool regression will ignore both 'ptestresult.rawlogs' & 'ptestresult.sections' as current regression operation focuses on comparing the "status" differences and it does not need the log as well as the new section information. By ignoring both 'ptestresult.rawlogs' & 'ptestresult.sections' , the regression time was optimized to seconds instead of minutes for ptest. For additional information inside 'ptestresult.sections', do we need similar regression? Any idea which data inside 'ptestresult.sections' will be useful for regression? Currently, resulttool regression will only print text based report, if html report was needed, it can be extend by using jinja2 framework. Do we need html report for this regression? Any requirement for the html report? http://lists.openembedded.org/pipermail/openembedded-core/2019-February/278971.html http://lists.openembedded.org/pipermail/openembedded-core/2019-February/278972.html http://lists.openembedded.org/pipermail/openembedded-core/2019-February/278973.html Thanks, Yeoh Ee Peng -Original Message- From: Richard Purdie [mailto:richard.pur...@linuxfoundation.org] Sent: Friday, February 1, 2019 7:40 AM To: Yeoh, Ee Peng ; 'openembedded-core@lists.openembedded.org' Cc: Eggleton, Paul ; Burton, Ross Subject: Re: [OE-core] [PATCH 1/2 v5] resultstool: enable merge, store, report and regression analysis On Thu, 2019-01-31 at 05:23 +, Yeoh, Ee Peng wrote: > Hi RP, > > I looked into ptest and regression. The existing "resultstool > regression" can be used to perform regression on ptest, since the > testresults.json capture ptest status. I had executed regression > script for the below 2 ptest testresults.json. Attached was the > regression report for ptest. > > https://autobuilder.yocto.io/pub/releases/yocto-2.7_M2.rc1/testresults > /qemux86-64-ptest/testresults.json > https://autobuilder.yocto.io/pub/releases/yocto-2.7_M1.rc1/testresults > /qemux86-64-ptest/testresults.json > > The only challenges now was since ptest result set was relatively > large, it was taking some time for computing the regression. Also > there was this "ptestresult.rawlogs" testcase that does not contain > status but the large rawlog. > > I did an experiment where I run the regression on testresults.json > with and without the ptest rawlog. It shows the time taken for > regression was significantly larger when it contain the rawlog. I will > try to improve the regression time by throw away the rawlog at runtime > when perform computing. > testresults.json with rawlog > Regression start time: 20190131122805 > Regression end time: 20190131124425 > Time taken: 16 mins 20 sec > > testresults.json without rawlog > Regression start time: 20190131124512 > Regression end time: 20190131124529 > Time taken: 17 sec Analysing the rawlog makes no sense so the tool needs to simply ignore that. 16 minutes is far too long! I've just merged some changes which mean there are probably some other sections it will need to ignore now too since the logs are now being split out per ptest (section). I've left rawlogs in as its useful for debugging but once the section splits are working we could remove it. This adds in timing data so we know how long each ptest took to run (in seconds), it also adds in exit code and timeout data. These all complicate the regression analysis but the fact that lttng has been timing out (for example) has been overlooked until now and shows we need to analyse these things. I'm considering whether we should have a command in resulttool which takes json data and writes it out in a "filesystem" form. The code in logparser.py already has a rudimentary version of this for ptest data. It could be extended to write out a X.log for each ptest based on the split out data and maybe duration and timeout information in some form too. The idea behind flat filesystem representations of the data is that a user can more easily explore or compare them, they also show up well in git. Its also worth thinking about how we'll end up using this. testresult will get called at the end of builds (particularly) release builds and we'll want it to generate a QA report for the automated test data. The autobuilder will likely put an http link in the "release build ready" email to an html like report stored alongside the testresults json files. I'm still trying to figure out how to make this all fit together and allow automated comparisons but the build performance data would also fit into this (and already has html reports). Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2 v7] resulttool: enable merge, store, report and regression analysis
OEQA outputs test results into json files and these files were
archived by Autobuilder during QA releases. Example: each oe-selftest
run by Autobuilder for different host distro generate a
testresults.json file.
These scripts were developed as a test result tools to manage
these testresults.json file.
Using the "store" operation, user can store multiple testresults.json
files as well as the pre-configured directories used to hold those files.
Using the "merge" operation, user can merge multiple testresults.json
files to a target file.
Using the "report" operation, user can view the test result summary
for all available testresults.json files inside a ordinary directory
or a git repository.
Using the "regression-file" operation, user can perform regression
analysis on testresults.json files specified. Using the "regression-dir"
and "regression-git" operations, user can perform regression analysis
on directory and git accordingly.
These resulttool operations expect the testresults.json file to use
the json format below.
{
"": {
"configuration": {
"": "",
"": "",
...
"": "",
},
"result": {
"": {
"status": "",
"log": ""
},
"": {
"status": "",
"log": ""
},
...
"": {
"status": "",
"log": ""
},
}
},
...
"": {
"configuration": {
"": "",
"": "",
...
"": "",
},
"result": {
"": {
"status": "",
"log": ""
},
"": {
"status": "",
"log": ""
},
...
"": {
"status": "",
"log": ""
},
}
},
}
To use these scripts, first source oe environment, then run the
entry point script to look for help.
$ resulttool
To store test result from oeqa automated tests, execute the below
$ resulttool store
To merge multiple testresults.json files, execute the below
$ resulttool merge
To report test report, execute the below
$ resulttool report
To perform regression file analysis, execute the below
$ resulttool regression-file
To perform regression dir analysis, execute the below
$ resulttool regression-dir
To perform regression git analysis, execute the below
$ resulttool regression-git
[YOCTO# 13012]
[YOCTO# 12654]
Signed-off-by: Yeoh Ee Peng
---
meta/lib/oeqa/files/testresults/testresults.json | 40
meta/lib/oeqa/selftest/cases/resulttooltests.py| 104 +++
scripts/lib/resulttool/__init__.py | 0
scripts/lib/resulttool/merge.py| 71 +++
scripts/lib/resulttool/regression.py | 208 +
scripts/lib/resulttool/report.py | 113 +++
scripts/lib/resulttool/resultsutils.py | 67 +++
scripts/lib/resulttool/store.py| 110 +++
.../resulttool/template/test_report_full_text.txt | 35
scripts/resulttool | 84 +
10 files changed, 832 insertions(+)
create mode 100644 meta/lib/oeqa/files/testresults/testresults.json
create mode 100644 meta/lib/oeqa/selftest/cases/resulttooltests.py
create mode 100644 scripts/lib/resulttool/__init__.py
create mode 100644 scripts/lib/resulttool/merge.py
create mode 100644 scripts/lib/resulttool/regression.py
create mode 100644 scripts/lib/resulttool/report.py
create mode 100644 scripts/lib/resulttool/resultsutils.py
create mode 100644 scripts/lib/resulttool/store.py
create mode 100644 scripts/lib/resulttool/template/test_report_full_text.txt
create mode 100755 scripts/resulttool
diff --git a/meta/lib/oeqa/files/testresults/testresults.json
b/meta/lib/oeqa/files/testresults/testresults.json
new file mode 100644
index 000..1a62155
--- /dev/null
+++ b/meta/lib/oeqa/files/testresults/testresults.json
@@ -0,0 +1,40 @@
+{
+"runtime_core-image-minimal_qemuarm_20181225195701": {
+"configuration": {
+"DISTRO": "poky",
+"HOST_DISTRO": "ubuntu-16.04",
+"IMAGE_BASENAME": "core-image-minimal",
+"IMAGE_PKGTYPE": "rpm",
+"LAYERS": {
+"meta": {
+"branch": "master",
+"commit": "801745d918e83f976c706f29669779f5b292ade3",
+"commit_count": 52782
+},
+"meta-poky": {
+"branch": "master",
+"commit": "801745d918e83f976c706f29669779f5b292ade3",
+"commit_count": 52782
+},
+"meta-yocto-bsp": {
+"branch": "master",
+"commit":
[OE-core] [PATCH 0/2 v7] test-case-mgmt
v1: Face key error from oe-git-archive Undesirable behavior when storing to multiple git branch v2: Include fix for oe-git-archive Include fix for store result to multiple git branch Improve git commit message v3: Enhance fix for oe-git-archive by using exception catch to improve code readability and easy to understand v4: Add new features, merge result files & regression analysis Add selftest to merge, store, report and regression functionalities Revise codebase for pythonic v5: Add required files for selftest testing store v6: Add regression for directory and git repository Enable regression pairing base set to multiple target sets Revise selftest testing for regression v7: Optimize regression computation for ptest results Rename entry point script to resulttool Mazliana (1): scripts/resulttool: enable manual execution and result creation Yeoh Ee Peng (1): resulttool: enable merge, store, report and regression analysis meta/lib/oeqa/files/testresults/testresults.json | 40 meta/lib/oeqa/selftest/cases/resulttooltests.py| 104 +++ scripts/lib/resulttool/__init__.py | 0 scripts/lib/resulttool/manualexecution.py | 137 ++ scripts/lib/resulttool/merge.py| 71 +++ scripts/lib/resulttool/regression.py | 208 + scripts/lib/resulttool/report.py | 113 +++ scripts/lib/resulttool/resultsutils.py | 67 +++ scripts/lib/resulttool/store.py| 110 +++ .../resulttool/template/test_report_full_text.txt | 35 scripts/resulttool | 92 + 11 files changed, 977 insertions(+) create mode 100644 meta/lib/oeqa/files/testresults/testresults.json create mode 100644 meta/lib/oeqa/selftest/cases/resulttooltests.py create mode 100644 scripts/lib/resulttool/__init__.py create mode 100755 scripts/lib/resulttool/manualexecution.py create mode 100644 scripts/lib/resulttool/merge.py create mode 100644 scripts/lib/resulttool/regression.py create mode 100644 scripts/lib/resulttool/report.py create mode 100644 scripts/lib/resulttool/resultsutils.py create mode 100644 scripts/lib/resulttool/store.py create mode 100644 scripts/lib/resulttool/template/test_report_full_text.txt create mode 100755 scripts/resulttool -- 2.7.4 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2 v7] scripts/resulttool: enable manual execution and result creation
From: Mazliana
Integrated “manualexecution” operation to resulttool scripts.
Manual execution script is a helper script to execute all manual
test cases in baseline command, which consists of user guideline
steps and the expected results. The last step will ask user to
provide their input to execute result. The input options are
passed/failed/blocked/skipped status. The result given will be
written in testresults.json including log error from the user
input and configuration if there is any.The output test result
for json file is created by using OEQA library.
The configuration part is manually key-in by the user. The system
allow user to specify how many configuration they want to add and
they need to define the required configuration name and value pair.
In QA perspective, "configuration" means the test environments and
parameters used during QA setup before testing can be carry out.
Example of configurations: image used for boot up, host machine
distro used, poky configurations, etc.
The purpose of adding the configuration is to standardize the
output test result format between automation and manual execution.
To use these scripts, first source oe environment, then run the
entry point script to look for help.
$ resulttool
To execute manual test cases, execute the below
$ resulttool manualexecution
By default testresults.json store in /tmp/log/manual/
[YOCTO #12651]
Signed-off-by: Mazliana
---
scripts/lib/resulttool/manualexecution.py | 137 ++
scripts/resulttool| 8 ++
2 files changed, 145 insertions(+)
create mode 100755 scripts/lib/resulttool/manualexecution.py
diff --git a/scripts/lib/resulttool/manualexecution.py
b/scripts/lib/resulttool/manualexecution.py
new file mode 100755
index 000..64ec581
--- /dev/null
+++ b/scripts/lib/resulttool/manualexecution.py
@@ -0,0 +1,137 @@
+# test case management tool - manual execution from testopia test cases
+#
+# Copyright (c) 2018, Intel Corporation.
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms and conditions of the GNU General Public License,
+# version 2, as published by the Free Software Foundation.
+#
+# This program is distributed in the hope it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+# more details.
+#
+import argparse
+import json
+import os
+import sys
+import datetime
+import re
+from oeqa.core.runner import OETestResultJSONHelper
+from resulttool.resultsutils import load_json_file
+
+class ManualTestRunner(object):
+def __init__(self):
+self.jdata = ''
+self.test_module = ''
+self.test_suite = ''
+self.test_cases = ''
+self.configuration = ''
+self.starttime = ''
+self.result_id = ''
+self.write_dir = ''
+
+def _get_testcases(self, file):
+self.jdata = load_json_file(file)
+self.test_cases = []
+self.test_module = self.jdata[0]['test']['@alias'].split('.', 2)[0]
+self.test_suite = self.jdata[0]['test']['@alias'].split('.', 2)[1]
+for i in self.jdata:
+self.test_cases.append(i['test']['@alias'].split('.', 2)[2])
+
+def _get_input(self, config):
+while True:
+output = input('{} = '.format(config))
+if re.match('^[a-zA-Z0-9_]+$', output):
+break
+print('Only alphanumeric and underscore are allowed. Please try
again')
+return output
+
+def _create_config(self):
+self.configuration = {}
+while True:
+try:
+conf_total = int(input('\nPlease provide how many
configuration you want to save \n'))
+break
+except ValueError:
+print('Invalid input. Please provide input as a number not
character.')
+for i in range(conf_total):
+print('-')
+print('This is configuration #%s ' % (i + 1) + '. Please provide
configuration name and its value')
+print('-')
+name_conf = self._get_input('Configuration Name')
+value_conf = self._get_input('Configuration Value')
+print('-\n')
+self.configuration[name_conf.upper()] = value_conf
+current_datetime = datetime.datetime.now()
+self.starttime = current_datetime.strftime('%Y%m%d%H%M%S')
+self.configuration['STARTTIME'] = self.starttime
+self.configuration['TEST_TYPE'] = self.test_module
+
+def _create_result_id(self):
+self.result_id = 'manual_' + self.test_module + '_' + self.starttime
+
+def _execute_test_steps(self, test_id):
+test_result = {}
+testcase_id = self.test_module + '.'
[OE-core] [PATCH 1/1 V2] base.bbclass: Check BuildStarted for HOSTTOOLS
There might be no bb.event.ConfigParsed event if bitbake server is running, so
check bb.event.BuildStarted too to make sure HOSTTOOLS_DIR exists.
Fixed:
$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
$ rm -fr tmp
$ bitbake quilt-native
ERROR: Error running gcc --version: /bin/sh: gcc: command not found
This error is caused by enable_uninative(), it runs twice (ConfigParsed and
BuildStarted), the error would happen when there is no ConfigParsed event
(no hosttools is created), but BuildStarted. This patch can fix the problem.
[YOCTO #13022]
Signed-off-by: Robert Yang
---
meta/classes/base.bbclass | 6 ++
1 file changed, 6 insertions(+)
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index f1a3c0e..8ece51f 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -228,6 +228,12 @@ python base_eventhandler() {
if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)
+
+# There might be no bb.event.ConfigParsed event if bitbake server is
+# running, so check bb.event.BuildStarted too to make sure ${HOSTTOOLS_DIR}
+# exists.
+if isinstance(e, bb.event.ConfigParsed) or \
+(isinstance(e, bb.event.BuildStarted) and not
os.path.exists(d.getVar('HOSTTOOLS_DIR'))):
# Works with the line in layer.conf which changes PATH to point here
setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS', d)
setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS_NONFATAL',
d, fatal=False)
--
2.7.4
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1 V2] base.bbclass: Check BuildStarted for HOSTTOOLS
* V2 - Tweak the position of BuildStarted a little and fix comments as Peter Kjellerstedt suggested. * V1 - Initial version // Robert The following changes since commit 430bd28093fb743ba3bd032e11e40b4dd8e5bd1f: linux-yocto: add baseline ARC support (2019-02-12 14:04:24 +) are available in the git repository at: git://git.openembedded.org/openembedded-core-contrib rbt/ht http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/ht Robert Yang (1): base.bbclass: Check BuildStarted for HOSTTOOLS meta/classes/base.bbclass | 6 ++ 1 file changed, 6 insertions(+) -- 2.7.4 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for HOSTTOOLS
On 2/14/19 11:43 AM, Peter Kjellerstedt wrote:
-Original Message-
From: openembedded-core-boun...@lists.openembedded.org On Behalf Of Robert Yang
Sent: den 14 februari 2019 04:22
To: Peter Kjellerstedt ; openembedded-
c...@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for
HOSTTOOLS
On 2/14/19 11:05 AM, Peter Kjellerstedt wrote:
-Original Message-
From: openembedded-core-boun...@lists.openembedded.org
core-boun...@lists.openembedded.org> On Behalf Of Robert Yang
Sent: den 14 februari 2019 03:59
To: openembedded-core@lists.openembedded.org
Subject: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for
HOSTTOOLS
There might be no bb.event.ConfigParsed event bitbake server is
running,
so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed
when
BuildStarted.
Fixed:
$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
$ rm -fr tmp
$ bitbake quilt-native
ERROR: Error running gcc --version: /bin/sh: gcc: command not found
This error is caused by enable_uninative(), it runs twice
(ConfigParsed
and
BuildStarted), the error would happen when there is no ConfigParsed
event
(no hosttools is created), but BuildStarted. This patch can fix the
problem.
[YOCTO #13022]
Signed-off-by: Robert Yang
---
meta/classes/base.bbclass | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index f1a3c0e..283cb0f 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -224,7 +224,11 @@ base_eventhandler[eventmask] =
"bb.event.ConfigParsed bb.event.MultiConfigParsed
python base_eventhandler() {
import bb.runqueue
-if isinstance(e, bb.event.ConfigParsed):
+# There might be no bb.event.ConfigParsed event bitbake server
is running,
+# so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR
existed when
+# BuildStarted.
+if isinstance(e, bb.event.ConfigParsed) or \
+(isinstance(e, bb.event.BuildStarted) and not
os.path.exists(d.getVar('HOSTTOOLS_DIR'))):
I don't think you want to do this here since not all of this if
statement
That can make the code simple and avoid duplicated, the only code which
may run twice is "d.setVar('BB_VERSION', bb.__version__)", I don't
think it's a problem since it is very light.
Well, true, but I still don't think you should mix them. Have one case
for what only needs to be done for bb.event.ConfigParsed, one for only
bb.event.BuildStarted and one for either.
Yeah, it's not a big problem, I will update it in V2.
should be done for the BuildStarted event. I think it is better to
add a
separate if statement just for the HOSTTOOLS part (see below). Also,
I don't
think you should be checking if the HOSTTOOLS_DIR exists, because the
contents of ${HOSTTOOLS} and ${HOSTTOOLS_NONFATAL} may have changed,
in
which case any new tools need to be added to the directory.
You don't have to worry about this since bitbake server can handle it
when HOSTTOOLS are changed in conf files, you can try this:
$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
Add HOSTTOOLS += "passwd" to conf/local.conf, and:
$ bitbake quilt-native
$ ls tmp/hosttools/ | grep passwd
passwd
The passwd will be created.
if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)
--
2.7.4
This is my suggestion for how to split the current handling of
bb.event.ConfigParsed (also note that I corrected the comment before
the new if statement):
if isinstance(e, bb.event.ConfigParsed):
if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)
# There might be no bb.event.ConfigParsed event if the bitbake server is
Thanks for the correction.
# running, so check bb.event.BuildStarted too to make sure
${HOSTTOOLS_DIR}
# exists.
if isinstance(e, bb.event.ConfigParsed) or isinstance(e,
bb.event.BuildStarted):
I'm afraid that it doesn't work since it would make hosttools generate
twice when bitbake sever is not running, which seems not a good idea.
I don't think that's a problem in practice since the generation of
It may cause a race issue when tools in tmp/hosttools are being used, but
BuildStarted removes them at the same time.
// Robert
${HOSTTOLS_DIR} should be pretty fast if it is already populated, but I
guess you could use your original if statement to avoid that:
if isinstance(e, bb.event.ConfigParsed) or \
(isinstance(e, bb.event.BuildStarted) and not
os.path.exists(d.getVar('HOSTTOOLS_DIR'))):
// Robert
# Works with the line in layer.conf which changes PATH to point here
setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS', d)
Re: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions
> -Original Message-
> From: openembedded-core-boun...@lists.openembedded.org core-boun...@lists.openembedded.org> On Behalf Of Khem Raj
> Sent: den 12 februari 2019 06:48
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install
> utility to create directories with right permissions
>
> Could fix issues where dirs with wrong permission is created with mkdir -p
>
> Signed-off-by: Khem Raj
> ---
> meta/classes/libc-package.bbclass | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes/libc-package.bbclass b/meta/classes/libc-
> package.bbclass
> index 0b4c666a74..34025635ce 100644
> --- a/meta/classes/libc-package.bbclass
> +++ b/meta/classes/libc-package.bbclass
> @@ -61,7 +61,7 @@ LOCALETREESRC ?= "${PKGD}"
> do_prep_locale_tree() {
> treedir=${WORKDIR}/locale-tree
> rm -rf $treedir
> - mkdir -p $treedir/${base_bindir} $treedir/${base_libdir}
> $treedir/${datadir} $treedir/${localedir}
> + install -d $treedir/${base_bindir} $treedir/${base_libdir}
> $treedir/${datadir} $treedir/${localedir}
This does not change anything (install -d and mkdir -p does the same thing).
> tar -cf - -C ${LOCALETREESRC}${datadir} -p i18n | tar -xf - -C
> $treedir/${datadir}
> # unzip to avoid parsing errors
> for i in $treedir/${datadir}/i18n/charmaps/*gz; do
> gunzip $i
> done
> tar -cf - -C ${LOCALETREESRC}${base_libdir} -p . | tar -xf - -C
> $treedir/${base_libdir}
This is where the problem happens. When the files are copied from
${LOCALETREESRC}, the ownership of $treedir${localedir} is changed to that
of ${LOCALETREESRC}${localedir}. However, I now realized that the problem
may be due to the fact that I have usrmerge enabled in DISTRO_FEATURES,
which causes the above line to copy things it was never meant to copy, in
this case ${LOCALETREESRC}${localedir}. I tried to just remove the line
altogether and then it built just fine. But I do not know what the
consequences are of removing it... I guess it is related to setting up an
environment for qemu to run in, but since that seems to work fine without
the files copied by the above line, maybe it is not needed? Or maybe it
is just not needed when having usrmerge in DISTRO_FEATURES? I tried to
backtrack it through the commit logs, but it was introduced with the
do_prep_locale_tree() function, so no help there.
> --
> 2.20.1
//Peter
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 4/4] cairo: upgrade 1.14.12 -> 1.16.0
* For changes, see:
https://www.cairographics.org/releases/ChangeLog.cairo-1.16.0
* Remove the backported patch.
* libcairo-trace isn't a version lib anymore, fix the packaging and
include it in perf-utils.
Signed-off-by: Anuj Mittal
---
.../cairo/0001-cairo-Fix-CVE-2017-9814.patch | 45 ---
.../{cairo_1.14.12.bb => cairo_1.16.0.bb} | 8 ++--
2 files changed, 3 insertions(+), 50 deletions(-)
delete mode 100644
meta/recipes-graphics/cairo/cairo/0001-cairo-Fix-CVE-2017-9814.patch
rename meta/recipes-graphics/cairo/{cairo_1.14.12.bb => cairo_1.16.0.bb} (92%)
diff --git
a/meta/recipes-graphics/cairo/cairo/0001-cairo-Fix-CVE-2017-9814.patch
b/meta/recipes-graphics/cairo/cairo/0001-cairo-Fix-CVE-2017-9814.patch
deleted file mode 100644
index 7d02ab9474..00
--- a/meta/recipes-graphics/cairo/cairo/0001-cairo-Fix-CVE-2017-9814.patch
+++ /dev/null
@@ -1,45 +0,0 @@
-From 042421e9e3d266ad0bb7805132041ef51ad3234d Mon Sep 17 00:00:00 2001
-From: Adrian Johnson
-Date: Wed, 16 Aug 2017 22:52:35 -0400
-Subject: [PATCH] cairo: Fix CVE-2017-9814
-
-The bug happens because in some scenarios the variable size can
-have a value of 0 at line 1288. And malloc(0) is not returning
-NULL as some people could expect:
-
-https://stackoverflow.com/questions/1073157/zero-size-malloc
-
-malloc(0) returns the smallest chunk possible. So the line 1290
-with the return is not execute. And the execution continues with
-an invalid map.
-
-Since the size is 0 the variable map is not initialized correctly
-at load_trutype_table. So, later when the variable map is accessed
-previous values from a freed chunk are used. This could allows an
-attacker to control the variable map.
-
-This patch have not merge in upstream now.
-
-Upstream-Status: Backport [https://bugs.freedesktop.org/show_bug.cgi?id=101547]
-CVE: CVE-2017-9814
-Signed-off-by: Dengke Du
- src/cairo-truetype-subset.c | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/src/cairo-truetype-subset.c b/src/cairo-truetype-subset.c
-index e3449a0..f77d11c 100644
a/src/cairo-truetype-subset.c
-+++ b/src/cairo-truetype-subset.c
-@@ -1285,7 +1285,7 @@ _cairo_truetype_reverse_cmap (cairo_scaled_font_t
*scaled_font,
- return CAIRO_INT_STATUS_UNSUPPORTED;
-
- size = be16_to_cpu (map->length);
--map = malloc (size);
-+map = _cairo_malloc (size);
- if (unlikely (map == NULL))
- return _cairo_error (CAIRO_STATUS_NO_MEMORY);
-
---
-2.8.1
-
diff --git a/meta/recipes-graphics/cairo/cairo_1.14.12.bb
b/meta/recipes-graphics/cairo/cairo_1.16.0.bb
similarity index 92%
rename from meta/recipes-graphics/cairo/cairo_1.14.12.bb
rename to meta/recipes-graphics/cairo/cairo_1.16.0.bb
index ad6745f60d..3e176930cc 100644
--- a/meta/recipes-graphics/cairo/cairo_1.14.12.bb
+++ b/meta/recipes-graphics/cairo/cairo_1.16.0.bb
@@ -24,11 +24,10 @@ DEPENDS = "fontconfig glib-2.0 libpng pixman zlib"
SRC_URI = "http://cairographics.org/releases/cairo-${PV}.tar.xz \
file://cairo-get_bitmap_surface-bsc1036789-CVE-2017-7475.diff \
- file://0001-cairo-Fix-CVE-2017-9814.patch \
"
-SRC_URI[md5sum] = "9f0db9dbfca0966be8acd682e636d165"
-SRC_URI[sha256sum] =
"8c90f00c500b2299c0a323dd9beead2a00353752b2092ead558139bd67f7bf16"
+SRC_URI[md5sum] = "f19e0353828269c22bd72e271243a552"
+SRC_URI[sha256sum] =
"5e7b29b3f113ef870d1e3ecf8adf21f923396401604bda16d44be45e66052331"
inherit autotools pkgconfig upstream-version-is-even gtk-doc multilib_script
@@ -81,7 +80,6 @@ DESCRIPTION_cairo-perf-utils = "The Cairo library performance
utilities"
FILES_${PN} = "${libdir}/libcairo.so.*"
FILES_${PN}-gobject = "${libdir}/libcairo-gobject.so.*"
FILES_${PN}-script-interpreter = "${libdir}/libcairo-script-interpreter.so.*"
-FILES_${PN}-perf-utils = "${bindir}/cairo-trace* ${libdir}/cairo/*.la
${libdir}/cairo/libcairo-trace.so.*"
-FILES_${PN}-dev += "${libdir}/cairo/*.so"
+FILES_${PN}-perf-utils = "${bindir}/cairo-trace* ${libdir}/cairo/*.la
${libdir}/cairo/libcairo-trace.so"
BBCLASSEXTEND = "native nativesdk"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/4] harfbuzz: upgrade 2.2.0 -> 2.3.1
For changes, see:
https://github.com/harfbuzz/harfbuzz/releases
Signed-off-by: Anuj Mittal
---
.../harfbuzz/{harfbuzz_2.2.0.bb => harfbuzz_2.3.1.bb} | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
rename meta/recipes-graphics/harfbuzz/{harfbuzz_2.2.0.bb => harfbuzz_2.3.1.bb}
(89%)
diff --git a/meta/recipes-graphics/harfbuzz/harfbuzz_2.2.0.bb
b/meta/recipes-graphics/harfbuzz/harfbuzz_2.3.1.bb
similarity index 89%
rename from meta/recipes-graphics/harfbuzz/harfbuzz_2.2.0.bb
rename to meta/recipes-graphics/harfbuzz/harfbuzz_2.3.1.bb
index b62082f53d..4b292d8ea9 100644
--- a/meta/recipes-graphics/harfbuzz/harfbuzz_2.2.0.bb
+++ b/meta/recipes-graphics/harfbuzz/harfbuzz_2.3.1.bb
@@ -11,8 +11,8 @@ LIC_FILES_CHKSUM =
"file://COPYING;md5=e021dd6dda6ff1e6b1044002fc662b9b \
DEPENDS = "glib-2.0 cairo fontconfig freetype"
SRC_URI = "http://www.freedesktop.org/software/harfbuzz/release/${BP}.tar.bz2;
-SRC_URI[md5sum] = "d9b8b2bc4d0cdf7f0824f4f0631d264b"
-SRC_URI[sha256sum] =
"b7ccfcbd56b970a709e8b9ea9fb46c922c606c2feef8f086fb6a8492e530f810"
+SRC_URI[md5sum] = "531de9df7c8a5405dd9c6a873fcee8c2"
+SRC_URI[sha256sum] =
"f205699d5b91374008d6f8e36c59e419ae2d9a7bb8c5d9f34041b9a5abcae468"
inherit autotools pkgconfig lib_package gtk-doc
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 3/4] pixman: upgrade 0.34.0 -> 0.38.0
For changes, see:
https://cgit.freedesktop.org/pixman/log/?qt=range=pixman-0.38.0...pixman-0.34.0
Signed-off-by: Anuj Mittal
---
.../xorg-lib/pixman/asm_include.patch | 29 ---
.../{pixman_0.34.0.bb => pixman_0.38.0.bb}| 5 ++--
2 files changed, 2 insertions(+), 32 deletions(-)
delete mode 100644 meta/recipes-graphics/xorg-lib/pixman/asm_include.patch
rename meta/recipes-graphics/xorg-lib/{pixman_0.34.0.bb => pixman_0.38.0.bb}
(89%)
diff --git a/meta/recipes-graphics/xorg-lib/pixman/asm_include.patch
b/meta/recipes-graphics/xorg-lib/pixman/asm_include.patch
deleted file mode 100644
index b52a5707d6..00
--- a/meta/recipes-graphics/xorg-lib/pixman/asm_include.patch
+++ /dev/null
@@ -1,29 +0,0 @@
-Fixes errors like
-
-Assembler messages:
-Fatal error: can't create .libs/pixman-mips-dspr2-asm.o: No such file or
directory
-
-it works with glibc because it uses gcc fixed-headers but thats not right.
-
-We move the include under C block
-
-Signed-off-by: Khem Raj
-Upstream-Status: Pending
-Index: pixman-0.32.6/pixman/pixman-private.h
-===
pixman-0.32.6.orig/pixman/pixman-private.h
-+++ pixman-0.32.6/pixman/pixman-private.h
-@@ -1,4 +1,3 @@
--#include
-
- #ifndef PIXMAN_PRIVATE_H
- #define PIXMAN_PRIVATE_H
-@@ -17,6 +16,8 @@
-
- #ifndef __ASSEMBLER__
-
-+#include
-+
- #ifndef PACKAGE
- # error config.h must be included before pixman-private.h
- #endif
diff --git a/meta/recipes-graphics/xorg-lib/pixman_0.34.0.bb
b/meta/recipes-graphics/xorg-lib/pixman_0.38.0.bb
similarity index 89%
rename from meta/recipes-graphics/xorg-lib/pixman_0.34.0.bb
rename to meta/recipes-graphics/xorg-lib/pixman_0.38.0.bb
index c290fa4614..49c3d62342 100644
--- a/meta/recipes-graphics/xorg-lib/pixman_0.34.0.bb
+++ b/meta/recipes-graphics/xorg-lib/pixman_0.38.0.bb
@@ -31,11 +31,10 @@ EXTRA_OECONF_class-nativesdk = "--disable-gtk"
SRC_URI += "\
file://0001-ARM-qemu-related-workarounds-in-cpu-features-detecti.patch \
- file://asm_include.patch \
file://0001-test-utils-Check-for-FE_INVALID-definition-before-us.patch \
"
-SRC_URI[md5sum] = "002a4fcb644ddfcb4b0e4191576a0d59"
-SRC_URI[sha256sum] =
"39ba3438f3d17c464b0cb8be006dacbca0ab5aee97ebde69fec7ecdbf85794a0"
+SRC_URI[md5sum] = "04ecad28edcc14e7eb5f7ed6df29c7a0"
+SRC_URI[sha256sum] =
"b768e3f7895ddebdc0f07478729d9cec4fe0a9d2201f828c900d67b0e5b436a8"
REQUIRED_DISTRO_FEATURES = ""
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/4] x264: update to latest on stable branch
For changes, see:
https://github.com/mirror/x264/compare/e9a5903edf8ca59ef20e6f4894c196f135af735e...0a84d986e7020f8344f00752e3600b9769cc1e85
Remove the perlnative dependency that doesn't seem to be needed and
remove the unnecessary copying of autotool scripts.
Signed-off-by: Anuj Mittal
---
meta/recipes-multimedia/x264/x264_git.bb | 8 +++-
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/meta/recipes-multimedia/x264/x264_git.bb
b/meta/recipes-multimedia/x264/x264_git.bb
index 842ea2547c..0d9d01d8a4 100644
--- a/meta/recipes-multimedia/x264/x264_git.bb
+++ b/meta/recipes-multimedia/x264/x264_git.bb
@@ -14,13 +14,13 @@ SRC_URI = "git://github.com/mirror/x264;branch=stable \
"
UPSTREAM_CHECK_COMMITS = "1"
-SRCREV = "e9a5903edf8ca59ef20e6f4894c196f135af735e"
+SRCREV = "0a84d986e7020f8344f00752e3600b9769cc1e85"
-PV = "r2854+git${SRCPV}"
+PV = "r2917+git${SRCPV}"
S = "${WORKDIR}/git"
-inherit lib_package pkgconfig perlnative
+inherit lib_package pkgconfig
X264_DISABLE_ASM = ""
X264_DISABLE_ASM_x86 = "--disable-asm"
@@ -44,8 +44,6 @@ EXTRA_OECONF = '--prefix=${prefix} \
'
do_configure() {
-install -m 0755 ${STAGING_DATADIR_NATIVE}/gnu-config/config.guess ${S}
-install -m 0755 ${STAGING_DATADIR_NATIVE}/gnu-config/config.sub ${S}
./configure ${EXTRA_OECONF}
}
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for HOSTTOOLS
> -Original Message-
> From: openembedded-core-boun...@lists.openembedded.org core-boun...@lists.openembedded.org> On Behalf Of Robert Yang
> Sent: den 14 februari 2019 04:22
> To: Peter Kjellerstedt ; openembedded-
> c...@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for
> HOSTTOOLS
>
> On 2/14/19 11:05 AM, Peter Kjellerstedt wrote:
> >> -Original Message-
> >> From: openembedded-core-boun...@lists.openembedded.org
> >> core-boun...@lists.openembedded.org> On Behalf Of Robert Yang
> >> Sent: den 14 februari 2019 03:59
> >> To: openembedded-core@lists.openembedded.org
> >> Subject: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for
> >> HOSTTOOLS
> >>
> >> There might be no bb.event.ConfigParsed event bitbake server is
> >> running,
> >> so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed
> when
> >> BuildStarted.
> >>
> >> Fixed:
> >> $ export BB_SERVER_TIMEOUT=-1
> >> $ bitbake quilt-native
> >> $ rm -fr tmp
> >> $ bitbake quilt-native
> >> ERROR: Error running gcc --version: /bin/sh: gcc: command not found
> >>
> >> This error is caused by enable_uninative(), it runs twice
> (ConfigParsed
> >> and
> >> BuildStarted), the error would happen when there is no ConfigParsed
> >> event
> >> (no hosttools is created), but BuildStarted. This patch can fix the
> >> problem.
> >>
> >> [YOCTO #13022]
> >>
> >> Signed-off-by: Robert Yang
> >> ---
> >> meta/classes/base.bbclass | 6 +-
> >> 1 file changed, 5 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> >> index f1a3c0e..283cb0f 100644
> >> --- a/meta/classes/base.bbclass
> >> +++ b/meta/classes/base.bbclass
> >> @@ -224,7 +224,11 @@ base_eventhandler[eventmask] =
> >> "bb.event.ConfigParsed bb.event.MultiConfigParsed
> >> python base_eventhandler() {
> >> import bb.runqueue
> >>
> >> -if isinstance(e, bb.event.ConfigParsed):
> >> +# There might be no bb.event.ConfigParsed event bitbake server
> is running,
> >> +# so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR
> existed when
> >> +# BuildStarted.
> >> +if isinstance(e, bb.event.ConfigParsed) or \
> >> +(isinstance(e, bb.event.BuildStarted) and not
> os.path.exists(d.getVar('HOSTTOOLS_DIR'))):
> >
> > I don't think you want to do this here since not all of this if
> statement
>
> That can make the code simple and avoid duplicated, the only code which
> may run twice is "d.setVar('BB_VERSION', bb.__version__)", I don't
> think it's a problem since it is very light.
Well, true, but I still don't think you should mix them. Have one case
for what only needs to be done for bb.event.ConfigParsed, one for only
bb.event.BuildStarted and one for either.
> > should be done for the BuildStarted event. I think it is better to
> add a
> > separate if statement just for the HOSTTOOLS part (see below). Also,
> I don't
> > think you should be checking if the HOSTTOOLS_DIR exists, because the
> > contents of ${HOSTTOOLS} and ${HOSTTOOLS_NONFATAL} may have changed,
> in
> > which case any new tools need to be added to the directory.
>
> You don't have to worry about this since bitbake server can handle it
> when HOSTTOOLS are changed in conf files, you can try this:
>
> $ export BB_SERVER_TIMEOUT=-1
> $ bitbake quilt-native
>
> Add HOSTTOOLS += "passwd" to conf/local.conf, and:
>
> $ bitbake quilt-native
> $ ls tmp/hosttools/ | grep passwd
> passwd
>
> The passwd will be created.
>
> >
> >> if not d.getVar("NATIVELSBSTRING", False):
> >> d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
> >> d.setVar('BB_VERSION', bb.__version__)
> >> --
> >> 2.7.4
> >
> > This is my suggestion for how to split the current handling of
> > bb.event.ConfigParsed (also note that I corrected the comment before
> > the new if statement):
> >
> > if isinstance(e, bb.event.ConfigParsed):
> > if not d.getVar("NATIVELSBSTRING", False):
> > d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
> > d.setVar('BB_VERSION', bb.__version__)
> >
> > # There might be no bb.event.ConfigParsed event if the bitbake server
> > is
>
> Thanks for the correction.
>
> > # running, so check bb.event.BuildStarted too to make sure
> > ${HOSTTOOLS_DIR}
> > # exists.
> > if isinstance(e, bb.event.ConfigParsed) or isinstance(e,
> > bb.event.BuildStarted):
>
> I'm afraid that it doesn't work since it would make hosttools generate
> twice when bitbake sever is not running, which seems not a good idea.
I don't think that's a problem in practice since the generation of
${HOSTTOLS_DIR} should be pretty fast if it is already populated, but I
guess you could use your original if statement to avoid that:
if isinstance(e, bb.event.ConfigParsed) or \
(isinstance(e, bb.event.BuildStarted) and not
os.path.exists(d.getVar('HOSTTOOLS_DIR'))):
> // Robert
>
Re: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for HOSTTOOLS
On 2/14/19 11:05 AM, Peter Kjellerstedt wrote:
-Original Message-
From: openembedded-core-boun...@lists.openembedded.org On Behalf Of Robert Yang
Sent: den 14 februari 2019 03:59
To: openembedded-core@lists.openembedded.org
Subject: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for
HOSTTOOLS
There might be no bb.event.ConfigParsed event bitbake server is
running,
so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when
BuildStarted.
Fixed:
$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
$ rm -fr tmp
$ bitbake quilt-native
ERROR: Error running gcc --version: /bin/sh: gcc: command not found
This error is caused by enable_uninative(), it runs twice (ConfigParsed
and
BuildStarted), the error would happen when there is no ConfigParsed
event
(no hosttools is created), but BuildStarted. This patch can fix the
problem.
[YOCTO #13022]
Signed-off-by: Robert Yang
---
meta/classes/base.bbclass | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index f1a3c0e..283cb0f 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -224,7 +224,11 @@ base_eventhandler[eventmask] =
"bb.event.ConfigParsed bb.event.MultiConfigParsed
python base_eventhandler() {
import bb.runqueue
-if isinstance(e, bb.event.ConfigParsed):
+# There might be no bb.event.ConfigParsed event bitbake server is running,
+# so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when
+# BuildStarted.
+if isinstance(e, bb.event.ConfigParsed) or \
+(isinstance(e, bb.event.BuildStarted) and not
os.path.exists(d.getVar('HOSTTOOLS_DIR'))):
I don't think you want to do this here since not all of this if statement
That can make the code simple and avoid duplicated, the only code which
may run twice is "d.setVar('BB_VERSION', bb.__version__)", I don't think
it's a problem since it is very light.
should be done for the BuildStarted event. I think it is better to add a
separate if statement just for the HOSTTOOLS part (see below). Also, I don't
think you should be checking if the HOSTTOOLS_DIR exists, because the
contents of ${HOSTTOOLS} and ${HOSTTOOLS_NONFATAL} may have changed, in
which case any new tools need to be added to the directory.
You don't have to worry about this since bitbake server can handle it when
HOSTTOOLS are changed in conf files, you can try this:
$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
Add HOSTTOOLS += "passwd" to conf/local.conf, and:
$ bitbake quilt-native
$ ls tmp/hosttools/ | grep passwd
passwd
The passwd will be created.
if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)
--
2.7.4
This is my suggestion for how to split the current handling of
bb.event.ConfigParsed (also note that I corrected the comment before the new
if statement):
if isinstance(e, bb.event.ConfigParsed):
if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)
# There might be no bb.event.ConfigParsed event if the bitbake server is
Thanks for the correction.
# running, so check bb.event.BuildStarted too to make sure ${HOSTTOOLS_DIR}
# exists.
if isinstance(e, bb.event.ConfigParsed) or isinstance(e,
bb.event.BuildStarted):
I'm afraid that it doesn't work since it would make hosttools generate
twice when bitbake sever is not running, which seems not a good idea.
// Robert
# Works with the line in layer.conf which changes PATH to point here
setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS', d)
setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS_NONFATAL',
d, fatal=False)
//Peter
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for HOSTTOOLS
> -Original Message-
> From: openembedded-core-boun...@lists.openembedded.org core-boun...@lists.openembedded.org> On Behalf Of Robert Yang
> Sent: den 14 februari 2019 03:59
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for
> HOSTTOOLS
>
> There might be no bb.event.ConfigParsed event bitbake server is
> running,
> so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when
> BuildStarted.
>
> Fixed:
> $ export BB_SERVER_TIMEOUT=-1
> $ bitbake quilt-native
> $ rm -fr tmp
> $ bitbake quilt-native
> ERROR: Error running gcc --version: /bin/sh: gcc: command not found
>
> This error is caused by enable_uninative(), it runs twice (ConfigParsed
> and
> BuildStarted), the error would happen when there is no ConfigParsed
> event
> (no hosttools is created), but BuildStarted. This patch can fix the
> problem.
>
> [YOCTO #13022]
>
> Signed-off-by: Robert Yang
> ---
> meta/classes/base.bbclass | 6 +-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index f1a3c0e..283cb0f 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -224,7 +224,11 @@ base_eventhandler[eventmask] =
> "bb.event.ConfigParsed bb.event.MultiConfigParsed
> python base_eventhandler() {
> import bb.runqueue
>
> -if isinstance(e, bb.event.ConfigParsed):
> +# There might be no bb.event.ConfigParsed event bitbake server is
> running,
> +# so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when
> +# BuildStarted.
> +if isinstance(e, bb.event.ConfigParsed) or \
> +(isinstance(e, bb.event.BuildStarted) and not
> os.path.exists(d.getVar('HOSTTOOLS_DIR'))):
I don't think you want to do this here since not all of this if statement
should be done for the BuildStarted event. I think it is better to add a
separate if statement just for the HOSTTOOLS part (see below). Also, I don't
think you should be checking if the HOSTTOOLS_DIR exists, because the
contents of ${HOSTTOOLS} and ${HOSTTOOLS_NONFATAL} may have changed, in
which case any new tools need to be added to the directory.
> if not d.getVar("NATIVELSBSTRING", False):
> d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
> d.setVar('BB_VERSION', bb.__version__)
> --
> 2.7.4
This is my suggestion for how to split the current handling of
bb.event.ConfigParsed (also note that I corrected the comment before the new
if statement):
if isinstance(e, bb.event.ConfigParsed):
if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)
# There might be no bb.event.ConfigParsed event if the bitbake server is
# running, so check bb.event.BuildStarted too to make sure ${HOSTTOOLS_DIR}
# exists.
if isinstance(e, bb.event.ConfigParsed) or isinstance(e,
bb.event.BuildStarted):
# Works with the line in layer.conf which changes PATH to point here
setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS', d)
setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS_NONFATAL', d,
fatal=False)
//Peter
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/1] base.bbclass: Check BuildStarted for HOSTTOOLS
There might be no bb.event.ConfigParsed event bitbake server is running,
so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when
BuildStarted.
Fixed:
$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
$ rm -fr tmp
$ bitbake quilt-native
ERROR: Error running gcc --version: /bin/sh: gcc: command not found
This error is caused by enable_uninative(), it runs twice (ConfigParsed and
BuildStarted), the error would happen when there is no ConfigParsed event
(no hosttools is created), but BuildStarted. This patch can fix the problem.
[YOCTO #13022]
Signed-off-by: Robert Yang
---
meta/classes/base.bbclass | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index f1a3c0e..283cb0f 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -224,7 +224,11 @@ base_eventhandler[eventmask] = "bb.event.ConfigParsed
bb.event.MultiConfigParsed
python base_eventhandler() {
import bb.runqueue
-if isinstance(e, bb.event.ConfigParsed):
+# There might be no bb.event.ConfigParsed event bitbake server is running,
+# so check bb.event.BuildStarted to make sure HOSTTOOLS_DIR existed when
+# BuildStarted.
+if isinstance(e, bb.event.ConfigParsed) or \
+(isinstance(e, bb.event.BuildStarted) and not
os.path.exists(d.getVar('HOSTTOOLS_DIR'))):
if not d.getVar("NATIVELSBSTRING", False):
d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
d.setVar('BB_VERSION', bb.__version__)
--
2.7.4
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] base.bbclass: Check BuildStarted for HOSTTOOLS
The following changes since commit 430bd28093fb743ba3bd032e11e40b4dd8e5bd1f: linux-yocto: add baseline ARC support (2019-02-12 14:04:24 +) are available in the git repository at: git://git.openembedded.org/openembedded-core-contrib rbt/ht http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/ht Robert Yang (1): base.bbclass: Check BuildStarted for HOSTTOOLS meta/classes/base.bbclass | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) -- 2.7.4 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] icecc-env: don't raise error when icecc not installed
When icecc is not installed, the `which icecc` command in icecc-env.sh returns nonzero. This happens when environment is being sourced. When the terminal has `set -e`, the whole script fails and terminal is closed. Fix this by ignoring errors from the which command. Signed-off-by: Adam Trhon --- .../icecc-toolchain/icecc-toolchain/icecc-env.sh| 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh b/meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh index 81b3018bb6..0dccc41c8a 100644 --- a/meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh +++ b/meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh @@ -21,7 +21,7 @@ # if [ -z "$ICECC_PATH" ]; then -ICECC_PATH=$(which icecc 2> /dev/null) +ICECC_PATH=$(which icecc 2> /dev/null || true) fi if [ -n "$ICECC_PATH" ]; then -- 2.20.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] systemd: RDEPENDS on util-linux-umount
On 2/13/19 5:00 AM, Jonas Bonn wrote: On 11/02/2019 13:15, mikko.rap...@bmw.de wrote: On Mon, Feb 11, 2019 at 12:08:46PM +, André Draszik wrote: Please ignore this patch. Looks like a red-herring. Sorry for the noise. FWIW, I would like to see this patch merged. Had some issues in the past with busybox umount and added same change as a bbappend. I'm also carrying a patch locally for this with the following explanation: systemd passes the "-c" parameter (do not canonicalize) to the umount command and busybox's umount does not understand this parameter. With the current busybox do you really still see a problem since busybox umount has been told to ignore "-c" now: https://git.busybox.net/busybox/commit/?id=426134128112738c97a665170b21153ef0764b7d umount: ignore -c "-c, --no-canonicalize: Do not canonicalize paths." As busybox doesn't canonicalize paths in the first place it is safe to ignore this option. See https://github.com/systemd/systemd/issues/7786 Signed-off-by: Shawn Landden Signed-off-by: Denys Vlasenko and that's released in: $ git tag --contains 426134128112738c97a665170b21153ef0764b7d 1_29_0 1_29_1 1_29_2 1_29_3 1_30_0 and we have: meta/recipes-core/busybox/busybox_1.29.3.bb -- # Randy MacLeod # Wind River Linux -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp
Peter did you have a chance to try
https://patchwork.openembedded.org/patch/158661/
On Sun, Feb 10, 2019 at 6:09 PM Khem Raj wrote:
>
> On Sun, Feb 10, 2019 at 3:32 PM Peter Kjellerstedt
> wrote:
> >
> > > -Original Message-
> > > From: openembedded-core-boun...@lists.openembedded.org > > core-boun...@lists.openembedded.org> On Behalf Of Peter Kjellerstedt
> > > Sent: den 10 februari 2019 07:25
> > > To: Richard Purdie ; Khem Raj
> > > ; openembedded-core@lists.openembedded.org
> > > Subject: Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using
> > > install utility instead of cp
> > >
> > > > -Original Message-
> > > > From: openembedded-core-boun...@lists.openembedded.org > > > core-boun...@lists.openembedded.org> On Behalf Of Peter Kjellerstedt
> > > > Sent: den 10 februari 2019 02:29
> > > > To: Richard Purdie ; Khem Raj
> > > > ; openembedded-core@lists.openembedded.org
> > > > Subject: Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using
> > > > install utility instead of cp
> > > >
> > > > > -Original Message-
> > > > > From: openembedded-core-boun...@lists.openembedded.org
> > > > > > > core-boun...@lists.openembedded.org> On Behalf Of Richard Purdie
> > > > > Sent: den 7 februari 2019 10:01
> > > > > To: Khem Raj ; openembedded-
> > > > > c...@lists.openembedded.org
> > > > > Subject: Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install
> > > using
> > > > > install utility instead of cp
> > > > >
> > > > > On Wed, 2019-02-06 at 16:35 -0800, Khem Raj wrote:
> > > > > > This has been a constant source of trouble for build failures due
> > > > to
> > > > > host-user-contaminated QA
> > > > > > errors of sort
> > > > > >
> > > > > > ERROR: QA Issue: glibc-locale: /glibc-binary-localedata-ca-
> > > > > es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY is owned by
> > > uid
> > > > > 3004, which is the same as the user running bitbake. This may be
> > > due
> > > > to
> > > > > host contamination [host-user-contaminated]
> > > > > >
> > > > > > So far we have tried to mould cp command into not carrying the
> > > > build
> > > > > > user permissions into install area but it is never entirely fixed
> > > > > since
> > > > > > the issue keeps popping up in various scenes
> > > > > >
> > > > > > This patch replaces use of cp with install utility and specifies
> > > > > install
> > > > > > mode for files explcitly
> > > > > >
> > > > > > Signed-off-by: Khem Raj
> > > > > > ---
> > > > > > meta/recipes-core/glibc/glibc-locale.inc | 44 ++
> > > --
> > > > --
> > > > > --
> > > > > > 1 file changed, 25 insertions(+), 19 deletions(-)
> > > > > >
> > > > > > diff --git a/meta/recipes-core/glibc/glibc-locale.inc
> > > > b/meta/recipes-
> > > > > core/glibc/glibc-locale.inc
> > > > > > index 6384f9cbf1..9b256a5108 100644
> > > > > > --- a/meta/recipes-core/glibc/glibc-locale.inc
> > > > > > +++ b/meta/recipes-core/glibc/glibc-locale.inc
> > > > > > @@ -72,27 +72,33 @@ FILES_localedef = "${bindir}/localedef"
> > > > > > LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-
> > > > > locale"
> > > > > >
> > > > > > do_install () {
> > > > > > - mkdir -p ${D}${bindir} ${D}${datadir}
> > > > > > - if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> > > > > > - cp -R --no-dereference --preserve=mode,links
> > > > > ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
> > > > > > - fi
> > > > > > - if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
> > > > > > - mkdir -p ${D}${localedir}
> > > > > > - cp -R --no-dereference --preserve=mode,links
> > > > > ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
> > > > > > - fi
> > > > > > +install -d ${D}${bindir}
> > > > > > +find "${LOCALETREESRC}/${bindir}" -maxdepth 1 -type f \
> > > > > > +-exec install -m 0755 -t "${D}${bindir}" {} \;
> > > > > > +
> > > > > > +for d in . $(find "${LOCALETREESRC}/${localedir}" -type
> > > d
> > > > -
> > > > > printf '%P ') ; do
> > > > > > +install -d "${D}${localedir}/$d"
> > > > > > +find "${LOCALETREESRC}/${localedir}/$d" -
> > > maxdepth
> > > > 1
> > > > > -type f \
> > > > > > +-exec install -m 0644 -t "${D}${localedir}/$d"
> > > {}
> > > > \;
> > > > > > +done
> > > > > > if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
> > > > > > - mkdir -p ${D}${libdir}
> > > > > > - if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
> > > > > > - cp -R --no-dereference --
> > > > > preserve=mode,links ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
> > > > > > - fi
> > > > > > - if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
> > > > > > - cp -R --no-dereference --
> > > > > preserve=mode,links ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
> > > > > > - fi
> > > > > > - fi
> > > >
[OE-core] [PATCH] classes/license_image.bbclass: Fix rootfs license file permissions
Fixes up the permissions on the license files when they are put on the
target file system so that they are readable by everyone. Previously,
they would have inherited whatever permissions the file had in the
recipe, which may not have been appropriate.
[YOCTO #13175]
Signed-off-by: Joshua Watt
---
meta/classes/license_image.bbclass | 12 +---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/meta/classes/license_image.bbclass
b/meta/classes/license_image.bbclass
index 6515ee8278..ba95ced3e6 100644
--- a/meta/classes/license_image.bbclass
+++ b/meta/classes/license_image.bbclass
@@ -37,6 +37,7 @@ python license_create_manifest() {
def write_license_files(d, license_manifest, pkg_dic, rootfs=True):
import re
+import stat
bad_licenses = (d.getVar("INCOMPATIBLE_LICENSE") or "").split()
bad_licenses = map(lambda l: canonical_license(d, l), bad_licenses)
@@ -146,12 +147,17 @@ def write_license_files(d, license_manifest, pkg_dic,
rootfs=True):
continue
os.link(pkg_license, pkg_rootfs_license)
-# Fixup file ownership
+# Fixup file ownership and permissions
for walkroot, dirs, files in os.walk(rootfs_license_dir):
for f in files:
-os.lchown(os.path.join(walkroot, f), 0, 0)
+p = os.path.join(walkroot, f)
+os.lchown(p, 0, 0)
+if not os.path.islink(p):
+os.chmod(p, stat.S_IRUSR | stat.S_IWUSR | stat.S_IRGRP
| stat.S_IROTH)
for dir in dirs:
-os.lchown(os.path.join(walkroot, dir), 0, 0)
+p = os.path.join(walkroot, dir)
+os.lchown(p, 0, 0)
+os.chmod(p, stat.S_IRWXU | stat.S_IRGRP | stat.S_IXGRP |
stat.S_IROTH | stat.S_IXOTH)
--
2.20.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] gl support for qemuaarch64
Thanks Alex, Yes i know, I'm targeting for arm though. I see '3d acceleration enabled' so it was pointing me to its availability still without KVM. I was finally able to have weston and its gl examples working for aarch64 and performance seems reasonable. I'm still evaluating it though. Br, Joao On Wed, Feb 13, 2019 at 10:52 AM Alexander Kanavin wrote: > On Tue, 12 Feb 2019 at 18:54, João Rasta wrote: > > I was trying to build virgl-gtk branch for ARM qemu target, but I'm not > completely sure if it's possible as it generally seems to require KVM. > > > > qemu-system-aarch64 has the virtio-gpu-pci device, which i'd assume to > support GL forwarding to the host if compiled with libvirglrenderer. I've > also been able to run a cross-compiled image reporting [drm] 3d > Acceleration enabled. However, I was not able to start any graphics > application on the guest. > > > > Could someone pls clarify this? > > First, do you need specifically qemuarm64? qemux86_64 is significantly > faster because it has kvm. I never tried arm or other emulation > targets because of that, it's all tested with x86*. > > The tests you need to do are: > a) run 'dmesg | grep drm'. What do you see? > > b) add kmscube to your image and run that. There is a demo video I > made so you can see how it works: > https://www.youtube.com/watch?v=0YBuku3JESc=youtu.be > > Alex > -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] gstreamer: improve metadata dependencies
Instead of maintaining its own copy of the 'system' packages, reuse
oe.utils.packages_filter_out_system().
This stops ${PN}-src being added to the meta package, which meant anything
depending on the meta package was also pulling in the sources.
Signed-off-by: Ross Burton
---
meta/recipes-multimedia/gstreamer/gst-plugins-package.inc | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/meta/recipes-multimedia/gstreamer/gst-plugins-package.inc
b/meta/recipes-multimedia/gstreamer/gst-plugins-package.inc
index c995e29e09d..6781e98c319 100644
--- a/meta/recipes-multimedia/gstreamer/gst-plugins-package.inc
+++ b/meta/recipes-multimedia/gstreamer/gst-plugins-package.inc
@@ -15,17 +15,17 @@ python split_gstreamer10_packages () {
python set_metapkg_rdepends () {
import os
+import oe.utils
pn = d.getVar('PN')
metapkg = pn + '-meta'
d.setVar('ALLOW_EMPTY_' + metapkg, "1")
d.setVar('FILES_' + metapkg, "")
-blacklist = [ pn, pn + '-locale', pn + '-dev', pn + '-dbg', pn + '-doc',
pn + '-meta' ]
+blacklist = [ pn, pn + '-meta' ]
metapkg_rdepends = []
-packages = d.getVar('PACKAGES').split()
pkgdest = d.getVar('PKGDEST')
-for pkg in packages[1:]:
-if not pkg in blacklist and not pkg in metapkg_rdepends and not
pkg.endswith('-dev') and not pkg.endswith('-dbg') and not pkg.count('locale')
and not pkg.count('-staticdev'):
+for pkg in oe.utils.packages_filter_out_system(d):
+if pkg not in blacklist and pkg not in metapkg_rdepends:
# See if the package is empty by looking at the contents of its
PKGDEST subdirectory.
# If this subdirectory is empty, then the package is.
# Empty packages do not get added to the meta package's RDEPENDS
--
2.11.0
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] utils: add -src to system package name blacklist
oe.utils.packages_filter_out_system() returns PACKAGES after removing "system"
packages but it doesn't handle ${PN}-src as generated by
PACKAGE_DEBUG_SPLIT_STYLE=debug-with-srcpkg.
Signed-off-by: Ross Burton
---
meta/lib/oe/utils.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index 7b574ffd30d..0c1d48a209e 100644
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -214,7 +214,7 @@ def packages_filter_out_system(d):
PN-dbg PN-doc PN-locale-eb-gb removed.
"""
pn = d.getVar('PN')
-blacklist = [pn + suffix for suffix in ('', '-dbg', '-dev', '-doc',
'-locale', '-staticdev')]
+blacklist = [pn + suffix for suffix in ('', '-dbg', '-dev', '-doc',
'-locale', '-staticdev', '-src')]
localepkg = pn + "-locale-"
pkgs = []
--
2.11.0
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] gl support for qemuaarch64
On Tue, 12 Feb 2019 at 18:54, João Rasta wrote: > I was trying to build virgl-gtk branch for ARM qemu target, but I'm not > completely sure if it's possible as it generally seems to require KVM. > > qemu-system-aarch64 has the virtio-gpu-pci device, which i'd assume to > support GL forwarding to the host if compiled with libvirglrenderer. I've > also been able to run a cross-compiled image reporting [drm] 3d Acceleration > enabled. However, I was not able to start any graphics application on the > guest. > > Could someone pls clarify this? First, do you need specifically qemuarm64? qemux86_64 is significantly faster because it has kvm. I never tried arm or other emulation targets because of that, it's all tested with x86*. The tests you need to do are: a) run 'dmesg | grep drm'. What do you see? b) add kmscube to your image and run that. There is a demo video I made so you can see how it works: https://www.youtube.com/watch?v=0YBuku3JESc=youtu.be Alex -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] systemd: RDEPENDS on util-linux-umount
On 11/02/2019 13:15, mikko.rap...@bmw.de wrote: On Mon, Feb 11, 2019 at 12:08:46PM +, André Draszik wrote: Please ignore this patch. Looks like a red-herring. Sorry for the noise. FWIW, I would like to see this patch merged. Had some issues in the past with busybox umount and added same change as a bbappend. I'm also carrying a patch locally for this with the following explanation: systemd passes the "-c" parameter (do not canonicalize) to the umount command and busybox's umount does not understand this parameter. /Jonas -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
