Re: [oe] [PATCH] yocto-compat-layer: improve error handling in signature creation

[Patrick Ohly]

On Wed, 2017-03-15 at 10:48 +0100, Patrick Ohly wrote:
> When "bitbake -k -S none world" failed, the error printed by
> yocto-compat-layer.py contained the stack trace multiple times and did not
> contain the stderr output from bitbake, making the error hard to understand
> and debug

Sorry, wrong list. I closed the patch in patchwork, too.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.



-- 
___
Openembedded-devel mailing list
Openembedded-devel@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-devel

[oe] [PATCH] yocto-compat-layer: improve error handling in signature creation

[Patrick Ohly]

When "bitbake -k -S none world" failed, the error printed by
yocto-compat-layer.py contained the stack trace multiple times and did not
contain the stderr output from bitbake, making the error hard to understand
and debug:

  INFO: ==
  INFO: ERROR: test_signatures (common.CommonCompatLayer)
  INFO: --
  INFO: Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in 
get_signatures
  stderr=subprocess.PIPE)
File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
  raise CalledProcessError(retcode, process.args, output=output)
  subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned 
non-zero exit status 1

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in 
test_signatures
  curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 149, in 
get_signatures
  raise RuntimeError(msg)
  RuntimeError: Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in 
get_signatures
  stderr=subprocess.PIPE)
File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
  raise CalledProcessError(retcode, process.args, output=output)
  subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned 
non-zero exit status 1

  Loading cache...done.
  Loaded 1328 entries from dependency cache.
  NOTE: Resolving any missing task queue dependencies
  NOTE: Runtime target 'zlib-qat' is unbuildable, removing...
  Missing or unbuildable dependency chain was: ['zlib-qat']
  ...
  Summary: There were 5 ERROR messages shown, returning a non-zero exit code.

The yocto-compat-layer.log was incomplete, it only had the first part
without the command output.

stderr was missing due to stderr=subprocess.PIPE.

Instead of the complicated try/except construct it's better to check
the return code ourselves and raise just a single exception. The
output (both on stderr and in the yocto-compat-layer.log) now is:

  INFO: ==
  INFO: ERROR: test_signatures (common.CommonCompatLayer)
  INFO: --
  INFO: Traceback (most recent call last):
File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in 
test_signatures
  curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 147, in 
get_signatures
  raise RuntimeError(msg)
  RuntimeError: Generating signatures failed. This might be due to some parse 
error and/or general layer incompatibilities.
  Command: bitbake -k -S none world
  Output:
  Loading cache...done.
  Loaded 1328 entries from dependency cache.
  NOTE: Resolving any missing task queue dependencies
  ERROR: Nothing PROVIDES 'qat16' (but 
/fast/work/meta-intel/common/recipes-extended/openssl-qat/openssl-qat_0.4.9-009.bb
 DEPENDS on or otherwise requires it)
  ERROR: qat16 was skipped: incompatible with machine qemux86 (not in 
COMPATIBLE_MACHINE)
  ...
  Missing or unbuildable dependency chain was: ['openssl-qat-dev']
  ...
  Summary: There were 5 ERROR messages shown, returning a non-zero exit code.

Signed-off-by: Patrick Ohly 
---
 scripts/lib/compatlayer/__init__.py | 20 +---
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/scripts/lib/compatlayer/__init__.py 
b/scripts/lib/compatlayer/__init__.py
index b3a166aa9ab..a7eb8625310 100644
--- a/scripts/lib/compatlayer/__init__.py
+++ b/scripts/lib/compatlayer/__init__.py
@@ -135,17 +135,15 @@ def get_signatures(builddir, failsafe=False):
 
 sigs = {}
 
-try:
-cmd = 'bitbake '
-if failsafe:
-cmd += '-k '
-cmd += '-S none world'
-output = subprocess.check_output(cmd, shell=True,
-stderr=subprocess.PIPE)
-except subprocess.CalledProcessError as e:
-import traceback
-exc = traceback.format_exc()
-msg = '%s\n%s\n' % (exc, e.output.decode('utf-8'))
+cmd = 'bitbake '
+if failsafe:
+cmd += '-k '
+cmd += '-S none world'
+p = subprocess.Popen(cmd, shell=True,
+ stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+output, _ = p.communicate()
+if p.returncode:
+msg = "Generating signatures failed. This might be due to some parse 
error and/or general layer incompatibilities.\nCommand: %s\nOutput:\n%s" % 
(cmd, output.decode('utf-8'))
 raise RuntimeError(msg)
 sigs_file = os.path.join(builddir, 'locked-sigs.inc')
 
--