Re: [OE-core] [PATCH 1/3] lib/oe/package: Improve filedeprunner subprocess handling

[Richard Purdie]

On Fri, 2017-09-01 at 16:32 +, Peter Kjellerstedt wrote:
> Even with these changes applied, I still see texinfo succeed to
> build 
> even when it should not (i.e., without having applied the patch to 
> package_deb.bbclass that I just sent).

Right, there are some nasty bugs with python's multiprocessing
exception handling in a few places. I've sent some patches for some
others which I ran into with other problems, Leo should have a patch
for this specific issue shortly. Thanks for the colon fix!

Cheers,

Richard
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/3] lib/oe/package: Improve filedeprunner subprocess handling

[Peter Kjellerstedt]

> -Original Message-
> From: openembedded-core-boun...@lists.openembedded.org
> [mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf Of
> Richard Purdie
> Sent: den 1 september 2017 16:21
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH 1/3] lib/oe/package: Improve filedeprunner
> subprocess handling
> 
> Currently the exit code of the spawned program isn't checked so it can
> fail and the do_package task will continue merrily upon its way.
> 
> Use subprocess.check_output() to ensure we check the exit code and
> redirect stderr to stdout so if it fails, we see the error output.
> 
> We can then drop the existing exception handling as the subprocess
> exception gives a much better error.
> 
> Signed-off-by: Richard Purdie <richard.pur...@linuxfoundation.org>
> ---
>  meta/lib/oe/package.py | 11 +++
>  1 file changed, 3 insertions(+), 8 deletions(-)
> 
> diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
> index fcee389..1e5c3aa 100644
> --- a/meta/lib/oe/package.py
> +++ b/meta/lib/oe/package.py
> @@ -176,8 +176,7 @@ def filedeprunner(arg):
> 
>  def process_deps(pipe, pkg, pkgdest, provides, requires):
>  file = None
> -for line in pipe:
> -line = line.decode("utf-8")
> +for line in pipe.split("\n"):
> 
>  m = file_re.match(line)
>  if m:
> @@ -226,12 +225,8 @@ def filedeprunner(arg):
> 
>  return provides, requires
> 
> -try:
> -dep_popen = subprocess.Popen(shlex.split(rpmdeps) + pkgfiles, 
> stdout=subprocess.PIPE)
> -provides, requires = process_deps(dep_popen.stdout, pkg, pkgdest, 
> provides, requires)
> -except OSError as e:
> -bb.error("rpmdeps: '%s' command failed, '%s'" % 
> (shlex.split(rpmdeps) + pkgfiles, e))
> -raise e
> +output = subprocess.check_output(shlex.split(rpmdeps) + pkgfiles, 
> stderr=subprocess.STDOUT).decode("utf-8")
> +provides, requires = process_deps(output, pkg, pkgdest, provides, 
> requires)
> 
>  return (pkg, provides, requires)
> 
> --
> 2.7.4

Even with these changes applied, I still see texinfo succeed to build 
even when it should not (i.e., without having applied the patch to 
package_deb.bbclass that I just sent).

//Peter

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] lib/oe/package: Improve filedeprunner subprocess handling

[Richard Purdie]

Currently the exit code of the spawned program isn't checked so it can
fail and the do_package task will continue merrily upon its way.

Use subprocess.check_output() to ensure we check the exit code and
redirect stderr to stdout so if it fails, we see the error output.

We can then drop the existing exception handling as the subprocess
exception gives a much better error.

Signed-off-by: Richard Purdie 
---
 meta/lib/oe/package.py | 11 +++
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index fcee389..1e5c3aa 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -176,8 +176,7 @@ def filedeprunner(arg):
 
 def process_deps(pipe, pkg, pkgdest, provides, requires):
 file = None
-for line in pipe:
-line = line.decode("utf-8")
+for line in pipe.split("\n"):
 
 m = file_re.match(line)
 if m:
@@ -226,12 +225,8 @@ def filedeprunner(arg):
 
 return provides, requires
 
-try:
-dep_popen = subprocess.Popen(shlex.split(rpmdeps) + pkgfiles, 
stdout=subprocess.PIPE)
-provides, requires = process_deps(dep_popen.stdout, pkg, pkgdest, 
provides, requires)
-except OSError as e:
-bb.error("rpmdeps: '%s' command failed, '%s'" % (shlex.split(rpmdeps) 
+ pkgfiles, e))
-raise e
+output = subprocess.check_output(shlex.split(rpmdeps) + pkgfiles, 
stderr=subprocess.STDOUT).decode("utf-8")
+provides, requires = process_deps(output, pkg, pkgdest, provides, requires)
 
 return (pkg, provides, requires)
 
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core