Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Hi Andrei,
On Monday, November 11, 2019 19:41, Andrei Gherzan wrote:
> Hi,
>
> On 11/11/2019 12:29, Grygorii Tertychnyi (gtertych) wrote:
>> Hi Andrei,
>>
>> From: Andrei Gherzan
>> Sent: Monday, November 11, 2019 13:18
>> Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
>>
>>>> Not sure I understand. Archiver class does not interpret "include"
>>>> directive.
>>>> It just parses text files. The regular expression looks correct:
>>>>
>>>> These lines:
>>>>
>>>> 440 elif include_re.match(line):
>>>> 441 incfile = include_re.match(line).group(1)
>>>>
>>>> put "${...}" _string_ into "incfile" variable. So, "incfile" is not None
>>>> at "this stage.
>>
>>> "${...}" it's already expanded to a white-space. So in that case it
>>> matches "include ".
>>
>> No. "incfile" is not expanded here, it actually contains "${...}", e.g.
>> ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
>
> Give me a simple way to reproduce this.
1. Put two lines to local.conf
% cat <> conf/local.conf
PACKAGECONFIG_remove_pn-perf = "scripting"
INHERIT += "archiver"
CONF
2. Run do_ar_recipe for perf
% bitbake -c ar_recipe perf
3. Observe the output error:
Exception: IsADirectoryError: [Errno 21] Is a directory
--
Grygorii
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Hi,
On 11/11/2019 12:29, Grygorii Tertychnyi (gtertych) wrote:
Hi Andrei,
From: Andrei Gherzan
Sent: Monday, November 11, 2019 13:18
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Not sure I understand. Archiver class does not interpret "include" directive.
It just parses text files. The regular expression looks correct:
These lines:
440 elif include_re.match(line):
441 incfile = include_re.match(line).group(1)
put "${...}" _string_ into "incfile" variable. So, "incfile" is not None at
"this stage.
"${...}" it's already expanded to a white-space. So in that case it
matches "include ".
No. "incfile" is not expanded here, it actually contains "${...}", e.g.
${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
Give me a simple way to reproduce this.
--
Andrei Gherzan
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Hi Andrei,
From: Andrei Gherzan
Sent: Monday, November 11, 2019 13:18
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
>> Not sure I understand. Archiver class does not interpret "include" directive.
>> It just parses text files. The regular expression looks correct:
>>
>> These lines:
>>
>> 440 elif include_re.match(line):
>> 441 incfile = include_re.match(line).group(1)
>>
>> put "${...}" _string_ into "incfile" variable. So, "incfile" is not None at
>> "this stage.
> "${...}" it's already expanded to a white-space. So in that case it
> matches "include ".
No. "incfile" is not expanded here, it actually contains "${...}", e.g.
${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
>> Then,
>>
>> 443 incfile = d.expand(incfile)
>>
>> Now "incfile" is empty and nobody checks it.
> incfile is not empty. It's actually a string containing one white-space.
It is actually empty, e.g. "".
>>
>> 444 incfile = bb.utils.which(bbpath, incfile)
>>
>> Now "incfile" is set to first directory name in BBPATH (wrong behavour?)
>>
>> 445 if incfile:
>> 446 shutil.copy(incfile, outdir)
>>
>> Exception here: "incfile" is directory, not a file.
>>
> The include regex is the following:
>
> include_re = re.compile( r"include\s+(.+)" )
>
> The issue is that when this is matched on a string suffixed with only
> spaces, it will match the last space as group(1). This is because ".+"
> forces to match the last white-space. Changing that to ".*" will make
> the group(1) be an empty string and later the if will evaluate False.
I don't see how it helps here.
--
Grygorii
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Hi Grygorii,
On 04/11/2019 15:09, Grygorii Tertychnyi (gtertych) wrote:
Andrei,
From: Andrei Gherzan
Sent: Friday, November 1, 2019 13:28
To: Grygorii Tertychnyi (gtertych); openembedded-core@lists.openembedded.org
Cc: xe-linux-external(mailer list)
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
do_ar_recipe fails on perf recipe on line:
include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '',
d)}
1. "${...}" part expands into empty string
2. bb.utils.which() takes empty string and returns first directory name from
bbpath
This doesn't sound sane. If the include directive has no argument,
incfile should end up None. That's what the code "assumes" at this
I agree.
point. I would fix it either at the regex expression level or
stripping the matched string. I reckon the former makes more sense
(.*).
Not sure I understand. Archiver class does not interpret "include" directive.
It just parses text files. The regular expression looks correct:
These lines:
440 elif include_re.match(line):
441 incfile = include_re.match(line).group(1)
put "${...}" _string_ into "incfile" variable. So, "incfile" is not None at
"this stage.
"${...}" it's already expanded to a white-space. So in that case it
matches "include ".
Then,
443 incfile = d.expand(incfile)
Now "incfile" is empty and nobody checks it.
incfile is not empty. It's actually a string containing one white-space.
444 incfile = bb.utils.which(bbpath, incfile)
Now "incfile" is set to first directory name in BBPATH (wrong behavour?)
445 if incfile:
446 shutil.copy(incfile, outdir)
Exception here: "incfile" is directory, not a file.
The include regex is the following:
include_re = re.compile( r"include\s+(.+)" )
The issue is that when this is matched on a string suffixed with only
spaces, it will match the last space as group(1). This is because ".+"
forces to match the last white-space. Changing that to ".*" will make
the group(1) be an empty string and later the if will evaluate False.
--
Andrei Gherzan
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
ping
From: Grygorii Tertychnyi (gtertych)
Sent: Monday, November 4, 2019 17:09
To: Andrei Gherzan; openembedded-core@lists.openembedded.org
Cc: xe-linux-external(mailer list)
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Andrei,
From: Andrei Gherzan
Sent: Friday, November 1, 2019 13:28
To: Grygorii Tertychnyi (gtertych); openembedded-core@lists.openembedded.org
Cc: xe-linux-external(mailer list)
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
>> do_ar_recipe fails on perf recipe on line:
>>
>> include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc',
>> '', d)}
>>
>> 1. "${...}" part expands into empty string
>> 2. bb.utils.which() takes empty string and returns first directory name from
>> bbpath
> This doesn't sound sane. If the include directive has no argument,
> incfile should end up None. That's what the code "assumes" at this
I agree.
> point. I would fix it either at the regex expression level or
> stripping the matched string. I reckon the former makes more sense
> (.*).
Not sure I understand. Archiver class does not interpret "include" directive.
It just parses text files. The regular expression looks correct:
These lines:
440 elif include_re.match(line):
441 incfile = include_re.match(line).group(1)
put "${...}" _string_ into "incfile" variable. So, "incfile" is not None at
this stage.
Then,
443 incfile = d.expand(incfile)
Now "incfile" is empty and nobody checks it.
444 incfile = bb.utils.which(bbpath, incfile)
Now "incfile" is set to first directory name in BBPATH (wrong behavour?)
445 if incfile:
446 shutil.copy(incfile, outdir)
Exception here: "incfile" is directory, not a file.
--
Grygorii
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Andrei,
From: Andrei Gherzan
Sent: Friday, November 1, 2019 13:28
To: Grygorii Tertychnyi (gtertych); openembedded-core@lists.openembedded.org
Cc: xe-linux-external(mailer list)
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
>> do_ar_recipe fails on perf recipe on line:
>>
>> include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc',
>> '', d)}
>>
>> 1. "${...}" part expands into empty string
>> 2. bb.utils.which() takes empty string and returns first directory name from
>> bbpath
> This doesn't sound sane. If the include directive has no argument,
> incfile should end up None. That's what the code "assumes" at this
I agree.
> point. I would fix it either at the regex expression level or
> stripping the matched string. I reckon the former makes more sense
> (.*).
Not sure I understand. Archiver class does not interpret "include" directive.
It just parses text files. The regular expression looks correct:
These lines:
440 elif include_re.match(line):
441 incfile = include_re.match(line).group(1)
put "${...}" _string_ into "incfile" variable. So, "incfile" is not None at
this stage.
Then,
443 incfile = d.expand(incfile)
Now "incfile" is empty and nobody checks it.
444 incfile = bb.utils.which(bbpath, incfile)
Now "incfile" is set to first directory name in BBPATH (wrong behavour?)
445 if incfile:
446 shutil.copy(incfile, outdir)
Exception here: "incfile" is directory, not a file.
--
Grygorii
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Hi,
On 01/11/2019 07:10, grygorii tertychnyi via Openembedded-core wrote:
> do_ar_recipe fails on perf recipe on line:
>
> include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc',
> '', d)}
>
> 1. "${...}" part expands into empty string
> 2. bb.utils.which() takes empty string and returns first directory name from
> bbpath
This doesn't sound sane. If the include directive has no argument,
incfile should end up None. That's what the code "assumes" at this
point. I would fix it either at the regex expression level or
stripping the matched string. I reckon the former makes more sense
(.*).
--
Andrei Gherzan
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Hi,
On 01/11/2019 07:10, grygorii tertychnyi via Openembedded-core wrote:
do_ar_recipe fails on perf recipe on line:
include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '',
d)}
1. "${...}" part expands into empty string
2. bb.utils.which() takes empty string and returns first directory name from
bbpath
This doesn't sound sane. If the include directive has no argument,
incfile should end up None. That's what the code "assumes" at this
point. I would fix it either at the regex expression level or strip the
match. I reckon former would make more sense. It sounds like the match
expression should use * (none or more).
3. shutil.copy() fails on copying directory:
Exception: IsADirectoryError: [Errno 21] Is a directory: ..
Hence, check "incfile" variable on each step.
Signed-off-by: grygorii tertychnyi
---
meta/classes/archiver.bbclass | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index 093e2d95af..7c46cff91f 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -441,9 +441,10 @@ python do_ar_recipe () {
incfile = include_re.match(line).group(1)
if incfile:
incfile = d.expand(incfile)
+if incfile:
incfile = bb.utils.which(bbpath, incfile)
-if incfile:
-shutil.copy(incfile, outdir)
+if incfile:
+shutil.copy(incfile, outdir)
create_tarball(d, outdir, 'recipe', d.getVar('ARCHIVER_OUTDIR'))
bb.utils.remove(outdir, recurse=True)
--
Andrei Gherzan
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
