Re: [lintian] 01/02: Use Lintian::Data for section/name mapping

[Bastien Roucaries]

Le 26 août 2017 08:59:00 GMT+02:00, Niels Thykier  a écrit :
>Bastien Roucariès:
>> This is an automated email from the git hooks/post-receive script.
>> 
>> rouca pushed a commit to branch master
>> in repository lintian.
>> 
>> commit f71f3901fcc23db666d3de176526e91fd4f228a2
>> Author: Bastien ROUCARIÈS 
>> Date:   Fri Aug 25 22:22:39 2017 +0200
>> 
>> Use Lintian::Data for section/name mapping
>
>
>Hi,
>
>This change causes a regression that need to be fixed (or the commit
>reverted)

will fix: what do you prefer : fix regex or fix lintian data to keep ordoring ?

>The problem being that @NAME_SECTION_MAPPINGS is an /ordered/ list with
>a "first match and we are done"-rule.  When migrated to L::Data, this
>order is lost because L::Data does not have an order guarantee.
>
>Therefore, when multiple rules match (see [1] for an example), the
>previous code would give a well-defined result (first rule listed)
>whereas the new code gives a non-deterministic result.
>
>Thanks,
>~Niels
>
>[1]
>
>libfoo-ocaml-dev matches the following two rules
>
>[qr/^lib.*-(?:ocaml|camlp4)-dev$/ => 'ocaml'],
>[qr/^lib.*-dev$/  => 'libdevel'],
>
>> ---
>>  checks/fields.pm  | 34
>+-
>>  data/fields/name_section_mappings | 16 
>>  debian/changelog  |  1 +
>>  3 files changed, 30 insertions(+), 21 deletions(-)
>> 
>> diff --git a/checks/fields.pm b/checks/fields.pm
>> index cb91491..4ad96ad 100644
>> --- a/checks/fields.pm
>> +++ b/checks/fields.pm
>> @@ -96,23 +96,14 @@ our @known_java_pkg = map { qr/$_/ } (
>>  );
>>  
>>  # Mapping of package names to section names
>> -my @NAME_SECTION_MAPPINGS = (
>> -[qr/-docs?$/  => 'doc'],
>> -[qr/-dbg(?:sym)?$/=> 'debug'],
>> -[qr/^(?:python-)?zope/=> 'zope'],
>> -[qr/^python3?-/   => 'python'],
>> -[qr/^r-(?:cran|bioc|other)-/  => 'gnu-r'],
>> -[qr/^lib.*-perl$/ => 'perl'],
>> -[qr/^lib.*-cil(?:-dev)?$/ => 'cli-mono'],
>> -[qr/^lib.*-(?:java|gcj)$/ => 'java'],
>> -[qr/^(?:lib)php-/ => 'php'],
>> -[qr/^lib(?:hugs|ghc6?)-/  => 'haskell'],
>> -[qr/^lib.*-ruby(?:1\.\d)?$/   => 'ruby'],
>> -[qr/^lib.*-(?:ocaml|camlp4)-dev$/ => 'ocaml'],
>> -[qr/^lib.*-dev$/  => 'libdevel'],
>> -[qr/^gir\d+\.\d+-.*-\d+\.\d+$/=> 'introspection'],
>> -[qr/^libjs-/  => 'javascript'],
>> -);
>> +my $NAME_SECTION_MAPPINGS = Lintian::Data->new(
>> +'fields/name_section_mappings',
>> +qr/\s*=>\s*/,
>> +sub {
>> +my $regex = qr/$_[0]/x;
>> +$_[0] = $_[1];
>> +return $regex;
>> +});
>>  
>>  my %VCS_EXTRACT = (
>>  browser => sub { return @_;},
>> @@ -547,14 +538,15 @@ sub run {
>>  # Check package name <-> section.  oldlibs is a special
>case; let
>>  # anything go there.
>>  if ($parts[-1] ne 'oldlibs') {
>> -foreach my $map (@NAME_SECTION_MAPPINGS) {
>> -next unless ($pkg =~ $map->[0]);
>> +foreach my $section ($NAME_SECTION_MAPPINGS->all())
>{
>> +my $regex =
>$NAME_SECTION_MAPPINGS->value($section);
>> +next unless ($pkg =~ m{$regex});
>>  
>>  my $area = '';
>>  $area = "$parts[0]/" if (scalar @parts == 2);
>>  tag 'wrong-section-according-to-package-name',
>> -  "$pkg => ${area}$map->[1]"
>> -  unless $parts[-1] eq $map->[1];
>> +  "$pkg => ${area}$section"
>> +  unless $parts[-1] eq $section;
>>  last;
>>  }
>>  }
>> diff --git a/data/fields/name_section_mappings
>b/data/fields/name_section_mappings
>> new file mode 100644
>> index 000..ca9c1de
>> --- /dev/null
>> +++ b/data/fields/name_section_mappings
>> @@ -0,0 +1,16 @@
>> +# map between regex (x) of package => section
>> +-docs?$  => doc
>> +-dbg(?:sym)?$=> debug
>> +^(?:python-)?zope=> zope
>> +^python3?-   => python
>> +^r-(?:cran|bioc|other)-  => gnu-r
>> +^lib.*-perl$ => perl
>> +lib.*-cil(?:-dev)?$  => cli-mono
>> +^lib.*-(?:java|gcj)$ => java
>> +^(?:lib)php- => php
>> +^lib(?:hugs|ghc6?)-  => haskell
>> +^lib.*-ruby(?:1\.\d)?$   => ruby
>> +^lib.*-(?:ocaml|camlp4)-dev$ => ocaml
>> +^lib.*-dev$  => libdevel
>> +^gir\d+\.\d+-.*-\d+\.\d+$=> introspection
>> +^libjs-  => javascript
>> \ No newline at end of file
>> diff --git a/debian/changelog b/debian/changelog
>> index e7cbe8d..af300a8 100644
>> --- a/debian/changelog
>> +++ b/debian/changelog

Re: [lintian] 01/02: Use Lintian::Data for section/name mapping

[Niels Thykier]

Bastien Roucaries:
> 
> 
> Le 26 août 2017 08:59:00 GMT+02:00, Niels Thykier  a écrit 
> :
>> Bastien Roucariès:
>>> This is an automated email from the git hooks/post-receive script.
>>>
>>> rouca pushed a commit to branch master
>>> in repository lintian.
>>>
>>> commit f71f3901fcc23db666d3de176526e91fd4f228a2
>>> Author: Bastien ROUCARIÈS 
>>> Date:   Fri Aug 25 22:22:39 2017 +0200
>>>
>>> Use Lintian::Data for section/name mapping
>>
>>
>> Hi,
>>
>> This change causes a regression that need to be fixed (or the commit
>> reverted)
> 
> will fix: what do you prefer : fix regex or fix lintian data to keep ordoring 
> ?
> 
> [...]

Do'er decides.

That said, I am happy to discuss the options if you would like input and
want to talk about pros/cons on the options.

Thanks,
~Niels

Re: [lintian] 01/02: Use Lintian::Data for section/name mapping

[Niels Thykier]

Bastien Roucariès:
> This is an automated email from the git hooks/post-receive script.
> 
> rouca pushed a commit to branch master
> in repository lintian.
> 
> commit f71f3901fcc23db666d3de176526e91fd4f228a2
> Author: Bastien ROUCARIÈS 
> Date:   Fri Aug 25 22:22:39 2017 +0200
> 
> Use Lintian::Data for section/name mapping


Hi,

This change causes a regression that need to be fixed (or the commit
reverted)

The problem being that @NAME_SECTION_MAPPINGS is an /ordered/ list with
a "first match and we are done"-rule.  When migrated to L::Data, this
order is lost because L::Data does not have an order guarantee.

Therefore, when multiple rules match (see [1] for an example), the
previous code would give a well-defined result (first rule listed)
whereas the new code gives a non-deterministic result.

Thanks,
~Niels

[1]

libfoo-ocaml-dev matches the following two rules

[qr/^lib.*-(?:ocaml|camlp4)-dev$/ => 'ocaml'],
[qr/^lib.*-dev$/  => 'libdevel'],

> ---
>  checks/fields.pm  | 34 +-
>  data/fields/name_section_mappings | 16 
>  debian/changelog  |  1 +
>  3 files changed, 30 insertions(+), 21 deletions(-)
> 
> diff --git a/checks/fields.pm b/checks/fields.pm
> index cb91491..4ad96ad 100644
> --- a/checks/fields.pm
> +++ b/checks/fields.pm
> @@ -96,23 +96,14 @@ our @known_java_pkg = map { qr/$_/ } (
>  );
>  
>  # Mapping of package names to section names
> -my @NAME_SECTION_MAPPINGS = (
> -[qr/-docs?$/  => 'doc'],
> -[qr/-dbg(?:sym)?$/=> 'debug'],
> -[qr/^(?:python-)?zope/=> 'zope'],
> -[qr/^python3?-/   => 'python'],
> -[qr/^r-(?:cran|bioc|other)-/  => 'gnu-r'],
> -[qr/^lib.*-perl$/ => 'perl'],
> -[qr/^lib.*-cil(?:-dev)?$/ => 'cli-mono'],
> -[qr/^lib.*-(?:java|gcj)$/ => 'java'],
> -[qr/^(?:lib)php-/ => 'php'],
> -[qr/^lib(?:hugs|ghc6?)-/  => 'haskell'],
> -[qr/^lib.*-ruby(?:1\.\d)?$/   => 'ruby'],
> -[qr/^lib.*-(?:ocaml|camlp4)-dev$/ => 'ocaml'],
> -[qr/^lib.*-dev$/  => 'libdevel'],
> -[qr/^gir\d+\.\d+-.*-\d+\.\d+$/=> 'introspection'],
> -[qr/^libjs-/  => 'javascript'],
> -);
> +my $NAME_SECTION_MAPPINGS = Lintian::Data->new(
> +'fields/name_section_mappings',
> +qr/\s*=>\s*/,
> +sub {
> +my $regex = qr/$_[0]/x;
> +$_[0] = $_[1];
> +return $regex;
> +});
>  
>  my %VCS_EXTRACT = (
>  browser => sub { return @_;},
> @@ -547,14 +538,15 @@ sub run {
>  # Check package name <-> section.  oldlibs is a special case; let
>  # anything go there.
>  if ($parts[-1] ne 'oldlibs') {
> -foreach my $map (@NAME_SECTION_MAPPINGS) {
> -next unless ($pkg =~ $map->[0]);
> +foreach my $section ($NAME_SECTION_MAPPINGS->all()) {
> +my $regex = $NAME_SECTION_MAPPINGS->value($section);
> +next unless ($pkg =~ m{$regex});
>  
>  my $area = '';
>  $area = "$parts[0]/" if (scalar @parts == 2);
>  tag 'wrong-section-according-to-package-name',
> -  "$pkg => ${area}$map->[1]"
> -  unless $parts[-1] eq $map->[1];
> +  "$pkg => ${area}$section"
> +  unless $parts[-1] eq $section;
>  last;
>  }
>  }
> diff --git a/data/fields/name_section_mappings 
> b/data/fields/name_section_mappings
> new file mode 100644
> index 000..ca9c1de
> --- /dev/null
> +++ b/data/fields/name_section_mappings
> @@ -0,0 +1,16 @@
> +# map between regex (x) of package => section
> +-docs?$  => doc
> +-dbg(?:sym)?$=> debug
> +^(?:python-)?zope=> zope
> +^python3?-   => python
> +^r-(?:cran|bioc|other)-  => gnu-r
> +^lib.*-perl$ => perl
> +lib.*-cil(?:-dev)?$  => cli-mono
> +^lib.*-(?:java|gcj)$ => java
> +^(?:lib)php- => php
> +^lib(?:hugs|ghc6?)-  => haskell
> +^lib.*-ruby(?:1\.\d)?$   => ruby
> +^lib.*-(?:ocaml|camlp4)-dev$ => ocaml
> +^lib.*-dev$  => libdevel
> +^gir\d+\.\d+-.*-\d+\.\d+$=> introspection
> +^libjs-  => javascript
> \ No newline at end of file
> diff --git a/debian/changelog b/debian/changelog
> index e7cbe8d..af300a8 100644
> --- a/debian/changelog
> +++ b/debian/changelog
> @@ -43,6 +43,7 @@ lintian (2.5.53) UNRELEASED; urgency=medium
>  + [CL] Ensure that python3-foo packages have "Section: python", not
>just python2-foo.  (Closes: #870272)
>  + [RG] Do no longer require debug packages to be priority extra.
> ++ [BR] Use Lintian::Data 

[lintian] 01/02: Use Lintian::Data for section/name mapping

[Bastien Roucariès]

This is an automated email from the git hooks/post-receive script.

rouca pushed a commit to branch master
in repository lintian.

commit f71f3901fcc23db666d3de176526e91fd4f228a2
Author: Bastien ROUCARIÈS 
Date:   Fri Aug 25 22:22:39 2017 +0200

Use Lintian::Data for section/name mapping
---
 checks/fields.pm  | 34 +-
 data/fields/name_section_mappings | 16 
 debian/changelog  |  1 +
 3 files changed, 30 insertions(+), 21 deletions(-)

diff --git a/checks/fields.pm b/checks/fields.pm
index cb91491..4ad96ad 100644
--- a/checks/fields.pm
+++ b/checks/fields.pm
@@ -96,23 +96,14 @@ our @known_java_pkg = map { qr/$_/ } (
 );
 
 # Mapping of package names to section names
-my @NAME_SECTION_MAPPINGS = (
-[qr/-docs?$/  => 'doc'],
-[qr/-dbg(?:sym)?$/=> 'debug'],
-[qr/^(?:python-)?zope/=> 'zope'],
-[qr/^python3?-/   => 'python'],
-[qr/^r-(?:cran|bioc|other)-/  => 'gnu-r'],
-[qr/^lib.*-perl$/ => 'perl'],
-[qr/^lib.*-cil(?:-dev)?$/ => 'cli-mono'],
-[qr/^lib.*-(?:java|gcj)$/ => 'java'],
-[qr/^(?:lib)php-/ => 'php'],
-[qr/^lib(?:hugs|ghc6?)-/  => 'haskell'],
-[qr/^lib.*-ruby(?:1\.\d)?$/   => 'ruby'],
-[qr/^lib.*-(?:ocaml|camlp4)-dev$/ => 'ocaml'],
-[qr/^lib.*-dev$/  => 'libdevel'],
-[qr/^gir\d+\.\d+-.*-\d+\.\d+$/=> 'introspection'],
-[qr/^libjs-/  => 'javascript'],
-);
+my $NAME_SECTION_MAPPINGS = Lintian::Data->new(
+'fields/name_section_mappings',
+qr/\s*=>\s*/,
+sub {
+my $regex = qr/$_[0]/x;
+$_[0] = $_[1];
+return $regex;
+});
 
 my %VCS_EXTRACT = (
 browser => sub { return @_;},
@@ -547,14 +538,15 @@ sub run {
 # Check package name <-> section.  oldlibs is a special case; let
 # anything go there.
 if ($parts[-1] ne 'oldlibs') {
-foreach my $map (@NAME_SECTION_MAPPINGS) {
-next unless ($pkg =~ $map->[0]);
+foreach my $section ($NAME_SECTION_MAPPINGS->all()) {
+my $regex = $NAME_SECTION_MAPPINGS->value($section);
+next unless ($pkg =~ m{$regex});
 
 my $area = '';
 $area = "$parts[0]/" if (scalar @parts == 2);
 tag 'wrong-section-according-to-package-name',
-  "$pkg => ${area}$map->[1]"
-  unless $parts[-1] eq $map->[1];
+  "$pkg => ${area}$section"
+  unless $parts[-1] eq $section;
 last;
 }
 }
diff --git a/data/fields/name_section_mappings 
b/data/fields/name_section_mappings
new file mode 100644
index 000..ca9c1de
--- /dev/null
+++ b/data/fields/name_section_mappings
@@ -0,0 +1,16 @@
+# map between regex (x) of package => section
+-docs?$  => doc
+-dbg(?:sym)?$=> debug
+^(?:python-)?zope=> zope
+^python3?-   => python
+^r-(?:cran|bioc|other)-  => gnu-r
+^lib.*-perl$ => perl
+lib.*-cil(?:-dev)?$  => cli-mono
+^lib.*-(?:java|gcj)$ => java
+^(?:lib)php- => php
+^lib(?:hugs|ghc6?)-  => haskell
+^lib.*-ruby(?:1\.\d)?$   => ruby
+^lib.*-(?:ocaml|camlp4)-dev$ => ocaml
+^lib.*-dev$  => libdevel
+^gir\d+\.\d+-.*-\d+\.\d+$=> introspection
+^libjs-  => javascript
\ No newline at end of file
diff --git a/debian/changelog b/debian/changelog
index e7cbe8d..af300a8 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -43,6 +43,7 @@ lintian (2.5.53) UNRELEASED; urgency=medium
 + [CL] Ensure that python3-foo packages have "Section: python", not
   just python2-foo.  (Closes: #870272)
 + [RG] Do no longer require debug packages to be priority extra.
++ [BR] Use Lintian::Data for name/section mapping
   * checks/files.pm:
 + [BR] Avoid false positive privacy-breach-generic for legal.xml.
   * checks/init.d.desc:

-- 
Alioth's /usr/local/bin/git-commit-notice on 
/srv/git.debian.org/git/lintian/lintian.git