Hi, Quoting Jakub Wilk (2014-03-19 12:01:56) > >In that case, can you give me an example of a tag that checks > >debian/control? > > binary-control-field-duplicates-source
thanks! That was very helpful :) please find attached a new patch with one more added tag. All three students forgot to add the build-profiles field for their patches so it seems something one easily forgets. cheers, josch
From 878e6fdeeeeaed3a1f88e58d3c4d619af576b486 Mon Sep 17 00:00:00 2001 From: josch <[email protected]> Date: Mon, 3 Mar 2014 15:58:28 +0100 Subject: [PATCH] Build-profiles support - added 3 new tags to detect errors in restriction list syntax * invalid-restriction-term-in-source-relation * invalid-restriction-namespace-in-source-relation * invalid-restriction-label-in-source-relation - added 4 new tags to ensure that if restrictions lists are used, a versioned build dependency on dpkg-dev and (if applicable) debhelper is added and no conflicts with them exist * restriction-list-without-versioned-dpkg-dev-dependency * restriction-list-with-versioned-dpkg-dev-conflict * restriction-list-with-debhelper-without-debhelper-version * restriction-list-with-debhelper-with-conflicting-debhelper-version - added one new tag to detect a missing build-profiles field in binary stanzas for build profiles stage1 and stage2 * stageX-profile-used-but-no-binary-package-dropped - added data/fields/dependency-restrictions and a parser to keep record of valid namespaces and labels for restriction lists --- checks/control-file.pm | 46 +++++++++++++ checks/fields.desc | 62 +++++++++++++++++ checks/fields.pm | 80 +++++++++++++++++++--- data/fields/dependency-restrictions | 4 ++ lib/Lintian/Relation.pm | 47 +++++++++---- .../debian/debian/control.in | 7 +- t/tests/fields-build-depends-general/desc | 8 +++ t/tests/fields-build-depends-general/tags | 8 +++ 8 files changed, 239 insertions(+), 23 deletions(-) create mode 100644 data/fields/dependency-restrictions diff --git a/checks/control-file.pm b/checks/control-file.pm index 2f16f31..0f9064c 100644 --- a/checks/control-file.pm +++ b/checks/control-file.pm @@ -292,6 +292,52 @@ sub run { } } + # check the Build-Profiles field + # this has to checked here because the Build-Profiles field does not appear + # in DEBIAN/control and even if it should in the future, some binary + # packages might never be built in the first place because of build + # profiles + + # check which profile names are supposedly supported according to the build + # dependencies + my %used_profiles=(); + for my $field ( + qw(build-depends build-depends-indep build-conflicts build-conflicts-indep) + ) { + if (defined $info->source_field($field)) { + for my $dep (split /\s*,\s*/, $info->source_field($field)) { + for my $alt (split /\s*\|\s*/, $dep) { + while ($alt =~ /<([^>]+)>/g) { + for my $restr (split /\s+/, $1) { + if ($restr =~ m/^!?profile\.(.*)/) { + $used_profiles{$1} = 0; + } + } + } + } + } + } + } + + # find those packages that do not get built because of a certain build + # profile + for my $bin (@package_names) { + my $raw = $info->binary_field($bin, "build-profiles"); + next unless $raw; + for my $prof (split /\s+/, $raw) { + if ($prof =~ s/^!//) { + $used_profiles{$prof} = 1; + } + } + } + + # find out if the developer forgot to mark binary packages as not being + # built + while (my ($k, $v) = each(%used_profiles)) { + tag 'stageX-profile-used-but-no-binary-package-dropped' + if (($k eq "stage1" || $k eq "stage2") && $v == 0); + } + return; } diff --git a/checks/fields.desc b/checks/fields.desc index d55c11a..97e17b9 100644 --- a/checks/fields.desc +++ b/checks/fields.desc @@ -637,6 +637,68 @@ Info: The architecture string in this source relation has some negated. This is not permitted by Policy. Either all architectures must be negated or none of them may be. +Tag: invalid-restriction-term-in-source-relation +Severity: important +Certainty: certain +Info: The restriction list in the source relation includes a term which + does not contain exactly one dot separating restriction namespace and label. + A term in a restriction list is of the form "namespace.label" (without the + quotes). + +Tag: invalid-restriction-namespace-in-source-relation +Severity: important +Certainty: possible +Info: The restriction list in the source relation includes a term with + an unknown namespace. The only allowed namespace is "profile" (without the + quotes). + +Tag: invalid-restriction-label-in-source-relation +Severity: important +Certainty: possible +Info: The restriction list in the source relation includes a term with + an unknown label. The only allowed labels are "stage1", "stage2", "notest" + and "cross". + +Tag: restriction-list-without-versioned-dpkg-dev-dependency +Severity: normal +Certainty: certain +Info: If a restriction list appears in the build dependencies, then the + source package has to build depend on dpkg-dev (>= 1.17.2) for minimal + restriction list support. + +Tag: restriction-list-with-versioned-dpkg-dev-conflict +Severity: normal +Certainty: certain +Info: If a restriction list appears in the build dependencies, then the + source package has to build depend on dpkg-dev (>= 1.17.2) for minimal + restriction list support. It must not conflict with version 1.17.2. + +Tag: restriction-list-with-debhelper-without-debhelper-version +Severity: normal +Certainty: certain +Info: If a restriction list appears in the build dependencies and the + package uses debhelper, then the source package has to depend on at least + debhelper 9.20140227. + +Tag: restriction-list-with-debhelper-with-conflicting-debhelper-version +Severity: normal +Certainty: certain +Info: If a restriction list appears in the build dependencies and the + package uses debhelper, then the source package has to depend on at least + debhelper 9.20140227. It must not conflict with version 9.20140227. + +Tag: stageX-profile-used-but-no-binary-package-dropped +Severity: normal +Certainty: certain +Info: You used a stage1 or stage2 build profile restriction in the build + dependencies but you did not mark any binary packages as not being built with + the used profile activated. Using a stage1 or stage2 build profile restriction + means that you intend to change the build process in a way such that some of + the build results will be different or not generated at all. All binary + packages which would provide different functionality, would be empty or not be + built at all under the stage1 or stage2 profiles must be marked as not being + generated with the Build-Profiles field. + Tag: depends-on-build-essential-package-without-using-version Severity: important Certainty: certain diff --git a/checks/fields.pm b/checks/fields.pm index aeee252..12f2dbb 100644 --- a/checks/fields.pm +++ b/checks/fields.pm @@ -46,6 +46,9 @@ our $known_build_essential = Lintian::Data->new('fields/build-essential-packages'); our $KNOWN_BINARY_FIELDS = Lintian::Data->new('fields/binary-fields'); our $KNOWN_UDEB_FIELDS = Lintian::Data->new('fields/udeb-fields'); +our $KNOWN_DEPENDENCY_RESTRICTIONS + = Lintian::Data->new('fields/dependency-restrictions', + qr/\./, \&_load_dependency_restrictions); our %KNOWN_ARCHIVE_PARTS = map { $_ => 1 } ('non-free', 'contrib'); @@ -730,7 +733,7 @@ sub run { && $known_obsolete_emacs{$alternatives[0]->[0]}); for my $part_d (@alternatives) { - my ($d_pkg, $d_march, $d_version, $d_arch, $rest, + my ($d_pkg, $d_march, $d_version, $d_arch, undef, $rest, $part_d_orig) = @$part_d; @@ -935,6 +938,7 @@ sub run { any { $_ eq $_[0] } qw(build-depends build-depends-indep); }; + my $restrictions_used = 0; my %depend; for my $field ( qw(build-depends build-depends-indep build-conflicts build-conflicts-indep) @@ -956,7 +960,7 @@ sub run { && &$is_dep_field($field)); for my $part_d (@alternatives) { - my ($d_pkg, $d_march, $d_version, $d_arch, $rest, + my ($d_pkg, $d_march, $d_version, $d_arch, $d_restr, $rest, $part_d_orig) = @$part_d; @@ -968,6 +972,30 @@ sub run { } } + if (scalar @{$d_restr} >= 1) { + $restrictions_used = 1; + } + + for my $restr (@{$d_restr}) { + my $dotcount = $restr =~ tr/.//; + if ($dotcount != 1) { + tag 'invalid-restriction-term-in-source-relation', + "$restr [$field: $part_d_orig]"; + next; + } + $restr =~ s/^!//; + my ($ns, $label) = split(/\./, $restr, 2); + if ($KNOWN_DEPENDENCY_RESTRICTIONS->known($ns)) { + tag 'invalid-restriction-label-in-source-relation', + "$label [$field: $part_d_orig]" + unless any { $_ eq $label } + @{$KNOWN_DEPENDENCY_RESTRICTIONS->value($ns)}; + } else { + tag 'invalid-restriction-namespace-in-source-relation', + "$ns [$field: $part_d_orig]"; + } + } + if ( $d_pkg =~ m/^openjdk-\d+-doc$/o or $d_pkg eq 'classpath-doc'){ tag 'build-depends-on-specific-java-doc-package', @@ -1077,6 +1105,24 @@ sub run { } } + # if restrictions are found in the build-depends/conflicts, then + # package must build-depend on dpkg (>= 1.17.2) + if ($restrictions_used) { + my $build_conflicts_all = $info->relation('build-conflicts-all'); + tag 'restriction-list-without-versioned-dpkg-dev-dependency' + unless ($build_all->implies('dpkg-dev (>= 1.17.2)')); + tag 'restriction-list-with-versioned-dpkg-dev-conflict' + if ($build_conflicts_all->implies_inverse('dpkg-dev (<< 1.17.2)')); + # if the package uses debhelper then it must require and not + # conflict with version >= 9.20140227 + if ($build_all->implies('debhelper')) { + tag 'restriction-list-with-debhelper-without-debhelper-version' + unless ($build_all->implies('debhelper (>= 9.20140227)')); + tag 'restriction-list-with-debhelper-with-conflicting-debhelper-version' + if ($build_conflicts_all->implies_inverse('debhelper (<< 9.20140227)')); + } + } + my (@arch_dep_pkgs, @dbg_pkgs); foreach my $gproc ($group->get_binary_processables) { my $binpkg = $gproc->pkg_name; @@ -1238,16 +1284,16 @@ sub run { return; } -# splits "foo:bar (>= 1.2.3) [!i386 ia64]" into -# ( "foo", "bar", [ ">=", "1.2.3" ], [ [ "i386", "ia64" ], 1 ], "" ) -# ^^^ ^^ -# count of negated arches, if ! was given || -# rest (should always be "" for valid dependencies) +# splits "foo:bar (>= 1.2.3) [!i386 ia64] <!profile.stage1 !profile.notest>" into +# ( "foo", "bar", [ ">=", "1.2.3" ], [ [ "i386", "ia64" ], 1 ], [ "!profile.stage1" "!profile.notest" ], "" ) +# ^^^ ^^ +# count of negated arches, if ! was given || +# rest (should always be "" for valid dependencies) sub _split_dep { my $dep = shift; - my ($pkg, $dmarch, $version, $darch) = ('', '', ['',''], [[], 0]); + my ($pkg, $dmarch, $version, $darch, $restr) = ('', '', ['',''], [[], 0], []); - my $pkgname = $1 if $dep =~ s/^\s*([^\s\[\(]+)\s*//; + my $pkgname = $1 if $dep =~ s/^\s*([^<\s\[\(]+)\s*//; if (defined $pkgname) { ($pkg, $dmarch) = split /:/, $pkgname, 2; $dmarch //= ''; # Ensure it is defined (in case there is no ":") @@ -1267,9 +1313,23 @@ sub _split_dep { } $darch->[1] = $negated; } + if ($dep && $dep =~ s/\s*<([^>]+)>\s*//) { + my $t = $1; + $restr = [split /\s+/, $t]; + } } - return ($pkg, $dmarch, $version, $darch, $dep); + return ($pkg, $dmarch, $version, $darch, $restr, $dep); +} + +sub _load_dependency_restrictions { + my ($key, $value, $pval) = @_; + my $ret = undef; + if (not defined $pval) { + $ret = $pval = []; + } + push @{$pval}, $value; + return $ret; } sub perl_core_has_version { diff --git a/data/fields/dependency-restrictions b/data/fields/dependency-restrictions new file mode 100644 index 0000000..270be3c --- /dev/null +++ b/data/fields/dependency-restrictions @@ -0,0 +1,4 @@ +profile.notest +profile.stage1 +profile.stage2 +profile.cross diff --git a/lib/Lintian/Relation.pm b/lib/Lintian/Relation.pm index 1ba783c..6d72176 100644 --- a/lib/Lintian/Relation.pm +++ b/lib/Lintian/Relation.pm @@ -115,21 +115,26 @@ sub parse_element { \s* (.*?) # architectures (5) \s* \] # closing bracket )? # end of optional architecture + (?: # start of optional restriction + \s* < # open bracket for restriction + \s* (.*?) # don't parse restrictions now + \s* > # closing bracket + )? # end of optional restriction /x; - my ($pkgname, $march, $relop, $relver, $bdarch) = ($1, $2, $3, $4, $5); + my ($pkgname, $march, $relop, $relver, $bdarch, $restr) = ($1, $2, $3, $4, $5, $6); my @array; if (not defined($relop)) { # If there's no version, we don't need to do any further processing. # Otherwise, convert the legacy < and > relations to the current ones. - @array = ('PRED', $pkgname, undef, undef, $bdarch, $march); + @array = ('PRED', $pkgname, undef, undef, $bdarch, $march, $restr); } else { if ($relop eq '<') { $relop = '<<'; } elsif ($relop eq '>') { $relop = '>>'; } - @array = ('PRED', $pkgname, $relop, $relver, $bdarch, $march); + @array = ('PRED', $pkgname, $relop, $relver, $bdarch, $march, $restr); } # Optimise the memory usage of the array. Understanding this @@ -191,26 +196,37 @@ sub new { return $self; } -=item new_noarch(RELATION) +=item new_norestriction(RELATION) Creates a new Lintian::Relation object corresponding to the parsed -relationship RELATION, ignoring architecture restrictions. This should be -used in cases where we only care if a dependency is present in some cases -and we don't want to require that the architectures match (such as when -checking for proper build dependencies, since if there are architecture -constraints the maintainer is doing something beyond Lintian's ability to -analyze). RELATION may be C<undef> or the empty string, in which case the -returned Lintian::Relation object is empty (always satisfied). +relationship RELATION, ignoring architecture restrictions and restriction +lists. This should be used in cases where we only care if a dependency is +present in some cases and we don't want to require that the architectures +match (such as when checking for proper build dependencies, since if there +are architecture constraints the maintainer is doing something beyond +Lintian's ability to analyze) or that the restrictions list match (Lintian +can't handle dependency implications with build profiles yet). RELATION +may be C<undef> or the empty string, in which case the returned +Lintian::Relation object is empty (always satisfied). =cut -sub new_noarch { +sub new_norestriction { my ($class, $relation) = @_; $relation = '' unless defined($relation); $relation =~ s/\[[^\]]*\]//g; + $relation =~ s/<[^>]*>//g; return $class->new($relation); } +=item new_noarch(RELATION) + +An alias for new_norestriction. + +=cut + +*new_noarch = \&new_norestriction; + =item and(RELATION, ...) Creates a new Lintian::Relation object produced by AND'ing all the @@ -351,6 +367,13 @@ sub implies_element { $$q[1] = '' unless defined $$q[1]; return if $$p[1] ne $$q[1]; + # Since the restriction list is not a set (as the architecture list) there + # is no way to calculate a superset or subset of one another. Furthermore, + # the evaluation depends on which build profiles are currently activated. + # With n being the number of possible build profiles, 2^n checks would + # have to be done. We decide not to do that (yet). + return if defined $$p[6] or defined $$q[6]; + # If the names match, then the only difference is in the architecture or # version clauses. First, check architecture. The architectures for p # must be a superset of the architectures for q. diff --git a/t/tests/fields-build-depends-general/debian/debian/control.in b/t/tests/fields-build-depends-general/debian/debian/control.in index f6546b9..80222ef 100644 --- a/t/tests/fields-build-depends-general/debian/debian/control.in +++ b/t/tests/fields-build-depends-general/debian/debian/control.in @@ -8,8 +8,13 @@ Build-Depends: debhelper (>= 9), bd-conflict, revision-1 (>= 1.0-1), xorg-dev, java-propose-classpath, python3.2-dev, foo [all], bar [i386 any], baz [source i3!86], baz [i386 !amd64], other-pkg [kfreebsd-any], yet-another [any-powerpc], + big <profile.stage1>, bpfail1 <foobar>, + bpfail2 <foo.bar>, bpfail3 <profile.bar>, packaging-dev, libdb5.1++-dev, libdb5.1-java-dev -Build-Conflicts: bd-conflict +Build-Conflicts: + bd-conflict, + dpkg-dev (>= 1.17.2), + debhelper (>= 9.20140227) Package: {$source} Architecture: {$architecture} diff --git a/t/tests/fields-build-depends-general/desc b/t/tests/fields-build-depends-general/desc index b8b7a1b..96c2918 100644 --- a/t/tests/fields-build-depends-general/desc +++ b/t/tests/fields-build-depends-general/desc @@ -16,5 +16,13 @@ Test-For: depends-on-build-essential-package-without-using-version depends-on-packaging-dev invalid-arch-string-in-source-relation + invalid-restriction-label-in-source-relation + invalid-restriction-namespace-in-source-relation + invalid-restriction-term-in-source-relation ored-build-depends-on-obsolete-package + restriction-list-with-debhelper-with-conflicting-debhelper-version + restriction-list-with-debhelper-without-debhelper-version + restriction-list-with-versioned-dpkg-dev-conflict + restriction-list-without-versioned-dpkg-dev-dependency + stageX-profile-used-but-no-binary-package-dropped References: Debian Bug#540594, Debian Bug#551793 diff --git a/t/tests/fields-build-depends-general/tags b/t/tests/fields-build-depends-general/tags index 0c0222f..5c70314 100644 --- a/t/tests/fields-build-depends-general/tags +++ b/t/tests/fields-build-depends-general/tags @@ -9,9 +9,17 @@ E: fields-build-depends-general source: depends-on-build-essential-package-witho E: fields-build-depends-general source: invalid-arch-string-in-source-relation all [build-depends: foo [all]] E: fields-build-depends-general source: invalid-arch-string-in-source-relation i3!86 [build-depends: baz [source i3!86]] E: fields-build-depends-general source: invalid-arch-string-in-source-relation source [build-depends: baz [source i3!86]] +E: fields-build-depends-general source: invalid-restriction-label-in-source-relation bar [build-depends: bpfail3 <profile.bar>] +E: fields-build-depends-general source: invalid-restriction-namespace-in-source-relation foo [build-depends: bpfail2 <foo.bar>] +E: fields-build-depends-general source: invalid-restriction-term-in-source-relation foobar [build-depends: bpfail1 <foobar>] I: fields-build-depends-general source: build-depends-on-python-dev-with-no-arch-any I: fields-build-depends-general source: ored-build-depends-on-obsolete-package build-depends: xlibmesa-gl-dev W: fields-build-depends-general source: build-depends-on-1-revision build-depends: revision-1 (>= 1.0-1) W: fields-build-depends-general source: build-depends-on-versioned-berkeley-db build-depends:libdb5.1++-dev W: fields-build-depends-general source: build-depends-on-versioned-berkeley-db build-depends:libdb5.1-java-dev W: fields-build-depends-general source: depends-on-packaging-dev build-depends +W: fields-build-depends-general source: restriction-list-with-debhelper-with-conflicting-debhelper-version +W: fields-build-depends-general source: restriction-list-with-debhelper-without-debhelper-version +W: fields-build-depends-general source: restriction-list-with-versioned-dpkg-dev-conflict +W: fields-build-depends-general source: restriction-list-without-versioned-dpkg-dev-dependency +W: fields-build-depends-general source: stageX-profile-used-but-no-binary-package-dropped -- 1.8.5.3
