Bug#1067653: lintian: debhelper-but-no-misc-depends and other relationship substvar hints are no longer universally applicable

[Niels Thykier]

Package: lintian
Severity: normal
X-Debbugs-Cc: ni...@thykier.net

Hi

With debhelper compat 14+ or any use of debputy (dh-sequence-debputy, 
dh-sequence-zz-debputy, or dh-sequence-zz-debputy-rrr), relationship 
substvars are now applied automatically.


This means that packages no longer need `Depends: ${misc:Depends}` or 
`Depends: ${shlibs:Depends}` to be correct.


See https://lists.debian.org/debian-devel/2024/03/msg00030.html for more 
details.


Best regards,
Niels

Re: debian-policy: Restrict deb822 field names more

[Niels Thykier]

Hi

I wanted you to be CCed on this bug. Unfortunately, my BTS X-CC appears 
to not have worked (I do not see it on the relevant lists), so here goes. :)


Note there is already a follow up, which is not included here. You may 
want to read https://bugs.debian.org/1064454 before answering.



On Thu, 22 Feb 2024 12:10:06 +0100 Niels Thykier  wrote:

Package: debian-policy
Severity: wishlist
X-Debbugs-Cc: ni...@thykier.net
X-Debbugs-Cc: lintian-ma...@debian.org
X-Debbugs-Cc: de...@lists.debian.org
X-Debbugs-Cc: debian-d...@lists.debian.org

Hi

I am hoping we can agree to some tighter bounds on the field names 
followed by relevant tools implementing the same restrictions in our 
next versions. Ideally all deb822 parsers should follow suit; I have 
CC'ed the major ones I could think of.
  I will submit a MR for the  python-debian's parser if a change is 
agreed, which would eventually cover dak.


My rationale for this request is as follows.


The policy says the following about a field name:


> The field name is composed of US-ASCII characters excluding control 
characters, space, and colon (i.e., characters in the ranges U+0021 (!) through 
U+0039 (9), and U+003B (;) through U+007E (~), inclusive). Field names must not 
begin with the comment character (U+0023 #), nor with the hyphen character (U+002D 
-).


Which leads to the interesting case that:

```
Package: foo
Depends: bar,
${shlib:Depends}
```

is a *valid* deb822 paragraph with 3 fields. The last being named 
`${shlib` with the value `Depends}`. I spotted this while debugging the 
python-debian parser that did not react to what I thought was a clear 
syntax error.


While the parser is technically correct given the Policy description 
above, I cannot see any case where this is a desirable outcome. Instead, 
I think we should classify this as a syntax error such that users are 
instructed to indent `${shlib:Depends}` token.


A simple fix would be to forbid `$` at the start of the field. Though, I 
think at this point it would make sense to prune more special characters 
from the list like `%&{}[]()/\\<>|$` from anywhere in the field. Note 
that we definitely need to keep `_` as it is used in debconf template 
files for translations. Both single and double underscore is used here, 
though they always come first in the field name.


The reason why I want a tighter bound is that currently, the following 
things are also considered valid field names:

 >
|foo(>=1:2.0),
foo(>=2:2.0),
,foo(>=3:2.0)
,foo(>=4:2.0)[amd64]
)': We can make inverted crying smileys as field names!
`Foo`: Now with markdown formatting of the field name!
$(foo): Can we trick something into running shell commands?
/etc/passwd: Maybe path traversal


In all cases, I see no value to allowing these absurd field names and 
only potential downsides.



Best regards,
Niels

PS: I doubt anything actually uses field names in an unsafe manner any 
more. But we do have some history with deb822 files.


But someone will eventually try to do this. We already received a bug 
report against debhelper long ago about being able to do path traversal 
via questionable package names. Largely, this was considered a non-issue 
because dpkg in its default configuration would abort and debhelper has 
a "run arbitrary code via the upstream build system" feature anyway.


Additionally, 10+ years ago. lintian would create a file per field of 
the file it scanned. Here, if it had used perl's "2-arg" open, you might 
have been able to do something "fun". I do not remember if it did and 
the exploit is a decade overdue even if.

Bug#977322: Following advice of cute-field tag for X-DhRuby-Root breaks build

[Niels Thykier]

Control: clone -1 -2
Control: retitle -2 gem2deb: Case sensitive handling of d/control
Control: reassign -2 gem2deb

On Sun, 13 Dec 2020 15:27:20 -0800 Russ Allbery  wrote:

Package: lintian
Version: 2.104.0
Severity: normal

Lintian now emits the following pedantic tag for packages using
X-DhRuby-Root:

P: remctl source: cute-field debian/control@ruby-remctl X-DhRuby-Root vs 
X-Dhruby-Root
N:
P: cute-field
N:
N:   The named field uses a free-style form of capitalization, which is
N:   permitted by policy. The alternative offered is probably a more common
N:   variant in the archive.
N:   
N:   Refer to Debian Policy Manual section 5.1 (Syntax of control files)

N:   for details.
N:   
N:   Severity: pedantic
N:   
N:   Check: fields/style


However, following this advice breaks the Ruby package tooling.  The
build products are not copied into the package correctly and the resulting
package is empty.  Apparently the Ruby package tooling requires that
specific capitalization.

This is with gem2deb 1.4.

[...]


Hi

On one end, I do not think `lintian` decides the canonical case / 
spelling of a field. In that sense, I feel this bug is valid.


On the flip side, gem2deb does not follow the deb822 if field name case 
matters. The format is defined as case-insensitive.


"""
Field names are not case-sensitive, but it is usual to capitalize the 
field names using mixed case as shown below. Field values are 
case-sensitive unless the description of the field says otherwise.

"""

Cloned and reassigned accordingly.

Best regards,
Niels

Bug#901631: Bug#901507: lintian: warn if debhelper is in B-D-Arch or B-D-Indep but not Build-Depends

[Niels Thykier]

On Mon, 18 Jun 2018 12:16:55 +0100 Simon McVittie  wrote:

On Sat, 16 Jun 2018 at 18:45:56 +0100, Chris Lamb wrote:
> I'm also seeing a bunch of false-positives in the addon checker -
> using the dh Python addon shouldn't mean that python can't be in
> Build-Depends-Indep. Or dpatch!

Sure - I hadn't intended that you'd add this mechanism for packages that
ship debhelper addons alongside other content, just the ones that have
little or no purpose other than their debhelper addons, like most/all
of the dh-$addon family, and maybe some of the pkg-$team-tools family.



Hi,

Drive by remark from the debhelper maintainer.

Some dh add-ons can be in ``Build-Depends-Indep`` now and is used to 
support cross-building in some cases. I do not remember when the feature 
was added though, so it might not have been possible at the time this 
was filed.


It is hard to tell which add-ons can be in B-D-I, and which cannot. If
lintian is going to warn about it, I would recommend a static data list
for this.
Best regards,
Niels

Bug#1023056: Bug#1014537: unnamed packaging files in a multibinary package should be an error

[Niels Thykier]

Control: tags 1023056 moreinfo
Control: tags 1023057 moreinfo

Axel Beckert:

Hi Niels,

Niels Thykier wrote:

I understand that you are unsatisfied with this proposal and that is
fair.


Thanks.


Though from my point of view, your email makes it hard for me to want to
engage with you to find a solution that would (ideally) satisfy your desires


I'm sorry, but at that point I have to say the very same about your
previous e-mail. IMHO this is not a one-sided thing. See below for an
explanation.


> [...]

Hi Axel,

First off, thanks for your reply.  I very much appreciate you reaching 
out here and putting your concerns forward as that enables to find 
common ground and work on resolving the conflict. :)


Also, sorry for the late reply. My "post-work" mental bandwidth this 
week has not enabled me to have adequate capacity for my follow up until 
now.



As for your concerns, I have read them and I feel that we have two 
conflicts that have been entangled into one.  Therefore, I would like to 
first attempt to untangle them and solve them separately.  As I see it, 
the current conflict is this bug (#1014537) but there is also the 
initial conflict related to the changelog trimming.



As I understood you, your issues are - *for this bug specifically*:


 1) You feel that I have made a final decision.  Here you mention the
cloning of bug and the debian/{TODO,README.Debian} as two things
that made you feel the decision to be final.

 2) While you agreed with Steve's feature request you did not feel my
proposed solution to Steve's request matched what you had expected
of me.

- For completeness, it is a bit unclear to me whether you also feel
  this ties in to point about changes not being discussed on the
  proper channel. It is possible for me to see narrative that given
  you also felt the proposal was a final decision - but maybe I am
  over reading you here.

 3) Current frustration with the changelog trimming.  I think a very
quick summary is that you feel changelog trimming proposal was not
discussed on the proper channels and also you strongly disagree with
the concept itself.

 - Note I am deliberately keeping this short because I understand it
   as an existing source of frustration caused by a different
   conflict.  Notably, this is the separate conflict I want to
   untangle from the current one about unnamed packaging files.
   However, since you explicitly brought the frustration caused by
   that conflict, I felt the summary would be incomplete without
   mentioning this part as well.

I hope you feel that is an accurate summary of the conflict related to 
*this* bug about unnamed packaging files in multi-binary packages.


  If you do *not* feel it was a good summary, then please stop reading
  more of this email and tell me where I misunderstood or misread you.

  The rest of my email is based on the above being a reasonably accurate
  understand of the conflict.  If I missed the mark on understanding
  you, the rest of the email will probably just feel like an aggravating
  straw man discussion to you and that is not what either of us want.


In a face to face discussion, I would normally wait for you to confirm 
that my summary was good enough. However, I am going to risk optimizing 
a bit by reducing the number of email round trips because I suspect the 
conflict related to this bug will be easy to resolve.




  => Again, if you felt I misread you above, please stop here and tell
 me where I was wrong.





I would like to start by addressing the part where you felt I made a 
final decision. My previous email was *not* intended to be a final 
decision. It was my request for feedback on my proposed solution.


Diving a bit deeper in what I intended with the actions you mentioned as 
making it feel like a final decision:


 * The cloned bug was me noting gregor suggesting a lintian tag for this
   and me thinking "let me just do it immediately, so I do not forget".
   Admittedly, the bugs should probably have been tagged moreinfo as
   well because the actual requested change had not (and is still not)
   fleshed out yet.  I have done this now.

 * My mention of debian/{TODO,README.Debian} was related to the part
   where I was leaving these files as exceptions to Steve's request.  If
   Steve (or someone else) wanted this particular exception "undone", I
   wanted that person to drive the discussion to show there was
   consensus for that change.

I hope the above clarified my actions and made it clear that I never 
intended this to be a "final decision".




As for my proposal not being what you had expected.  My intention with 
the email was to discuss my proposed solution. We all have different 
priorities and I made my proposal in a way I felt were aligned with the 
original proposal while also helping me with some of my priorities. 
However, I was aware t

Re: Bug#1014537: unnamed packaging files in a multibinary package should be an error

[Niels Thykier]

Axel Beckert:

Hi,


I am looking at making `debian/foo` an error by default in debhelper compat
15 (triggering a warning from compat 14).


Uargh, yet another bad decision which makes one want to no more using
debhelper. :-(



Hi Axel,

I understand that you are unsatisfied with this proposal and that is 
fair. Though from my point of view, your email makes it hard for me to 
want to engage with you to find a solution that would (ideally) satisfy 
your desires as well as solve the underlying problem that Steve wants to 
have solved.


Notably, nowhere in your email can I see any attempt from you to say 
"Could we find an alternative where single binary source packages can 
still leverage this short cut, because I find it very valuable?" in a 
neutral or constructive manner.


Instead, it feels to me like you are dumping your frustration on me and 
have given up on working with me in solving the problem in the best way 
for all parties (including you!).
   I find emails like this super draining on my motivation. I have no 
interest in spending my volunteer time working with people that writing 
emails phrased such that they make me feel like that person has given up 
on me.



I ask that you rephrase your email in a way where I do not feel you have 
given up on working with me, if you want me to engage any further with 
you. I hope we can agree on that minimum baseline for future communication.



~Niels

Re: Bug#1014537: unnamed packaging files in a multibinary package should be an error

[Niels Thykier]

Control: clone -1 -2 -3
Control: reassign -2 lintian
Control: reassign -3 lintian-brush
Control: block -3 by -2
Control: block -3 by -1
Control: block -2 by -1

Hi,

CC'ing relevant maintainers for clone + reassign for lintian + 
lintian-brush.


I am looking at making `debian/foo` an error by default in debhelper 
compat 15 (triggering a warning from compat 14).  The choice of having 
an intermediate compat level is to align the change with another 
"single-binary vs. multi-binary" change.


My current code proposal is at 
https://salsa.debian.org/debian/debhelper/-/merge_requests/97 for people 
who are curious or want to help by testing.


@Michael: This has some interesting affects on the 
dh_installsystemd{,user} test cases.  Could you have a look at 
dh_installsystemd{,user} and see if you agree with where this is headed 
for that helper?  If it is becoming unslightly, maybe we should combine 
it with a fix for #932152.


On Sat, 9 Jul 2022 00:11:47 +0200 gregor herrmann  wrote:

On Thu, 07 Jul 2022 08:48:36 -0700, Steve Langasek wrote:

> This kind of packaging, with some packaging files under debian/ having an
> associated binary package name and some not, is an antipattern.

+1

(I also don't like packaging files without binary package name for
single-binary source package, but that's just a matter of taste.)
 


I will standardize forbidding `debian/foo` in the general case with the 
following exceptions:


 * debian/copyright
 * debian/changelog
 * debian/NEWS

These have historically applied to all packages and it does not seem 
useful to force everyone to maintain N copies of {changelog,copyright,...}.


 * debian/README.Debian
 * debian/TODO

These have historically been installed into the main package and a note 
in debhelper (dh_installdocs) suggests these (or at least README.Debian) 
is a name standardized by Policy.  I am open to not installing them by 
default if one of you are willing to drive the discussion on 
debian-devel to assert there is consensus for that.


Otherwise, I will keep the special-case for these two files for now.


> I would like to suggest that in the next debhelper compat level, debhelper
> should consider it an error when debian/control lists more than one binary
> package, but contains unnamed packaging files under debian/.  


Or maybe start with a warning (maybe immediately) and then proceed to
an error?

I also wonder if a lintian warning might be helpful to get this
going.
(And I admit that I haven't checked if such a lintian tag exists but
I can't remember seeing one.)


Cheers,
gregor

[...]
I have CC'ed lintian + lintian-brush on CC and cloned bugs for them. 
Having a lintian tag for them will enable the Janitor to help with the 
migration, so I agree we should have a lintian tag for this.


Thanks,
~Niels

Bug#1021502: lintian: incorrectly complains about debian-news-entry-has-unknown-version

[Niels Thykier]

Control: tags 1021608 wontfix

Axel Beckert:

[...]
Control: retitle -3 debhelper: Revert change to trim changelog.Debian.gz by 
default
Control: severity -3 wishlist

[...]


Hi Axel,

I note you have cloned a bug for doing a revert of the automatic 
trimming with an email that I interpreted as a interest in a permanent 
revert.


I am perfectly open towards the current implementation having bugs - and 
we can discuss whether some of them are bad enough to warrant a timely 
solution or *temporary* revert.


However, be advised that the change to introduce automatic trimming was 
discussed on debian-devel and the consensus of that mail thread was to 
my understand that we should enable automatic trimming.  Therefore, I am 
now wontfix'ing #1021608 on a conceptual basis.  If you with #1021608 
wanted to a temporary revert, feel free to remove the wontfix tag again.


If you disagree the automatic trimming in general, I will require that 
you have ensured that there is project wide consensus or the tech-ctte 
behind that revert.


Thanks,
~Niels

Re: Literal Unicode COMBINING TILDE OVERLAY in checks/group-check.pm (now lib/Lintian/Check/GroupChecks.pm)

[Niels Thykier]

Axel Beckert:
> Hi Niels,
> 
> [...]
> 
> Nevertheless I would like to know if this was an accident which has
> been undetected for 10 years (and one month :-) or if this COMBINING
> TILDE OVERLAY character was added was on purpose. (In case of the
> latter, I'd be very curious about that purpose. :-)
> 
>   Kind regards, Axel

Hi Axel,

I can only assume it was an accident.

Thanks,
~Niels

Bug#992465: WARNING: dh_installsystemd is moving unit files to /usr/lib/systemd/system

[Niels Thykier]

Michael Biebl:
> Am 21.08.21 um 10:34 schrieb Niels Thykier:
>> Also, before that, we will need a solution to the generators issue
>> (#992554 comment #25 and below).
> 
> Not only generators. For systemd we have to consider
> 
> [... about 50 packages ...]
> 
> systemd does not support reading scripts/files from a /usr/lib
> counterpart and we'd have to patch various places in systemd to allow
> that. Those patches should be simple though.
> 

Ok, I think at this level we should do that transition manually in the
affected packages (given number of packages involved and the lack of
support in systemd/stable).


> For udev, it's a bit trickier.
> While udev supports reading udev .rules files from /lib/udev/rules.d and
> /usr/lib/udev/rules.d, udev allows to run programs via RUN+= or
> IMPORT{program}=
> If not specified with a full path, it will look up the binary in /lib/udev
> There are quite a few packages which hard-code a full path, like e.g.
> 
> 
> [...]
> 
> Moving everything from /lib/udev to /usr/lib/udev would break that
> reference.
> 
> Then there are also quite a few .rules files which don't hard-code the
> full path. For those we'd have to patch udev to lookup the binaries in
> /lib/udev and /usr/lib/udev.
> 
> Not quite sure, how we'd coordinate that. Maybe only automatically move
> the .rules files and then file bugs against packages shipping helpers in
> /lib/udev to do the migration manually (by moving the helper to
> /usr/lib/udev and updating the .rules files accordingly).
> 
> A rough estimation suggests that this affects 60+ packages.
> 
> # apt-file search -x ^/lib/udev/  | grep -v rules.d | grep -v hwdb |
> grep -v rc_keymaps | grep -v ^udev: | grep -v ^systemd: | cut -f2 -d':'
>  [...]
> 
> Regards,
> Michael
> 
> 

Would work for me.  There about 240 packages using /lib/udev/rules.d
(unstable/testing) so we are looking at auto-migrating somewhere between
180 and 240 packages away from /lib/udev automatically with that flow
leaving 60 to be patched manually in a slower pace.


Before I start, can you please confirm that:

 * udev in stable/oldstable support /usr/lib/rules.d ?
   (oldstable is for backports, re: #992711)

 * that you feel we are ready to go with debhelper doing this from the
   next upload.  For reference, I am out of touch with lintian (etc.)
   on this topic.

Thanks,
~Niels

Bug#992465: WARNING: dh_installsystemd is moving unit files to /usr/lib/systemd/system

[Niels Thykier]

Michael Biebl:
> Hi Nils, hi Peter
> 
> Am 20.08.21 um 07:54 schrieb Niels Thykier:
> 
>> As people have concluded already, the change was intended and functional
>> although lintian was not updated to match.  Accordingly, I will close
>> this bug against debhelper.
> 
> I think we should start doing the same for udev files in
> /lib/udev/rules.d (i.e. move rules files vi dh_installudev). I will file
> bugs against debhelper and lintian so we can coordinate that.
> 

OK. :)

>> @Michael: Re: lintian, then there is already a bug now with a patch
>> https://bugs.debian.org/992465
> 
> Thanks for the hint.
> 
> I guess after some point, I'd like to see lintian switched to actually
> flag packages that still install files in /lib/systemd/system, i.e. warn
> about them, so we can track the progress of the conversion to
> /usr/lib/systemd/system
> We currently have well over 1100 packages installing over 2200 service
> files in /lib, so it's probably too early to do that. Maybe in a couple
> of months.
> 
> Regards,
> Michael
> 
> 

Also, before that, we will need a solution to the generators issue
(#992554 comment #25 and below).

Thanks,
~Niels

Bug#958500: False positive for missing-build-dependency-for-dh_-command dh_gnome => gnome-pkg-tools

[Niels Thykier]

On Thu, 23 Apr 2020 22:23:10 +0100 "Chris Lamb"  wrote:
> Hi Laurent,
> 
> > I get a false positive in libmanette for the following error:
> > 
> > E: libmanette source: missing-build-dependency-for-dh_-command dh_gnome 
> > => gnome-pkg-tools
> > 
> > libmanette build-depends on dh-sequence-gnome which is provided by 
> > gnome-pkg-tools
> 
> Hm. Not sure what is going on here as we appear to already cover
> this since:
> 
>   
> https://salsa.debian.org/lintian/lintian/commit/f8887189f6bafcf799889b271ce30e8cd7311c03
> 
> 
> Regards,
> 
> -- 
>   ,''`.
>  : :'  : Chris Lamb
>  `. `'`  la...@debian.org  chris-lamb.co.uk
>`-
> 
> 

Hi Chris,

I think that commit is missing "dh_gnome" in
data/debhelper/dh_commands-manual?   At least matches the tag output.

~Niels

Bug#950385: lintian: Crash error on lintian

[Niels Thykier]

Package: lintian
Version: 2.47.0
Severity: normal

Hi,

I spotted this error on lindsay.d.o in a log file that was not exposed
in the published log (presumably, lintian crashed to the point where
harness did not include the log):


"""
[...]
tar: nexuiz-data-2.5.2/misc: Cannot mkdir: No space left on device
tar: 
nexuiz-data-2.5.2/misc/mediasource/menuskins/wickedz/background_builder/background_ingame_25.tga:
 Cannot open: No such file or directory
tar: nexuiz-data-2.5.2/misc: Cannot mkdir: No space left on device
tar: 
nexuiz-data-2.5.2/misc/mediasource/menuskins/wickedz/background_builder/background_l2.tga:
 Cannot open: No such file or directory
tar: nexuiz-data-2.5.2/misc: Cannot mkdir: No space left on device
tar: 
nexuiz-data-2.5.2/misc/mediasource/menuskins/wickedz/background_builder/vfont_7.tga:
 Cannot open: No such file or directory
tar: nexuiz-data-2.5.2/misc: Cannot mkdir: NoN: Finished processing group 
nexuiz-data/2.5.2-9
N: Starting on group nickle/2.85-1
N: Unpacking packages in group nickle/2.85-1
warning: collection unpacked failed for source package nexuiz-data, skipping 
check error: 512
Can't close filehandle 'STDOUT': 'No space left on device' at 
/srv/lintian.debian.org/lintian/checks/manpages.pm line 243
Lintian::manpages::close("STDOUT") called at 
/srv/lintian.debian.org/lintian/checks/manpages.pm line 243
Lintian::manpages::files(Lintian::manpages=HASH(0x564ed385be60), 
Lintian::Path=HASH(0x564ed3206e90)) called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Che
ck.pm line 79
Lintian::Check::run(Lintian::manpages=HASH(0x564ed385be60)) called at 
/srv/lintian.debian.org/lintian/lib/Lintian/CheckScript.pm line 235

Lintian::CheckScript::run_check(Lintian::CheckScript=HASH(0x564ec30439f8), 
Lintian::Processable::Binary=HASH(0x564ec3291a88), 
Lintian::Processable::Group=HASH
(0x564ec3269438)) called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Processable/Group.pm line 708
eval {...} called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Processable/Group.pm line 708

Lintian::Processable::Group::process(Lintian::Processable::Group=HASH(0x564ec3269438),
 Lintian::Tags=HASH(0x564ec305e130), SCALAR(0x564ec1d2b688), HASH(0x564e
c28e3ba8), HASH(0x564ec18de068), CODE(0x564ec2a92340), 
Lintian::Output::FullEWI=HASH(0x564ec2d46788)) called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Processabl
e/Pool.pm line 266

Lintian::Processable::Pool::process(Lintian::Processable::Pool=HASH(0x564ec1d2a908),
 "check", Lintian::Profile=HASH(0x564ec2d174d8), Lintian::Tags=HASH(0x564e
c305e130), SCALAR(0x564ec1d2b688), HASH(0x564ec18de068), CODE(0x564ec2a92340), 
GLOB(0x564ec2d74c60), ...) called at 
/srv/lintian.debian.org/lintian/commands/lintian.p
m line 844
main::main() called at /srv/lintian.debian.org/lintian/frontend/lintian 
line 46
eval {...} called at /srv/lintian.debian.org/lintian/frontend/lintian 
line 46
main::__ANON__("/srv/lintian.debian.org/lintian/commands/lintian.pm") 
called at /srv/lintian.debian.org/lintian/frontend/lintian line 115
dplint::run_tool("/srv/lintian.debian.org/lintian/frontend/lintian", 
"lintian") called at /srv/lintian.debian.org/lintian/frontend/lintian line 291
dplint::main() called at 
/srv/lintian.debian.org/lintian/frontend/lintian line 375
internal error: cannot run manpages checkwarning: skipping check of 
binary:nickle/2.85-1/i386
Filehandle STDOUT reopened as $_[...] only for input at (eval 143) line 149.
Filehandle STDOUT reopened as $_[...] only for input at (eval 328) line 149.
Filehandle STDOUT reopened as $_[...] only for input at (eval 129) line 149.
Filehandle STDOUT reopened as $_[...] only for input at (eval 129) line 149.
Filehandle STDOUT reopened as $_[...] only for input at (eval 129) line 149.
N: Interrupted.
internal error: cannot run manpages checkwarning: skipping check of 
binary:nickle/2.85-1/i386
Can't close filehandle 'STDOUT': 'No space left on device' at 
/srv/lintian.debian.org/lintian/checks/manpages.pm line 243
Lintian::manpages::close("STDOUT") called at 
/srv/lintian.debian.org/lintian/checks/manpages.pm line 243
Lintian::manpages::files(Lintian::manpages=HASH(0x564ed3893798), 
Lintian::Path=HASH(0x564ed3227cc8)) called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Check.pm line 79
Lintian::Check::run(Lintian::manpages=HASH(0x564ed3893798)) called at 
/srv/lintian.debian.org/lintian/lib/Lintian/CheckScript.pm line 235

Lintian::CheckScript::run_check(Lintian::CheckScript=HASH(0x564ec30439f8), 
Lintian::Processable::Binary=HASH(0x564ec3288a80), 
Lintian::Processable::Group=HASH(0x564ec3269438)) called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Processable/Group.pm line 708
eval {...} called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Processable/Group.pm line 708

Lintian::Processable::Group::process(Lintian::Processable::Group=HASH(0x564ec3269438),
 

Bug#924449: lintian: Periodic "out of disk space" errors from lindsay.d.o

[Niels Thykier]

Adam D. Barratt:
> On 2019-03-13 06:40, Niels Thykier wrote:
>> Package: lintian
>> Version: 2.10.0
>> Severity: important
>>
>> We seem to be getting regular "out of disk space" errors on
>> lindsay.d.o after resuming the archive-wide processing.
> [...]
>> AFAICT, lintian correctly cleans up old packages during the run.
>> I.e. we are not accumulating disk usage from previous packages in the
>> run.
>>
>> For reference, we currently have 32GB disk available for lintian ("all
>> inclusive").  About 23GB of that is available for checking with our
>> current usage for reports and stats.
> 
> For the record, lintian.d.o filled all the available space on lindsay's
> /srv again in the past day or so.
> 
> I killed a running check for insighttoolkit4, but there was still 23GB
> in scratch/tmp-lintian-lab*, in several folders dating back to the 28th
> of this month. I've removed a few of the older ones and checks were
> progressing again, but it seems like there are still some issues here.
> 
> Regards,
> 
> Adam

Hi Adam,

I added a crontab entry to automatically clean up scratch in case
something like that happened.  Perhaps we want to tweak the limit for
that to have lindsay.d.o recover faster on its own.

Thanks,
~Niels

Bug#949799: lintian: "Can't call method "upstream" on an undefined value at /srv/lintian.debian.org/lintian/checks/debian/watch.pm line 64."

[Niels Thykier]

Felix Lechner:
> Hi Niels,
> 
> On Sat, Jan 25, 2020 at 12:45 AM Niels Thykier  wrote:
>>
>> Version: 2.45.0
> 
> I cannot reproduce the error with version 2.45.0. (Plus in that
> version, line 64 is a comment.)
> 
>> E: cachefilesd source (0.10.10-0.2) [source]: 
>> malformed-debian-changelog-version 0.10.10-0.2 (for native)
>> Can't call method "upstream" on an undefined value at 
>> /srv/lintian.debian.org/lintian/checks/debian/watch.pm line 64.
>> internal error: cannot run debian/watch check on package 
>> source:cachefilesd/0.10.10-0.2
>> warning: skipping check of source:cachefilesd/0.10.10-0.2
>> N: 
> 
> I noticed the error in version 2.40.0 and believe it was fixed with
> 
> 
> https://salsa.debian.org/lintian/lintian/commit/e580062fee8a81e91a17150eff3abebf51702b71
> 
> Did you maybe look through an older log?
> 

Sounds reasonable; the lintian.log does contain results from older runs
without an indication of which version processed it. :)

> Closing this bug. Please reopen if a mistake was made.
> 
> Kind regards
> Felix Lechner
> 

I will trust you on this; let us leave it closed. :)

Thanks,
~Niels

Bug#949805: lintian: --status-log says everything was processed with an error

[Niels Thykier]

Package: lintian
Version: 2.45.0
Severity: important

Hi,

Consider the following command:

"""
$ frontend/lintian --status-log '&2' ../debhelper_12.8_source.changes  ; echo $?
error debhelper/12.8 (2.869s)
N: 1 tag overridden (1 info)
0
"""

This should have said "complete debhelper/12.8 (...s)" as it
successfully processed the package provided.  


I *think* this is due to coll_hook in Lintian::Processible::Group
returning 0 for the "finish" hook rather than 1 (the caller interprets
this as a failure, which matches how "start-failed" hook is handled).

This bug is causing lindsay.d.o to flag everything as failed and
reprocess everything even when some of them were successfully
processed.

Thanks,
~Niels

Bug#949799: lintian: "Can't call method "upstream" on an undefined value at /srv/lintian.debian.org/lintian/checks/debian/watch.pm line 64."

[Niels Thykier]

Package: lintian
Version: 2.45.0
Severity: normal

Hi,

I noticed that lintian reported a high number of packages with processing
errors on https://lintian.debian.org/.  Digging into the log, I noted the
following error:

"""
N: Starting on group cachefilesd/0.10.10-0.2
N: Unpacking packages in group cachefilesd/0.10.10-0.2
N: 
N: Processing source package cachefilesd (version 0.10.10-0.2, arch source) ...
[...]
E: cachefilesd source (0.10.10-0.2) [source]: 
malformed-debian-changelog-version 0.10.10-0.2 (for native)
Can't call method "upstream" on an undefined value at 
/srv/lintian.debian.org/lintian/checks/debian/watch.pm line 64.
internal error: cannot run debian/watch check on package 
source:cachefilesd/0.10.10-0.2
warning: skipping check of source:cachefilesd/0.10.10-0.2
N: 
"""

Thanks,
~Niels

Bug#949797: lintian: "Use of uninitialized value in numeric comparison (<=>) at /srv/lintian.debian.org/lintian/lib/Lintian/Output/FullEWI.pm line 113."

[Niels Thykier]

Package: lintian
Version: 2.45.0
Severity: minor

Hi,

With #949398 fixed, lintian has now started processing packages again
on lindsay.d.o.  I spotted this warning in the logs:

"""
Use of uninitialized value in numeric comparison (<=>) at 
/srv/lintian.debian.org/lintian/lib/Lintian/Output/FullEWI.pm line 113.
"""

It appears repeatedly for some packages.

Thanks,
~Niels

Bug#949398: marked as pending in lintian

[Niels Thykier]

Chris Lamb:
> Hi Felix,
> 
>> Please feel free to restart lintian.d.o. (I do not have access.) I
>> think it's been down for months.
>  
> Good thinking. :)
> 
> Just for a bit of background my preference is to keep lintian.d.o
> running a specific released version of Lintian, otherwise we make our
> bug reports ambiguous about what version they are using and thus cause
> more work for everyone.
> 
> However, I plan to release Lintian tomorrow (ie. Friday 24th Jan) or
> more accurately when the current version in unstable migrates to
> testing. I will assume that (another 24 hours delay would not be a
> problem for you both.
> 
> 
> Regards,
> 

Hi,

Personally, I am fine with the delay.

Though I can tell you it already migrated:

"""
$ dak ls lintian
lintian| 2.5.30+deb8u4  | oldoldstable  | source, all
lintian| 2.5.50.4   | oldstable | source, all
lintian| 2.15.0 | stable| source, all
lintian| 2.45.0~bpo9+1  | stretch-backports | source, all
lintian| 2.45.0~bpo10+1 | buster-backports  | source, all
lintian| 2.46.0 | testing   | source, all
lintian| 2.46.0 | unstable  | source, all
"""

It will probably be several hours until it will be published on the
mirrors and the migration mail will be sent.

Thanks,
~Niels

Bug#949398: marked as pending in lintian

[Niels Thykier]

Felix Lechner:
> Hi,
> 
> On Thu, Jan 23, 2020 at 1:57 PM Niels Thykier  wrote:
>>
>> Thanks for the quick response time and a fix. :)
> 
> Please feel free to restart lintian.d.o. (I do not have access.) I
> think it's been down for months.
> 

I do not have write access either any more.  We will need Chris or
another lintian maintainer to do that.

Technically, lintian.d.o should be running already, but this bug is/was
causing lintian to skip all the work making it appear as if lintian on
lintian.d.o was stopped.

> Also, please note that I fixed much of the blacklist (per #946320) but
> cannot adjust it.
> 
> Kind regards
> Felix
> 

Same issue here for me.  Perhaps it would make sense to move the live
config file under git and enable you to make changes to it via git /
merge requests.

Thanks,
~Niels

Bug#949398: marked as pending in lintian

[Niels Thykier]

Felix Lechner:
> Control: tag -1 pending
> 
> Hello,
> 
> Bug #949398 in lintian reported by you has been fixed in the
> Git repository and is awaiting an upload. You can see the commit
> message below and you can check the diff of the fix at:
> 
> https://salsa.debian.org/lintian/lintian/commit/544acaf9d364fb8285b8e79d3d4dadb56d9ce1db
> 
> 
> Skip only empty lines when packages are specified in a file. (Closes: #949398)
> 
> All lines were skipped before the end-of-string marker was added.
> 
> 
> (this message was generated automatically)
> 

Thanks for the quick responds time and a fix. :)

Thanks,
~Niels

Bug#949398: lintian: --packages-from-file is broken

[Niels Thykier]

Package: lintian
Version: 2.45.0
Severity: normal

Hi,

At some point, "--packages-from-file -" stopped working.  Below is the
output of --packages-from-file from 2.45.0 and 2.6.0.

"""
$ lintian --print-version ; echo '../debhelper_12.8_amd64.changes' | lintian 
-EvIL +pedantic  --packages-from-file -
2.45.0
N: Using profile debian/main.
N: No packages selected.
$ frontend/lintian --print-version ; echo '../debhelper_12.8_amd64.changes' | 
frontend/lintian -EvIL +pedantic  --packages-from-file -
2.6.0
N: Using profile debian/main.
N: Setting up lab in /tmp/temp-lintian-lab-K3fvWmB1T5 ..
[...]
"""

It appears that 2.45.0 just says "No packages selected." while 2.6.0
starts processing the package.  I suspect (but do not know for
certain) that this is related to why lindsay.d.o is not processing any
packages at the moment.  At least the log file on from lintian.d.o
starts with:

"""
N: Using profile debian/main.
N: No packages selected.
N: ---start-of-old-lintian-log-file---
N: Processing source package aether-ant-tasks (version 1.0.1-5, arch source) ...
"""

Thanks,
~Niels

Bug#939656: dh_strip should strip sections with LTO information from .a and .o files / lintian should warn about these

[Niels Thykier]

Matthias Klose:
> On 16.10.19 22:08, Niels Thykier wrote:
>> Control: tags -1 moreinfo
>>
>> Matthias Klose:
>>> Control: tags -1 - moreinfo
>>>
>>> On 29.09.19 11:03, Niels Thykier wrote:
>>>> Control: tags -1 moreinfo
>>>>
>>>> [...]
>>>>
>>>> When would you need to keep these LTO sections (but not use -k to keep
>>>> everything)?
>>>
>>> -k is aliased as --keep-debug, this has nothing to do with LTO
>>> sections.  Why do you want to blow up the size of a package needing LTO
>>> when only LTO is needed?
> 
> sorry, but it feels like you are evading that question.

Honestly, I do not think that question was anything other then
rhetorical.  But also, the question is irrelevant to whether I am
willing to support the extra complexity to support "yet-another-switch"
in dh_strip.
  Either there is a use-case for /optionally/ keeping LTO in a
non-trivial amount of special-cases or dh_strip will simply not support
the option and will strip LTO symbols unconditionally.  Or a third
alternative is that the problem degenerates into a variant of
#941065/#595810 (in which case the bug will be merged with the relevant
bug).

~Niels

Bug#939656: dh_strip should strip sections with LTO information from .a and .o files / lintian should warn about these

[Niels Thykier]

Control: tags -1 moreinfo

Matthias Klose:
> Control: tags -1 - moreinfo
> 
> On 29.09.19 11:03, Niels Thykier wrote:
>> Control: tags -1 moreinfo
>>
>> [...]
>>
>> When would you need to keep these LTO sections (but not use -k to keep
>> everything)?
> 
> -k is aliased as --keep-debug, this has nothing to do with LTO
> sections.  Why do you want to blow up the size of a package needing LTO
> when only LTO is needed?
> 
> Keeping LTO information in a package can be useful when you have tightly
> coupled packages where you know that LTO optimization can be done across
> package boundaries.
> 
> Matthias

Can you name me some concrete examples of packages / relations where
keeping LTO information would be beneficial, useful and worth the extra
complexity/cognitive load of "yet-another-toggle-in-debhelper" that
everyone needs to consider whether applies to them?

Thanks,
~Niels

Bug#939656: dh_strip should strip sections with LTO information from .a and .o files / lintian should warn about these

[Niels Thykier]

Control: tags -1 moreinfo

Matthias Klose:
> Package: debhelper,lintian
> Severity: important
> 
> Some packages build with link time optimizations enabled, which is ok,
> whoever then these packages may ship with static libs which still have
> the LTO information in some sections of the object files (e.g.
> ext2fsprogs).  This is not desired in most cases, so this information
> should be removed from these files, and not shipped in the archive. Plus
> the streaming format for the LTO information changes (even in GCC minor
> releases), and leads to build errors when you try to use an old
> streaming format with a newer compiler.  I'm asking for
> 
>  - dh_strip removing sections, as in
> 
>  strip -R .gnu.lto_* -R .gnu.debuglto_* -N __gnu_lto_slim -N
> __gnu_lto_v1
> 
>    which is turned on by default.
> 
>  - dh_strip providing an option not to remove these sections.
> 
>  - lintian warning about object files and static archives having such
>    sections.
> 
> I'd like to see that implemented in debhelper, because LTO builds are
> also sometimes enabled in upstream sources.
> 
> LTO is turned on by default in Suse, and their dh_strip equivalent
> provides the functionality above for the removal of the LTO information.
> 
> Please feel free to split this issue into separate debhelper and lintian
> tasks once a solution is agree upon.
> 

When would you need to keep these LTO sections (but not use -k to keep
everything)?

Thanks,
~Niels

Bug#935292: lintian: reporing-harness get stuck when lintian deadlocks

[Niels Thykier]

On Wed, 21 Aug 2019 11:35:00 + Niels Thykier  wrote:
> Niels Thykier:
> > Package: lintian
> > Version: 2.18.0
> > Severity: important
> > 
> > Spotted on lindsay.d.o:
> > 
> > The reporting framework runs lintian on a timer and can send it
> > SIGTERM if it gets stuck (usually when the processing time is over).
> > However, it seems that the reporting framework sometimes just
> > dies/disappeares after lintian is killed.  Below is the log from where
> > lintian was manually killed by me on a stuck package - but there are
> > plenty of examples that suggest it also happens when the
> > reporting-lintian-harness automatically kills lintian on a timeout.
> > 
> > """
> > [2019-08-21T10:26:24]: New lintian log at 
> > /srv/lintian.debian.org/logs/lintian.log-bPGtadL
> > [2019-08-21T10:29:05]:   [lintian] error processing golang-1.11/1.11.13-1 
> > (time: 122.898s)
> > [2019-08-21T10:29:06]:   [lintian] processed 
> > golang-github-mcuadros-go-gin-prometheus/0.1.0+git20190723.c7374e9-2 
> > successfully (time: 1.396s)
> > [2019-08-21T10:32:37]:   [lintian] processed grass/7.8.0~rc1-1~exp1 
> > successfully (time: 210.685s)
> > [2019-08-21T10:33:45]:   [lintian] processed grpc/1.23.0-1 successfully 
> > (time: 68.446s)
> > [2019-08-21T10:33:50]:   [lintian] processed guake/3.6.3-1 successfully 
> > (time: 5.118s)
> > [2019-08-21T10:33:56]:   [lintian] processed intel-cmt-cat/3.1.0-2 
> > successfully (time: 5.953s)
> > [2019-08-21T10:34:03]:   [lintian] processed intel-processor-trace/2.0.1-1 
> > successfully (time: 6.566s)
> > [2019-08-21T10:34:06]:   [lintian] processed libbpp-qt/2.4.1-2 successfully 
> > (time: 3.384s)
> > [2019-08-21T10:34:13]:   [lintian] processed libbpp-seq-omics/2.4.1-4 
> > successfully (time: 7.356s)
> > [2019-08-21T10:34:17]:   [lintian] processed 
> > libconfig-model-itself-perl/2.018-1 successfully (time: 3.495s)
> > [2019-08-21T10:34:21]:   [lintian] processed libseqlib/1.1.2+dfsg-4 
> > successfully (time: 3.913s)
> > [2019-08-21T10:55:19]:   [lintian] processed linux-signed-i386/5.2.9+1 
> > successfully (time: 1258.298s)
> > [2019-08-21T10:55:42]:   [lintian] processed mandos/1.8.8-1 successfully 
> > (time: 22.420s)
> > [2019-08-21T11:15:26]: Signal SIGTERM acknowledged: disabled timed alarms
> > """
> > 
> > After this, the harness.log simply "stops".
> > 
> > [...]
> 
> Correction, it died.  I spotted a mail from cron in
> /var/spool/mail/lintian saying it could not send a main due to invalid
> cmd-line parameters to the mail-cmd in our crontab.  I have "fixed" this
> setting MAILTO and removing the explicit mail-cmd - it has apparently
> never worked.
> 
> Thanks,
> ~Niels
> 
> 
> 
> 

We are still experiencing this issue and have now almost 60 blacklisted
packages to work around this (and a few other issues).  Sadly, mailing
does not work, so we do not get the actual error from
reporting-lintian-harness (so I do not have any additional debug
information).

This issue neuters the lintian processing as all results end up being
discarded due to this issue.

Thanks,
~Niels

Bug#935292: lintian: reporing-harness get stuck when lintian deadlocks

[Niels Thykier]

Niels Thykier:
> Package: lintian
> Version: 2.18.0
> Severity: important
> 
> Spotted on lindsay.d.o:
> 
> The reporting framework runs lintian on a timer and can send it
> SIGTERM if it gets stuck (usually when the processing time is over).
> However, it seems that the reporting framework sometimes just
> dies/disappeares after lintian is killed.  Below is the log from where
> lintian was manually killed by me on a stuck package - but there are
> plenty of examples that suggest it also happens when the
> reporting-lintian-harness automatically kills lintian on a timeout.
> 
> """
> [2019-08-21T10:26:24]: New lintian log at 
> /srv/lintian.debian.org/logs/lintian.log-bPGtadL
> [2019-08-21T10:29:05]:   [lintian] error processing golang-1.11/1.11.13-1 
> (time: 122.898s)
> [2019-08-21T10:29:06]:   [lintian] processed 
> golang-github-mcuadros-go-gin-prometheus/0.1.0+git20190723.c7374e9-2 
> successfully (time: 1.396s)
> [2019-08-21T10:32:37]:   [lintian] processed grass/7.8.0~rc1-1~exp1 
> successfully (time: 210.685s)
> [2019-08-21T10:33:45]:   [lintian] processed grpc/1.23.0-1 successfully 
> (time: 68.446s)
> [2019-08-21T10:33:50]:   [lintian] processed guake/3.6.3-1 successfully 
> (time: 5.118s)
> [2019-08-21T10:33:56]:   [lintian] processed intel-cmt-cat/3.1.0-2 
> successfully (time: 5.953s)
> [2019-08-21T10:34:03]:   [lintian] processed intel-processor-trace/2.0.1-1 
> successfully (time: 6.566s)
> [2019-08-21T10:34:06]:   [lintian] processed libbpp-qt/2.4.1-2 successfully 
> (time: 3.384s)
> [2019-08-21T10:34:13]:   [lintian] processed libbpp-seq-omics/2.4.1-4 
> successfully (time: 7.356s)
> [2019-08-21T10:34:17]:   [lintian] processed 
> libconfig-model-itself-perl/2.018-1 successfully (time: 3.495s)
> [2019-08-21T10:34:21]:   [lintian] processed libseqlib/1.1.2+dfsg-4 
> successfully (time: 3.913s)
> [2019-08-21T10:55:19]:   [lintian] processed linux-signed-i386/5.2.9+1 
> successfully (time: 1258.298s)
> [2019-08-21T10:55:42]:   [lintian] processed mandos/1.8.8-1 successfully 
> (time: 22.420s)
> [2019-08-21T11:15:26]: Signal SIGTERM acknowledged: disabled timed alarms
> """
> 
> After this, the harness.log simply "stops".
> 
> [...]

Correction, it died.  I spotted a mail from cron in
/var/spool/mail/lintian saying it could not send a main due to invalid
cmd-line parameters to the mail-cmd in our crontab.  I have "fixed" this
setting MAILTO and removing the explicit mail-cmd - it has apparently
never worked.

Thanks,
~Niels

Bug#935292: lintian: reporing-harness get stuck when lintian deadlocks

[Niels Thykier]

Package: lintian
Version: 2.18.0
Severity: important

Spotted on lindsay.d.o:

The reporting framework runs lintian on a timer and can send it
SIGTERM if it gets stuck (usually when the processing time is over).
However, it seems that the reporting framework sometimes just
dies/disappeares after lintian is killed.  Below is the log from where
lintian was manually killed by me on a stuck package - but there are
plenty of examples that suggest it also happens when the
reporting-lintian-harness automatically kills lintian on a timeout.

"""
[2019-08-21T10:26:24]: New lintian log at 
/srv/lintian.debian.org/logs/lintian.log-bPGtadL
[2019-08-21T10:29:05]:   [lintian] error processing golang-1.11/1.11.13-1 
(time: 122.898s)
[2019-08-21T10:29:06]:   [lintian] processed 
golang-github-mcuadros-go-gin-prometheus/0.1.0+git20190723.c7374e9-2 
successfully (time: 1.396s)
[2019-08-21T10:32:37]:   [lintian] processed grass/7.8.0~rc1-1~exp1 
successfully (time: 210.685s)
[2019-08-21T10:33:45]:   [lintian] processed grpc/1.23.0-1 successfully (time: 
68.446s)
[2019-08-21T10:33:50]:   [lintian] processed guake/3.6.3-1 successfully (time: 
5.118s)
[2019-08-21T10:33:56]:   [lintian] processed intel-cmt-cat/3.1.0-2 successfully 
(time: 5.953s)
[2019-08-21T10:34:03]:   [lintian] processed intel-processor-trace/2.0.1-1 
successfully (time: 6.566s)
[2019-08-21T10:34:06]:   [lintian] processed libbpp-qt/2.4.1-2 successfully 
(time: 3.384s)
[2019-08-21T10:34:13]:   [lintian] processed libbpp-seq-omics/2.4.1-4 
successfully (time: 7.356s)
[2019-08-21T10:34:17]:   [lintian] processed 
libconfig-model-itself-perl/2.018-1 successfully (time: 3.495s)
[2019-08-21T10:34:21]:   [lintian] processed libseqlib/1.1.2+dfsg-4 
successfully (time: 3.913s)
[2019-08-21T10:55:19]:   [lintian] processed linux-signed-i386/5.2.9+1 
successfully (time: 1258.298s)
[2019-08-21T10:55:42]:   [lintian] processed mandos/1.8.8-1 successfully (time: 
22.420s)
[2019-08-21T11:15:26]: Signal SIGTERM acknowledged: disabled timed alarms
"""

After this, the harness.log simply "stops".

The package triggering it in this case was "mgcv" (now blacklisted on
lindsay.d.).  The lintian log looks like the following:

"""
N: Finished processing group mandos/1.8.8-1
N: Starting on group mgcv/1.8-28-2
N: Unpacking packages in group mgcv/1.8-28-2
N: Interrupted.
"""

The last line appeared after I killed lintian.  Please see [1] for the
last bit of the performance log which tells you what tasks lintian
managed to complete on mgcv.


This has probably been a key reason for why the lintian.d.o website
has not been updated in the past few days.

I hope it is helpful in debugging the issue.

Thanks,
~Niels


[1] Note the first line (mandos) is from the previous source just to ensure
that everything from mgcv is present.

"""
mandos/1.8.8-1,total-group-auto-remove,0.310351
binary:r-cran-mgcv/1.8-28-2/i386,coll/unpacked,1.007855
binary:r-cran-mgcv/1.8-28-2/amd64,coll/unpacked,1.168259
binary:r-cran-mgcv/1.8-28-2/i386,coll/file-info,0.524697
source:mgcv/1.8-28-2,coll/unpacked,0.9776
binary:r-cran-mgcv/1.8-28-2/amd64,coll/file-info,0.662295
binary:r-cran-mgcv/1.8-28-2/i386,coll/md5sums,0.216517
binary:r-cran-mgcv/1.8-28-2/amd64,coll/md5sums,0.187911
source:mgcv/1.8-28-2,coll/md5sums,0.230758
binary:r-cran-mgcv/1.8-28-2/i386,coll/objdump-info,0.317088
binary:r-cran-mgcv/1.8-28-2/amd64,coll/objdump-info,0.31345
binary:r-cran-mgcv/1.8-28-2/i386,coll/strings,0.342909
binary:r-cran-mgcv/1.8-28-2/amd64,coll/strings,0.366867
source:mgcv/1.8-28-2,coll/src-orig-index,0.306694
source:mgcv/1.8-28-2,coll/file-info,2.820701
binary:r-cran-mgcv/1.8-28-2/i386,coll/bin-pkg-control,0.569369
binary:r-cran-mgcv/1.8-28-2/i386,coll/override-file,0.083423
binary:r-cran-mgcv/1.8-28-2/amd64,coll/override-file,0.08031
binary:r-cran-mgcv/1.8-28-2/amd64,coll/bin-pkg-control,0.695504
source:mgcv/1.8-28-2,coll/override-file,0.127517
binary:r-cran-mgcv/1.8-28-2/i386,coll/ar-info,0.095013
binary:r-cran-mgcv/1.8-28-2/amd64,coll/ar-info,0.107359
binary:r-cran-mgcv/1.8-28-2/i386,coll/java-info,0.108112
binary:r-cran-mgcv/1.8-28-2/amd64,coll/java-info,0.125803
binary:r-cran-mgcv/1.8-28-2/i386,coll/copyright-file,0.072609
binary:r-cran-mgcv/1.8-28-2/amd64,coll/copyright-file,0.060382
binary:r-cran-mgcv/1.8-28-2/i386,coll/scripts,0.095971
binary:r-cran-mgcv/1.8-28-2/amd64,coll/scripts,0.096182
source:mgcv/1.8-28-2,coll/diffstat,0.065437
binary:r-cran-mgcv/1.8-28-2/i386,coll/changelog-file,0.138974
"""

Bug#926543: lintian: Deadlock in source-copyright check on source:khronos-opencl-man/1.0~svn33624-4

[Niels Thykier]

On Fri, 9 Aug 2019 08:37:48 -0700 Felix Lechner
 wrote:
> Hi,
> 
> On Fri, Aug 9, 2019 at 3:57 AM Andrey Rahmatullin  wrote:
> >
> > It's the same with lintian from sid (2.16.0) on xhtml2pdf 0.2.2-2 and 
> > 0.2.2-3.
> 
> I do not see any issues terminating locally. I used both Lintian
> master and 2.16.0 on the xhtml2pdf source packages, per below. I also
> tried, locally, the more complex command from above on
> khronos-opencl-man_1.0~svn33624-4.dsc. Attached please find the log
> showing that success, as well.
> 
> Can your errors be reproduced locally, or are they limited to lindsay?
> 
> Kind regards,
> Felix
> 
> [...]

FTR: I can still reproduce this issue with 2.18 on lindsay.d.o today
(tested hronos-opencl-man again).

Thanks,
~Niels

Bug#935257: lintian: Dies with "Already have a handler for XXXXX at /usr/share/perl5/IO/Async/Loop.pm line 2825."

[Niels Thykier]

Package: lintian
Version: 2.18.0
Severity: important

Spotted on lindsay.d.o while processing gcc-9.

"""
C: libubsan1 binary (9.2.1-5) [i386]: control-tarball-compression-format xz
C: libubsan1 binary (9.2.1-5) [i386]: data-tarball-compression-format xz
I: libubsan1 binary (9.2.1-5) [i386]: 
symbols-file-missing-build-depends-package-field
Already have a handler for 10981 at /usr/share/perl5/IO/Async/Loop.pm line 2825.

IO::Async::Loop::watch_child(IO::Async::Loop::Epoll=HASH(0x563f9ed2bea0), 
10981, CODE(0x563fa13ec270)) called at 
/usr/share/perl5/IO/Async/Internals/ChildMana
ger.pm line 291

IO::Async::Internals::ChildManager::_spawn_in_parent(IO::Async::Internals::ChildManager=HASH(0x563f9f5207e8),
 GLOB(0x563fa176a868), 10981, CODE(0x563fbdeac808
)) called at /usr/share/perl5/IO/Async/Internals/ChildManager.pm line 145

IO::Async::Internals::ChildManager::spawn_child(IO::Async::Internals::ChildManager=HASH(0x563f9f5207e8),
 "on_exit", CODE(0x563fbdeac808), "code", CODE(0x563fb
deb2b58), "setup", ARRAY(0x563f9a91c5d0), "command", ...) called at 
/usr/share/perl5/IO/Async/Loop.pm line 1098

IO::Async::Loop::spawn_child(IO::Async::Loop::Epoll=HASH(0x563f9ed2bea0), 
"code", CODE(0x563fbdeb2b58), "command", undef, "setup", ARRAY(0x563f9a91c5d0), 
"on_
exit", ...) called at /usr/share/perl5/IO/Async/Process.pm line 532

IO::Async::Process::_add_to_loop(IO::Async::Process=HASH(0x563fbe7643a8), 
IO::Async::Loop::Epoll=HASH(0x563f9ed2bea0)) called at 
/usr/share/perl5/IO/Async/Not
ifier.pm line 301

IO::Async::Notifier::__set_loop(IO::Async::Process=HASH(0x563fbe7643a8), 
IO::Async::Loop::Epoll=HASH(0x563f9ed2bea0)) called at 
/usr/share/perl5/IO/Async/Loop
.pm line 407

IO::Async::Loop::_add_noparentcheck(IO::Async::Loop::Epoll=HASH(0x563f9ed2bea0),
 IO::Async::Process=HASH(0x563fbe7643a8)) called at /usr/share/perl5/IO/Async/
Loop.pm line 395
IO::Async::Loop::add(IO::Async::Loop::Epoll=HASH(0x563f9ed2bea0), 
IO::Async::Process=HASH(0x563fbe7643a8)) called at 
/usr/share/perl5/IO/Async/Notifier.pm lin
e 422

IO::Async::Notifier::add_child(IO::Async::Function::Worker=HASH(0x563fbe7454a0),
 IO::Async::Process=HASH(0x563fbe7643a8)) called at /usr/share/perl5/IO/Async/
Routine.pm line 291

IO::Async::Routine::_setup_fork(IO::Async::Function::Worker=HASH(0x563fbe7454a0))
 called at /usr/share/perl5/IO/Async/Routine.pm line 202

IO::Async::Routine::_add_to_loop(IO::Async::Function::Worker=HASH(0x563fbe7454a0),
 IO::Async::Loop::Epoll=HASH(0x563f9ed2bea0)) called at 
/usr/share/perl5/IO/Async/Notifier.pm line 301

IO::Async::Notifier::__set_loop(IO::Async::Function::Worker=HASH(0x563fbe7454a0),
 IO::Async::Loop::Epoll=HASH(0x563f9ed2bea0)) called at 
/usr/share/perl5/IO/Async/Loop.pm line 407

IO::Async::Loop::_add_noparentcheck(IO::Async::Loop::Epoll=HASH(0x563f9ed2bea0),
 IO::Async::Function::Worker=HASH(0x563fbe7454a0)) called at 
/usr/share/perl5/IO/Async/Loop.pm line 395
IO::Async::Loop::add(IO::Async::Loop::Epoll=HASH(0x563f9ed2bea0), 
IO::Async::Function::Worker=HASH(0x563fbe7454a0)) called at 
/usr/share/perl5/IO/Async/Notifier.pm line 422

IO::Async::Notifier::add_child(IO::Async::Function=HASH(0x563fa23ef0c8), 
IO::Async::Function::Worker=HASH(0x563fbe7454a0)) called at 
/usr/share/perl5/IO/Async/Function.pm line 538

IO::Async::Function::_new_worker(IO::Async::Function=HASH(0x563fa23ef0c8)) 
called at /usr/share/perl5/IO/Async/Function.pm line 552

IO::Async::Function::_get_worker(IO::Async::Function=HASH(0x563fa23ef0c8)) 
called at /usr/share/perl5/IO/Async/Function.pm line 577

IO::Async::Function::_dispatch_pending(IO::Async::Function=HASH(0x563fa23ef0c8))
 called at /usr/share/perl5/IO/Async/Function.pm line 710

IO::Async::Function::Worker::__ANON__(IO::Async::Function::Worker=HASH(0x563fa16c9158),
 IO::Async::Future=HASH(0x563fbdeb2d50)) called at /usr/share/perl5/Future.pm 
line 462
Future::_mark_ready(IO::Async::Future=HASH(0x563fbdeb2d50), "done") 
called at /usr/share/perl5/Future.pm line 600
Future::done(IO::Async::Future=HASH(0x563fbdeb2d50)) called at 
/usr/share/perl5/Future.pm line 773
Future::on_ready(Future=HASH(0x563fa16c6d10), 
IO::Async::Future=HASH(0x563fbdeb2d50)) called at /usr/share/perl5/Future.pm 
line 453
Future::_mark_ready(IO::Async::Future=HASH(0x563fa1777988), "done") 
called at /usr/share/perl5/Future.pm line 600
Future::done(IO::Async::Future=HASH(0x563fa1777988), 
ARRAY(0x563f9f4fa3b0)) called at /usr/share/perl5/IO/Async/Channel.pm line 451
IO::Async::Channel::__ANON__(IO::Async::Channel=HASH(0x563f9f4fcc00), 
"recv", ARRAY(0x563f9f4fa3b0)) called at /usr/share/perl5/IO/Async/Channel.pm 
line 496
IO::Async::Channel::_on_stream_read(undef, undef, undef, 
IO::Async::Stream=HASH(0x563fa165dca8), SCALAR(0x563f9f506448), "") called at 

[Git][lintian/lintian] Deleted branch unpacker-io-async

[Niels Thykier]

Niels Thykier deleted branch unpacker-io-async at lintian / lintian

-- 

You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian] Deleted branch smoke-me/doc-rewrite

[Niels Thykier]

Niels Thykier deleted branch smoke-me/doc-rewrite at lintian / lintian

-- 

You're receiving this email because of your account on salsa.debian.org.

Re: [rt.debian.org #7900] AutoReply CC: lindsay.debian.org ENOSPC

[Niels Thykier]

Julien Cristau via RT:
> A new trouble ticket, a summary of which appears below the dashed line, 
> regarding
> 
>   "lindsay.debian.org ENOSPC",
> 
> has been created and the Requestor set you as a CC, which is why you are 
> receiving this autoreply-on-ticket-creation message.
> 
> In case you reply to this mail, please include the following string in the 
> subject line (excluding quotation marks):
> 
>   [rt.debian.org #7900]
> 
> -
> lindsay is out of disk space on /srv.  scratch/temp-lintian-lab-XX looks 
> huge.
> 
> https://munin.debian.org/debian.org/lindsay.debian.org/df.html
> 
> cc lintian-maint who can hopefully clean up and fix this.
> 

Hi,

I have cleaned up /srv now and inserted a cronjob that remove old cruft
from lintian in /srv/lintian.debian.org/scratch automatically after 24
hours (runs at 23:50).  It is definitely a work around as lintian is
supposed to clean up after itself (even on error).  However, at least it
is self-healing.

Thanks,
~Niels

Bug#931891: dh_dwz: handle compressed debug sections?

[Niels Thykier]

Control: reassign -1 dwz

Matthias Klose:
> Package: debhelper,lintian
> Severity: wishlist
> 
> dwz currently doesn't handle compressed debug sections.  In the first place, 
> why
> should we have these, because or dbg packages are compressed anyway. otoh, 
> these
> are enabled by some upstreams like ruby.
> 
> So what dh_dwz could do, is
> 
>  - look for SHF_COMPRESSED
>  - objcopy --decompress-debug-sections
>  - dwz
>  - objcopy --compress-debug-sections
> 
> Or lintian could just warn about compressed debug sections.
> 

Alternatively, dwz(1) could transparently decompress the debug sections,
do its thing and just leave the optimized file decompressed (or
recompress it).  Given dwz already knows how to detect this and is
parsing ELF binaries at an entirely different level than debhelper, it
is much better suited for this kind of logic (compared to debhelper).

Based on #933541, it would seem that golang is also using compressed
debug sections by default now.

Thanks,
~Niels

Bug#930679: Please add overridable tag for not using dh sequencer

[Niels Thykier]

Mattia Rizzolo:
> On Sun, Aug 04, 2019 at 11:05:23PM +0100, Chris Lamb wrote:
>> Somewhat related, but if we introduce this mooted "package-does-not-
>> use- dh-sequencer" we need to work out what to do with:
>>
>>   https://lintian.debian.org/tags/package-does-not-use-debhelper-or-cdbs.html
>>
>> One thing we can probably all agree with is that the severity of
>> package-does-not-use-debhelper-or-cdbs should be equal to or exceed
>> the severity of package-does-not-use-dh-sequencer as one is a logical
>> subset of another.
> 
> I just reported 3 bugs (#933901, #933902, #933903) with false positives
> of that tag.  I just looked a bunch of them and couldn't find a single
> true positive, so I think that tag should be reviewed before bumping its
> severity.
> 
> I think those bugs should be squashed, reviewed, and then bumped to W.
> Only once there are very few packages with that should
> package-goes-not-use-dh-sequencer be bumped to W as well.
> (note that package-does-not-)se-debhelper-or-cdbs does not emit for
> classic-style debhelper rules files.)
> 

The tag package-does-not-use-debhelper-or-cdbs would probably benefit
from using the same logic as debian-build-system, which tries to handle
some of these cases.  As I recall the %build_systems table holds exactly
this kind of information already.

Thanks,
~Niels

Bug#930311: lintian: Possible exception to package-contains-documentation-outside-usr-share-doc

[Niels Thykier]

Chris Lamb:
> Hi Niels,
> 
>> My question is: Should we move this exception to lintian itself and
>> stop having people automate overrides
> 
> Oh, without any doubt here — the idea of automatically-generated
> overrides simply makes me squirm.
> 
> (Shall we begin by cloning this bug "against" dh-r?)
> 
> 
> Regards,
> 

Re:
https://salsa.debian.org/lintian/lintian/commit/a16cd3a1c812c8894bddf9b920561eb0dd602d85

I suspect we should probably match usr/lib/R/site-library/ as a prefix
rather than an exact match.  My guess is that they have a "per-package"
folder structure beneath that directory.

Thanks,
~Niels

Bug#930311: lintian: Possible exception to package-contains-documentation-outside-usr-share-doc

[Niels Thykier]

Chris Lamb:
> Hi Niels,
> 
>> My question is: Should we move this exception to lintian itself and
>> stop having people automate overrides
> 
> Oh, without any doubt here — the idea of automatically-generated
> overrides simply makes me squirm.
> 
> (Shall we begin by cloning this bug "against" dh-r?)
> 
> 
> Regards,
> 

If we intend to create the exception in lintian, I would personally
probably go with making the exception first and then filing the bug
against dh-r to remove the auto-generation.  But either way works (for
me at least) and if you prefer the other order, then lets do that by all
means.

Thanks,
~Niels

Bug#930311: lintian: Possible exception to package-contains-documentation-outside-usr-share-doc

[Niels Thykier]

Package: lintian
Version: 2.15.0
Severity: normal

Hi,

I noticed that the dh-r package by default creates an override for
package-contains-documentation-outside-usr-share-doc when the R
package puts documentation in usr/lib/R/site-library:

"""
my $check_for_docs = `find debian/$dh{FIRSTPACKAGE} -type f -name "*.md" -o 
-name "*.Rmd" -o -name "README" -o -name "README.md" | grep -v 'usr/share/doc'`;
if ( $check_for_docs ) {
say "Create lintian-override for 
package-contains-documentation-outside-usr-share-doc due to $check_for_docs";
open(my $lintian, ">>", "debian/lintian-overrides");
say $lintian "# The documentation is where it is expected by GNU R 
users";
say $lintian "$dh{FIRSTPACKAGE}: 
package-contains-documentation-outside-usr-share-doc usr/lib/R/site-library/*";
close $lintian;
}
"""
(source: https://sources.debian.org/src/dh-r/20190121/dh/R.pm/?hl=3#L268)

My question is: Should we move this exception to lintian itself and
stop having people automate overrides or should something else be
done?  (To be explicit: The latter is an open question as I am not
sure what the "proper something else" would be in this case)

Thanks,
~Niels

Bug#928809: lintian: suggest adding gitlab-ci file

[Niels Thykier]

Chris Lamb:
> Dmitry Bogatov wrote:
> 
>>> [..]  I just think that lintian should be less pro-active at adding
>>> checks for things that are far from accepted.
>>
>> That is why I propose introducing concept of "controversial" checks.
> 
> I think we are all violently agreeing here. 
> 
>> Having Lintian plainly reject such proposals would mean need in creation
>> of something like "lintian-unofficial"
> 
> (Personally, I doubt someone would fork Lintian, more likely its
> output would become less and less "trusted". But both outcomes suck.)
> 
> 
> Best wishes,
> 

There would be no need to fork lintian over this.  Lintian supports
third-party checks (via third-party lintian profiles).  With files in
the proper place, you can have:

  lintian --profile debian/unofficial ...

run lintian's built-in plus the extra unofficial checks.

Thanks,
~Niels

Bug#927476: lintian: Please add --onlyrun examples

[Niels Thykier]

Chris Lamb:
> [...]
> Niels Thykier wrote:
> 
>> We already examples in Lintian::Tutorial::TestSuite
>> (doc/tutorial/Lintian/Tutorial/TestSuite.pod).
> […]
> 
> Agreed, although I was actually following t/bin/runtests --help here,
> not the POD documentation. Thus, both need to be updated? (Cloned bug)
> 
>> have not been updated to reflect the "recent" changes to the test suite
> 
> Presumably you are referring to the requirement (?) to first call t/
> bin/build-test-packages first (which I thought about when away from
> the keyboard last night).
> 

Not only that, but all the examples of how to only run a particular test
does not seem to match what you were using (the examples were from
before the test suite got restructured).

> If so, I think it's a really bizarre user interface that does not warn
> you specifically when these are missing so I have retitled this bug
> to match.
> 
> (I think I've just hit another bug in the tag extraction process which
> I will file after this mail...)
> 
> Thanks.
> 
> 
> Best wishes,
> 

Thanks,
~Niels

Bug#927476: lintian: Please add --onlyrun examples

[Niels Thykier]

Chris Lamb:
> Package: lintian
> Version: 2.12.0
> Severity: wishlist
> X-Debbugs-CC: Felix Lechner 
> 
> Hi,
> 
> I'm currently really failing to understand how --onlyrun works. For
> example:
> 
>   $ t/bin/runtests 
> --onlyrun=test:control-file-rules-requires-root-binary-targets
> 
> ... does not run any tests.
> 
> I'm sure I'm just calling it wrong and, to be clear, this bug report
> is not about this issue but rather could you please add some concrete
> examples to the manual page for each selector type (script, tag, etc.)
> 
> 
> Best wishes,
> 

We already examples in Lintian::Tutorial::TestSuite
(doc/tutorial/Lintian/Tutorial/TestSuite.pod).  However, I suspect they
have not been updated to reflect the "recent" changes to the test suite
and its runner.  I guess Lintian::Tutorial::WritingTests is equally out
of date.

Note: I am fine with removing those POD documents if we now prefer to
keep this documentation elsewhere.  However, if they remain, they should
be updated to reflect the reality.

Thanks,
~Niels

Bug#927220: lintian: Please detect and warn about libs exporting common symbols

[Niels Thykier]

Package: lintian
Version: 2.9.1
Severity: wishlist

Hi,

We discovered that libqb was accidentially exporting symbols such as
strlcat and strlcpy, which are implemented in libbsd.

Packages should *not* export strlcpy (or other common symbols) simply
because they happen to use an embeeded compat version of the symbol.

Possible solutions:

 * use symbol hiding to hide the symbol (e.g. 
"""__attribute__((visibility("hidden")))""" )

 * Build-Depend on or/and add the proper defines to expose the symbol
   (and thereby avoid using the embedded version).


Thanks,
~Niels

[Git][lintian/lintian] Deleted branch faster-test-builds

[Niels Thykier]

Niels Thykier deleted branch faster-test-builds at lintian / lintian

-- 

You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian] Pushed new branch faster-test-builds

[Niels Thykier]

Niels Thykier pushed new branch faster-test-builds at lintian / lintian

-- 
View it on GitLab: 
https://salsa.debian.org/lintian/lintian/tree/faster-test-builds
You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian][master] 12 commits: Implement universal output format in Lintian.

[Niels Thykier]

Niels Thykier pushed to branch master at lintian / lintian


Commits:
0f403c3d by Felix Lechner at 2019-04-15T09:46:09Z
Implement universal output format in Lintian.

This is a new output option that shortcuts to the output of
tagextract. This was done to avoid processing tags during the test, to
save time. This format is a tag-oriented, common subset of all the
other output formats. It is well-suited to evaluate how well a
particular check is working.

Among the notable features is the absence of the severity tag, which
has no bearing on the effectiveness of a check, as well as that the
output is always sorted with the packages on a particular order. For
example, source packages always come before binary packages.

The format looks like this:

scripts (source): binary-arch-rules-but-pkg-is-arch-indep
scripts (source): ancient-standards-version 3.2.1 (released 2000-08-24) 
(current is CURRENT)
scripts (binary): wrong-path-for-interpreter usr/bin/rubyfoo (#!/bin/ruby1.8 != 
/usr/bin/ruby1.8)
scripts (binary): wrong-path-for-interpreter usr/bin/lefty-foo 
(#!/usr/local/bin/lefty != /usr/bin/lefty)
scripts (binary): unusual-interpreter usr/bin/suidperlfoo #!/usr/bin/suidperl

- - - - -
ee7eb262 by Felix Lechner at 2019-04-15T09:52:19Z
Add default match strategy tags for tests; set default output format 
to universal; drop sort.

Adjusts the test case defaults to make the matching strategy for
tags, which was previously used in all tests, the explicit default.
This is done to so that another matching strategy, literal, can be
introduced. The second strategy will compare lintian output literally,
without any processing or sorting. It is useful, for example, when
comparing warning messages not in tag format.

Also sets the default output format to universal. This format is
newly available and saves calls to several Perl scripts in the test
runner. Hopefully, that will shorten the test execution times.

Tags in the universal format are always sorted. It generally does
not make sense to test for the order in which tags appear. The sorting
should really be done in Lintian for all output formats. It is no
longer available in the test runner.

Gbp-Dch: ignore

- - - - -
3657d2a0 by Felix Lechner at 2019-04-15T10:33:09Z
Add literal match strategy and output format, as needed, to tests that need it.

Adjusts the test specifications for a number of tests that benefit
from employing the literal match strategy.

Among those are test that use uncommon output formats, such as xml.
Their outputs were previously converted using tagextract, which took
extra time. Now their output compared directly. Very soon, their
outputs will be compared literally with a file called output in the
test specification directory.

There are also some false positive test that produce no output. Those
tests have also been switched to the literal matching stategy to
minimize the processing of their output. The theory is that any
discrepancies would be more easily found.

Gbp-Dch: ignore

- - - - -
a1282aec by Felix Lechner at 2019-04-15T10:33:09Z
Dont offer to calibrate tests unless their match strategy is 
tags.

The interactive test calibration, which adds or removes tags after
prompting the user, only works with the tags matching strategy. The
mechanisms is now disabled for other matching stategies.

Gbp-Dch: ignore

- - - - -
02d56d16 by Felix Lechner at 2019-04-15T10:39:12Z
Implement literal match strategy in test runner; adjust for universal tag 
format; drop sorting.

This provides for the new mechanisms in the test runner. The tag
processing is now done inside the runner, which saves three calls to
Perl scripts: two calls to tagextract and one call to tagdiff.

This step is thought to save about seven minutes in an average
Gitlab run (but the evidence is still unclear).

Gbp-Dch: ignore

- - - - -
68b2d7f3 by Felix Lechner at 2019-04-15T10:39:17Z
Move expected output to output for tests using a literal match 
strategy.

The literal matching strategy uses files named output to store the
expected output from Lintian. This moves the old files, which were
called tags, into the new locations.

The expected tags were previously all in the EWI format, so this will
work for any tests that uses that format going forward. Tests that use
other output formats will have their output files converted.

Gbp-Dch: ignore

- - - - -
6eed75c5 by Felix Lechner at 2019-04-15T10:39:17Z
Adjust expected output for literal matching strategy for lack of sorting and 
revert to original output format.

Some tests using the literal matching strategy employed the Sort
setting, which was removed, or used uncommon output formats. For those
tests, the expected output is converted to match, literally, the
expected output from Lintian.

Gbp-Dch: ignore

- - - - -
2e744ed0 by Felix Lechner at 2019-04-15T10:39:17Z
Convert all expected tags to universal format.

This converts the expected tags for all tests under the tags
matching strategy to the universal format

[Git][lintian/lintian] Deleted branch remove-obsolete-static-lab-code

[Niels Thykier]

Niels Thykier deleted branch remove-obsolete-static-lab-code at lintian / 
lintian

-- 

You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian][master] 3 commits: Remove coll completion status + auto-removal; it was only relevant for static labs

[Niels Thykier]

Niels Thykier pushed to branch master at lintian / lintian


Commits:
1329683c by Niels Thykier at 2019-04-12T19:12:22Z
Remove coll completion status + auto-removal; it was only relevant for static 
labs

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
a759a8e5 by Niels Thykier at 2019-04-13T12:39:31Z
Remove code that was only relevant for static-labs

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
7f0d7227 by Niels Thykier at 2019-04-13T12:40:42Z
.gitlab-ci: Run lintian --version before the tests

If lintian --version fails, then the test runner will bail as well
(but will not emit the error messages).  Also, if lintian --version
fails, we might as well stop very early.

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -


30 changed files:

- .gitlab-ci.yml
- collection/ar-info.desc
- collection/bin-pkg-control.desc
- collection/changelog-file.desc
- collection/copyright-file.desc
- collection/diffstat.desc
- collection/file-info.desc
- collection/java-info.desc
- collection/md5sums.desc
- collection/objdump-info.desc
- collection/override-file.desc
- collection/scripts.desc
- collection/src-orig-index.desc
- collection/strings.desc
- collection/unpacked.desc
- commands/lintian.pm
- commands/reporting-harness.pm
- lib/Lintian/CollScript.pm
- lib/Lintian/Lab.pm
- lib/Lintian/Lab/Entry.pm
- − lib/Lintian/Lab/Manifest.pm
- − lib/Lintian/Lab/ManifestDiff.pm
- lib/Lintian/Unpacker.pm
- − t/scripts/Lintian/Lab/Manifest/01-basic.t
- − t/scripts/Lintian/Lab/Manifest/02-io.t
- − t/scripts/Lintian/Lab/Manifest/03-diff.t
- − t/scripts/Lintian/Lab/Manifest/data/changes1-info
- − t/scripts/Lintian/Lab/Manifest/data/new-list-info
- − t/scripts/Lintian/Lab/Manifest/data/orig-list-info
- − t/scripts/Lintian/Lab/auto-repair.t


The diff was not included because it is too large.


View it on GitLab: 
https://salsa.debian.org/lintian/lintian/compare/844b2530f4d123567bc09e39ae745b20cc20cb68...7f0d72278b1e0a6d08039f55632d3e1a54a54bf9

-- 
View it on GitLab: 
https://salsa.debian.org/lintian/lintian/compare/844b2530f4d123567bc09e39ae745b20cc20cb68...7f0d72278b1e0a6d08039f55632d3e1a54a54bf9
You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian][remove-obsolete-static-lab-code] 12 commits: Fix previously undetected problem in template deb-make-builder.

[Niels Thykier]

Niels Thykier pushed to branch remove-obsolete-static-lab-code at lintian / 
lintian


Commits:
69d60295 by Felix Lechner at 2019-04-12T17:29:54Z
Fix previously undetected problem in template deb-make-builder.

The call to gzip fails when the file is a link. That feature is used
in some tests. Ignores the status of the entire conditional.

Gbp-Dch: ignore

- - - - -
d158cf8b by Felix Lechner at 2019-04-12T17:31:19Z
Fix some previously undetected problems in test changes-file-bad-section.

The filename log conflicts with the default location for the test
logs. That behavior may change in future test runners. Changes the
file thats part of the test case to another name.

Gbp-Dch: ignore

- - - - -
7b06eb39 by Felix Lechner at 2019-04-12T17:32:40Z
Fix some previously undetected problems in test 
changes-file-size-checksum-mismatch.

The filename log conflicts with the default location for the test
logs. That behavior may change in future test runners. Changes the
file thats part of the test case to another name.

Gbp-Dch: ignore

- - - - -
e9ff5021 by Felix Lechner at 2019-04-12T17:32:55Z
Fix previously undetected problem in test control-field-traversal-4.

The package builder is apparently unable to copy the changelog to the
docdir when the docdir is a link whose destination does not exist. Not
sure why this did not cause problems before. Creates the destination.

Gbp-Dch: ignore

- - - - -
18a94a88 by Felix Lechner at 2019-04-12T17:34:19Z
Fix some previously undetected problems in test files-mtime.

The file was already zipped somewhere else. Remove the destination to
make sure the gzip command succeeds.

Gbp-Dch: ignore

- - - - -
8582254b by Felix Lechner at 2019-04-12T17:35:16Z
Add missing import to lib/Test/Lintian/Helper.pm.

The package uses Lintian::Profile. Imports that module.

Gbp-Dch: ignore

- - - - -
c767887f by Felix Lechner at 2019-04-12T18:33:46Z
Separate builder for test packages.

The command build-test-packages is now used to build packages
separately from runtests. That cuts the development cycle
approximately in half. Does not affect GitLab pipelines, which run
both commands consecutively.

>From now on, please remember to rebuild the test packages manually if
you change any of the test specifications while contributing to
Lintian checks.

Gbp-Dch: ignore

- - - - -
d367faf7 by Felix Lechner at 2019-04-12T18:33:46Z
Only run tests in t/bin/runtests but do not build any packages.

The command build-test-packages is now used to build packages
separately from runtests. That cuts the development cycle
approximately in half. Does not affect GitLab pipelines, which run
both commands consecutively.

>From now on, please remember to rebuild the test packages manually if
you change any of the test specifications while contributing to
Lintian checks.

- - - - -
546c0d36 by Felix Lechner at 2019-04-12T18:46:53Z
Update rules and autopkgtest for separate package building.

Runs build-test-packages before runtests to make sure the 
test
packages are available.

Gbp-Dch: ignore

- - - - -
844b2530 by Niels Thykier at 2019-04-12T19:01:06Z
.gitlab-ci.yml: Build the packages before running tests

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
1329683c by Niels Thykier at 2019-04-12T19:12:22Z
Remove coll completion status + auto-removal; it was only relevant for static 
labs

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
f39f024b by Niels Thykier at 2019-04-12T19:24:01Z
Remove code that was only relevant for static-labs

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -


30 changed files:

- .gitlab-ci.yml
- collection/ar-info.desc
- collection/bin-pkg-control.desc
- collection/changelog-file.desc
- collection/copyright-file.desc
- collection/diffstat.desc
- collection/file-info.desc
- collection/java-info.desc
- collection/md5sums.desc
- collection/objdump-info.desc
- collection/override-file.desc
- collection/scripts.desc
- collection/src-orig-index.desc
- collection/strings.desc
- collection/unpacked.desc
- commands/lintian.pm
- commands/reporting-harness.pm
- debian/rules
- debian/tests/testsuite
- lib/Lintian/CollScript.pm
- lib/Lintian/Lab.pm
- lib/Lintian/Lab/Entry.pm
- − lib/Lintian/Lab/Manifest.pm
- − lib/Lintian/Lab/ManifestDiff.pm
- lib/Lintian/Unpacker.pm
- lib/Test/Lintian/Helper.pm
- lib/Test/Lintian/Run.pm
- + t/bin/build-test-packages
- t/bin/runtests
- − t/scripts/Lintian/Lab/Manifest/01-basic.t


The diff was not included because it is too large.


View it on GitLab: 
https://salsa.debian.org/lintian/lintian/compare/a6204254fdc75c621f22d40e1258601e8b3cf641...f39f024bac90375c2ddfd9c82454aed7908e712b

-- 
View it on GitLab: 
https://salsa.debian.org/lintian/lintian/compare/a6204254fdc75c621f22d40e1258601e8b3cf641...f39f024bac90375c2ddfd9c82454aed7908e712b
You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian] Pushed new branch ci-run-tests-again

[Niels Thykier]

Niels Thykier pushed new branch ci-run-tests-again at lintian / lintian

-- 
View it on GitLab: 
https://salsa.debian.org/lintian/lintian/tree/ci-run-tests-again
You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian] Pushed new branch remove-obsolete-static-lab-code

[Niels Thykier]

Niels Thykier pushed new branch remove-obsolete-static-lab-code at lintian / 
lintian

-- 
View it on GitLab: 
https://salsa.debian.org/lintian/lintian/tree/remove-obsolete-static-lab-code
You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian] Deleted branch slightly-faster-tests

[Niels Thykier]

Niels Thykier deleted branch slightly-faster-tests at lintian / lintian

-- 

You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian][master] 11 commits: tests: Optimize a few find calls to avoid spawning a process per match

[Niels Thykier]

Niels Thykier pushed to branch master at lintian / lintian


Commits:
193f9e3e by Niels Thykier at 2019-04-12T10:33:43Z
tests: Optimize a few find calls to avoid spawning a process per match

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
5ecfe473 by Niels Thykier at 2019-04-12T10:33:44Z
tests: Remove obsolete test helper tool

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
7d93cf2e by Niels Thykier at 2019-04-12T10:33:45Z
Cache all of dpkg-architectures values in the test runner

This may enable the dpkg tooling to skip a few computations.

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
32b72978 by Niels Thykier at 2019-04-12T11:13:17Z
Test::Lintian::Helper: Remove redundant sub

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
1798d27e by Niels Thykier at 2019-04-12T12:03:15Z
tests: Skip some mkdir and cp calls when they are redundant

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
4673f713 by Niels Thykier at 2019-04-12T13:09:08Z
Extract reporting related utilities into a separate module

Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
af1a2d51 by Niels Thykier at 2019-04-12T13:09:08Z
Remove unused system_env function from Lintian::Util

Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
ae5bc52d by Niels Thykier at 2019-04-12T13:37:36Z
Move Deb822 parsing into its own module called Lintian::Deb822Parser

Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
487e3c64 by Felix Lechner at 2019-04-12T13:37:37Z
Drop safe_qx from test infrastructure.

Drops safe_qx in order to further isolate the test infrastructure from
Lintian. This makes the tests more robust and independent of the test
subject.

When complete, the separation will prevent the needless rebuilding of
test package when only the Lintian executable has changed. It will
eventually make the test suite faster.

Gbp-Dch: ignore

- - - - -
81b07d50 by Niels Thykier at 2019-04-12T13:46:07Z
Lazy load L::Command in L::Util

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
597b9303 by Niels Thykier at 2019-04-12T13:54:17Z
Test::Lintian::Run: Remove import of unused module

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -


30 changed files:

- checks/control-file.pm
- checks/copyright-file.pm
- checks/debconf.pm
- checks/source-copyright.pm
- checks/testsuite.pm
- commands/reporting-harness.pm
- commands/reporting-html-reports.pm
- commands/reporting-lintian-harness.pm
- commands/reporting-sync-state.pm
- lib/Lintian/CheckScript.pm
- lib/Lintian/CollScript.pm
- lib/Lintian/Collect/Binary.pm
- lib/Lintian/Collect/Source.pm
- lib/Lintian/Command/Simple.pm
- + lib/Lintian/Deb822Parser.pm
- lib/Lintian/Lab/Entry.pm
- lib/Lintian/Profile.pm
- + lib/Lintian/Reporting/Util.pm
- lib/Lintian/Util.pm
- lib/Test/Lintian.pm
- lib/Test/Lintian/ConfigFile.pm
- lib/Test/Lintian/Helper.pm
- lib/Test/Lintian/Run.pm
- private/generate-lintian-pod
- private/generate-profiles.pl
- private/refresh-archs
- private/tag-stats
- private/update-coverage
- − t/bin/create-deb
- t/bin/runtests


The diff was not included because it is too large.


View it on GitLab: 
https://salsa.debian.org/lintian/lintian/compare/130a75cc8f98cbce35a18e6aad4be260e0ee88a1...597b9303db9957ef426e729f307aca6cacfb3fc0

-- 
View it on GitLab: 
https://salsa.debian.org/lintian/lintian/compare/130a75cc8f98cbce35a18e6aad4be260e0ee88a1...597b9303db9957ef426e729f307aca6cacfb3fc0
You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian][slightly-faster-tests] 3 commits: Move Deb822 parsing into its own module called Lintian::Deb822Parser

[Niels Thykier]

Niels Thykier pushed to branch slightly-faster-tests at lintian / lintian


Commits:
ae5bc52d by Niels Thykier at 2019-04-12T13:37:36Z
Move Deb822 parsing into its own module called Lintian::Deb822Parser

Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
487e3c64 by Felix Lechner at 2019-04-12T13:37:37Z
Drop safe_qx from test infrastructure.

Drops safe_qx in order to further isolate the test infrastructure from
Lintian. This makes the tests more robust and independent of the test
subject.

When complete, the separation will prevent the needless rebuilding of
test package when only the Lintian executable has changed. It will
eventually make the test suite faster.

Gbp-Dch: ignore

- - - - -
e207d575 by Niels Thykier at 2019-04-12T13:37:38Z
Lazy load L::Command in L::Util

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -


30 changed files:

- checks/control-file.pm
- checks/copyright-file.pm
- checks/debconf.pm
- checks/source-copyright.pm
- checks/testsuite.pm
- commands/reporting-harness.pm
- commands/reporting-html-reports.pm
- commands/reporting-sync-state.pm
- lib/Lintian/CheckScript.pm
- lib/Lintian/CollScript.pm
- lib/Lintian/Collect/Binary.pm
- lib/Lintian/Collect/Source.pm
- + lib/Lintian/Deb822Parser.pm
- lib/Lintian/Lab/Entry.pm
- lib/Lintian/Profile.pm
- lib/Lintian/Util.pm
- lib/Test/Lintian.pm
- lib/Test/Lintian/ConfigFile.pm
- lib/Test/Lintian/Helper.pm
- lib/Test/Lintian/Run.pm
- private/generate-lintian-pod
- private/generate-profiles.pl
- private/refresh-archs
- private/tag-stats
- private/update-coverage
- t/bin/runtests
- t/scripts/Lintian/Util/dctrl-parser.t
- t/scripts/needs-info-exists.t
- t/scripts/needs-info-missing.t
- t/scripts/profiles-coverage.t


The diff was not included because it is too large.


View it on GitLab: 
https://salsa.debian.org/lintian/lintian/compare/a6670ef14aeb449fd180863e9ac2cf902f21f2f4...e207d5755b6932dbae9a12da67bb76793b93a691

-- 
View it on GitLab: 
https://salsa.debian.org/lintian/lintian/compare/a6670ef14aeb449fd180863e9ac2cf902f21f2f4...e207d5755b6932dbae9a12da67bb76793b93a691
You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian][slightly-faster-tests] 7 commits: Test::Lintian::Helper: Remove redundant sub

[Niels Thykier]

Niels Thykier pushed to branch slightly-faster-tests at lintian / lintian


Commits:
32b72978 by Niels Thykier at 2019-04-12T11:13:17Z
Test::Lintian::Helper: Remove redundant sub

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
1798d27e by Niels Thykier at 2019-04-12T12:03:15Z
tests: Skip some mkdir and cp calls when they are redundant

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
4673f713 by Niels Thykier at 2019-04-12T13:09:08Z
Extract reporting related utilities into a separate module

Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
af1a2d51 by Niels Thykier at 2019-04-12T13:09:08Z
Remove unused system_env function from Lintian::Util

Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
1efc9eff by Niels Thykier at 2019-04-12T13:09:10Z
Move Deb822 parsing into its own module called Lintian::Deb822Parser

Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -
a265 by Felix Lechner at 2019-04-12T13:15:41Z
Drop safe_qx from test infrastructure.

Drops safe_qx in order to further isolate the test infrastructure from
Lintian. This makes the tests more robust and independent of the test
subject.

When complete, the separation will prevent the needless rebuilding of
test package when only the Lintian executable has changed. It will
eventually make the test suite faster.

Gbp-Dch: ignore

- - - - -
a6670ef1 by Niels Thykier at 2019-04-12T13:24:11Z
Lazy load L::Command in L::Util

Gbp-Dch: ignore
Signed-off-by: Niels Thykier ni...@thykier.net

- - - - -


30 changed files:

- checks/control-file.pm
- checks/copyright-file.pm
- checks/debconf.pm
- checks/source-copyright.pm
- checks/testsuite.pm
- commands/reporting-harness.pm
- commands/reporting-html-reports.pm
- commands/reporting-lintian-harness.pm
- commands/reporting-sync-state.pm
- lib/Lintian/CheckScript.pm
- lib/Lintian/CollScript.pm
- lib/Lintian/Collect/Binary.pm
- lib/Lintian/Collect/Source.pm
- lib/Lintian/Command/Simple.pm
- + lib/Lintian/Deb822Parser.pm
- lib/Lintian/Lab/Entry.pm
- lib/Lintian/Profile.pm
- + lib/Lintian/Reporting/Util.pm
- lib/Lintian/Util.pm
- lib/Test/Lintian.pm
- lib/Test/Lintian/ConfigFile.pm
- lib/Test/Lintian/Helper.pm
- lib/Test/Lintian/Run.pm
- private/generate-lintian-pod
- private/generate-profiles.pl
- private/refresh-archs
- private/tag-stats
- private/update-coverage
- t/bin/runtests
- t/scripts/Lintian/Util/dctrl-parser.t


The diff was not included because it is too large.


View it on GitLab: 
https://salsa.debian.org/lintian/lintian/compare/7d93cf2e9c0bd1c66b8011bf659740df4aa0b53f...a6670ef14aeb449fd180863e9ac2cf902f21f2f4

-- 
View it on GitLab: 
https://salsa.debian.org/lintian/lintian/compare/7d93cf2e9c0bd1c66b8011bf659740df4aa0b53f...a6670ef14aeb449fd180863e9ac2cf902f21f2f4
You're receiving this email because of your account on salsa.debian.org.

[Git][lintian/lintian] Pushed new branch slightly-faster-tests

[Niels Thykier]

Niels Thykier pushed new branch slightly-faster-tests at lintian / lintian

-- 
View it on GitLab: 
https://salsa.debian.org/lintian/lintian/tree/slightly-faster-tests
You're receiving this email because of your account on salsa.debian.org.

Bug#926543: lintian: Deadlock in source-copyright check on source:khronos-opencl-man/1.0~svn33624-4

[Niels Thykier]

Niels Thykier:
> Niels Thykier:
>> Niels Thykier:
>>> Package: lintian
>>> Version: 2.9.1
>>> Severity: important
>>>
>>> Hi,
>>>
>>> Discovered in the archive-wide run on lindsay.d.o; lintian does not
>>> terminate when run on khronos-opencl-man/1.0~svn33624-4 (source).
>>>
>>> Thanks,
>>> ~Niels
>>>
>>
>> For reference, I used the following command line to confirm it on lindsay:
>>
>> lintian -EvIL +pedantic -j2 -ddd \
>>   /srv/mirrors/debian/pool/main/k/khronos-opencl-man/*33624*.{deb,dsc}
>>
>> I.e. I ran it with source and binaries available (didn't check of the
>> source alone was enough to trigger the issue..
>>
>> Thanks,
>> ~Niels
>>
> 
> The source package is enough to reproduce it but I cannot reproduce it
> on buster.  It has to be stretch or older; this implies that the
> underlying issue might have been fixed in a dependency (presumably
> IPC::Run).
> 
> Thanks,
> ~Niels
> 

lindsay.debian.org has been upgraded to buster but the problem still
persist.  I reproduced it today with 2.12.0 (git checkout).

Thanks,
~Niels

Bug#926543: lintian: Deadlock in source-copyright check on source:khronos-opencl-man/1.0~svn33624-4

[Niels Thykier]

Niels Thykier:
> Niels Thykier:
>> Package: lintian
>> Version: 2.9.1
>> Severity: important
>>
>> Hi,
>>
>> Discovered in the archive-wide run on lindsay.d.o; lintian does not
>> terminate when run on khronos-opencl-man/1.0~svn33624-4 (source).
>>
>> Thanks,
>> ~Niels
>>
> 
> For reference, I used the following command line to confirm it on lindsay:
> 
> lintian -EvIL +pedantic -j2 -ddd \
>   /srv/mirrors/debian/pool/main/k/khronos-opencl-man/*33624*.{deb,dsc}
> 
> I.e. I ran it with source and binaries available (didn't check of the
> source alone was enough to trigger the issue..
> 
> Thanks,
> ~Niels
> 

The source package is enough to reproduce it but I cannot reproduce it
on buster.  It has to be stretch or older; this implies that the
underlying issue might have been fixed in a dependency (presumably
IPC::Run).

Thanks,
~Niels

Bug#926543: lintian: Deadlock in source-copyright check on source:khronos-opencl-man/1.0~svn33624-4

[Niels Thykier]

Niels Thykier:
> Package: lintian
> Version: 2.9.1
> Severity: important
> 
> Hi,
> 
> Discovered in the archive-wide run on lindsay.d.o; lintian does not
> terminate when run on khronos-opencl-man/1.0~svn33624-4 (source).
> 
> Thanks,
> ~Niels
> 

For reference, I used the following command line to confirm it on lindsay:

lintian -EvIL +pedantic -j2 -ddd \
  /srv/mirrors/debian/pool/main/k/khronos-opencl-man/*33624*.{deb,dsc}

I.e. I ran it with source and binaries available (didn't check of the
source alone was enough to trigger the issue..

Thanks,
~Niels

Bug#926543: lintian: Deadlock in source-copyright check on source:khronos-opencl-man/1.0~svn33624-4

[Niels Thykier]

Package: lintian
Version: 2.9.1
Severity: important

Hi,

Discovered in the archive-wide run on lindsay.d.o; lintian does not
terminate when run on khronos-opencl-man/1.0~svn33624-4 (source).

Thanks,
~Niels

Bug#924715: lintian: Please rename the process based on what it is done (i.e. set $0)

[Niels Thykier]

Package: lintian
Version: 2.9.1
Severity: wishlist

Hi,

It is very useful when lintian changes its process name based on what
it is doing (in particularly when debugging things like #924714).
Though, only unpack jobs are currently the only parts that use this
feature.  Please expand it to more parts so it is easier to see what
is running amock.

Thanks,
~Niels

Bug#924714: lintian: Way too many threads running

[Niels Thykier]

Niels Thykier:
> Package: lintian
> Version: 2.9.1
> Severity: important
> 
> Observed on lindsay.d.o with -j 1:
> 
> 
> """
>│ 
> └─reporting-harne(11215)───reporting-linti(11461)───lintian(11482)─┬─Lab 
> entry binar(+
>│  
>   ├─Lab entry sourc(+
>│  
>   ├─lintian(9769)
>│  
>   ├─lintian(9770)
>│  
>   ├─lintian(9771)
>│  
>   ├─lintian(9772)
>│  
>   ├─lintian(9773)
>│  
>   ├─lintian(9774)
>│  
>   ├─lintian(9776)
>│  
>   ├─lintian(9777)
>│  
>   ├─lintian(9778)
>│  
>   ├─lintian(9780)
>│  
>   ├─lintian(9783)
>│  
>   ├─lintian(9965)
>│  
>   ├─lintian(9975)
>│  
>   ├─lintian(1)
>│  
>   ├─lintian(10003)
>│  
>   ├─lintian(10020)
>│  
>   ├─lintian(10030)
>│  
>   ├─lintian(10033)
>│  
>   ├─lintian(10037)
>│  
>   ├─lintian(10067)
>│  
>   ├─lintian(10118)
>│  
>   ├─lintian(10139)
>│  
>   ├─lintian(10146)
>│  
>   ├─lintian(10161)
>│  
>   ├─lintian(10194)
>│  
>   ├─lintian(10266)
>│  
>   ├─lintian(10337)
>│  
>   ├─lintian(10417)
>│  
>   ├─lintian(10461)
>│  
>   ├─lintian(10480)
>│  
>   ├─lintian(10481)
>│ 

Bug#924714: lintian: Way too many threads running

[Niels Thykier]

Package: lintian
Version: 2.9.1
Severity: important

Observed on lindsay.d.o with -j 1:


"""
   │ 
└─reporting-harne(11215)───reporting-linti(11461)───lintian(11482)─┬─Lab entry 
binar(+
   │
├─Lab entry sourc(+
   │
├─lintian(9769)
   │
├─lintian(9770)
   │
├─lintian(9771)
   │
├─lintian(9772)
   │
├─lintian(9773)
   │
├─lintian(9774)
   │
├─lintian(9776)
   │
├─lintian(9777)
   │
├─lintian(9778)
   │
├─lintian(9780)
   │
├─lintian(9783)
   │
├─lintian(9965)
   │
├─lintian(9975)
   │
├─lintian(1)
   │
├─lintian(10003)
   │
├─lintian(10020)
   │
├─lintian(10030)
   │
├─lintian(10033)
   │
├─lintian(10037)
   │
├─lintian(10067)
   │
├─lintian(10118)
   │
├─lintian(10139)
   │
├─lintian(10146)
   │
├─lintian(10161)
   │
├─lintian(10194)
   │
├─lintian(10266)
   │
├─lintian(10337)
   │
├─lintian(10417)
   │
├─lintian(10461)
   │
├─lintian(10480)
   │
├─lintian(10481)
   │
├─lintian(10495)
   │
└─lintian(10507)
"""

It appears to be processing firefox-esr (check phase).

Thanks,
~Niels

Bug#924449: lintian: Periodic "out of disk space" errors from lindsay.d.o

[Niels Thykier]

Package: lintian
Version: 2.10.0
Severity: important

We seem to be getting regular "out of disk space" errors on
lindsay.d.o after resuming the archive-wide processing.

I found the following in the [log file].  We see the following 3 issues
(AFAICT):
 * linux
 * python3-zaqar-ui
 * Closing of the perf log ("/srv/lintian.debian.org/logs/lintian-perf.log")

This might have been the underlying root cause for #890873, it may be
a regression with the rewrite of L::Unpacker, or maybe unpacking Linux
simply takes more disk space that we have.

AFAICT, lintian correctly cleans up old packages during the run.
I.e. we are not accumulating disk usage from previous packages in the
run.

For reference, we currently have 32GB disk available for lintian ("all
inclusive").  About 23GB of that is available for checking with our
current usage for reports and stats.

Thanks,
~Niels

[log file]:
"""
N: Starting on group linux/4.19.20-1
N: Unpacking packages in group linux/4.19.20-1

gzip: stdout: No space left on device

gzip: 
gzip: stdout: No space left on device
stdout: No space left on device
mkdir: cannot create directory 
'/srv/lintian.debian.org/scratch/temp-lintian-lab-HPiNntNduD/pool/l/linux/linux-image-4.19.0-3-rt-amd64-unsigned_4.19.20-1_amd64_binary
/strings/lib/modules/4.19.0-3-rt-amd64/kernel/drivers/misc': No space left on 
device

gzip: stdout: No space left on device

gzip: stdout: No space left on device
mkdir -p 
/srv/lintian.debian.org/scratch/temp-lintian-lab-HPiNntNduD/pool/l/linux/linux-image-4.19.0-3-rt-amd64-unsigned_4.19.20-1_amd64_binary/strings/lib/modules/4.19.0-3-rt-amd64/kernel/drivers/misc
 failed: 0

gzip: stdout: No space left on device
command failed with error code 1 at 
/srv/lintian.debian.org/lintian/lib/Lintian/Command.pm line 344.
Lintian::Command::reap(HASH(0x55ea04300b40)) called at 
/srv/lintian.debian.org/lintian/collection/objdump-info line 81

Lintian::coll::objdump_info::collect("linux-image-4.19.0-3-rt-686-pae-dbg", 
"binary", "/srv/lintian.debian.org/scratch/temp-lintian-lab-HPiNntNduD/p"...) 
called at /srv/lintian.debian.org/lintian/lib/Lintian/CollScript.pm line 242
Lintian::CollScript::collect(Lintian::CollScript=HASH(0x55ea043df688), 
"linux-image-4.19.0-3-rt-686-pae-dbg", "binary", 
"/srv/lintian.debian.org/scratch/temp-lintian-lab-HPiNntNduD/p"...) called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Unpacker.pm line 412
eval {...} called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Unpacker.pm line 412
Lintian::Unpacker::__ANON__() called at 
/usr/share/perl5/IO/Async/Loop.pm line 1943
eval {...} called at /usr/share/perl5/IO/Async/Loop.pm line 1943
IO::Async::Loop::fork(IO::Async::Loop::Poll=HASH(0x55ea0436bfa0), 
"code", CODE(0x55ea24cdc728), "on_exit", CODE(0x55ea313c8288)) called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Unpacker.pm line 461
eval {...} called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Unpacker.pm line 385

Lintian::Unpacker::__ANON__("objdump-info-binary:linux-image-4.19.0-3-rt-686-pae-dbg/4.19."...,
 Lintian::CollScript=HASH(0x55ea043df688), 
Lintian::Lab::Entry=HASH(0x55ea02c950f8), 
Lintian::DepMap::Properties=HASH(0x55ea303ced58)) called at 
/srv/lintian.debian.org/lintian/lib/Lintian/Unpacker.pm line 453
Lintian::Unpacker::__ANON__(2475, 0) called at 
/usr/share/perl5/IO/Async/Loop.pm line 2604
IO::Async::Loop::_reap_children(HASH(0x55ea03de6920)) called at 
/usr/share/perl5/IO/Async/Loop.pm line 2663
IO::Async::Loop::__ANON__() called at /usr/share/perl5/IO/Async/Loop.pm 
line 805
IO::Async::Loop::__ANON__() called at /usr/share/perl5/IO/Async/OS.pm 
line 577
IO::Async::OS::_Base::__ANON__(IO::Async::Handle=HASH(0x55ea05c557a8)) 
called at /usr/share/perl5/IO/Async/Loop/Poll.pm line 172

IO::Async::Loop::Poll::post_poll(IO::Async::Loop::Poll=HASH(0x55ea0436bfa0)) 
called at /usr/share/perl5/IO/Async/Loop/Poll.pm line 279

IO::Async::Loop::Poll::loop_once(IO::Async::Loop::Poll=HASH(0x55ea0436bfa0), 
undef) called at /usr/share/perl5/IO/Async/Loop.pm line 524
IO::Async::Loop::run(IO::Async::Loop::Poll=HN: Finished processing 
group zaqar-ui/5.0.0-2
warning: could not create the package entry in the lab: No space left on device
warning: skipping check of binary package python3-zaqar-ui
warning: closing +/srv/lintian.debian.org/logs/lintian-perf.log failed: Can't 
close(GLOB(0x55ea01364618)) filehandle: 'No space left on device' at 
/srv/lintian.debian.org/lintian/commands/lintian.pm line 1818
main::close(GLOB(0x55ea01364618)) called at 
/srv/lintian.debian.org/lintian/commands/lintian.pm line 1818
eval {...} called at 
/srv/lintian.debian.org/lintian/commands/lintian.pm line 1818
main::END() called at /srv/lintian.debian.org/lintian/frontend/lintian 
line 0
eval {...} called at /srv/lintian.debian.org/lintian/frontend/lintian 
line 0

"""

Bug#919162: lintian.d.o apears to report many failures twice

[Niels Thykier]

Chris Lamb:
> Ian Campbell wrote:
> 
>>> aapt 1:8.1.0+r23-3 (binary) ($maintainer)
>>> 
>>> * usr/lib/android-sdk/build-tools/debian/aapt [amd64, i386]
>>> * usr/lib/android-sdk/build-tools/debian/aapt2
>>
>> That's what I was thinking for the ideal case too.
> 
> (This is probably the same effort as the separate "[amd64]" and
> "[i386]" lines; its the duplicate detection and updating the data
> structures that will be problematic, if anything.)
> 
> Niels, any tips/pointers on how I get a very very simple reporting
> framework running locally? Perhaps even just one package? I've been
> previously just making my changes here blind...
> 
> 
> Regards,
> 

In this case, you only need the report-generator to work.  This
simplifies some things as it can be called directly if you do some setup
manually.

 * Create a config.yml with the relevant bits (copy and adapt the one
   from lindsay if needed).  I believe you only need to consider:
 storage.state-cache (writable directory)
 storage.reports-work-dir (creatable non-existent directory)
 storage.historical-data-dir (optional; only needed for graphs)
 template-variables.* (for variables used in the templates)

 * (optionally) create an harness state cache file (copy and adapt the
   one from lindsay if in doubt).  As I recall, it is redundant in this
   case and its absence will just mean the results is tied to the
   report for the maintainer "unknown".

That was the setup.  Then the testing works like this:

 * Generate a lintian log file that should be parsed/generate the
   report[1].

 * Delete/rename the directory denoted by storage.reports-work-dir if it
   exists (e.g. from a previous test)

 * Run the reporting html command with:
   frontend/dplint reporting-html-reports \
 --reporting-config 
 

The above is from memory and may need some tweaking to work in practise.

Thanks,
~Niels


[1] Minimum being:
   lintian \
 --verbose \
 --exp-output=format=fullewi \
 

Bug#919162: lintian.d.o apears to report many failures twice

[Niels Thykier]

Chris Lamb:
> [Adding ni...@thykier.net to CC]
> 
> Hi Ian,
> 
>> It seems that lintian.d.o lists some (all?) failures in a binary
>> package twice.
> […]
>>  * usr/bin/grub-editenv
>>  * usr/bin/grub-editenv
> […]
> 
>> I failed to find the raw reports anywhere so I can't see if these are
>> duplicated there
> 
> So I dug them out from 
> /srv/lintian.debian.org/reports-directory/www/lintian.log.gz
> on lindsay.debian.org:
> 
> [...]
> 
> … so I think that this is something depper in the reporting system
> itself (and they are not emitted twice locally).
> 
> Unfortunately this is an area that I don't know very well at all so
> I'm not even sure where to start. Niels (CC'd), can you help point
> me at least in the right direction, perhaps?
> 
> 
> Regards,
> 

What you see is that grub-editenv have the tag twice because the tag
appears on different architectures (i386 vs. amd64).  The reporting
framework was never updated to include this information more smoothly
than simply adding the tag twice.

Thanks,
~Niels

[Git][lintian/lintian] Pushed new branch unpacker-io-async

[Niels Thykier]

Niels Thykier pushed new branch unpacker-io-async at lintian / lintian

-- 
View it on GitLab: 
https://salsa.debian.org/lintian/lintian/tree/unpacker-io-async
You're receiving this email because of your account on salsa.debian.org.

Bug#914873: lintian: Reduce the number of overrides for binary-or-shlib-defines-rpath

[Niels Thykier]

Package: lintian
Version: 2.5.113
Severity: wishlist

Hi,

I noticed on
https://lintian.debian.org/tags/binary-or-shlib-defines-rpath.html
that this tag has over 25000 overrides.  This implies that either the
certainity of this tag has dropped to a "wild-guess", someone is
absuing overrides or the tag has a bug that no one reported so far.

At a quick glance, it would appear that many of these overrides are
from haskell packages.  Example:

"""
libghc-active-dev 0.2.0.13-6 (binary) (Debian Haskell Group 
) overridden

  * O 
usr/lib/haskell-packages/ghc/lib/x86_64-linux-ghc-8.4.3/libHSactive-0.2.0.13-Cwvr7dc67LjAJlq5Xu7krQ-ghc8.4.3.so
 /usr/lib/ghc/text-1.2.3.0
  * O 
usr/lib/haskell-packages/ghc/lib/x86_64-linux-ghc-8.4.3/libHSactive-0.2.0.13-Cwvr7dc67LjAJlq5Xu7krQ-ghc8.4.3.so
 /usr/lib/ghc/template-haskell-2.13.0.0
  * O 
usr/lib/haskell-packages/ghc/lib/x86_64-linux-ghc-8.4.3/libHSactive-0.2.0.13-Cwvr7dc67LjAJlq5Xu7krQ-ghc8.4.3.so
 /usr/lib/ghc/array-0.5.2.0
"""

(Note this example may be trimmed in number of occurances)

It smells like the haskell team has invented some scheme for RPATH
that they now rely on, which lintian should simply accept as a "valid"
use of RPATH.

Thanks,
~Niels

Bug#908915: Several tests fail (not skipped) without certain packages installed

[Niels Thykier]

Josh Triplett:
> Package: lintian
> Version: 2.5.103
> Severity: normal
> 
> When trying to run the lintian testsuite, I got several test failures:
> 
> Failed tests (6)
> tests::binaries-missing-depends-on-numpy-abi
> tests::debhelper-dh-quilt-addon-but-quilt-source-format
> tests::debhelper-dh-quilt-addon-but-quilt-source-format-unrel
> tests::debhelper-dh-with-quilt
> tests::rules-including-deprecated-makefiles
> tests::rules-missing-targets-with-known-includes
> 
> Looking at the log, these seem to fail without certain packages
> installed. I'm not sure if the bug is that those tests lack
> Test-Depends, or in some cases if the tests shouldn't be trying to
> build.
> 
> For example, rules-missing-targets-with-known-includes fails due to
> missing /usr/share/javahelper/java-vars.mk:
> 
> [...]
> 
> And rules-including-deprecated-makefiles fails due to missing
> /usr/share/cdbs/1/rules/simple-patchsys.mk:
> 
> [...]
> 
> Should these tests use Test-Depends, or for some of these tests should
> lintian just build a source package and check that without trying to
> build binary packages?
> 
> [...]
Hi,

Both cdbs and javahelper are in the Build-Depends of lintian (marked
with ).  Just to be sure, have you installed *all* the
build-depends of lintian (including those marked with "")?

Note: Test-Depends is used for skipping tests instead of adding a
Build-Depends to lintian itself (e.g. to aid backporting).  It should
not duplicate the lintian Build-Depends field.

Thanks,
~Niels

Bug#907727: source-contains-empty-directory when a patch adds a file to that directory

[Niels Thykier]

Chris Lamb:
> Hi Russ,
> 
>> P: xfonts-jmk source: source-contains-empty-directory neep/ascii/
>>
>> but a patch in the quilt series adds a file to that directory. 
> 
> So, we currently loop over:
> 
> foreach my $file ($info->sorted_orig_index) {
> 
> … which with "3.0 (quilt)" has the patches applied. I'm not sure how we
> can get the unpatched output in this case. Any ideas?
> 
> (Parsing the diffs to catch this false-positive might be a little too
> much for this corner-case.)
> 
> 
> Regards,
> 

Hi Chris,

Have you checked if the src-orig-index collection
($info->{sorted_,}orig_index) has the data you want?

As I recall, it is based solely on the tarballs (i.e. no patches applied).

Thanks,
~Niels

Bug#907845: Warn if compat level 11 is used and dh_system_*-arch helpers are used too

[Niels Thykier]

Chris Lamb:
> tags 907845 + pending
> retitle 907845 lintian: Warn if compat level 11 is used and dh_system_*-arch 
> helpers are used too
> thanks
> 
>> I wonder why it didn't trigger for sysstat
>> https://salsa.debian.org/debian/sysstat/blob/master/debian/rules#L68
>>
>> Is the -arch prefix confusing lintian?
> 
> Indeed. Now fixed in Git, pending upload:
> 
>   
> https://salsa.debian.org/lintian/lintian/commit/c4923ba1353f6238955bef6f6c0a4c2175f010cf
> 
>   checks/debhelper.pm| 2 +-
>   debian/changelog   | 3 +++
>   t/tests/rules-uses-deprecated-systemd-override/debian/debian/rules | 3 +++
>   t/tests/rules-uses-deprecated-systemd-override/tags| 1 +
>   4 files changed, 8 insertions(+), 1 deletion(-)
> 
> 
> Regards,
> 

There is also an -indep variant of override targets.  :)

Thanks,
~Niels

Bug#907667: lintian: should html escape output if --color=html is used

[Niels Thykier]

Chris Lamb:
> Hi Niels,
> 
>> Any reason for introducing the CGI dependency over simply applying the
>> same escape rules for the $information variable?
> 
> Only because well-used libraries are preferred, particularly for data
> sanitisation (!) operations.
> 
> Is the extra dependency problematic? We use some far-more esoteric
> libraries than CGI, so I did not think it would be an issue.
> 
> 
> Regards,
> 

If we are consistent with how we perform the quoting, I do not mind the
extra dependency.  Particularly because it should be doable to reduce it
to a suggests given --color=html is not a default (which we can add
later if relevant).

Though, reminder - if you introduce a new dependency, you will have to
get DSA to install it on lindsay.d.o before you can upgrade lintian there.

Thanks,
~Niels

Bug#907667: lintian: should html escape output if --color=html is used

[Niels Thykier]

Chris Lamb:
> tags 907667 + pending
> thanks
> 
> Fixed in Git, pending upload:
> 
>   
> https://salsa.debian.org/lintian/lintian/commit/897c485d61387adc5689f287c7e0404e604136e7
> 
>   debian/changelog  | 5 +
>   debian/control| 2 ++
>   lib/Lintian/Output.pm | 7 +++
>   t/tests/lintian-color-html/debian/debian/docs | 1 +
>   t/tests/lintian-color-html/debian/foo.xml | 1 +
>   t/tests/lintian-color-html/desc   | 8 
>   t/tests/lintian-color-html/tags   | 1 +
>   7 files changed, 21 insertions(+), 4 deletions(-)
> 
> 
> Regards,
> 


Any reason for introducing the CGI dependency over simply applying the
same escape rules for the $information variable?  Possibly we could
extract the html_quote from commands/reporting-html-reports.pm and put
it in L::Util (or similar) and share the code from there.

Alternatively, if we are moving to a dependency to solve this issue,
then we should use it consistently (i.e. remove html_quote from
commands/reporting-html-reports.pm).

Thanks,
~Niels

Bug#906209: lintian: bugs-field-does-not-refer-to-debian-infrastructure: cannot override

[Niels Thykier]

Chris Lamb:
> tags 906209 + moreinfo
> thanks
> 
> Thorsten,
> 
>> $ tail -2 debian/simkolab-blackberry.lintian-overrides
> 
> Should this not be debian/source.lintian-overrides or debian/source/
> lintian-overrides? See the "FILES" section in lintian(1) for more info.
> 
> 
> Regards,
> 



I suspect the file name is correct (I would have expected a "source"
between the code and the tag if it was a source tag).  But regardless,
the key issue is that the override file does not have the correct
content (and lintian is not very helpful in this case).

The lintian extra is: "mailto:t.gla...@tarent.de (line 23)"
The extra for the override is: "mailto:t.gla...@tarent.de;
Since:

  "mailto:t.gla...@tarent.de (line 23)" != "mailto:t.gla...@tarent.de;

the override is not applicable for that tag (or any tag emitted at all),
which makes it an unused override.

This is not the first time this issue has appeared and I doubt it will
be the last unless lintian gets some features to be more helpful.

Thanks,
~Niels

Lintian test suite proposal

[Niels Thykier]

Hi Felix,

Context for lint-maint: Felix Lechner sent me a proposed change for the
test suite and asked me for write an email with the feedback.  I thought
it was useful to include lint-maint because it affects all contributors
of lintian (if implemented).  The proposal is quoted below interleaved
with my comments and is to be an update/edit to t/tests/README.


Thanks for working on improving Lintian.  I should once again repeat
that my work on lintian has dwindled and I may not be the right
contributor any longer to decide which way we are going with the test
suite (IOW, take my review with a grain of salt).


As I read it, there are two orthogonal but interesting ideas.

 1. The tags.d layout

 2. The Abbreviated-Tags proposal pigging backed on top of tags.d.

As I read it, these proposals should be feasible implement separately
(i.e. Abbreviated-Tags could be retrofitted into the existing framework
without adding tags.d).  This is mostly relevant if one of the proposals
turn out to be controversial and the other is not - in that case, we can
move forward with the uncontroversial one.


Personally, I feel I understand Abbreviated-Tags good enough that I can
say I would support it and I would recommend it was made the default for
new tests (modulo a few minor remarks).
  On the tags.d layout, I can mostly see where you would like to go but
it is not clear to me when the test runner will accept vs. fail a test.


> New tags.d layout
> -

General remarks about the text (assuming it is for t/tests/README):

 * I think we should avoid the "New" / "Old" in this document if it is
   an addition to t/tests/README.  The problem with using "new" for
   everything is that in a year it will no longer be new and at some
   point it could even be superseded by a different format/layout as we
   improve ourselves.

 * The text sometimes stops being a README document and starts being
   sales-pitch and then jumps back to being a README a bit later only
   to become a TODO list in the end.  I have highlighted a few of these
   but I stopped when I realised it was a general problem.
   - Note there is nothing wrong with the sales-pitch/rationale for the
 proposals.  The "issue" is that I was told to read this as a
 t/tests/README change and some of the text need tweaking to fit
 there.

> 
> In addition to the established format described in the remainder of
> this document, our test suite now supports a new directory-based
> format for expected tags.
> 
> The new format no longer compares the entire lintian output. Instead,
> it uses separate files for each lintian check. The files are located
> in a directory called tags.d.
>> Each file is named after a lintian check. It contains all tags issued
> by a particular check. The tags can be in any order. Tags from checks
> are ignored and will no longer interfere.

The last sentence is a bit unclear to me.  I suspect you mean that tags
are ignored if they do not belong to one of the checks in tags.d?

Assuming there is tags.d/X file and it contains the tag "Y".  Will the
test fail if the check X emits the tag "Z" in addition to "Y"?

> This decoupling has several
> advantages. For example, one can now enable '--pedantic' without
> having to deal with any noise from other checks. Instead, one can
> conveniently test just the tags expected from a particular check.
> 
> The tags.d directory also contains a file named 'options' which gives
> the user a way to restore the old behavior. 

How come you are going for a new "options" file for these options
instead of adding them to the per-test desc file?

> When set, 'Exhaustive-Set:
> yes" will cause any extra tags in the output to fail the test. Since
> checks generally have little correlation, this option will not be
> useful in many cases. For most tests, the default setting of
> 'Exhaustive-Set: no' will be a great relief.

I could do with some examples of when the test succeeds and fails given
a given tags.d folder/content, different Exhaustive-Set values and
different lintian output.

> The lintian option
> '--pedantic' might even become the default setting for the test suite
> going forward.
> 

The last sentence (about making --pedantic default in tests) makes sense
in the proposal as it is part of the reason why this proposal might be
useful.  However, I do not think it would belong in t/tests/README.

> For more flexibility, there is another option 'Abbreviated-Tags: yes'.
> It allows expected tags to be listed in a shorter and possibly
> more convenient format. First, the letter code at the beginning of
> each line is dropped and reconstructed automatically. Tags are only
> ever issued with one of these letters, so the chance of an internal
> error is small. Instead of worrying about a tag's severity and
> certainty, you can focus on which tags are being issued and why.
> 

Note that AFAICT, the Abbreviated-Tags could be added to the existing
framework without the new layout (after resolving 

Bug#905747: lintian: Better checking of DEP-5 copyright Files field

[Niels Thykier]

Package: lintian
Version: 2.5.94
Severity: minor

I saw the following in a d/copyright today and expected lintian to
give some form of warning:

"""
Files: *
   debian/patches/*
Copyright: [...]
License:   License1

Files: debian/*
Copyright: [...]
License:   License2
"""

There are two distrinct issues here:

 1) The debian/patches/* entry in the first paragraph is completely
overwritten by the debian/*.  There is no warning about this
even though it is a mistake (redundant at best; most likely
an incorrect license is recorded for the overshadowed entry).


 2) Having "*" together with anything else is redundant at best and
almost certainly a mistake.


In the concrete case, what the person wanted was:

"""
Files: *
   debian/patches/*
Copyright: [...]
License:   License1

Files: debian/*
Copyright: [...]
License:   License2

Files: debian/patches/*
Copyright: [...]
License:   License1
"""

I.e. debian/* is under License2 except for debian/patches which has
the same license as all of upstreams code.

Thanks,
~Niels

Bug#904886: lintian: Support "debhelper-compat (= X)" B-D as replacement for "debhelper (>= X~)"

[Niels Thykier]

Chris Lamb:
> Dear Niels,
> 
>> debhelper-and-debhelper-compat-virtual-relation-with-unsupported-version
>> might be better worded as
>> "debhelper-build-dependency-implied-by-debhelper-compat-relation" and E
>> might too high for this.  There is not much harm in having the redundant
>> build-dependency.
>>
>> On a related note, the text for that tag saying  "However, this requires
>> a debhelper 11.3.6 or later." is incorrect.  Having debhelper-compat (=
>> X) implies debhelper (>= 11.3.6~) - i.e. you can rely on any feature in
>> debhelper up and including version 11.3.6~ without an explicit versioned
>> relation on debhelper (but I am fine with lintian not keeping track of
>> that).
> 
> Hm, perhaps I misunderstood what you wrote a few mails back - could you
> provide an example "good" and example "bad" case of a Build-Depends
> that uses both just to clarify? I think I've managed to confuse myself.
> 
> 
> Best wishes,
> 


# Good:

  Build-Depends: debhelper-compat (= 11)

  Build-Depends: debhelper-compat (= 11),
 debhelper (>= 11.2~)

The latter of the two also holds for any:

  debhelper-compat (= X), debhelper (>= Y)

Where X << Y.


# Bad:

  Build-Depends: debhelper-compat (= 11),
 debhelper (>= 10~)

The debhelper-compat would imply debhelper (>= 10) already.  This also
holds for any:

  debhelper-compat (= X), debhelper (>= Y)

Where X >= Y.

This is "bad" because the debhelper-compat relation makes the debhelper
relation redundant.  IOW you could delete the debhelper relation without
any loss in functionality/support/features from debhelper.

## The special case

Strictly speaking this also holds for any where X >= 9 and Y <= 11.3.6~
as the first version of debhelper to introduce debhelper-compat was
11.3.6~ (and accordingly, if apt can satisfy the debhelper-compat
relation, the debhelper relation is satisfied along with it).

However, I am not entirely convinced it makes sense for lintian to keep
track of this special-case.  Over time, the value of handling this
special case will decline.


I hope this clarifies the situation.

Thanks,
~Niels

Bug#904886: lintian: Support "debhelper-compat (= X)" B-D as replacement for "debhelper (>= X~)"

[Niels Thykier]

Chris Lamb:
> tags 904886 + pending
> thanks
> 
> Fixed in Git, pending upload:
> 
>   
> https://salsa.debian.org/lintian/lintian/commit/f61be381930e8087b2d00cf85687b51eecc7d5ec
> 
>   checks/debhelper.desc  | 34 
> --
>   checks/debhelper.pm| 31 
>   debian/changelog   |  3 ++
>   .../debian/debian/control.in   | 16 ++
>   .../desc   |  7 +
>   .../pre_build  |  5 
>   .../tags   |  1 +
>   .../debian/debian/control.in   | 16 ++
>   .../debhelper-compat-virtual-relation-both/desc|  7 +
>   .../pre_build  |  5 
>   .../debhelper-compat-virtual-relation-both/tags|  0
>   .../debian/debian/control.in   | 16 ++
>   t/tests/debhelper-compat-virtual-relation/desc | 12 
>   .../debhelper-compat-virtual-relation/pre_build|  5 
>   t/tests/debhelper-compat-virtual-relation/tags |  2 ++
>   15 files changed, 151 insertions(+), 9 deletions(-)
> 
> 
> Regards,
> 

Thanks for implementing this. :)


debhelper-and-debhelper-compat-virtual-relation-with-unsupported-version
might be better worded as
"debhelper-build-dependency-implied-by-debhelper-compat-relation" and E
might too high for this.  There is not much harm in having the redundant
build-dependency.

On a related note, the text for that tag saying  "However, this requires
a debhelper 11.3.6 or later." is incorrect.  Having debhelper-compat (=
X) implies debhelper (>= 11.3.6~) - i.e. you can rely on any feature in
debhelper up and including version 11.3.6~ without an explicit versioned
relation on debhelper (but I am fine with lintian not keeping track of
that).

Thanks,
~Niels

Bug#905467: lintian: Please detect source-only non-free uploads w/o opt-in autobuild

[Niels Thykier]

Chris Lamb:
> tags 905467 + pending
> thanks
> 
> Implemented in Git, pending upload:
> 
>   
> https://salsa.debian.org/lintian/lintian/commit/514313860d51930c986bbcb3eca6ec4ac7905852
> 
>   checks/changes-file.desc   | 12 
>   checks/changes-file.pm |  8 +++-
>   debian/changelog   |  3 +++
>   t/changes/changes-file-source-only-non-free.changes.in | 18 
> ++
>   t/changes/changes-file-source-only-non-free.desc   |  5 +
>   t/changes/changes-file-source-only-non-free.log|  1 +
>   t/changes/changes-file-source-only-non-free.tags   |  1 +
>   7 files changed, 47 insertions(+), 1 deletion(-)
> 
> 
> Regards,
> 

I seem to recall that "XS-Autobuild: Yes" requires being added to a
whitelist as well as having the header (A slightly dated reference:
https://lwn.net/Articles/211820/ - given it is using ".net" and not ".org")

Thanks,
~Niels

Bug#905423: lintian: Fails to process (coll/src-orig-index) several source packages

[Niels Thykier]

Package: lintian
Version: 2.5.94
Severity: important

Spotted on lindsay.d.o:

"""
N: Starting on group libsvn-svnlook-perl/0.04-3
N: Unpacking packages in group libsvn-svnlook-perl/0.04-3
N: 
N: Processing source package libsvn-svnlook-perl (version 0.04-3, arch source) 
...
C: libsvn-svnlook-perl source (0.04-3) [source]: rules-requires-root-implicitly
cannot parse tar output from src-orig-index: -r--r--r-- 
sscotto/ABAOFFICE\domain users 319 2007-08-17 Build.PL at 
/srv/lintian.debian.org/lintian/checks/cruft.pm line 462.
internal error: cannot run cruft check on package 
source:libsvn-svnlook-perl/0.04-3
warning: skipping check of source:libsvn-svnlook-perl/0.04-3
N: 
N: Processing binary package libsvn-svnlook-perl (version 0.04-3, arch all) ...
C: libsvn-svnlook-perl binary (0.04-3) [all]: no-ctrl-scripts
C: libsvn-svnlook-perl binary (0.04-3) [all]: 
control-tarball-compression-format gz
C: libsvn-svnlook-perl binary (0.04-3) [all]: data-tarball-compression-format xz
N: Finished processing group libsvn-svnlook-perl/0.04-3
N: Starting on group libtest-mock-guard-perl/0.10-2
N: Unpacking packages in group libtest-mock-guard-perl/0.10-2
N: 
N: Processing source package libtest-mock-guard-perl (version 0.10-2, arch 
source) ...
C: libtest-mock-guard-perl source (0.10-2) [source]: 
rules-requires-root-implicitly
cannot parse tar output from src-orig-index: -rw-r--r-- 
shimada.yuji/DENA\Domain Users 1878 2014-01-16 Build.PL at 
/srv/lintian.debian.org/lintian/checks/cruft.pm line 462.
internal error: cannot run cruft check on package 
source:libtest-mock-guard-perl/0.10-2
warning: skipping check of source:libtest-mock-guard-perl/0.10-2
"""

Thanks,
~Niels

Bug#904886: lintian: Support "debhelper-compat (= X)" B-D as replacement for "debhelper (>= X~)"

[Niels Thykier]

Chris Lamb:
> Hi Niels,
> 
>> The two methods for setting the compat level (d/compat
>> vs. debhelper-compat) will mutually exclusive (enforced in debhelper
>> itself) but will co-exist in the foreseeable future.  I.e. there are
>> currently no plans to remove d/compat.
> 
> Can you just clarify this? If you specify both you get:
> 

Yes, that is what I meant with *mutually exclusive* (I see I butchered
the original sentence, which may have resulted in your confusion).
Either debian/compat containing COMPAT XOR a B-D on debhelper-compat (=
COMPAT).

> [...]
> 
> ... thus, I'm not 100% sure what you mean by "no plans" to. :)
> 
> 
> Regards,
> 

That you are free to pick either method without (deprecation) warnings
or errors as long as you use *exactly one* of them in a given package.

Thanks,
~Niels

Bug#904886: lintian: Support "debhelper-compat (= X)" B-D as replacement for "debhelper (>= X~)"

[Niels Thykier]

Package: lintian
Version: 2.5.94
Severity: wishlist

Hi,

Please update the lintian code to support the new alternative way of
setting compat levels (supported in debhelper (>= 11.3~) and later).

Instead of having a debian/compat file and a simple Build-Depends on
"debhlelper (>= ~)", the maintainer simply Build-Depends on
the versioned virtual package "debhelper-compat"
e.g. "debhelper-compat (= 11)", which debhelper provides.

I have used mscgen and apt-file to test this in the archive if you
want a live example (other maintainers have started to pick it up as
well).  These packages should basically have no debhelper related
lintian tags.

Special cases / things that work differently from the original method:

 * maintainer *can* use "debhelper (>= 11.3.6~), debhelper-compat (= 10)"
   to use compat 10 but need debhelper 11.3.6 or later.

 * The virtual provides only supports compat 9 or later.

 * The virtual provides relation should *not* include the "~" in the
   version.

The two methods for setting the compat level (d/compat
vs. debhelper-compat) will mutually exclusive (enforced in debhelper
itself) but will co-exist in the foreseeable future.  I.e. there are
currently no plans to remove d/compat.

Thanks,
~Niels

Bug#899192: lintian: header-has-overly-generic-name false positives on names merely containing util.h

[Niels Thykier]

Chris Lamb:
> tags 899192 + pending
> thanks
> 
> Russ,
> 
>>  E: libmutter-2-dev: header-has-overly-generic-name 
>> usr/include/mutter/meta/util.h
>>
>> I think this should only trigger on usr/include/util.h specifically, not
>> if the file is in any subdirectory.  Generic header names are only a
>> problem at the top level of the include hierarchy.
> 
> But of course. Almost confused why you thought an explanation was needed ;)
> 
> Fixed in Git, pending upload:
> 
>   
> https://salsa.debian.org/lintian/lintian/commit/7cc105cbf3b46ff248faa74d56a86ba6f3af912e
> 
>   data/files/generic-header-files| 2 +-
>   debian/changelog   | 4 
>   t/tests/files-header-has-overly-generic-name/debian/debian/install | 1 +
>   3 files changed, 6 insertions(+), 1 deletion(-)
> 
> 
> Regards,
> 

Does this still trigger if people install it into
"/usr/include/$MA_DIR/util.h"?  Asking because a number of -dev packages
have begun to use /usr/include/$MA_DIR (in some cases even enabling them
to become M-A: same).

Thanks,
~Niels

Bug#898431: lintian.debian.org should emit source-contains-prebuilt-wasm-binary (backport file?)

[Niels Thykier]

Chris Lamb:
> retitle 898431 please update version of file(1) on lindsay.debian.org to 
> detect .wasm files
> thanks
> 
> Bastien,
> 
>> source-contains-prebuilt-wasm-binary source tag is not emitted due to
>> too old file.
> 
> To clarify anyone else who had difficult parsing this, "file" here
> refers to file(1)/src:file, not the to the prebuilt .wasm file itself.
> 
> Niels, is this one for us or DSA?
> 

The version of file(1) we need most be in stable or stable-backports.
Once that is there, we send a patch to the DSA metapackage for lindsay
(git mirror at https://salsa.debian.org/dsa-team/mirror/debian.org) to
ensure the proper version is pulled.

Note that I am not sure whether we have/can get backports enabled for
lindsay, but if we can, the above outlines the process.

Thanks,
~Niels

Bug#898077: lintian: False positive in missing-build-dependency-for-dh-addon, python package

[Niels Thykier]

Chris Lamb:
> tags 898077 + pending
> thanks
> 
>> Lintian should perhaps check of there is a python package that meets the
>> dependency requirement? Or allow e.g. "*scour"?
> 
> We can't do a wildcard (!) but we can also check for
> python-scour. I've done this in Git, pending upload:
> 
>   
> https://salsa.debian.org/lintian/lintian/commit/fc69686ae2d6aff415762812e805af4e5e9ca627
> 
>   data/debhelper/dh_addons-manual | 1 +
>   debian/changelog| 4 
>   2 files changed, 5 insertions(+)
> 
> 
> Regards,
> 

Hi,

I am not entirely convinced this is a good idea.  As I understand it,
that dependency is a temporary measure to avoid breakage but the plan is
to remove the dependency (i.e. packages will be required to build-depend
directly on "scour" for the add-on).

I have taken the liberty of CC'ing Martin (maintainer of scour) to
clarify the situation.

Thanks,
~Niels

Bug#897608: lintian: warn that debug symbol migration is complete

[Niels Thykier]

Control: tags -1 moreinfo

On Thu, 3 May 2018 16:30:29 +0200 Graham Inggs  wrote:
> Package: lintian
> Version: 2.5.84
> Severity: wishlist
> 
> Hi Lintian maintainers
> 
> Maintainers should be warned that debug symbol migration is complete and 
> the following lines can be removed if found in debian/rules:
> 
> override_dh_strip:
>   dh_strip --ddeb-migration=...
> 
> or
> override_dh_strip:
>   dh_strip --dbgsym-migration=...
> 
> Regards
> Graham
> 
> 

lintian cannot possibly know whether the migration is complete or not
for a given package just from those lines as not everyone has migrated
at the same time (and indeed, some still have not migrated).

Thanks,
~Niels

Bug#897213: lintian: Please remove dependency-on-python-version-marked-for-end-of-life until after Buster releases

[Niels Thykier]

Chris Lamb:
> tags 897213 + pending
> thanks
> 
>> People pay attention to lintian results and so the tags
>> should be actionable.
> 
> Indeed, that's convincing enough and I have not heard anything from
> doko... :)
> 
> However, instead of removing it I've marked the tag as 
> "experimental" and with a "pedantic" severity, thus essentially
> hiding it from 99.999% of Lintian users (yet allowing us to continue
> to continue collect statistics and make it easier to re-introduce
> later)
> 
> I also updated the tag description to say:
> 
>You should not make any changes to your package based on this
>presence of this tag.
> 
> [...]
> 
> 
> Best wishes,
> 

Hi Chris,

Worst case, we can also move the tag to a separate profile (e.g.
"debian/future.profile"), which we did for the hardening tags that had a
lot of false-positives (until we figured out how to deal with them).  I
hope it will not be relevant, but in case it does... :)

Admittedly, it does have the down-side of not being collected on
lindsay.d.o.

Thanks,
~Niels

Bug#897248: lintian: Fails to process haskell-crypto on lindsay.d.o

[Niels Thykier]

Package: lintian
Version: 2.5.84
Severity: normal

I noticed that lintian.d.o reported 12 groups with issues.  After a
very quick dive, the first item in the lintian log is:

"""
N: Unpacking packages in group haskell-crypto/4.2.5.1-8
N: 
N: Processing source package haskell-crypto (version 4.2.5.1-8, arch source) ...
I: haskell-crypto source (4.2.5.1-8) [source]: duplicate-short-description 
libghc-crypto-dev libghc-crypto-prof libghc-crypto-doc
I: haskell-crypto source (4.2.5.1-8) [source]: duplicate-long-description 
libghc-crypto-dev libghc-crypto-prof libghc-crypto-doc
C: haskell-crypto source (4.2.5.1-8) [source]: rules-requires-root-implicitly
Use of uninitialized value $ownership in substitution (s///) at 
/srv/lintian.debian.org/lintian/lib/Lintian/Collect/Package.pm line 448, 
<$_[...]> line 1.
Use of uninitialized value $date in concatenation (.) or string at 
/srv/lintian.debian.org/lintian/lib/Lintian/Collect/Package.pm line 450, 
<$_[...]> line 1.
Use of uninitialized value $time in concatenation (.) or string at 
/srv/lintian.debian.org/lintian/lib/Lintian/Collect/Package.pm line 450, 
<$_[...]> line 1.
Use of uninitialized value $perm in substr at 
/srv/lintian.debian.org/lintian/lib/Lintian/Collect/Package.pm line 451, 
<$_[...]> line 1.
Use of uninitialized value $perm in exists at 
/srv/lintian.debian.org/lintian/lib/Lintian/Collect/Package.pm line 463, 
<$_[...]> line 1.
Use of uninitialized value $t in pattern match (m//) at 
/srv/lintian.debian.org/lintian/lib/Lintian/Util.pm line 1002, <$_[...]> line 1.
Use of uninitialized value $t in concatenation (.) or string at 
/srv/lintian.debian.org/lintian/lib/Lintian/Util.pm line 1009, <$_[...]> line 1.
 does not appear to be a permission string at 
/srv/lintian.debian.org/lintian/lib/Lintian/Collect/Package.pm line 466.
internal error: cannot run cruft check on package 
source:haskell-crypto/4.2.5.1-8
warning: skipping check of source:haskell-crypto/4.2.5.1-8
"""

Bug#897082: lintian: Please do not warn about debian-watch-uses-insecure-uri for ftp:// URIs

[Niels Thykier]

Chris Lamb:
> Hi Andreas,
> 
>> [...]
> ... which does seem to cover the ftp:// case. Perhaps you were
> thinking of something like:
> 
>  The watch file uses an unencrypted transport protocol for the
>  URI such as http:// or ftp://. It is recommended to use a secure
>  transport such as HTTPS for anonymous read-only access.
> 

Perhaps "... such as HTTPS or FTPS (FTP + TLS) for anonymous read-only
access." would help cover the FTP-case?

Thanks,
~Niels

Bug#895574: lintian: binary-compiled-with-profiling-enabled test fails on Ubuntu's armhf

[Niels Thykier]

Steve McIntyre:
> [...] Wow, debugging lintian tests is a lot of effort!
> 

Hi Steve,

Thanks for your work on lintian. :)

When you write that it is a lot of effort to debug the lintian tests,
then I wonder if it is because our documentation is lacking.  Like:

 * We can run the test in isolation (i.e. you don't have to run all the
   tests).
 * The output artefacts are left in the test directory (debian/test-out
   by default) for manual analysis.
 * Lintian has a --keep-lab (-v) that leaves the lab behind (and prints
   where it left it) for manual analysis.
 * Lintian can be run directly from the checkout (with any local
   modifications) to able to experimental with changes.  Including
   running it on the test artefacts to shave some overhead in the test
   runner.

Is there any of the above points, that would have made your debugging if
you had known it ahead of time?  Or any of it, that we should have made
more easily available in the documentation?


Thanks,
~Niels

Bug#895656: lintian: not-binnmuable-any-depends-all description is confusing

[Niels Thykier]

Package: lintian
Version: 2.5.81
Severity: minor

Hi,

Description says:
"""
The package is not safely binNMUable because an arch:any package
depends on an arch:all package with a (= ${source:Version}) or (=
${binary:Version}) relationship. Please use (= ${source:Version})
instead.
"""

We basically say "Don't use X or Y.  Please X instead".  Probably this
is caused by a blind replacement of "${source-Version}".

Thanks,
~Niels

Bug#890873: lintian: gcc-8-cross-ports makes lintian fork-bomb, consume all memory and fill the disk

[Niels Thykier]

Niels Thykier:
> Chris Lamb:
>> Hi Niels,
>>
> [...]
>>> Thanks, looks good to me with a quick review.
>>
>> I'll give it a whirl!
>>  
>>
>> Best wishes,
>>
> 
> Looking forward to seeing updated results on lintian.debian.org. :)
> 
> Thanks,
> ~Niels
> 

Hi,

I had a look in the logs and it is not clear to me that the blacklist is
working.  Looking for gcc-8-cross-ports I get the following:

> $ grep  gcc-8-cross-ports /srv/lintian.debian.org/logs/sync_state.log*
> /srv/lintian.debian.org/logs/sync_state.log.0:gcc-8-cross-ports/6 is 
> out-of-date: New group (triggered by 
> binary:cpp-8-alpha-linux-gnu/8-20180402-1cross2/i386)
> /srv/lintian.debian.org/logs/sync_state.log.0:Group gcc-8-cross-ports/2 
> dropped: It is not an active group

Looking for "blacklisted" (which should appear in the log if something
is blacked listed), I get nothing:

> $ grep -r blacklisted /srv/lintian.debian.org/logs
> $

I would have expected to at least see:

> Ignoring blacklisted package src:gcc-8-cross-ports

Thanks,
~Niels

Bug#890873: lintian: gcc-8-cross-ports makes lintian fork-bomb, consume all memory and fill the disk

[Niels Thykier]

Chris Lamb:
> Hi Niels,
> 
>> Thanks for having a look at this.
> 
> Thanks for the review. I've updated my patch here:
> 
>   
> https://gist.githubusercontent.com/lamby/996d85ac0e6ba0a18af2758e79912299/raw
> 
>> I have one design concern; I think the blacklist should be global
>> not per archive.
> 
> ACK, updated.
> 
>> On the more minor side of things, I would have liked to see lintian
>> still record the packages that are blacklisted.
> 
> In lieu of marking them as such on the reports, I've made it spit out log
> messages when it skips something. (For the offending packages, it will at
> least still show "processed with Lintian $very_old_version" so it should
> not be /too/ magic.)
> 
> 
>> We will probably want to map the blacklist into a hashref so we can do
>> O(1) testing to see if a source is blacklisted if the blacklist grows
>> more than a handful of packages.
> 
> I hope this is unnecessary too (!), but I'm a bit stubborn so have
> implemented it regardless.
> 
> Looking forward to your feedback... :)
> 
> 
> Best wishes,
> 

Thanks, looks good to me with a quick review.

I think we should try that variant and see what breaks. :)  Honestly, I
do not remember if you can use $foo->{'bar'} in a string interpolation,
but I guess are about to find out. :)

Thanks,
~Niels

[lintian] branch master updated (d1d7333 -> c28a134)

[Niels Thykier]

This is an automated email from the git hooks/post-receive script.

nthykier pushed a change to branch master
in repository lintian.

  from  d1d7333   spelling: Add another correction
   new  c28a134   do_fork: Fix possible fork-bomb situation on error

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "adds" were already present in the repository and have only
been added to this reference.


Summary of changes:
 debian/changelog|  3 +++
 lib/Lintian/Util.pm | 12 ++--
 2 files changed, 13 insertions(+), 2 deletions(-)

-- 
Alioth's /usr/local/bin/git-commit-notice on 
/srv/git.debian.org/git/lintian/lintian.git

[lintian] 01/01: do_fork: Fix possible fork-bomb situation on error

[Niels Thykier]

This is an automated email from the git hooks/post-receive script.

nthykier pushed a commit to branch master
in repository lintian.

commit c28a134b64c11954810164c1e08817477796007e
Author: Niels Thykier <ni...@thykier.net>
Date:   Sat Apr 7 06:28:24 2018 +

do_fork: Fix possible fork-bomb situation on error

Signed-off-by: Niels Thykier <ni...@thykier.net>
---
 debian/changelog|  3 +++
 lib/Lintian/Util.pm | 12 ++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index 7c0ebeb..70d5015 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -68,6 +68,9 @@ lintian (2.5.81) UNRELEASED; urgency=medium
   prevents, for example, "directory(s)" from triggering false-positive
   whilst still warning about "directorys".  Thanks to Patrick Matthäi
   for the report.  (Closes: #894077)
+  * lib/Lintian/Util.pm:
++ [NT] Fix a bug in do_fork that could cause lintian to fork bomb.
+  (See #890873)
 
  -- Chris Lamb <la...@debian.org>  Sun, 18 Mar 2018 22:55:23 -0400
 
diff --git a/lib/Lintian/Util.pm b/lib/Lintian/Util.pm
index bd18052..be8c6cb 100644
--- a/lib/Lintian/Util.pm
+++ b/lib/Lintian/Util.pm
@@ -904,8 +904,16 @@ sub do_fork() {
 }
 }
 }
-sigprocmask(SIG_SETMASK, $orig_mask, undef)
-  or die("sigprocmask failed: $!\n");
+if (!sigprocmask(SIG_SETMASK, $orig_mask, undef)) {
+# The child MUST NOT use die as the caller cannot distinguish
+# the caller from the child via an exception.
+my $sigproc_error = $!;
+if (not defined($pid) or $pid != 0) {
+die("sigprocmask failed (do_fork, parent): $sigproc_error\n");
+}
+print STDERR "sigprocmask failed (do_fork, child): $sigproc_error\n";
+POSIX::_exit(255);
+}
 if (not defined($pid)) {
 $! = $fork_error;
 }

-- 
Alioth's /usr/local/bin/git-commit-notice on 
/srv/git.debian.org/git/lintian/lintian.git

Bug#890873: lintian: gcc-8-cross-ports makes lintian fork-bomb, consume all memory and fill the disk

[Niels Thykier]

Olly Betts:
> On Tue, Feb 20, 2018 at 07:48:31AM +0100, Niels Thykier wrote:
>> Seen twice on lindsay.d.o in the last 24 hours.  The exact reason is
>> unknown, but it is probably during the unpacker (last non-error in the
>> log is lintian starting the unpack, plus previously we had a race-condition
>> in the unpacker that could trigger a similar fork-bomp situation).
>>
>> Most likely we never fixed this condition and only made it
>> "sufficiently unlikely" until gcc-8-cross-ports showed up.
>>
>> For now, I have disabled the crontab on lindsay.d.o to avoid it taking
>> down our host.
> 
> I tried to build the wxwidgets3.0 3.0.4+dfsg-1 in an unstable chroot a
> couple of days ago.  I use sbuild and it's configured to run lintian in
> the chroot on the built packages.  However lintian ran out of disk space
> in the chroot, which isn't something I have hit before.  And I would
> expect the build itself to need more disk space than lintian (assuming
> that the debian/rules clean target is run before lintian is).  Lintian
> appeared to be slowly failing on each binary package in turn, so I just
> hit Ctrl-C on it.
> 
> I asked on #debian-devel in case this was a known problem, and lamby
> pointed out this ticket and suggested adding a note in case this is the
> same underlying issue, so I'm doing so.  I didn't attempt to investigate
> further yet.
> 
> Cheers,
> Olly
> 

Hi Olly,

Can you check if you end up with 10+ lintian processes when you
experience the "out of disk"-issue?

If possible, can you also try lintian with the [io-async branch] to see
if that fixes your issue?  Note: It will require libio-async-perl to be
available when lintian runs (in case you do ad-hoc patching of the
installed lintian).

Thanks,
~Niels

[io-async branch]:
https://anonscm.debian.org/git/users/nthykier/lintian.git/log/?h=unpacker-io-async

Re: [lintian] 02/02: Check all subdirectories under /usr/share/doc/foo to test whether we ship example files, not just /usr/share/doc/foo/examples/.

[Niels Thykier]

Chris Lamb:
> This is an automated email from the git hooks/post-receive script.
> 
> lamby pushed a commit to branch master
> in repository lintian.
> 
> commit 9027b8319fb4635355abda605eb1fe929e982f2a
> Author: Chris Lamb 
> Date:   Tue Mar 6 22:15:26 2018 -0800
> 
> Check all subdirectories under /usr/share/doc/foo to test whether we ship 
> example files, not just /usr/share/doc/foo/examples/.
> ---
>  checks/cruft.pm  | 2 +-
>  debian/changelog | 3 +++
>  2 files changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/checks/cruft.pm b/checks/cruft.pm
> index db21095..5a7ef79 100644
> --- a/checks/cruft.pm
> +++ b/checks/cruft.pm
> @@ -1575,7 +1575,7 @@ sub _ships_examples {
>  # If we have an -examples package, assume we ship examples.
>  return 1 if $name =~ m{-examples$};
>  my @files = $binpkg->info->sorted_index;
> -return 1 if any { m{^usr/share/doc/$name/examples/$} } @files;
> +return 1 if any { m{^usr/share/doc/$name/(.+/)?examples/$} } @files;
  ^

While you are updating that code path, could you add the missing the \Q
\E around that $name (or quotemeta)?  Otherwise we get incorrect results
for packages like "g++" (interpreted as "1 or more instances of the
letter g" rather than "literally g++").

I believe this is a reoccurring problem in the code base, so you will
probably notice it elsewhere as well.

Thanks,
~Niels

Bug#891555: Detect and warn about debhelper >= 11 instead of 11~ b-d

[Niels Thykier]

Thomas Goirand:
> [...]
> 
> And then debhelper 12 is released, people start to use it, and guess
> what happens... :)
> 

I am prototyping an alternative in debhelper that may make this issue
obsolete.  At the moment (since 11.1.5) you can use:

 Build-Depends: debhelper-compat (= 11)

as a replacement for:

 Build-Depends: debhelper (>= 11~)
 and d/compat set to 11.

Note the lack of "~" in the debhelper-compat version.


It spews out tons of warnings as it is still experimental.  Nonetheless,
feedback appreciated.

Thanks,
~Niels

Bug#890959: lintian: inconsistent use of “header” terminology

[Niels Thykier]

Ben Finney:
> Control: tags -1 + patch
> 
> I have made a series of commits in my fork repository, in the branch
> ‘wip/issue/one-header-multiple-fields’, to address this bug
> .
> 
> [...]
> 



Hi Ben,

I have just a quick scroll through the changes.  I notice that we will
be renaming the tags, but I do not see any changes to:

  data/override/renamed-tags

If the patch series also update that file, then existing overrides will
not be broken by the renamed tags.

Thanks,
~Niels

Bug#890873: lintian: gcc-8-cross-ports makes lintian fork-bomb, consume all memory and fill the disk

[Niels Thykier]

Niels Thykier:
> Package: lintian
> Version: 2.5.74
> Severity: important
> 
> Seen twice on lindsay.d.o in the last 24 hours.  The exact reason is
> unknown, but it is probably during the unpacker (last non-error in the
> log is lintian starting the unpack, plus previously we had a race-condition
> in the unpacker that could trigger a similar fork-bomp situation).
> 
> Most likely we never fixed this condition and only made it
> "sufficiently unlikely" until gcc-8-cross-ports showed up.
> 
> 
> For now, I have disabled the crontab on lindsay.d.o to avoid it taking
> down our host.
> 
> Thanks,
> ~Niels
> 

Btw, I got an old prototype rewrite where the custom parallelization
handling is replaced by IO::Async in
https://anonscm.debian.org/git/users/nthykier/lintian.git/log/?h=unpacker-io-async

No clue if this handles this particular problem, but I wanted to
mentioned it in case someone else got the same idea of rewriting things
to use IO::Async.

Thanks,
~Niels

Bug#890873: lintian: gcc-8-cross-ports makes lintian fork-bomb, consume all memory and fill the disk

[Niels Thykier]

Package: lintian
Version: 2.5.74
Severity: important

Seen twice on lindsay.d.o in the last 24 hours.  The exact reason is
unknown, but it is probably during the unpacker (last non-error in the
log is lintian starting the unpack, plus previously we had a race-condition
in the unpacker that could trigger a similar fork-bomp situation).

Most likely we never fixed this condition and only made it
"sufficiently unlikely" until gcc-8-cross-ports showed up.


For now, I have disabled the crontab on lindsay.d.o to avoid it taking
down our host.

Thanks,
~Niels