On 09/09/26 at 15:35 +1000, Stuart Prescott wrote:
> lintian looks for documentation outside /usr/share/doc and emits the
> package-contains-documentation-outside-usr-share-doc tag if it finds
> some. Of course there are some files that might look like documentation
> but are actually in the right place and so should not cause this tag to
> be emitted.
>
> One such set of files are in the `dist-info` directory of a python
> module, where various pieces of metadata that represent how the module
> was built, its dependencies (for the Python world) and licence data are
> stored. For example:
>
> /usr/lib/python3/dist-packages/python_debian-1.1.1.dist-info
> /usr/lib/python3/dist-packages/python_debian-1.1.1.dist-info/INSTALLER
> /usr/lib/python3/dist-packages/python_debian-1.1.1.dist-info/METADATA
> /usr/lib/python3/dist-packages/python_debian-1.1.1.dist-info/top_level.txt
>
> However, lintian has started flagging these files with
> package-contains-documentation-outside-usr-share-doc.
>
> I: python3-debian: package-contains-documentation-outside-usr-share-doc
> [usr/lib/python3/dist-packages/python_debian-1.1.1.dist-info/top_level.txt]
> N:
> N: This package ships a documentation file outside /usr/share/doc
> N: Documentation files are normally installed inside /usr/share/doc.
> N:
> N: If this file doesn't describe the contents or purpose of the directory it
> N: is in, please consider moving this file to /usr/share/doc/ or maybe even
> N: removing it. If this file does describe the contents or purpose of the
> N: directory it is in, please add a lintian override.
> N:
> N: Visibility: info
> N: Show-Always: no
> N: Check: documentation
> N:
> N: Screen: python/egg/metadata
> N: Advocates: "Scott Kitterman" <[email protected]>
> N: Reason: The folders XXX.dist-info/ and XXX.egg-info/ hold metadata for
> N: Python modules. Those files are not documentation even though
> N: some of their names carry the .txt file extension.
> N:
> N: Python modules can be both public and private.
> N:
> N: Read more in
> N:
> https://www.python.org/dev/peps/pep-0427/#the-dist-info-directory,
> N: https://www.python.org/dev/peps/pep-0376/#id16,
> N: https://www.python.org/dev/peps/pep-0610/,
> N: https://www.python.org/dev/peps/pep-0639/,
> N:
> https://setuptools.pypa.io/en/latest/deprecated/python_eggs.html,
> N: and Bug#1003913.
>
> (I leave the details of the intended streen in the output.)
>
> This is a regression - up until recently, lintian did not complain about
> these text files within the Python module definition.
Hi Stuart,
I looked into this.
(What follows is the AI-generated summary of a long human-driven
discussion with an agent)
TL;DR: This is not a regression in lintian itself - 2.122.0, 2.136.1, 2.139.0
and
2.140.0 all behave identically for your package. What changed is the package:
python3-debian 1.1.1 no longer ships a WHEEL file in its .dist-info directory
(1.0.1 and 1.1.0 did). That newly surfaces two long-standing lintian bugs. The
report is valid - these files should not be flagged - but it's not a recent
regression.
--------------------------------------------------
Problem analysis
The tag package-contains-documentation-outside-usr-share-doc is suppressed for
Python metadata by two independent mechanisms, and both fail for your package:
1. The screen requires a WHEEL file.
lib/Lintian/Screen/Python/Egg/Metadata.pm suppresses a .dist-info/ directory
only when it contains both METADATA and WHEEL:
return 1
if $item->dirname =~ m{ [^/] [.] dist-info / $}x
&& defined $item->parent_dir->child('METADATA')
&& defined $item->parent_dir->child('WHEEL');
WHEEL is only produced for packages built as wheels. A package installed via
setup.py install (like python_debian-1.1.1) has METADATA but no WHEEL - your
dist-info contains INSTALLER, METADATA, top_level.txt, and nothing else.
Earlier package versions (1.0.1, 1.1.0) happened to ship a WHEEL file, so the
screen worked for them. The METADATA file is the real, definitive marker of a
dist-info directory (PEP 427/376), so requiring WHEEL is too strict.
Required change: drop the WHEEL check so a .dist-info/ directory is recognized
by METADATA alone (keep the egg-info/PKG-INFO branch as-is):
return 1
if $item->dirname =~ m{ [^/] [.] dist-info / $}x
&& defined $item->parent_dir->child('METADATA');
2. The exclusion filename list never actually takes effect.
lib/Lintian/Check/Documentation.pm already lists top_level.txt (and robots.txt,
entry_points.txt, etc.) in @NOT_DOCUMENTATION_FILE_REGEXES, but the guard is
written as:
and any { $item->basename !~ m{$_}xi } @NOT_DOCUMENTATION_FILE_REGEXES)
any { BLOCK } LIST returns true if the block is true for at least one element
of the list. Here the block is "the basename does not match this regex", so the
whole test asks: "Is there at least one exclusion regex that the basename does
not match?"
The list holds ten different regexes (e.g. ^top_level[.]txt$, ^robots[.]txt$,
^entry_points[.]txt$, ...). A single basename can match at most a couple of
them. So for top_level.txt, although it matches ^top_level[.]txt$, it does not
match ^robots[.]txt$, ^entry_points[.]txt$, ^dependency_links[.]txt$, etc.
Because there are many regexes it does not match, the answer to "is there at
least one it doesn't match?" is almost always yes - so the test is true in
nearly every case, and the exclusion never actually excludes anything.
What was intended is the opposite: a file should only be flagged if it matches
none of the exclusion regexes - i.e. "the basename should not match any of the
exclusion regexes." The bug is that any { !~ } tests the wrong thing.
Required change: invert the test so a filename is only flagged when it matches
none of the exclusion regexes:
and none { $item->basename =~ m{$_}xi } @NOT_DOCUMENTATION_FILE_REGEXES)
(none is already exported by List::SomeUtils elsewhere in the codebase.)
--------------------------------------------------
Why it looked like a regression
I tested your exact scenario across multiple lintian releases:
Package 2.122.0 2.136.1 2.139.0 2.140.0
python3-debian 1.1.1 (no WHEEL) flagged flagged flagged flagged
python3-debian 1.0.1 (has WHEEL) suppress suppress suppress suppress
plain top_level.txt flagged flagged flagged flagged
The WHEEL-dependent screen logic and the broken exclusion guard are both
present in all these versions. The behavior is constant - what changed is the
package. Since you "up until recently" had a version of python3-debian that
shipped WHEEL, lintian suppressed these files; upgrading to 1.1.1 exposed the
latent bugs.
Verification considerations for the fix
- A legitimate documentation file such as user-guide.txt must still be flagged
(no false negatives).
- A .dist-info/ directory without METADATA should still be flagged (don't
over-suppress).
- Add a regression test for a .dist-info/ directory containing METADATA but no
WHEEL (e.g. under t/recipes/checks/documentation/)
Best,
Lucas
