Bug#1041317: dgit: table too wide in man page, trashes autopkgtests

2023-07-17 Thread Ian Jackson
Control: severity -1 serious

Hi.  Thanks for your comprehensive, detailed and helpful email.

G. Branden Robinson writes ("Bug#1041317: dgit: table too wide in man page, 
trashes autopkgtests"):
> Package: dgit
> Version: 9.13
> Severity: Important
> Justification: causes problems for everybody's autopkgtests
> 
> I noticed that groff 1.23.0-2 (and -1 before it) will be blocked from
> ever migrating to testing due to an autopkgtest failure on EVERY
> architecture.

I'm sorry that this test is causing you an inconvenience.  Thanks for
filing the bug.  I think the right severity for a bug which is
preventing another package from migrating is serious, so I am raising
the severity of your report.

> https://tracker.debian.org/pkg/groff
> 
> So I downloaded the 2.4MB compressed autopkgtest log for groff 1.23.0-2
> on amd64 and trawled through the 325,000+ lines of output looking for
> the problem.  On the one hand, that is an obnoxiously huge amount of
> information.  On the other, the log is annotated well enough that I was
> able to quickly locate the problem.

Thanks very much for investigating.  And thanks also to Colin for the
MR adjusting the regexp.

> Here are the juicy bits.
...
> 941s RE (?^:^(?:(?^:^(?=a)b))$)|(?^:^(?:ERROR.*)$)|(?^:^(?:.* # table wider 
> than line width)$)
...
> That regex is sufficiently complex that I can't tell if it's trying to
> filter the diagnostic message "table wider than line width" or not.  If
> it is, the fact that I have recast the language of the diagnostic
> message in groff 1.23.0 has fooled it.

Yes, that is what it is doing.  Unfortunately as far as I'm aware,
groff doesn't offer a formal warning suppression or classification
mechanism.  So instead there is this regexp, which (as might be
expected of such a thing) is fragile and keeps growing cases as
messages change, becoming ever more baroque and inscrutable.

> It is a bad idea to rely on the exact wording of diagnostic messages.
> That is one reason error conditions in many software systems are given
> an unintelligible identifier.

Quite so.  If you had an alternative way to suppres this, I would
have been happy to hear it!  However later you tell me how to avoid
the warning completely, which is much better.

> https://git.savannah.gnu.org/cgit/groff.git/commit/?id=7111d3378f0c2ceab891d66ae815d393ff87dae5
> 
> Yes, the language in the regex could be updated if it's doing what I
> think it is, but that just kicks the can down the road.  It is better
> for groff to have the freedom to continue to improve its diagnostic
> messages for the comprehension of the user.

I agree.

> So let's the fix the problem in the dgit man page.
> 
> Man pages are formatted for a width of 78n if the terminal width is 80
> columns.  This origin of this practice is not well documented but
> experience with groff upstream leads me to believe that it is a
> workaround for bugs in GNU tbl(1).  (In AT Unix Version 7, they were
> formatted for a line length of 65n, with a page offset of one tenth of
> an inch.  On Western Electric Teletype Model 37 printing terminals.)
> 
> How wide is dgit's man page?
...
> 79
...
> Hence the warning.
> 
> groff 1.22.4 didn't used to throw this warning in this circumstance.

I'm pretty sure at least one version of groff I encountered produced
this warning, and that's why I have this adhoc regexp-based ignore
rule.

> The diagnostic is wholly legitimate.
> 
> Let's have a look:
> 
> $ MANPAGER=cat MANWIDTH=80 command man --warnings -Tascii dgit | sed -n 
> '/DESCRIPTION/,/OPERATIONS/p'
> :46: warning: table wider than line length minus indentation
> DESCRIPTION
...
>dgit-maint-gbp(7) for maintainers already using 
> git-buildpackage
...
> Yup, if we look carefully, we can see that the word "git-buildpackage"
> encroaches into the right margin.

This is true in a strictly technical sense: the generated formatting
does violate groff's the intended behaviour.  However, the output is,
in fact, perfectly fine, when looked at from a user's point of view:
the output line is 79 characters long, which as you note doesn't in
fact currently cause trouble (other than this warning).

So I felt justified in ignoring the warning.  But, I didn't want to
ignore all warnings since it is so easy to introduce syntax errors
etc.  Hence the fragile regexp.

I think that the best fix would be this:

> I would like in the future to improve GNU tbl to the point where tables
> can spread their wings to the full 80 column span of the widely accepted
> minimum terminal width, but a deeply ingrained feature of GNU tbl makes
> that tough.
> 
> Maybe it will happen for groff 1.24.

I quite believe you that it's difficult.

> My recommendation is a simple tweak to the table format, stealing one en

Bug#1041317: dgit: table too wide in man page, trashes autopkgtests

2023-07-17 Thread Colin Watson
On Mon, Jul 17, 2023 at 06:24:41AM -0500, G. Branden Robinson wrote:
> I noticed that groff 1.23.0-2 (and -1 before it) will be blocked from
> ever migrating to testing due to an autopkgtest failure on EVERY
> architecture.
> 
> https://tracker.debian.org/pkg/groff

Note that I also filed an MR to fix the same thing:

  https://salsa.debian.org/dgit-team/dgit/-/merge_requests/7

I have no objection to Branden's patch here though, and if that's
applied then I believe mine can be rejected.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]



Bug#1041317: dgit: table too wide in man page, trashes autopkgtests

2023-07-17 Thread G. Branden Robinson
Package: dgit
Version: 9.13
Severity: Important
Justification: causes problems for everybody's autopkgtests

I noticed that groff 1.23.0-2 (and -1 before it) will be blocked from
ever migrating to testing due to an autopkgtest failure on EVERY
architecture.

https://tracker.debian.org/pkg/groff

So I downloaded the 2.4MB compressed autopkgtest log for groff 1.23.0-2
on amd64 and trawled through the 325,000+ lines of output looking for
the problem.  On the one hand, that is an obnoxiously huge amount of
information.  On the other, the log is annotated well enough that I was
able to quickly locate the problem.

Here are the juicy bits.

[...]
941s + for roff in $manpages
941s + section=1
941s + page=dgit
[...]
941s + cmd='man --warnings "$@" $section $page'
941s + eval 'man --warnings "$@" $section $page 2>&1 >/dev/null |tee 
/tmp/autopkgtest-lxc.3j70k01o/downtmp/autopkgtest_
tmp/dgit.1.txt-errs'
941s ++ man --warnings 1 dgit
941s ++ tee 
/tmp/autopkgtest-lxc.3j70k01o/downtmp/autopkgtest_tmp/dgit.1.txt-errs
941s :46: warning: table wider than line length minus 
indentation
[...]
941s RE (?^:^(?:(?^:^(?=a)b))$)|(?^:^(?:ERROR.*)$)|(?^:^(?:.* # table wider 
than line width)$)
941s unexpected: :46: warning: table wider than line length 
minus indentation
941s unexpected errors
941s END failed--call queue aborted, <> line 1.
[...]
941s  EXITING 22 
[...]
941s TEST FAILED
[...]
2435s manpages-format  FAIL non-zero exit status 16

That regex is sufficiently complex that I can't tell if it's trying to
filter the diagnostic message "table wider than line width" or not.  If
it is, the fact that I have recast the language of the diagnostic
message in groff 1.23.0 has fooled it.

It is a bad idea to rely on the exact wording of diagnostic messages.
That is one reason error conditions in many software systems are given
an unintelligible identifier.

https://git.savannah.gnu.org/cgit/groff.git/commit/?id=7111d3378f0c2ceab891d66ae815d393ff87dae5

Yes, the language in the regex could be updated if it's doing what I
think it is, but that just kicks the can down the road.  It is better
for groff to have the freedom to continue to improve its diagnostic
messages for the comprehension of the user.

So let's the fix the problem in the dgit man page.

Man pages are formatted for a width of 78n if the terminal width is 80
columns.  This origin of this practice is not well documented but
experience with groff upstream leads me to believe that it is a
workaround for bugs in GNU tbl(1).  (In AT Unix Version 7, they were
formatted for a line length of 65n, with a page offset of one tenth of
an inch.  On Western Electric Teletype Model 37 printing terminals.)

How wide is dgit's man page?

$ MANPAGER=cat MANWIDTH=80 command man --warnings dgit |wc -L
:46: warning: table wider than line length minus indentation
79

Hence the warning.

groff 1.22.4 didn't used to throw this warning in this circumstance.

That was a bug.

https://savannah.gnu.org/bugs/index.php?61854

Now it does.

The diagnostic is wholly legitimate.

Let's have a look:

$ MANPAGER=cat MANWIDTH=80 command man --warnings -Tascii dgit | sed -n 
'/DESCRIPTION/,/OPERATIONS/p'
:46: warning: table wider than line length minus indentation
DESCRIPTION
   dgit allows you to treat the Debian archive as if it were a git reposi-
   tory.   Conversely, it allows Debian to publish the source of its pack-
   ages as git branches, in a format which is directly useable by ordinary
   people.

   This is the command line reference.  Please read the tutorial(s):
   dgit-user(7)  for users: edit, build and share packages
   dgit-nmu-simple(7)for DDs: do a straightforward NMU
   dgit-maint-native(7)  for maintainers of Debian-native packages
   dgit-maint-debrebase(7)   for maintainers: a pure-git rebasish workflow
   dgit-maint-merge(7)   for maintainers: a pure-git merging workflow
   dgit-maint-gbp(7) for maintainers already using git-buildpackage
   dgit-sponsorship(7)   for sponsors and sponsored contributors
   dgit-downstream-dsc(7)setting up dgit push for a new distro

   See dgit(7) for detailed information about the data model, common prob-
   lems likely to arise with certain kinds of package, etc.

OPERATIONS

Yup, if we look carefully, we can see that the word "git-buildpackage"
encroaches into the right margin.

My recommendation is a simple tweak to the table format, stealing one en
of column separation to make the table fit.

$ diff -u ./dgit.1.orig ./dgit.1
--- ./dgit.1.orig   2023-07-17 06:03:04.465368217 -0500
+++ ./dgit.12023-07-17 06:03:26.309288710 -0500
@@ -42,7 +42,7 @@
 This is the command line reference.
 Please read the tutorial(s):
 .TS
-lb l.
+lb2 l.
 dgit-user(7)   for users: edit, build and share packages
 dgit-nmu-simple(7) for DDs: do a straightforward NMU
 dgit-maint-native(7)   for maintainers of