Re: groff warning: TE macro called with TW register undefined

[Hugh McMaster]

Hi Branden,

On Thu, 17 Aug 2023 at 10:08, G. Branden Robinson wrote:
>
> Hi Hugh,
>
> [...]
> So whether the man page is hand-crafted or generated from some other
> format at build time is not relevant.  In fact, it's _meta_ irrelevant
> (if that's even a thing) because the only reason (apart from the
> validation Lintian's doing) for a source project to run groff at build
> time on a man page is to generate a PDF, "cat page", or other
> _formatted_ version of the man page.[1]  Examples of these are rare, but
> Bash and NetHack generate cat pages, and groff itself generates a PDF
> compilation of all its man pages.[2]

Thanks for the explanation. While my research indicated that tbl was
not being called, I couldn't find any way of calling it, short of
`groff -t`.

> > > What exact check is failing here (is it lintian, or something else)?
> > > Could you please give us a pointer to the man page in question?
> >
> > Lintian issues the warning when checking for man-page issues using
> > groff (via man). This particular warning has only appeared since the
> > recent groff 1.23.0 upload.
>
> The warning is new to groff 1.23.0.[3]
>
> > The Lintian tag is 'groff-message'. The tag description helpfully
> > provides the exact command used during the check:
> >
> > LC_ALL=C.UTF-8 MANROFFSEQ='' MANWIDTH=80 \
> > man --warnings -E UTF-8 -l -Tutf8 -Z  >/dev/null
> >
> > The man page in question -- ftlint.1 -- is in the freetype2-demos
> > package [1], or you can get an online copy from [2].
>
> In the course of composing this message, I see that Colin covered this
> point.[4]

Yes. I had no idea preprocessors could be invoked via man(1).

> I would also note that you can use the grog(1) command (new and improved
> in groff 1.23.0![5]) to help you figure out which preprocessors (and
> macro package) a *roff document needs.[6]
>
> Let me know if this helps, or does not.

It does help, yes. I ran grog(1) on the man page in question and the
output immediately indicated that `tbl` was needed.

Thank you for your help.

> Regards,
> Branden
>
> [1] Another use case is to produce non-man-page manuals from *roff
> sources using a macro package like "ms", "me", or "mm".  groff
> supports all of these and it was commonly done in pre-Web days, but
> it's now sadly close to a lost art.  The _Unix Time-Sharing System
> Programmer's Manual Seventh Edition Volume 2_ (1979) remains
> valuable reading and an example of high-quality technical writing
> apply *roff to ends other than man pages.
>
> 
> https://archive.org/details/bitsavers_attunix7thersManualSeventhEditionVol21983_34117955/
>
> It's particularly valuable for learning "classical Unix" tools in
> their early and more easily grasped forms.  I've found that GNU
> manuals, in spite of the advantages touted for Texinfo for
> preparation of book-length works over mere reference guides (a.k.a.
> man pages), are nevertheless often written in the style of man pages
> with little effort made to give the reader a perspective from which
> to integrate knowledge of the (nearly always) larger interface and
> feature list of GNU replacements for Unix tools.  In other words,
> they too often suffer from the same defects that the GNU Coding
> Standards attribute to man pages.
>
> 
> https://web.archive.org/web/20041029120203/http://www.gnu.org/prep/standards/html_node/GNU-Manuals.html#GNU-Manuals
>
> (To be fair, more recent versions of the GNU Coding Standards have
> moved--slightly--in the direction of acknowledging that the
> quality of the technical writing, not the formatting language used
> to compose it, that is the predominant factor in production of
> useful manuals.)
>
> [2] https://www.gnu.org/software/groff/manual/groff-man-pages.pdf
> [3] 
> https://git.savannah.gnu.org/cgit/groff.git/commit/?id=80ee140eb0616b794b853bbad623263cbea06abc
> [4] https://lists.debian.org/debian-devel/2023/08/msg00220.html
> [5] Yes, I can feel my eternal soul tumbling down several rungs in Hell
> for engaging in promotion.
>
> [6] https://man.cx/grog
>
> I was going to link to
> https://manpages.debian.org/unstable/groff/grof.1.en.html here, but
> the man page is missing!  groff definitely ships it.  Any advice?

Re: groff warning: TE macro called with TW register undefined

[Hugh McMaster]

On Thu, 17 Aug 2023 at 09:10, Colin Watson wrote:
>
> Your problem is that you need this line as the very first line of the
> page to instruct man(1) to run the tbl preprocessor:
>
>   '\" t
>
> See https://manpages.debian.org/bookworm/man-db/man.1.en.html#DEFAULTS

That does the trick. Thank you!

I knew there had to be a simple solution, but I didn't know man(1)
could be instructed to call a preprocessor.

Re: groff warning: TE macro called with TW register undefined

[G. Branden Robinson]

At 2023-08-17T01:37:52+0100, Colin Watson wrote:
> On Wed, Aug 16, 2023 at 07:08:18PM -0500, G. Branden Robinson wrote:
> > [6] https://man.cx/grog
> > 
> > I was going to link to
> > https://manpages.debian.org/unstable/groff/grof.1.en.html here, but
> > the man page is missing!  groff definitely ships it.  Any advice?
> 
> Aside from the typo in the last URL segment, the penultimate segment
> identifies the binary package, and grog(1) is in the groff-base package
> rather than groff.

Thanks, Colin.  That explains why I also couldn't find it under
.

Upstream mentality plus a senior moment--not a good combination!

>   https://manpages.debian.org/unstable/groff-base/grog.1.en.html

Regards,
Branden


signature.asc
Description: PGP signature

Re: groff warning: TE macro called with TW register undefined

[Colin Watson]

On Wed, Aug 16, 2023 at 07:08:18PM -0500, G. Branden Robinson wrote:
> [6] https://man.cx/grog
> 
> I was going to link to
> https://manpages.debian.org/unstable/groff/grof.1.en.html here, but
> the man page is missing!  groff definitely ships it.  Any advice?

Aside from the typo in the last URL segment, the penultimate segment
identifies the binary package, and grog(1) is in the groff-base package
rather than groff.

  https://manpages.debian.org/unstable/groff-base/grog.1.en.html

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: groff warning: TE macro called with TW register undefined

[G. Branden Robinson]

Hi Hugh,

At 2023-08-17T07:54:03+1000, Hugh McMaster wrote:
> On Thu, 17 Aug 2023 at 03:39, Colin Watson wrote:
> > On Wed, Aug 16, 2023 at 09:29:45PM +1000, Hugh McMaster wrote:
> > > This all seems promising. Unfortunately, the man page is
> > > hand-crafted, not generated from another source, so groff is never
> > > invoked, and no other documents are referenced in the file.
> >
> > groff must be run somewhere, because you're seeing a warning from
> > groff.
> 
> I meant that groff is never called by FreeType's build system, as the
> man page is hand-crafted.

Okay.  But Lintian _is_ calling groff, to format man pages and sniff out
problems with them.

So whether the man page is hand-crafted or generated from some other
format at build time is not relevant.  In fact, it's _meta_ irrelevant
(if that's even a thing) because the only reason (apart from the
validation Lintian's doing) for a source project to run groff at build
time on a man page is to generate a PDF, "cat page", or other
_formatted_ version of the man page.[1]  Examples of these are rare, but
Bash and NetHack generate cat pages, and groff itself generates a PDF
compilation of all its man pages.[2]

> > What exact check is failing here (is it lintian, or something else)?
> > Could you please give us a pointer to the man page in question?
> 
> Lintian issues the warning when checking for man-page issues using
> groff (via man). This particular warning has only appeared since the
> recent groff 1.23.0 upload.

The warning is new to groff 1.23.0.[3]

> The Lintian tag is 'groff-message'. The tag description helpfully
> provides the exact command used during the check:
> 
> LC_ALL=C.UTF-8 MANROFFSEQ='' MANWIDTH=80 \
> man --warnings -E UTF-8 -l -Tutf8 -Z  >/dev/null
> 
> The man page in question -- ftlint.1 -- is in the freetype2-demos
> package [1], or you can get an online copy from [2].

In the course of composing this message, I see that Colin covered this
point.[4]

I would also note that you can use the grog(1) command (new and improved
in groff 1.23.0![5]) to help you figure out which preprocessors (and
macro package) a *roff document needs.[6]

Let me know if this helps, or does not.

Regards,
Branden

[1] Another use case is to produce non-man-page manuals from *roff
sources using a macro package like "ms", "me", or "mm".  groff
supports all of these and it was commonly done in pre-Web days, but
it's now sadly close to a lost art.  The _Unix Time-Sharing System
Programmer's Manual Seventh Edition Volume 2_ (1979) remains
valuable reading and an example of high-quality technical writing
apply *roff to ends other than man pages.


https://archive.org/details/bitsavers_attunix7thersManualSeventhEditionVol21983_34117955/

It's particularly valuable for learning "classical Unix" tools in
their early and more easily grasped forms.  I've found that GNU
manuals, in spite of the advantages touted for Texinfo for
preparation of book-length works over mere reference guides (a.k.a.
man pages), are nevertheless often written in the style of man pages
with little effort made to give the reader a perspective from which
to integrate knowledge of the (nearly always) larger interface and
feature list of GNU replacements for Unix tools.  In other words,
they too often suffer from the same defects that the GNU Coding
Standards attribute to man pages.


https://web.archive.org/web/20041029120203/http://www.gnu.org/prep/standards/html_node/GNU-Manuals.html#GNU-Manuals

(To be fair, more recent versions of the GNU Coding Standards have
moved--slightly--in the direction of acknowledging that the
quality of the technical writing, not the formatting language used
to compose it, that is the predominant factor in production of
useful manuals.)

[2] https://www.gnu.org/software/groff/manual/groff-man-pages.pdf
[3] 
https://git.savannah.gnu.org/cgit/groff.git/commit/?id=80ee140eb0616b794b853bbad623263cbea06abc
[4] https://lists.debian.org/debian-devel/2023/08/msg00220.html
[5] Yes, I can feel my eternal soul tumbling down several rungs in Hell
for engaging in promotion.

[6] https://man.cx/grog

I was going to link to
https://manpages.debian.org/unstable/groff/grof.1.en.html here, but
the man page is missing!  groff definitely ships it.  Any advice?


signature.asc
Description: PGP signature

Re: groff warning: TE macro called with TW register undefined

[Colin Watson]

On Thu, Aug 17, 2023 at 07:54:03AM +1000, Hugh McMaster wrote:
> The man page in question -- ftlint.1 -- is in the freetype2-demos
> package [1], or you can get an online copy from [2].
> 
> [1] /usr/share/man/man1/ftlint.1.gz
> [2] 
> https://sources.debian.org/src/freetype/2.13.0%2Bdfsg-1/ft2demos/man/ftlint.1/

Your problem is that you need this line as the very first line of the
page to instruct man(1) to run the tbl preprocessor:

  '\" t

See https://manpages.debian.org/bookworm/man-db/man.1.en.html#DEFAULTS.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: groff warning: TE macro called with TW register undefined

[Hugh McMaster]

On Thu, 17 Aug 2023 at 03:39, Colin Watson wrote:
>
> On Wed, Aug 16, 2023 at 09:29:45PM +1000, Hugh McMaster wrote:
> > This all seems promising. Unfortunately, the man page is hand-crafted,
> > not generated from another source, so groff is never invoked, and no
> > other documents are referenced in the file.
>
> groff must be run somewhere, because you're seeing a warning from groff.

I meant that groff is never called by FreeType's build system, as the
man page is hand-crafted.

> What exact check is failing here (is it lintian, or something else)?
> Could you please give us a pointer to the man page in question?

Lintian issues the warning when checking for man-page issues using
groff (via man). This particular warning has only appeared since the
recent groff 1.23.0 upload.

The Lintian tag is 'groff-message'. The tag description helpfully
provides the exact command used during the check:

LC_ALL=C.UTF-8 MANROFFSEQ='' MANWIDTH=80 \
man --warnings -E UTF-8 -l -Tutf8 -Z  >/dev/null

The man page in question -- ftlint.1 -- is in the freetype2-demos
package [1], or you can get an online copy from [2].

[1] /usr/share/man/man1/ftlint.1.gz
[2] 
https://sources.debian.org/src/freetype/2.13.0%2Bdfsg-1/ft2demos/man/ftlint.1/

Re: groff warning: TE macro called with TW register undefined

[Colin Watson]

On Wed, Aug 16, 2023 at 09:29:45PM +1000, Hugh McMaster wrote:
> This all seems promising. Unfortunately, the man page is hand-crafted,
> not generated from another source, so groff is never invoked, and no
> other documents are referenced in the file.

groff must be run somewhere, because you're seeing a warning from groff.
What exact check is failing here (is it lintian, or something else)?
Could you please give us a pointer to the man page in question?

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Re: groff warning: TE macro called with TW register undefined

[Hugh McMaster]

Hi Branden,

On Wed, 16 Aug 2023 at 00:30, G. Branden Robinson wrote:
>
> Hi Hugh,
>
> I work on groff upstream.

Perfect :-)

> At 2023-08-15T22:46:30+1000, Hugh McMaster wrote:
> > groff-message an.tmac::66: warning: tbl preprocessor
> > failed, or it or soelim was not run; table(s) likely not rendered (TE
> > macro called with TW register undefined)
> >
> > It seems TW is not defined in the number register.
> >
> > While I can define TW by adding `.nr TW 0` before .TS, I don't know if
> > that's the correct solution.
>
> It is a bad idea.  The purpose of the `TW` register is defined in the
> tbl(1) man page.
>
> The register TW stores the width of the table region in basic units;
> it can’t be used within the region itself, but is defined before the
> .TE token is output so that a groff macro named TE can make use of
> it. [...]

Okay. I didn't think this approach was the solution.

> This information is necessary for macro packages that want to center
> tables, a common feature of packages _except_ those that render man
> pages.  Because forgetting to run the tbl preprocessor is a common
> error, during groff 1.23.0 development it occurred to me that I could
> use this register as an indicator.
>
> > Any ideas on fixing this warning, or is it best to ignore it for now?
>
> The diagnostic message is trying to warn you that you _probably_ forgot
> to run tbl.  But there are other causes.
>
> 1.  tbl crashed or aborted.
>
> 2.  The man(7) document embedded the beginning of table in a different
> file that it "sourced" with the `so` request, but soelim(1) was not
> run on the document.  This is valid, but not considered portable man
> page composition style.[1]
>
> 3.  A `TE` macro call is in the man page with no `TS` preceding it.
> This could arise due to clumsy editing, unfamiliarity with the
> man(7) language, or bad luck combined with inexperience (an ordinary
> text line happened to begin with `.TE` and the page author didn't
> realize that *roff would attempt to interpret that as a macro call).
>
> I tried to cover these bases in the diagnostic, but might have failed.

This all seems promising. Unfortunately, the man page is hand-crafted,
not generated from another source, so groff is never invoked, and no
other documents are referenced in the file.

AFAICT, the syntax is correct, with .TS, table content, and .TE macros
all present.

> Can you tell me what part of the message was confusing?

I woudn't say it's confusing; the message clearly states the problem
is that TW is undefined.

For me, I see more of a "chicken and egg" problem. `groff -t` needs to
be invoked to register the TW value, but the file is hand-crafted, so
groff isn't run.

> Regards,
> Branden
>
> [1] Any *roff can handle this.  So can mandoc(1)--I just found that out.
> This fact raises some interesting possibilities, since everything
> that was ever called "man2html", the original motive for
> groff_man(7)'s Portability section[2] has long gone to the dustbin.
>
> [2] In groff 1.23.0, this section is now in groff_man_style(7).

Re: groff warning: TE macro called with TW register undefined

[Hugh McMaster]

Hi Colin,

On Tue, 15 Aug 2023 at 23:55, Colin Watson wrote:
>
> On Tue, Aug 15, 2023 at 10:46:30PM +1000, Hugh McMaster wrote:
> > Lintian has recently started emitting this groff warning for ftlint.1
> > in freetype-demos as part of its man pages check:
> >
> > groff-message an.tmac::66: warning: tbl preprocessor
> > failed, or it or soelim was not run; table(s) likely not rendered (TE
> > macro called with TW register undefined)
>
> I haven't looked at your code in detail, but this sounds similar to
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1042358.  groff is most 
> likely observing that
> you're using the tbl macros but haven't invoked groff with -t, which
> means any warnings it issues are going to be inaccurate representations
> of how pages are actually rendered in practice.  Your starting point
> should be to add -t to the groff command line in this check, and then
> see what else shows up after that.

It is similar, yes, but unfortunately, I'm working with a hand-crafted
man page file.

Re: groff warning: TE macro called with TW register undefined

[G. Branden Robinson]

Hi Hugh,

I work on groff upstream.

At 2023-08-15T22:46:30+1000, Hugh McMaster wrote:
> groff-message an.tmac::66: warning: tbl preprocessor
> failed, or it or soelim was not run; table(s) likely not rendered (TE
> macro called with TW register undefined)
> 
> It seems TW is not defined in the number register.
> 
> While I can define TW by adding `.nr TW 0` before .TS, I don't know if
> that's the correct solution.

It is a bad idea.  The purpose of the `TW` register is defined in the
tbl(1) man page.

The register TW stores the width of the table region in basic units;
it can’t be used within the region itself, but is defined before the
.TE token is output so that a groff macro named TE can make use of
it. [...]

This information is necessary for macro packages that want to center
tables, a common feature of packages _except_ those that render man
pages.  Because forgetting to run the tbl preprocessor is a common
error, during groff 1.23.0 development it occurred to me that I could
use this register as an indicator.

> Any ideas on fixing this warning, or is it best to ignore it for now?

The diagnostic message is trying to warn you that you _probably_ forgot
to run tbl.  But there are other causes.

1.  tbl crashed or aborted.

2.  The man(7) document embedded the beginning of table in a different
file that it "sourced" with the `so` request, but soelim(1) was not
run on the document.  This is valid, but not considered portable man
page composition style.[1]

3.  A `TE` macro call is in the man page with no `TS` preceding it.
This could arise due to clumsy editing, unfamiliarity with the
man(7) language, or bad luck combined with inexperience (an ordinary
text line happened to begin with `.TE` and the page author didn't
realize that *roff would attempt to interpret that as a macro call).

I tried to cover these bases in the diagnostic, but might have failed.

Can you tell me what part of the message was confusing?

Regards,
Branden

[1] Any *roff can handle this.  So can mandoc(1)--I just found that out.
This fact raises some interesting possibilities, since everything
that was ever called "man2html", the original motive for
groff_man(7)'s Portability section[2] has long gone to the dustbin.

[2] In groff 1.23.0, this section is now in groff_man_style(7).


signature.asc
Description: PGP signature

Re: groff warning: TE macro called with TW register undefined

[Colin Watson]

On Tue, Aug 15, 2023 at 10:46:30PM +1000, Hugh McMaster wrote:
> Lintian has recently started emitting this groff warning for ftlint.1
> in freetype-demos as part of its man pages check:
> 
> groff-message an.tmac::66: warning: tbl preprocessor
> failed, or it or soelim was not run; table(s) likely not rendered (TE
> macro called with TW register undefined)

I haven't looked at your code in detail, but this sounds similar to
https://bugs.debian.org/1042358.  groff is most likely observing that
you're using the tbl macros but haven't invoked groff with -t, which
means any warnings it issues are going to be inaccurate representations
of how pages are actually rendered in practice.  Your starting point
should be to add -t to the groff command line in this check, and then
see what else shows up after that.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]