Re: [groff] groff as the basis for comprehensive documentation?

[Ralph Corderoy]

Hi Steve,

> I'm not sure why you consider `.\&' important: is that for
> end-of-sentence recognition?

No, the opposite, it's to stop an end-of-sentence character being taken
as the end of a sentence.

> I've never used double spaces for sentences, but I recognize there are
> good arguments in favour of it.

If your input has only a single space after the full stop that is the
end of a sentence but not the end of a line, then it will typeset
different to one that does fall at the end of a line.  Compare
`appropriate.' and `wrong.'.

$ nroff <<\E | grep .
.na
Indeed, I did have a relationship with Miss Lewinsky that was not
appropriate. In fact, it was wrong.
It constituted a critical lapse in judgment and a personal failure
on my part for which I am solely and completely responsible.
E
Indeed, I did have a relationship with Miss Lewinsky that was not
appropriate. In fact, it was wrong.  It constituted a critical
lapse in judgment and a personal failure on my part for which I
am solely and completely responsible.
$ 

I prefer the troff norm of a wider space, but have to look out for `Data
by Spock et al.\& shows' because editing the input might place the `.\&'
at the end of a line.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi Ralph,

Ralph Corderoy wrote on Wed, Apr 25, 2018 at 05:00:01PM +0100:
> Ingo Schwarze wrote:

>> I work on mdoc(7) manual pages a lot, and i almost never look at any
>> kind of output to see whether the final formatting comes out in the
>> desired visual form.  While writing, i exclusively think about the
>> logical structure of the text and the semantic function of each word
>> and symbol.  (I do periodically check the rendered console output,
>> though - but only because finding typos is easier in the rendered form
>> than in the source code.)  I certainly never look at HTML or
>> PDF/PostScript output to see whether it comes out right.  I just
>> *know* it will - or if it won't then that's a bug in the formatter
>> which i have to fix.

> Don't you ever forget to `\&' after a `.' that's not the end of
> sentence,

That problem is much less common than one might think because of
the "new sentence, new line" rule - basically, only dots at the
end of an input line that are not full stops need the escaping,
and that situation occurs quite rarely.

> for example, and not notice because it fortunately falls where
> it doesn't matter on the TTY but stands out when viewed in a
> higher-resolution format?

It almost always stands out on the terminal, too, because -Tascii
inserts two blanks after the end of a sentence, so the problem
is clearly visible unless the dot also happens to be at the end
of an output line.  But the latter can happen in PostScript or PDF
output, too.

> I find looking at the PostScript/PDF valuable precisely because
> it's a different rendering and thus shows problems hidden by the
> rendering used when writing.

I think in theory, you are right - there are formatting oversights
that are invisible in ASCII output but may affect typeset output.
The fact the your example is not very convincing is not a coincidence,
though.  Such issues are relatively uncommon and many people regard
them as somewhat arcane.  The most common such problem is probably
confusion of the input characters "-", "\-", "\(mi", and "\(en"
which is invisible in -Tascii - but can be seen in -Tutf8 if you
look closely.  The next one is confusing "\(lq", "\(rq", and "\(dq"
- again, invisible in -Tascii, visible in -Tutf8.  Same for "\(oq",
"\(cq", and "\(aq".  Confusing "\ ", "\~", and "\0" is even more
arcane, invisible in -Tutf8 and easy to miss even in typeset output.
There may be a few other points, but i expect them all to be
relatively rare and minor.

Besides, in stark contrast to man(7), mdoc(7) rarely requires using
these characters by hand (with the exception of "-", "\-", and
"\(en"), because you use .Qq, .Dq, .Sq, and .Ql, which cannot go
wrong.  Besides, we don't harass manual page authors over errors
of this kind, so scrutinizing my own work for them would be
considerable effort for minor benefit - after all, these kinds of
issues are not easy to see even in PDF output.

In principle, you are right, different output formats allow bugs
that only show in some or even only in one of them.  But the standard
process of writing manuals with mdoc(7) is deliberately set up in
a way to make this effect in practice almost irrelevant.  It's a
*feature* that authors almost never need to worry about all that
and can concentrate on the content and logic.

Yours,
  Ingo

Re: [groff] groff as the basis for comprehensive documentation?

[Steve Izma]

On Wed, Apr 25, 2018 at 10:49:25PM +0100, Ralph Corderoy wrote:
> Subject: Re: [groff] groff as the basis for comprehensive documentation?
> 
> > When I write, I only want to think about the words on the screen and
> > the structure of my argument
> 
> And when you've finished writing, do you not look at an output and spot
> problems due to concentrating on the content?  It's also troff input,
> e.g. `.\&', and although that shouldn't interrupt your flow when
> writing, it does need to be considered in some pass.

I see what you mean. I didn't clarify that I write using a simple
markup scheme (much simpler than markdown-style systems), so I'm
not using troff in the source (except for more complicated, short
items like posters and theatre or music programmes, in which I'll
enter troff requests directly). For longer pieces using only the
structural elements I mentioned, I need very little in markings
(e.g., blank lines for paragraphs, dashes around subheads). I
have simple awk scripts (sometimes a more complicated python
script, depending on the output) to run the finished text through
groff and to PS or PDF or XML, etc.

I'm not sure why you consider `.\&' important: is that for
end-of-sentence recognition? I would never start a text line with
a character that needs '\&' for escaping, although I'll use it
often in my conversion scripts. I very often write at least my
first drafts with each sentence beginning a new line, if I want
sentence-ending recognition in vim or groff. I've never used double
spaces for sentences, but I recognize there are good arguments in
favour of it.

> > I should not need to make significant changes to the source text in
> > order for it to read properly in these different formats.
> 
> Only if you've no errors in the first place, errors that might not show
> up if you don't check a formatted output, or check only one.

For *content* errors, one should be able to use a text editor
that makes the source readable enough to do this (until you reach
the author-self-proofing problem I mentioned earlier; then it's
up to others to read a cleanly formatted output).

For *typesetting* or *style* errors, of course there's going to
be back and forth. But I don't reach that point until I'm
satisfied with the textual content. After that point and to
adjust the typeset output, I work in either the troff or XML
source file to add processing instructions (mostly things like
kerning and extra spacing, which are really just last-resort
fixes to formatting when the troff macro can't handle all
possible situations), but since I'm no longer editing the content
all this extra markup isn't interfering with reading the text.

-- Steve

-- 
Steve Izma
-
Home: 35 Locust St., Kitchener, Ontario, Canada  N2H 1W6
E-mail: si...@golden.net  phone: 519-745-1313  cell: 519-998-2684

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
<http://en.wikipedia.org/wiki/Posting_style>

Re: [groff] groff as the basis for comprehensive documentation?

[Ralph Corderoy]

Hi Steve,

> When I write, I only want to think about the words on the screen and
> the structure of my argument

And when you've finished writing, do you not look at an output and spot
problems due to concentrating on the content?  It's also troff input,
e.g. `.\&', and although that shouldn't interrupt your flow when
writing, it does need to be considered in some pass.

> I should not need to make significant changes to the source text in
> order for it to read properly in these different formats.

Only if you've no errors in the first place, errors that might not show
up if you don't check a formatted output, or check only one.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [groff] groff as the basis for comprehensive documentation?

[Mike Bianchi]

On Wed, Apr 25, 2018 at 03:58:33PM -0400, Steve Izma wrote:
> When I write, I only want to think about the words  ...
> and the structure of my argument ...

May I add a big Amen!

-- 
 Mike Bianchi

Re: [groff] groff as the basis for comprehensive documentation?

[Steve Izma]

On Wed, Apr 25, 2018 at 05:00:01PM +0100, Ralph Corderoy wrote:
> Subject: Re: [groff] groff as the basis for comprehensive documentation?
> 
> > I work on mdoc(7) manual pages a lot, and i almost never look at any
> > kind of output to see whether the final formatting comes out in the
> > desired visual form.  While writing, i exclusively think about the
> > logical structure of the text and the semantic function of each word
> > and symbol.  (I do periodically check the rendered console output,
> > though - but only because finding typos is easier in the rendered form
> > than in the source code.)  I certainly never look at HTML or
> > PDF/PostScript output to see whether it comes out right.  I just
> > *know* it will - or if it won't then that's a bug in the formatter
> > which i have to fix.
> 
> ... I find looking at the PostScript/PDF valuable
> precisely because it's a different rendering and thus shows problems
> hidden by the rendering used when writing.

I completely agree with Ingo's approach because it demonstrates
the whole point of differentiating between content and format.
When I write, I only want to think about the words on the screen
and the structure of my argument, which is my interpretation of
semantic structure -- paragraph breaks (corresponding to the
notion of a paragraph presenting a single coherent part of the
overall argument), sub-heads to flag a shift in the discussion,
blockquotes to distinguish other people's thoughts, etc. Maybe
also lists, but not much else structually speaking.

This, of course, is a statement about writing in general, not
specifically about writing man pages, but I think the logic
holds.

I also usually assume that I will end up rendering the text in
various ways, e.g., typeset on paper, plain text in an e-mail,
similarly structured input for a wiki. And the typeset output
is not necessarily fixed either: sometimes I'll want it in two
columns on a letter-sized page, sometimes a single column on a
book page. I should not need to make significant changes to the
source text in order for it to read properly in these different
formats. We all know that's the job of the tmac package.

In respect to proofreading, I don't have problems writing and
editing within a text editor. But, over the decades I've worked
in publishing, I've come to see it as a rule that authors
fairly quickly get so familiar with their text that they see what
they want to see, rather than what's actually on the page, so
proofreading is never complete until someone else does it. That
often means an additional way of formatting the text for others
to correct it (e.g., additional line spacing, wider margins for
notes).

-- Steve

-- 
Steve Izma
-
Home: 35 Locust St., Kitchener, Ontario, Canada  N2H 1W6
E-mail: si...@golden.net  phone: 519-745-1313  cell: 519-998-2684

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
<http://en.wikipedia.org/wiki/Posting_style>

Re: [groff] groff as the basis for comprehensive documentation?

[Ralph Corderoy]

Hi Ingo,

> I work on mdoc(7) manual pages a lot, and i almost never look at any
> kind of output to see whether the final formatting comes out in the
> desired visual form.  While writing, i exclusively think about the
> logical structure of the text and the semantic function of each word
> and symbol.  (I do periodically check the rendered console output,
> though - but only because finding typos is easier in the rendered form
> than in the source code.)  I certainly never look at HTML or
> PDF/PostScript output to see whether it comes out right.  I just
> *know* it will - or if it won't then that's a bug in the formatter
> which i have to fix.

Don't you ever forget to `\&' after a `.' that's not the end of
sentence, for example, and not notice because it fortunately falls where
it doesn't matter on the TTY but stands out when viewed in a
higher-resolution format?  I find looking at the PostScript/PDF valuable
precisely because it's a different rendering and thus shows problems
hidden by the rendering used when writing.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

It's taken me several hours to write this, so it damn well better be
enlightening.

> What i meant by the above sentence is that the CSS you gave,
> with the exception of the dfn{} and kbd{} selectors, selects
> nothing based on kind or class of content or semantic function.
> …
> consequently, the EXAMPLES section is indeed rendered in a
> misleading way on your example page

Actually, that stylesheet is unfinished. It's because of the browser's
default styling that most of the page's content is convincingly styled. As
for the examples section, yeah, it was jarring enough I pushed a fix just
for that. Note the selector I used in the ruleset
:
‘samp kbd’ applies its properties to any  element nested inside a
, which as the spec states

represents
the part of a user's input as echoed back by the system (for a command
prompt, that means the entire line). Contextually applying or removing
properties is common in CSS.

> So, with your stylesheet, almost all semantic information from all
> macros seems completely unhandled to me - or what am i missing?

I don't understand what you mean when you say "semantic information". The
stylesheet is building off the default browser styling, which doesn't need
to be repeated. It's senseless to write this, for example:

b { font-weight: bold; }
i { font-style: italic; }

u { text-decoration: underline; }


... because  tags are already bold by default,  tags are italic by
default, etc. The same is true for this:

kbd { font-weight: bold; }



Since a browser's default styling varies between vendors, it's common
practice to use something like Normalize.css
 to smooth out variances. I
use a modified
version of it

which the HTML page I wrote uses.

> They are not redundant. Their purpose is to display the semantic
> function of the word in a tooltip when hovering the mouse over it.

What the hell. Why? Is every reader expected to understand what Ar means if
the cursor rolls over “*options”* or what-have-you? I'm puzzled, and can't
see any *practical* reason for this. Compare that with the tooltips of the
links you mentioned, which show a short summary for *unrelated* content:

The HTMLElement interface represents any HTML element.
Some elements directly implement this interface, others
implement it via an interface that inherits it.


> The title element would be read *in addition* to the contents of
> the element, rather than instead of the contentis, right?

No. In place of. Moreover, this depends on how a screen-reader is
configured to handle title attributes.

> That is *not* at all what i want.  It is vital that any browser,
> even those i never heard of, is able to show this information to
> the user.
>
> It is neither "non-visible" nor "private" data, but an important
> part of the output, to be shown directly to the user.

Again, you'll have to explain why tag-names in a tooltip are at all
relevant to the authored material. If information is important for the
reader, it should obviously be part of the content itself. Whether
something is an argument, a flag, or a command modifier should be quite
evident to a reader by context.

> I want the output to be comprehensible even without any stylesheet.

I'd say the following is comprehensible without *anything*:

[-ac] [-I os=name] [-K encoding] [-mdoc | -man]


> I could not find any HTML element that is adequate for fixed
> syntax elements and clearly distinguishable from example code.
> Can you point me to one?

What do you mean by "fixed syntax"? Monospaced text? There are several:

   -   is
   used to represent user input verbatim. Historically, this was only used to
   represent keys on a keyboard (hence the name). Its meaning was broadened in
   HTML5 to refer to any form of user input: menus, command lines, spoken
   commands, et al.
   -  
   is used for program output, such as a command transcript
   

   .
   -   is
   used for anything else, and is typically what you'll use for inline
   monospace text. For code blocks, it should be used inside 
   :
e.g., 
   … . Note that  alone doesn't mean source code: it
   could be used for chat logs and ASCII art, for example. Furthermore, it
   doesn't imply a fixed-pitch typeface will *always* be used: simply that
   whitespace must be preserved by the user-agent verbatim.


> I dimly remember that i considered that, but decided against it
> because 

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi John,

John Gardner wrote on Sat, Apr 21, 2018 at 04:48:33PM +1000:

> Ingo, I've spent the last 13 years in front-end web development,
> and I've been writing standards-compliant websites for almost
> a decade.

Sounds like you might have valueable input that could end up
improving the mandoc -Thtml output.  Note that i did *not*
claim that i specialize in anything related to HTML/CSS, which
i actually do not.  Quite to the contrary, whenever i had
questions related to HTML/CSS, i had a hard time finding any
developer who knew much about it.  So i might finally get some
real help, looking forward to that...

>> I see absolutely nothing semantic in there, it looks like a
>> purely presentational style sheet to me.

> ... yes, that's the entire reason CSS exists: to separate
> presentation from content (the latter being tantamount with
> "semantics" as understood by web authors and those of us who
> actively follow modern web standards).

Wait - the point of CSS is to select adequate presentation
for content of a given kind or class, right?  So the CSS, on its
input side, first needs to be told, by HTML elements and attributes,
what kind or class of content some text belongs to, and then has
to select the presentational attributes using selectors addressing
these kinds and classes of HTML elements, right?

What i meant by the above sentence is that the CSS you gave,

https://rawgit.com/Alhadis/Stylesheets/master/complete/manpage/manpage.css

with the exception of the dfn{} and kbd{} selectors, selects nothing
based on kind or class of content or semantic function.  What you
do with def{} handles one single macro, and kbd{} is used for very
different kinds of content that need completely different formatting,
namely fixed syntax elements like command line options (.Fl) and
fixed option arguments (.Cm) on the one hand but also code examples
(.Dl) on the other hand - consequently, the EXAMPLES section is
indeed rendered in a misleading way on your example page, compare to

  https://man.openbsd.org/mandoc.1#EXAMPLES

So, with your stylesheet, almost all semantic information from all
macros seems completely unhandled to me - or what am i missing?

> Redundant title attributes on everything.

They are not redundant.  Their purpose is to display the semantic
function of the word in a tooltip when hovering the mouse over it.
I'll gladly do that in a better way if i find one or someone directs
me into the right direction, but when i read the CSS standard, i
failed to find any other way.

As a matter of fact, the very document you quote uses title
attributes for exactly the same purpose:

https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/data-*

Look at the source code for the elements that show tool tips,
like "HTMLElement" or "DOMStringMap".

> Actually, worse than redundant:
>it screws with assistive technologies like screen-readers, which
>might read the contents of a tag to the user using the title
>attribute if one is present.

The title element would be read *in addition* to the contents of
the element, rather than instead of the contentis, right?  That
would actually be useful, because not being able get a visual
impression of the page as a whole, hearing the page word by word,
it is even harder to correctly guess the semantic function of key
words because you at first lack the necessary context of these
words, which a visual impression of the page can provide without
reading anything.  Besides, listening to a screen reader, you
wouldn't hear what is bold or italic, so getting the meta-information
across in some different, verbal way seems useful to me.

I admit, though, that i relatively rarely work with people who use
screen readers, only every few months maybe, and never asked them
to test the mandoc output.

>If you want to attach page or application-specific metadata to
>elements, use data-*
>
> 
>instead.

Looking at the link given there,

https://html.spec.whatwg.org/multipage/dom.html#embedding-custom-non-visible-data-with-the-data-*-attributes

i read:

  3.2.6.6 Embedding custom non-visible data with the data-* attributes
  [...]
  These attributes are not intended for use by software that is not
  known to the administrators of the site that uses the attributes.

That is *not* at all what i want.  It is vital that any browser,
even those i never heard of, is able to show this information to
the user.

It is neither "non-visible" nor "private" data, but an important
part of the output, to be shown directly to the user.

So i don't quite understand why you suggest data-*.
No user would never see those attributes, right?

> *Presentational tags used instead of those conveying text-level
> semantics: *You're
>literally doing what mdoc(7) tells you not to do, except in HTML form:
>   - -b, -S, -o:
>   Flags/options should be represented using kbd tags, as they 

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi Branden,

G. Branden Robinson wrote on Sat, Apr 21, 2018 at 03:06:14AM -0400:
> At 2018-04-20T23:19:44+0200, Ingo Schwarze wrote:
>> John Gardner wrote:
>>> Ingo Schwarze wrote:

 man://mandoc.1#EXIT_STATUS

>>> Now, as for the SHOUTY SHOUTY...

>> That's not a matter of SHOUTING, but of case sensitivity.
>> The name of that standard section in man(7) and mdoc(7)
>> is "EXIT STATUS", not "Exit Status" nor "Exit status"
>> nor "exit status".  Case is preserved, consider:

>> That's a bad idea.  I admit that many authors use unusual and even
>> inconsistent casing in section headers (even in the very mandoc.1)-:,
>> which may sometimes seem awkward.  But in technical documentation,
>> casing is often deliberate, and automatically changing it based on
>> natural language rules is prone to make information incorrect in
>> some cases.

> I disagree with most of this analysis.  As far as I can tell this
> was a presentational decision,

True.  The convention "write the first argument of .TH, .Dt,
and all arguments of .Sh in ALL CAPS" is a premature presentational
decision.

> In my opinion, which I am far too young and poorly-connected to have
> proffered when it would have made any difference, the
> forced-full-capitalization of section titles in man page sources is an
> information-destroying transform

I strongly agree with that: once done, it cannot be undone.

> done in the wrong place at the wrong
> time.  Section headings should be capitalized as section titles normally
> are in technical documentation: either like work titles, or first-letter
> only, with the normal rules for proper nouns and adjectives respected.

I mildly agree with that sentiment, as explained in my other
posting - except for the large amount of work it would cause me.

> It would be better if man-db (or similar) set a *roff variable
> that the macro package would check to see if case transformation on
> section headings was desired.  The default, for the next n years, of
> course, would be to go ahead and do the transformation to avoid shocking
> people.

So what?  With two explicit OKs - namely, from Michael Kerrisk and
Jason McIntyre - i would switch that right away and not even provide
a knob for people to turn.  Let them be shocked.  Things change.

The problem is, i have to change several thousand manual pages first
even if we all agree it would be nice to have - is that amount of
work well invested, or are there more important things to do?


The point at *this* point of the thread is a *completely*
different one:  HTML anchors are not only used for section
headers, but also for command names, function names, variable
names, and many other syntax elements where case is no doubt
syntactically important.  So as long as the section name
appearing in a specific manual page is "EXIT STATUS", the
anchor has to be exactly that, and no case variation.

> This has been itching me for many years; thanks for the excuse to
> air my grievance.  ;-)

Yours,
  Ingo

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi Larry,

Larry Kollar wrote on Sat, Apr 21, 2018 at 10:44:43AM -0400:

> I'm in violent agreement with your and Brandon's assessment
> of all-caps headings.  Shoot, there aren't many
> terminal programs that *can't* do bold these days;
> I think even that VT220 I keep
> planning to connect to my iMac can do it.
> 
> Given a choice, I never all-caps my manpage headings.

Not a real problem, your pages are still readable and correct -
and you argue they are even more readable and nicer.  Fair enough.

Some parsers may warn, including mandoc:

mandoc: :1:5: STYLE: lower case character in document title: Dt foo
mandoc: :1:5: STYLE: lower case character in document title: TH foo

But it's only flagged as a STYLE level issue, meaning that there
is no ambiguity, portability, or formatting problem.

> I've succumbed to convention once or twice, maybe.
> It would be cool if man(1) could recognize an all-caps string
> flush left, and turn it into bolded title case.

Err, no, serious layering violation alert.  In implementations
were the formatter and man(1) are separate programs, the man(1)
utility should please *not* try to change the formatting, not
even in a well-intentioned way.  It should only display the
rendered version it got, unchanged.

The formatter can't do the tranformation either: once the case of
some string was clobbered into ALL CAPS, going back programmatically
is impossible, information was lost.  For example, the OpenBSD
ipsec.conf(5) manual contains

  .Sh IPSEC.CONF FILE FORMAT

and it might contain

  .Sh IPSEC AUTOMATIC KEYING

No formatter can figure out that the correct capitalization is

  .Sh ipsec.conf file format
  .Sh IPsec automatic keying

I'm not necessarily opposed to getting rid of this antiquated
convention (though i would have to check with Jason, who sometimes
is aware of arguments that i miss), except that i would probably
be the poor guy who would have to edit thousands of manual pages,
a process that can only partly be automated and certainly requires
checking all of them by hand.

You will understand that while i see the point why the convention
is not good - and even worse for .TH and .Dt than for .Sh - i'm not
thrilled by the idea of getting rid of it so late in the game, for
the sheer volume of churn it would cause.

Yours,
  Ingo

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi Nate,

Nate Bargmann wrote on Sat, Apr 21, 2018 at 12:59:16PM -0500:

> But why do we focus on presentation when authoring a document?
> I think that in large part we have trained ourselves to do so
> using the various desktop word processing/publishing programs
> that came down the pike.
> I've done plenty of documents where the content was subservient to the
> presentation (newsletters and such).  A word processor forces a
> presentation first mindset because its only real output is the printed
> page.  It's not a GUI thing, I was spending as much time tweaking the
> presentation in the codes editor in Word Perfect 5.1 on MS-DOS in 1990
> as I was writing lab reports.

I think it depends on where you come from.  Not counting pure-ASCII
text editing for fixed-width line printers that couldn't do *any*
kind of formatting at all in the early 80ies, i grew up with LaTeX
as *the* one and only text processor for serious work (not counting
the occasional Word document for quickly preparing a leaflet to
distribute on Campus, which we didn't really take seriously in the
sense of "text processing").  And when i typeset formulae for
theoretical high enery phsics, i used to not only make sure that
the content of the formulae was correct and that the printed output
was easy to read and pretty, but also that the source code was
easily human-readable and conveyed the structure of the formulae
that were often several lines long.  That applied even when i had
to design new glyphs for specialized symbols, typically by grouping
existing component glyphs, or to code new environments for specialized
function-like notations.

> Working in the presentation mindset for so long makes it difficult to
> think in terms of semantic markup.  I'm not sure I really understand it
> fully.  Even working with the man macros, I still spend some amount of
> time checking the presentation on the terminal and with PDF and HTML
> output.

I work on mdoc(7) manual pages a lot, and i almost never look at
any kind of output to see whether the final formatting comes out
in the desired visual form.  While writing, i exclusively think
about the logical structure of the text and the semantic function
of each word and symbol.  (I do periodically check the rendered
console output, though - but only because finding typos is easier
in the rendered form than in the source code.)  I certainly
never look at HTML or PDF/PostScript output to see whether it
comes out right.  I just *know* it will - or if it won't then
that's a bug in the formatter which i have to fix.

This anecdotal evidence still confirms your observation, though.
The chief maintainer of the OpenBSD manuals, Jason McIntyre, trained
himself for many years by editing mdoc(7) manual pages with an
ancient, now long obsolete version of groff, groff-1.15, which was
much more quirky than the current, much cleaner groff after Werner's
major work that led up to groff-1.17.  I know definitely that Jason
is still in the habit of checking that every change renders in the
way he desires.  During those years, he got so used to work around
quirks in the parsers and formatters by tweaking the source that
for some years, i found more bugs in mandoc by inspecting his
workaround commits than by explicit bug reports - even though he
did report many bugs.  He *expected* the formatter to be quirky and
thoroughly trained himself to "tweak for effect" by using quirky
software for a long time.

That effect does exist, but is is not unavoidable, and i think
it is even reversible, though of course unlearning is always
harder than learning.

Yours,
  Ingo

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi James and Nate,

James K. Lowden wrote on Sat, Apr 21, 2018 at 06:58:59PM -0400:
> On Sat, 21 Apr 2018 08:16:36 -0500 Nate Bargmann wrote:

>> Note that I am only working with Groff's man macro package and do
>> understand that other macro packages may have greater demands on the
>> HTML generator.

> It's not the demands on the HTML *generator* that present the problem.
> The input to grohtml is devoid of information HTML itself demands, and
> is thick with information it can't use.  
> 
>   HTML needs: titles, paragraphs, tables
>   ditroff provides: positions, text, fonts
> 
> The only reason it works at all is that the pre-grohtml preprocessor
> sneaks some useful information to the postprocessor via ditroff
> escapes.  That allows grohtml to generate MathML from eqn output, for
> example.  
> 
> While it's possible to work that way, I see no advantage to squirting
> most of the needed information to the post-processor through the
> formatter, when the formatter's work is discarded by the
> post-processor.  Why bother?  just translate the macros directly.  
> 
> Which, I suppose, it what Ingo is doing already.  

Yes, confirmed, that is entirely accurate.  So the program
doing exactly that will be ten years old the coming winter.

It is also the reason why the former, original name of the mandoc
project, before it grew support for the man(7) input language
and the -Tascii, -Tutf8, -Tps, and -Tpdf output modes was
chosen as "mdocml" by the original author, Kristaps Dzonsons:
It meant "mdoc->html", i.e. "go directly from mdoc(7) to HTML,
without the devastating detour via the troff intermediate format".

Yours,
  Ingo

Re: [groff] groff as the basis for comprehensive documentation?

[Deri James]

On Sat 21 Apr 2018 08:16:36 Nate Bargmann wrote:
> Note that I am only working with Groff's man macro package and do
> understand that other macro packages may have greater demands on the
> HTML generator.

You might consider supporting mdoc macros. You may have seen:-

http://chuzzlewit.co.uk/WebManPDF.pl/man:/7/groff_mdoc

Which shows what a preprocessor inserting Keith's pdfmark extensions can do. 
Compare with:-

http://chuzzlewit.co.uk/WebManPDF.pl/man:/7/groff_man

Which has less semantic markup to use.

Note this runs on a very small arm powered computer, so there is a noticeable 
delay in serving the pages, it is much faster on my desktop. 

Re: [groff] groff as the basis for comprehensive documentation?

[G. Branden Robinson]

At 2018-04-21T18:21:28-0400, James K. Lowden wrote:
> On Sat, 21 Apr 2018 12:59:16 -0500
> Nate Bargmann  wrote:
> 
> > But why do we focus on presentation when authoring a document?  
> 
> Because the document we see is what we're creating.  The only purpose
> of the document is to be read, and appearance matters to the reader.
> The reader assuredly doesn't care about semantic markup.  
> 
> Semantic markup is an abstraction that assists the author.  It can
> ensure that semantically similar things -- section headers, quotations,
> etc. -- are rendered consistently.  It can help -- somewhat -- in
> rendering the same input text pleasantly in different output formats.
> It can help indexing programs and such knit together different
> documents in ways that are useful to the user (and thus to the
> author).  
> 
> There's no Higher Good honored by using semantic markup.  It's a tool
> to use if it gets you where you want to go.  Ultimately, though, you
> want your text to be read, and it is altogether fitting and proper that
> you be concerned with its appearance.  

This is true for literature, like poetry or a novel.

For nonfiction, there are additional objectives--full-text searchability
and cross-referencing.  For technical documentation of software systems,
we can further add templating.

By "templating" I mean the little markup-convention languages we use to
distinguish literal text from variable text, and punctuation symbols
like brackes, braces, bars, and ellipses we draft into service to
communicate the regularity (in the formal sense) of the underlying
expressions.

-- 
Regards,
Branden


signature.asc
Description: PGP signature

Re: [groff] groff as the basis for comprehensive documentation?

[James K. Lowden]

On Sat, 21 Apr 2018 08:16:36 -0500
Nate Bargmann  wrote:

> > For lack of a better term, I think it's an abstraction mismatch.
> > The ditroff language presupposes a dot-addressable canvas, onto
> > which lines and strings of text are drawn.  That model fits most
> > printers (these days) and terminals.  But it doesn't describe HTML
> > at all.  
> 
> I suppose it depends on what one expects from the generated HTML.  As
> one who reads pages more than writes them, I've been impressed with
> the presentation on the man-pages project Web site (hosted at
> kernel.org). For example, here is the rendering of groff_man(7):
> 
> http://man7.org/linux/man-pages/man7/groff_man.7.html
> 
> I couldn't find the generator being used in the Git repository and a
> lot of it may be done with CSS.  The text is rendered using the 
> tag so it looks much like tty output though it is not fully justified
> yet the text blocks are indented.  Aside from the justification, the
> rest looks very familiar to me. 

I find very little to commend that version.  In fact, it's an excellent
example of the widespread dunderheaded monospace manpage rendering on
the web.  I invite you to compare it with something better: 

https://linux.die.net/man/7/groff_man

(Better url, too, btw: "man/7/groff_man" captures everything
"man-pages/man7/groff_man.7.html" does in half the space.)  

(One has to wonder, though, at the age of that document.  For
amusement, follow the link in See Also.)  

There's nothing in the manpage input text that specifies the font family
to be used.  The Postscript rendering uses proportional fonts and
italics, much more pleasant to read than nroff output in a terminal.
Why shouldn't the HTML output make the highest and best use of its
medium, instead of poorly emulating a 40 year-old obsolete hardware?  

> Note that I am only working with Groff's man macro package and do
> understand that other macro packages may have greater demands on the
> HTML generator.

It's not the demands on the HTML *generator* that present the problem.
The input to grohtml is devoid of information HTML itself demands, and
is thick with information it can't use.  

HTML needs: titles, paragraphs, tables
ditroff provides: positions, text, fonts

The only reason it works at all is that the pre-grohtml preprocessor
sneaks some useful information to the postprocessor via ditroff
escapes.  That allows grohtml to generate MathML from eqn output, for
example.  

While it's possible to work that way, I see no advantage to squirting
most of the needed information to the post-processor through the
formatter, when the formatter's work is discarded by the
post-processor.  Why bother?  just translate the macros directly.  

Which, I suppose, it what Ingo is doing already.  

--jkl

Re: [groff] groff as the basis for comprehensive documentation?

[James K. Lowden]

On Sat, 21 Apr 2018 12:59:16 -0500
Nate Bargmann  wrote:

> But why do we focus on presentation when authoring a document?  

Because the document we see is what we're creating.  The only purpose
of the document is to be read, and appearance matters to the reader.
The reader assuredly doesn't care about semantic markup.  

Semantic markup is an abstraction that assists the author.  It can
ensure that semantically similar things -- section headers, quotations,
etc. -- are rendered consistently.  It can help -- somewhat -- in
rendering the same input text pleasantly in different output formats.
It can help indexing programs and such knit together different
documents in ways that are useful to the user (and thus to the
author).  

There's no Higher Good honored by using semantic markup.  It's a tool
to use if it gets you where you want to go.  Ultimately, though, you
want your text to be read, and it is altogether fitting and proper that
you be concerned with its appearance.  

--jkl

Re: [groff] groff as the basis for comprehensive documentation?

[Steffen Nurpmeso]

Nate Bargmann  wrote:
 |* On 2018 19 Apr 18:47 -0500, James K. Lowden wrote:
 |> On Fri, 20 Apr 2018 01:44:06 +1000
 |> John Gardner  wrote:
  ...
 |I, for one, do not expect an HTML rendered version of *anything* to
 |be a faithful representation of a printed page.  The output of
 |"groff -man -Thtml ..." is reasonable for my purposes, except that I'd
 |like to be able to generate relative links in a page to other pages in
 |my project.  Maybe I just need to do some sed(1) transformation after
 |groff is done.

I have for example

  .   el .ie 'html'\*[.T]' \
  . mx:dump-init-html
[...]
  .de mx:dump-init-html
  . \" grohtml is real shit..
  . mso html.tmac
[get HTML macros]
  . als mx:dump-xr  mx:dump-xr-html
  . als mx:dump-anchor  mx:dump-anchor-html
  . als mx:dump-ref mx:dump-ref-html
  ..
  .
  .de mx:dump-xr-html
  . \" Cannot help it
  ..
  .
  .de mx:dump-anchor-html
  . TAG "\$2"
  . if d mx-debug \
  .   HTML #\$2#
  ..
  .
  .de mx:dump-ref-html
  . URL "#\$1" "[\$1]"
[create an active reference]
  ..

I have to excuse myself for the strong wording.

 |Note that I am only working with Groff's man macro package and do
 |understand that other macro packages may have greater demands on the
 |HTML generator.

I really want to excuse my strong wording.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [groff] groff as the basis for comprehensive documentation?

[Nate Bargmann]

* On 2018 21 Apr 09:56 -0500, John Gardner wrote:
> We just need a post-processor that can generate reusable, *stylable* HTML
> output for an endless and unpredictable range of both site layouts and
> portable devices. Intelligently-structured markup is the first step – most
> front-end developers these days are reusing existing stylesheets written in
> a variety of CSS precompilers. The only way to accommodate this
> all-pervasive cesspool of copy+pasta is *to focus on content and semantics,
> not its presentation*.

I like the ambition on display, John.  :-)

Beyond the things you've already outlined and the thing I've mentioned,
I forgot about things like Media-Wiki markup and now the use of Markdown
for Wiki markup (that reads like we're going one way to go the other!)

I've done project documentation in each of those mentioned and had to
convert the Media-Wiki stuff to SourceForge.net's dialect of Wiki
Markdown when they dumped the use of Media-Wiki.  Then earlier this year
SF.net became wholly unreliable so I moved our stuff over to the GitHub
wiki and had to do some editing to make it right there again.

If I were emperor of the universe I would put a moratorium on document
formats!

But why do we focus on presentation when authoring a document?  I think
that in large part we have trained ourselves to do so using the various
desktop word processing/publishing programs that came down the pike.
I've done plenty of documents where the content was subservient to the
presentation (newsletters and such).  A word processor forces a
presentation first mindset because its only real output is the printed
page.  It's not a GUI thing, I was spending as much time tweaking the
presentation in the codes editor in Word Perfect 5.1 on MS-DOS in 1990
as I was writing lab reports.

Working in the presentation mindset for so long makes it difficult to
think in terms of semantic markup.  I'm not sure I really understand it
fully.  Even working with the man macros, I still spend some amount of
time checking the presentation on the terminal and with PDF and HTML
output.

- Nate

-- 

"The optimist proclaims that we live in the best of all
possible worlds.  The pessimist fears this is true."

Web: http://www.n0nb.us  GPG key: D55A8819  GitHub: N0NB


signature.asc
Description: PGP signature

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

>
>
>
> *So you’re going to insert devtags to pass semantic info to the
> postprocessor?...Personally, I think you’re going to have better luck
> passing semantic hints through as you mentioned above.*


Sort of. Sticking to pre- and post-processors is really about separation of
concerns more than anything else. There're no assumptions about what macro
packages an author's using, which preprocessors they're using, or even what
fonts they use. As long as Kernighan's output language is what we're
dealing with... game on.

*Any chance you could use an MUA that has a consistent quoting style and
> ties In-Reply-To, References, and the bits being quoted together? Otherwise
> threading for those of us using it suffers.*


Shit, I'm so sorry. =( I'm writing from Gmail (in Chrome), and applying
formatting manually. This  is what I'm
looking at right now...

I keep forgetting how this must look to users of other mail clients... :(


On 22 April 2018 at 01:06, Larry Kollar  wrote:

> Some of this is really cool, and ties in with a couple things I’ve tried
> in the past.
>
> John Gardner  wrote:
>
> > *1. Handling semantics*
> > We all know you can't draw semantics from cold, low-level formatting
> > commands. But for certain contexts - hierarchically sorted documents,
> > consistently indented code-samples and tables marked as tables, I believe
> > (okay, *hoping)* it's possible to reconstruct meaning from... well, stuff
> > that looks like this:
> >
> > n12000 0 V84000 H72000
> > x X devtag:.NH 1
> > x font 36 TB
> > f36s10950V84000H72000
> >
> > How? See the x X devtag line? That's what inspired this whole landslide
> of
> > absurd ambition. I wondered what we could do if more metadata were
> provided
> > that way – as device-specific control strings from, say, a preprocessor.
>
> So you’re going to insert devtags to pass semantic info to the
> postprocessor?
> Cool idea. I wrote a script called “htbl” some years back to go with
> grothml; it
> turns a subset of tbl markup into HTML tables. I never thought of using
> devtags
> to mark rows/cells like that; it might have worked better.
>
> > ...
> >
> > We know the widths and heights of each mounted device-font, their
> > kerning-pairs, ligatures, and lord knows what else. We milk this for all
> > it's worth: by plotting each glyph's bounding box in a scaled space
> > representing the output medium, we identify the most obvious constructs
> > first.
>
> That’s pretty similar to the PDF-to-markup thing I blithered about earlier.
> I think a more skilled programmer than myself (I’m a jumped up tech writer)
> could really make it work well… although as i said before, each document
> is unique. Personally, I think you’re going to have better luck passing
> semantic
> hints through as you mentioned above. But it does sound like fun! I hope
> you
> keep us posted.
>
> Larry
>
>
>

Re: [groff] groff as the basis for comprehensive documentation?

[Ralph Corderoy]

Hi John,

> >
> >

> * *

Any chance you could use an MUA that has a consistent quoting style and
ties In-Reply-To, References, and the bits being quoted together?
Otherwise threading for those of us using it suffers.
I realise it may mean two or three short replies when one summary would
do, but most of the time the extra noise is at an accepted level.

> Somedays I'll write `e-mail`, and other days it just looks like I'm
> subtracting the value of the `mail` variable

Teach your spelling checker right from wrong?  :-)

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

>
> *I, for one, do not expect an HTML rendered version of *anything* to be a
> faithful representation of a printed page.*


Barring physical factors like paper size, overprint, bleed and other
print-specific stuff, it's actually possible – thanks to CSS. It can
declaratively target 3 very different output mediums
:
screen (browsers, tablets, phones, projectors, etc), print, and speech (the
last medium is particularly crippled by poorly structured markup...) There
were originally others, but they were axed from the spec largely due to the
shift in the technology landscape. Having tv, projection, and handheld as
distinct media types made less sense as smartphones took off and browsers
went mobile. Others like braille and embossed were narrow attempts at
accessibility - WHATWG realised that disabilities and assistive
technologies were too complex to be addressed by CSS alone, so it was
deferred to WAI-ARIA instead...

On 22 April 2018 at 00:54, John Gardner  wrote:

> None of those, and certainly not an em dash. It's `man page', short for
>> `manual pages'.
>
>
> Haha, don't worry! =) I was being facetious. Probably could've
> communicated the joke better with `man\D'l 99n 0'pages` instead.
>
> In all seriousness, I can be horribly inconsistent with hyphenation in
> general, and I drive myself nuts. Somedays I'll write `e-mail`, and other
> days it just looks like I'm subtracting the value of the `mail` variable
> from whatever value is bound to `e` =(
>
> *I suppose it depends on what one expects from the generated HTML.*
>
>
> I should probably explain my other motive for developing these
> post-processors. Schooling Ingo on HTML earlier has probably left me
> looking way more critical and judgemental of people's markup than I really
> am. It's arrogance and ignorance (and too much holding-my-tongue) that
> caused me the long-winded bitchslap I gave before.
>
> As I mentioned earlier, I recently refactored Node.js's manpage to use
> mdoc(7) macros instead of man(7). My motive was to make formatting errors
> harder for the majority of contributors, most of whom are unfamiliar with
> roff or its macros. I was checking Node.js's repository to see if a patch
> for OpenBSD had landed, and saw a recent commit
> 
> titled *"doc: fix manpage warnings"*.  (Keep in mind this is the *only*
> Roff file in a sea of markdown files. Heck, the project's *canonical
> source* of documentation are markdown files split, parsed, and fed
> through a variety of interlocked JavaScript modules to produce parsable
> output suitable for their site's API pages
> .
>
> An attempt  was made to do the
> same for generating manpages, but the contributor couldn't find the time to
> finish the PR. I'd assumed it may have had something to do with lack of
> Roff-knowledge, rather than the fact he was using Node.js's JSON-formatted
> docs to generate the manpages. After all, anything in JSON format is
> *surely* done with the intent of making content portable and
> not-at-all-format specific, right? Right? I *OH GOD, IS THAT RAW HTML
> IN JSON THAT WAS GENERATED FROM MARKDOWN?! *
>
> Yes. Yes it is. *And the entire web industry can't stop doing this shit.*
>
> The web industry's obsession with markdown is going too far. Honestly, I
> get the appeal of markdown being contributor-friendly, but when you've got
> 30-odd dependencies to generate JSON, static HTML, PDF and
> god-knows-what-else, and you're *still* tasked with checking a manpage to
> see if its options are "in sync with the rest of the documentation", you
> can't help but wonder where the hell everybody went wrong.
>
> Troff is a powerful, horribly under-appreciated system. If people actually
> bothered to write manpages by hand, they'd be amazed by what the language
> and its associated programs can really do. But today's generation of devs
> have zero incentive to even give it a try. Those who are impelled to learn
> any sort of programmatic document preparation will default to LaTeX,
> period. Even if compiling LaTeX documents is as pleasant as pulling teeth,
> and even if LaTeX formats are binary and completely opaque to the casual
> observer.
>
> *The only sane response to insanity is more insanity, *and I'm probably
> crazy enough to believe more devs might give Roff a crack if they knew it
> was easier to generate markdown and HTML from Roff, rather than the other
> way around. Trouble is, grohtml's HTML output is, uh, somewhat archaic in
> the age when HTML5 is now considered an umbrella family of specifications
> , rather than a language revision.
> Now, I'm not holding this against anybody - when grohtml was 

Re: [groff] groff as the basis for comprehensive documentation?

[Larry Kollar]

Some of this is really cool, and ties in with a couple things I’ve tried in the 
past.

John Gardner  wrote:

> *1. Handling semantics*
> We all know you can't draw semantics from cold, low-level formatting
> commands. But for certain contexts - hierarchically sorted documents,
> consistently indented code-samples and tables marked as tables, I believe
> (okay, *hoping)* it's possible to reconstruct meaning from... well, stuff
> that looks like this:
> 
> n12000 0 V84000 H72000
> x X devtag:.NH 1
> x font 36 TB
> f36s10950V84000H72000
> 
> How? See the x X devtag line? That's what inspired this whole landslide of
> absurd ambition. I wondered what we could do if more metadata were provided
> that way – as device-specific control strings from, say, a preprocessor.

So you’re going to insert devtags to pass semantic info to the postprocessor?
Cool idea. I wrote a script called “htbl” some years back to go with grothml; it
turns a subset of tbl markup into HTML tables. I never thought of using devtags
to mark rows/cells like that; it might have worked better.

> ...
> 
> We know the widths and heights of each mounted device-font, their
> kerning-pairs, ligatures, and lord knows what else. We milk this for all
> it's worth: by plotting each glyph's bounding box in a scaled space
> representing the output medium, we identify the most obvious constructs
> first.

That’s pretty similar to the PDF-to-markup thing I blithered about earlier.
I think a more skilled programmer than myself (I’m a jumped up tech writer)
could really make it work well… although as i said before, each document
is unique. Personally, I think you’re going to have better luck passing semantic
hints through as you mentioned above. But it does sound like fun! I hope you
keep us posted.

Larry

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

>
> None of those, and certainly not an em dash. It's `man page', short for
> `manual pages'.


Haha, don't worry! =) I was being facetious. Probably could've communicated
the joke better with `man\D'l 99n 0'pages` instead.

In all seriousness, I can be horribly inconsistent with hyphenation in
general, and I drive myself nuts. Somedays I'll write `e-mail`, and other
days it just looks like I'm subtracting the value of the `mail` variable
from whatever value is bound to `e` =(

*I suppose it depends on what one expects from the generated HTML.*


I should probably explain my other motive for developing these
post-processors. Schooling Ingo on HTML earlier has probably left me
looking way more critical and judgemental of people's markup than I really
am. It's arrogance and ignorance (and too much holding-my-tongue) that
caused me the long-winded bitchslap I gave before.

As I mentioned earlier, I recently refactored Node.js's manpage to use
mdoc(7) macros instead of man(7). My motive was to make formatting errors
harder for the majority of contributors, most of whom are unfamiliar with
roff or its macros. I was checking Node.js's repository to see if a patch
for OpenBSD had landed, and saw a recent commit

titled *"doc: fix manpage warnings"*.  (Keep in mind this is the *only*
Roff file in a sea of markdown files. Heck, the project's *canonical source* of
documentation are markdown files split, parsed, and fed through a variety
of interlocked JavaScript modules to produce parsable output suitable for
their site's API pages .

An attempt  was made to do the
same for generating manpages, but the contributor couldn't find the time to
finish the PR. I'd assumed it may have had something to do with lack of
Roff-knowledge, rather than the fact he was using Node.js's JSON-formatted
docs to generate the manpages. After all, anything in JSON format is
*surely* done with the intent of making content portable and
not-at-all-format specific, right? Right? I *OH GOD, IS THAT RAW HTML
IN JSON THAT WAS GENERATED FROM MARKDOWN?! *

Yes. Yes it is. *And the entire web industry can't stop doing this shit.*

The web industry's obsession with markdown is going too far. Honestly, I
get the appeal of markdown being contributor-friendly, but when you've got
30-odd dependencies to generate JSON, static HTML, PDF and
god-knows-what-else, and you're *still* tasked with checking a manpage to
see if its options are "in sync with the rest of the documentation", you
can't help but wonder where the hell everybody went wrong.

Troff is a powerful, horribly under-appreciated system. If people actually
bothered to write manpages by hand, they'd be amazed by what the language
and its associated programs can really do. But today's generation of devs
have zero incentive to even give it a try. Those who are impelled to learn
any sort of programmatic document preparation will default to LaTeX,
period. Even if compiling LaTeX documents is as pleasant as pulling teeth,
and even if LaTeX formats are binary and completely opaque to the casual
observer.

*The only sane response to insanity is more insanity, *and I'm probably
crazy enough to believe more devs might give Roff a crack if they knew it
was easier to generate markdown and HTML from Roff, rather than the other
way around. Trouble is, grohtml's HTML output is, uh, somewhat archaic in
the age when HTML5 is now considered an umbrella family of specifications
, rather than a language revision. Now,
I'm not holding this against anybody - when grohtml was devised, it was...
what, early 2001? 2002? CSS support was spotty and inconsistent, and the W3
kept claiming XML/XHTML was the way of the future. Zero progress was made
for about 7 years, and I honestly can't blame Groff for using very outdated
layout techniques. Basically, anything laid out using HTML tables clearly
wasn't written within the last decade...

We just need a post-processor that can generate reusable, *stylable* HTML
output for an endless and unpredictable range of both site layouts and
portable devices. Intelligently-structured markup is the first step – most
front-end developers these days are reusing existing stylesheets written in
a variety of CSS precompilers. The only way to accommodate this
all-pervasive cesspool of copy+pasta is *to focus on content and semantics,
not its presentation*.



On 21 April 2018 at 23:25, Ralph Corderoy  wrote:

> Hi John,
>
> > Manpages, man-pages, or man\(empages?
>
> None of those, and certainly not an em dash.
> It's `man page', short for `manual pages'.
>
> $ dict -ms regexp 'man.*page'
> jargon:  "man page"
> foldoc:  "demand paged"  "man page"  "unix man page"
>   "unix manual page"
>   

Re: [groff] groff as the basis for comprehensive documentation?

[Larry Kollar]

John Gardner  wrote:

> It's ironic how this convention of unconventional formatting has stuck
> around since the beginning, but in over 40 years of writing manpages,
> nobody's been able to agree on a consistent way of hyphenating the damn
> word.
> 
> Manpages, man-pages, or man\(empages?

I’ve always made it a compound word, “manpages.” And I’m in violent agreement
with your and Brandon’s assessment of all-caps headings. Shoot, there aren’t 
many
terminal programs that *can’t* do bold these days; I think even that VT220 I 
keep
planning to connect to my iMac can do it.

Given a choice, I never all-caps my manpage headings. I’ve succumbed to 
convention
once or twice, maybe. It would be cool if man(1) could recognize an all-caps 
string
flush left, and turn it into bolded title case.

Larry

Re: [groff] groff as the basis for comprehensive documentation?

[Ralph Corderoy]

Hi John,

> Manpages, man-pages, or man\(empages?

None of those, and certainly not an em dash.
It's `man page', short for `manual pages'.

$ dict -ms regexp 'man.*page'
jargon:  "man page"
foldoc:  "demand paged"  "man page"  "unix man page"
  "unix manual page"
$

It might get hyphenated as a compound adjective, `the man-page macros',
or in a filename where as space isn't wanted, `man-pages.7'.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [groff] groff as the basis for comprehensive documentation?

[Nate Bargmann]

* On 2018 19 Apr 18:47 -0500, James K. Lowden wrote:
> On Fri, 20 Apr 2018 01:44:06 +1000
> John Gardner  wrote:
> 
> > > You might like to believe that eqn, tbl, and pic could be processed
> > > with grohtml
> > 
> > I've seen grohtml's complexity and was bewildered.  Hence why I
> > intend to write my own. The procedures for inferring structural or
> > semantic metadata from low-level intermediate output commands will be
> > an entertaining challenge. =)
> 
> For lack of a better term, I think it's an abstraction mismatch.  The
> ditroff language presupposes a dot-addressable canvas, onto which lines
> and strings of text are drawn.  That model fits most printers (these
> days) and terminals.  But it doesn't describe HTML at all.  
> 
> I discussed HTML output with Ted Faber, of grap, upon a time.  He
> produces HTML from a handful of his own macros.  ISTM ms macros map
> onto HTML pretty well:
> 
>   .SH => 
>   .PP => 
>   .I  => 
>   .B  => 
> 
> and so on.  But what, for example, is HTML to do with line
> justification, and why should the browser honor the (implied) line
> breaks, when it has its own line-wrapping logic and style sheet, and
> the page size is dynamic?

I suppose it depends on what one expects from the generated HTML.  As
one who reads pages more than writes them, I've been impressed with the
presentation on the man-pages project Web site (hosted at kernel.org).
For example, here is the rendering of groff_man(7):

http://man7.org/linux/man-pages/man7/groff_man.7.html

I couldn't find the generator being used in the Git repository and a lot
of it may be done with CSS.  The text is rendered using the  tag so
it looks much like tty output though it is not fully justified yet the
text blocks are indented.  Aside from the justification, the rest looks
very familiar to me.  Links to other hosted man pages are present.

I, for one, do not expect an HTML rendered version of *anything* to
be a faithful representation of a printed page.  The output of
"groff -man -Thtml ..." is reasonable for my purposes, except that I'd
like to be able to generate relative links in a page to other pages in
my project.  Maybe I just need to do some sed(1) transformation after
groff is done.

Note that I am only working with Groff's man macro package and do
understand that other macro packages may have greater demands on the
HTML generator.

- Nate

-- 

"The optimist proclaims that we live in the best of all
possible worlds.  The pessimist fears this is true."

Web: http://www.n0nb.us  GPG key: D55A8819  GitHub: N0NB


signature.asc
Description: PGP signature

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

*> The section heading examples in man-pages(7) are in all upper case.*

It's ironic how this convention of unconventional formatting has stuck
around since the beginning, but in over 40 years of writing manpages,
nobody's been able to agree on a consistent way of hyphenating the damn
word.

Manpages, man-pages, or man\(empages?

On 21 April 2018 at 22:16, Nate Bargmann  wrote:

> * On 2018 21 Apr 02:07 -0500, G. Branden Robinson wrote:
> > In my opinion, which I am far too young and poorly-connected to have
> > proffered when it would have made any difference, the
> > forced-full-capitalization of section titles in man page sources is an
> > information-destroying transform done in the wrong place at the wrong
> > time.  Section headings should be capitalized as section titles normally
> > are in technical documentation: either like work titles, or first-letter
> > only, with the normal rules for proper nouns and adjectives respected.
>
> To be effective, I think one needs to get Michael Kerrisk of the
> man-pages project https://www.kernel.org/doc/man-pages/ on board.  The
> section heading examples in man-pages(7) are in all upper case.  I've
> been using this page and others of the project along with groff_man(7)
> as a guide for formatting my own pages.
>
> > This has been itching me for many years; thanks for the excuse to air
> > my grievance.  ;-)
>
> Glad to start the thread that has led a number of different directions!
> :-D
>
> - Nate
>
> --
>
> "The optimist proclaims that we live in the best of all
> possible worlds.  The pessimist fears this is true."
>
> Web: http://www.n0nb.us  GPG key: D55A8819  GitHub: N0NB
>

Re: [groff] groff as the basis for comprehensive documentation?

[G. Branden Robinson]

At 2018-04-20T23:19:44+0200, Ingo Schwarze wrote:
> >> man://mandoc.1#EXIT_STATUS
> 
> > Now, as for the SHOUTY SHOUTY...
> 
> That's not a matter of SHOUTING, but of case sensitivity.
> The name of that standard section in man(7) and mdoc(7)
> is "EXIT STATUS", not "Exit Status" nor "Exit status"
> nor "exit status".  Case is preserved, consider:

> That's a bad idea.  I admit that many authors use unusual and even
> inconsistent casing in section headers (even in the very mandoc.1)-:,
> which may sometimes seem awkward.  But in technical documentation,
> casing is often deliberate, and automatically changing it based on
> natural language rules is prone to make information incorrect in
> some cases.

I disagree with most of this analysis.  As far as I can tell this was a
presentational decision, similar to the one that led to the Unix
trademark being shown in small caps.  I don't recall the reference but
the reason was not because Unix was supposed to be in full caps--it's
not an acronym, after all--but just to show off a fancy font on the
typesetter.

In my opinion, which I am far too young and poorly-connected to have
proffered when it would have made any difference, the
forced-full-capitalization of section titles in man page sources is an
information-destroying transform done in the wrong place at the wrong
time.  Section headings should be capitalized as section titles normally
are in technical documentation: either like work titles, or first-letter
only, with the normal rules for proper nouns and adjectives respected.

It would be better if man-db (or similar) set a *roff variable
that the macro package would check to see if case transformation on
section headings was desired.  The default, for the next n years, of
course, would be to go ahead and do the transformation to avoid shocking
people.

This has been itching me for many years; thanks for the excuse to air
my grievance.  ;-)

-- 
Regards,
Branden


signature.asc
Description: PGP signature

Re: [groff] groff as the basis for comprehensive documentation?

[G. Branden Robinson]

At 2018-04-17T00:05:59+1000, John Gardner wrote:
> I'm referring to a select choice of words that just happens to neatly fall
> against the 72-character limit... =) Here's the commit message I was
> referring to:
> 
> Like man(7), mdoc(7) is a macro package for marking up computer manuals.
> The main difference is that mdoc is semantic rather than presentational,
> providing authors with a full DSL to abstract page markup from low-level
> Roff commands. By contrast, `man` is minimalist and leaves formatting to
> the author, who is expected to have a working amount of Roff knowledge.
> 
> Therefore the use of `mdoc` for marking up Node's manpage is a decidedly
> better choice than bare `man`. Less room is left for error and mandoc(1)
> offers very robust error-checking and linting.
> 
> I've been writing every commit-message like this for years and I got too
> good at it, now I look completely mental... :-\

Years ago, we Debian people noticed that our comrade Joey Hess bore a
talent for doing this in ordinary email list traffic.  Hasty research
was done and evidently the term for this is "bricktext".  Some people
have a natural talent for it.  I am not one of them.  :)

-- 
Regards,
Branden


signature.asc
Description: PGP signature

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

Apologies for the out-of-sync response: Ingo's e-mail was binned as spam by
Gmail. Apologies if it sounded like I was ignoring you. =) (I was wondering
where Ralph drew this from:


> *The name of that standard section in man(7) and mdoc(7) is "EXIT**STATUS",
> not "Exit Status" nor "Exit status" nor "exit status".*


Now then. This is gonna be great.


> *Oops, you are rolling your own CSS from scratch.*


Ingo, I've spent the last 13 years in front-end web development, and I've
been writing standards-compliant websites for almost a decade. You are in
absolutely *no* position to be lecturing me on CSS, especially not when
your own is a pigsty.

*I see absolutely nothing semantic in there, it looks like a
> purely presentational style sheet to me.*


... yes, that's the entire reason CSS exists: to separate presentation from
content (the latter being tantamount with "semantics" as understood by web
authors and those of us who actively follow modern web standards). Whatever
your interpretation of "semantic" is, it's firmly-entrenched in some
mdoc-centric little realm far removed from the web platform.

*Are you aware of this semantic style sheet for manual pages:*


The stylesheet is atrocious, and only "semantic" insofar as direct
likenesses to mdoc markup is concerned. Probably looks adequate to systems
programmers, but to a web developer, it more resembles the auto-generated
slop Microsoft Office generated in the old days when saving Word documents
as HTML.

*With HTML code containing the correct attributes*


Correct? You've managed to get *everything* wrong. HTML and CSS is *NOT* mdoc,
yet you write it as such and then brag frequently about mandoc's "superior
HTML output". I've had enough.

*Shortlist of what mandoc fucks up with HTML:*

   1.
*Redundant title attributes on everything. *Actually, worse than redundant:
   it screws with assistive technologies like screen-readers, which might read
   the contents of a tag to the user using the title attribute if one is
   present. If you want to attach page or application-specific metadata to
   elements, use data-*
   
   instead.


   -
*Presentational tags used instead of those conveying text-level
semantics: *You're
   literally doing what mdoc(7) tells you not to do, except in HTML form:
  - -b, -S, -o:
  Flags/options should be represented using kbd tags, as they describe user
  input .
  - =*option*:
  Parameters should use var tags to indicate a placeholder name for an
  expectant value
  - Use dfn to markup the defining subject's name. For mdoc, this means
  *Nm*
   - *Inconsistent or incorrect use of sectioning elements*
   You linked to https://man.openbsd.org/gcc.1 as an example. CTRL+F and
   search for "Options Controlling the Kind of Output".
   I'd hotlink the section directly, but you neglected to use an ID
   attribute or even an anchor element with a name attribute. Did you mean to
   use all those separate  tags as an indication of quality output, or was
   that an oversight?
   - *Pointless empty elements everywhere*
   - *Class attributes assigned to elements which should be styled using
   SIMPLE stylesheet declarations*

Also, half of your "semantic stylesheet "
is redundant and repeating default properties. Many values aren't actually
doing anything, and several rulesets are empty altogether.

I can't go on. I'm feeling queasy with fremdschämen. Seriously.


On 21 April 2018 at 07:19, Ingo Schwarze  wrote:

> Hi John,
>
> John Gardner wrote on Sat, Apr 21, 2018 at 06:21:33AM +1000:
> > Ingo Schwarze wrote:
>
> >> and blanks in fragment names replaced by underscores rather than
> >> hyphens, for example:
>
> > The underscores look really jarring...
> > what's the argument against using dashes instead?
>
>$ man -k Sh,Ss=- | wc -l
> 43
>$ man -k Sh,Ss=_ | wc -l
> 11
>
> Dashes are much more common in normal English text (which section
> and subsection headings usually consist of).  If you see a hyphen
> there, you expect that it represents a hyphen, right?  Besides,
> i regard the underscore as the ASCII printable character most
> visually similar to the blank as it draws nothing *inside* the
> box, but just at the edge.
>
> But really, it's no big deal, i could have gone with the hyphen
> back then, but nobody said they would prefer it when the decision
> was made.  I'm merely pointing out there is an opportunity here to
> consciously choose to be compatible, or to not be compatible...  :)
>
> >> man://mandoc.1#EXIT_STATUS
>
> > Now, as for the SHOUTY SHOUTY...
>
> That's not a matter of SHOUTING, but of case sensitivity.
> The name of that standard section in man(7) and mdoc(7)
> is "EXIT STATUS", not "Exit Status" nor "Exit status"
> nor "exit status".  Case is preserved, consider:
>
>  

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

First, leave performance expectations at the door. The ambitious experiment
I describe below is intended to provide airtight handling for a conversion
medium which is inherently lossy (Roff -> HTML/SVG/CSS/et al, Markdown, and
Markdown with GitHub-flavoured options).


*1. Handling semantics*
We all know you can't draw semantics from cold, low-level formatting
commands. But for certain contexts - hierarchically sorted documents,
consistently indented code-samples and tables marked as tables, I believe
(okay, *hoping)* it's possible to reconstruct meaning from... well, stuff
that looks like this:

n12000 0 V84000 H72000
x X devtag:.NH 1
x font 36 TB
f36s10950V84000H72000


How? See the x X devtag line? That's what inspired this whole landslide of
absurd ambition. I wondered what we could do if more metadata were provided
that way – as device-specific control strings from, say, a preprocessor.

I intend to have a complementary preprocessor (probably named infer)
perform preliminary scans in the document pipeline to unintrusively tag
regions of particular interest. "Particular interest" here refers mainly to
preprocessors like tbl, eqn and pic which generate output that's mangled
beyond recognition.

It also refers to tracking any macro packages like mdoc(7) which *may* carry
semantic meaning with their command-set. Bear in mind these are really just
hints it's dropping for the post-processor phase: it certainly doesn't
attempt to go any further than recognising unparsed requests and macro
calls. It's not trying to be a genius. It's just annotating context for
more reliable interpretation.

Now, about that...

*2. We're gonna abuse metrics as a cloudy way to predict what the reader is
supposed to see*
We know the widths and heights of each mounted device-font, their
kerning-pairs, ligatures, and lord knows what else. We milk this for all
it's worth: by plotting each glyph's bounding box in a scaled space
representing the output medium, we identify the most obvious constructs
first.

This is actually where it becomes impossible to continue explaining without
illustrations or diagrams, and the whole process I'm envisioning is very
indirect, and influenced by numerous assumptions about output.

Now, this might turn out and be fantastic. Or it might be a flop. One way
or the other, I'm gonna have a hell of a lot of fun seeing how far I can
get and whether it's possible.

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

>
> *Hmm, that must be new in CSS (i stopped at CSS2).*


Do you mean attribute selectors?

these[ones-like$="this?"] { }


They've actually enjoyed universal support for quite some time now... =)
They were included in the first revision of the CSS2 specification, IIRC.

*But that has nothing to do with roff, let alone groff, right?*


It does, but not directly. What I intend to write is a
*post-processor. *Something
which will accept the following:

$ troff -Tutf8 /path/to/manpage.1 -Z | webroff

$ troff -T pdf /some/book.me -Z | w3conv


I haven't decided on what to name the actual executable (you see two
different names above). You'll also notice I'm piping output intended for
-Tutf8 and -T pdf into a foreign postprocessor. That's the point where I
divert the stream of intermediate output commands (which I've nicknamed
"DITROFF DATA" for slang, because damn that mouthful).

Unfortunately, this means it won't support mandoc because it lacks a
similar intermedia output language  (I *did* email Ingo some time ago about
a stable AST format I could work with... he probably thought I was nuts
after I said I was interested in writing my own HTML processor

*I referred to for example mdoc(7)'s .Sx command.*


Yes, that will be possible in the sense of a conventional HTML anchor. To
use a frivolous example:

Yeah John, go back to that thing


Hrm, I'm pretty sure I can hear the thoughts of somebody on this list
reading this email...

*"Hah! Good luck building your magical, semantic-detection from
> pixel-drawing commands, kid!"*


Brace yourselves for the gory details of how I'm gonna have a crack at
this...

On 21 April 2018 at 09:04, Steffen Nurpmeso <stef...@sdaoden.eu> wrote:

> John Gardner <gardnerjo...@gmail.com> wrote:
>  |Every instance of the "SHOUTED" headings can be uppercased too, even when
>  |used outside their role as a heading.
>  |
>  |The CSS to achieve this:
>  |
>  |a[href="#name"],
>  |a[href="#description"],
>  |a[href="#authors"] {
>  |
>  |text-transform: uppercase;
>  |}
>  |
>  |Will typecast any link pointing to  in majuscule
> "NAME".
>  |It's all CSS. =)
>
> Hmm, that must be new in CSS (i stopped at CSS2).
> But that has nothing to do with roff, let alone groff, right?
> I referred to for example mdoc(7)'s .Sx command.  And i think even
> Kristap's and Ingo's mandoc C parse tree will not automatically
> perform this adjustment (so that the tag for less(1) that mandoc
> can generate, somehow, is correct), but i have not verified that.
>
> --steffen
> |
> |Der Kragenbaer,The moon bear,
> |der holt sich munter   he cheerfully and one by one
> |einen nach dem anderen runter  wa.ks himself off
> |(By Robert Gernhardt)
>
>
> ------ Forwarded message --
> From: John Gardner <gardnerjo...@gmail.com>
> To: groff <groff@gnu.org>
> Cc:
> Bcc:
> Date: Sat, 21 Apr 2018 08:39:36 +1000
> Subject: Re: [groff] groff as the basis for comprehensive documentation?
> Every instance of the "SHOUTED" headings can be uppercased too, even when
> used outside their role as a heading.
>
> The CSS to achieve this:
>
> a[href="#name"],
> a[href="#description"],
> a[href="#authors"] {
>
> text-transform: uppercase;
> }
>
> Will typecast any link pointing to  in majuscule "NAME".
> It's all CSS. =)
>
> On 21 April 2018 at 08:29, Steffen Nurpmeso <stef...@sdaoden.eu> wrote:
>
> > Ralph Corderoy <ra...@inputplus.co.uk> wrote:
> >  |Ingo wrote:
> >  |> The name of that standard section in man(7) and mdoc(7) is "EXIT
> >  |> STATUS", not "Exit Status" nor "Exit status" nor "exit status".
> >  |
> >  |The shouting section heading makes it easier to find that heading
> rather
> >  |than the same word occurring elsewhere, e.g. `ENVIRONMENT'.
> >
> > Alternatively you have active links and an index and can jump to
> > whatever section or anchor there is.  For print-outs normal case
> > would be much nicer.
> >
> >  |And if the .SH's parameter isn't shouting then perhaps there's a reason
> >  |for it and it should be preserved, even if it just shows up the bug to
> >  |fix.
> >
> > Much nicer.
> >
> > --steffen
> > |
> > |Der Kragenbaer,The moon bear,
> > |der holt sich munter   he cheerfully and one by one
> > |einen nach dem anderen runter  wa.ks himself off
> > |(By Robert Gernhardt)
> >
> >
>
>
>

Re: [groff] groff as the basis for comprehensive documentation?

[Steffen Nurpmeso]

John Gardner  wrote:
 |Every instance of the "SHOUTED" headings can be uppercased too, even when
 |used outside their role as a heading.
 |
 |The CSS to achieve this:
 |
 |a[href="#name"],
 |a[href="#description"],
 |a[href="#authors"] {
 |
 |text-transform: uppercase;
 |}
 |
 |Will typecast any link pointing to  in majuscule "NAME".
 |It's all CSS. =)

Hmm, that must be new in CSS (i stopped at CSS2).
But that has nothing to do with roff, let alone groff, right?
I referred to for example mdoc(7)'s .Sx command.  And i think even
Kristap's and Ingo's mandoc C parse tree will not automatically
perform this adjustment (so that the tag for less(1) that mandoc
can generate, somehow, is correct), but i have not verified that.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
--- Begin Message ---
Every instance of the "SHOUTED" headings can be uppercased too, even when
used outside their role as a heading.

The CSS to achieve this:

a[href="#name"],
a[href="#description"],
a[href="#authors"] {

text-transform: uppercase;
}

Will typecast any link pointing to  in majuscule "NAME".
It's all CSS. =)

On 21 April 2018 at 08:29, Steffen Nurpmeso  wrote:

> Ralph Corderoy  wrote:
>  |Ingo wrote:
>  |> The name of that standard section in man(7) and mdoc(7) is "EXIT
>  |> STATUS", not "Exit Status" nor "Exit status" nor "exit status".
>  |
>  |The shouting section heading makes it easier to find that heading rather
>  |than the same word occurring elsewhere, e.g. `ENVIRONMENT'.
>
> Alternatively you have active links and an index and can jump to
> whatever section or anchor there is.  For print-outs normal case
> would be much nicer.
>
>  |And if the .SH's parameter isn't shouting then perhaps there's a reason
>  |for it and it should be preserved, even if it just shows up the bug to
>  |fix.
>
> Much nicer.
>
> --steffen
> |
> |Der Kragenbaer,The moon bear,
> |der holt sich munter   he cheerfully and one by one
> |einen nach dem anderen runter  wa.ks himself off
> |(By Robert Gernhardt)
>
>

--- End Message ---

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

Every instance of the "SHOUTED" headings can be uppercased too, even when
used outside their role as a heading.

The CSS to achieve this:

a[href="#name"],
a[href="#description"],
a[href="#authors"] {

text-transform: uppercase;
}

Will typecast any link pointing to  in majuscule "NAME".
It's all CSS. =)

On 21 April 2018 at 08:29, Steffen Nurpmeso  wrote:

> Ralph Corderoy  wrote:
>  |Ingo wrote:
>  |> The name of that standard section in man(7) and mdoc(7) is "EXIT
>  |> STATUS", not "Exit Status" nor "Exit status" nor "exit status".
>  |
>  |The shouting section heading makes it easier to find that heading rather
>  |than the same word occurring elsewhere, e.g. `ENVIRONMENT'.
>
> Alternatively you have active links and an index and can jump to
> whatever section or anchor there is.  For print-outs normal case
> would be much nicer.
>
>  |And if the .SH's parameter isn't shouting then perhaps there's a reason
>  |for it and it should be preserved, even if it just shows up the bug to
>  |fix.
>
> Much nicer.
>
> --steffen
> |
> |Der Kragenbaer,The moon bear,
> |der holt sich munter   he cheerfully and one by one
> |einen nach dem anderen runter  wa.ks himself off
> |(By Robert Gernhardt)
>
>

Re: [groff] groff as the basis for comprehensive documentation?

[Steffen Nurpmeso]

Ralph Corderoy  wrote:
 |Ingo wrote:
 |> The name of that standard section in man(7) and mdoc(7) is "EXIT
 |> STATUS", not "Exit Status" nor "Exit status" nor "exit status".
 |
 |The shouting section heading makes it easier to find that heading rather
 |than the same word occurring elsewhere, e.g. `ENVIRONMENT'.

Alternatively you have active links and an index and can jump to
whatever section or anchor there is.  For print-outs normal case
would be much nicer.

 |And if the .SH's parameter isn't shouting then perhaps there's a reason
 |for it and it should be preserved, even if it just shows up the bug to
 |fix.

Much nicer.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

>
> Begging your pardon ... who's pdfmark macros?
>

Ahaha, my bad. I recall well the credit Deri gave in pdf.tmac:


> *Much of the code in this macro has come from the excellent original work
> by*
> *Keith Marshall (see attribution in the pdfmark.tmac file). I, however,**am
> solely responsible for any bugs I may have introduced into this file*.


I apologise for my dimwittedness nonetheless...

@Ralph The shouting section heading makes it easier to find that heading
> rather
> than the same word occurring elsewhere, e.g. `ENVIRONMENT'.
> And if the .SH's parameter isn't shouting then perhaps there's a reason
> for it and it should be preserved, even if it just shows up the bug to
> fix.


They're still presented as such. CSS is used to stylise the capitalisation
so the heading assumes uppercase. The effect is a purely presentational one
to readers, but one that can have an adverse effect on screen-readers and
other accessibility software which interpret  ALLCAPS  as a list of letters
to read back to the user, one by one...


On 21 April 2018 at 07:43, Keith Marshall 
wrote:

> On 20/04/18 19:19, John Gardner wrote:
> > And as if that weren't enough, the renderer includes first-class
> > support for Deri Jame's pdfmark macros ...
>
> Begging your pardon ... who's pdfmark macros?
>

Re: [groff] groff as the basis for comprehensive documentation?

[Ralph Corderoy]

Hi,

Ingo wrote:
> The name of that standard section in man(7) and mdoc(7) is "EXIT
> STATUS", not "Exit Status" nor "Exit status" nor "exit status".

The shouting section heading makes it easier to find that heading rather
than the same word occurring elsewhere, e.g. `ENVIRONMENT'.
And if the .SH's parameter isn't shouting then perhaps there's a reason
for it and it should be preserved, even if it just shows up the bug to
fix.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [groff] groff as the basis for comprehensive documentation?

[Keith Marshall]

On 20/04/18 19:19, John Gardner wrote:
> And as if that weren't enough, the renderer includes first-class
> support for Deri Jame's pdfmark macros ...

Begging your pardon ... who's pdfmark macros?

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi John,

John Gardner wrote on Sat, Apr 21, 2018 at 06:21:33AM +1000:
> Ingo Schwarze wrote:

>> and blanks in fragment names replaced by underscores rather than
>> hyphens, for example:

> The underscores look really jarring...
> what's the argument against using dashes instead?

   $ man -k Sh,Ss=- | wc -l
43
   $ man -k Sh,Ss=_ | wc -l 
11

Dashes are much more common in normal English text (which section
and subsection headings usually consist of).  If you see a hyphen
there, you expect that it represents a hyphen, right?  Besides,
i regard the underscore as the ASCII printable character most
visually similar to the blank as it draws nothing *inside* the
box, but just at the edge.

But really, it's no big deal, i could have gone with the hyphen
back then, but nobody said they would prefer it when the decision
was made.  I'm merely pointing out there is an opportunity here to
consciously choose to be compatible, or to not be compatible...  :)

>> man://mandoc.1#EXIT_STATUS

> Now, as for the SHOUTY SHOUTY...

That's not a matter of SHOUTING, but of case sensitivity.
The name of that standard section in man(7) and mdoc(7)
is "EXIT STATUS", not "Exit Status" nor "Exit status"
nor "exit status".  Case is preserved, consider:

  https://man.openbsd.org/mandoc.1#EXIT_STATUS
  https://man.openbsd.org/mandoc.1#PAGER
  https://man.openbsd.org/mandoc.1#HTML_Output
  https://man.openbsd.org/mandoc.1#Output_Formats
  https://man.openbsd.org/mandoc.1#Syntax_tree_output
  https://man.openbsd.org/mandoc.1#Errors_related_to_tables
  https://man.openbsd.org/mandoc.1#error
  https://man.openbsd.org/mandoc.1#mdoc
  https://man.openbsd.org/mandoc.1#c
  https://man.openbsd.org/mandoc.1#K
  https://man.openbsd.org/ls.1#a
  https://man.openbsd.org/ls.1#A
  https://man.openbsd.org/ksh.1#pwd
  https://man.openbsd.org/ksh.1#PWD
  https://man.openbsd.org/ksh.1#[[
  https://man.openbsd.org/ksh.1##
  https://man.openbsd.org/ksh.1#-
  https://man.openbsd.org/ksh.1#_

> for HTML output, I'll be using correctly cased section headings,
> with the correct application of `text-transform: uppercase;`
> being applied by CSS.

That's a bad idea.  I admit that many authors use unusual and even
inconsistent casing in section headers (even in the very mandoc.1)-:,
which may sometimes seem awkward.  But in technical documentation,
casing is often deliberate, and automatically changing it based on
natural language rules is prone to make information incorrect in
some cases.

> In fact you can a start I made on a semantic HTML5 output example:
> https://rawgit.com/Alhadis/Stylesheets/master/complete/manpage/example.html#name

Oops, you are rolling your own CSS from scratch.

I see absolutely nothing semantic in there, it looks like a purely
presentational style sheet to me.

Are you aware of this semantic style sheet for manual pages:

  https://man.openbsd.org/mandoc.css
  http://mandoc.bsd.lv/cgi-bin/cvsweb/mandoc.css

It has matured a lot since Kristaps started it in Dec. 2008.  With
HTML code containing the correct attributes, it can be used with
both man(7) and mdoc(7) documents - of course it is much more
powerful with mdoc(7), though.  Man(7) HTML output can never be
very pretty (nor semantically rich) due to the limitations of the
language, but it more or less kind of works all the same:

  https://man.openbsd.org/gcc.1
  https://man.openbsd.org/perl.1
  https://man.openbsd.org/curses.3

Yours,
  Ingo

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

>
>
>
> *Unless you have strong reasons for the different syntax, pleaseconsider
> using the syntax established in the new man.cgi(8) a fewyears ago: *


> *  protocol://[manpath/][arch/]name[.sec][#fragment]*


Thank you for bringing this to me. =) Yes I most certainly will use this
syntax (didn't consider the possibility of including $MANPATH in the URI).

*and blanks in fragment names replaced by underscores rather than hyphens,
> for example:*

The underscores look really jarring... what's the argument against using
dashes instead? Slugs like "#camel-kebab-case" tend to be formatted that
way, for example...


man://mandoc.1#EXIT_STATUS


Now,  as for the SHOUTY SHOUTY... for HTML output, I'll be using correctly
cased section headings, with the correct application of `text-transform:
uppercase;` being applied by CSS. In fact you can a start I made on a
semantic HTML5  output example:

*https://rawgit.com/Alhadis/Stylesheets/master/complete/manpage/example.html#name
*

This will be generated by one of two projects I intend to start once this
is finished - postprocessors for purely semantic web technologies that
follow WAI-ARIA accessibility practices and uphold contemporary
web-authoring recommendations.

Note the minimalism in the code I've linked you to... So far, this is what
I've written for its stylesheet

:

On 21 April 2018 at 05:54, Ingo Schwarze  wrote:

> Hi John,
>
> John Gardner wrote on Sat, Apr 21, 2018 at 04:19:06AM +1000:
>
> > My Troff previewer will be doing just that for
> > man://mandoc/1/. =)
> > Will probably add support for subsection-linking with fragment
> > identifiers too:
> > man://mandoc/1/#exit-status
>
> Unless you have strong reasons for the different syntax, please
> consider using the syntax established in the new man.cgi(8) a few
> years ago:
>
>   protocol://[manpath/][arch/]name[.sec][#fragment]
>
> with all components case-sensitive and blanks in fragment names
> replaced by underscores rather than hyphens, for example:
>
>   man://mandoc.1#EXIT_STATUS
>   man://sparc64/lom.4
>
> I'm not saying either syntax is better - as a matter of fact, the
> differences are minimal, but avoiding gratuitious variations may
> benefit the overall ecosystem in the long term.
>
> The [manpath/] component can be used to identify operating systems
> and operating system releases; you may not need it in your context,
> to access local manual pages only.
>
> Note that i didn't invent a new syntax lightly, but there was no
> precedent to follow that i could find.  The old syntax of the
> classical man.cgi was a horrible thing involving
>   ?query=...=...=...
> and so on, so reusing it was not an acceptable option (though
> the new man.cgi still supports it for backward compatibility).
>
> Note that Debian mostly follows that syntax, too:
>
>   https://manpages.debian.org/stretch/mandoc/mandoc.1.en.html#HTML_Output
>
> Except for the .lang.html insertion.
> They are using [manpath/] for "release/package/",
> so that component is somewhat flexible depending on the context.
>
> Yours,
>   Ingo
>

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi John,

John Gardner wrote on Sat, Apr 21, 2018 at 04:19:06AM +1000:

> My Troff previewer will be doing just that for
> man://mandoc/1/. =)
> Will probably add support for subsection-linking with fragment
> identifiers too:
> man://mandoc/1/#exit-status

Unless you have strong reasons for the different syntax, please
consider using the syntax established in the new man.cgi(8) a few
years ago:

  protocol://[manpath/][arch/]name[.sec][#fragment]

with all components case-sensitive and blanks in fragment names
replaced by underscores rather than hyphens, for example:

  man://mandoc.1#EXIT_STATUS
  man://sparc64/lom.4

I'm not saying either syntax is better - as a matter of fact, the
differences are minimal, but avoiding gratuitious variations may
benefit the overall ecosystem in the long term.

The [manpath/] component can be used to identify operating systems
and operating system releases; you may not need it in your context,
to access local manual pages only.

Note that i didn't invent a new syntax lightly, but there was no
precedent to follow that i could find.  The old syntax of the
classical man.cgi was a horrible thing involving
  ?query=...=...=...
and so on, so reusing it was not an acceptable option (though
the new man.cgi still supports it for backward compatibility).

Note that Debian mostly follows that syntax, too:

  https://manpages.debian.org/stretch/mandoc/mandoc.1.en.html#HTML_Output

Except for the .lang.html insertion.
They are using [manpath/] for "release/package/",
so that component is somewhat flexible depending on the context.

Yours,
  Ingo

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

>
>
> *This is the thing I miss most about Konqueror: you could type a URI 
> like**“man:mdoc”
> and it would format and display the page*


There'll be a feature like that in Atom. The editor recently introduced a
feature where extension authors can register an external/custom protocol to
open links in Atom from browsers/emails. My Troff previewer will be doing
just that for man://mandoc/1/. =) Will probably add support for
subsection-linking with fragment identifiers too:
man://mandoc/1/#exit-status

I'll even made manpage refs like groff(1) hotlinks for navigating between
manpages in real-time. And as if that weren't enough, the renderer includes
first-class support for Deri Jame's pdfmark macros, enabling display and
traversal of PDF bookmarks in the viewport itself..

I've been so anxious to finish this and show everybody but I'm blocked on a
retarded issue of panning/zooming transformations that require a
math-empowered brain that's better than mine... :(

On 21 April 2018 at 04:09, Larry Kollar  wrote:

>
> Ingo Schwarze  wrote:
>
> > So yeah, even though proportional font is slowly becoming more
> > widely used, you may be right:  The legacy of Wolfram Schneider's
> > FreeBSD man.cgi is still pretty widespread and even motivated Michael
> > Stapelberg to use a fixed width font for Debian, even though the
> > rendering engine he uses would happily support proportional fonts.
>
> This is the thing I miss most about Konqueror: you could type a URI like
> “man:mdoc” and it would format and display the page. Whatever they were
> using for an algorithm, it worked better to display a manpage as HTML than
> anything else available at the time.
>
> When Apple first announced Safari, and its Webkit origins, I had high hopes
> they would carry that feature over. No such luck.
>
> Larry
>

Re: [groff] groff as the basis for comprehensive documentation?

[Larry Kollar]

Ingo Schwarze  wrote:

> So yeah, even though proportional font is slowly becoming more
> widely used, you may be right:  The legacy of Wolfram Schneider's
> FreeBSD man.cgi is still pretty widespread and even motivated Michael
> Stapelberg to use a fixed width font for Debian, even though the
> rendering engine he uses would happily support proportional fonts.

This is the thing I miss most about Konqueror: you could type a URI like
“man:mdoc” and it would format and display the page. Whatever they were
using for an algorithm, it worked better to display a manpage as HTML than
anything else available at the time.

When Apple first announced Safari, and its Webkit origins, I had high hopes
they would carry that feature over. No such luck.

Larry

Re: [groff] groff as the basis for comprehensive documentation?

[Larry Kollar]

John Gardner  wrote:

> It's easier than you think.You just have to separate presentational
> semantics from structural and content-related ones.

I’m fond of saying ‘All you have to do is…’ is one of the biggest lies ever 
told. ;-D

> I've seen grohtml's complexity and was bewildered.  Hence why I intend to
> write my own. The procedures for inferring structural or semantic metadata
> from low-level intermediate output commands will be an entertaining
> challenge. =)

Good luck! i’ve actually had some limited success using ps2ascii’s more complex
output modes to derive structure from font/size specifications and 
spacing/location,
converting a PDF file to some kind of markup. Of course, it’s very specific to
individual documents — it’s actually a collection of scripts, one of which 
returns
a list of fonts and sizes used in a document and the number of characters used
for each. i would use that to build a table, specifying whether strings with 
that
format were inline or block, and the kind of markup to wrap them in.

Paragraph detection is, um, fun. Some books use indents, others use vertical
space. And don’t get me started on definition lists or tables…

Larry

Re: [groff] groff as the basis for comprehensive documentation?

[Colin Watson]

On Thu, Apr 19, 2018 at 06:48:19PM +0200, Ingo Schwarze wrote:
> Colin Watson wrote on Thu, Apr 19, 2018 at 10:06:28AM +0100:
> > "man ./apropos.1", as Nate pointed out.  man-db's heuristic is that if
> > the page name contains a slash then it's surely a path name instead and
> > should be treated as such; I think that's a reasonable one.
> 
> Thank you for explaining the heuristic and for pointing out the
> missing feature in mandoc.  Given the existence of the -l option,
> having the heuristic is maybe not absolutely required, but i
> agree that it is not unreasonable, and we have seen that the absence
> of the heuristic can confuse casual users who are used to man-db.
> 
> So with the commit below, i added the same heuristic to mandoc.

Thanks.

> By the way, the old version of man-db in jessie exhibits a strange
> behaviour in that case, but probably that has been fixed long ago:
> 
>$ lsb_release -d
>   Description:Debian GNU/Linux 8.10 (jessie)
>$ dpkg-query -l man-db | tail -n 1
>   ii  man-db 2.7.0.2-5i386 on-line manual pager
>$ man --version
>   man 2.7.0.2
>$ man -w man ./man.1
>   man: man-./man.1: No such file or directory
>   man: man_./man.1: No such file or directory
>   /usr/share/man/man1/man.1.gz
>   ./man.1

This is still incorrect in current versions: man(1)'s command-line
parsing is not quite as elegantly well-factored as it ought to be ...  I
don't quite have time to sort it out just now, but I've filed
https://savannah.nongnu.org/bugs/index.php?53708 so that I don't forget
about it.  Thanks for the report!

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi Nate,

Nate Bargmann wrote on Thu, Apr 19, 2018 at 02:19:04PM -0500:
> * On 2018 19 Apr 13:13 -0500, Ingo Schwarze wrote:

>> But the -l option does exactly one thing that is easily described
>> in one short sentence that can hardly be misunderstood ("The name
>> arguments are interpreted as filenames").

> As it has been a few days since I first read mandoc's man(1) page (does
> that make sense?), thinking about it, it may be that the first sentence
> of -l's description is, "A synonym for mandoc(1) -a."  I honestly don't
> recall if I simply stopped reading there and went on to other options or
> closed that page and opened mandoc(1) to look at the -a option.

Ouch.

> As I have looked at it again, as the primary purpose of -l, as I now
> understand it, is found in the second and third sentences, then the
> first sentence should be moved after them at the very least or to the
> end of the description.

Not quite.

Look at the options -f, -k, and -l.  They all start with "a synonym
for", and that is important because these three options make man(1)
behave like other commands that were historically separate: whatis(1),
apropos(1), and mandoc(1).  Saying so up front is crucial because the
one or two sentences following each "synonym" statement merely give
an extremely superficial summary of the purpose of these commands.
To fully understand how to use them, you really need the dedicated
manual pages.  So "A synonym for mandoc(1)" must not be moved to the
end, it is the most important part of the description.

> When I look at mandoc(1), the -a option is described as the output being
> paginated with more if the output device is stdout.  As I read it, it is
> only concerned with output, not file input.

Right, -a means "show all" which traditionally implies two aspects:

 (1) Do not exit after the first matching page.
 (2) Really "show", i.e. with a pager, rather than the default output
 behaviour, which is, for example, printing to stdout for mandoc(1)
 and merely listing title lines for apropos(1).

Now for man(1), aspect 2, using a pager, is the default anyway,
while for mandoc(1), aspect 1, processing all files, is the default
anyway.  So even though -a always does the same, what needs to be
said about it in the man(1) and mandoc(1) manual pages sounds
substantially different if you are not aware of the big picture.

> I hope you can see where I got off track.  Perhaps rearranging the
> description of man(1)'s -l option will help others in the future.

I committed a fix to make the misunderstanding you suffered from
less likely, see the commit at the end of this mail.

Thanks for explaining where your confusion came from.  As a developer
knowing the software inside out, you indeed have a little chance to
find such issues.  The help of new users is required for this kind
of improvements.

Yours,
  Ingo


Log Message:
---
Tweak the description of -l:

Avoid the misunderstanding that the essential purpose of -l is 
similar to the purpose of the -a option in mandoc(1), which is not 
the point: the fact that -l implies -a is merely a minor detail. 
The point of -l is to make man(1) behave like mandoc(1).
Move the mention of -a to the end to de-emphasize it.

Nate Bargmann reported that this seriously confused him, 
and i can see why.

Modified Files:
--
mandoc:
man.1

Revision Data
-
Index: man.1
===
RCS file: /home/cvs/mandoc/mandoc/man.1,v
retrieving revision 1.32
retrieving revision 1.33
diff -Lman.1 -Lman.1 -u -p -r1.32 -r1.33
--- man.1
+++ man.1
@@ -114,8 +114,7 @@ manual.
 By default, it displays the header lines of all matching pages.
 .It Fl l
 A synonym for
-.Xr mandoc 1
-.Fl a .
+.Xr mandoc 1 .
 The
 .Ar name
 arguments are interpreted as filenames.
@@ -127,6 +126,8 @@ No search is done and
 and
 .Fl w
 are ignored.
+This option implies
+.Fl a .
 .It Fl M Ar path
 Override the list of standard directories which
 .Nm

Re: [groff] groff as the basis for comprehensive documentation?

[James K. Lowden]

On Fri, 20 Apr 2018 01:44:06 +1000
John Gardner  wrote:

> > You might like to believe that eqn, tbl, and pic could be processed
> > with grohtml
> 
> I've seen grohtml's complexity and was bewildered.  Hence why I
> intend to write my own. The procedures for inferring structural or
> semantic metadata from low-level intermediate output commands will be
> an entertaining challenge. =)

For lack of a better term, I think it's an abstraction mismatch.  The
ditroff language presupposes a dot-addressable canvas, onto which lines
and strings of text are drawn.  That model fits most printers (these
days) and terminals.  But it doesn't describe HTML at all.  

I discussed HTML output with Ted Faber, of grap, upon a time.  He
produces HTML from a handful of his own macros.  ISTM ms macros map
onto HTML pretty well:

.SH => 
.PP => 
.I  => 
.B  => 

and so on.  But what, for example, is HTML to do with line
justification, and why should the browser honor the (implied) line
breaks, when it has its own line-wrapping logic and style sheet, and
the page size is dynamic?

Similarly, it's fairly easy, up to a point, to imagine tbl generating
HTML tables directly, easier (for me, anyway) than imagining how to
infer the table structure after it's passed through groff.  But then
there are little niggles like  and HTML's table/cell borders.  


I've said before that it's a terrible, terrible shame HTML ever was
invented.  It didn't need to be, and the cost is surely measured in
billions of man-hours.  When it was invented, the roff language was
already 10 years old and in widespread use.  It could have been adapted
to the www through the simple expediency of writing a ditroff interface
to the browser, such that the browser accepted and rendered ditroff
output, as a postprocessor.  The few novel aspects of that HTML
introduced (relative to roff) -- hypherlinks, POST -- could have been
added to roff.  The escape mechanism was already there.  And, voila!
The thousands of roff documents already extant would have been
instantantly accessible, and the skill of using the language extended
to a new realm.  We wouldn't be talking about how to
convert/adapt/generate, because there'd be only one language.  

Yeah, I inow the history.  Yet, even now, there's no low-level
dot-addressable interface to the browser.  It has to *interpret*
everything: SVG, PNG, you name it.   Why use anything if you can
reinvent it?
.rant off

--jkl

Re: [groff] groff as the basis for comprehensive documentation?

[Nate Bargmann]

* On 2018 19 Apr 13:13 -0500, Ingo Schwarze wrote:
> But the -l option does exactly one thing that is easily described
> in one short sentence that can hardly be misunderstood ("The name
> arguments are interpreted as filenames").

As it has been a few days since I first read mandoc's man(1) page (does
that make sense?), thinking about it, it may be that the first sentence
of -l's description is, "A synonym for mandoc(1) -a."  I honestly don't
recall if I simply stopped reading there and went on to other options or
closed that page and opened mandoc(1) to look at the -a option.

As I have looked at it again, as the primary purpose of -l, as I now
understand it, is found in the second and third sentences, then the
first sentence should be moved after them at the very least or to the
end of the description.

When I look at mandoc(1), the -a option is described as the output being
paginated with more if the output device is stdout.  As I read it, it is
only concerned with output, not file input.

I hope you can see where I got off track.  Perhaps rearranging the
description of man(1)'s -l option will help others in the future.

Anyway,  I'm glad this bit of feedback gave you the inspiration to allow
direct processing of files with a path name.  This is the way Free
Software works best when we build off each other.

> While the main strength of groff(1) is no doubt high-quality
> typesetting, i guess nowadays, more people are using it to read
> manual pages than for typesetting work, and i think that most
> users are not even aware they are using it.

I think this is a very true statement.  It was probably about a year ago
that I really dug in and learned a bit about the role of groff in man
pages.  I had bumped into roff by editing some pages but was occupied
by other things at the time.  That is a bit of a poor track record for
someone using Linux since 1996!

I know that I was led back here by the failed promises of other "one
size fits all" documenting systems.  I now prefer the tried and true.

- Nate

-- 

"The optimist proclaims that we live in the best of all
possible worlds.  The pessimist fears this is true."

Web: http://www.n0nb.us  GPG key: D55A8819  GitHub: N0NB


signature.asc
Description: PGP signature

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi Nate,

Nate Bargmann wrote on Wed, Apr 18, 2018 at 08:28:29PM -0500:

> I now looked at mandoc's man(1) page again and the '-l' option is
> explained as you have used it here.  Somehow I missed that the other day
> and had tried '-m' and '-M' to no avail.  Then I managed to construct a
> directory local .db and could read my test file.
> 
> Perhaps an example would be a benefit.  I often check the EXAMPLE
> section of pages as I'm the sort that gets more out of working example
> than even a good explanation.

I agree that EXAMPLES can often be useful, but as anything else
in documentation, they are not a cure-all.  They are useful for:

 - complicated or non-obvious combinations of functionalities
   that are particularly useful but far separated in the reference
   description
 - supplementing the description of syntax or semantics that is
   already unusually complicated even when not combining it with
   anything else
 - showing recommended coding idioms, in particular when insecure
   or cumbersome usage is widespread

But the -l option does exactly one thing that is easily described
in one short sentence that can hardly be misunderstood ("The name
arguments are interpreted as filenames").  In practice, it is rarely
combined with other features: most other options and arguments lose
their effect when combined with -l.  In such a simple situation, i
would consider an example detrimental.  It would merely repeat the
description and harm conciseness.

> I'll admit to not being a 'man' power user.  I use it to quickly
> pull up a page

Don't worry; the man(1) command is intended to be easy to use for
people in a hurry, for people who think hard about something else,
and for novice users.  If man(1) requires expertise to be usable,
something has gone wrong.  That said, no doubt it also has features
catering to the needs of very experienced programmers, but there
is nothing wrong with not using those.

> and don't even think of 'apropos' or the other command line
> options.  Like a lot of us, I'm probably lazy in all the wrong
> areas.

That are many options an average user rarely needs, even though
i have been working hard to reduce the number of options of low
usefulness.  The apropos(1) command, however, is a very helpful
tool, and learning to use it provides great value to both
novice and experienced users.

> I know we're veering off topic for this list.

While the main strength of groff(1) is no doubt high-quality
typesetting, i guess nowadays, more people are using it to read
manual pages than for typesetting work, and i think that most
users are not even aware they are using it.  There is nothing
wrong with that, the man(1) command ought to be simple to use.
Keeping it so requires that developers working on groff remain
aware of the requirements for manual-page display.

For that reason, my impression is that most members of this list
probably regard discussions related to manual pages as on-topic,
even if not related to typesetting.

Yours,
  Ingo

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi James,

James K. Lowden wrote on Thu, Apr 19, 2018 at 10:45:49AM -0400:

> The roff language is the only markup language in current use that was
[...]
> As Hoare said of Algol, it is an improvement over its successors.  

Heh.  That's a good one!

> Cross references in mdoc, for example, do not generate links in HTML

They do with mandoc(1) -Thtml.

> or PDF documents.

Right, that looks like a missing feature in both groff and mandoc.

> It's pretty rare just to find manpages rendered in proportional HTML
> fonts.  

Reading this, my first thought was "that can hardly be true any
longer", and indeed:

  https://man.openbsd.org/roff.7
  http://jlk.fjfi.cvut.cz/arch/manpages/man/core/groff/roff.7.en
  https://linux.die.net/man/7/roff

Then again, admittedly,

  https://manpages.debian.org/stretch/groff/roff.7.en.html
  https://www.freebsd.org/cgi/man.cgi?query=roff
  http://netbsd.gw.com/cgi-bin/man-cgi?roff
  https://leaf.dragonflybsd.org/cgi/web-man?command=roff
  https://illumos.org/man/5/mandoc_roff
  http://man.minix3.org/cgi-bin/man.cgi?query=man=7
  http://man7.org/linux/man-pages/man7/roff.7.html

So yeah, even though proportional font is slowly becoming more
widely used, you may be right:  The legacy of Wolfram Schneider's
FreeBSD man.cgi is still pretty widespread and even motivated Michael
Stapelberg to use a fixed width font for Debian, even though the
rendering engine he uses would happily support proportional fonts.

Yours,
  Ingo

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi Colin, hi Nate,

Colin Watson wrote on Thu, Apr 19, 2018 at 10:06:28AM +0100:

> "man ./apropos.1", as Nate pointed out.  man-db's heuristic is that if
> the page name contains a slash then it's surely a path name instead and
> should be treated as such; I think that's a reasonable one.

Thank you for explaining the heuristic and for pointing out the
missing feature in mandoc.  Given the existence of the -l option,
having the heuristic is maybe not absolutely required, but i
agree that it is not unreasonable, and we have seen that the absence
of the heuristic can confuse casual users who are used to man-db.

So with the commit below, i added the same heuristic to mandoc.
Even mixing page names and file names on the command line now works:

   $ man -w man ./man.1 apropos ./apropos.1
  /usr/share/man/man1/apropos.1
  /usr/share/man/man1/man.1
  /usr/share/man/man7/man.7
  ./man.1
  ./apropos.1

By the way, the old version of man-db in jessie exhibits a strange
behaviour in that case, but probably that has been fixed long ago:

   $ lsb_release -d
  Description:Debian GNU/Linux 8.10 (jessie)
   $ dpkg-query -l man-db | tail -n 1
  ii  man-db 2.7.0.2-5i386 on-line manual pager
   $ man --version
  man 2.7.0.2
   $ man -w man ./man.1
  man: man-./man.1: No such file or directory
  man: man_./man.1: No such file or directory
  /usr/share/man/man1/man.1.gz
  ./man.1

Yours,
  Ingo


Log Message:
---
Compatibility with man-db: 
In page name mode (= man(1) default mode), also try to interpret 
names containing slashes as absolute or relative file names.

Missing feature reported by Nate Bargmann on ,
and the man-db maintainer Colin Watson  
kindly explained to me how this is supposed to work.

Modified Files:
--
mandoc:
main.c

Revision Data
-
Index: main.c
===
RCS file: /home/cvs/mandoc/mandoc/main.c,v
retrieving revision 1.304
retrieving revision 1.305
diff -Lmain.c -Lmain.c -u -p -r1.304 -r1.305
--- main.c
+++ main.c
@@ -132,7 +132,7 @@ main(int argc, char *argv[])
size_t   i, sz;
int  prio, best_prio;
enum outmode outmode;
-   int  fd;
+   int  fd, startdir;
int  show_usage;
int  options;
int  use_pager;
@@ -386,15 +386,34 @@ main(int argc, char *argv[])
argc, argv, , ))
usage(search.argmode);
 
-   if (sz == 0) {
-   if (search.argmode == ARG_NAME)
-   fs_search(, ,
-   argc, argv, , );
-   else
-   warnx("nothing appropriate");
+   if (sz == 0 && search.argmode == ARG_NAME)
+   fs_search(, ,
+   argc, argv, , );
+
+   if (search.argmode == ARG_NAME) {
+   for (c = 0; c < argc; c++) {
+   if (strchr(argv[c], '/') == NULL)
+   continue;
+   if (access(argv[c], R_OK) == -1) {
+   warn("%s", argv[c]);
+   continue;
+   }
+   res = mandoc_reallocarray(res,
+   sz + 1, sizeof(*res));
+   res[sz].file = mandoc_strdup(argv[c]);
+   res[sz].names = NULL;
+   res[sz].output = NULL;
+   res[sz].ipath = SIZE_MAX;
+   res[sz].bits = 0;
+   res[sz].sec = 10;
+   res[sz].form = FORM_SRC;
+   sz++;
+   }
}
 
if (sz == 0) {
+   if (search.argmode != ARG_NAME)
+   warnx("nothing appropriate");
rc = MANDOCLEVEL_BADARG;
goto out;
}
@@ -478,7 +497,29 @@ main(int argc, char *argv[])
parse(, STDIN_FILENO, "");
}
 
+   /*
+* Remember the original working directory, if possible.
+* This will be needed if some names on the command line
+* are page names and some are relative file names.
+* Do not error out if the current directory is not
+* readable: Maybe it won't be needed after all.
+*/
+   startdir = open(".", O_RDONLY | O_DIRECTORY);
+
while (argc > 0) {
+
+   /*
+* Changing directories is not needed in ARG_FILE mode.
+* Do it on a best-effort basis.  Even in case of
+* failure, some functionality may still work.
+ 

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

>
> *Groff is not the ideal system for generating HTML.*


It's easier than you think.You just have to separate presentational
semantics from structural and content-related ones.

Personally, I feel HTML generators should emit only semantic markup and
leave it to structure and external stylesheets to take care of the rest.


> You might like to believe that eqn, tbl, and pic could be processed with
> grohtml


I've seen grohtml's complexity and was bewildered.  Hence why I intend to
write my own. The procedures for inferring structural or semantic metadata
from low-level intermediate output commands will be an entertaining
challenge. =)


On 20 April 2018 at 00:45, James K. Lowden  wrote:

> On Mon, 16 Apr 2018 13:19:31 -0500
> Nate Bargmann  wrote:
>
> > I'm still undecided on the Texinfo part, though it may serve as the
> > portion that ties everything together.  I have man pages for utility
> > programs of the project and will be writing man pages for the C
> > library.  Being able to collate this nicely would be a great
> > benefit.
>
> I went down your very same road some years ago, except I used jade and
> SGML instead of XML for DocBook.  I found LaTex too confining and
> complex.  Once I bothered to learn mdoc, I wished I'd started there.
>
> The roff language is the only markup language in current use that was
>
> 1.  designed to be typed by humans, and
> 2.  designed to produce typeset documentation.
>
> I think there was hope, once upon a time, that a free implementation of
> something like Interleaf would become the UI for DocBook, and that mere
> mortals wouldn't have to balance their tags.  Needless to say, it never
> came to pass.  Lyx isn't it.
>
> The design of the roff language, while not "modern", is minimalistic;
> it has the least markup as a percentage of text.  It makes few
> assumptions about how the text should appear, and those assumption are
> well documented and easily adjusted.  The groff implementation is fast
> and small.  As Hoare said of Algol, it is an improvement over its
> successors.
>
> The full current capability of groff is harder to exploit than it could
> be, however.  There's still a bias toward printed output.  To create a
> document like Deri's, with hyperlinks, you have to understand the
> system pretty well, and piece together a few documents, some of which
> are incomplete.  Cross references in mdoc, for example, do not generate
> links in HTML or PDF documents.  It's possible to produce presentation
> slides, too, but you have to do a little digging.
>
> > Ideally, if the same sort of collation could be done with HTML, that
> > would be perfect.
>
> Groff is not the ideal system for generating HTML.  You might like to
> believe that eqn, tbl, and pic could be processed with grohtml and come
> out lovely on the other side, but that goal remains over the horizon.
> It's pretty rare just to find manpages rendered in proportional HTML
> fonts.
>
> --jkl
>
>

Re: [groff] groff as the basis for comprehensive documentation?

[James K. Lowden]

On Mon, 16 Apr 2018 13:19:31 -0500
Nate Bargmann  wrote:

> I'm still undecided on the Texinfo part, though it may serve as the
> portion that ties everything together.  I have man pages for utility
> programs of the project and will be writing man pages for the C
> library.  Being able to collate this nicely would be a great
> benefit.  

I went down your very same road some years ago, except I used jade and
SGML instead of XML for DocBook.  I found LaTex too confining and
complex.  Once I bothered to learn mdoc, I wished I'd started there.  

The roff language is the only markup language in current use that was

1.  designed to be typed by humans, and 
2.  designed to produce typeset documentation.  

I think there was hope, once upon a time, that a free implementation of
something like Interleaf would become the UI for DocBook, and that mere
mortals wouldn't have to balance their tags.  Needless to say, it never
came to pass.  Lyx isn't it.  

The design of the roff language, while not "modern", is minimalistic;
it has the least markup as a percentage of text.  It makes few
assumptions about how the text should appear, and those assumption are
well documented and easily adjusted.  The groff implementation is fast
and small.  As Hoare said of Algol, it is an improvement over its
successors.  

The full current capability of groff is harder to exploit than it could
be, however.  There's still a bias toward printed output.  To create a
document like Deri's, with hyperlinks, you have to understand the
system pretty well, and piece together a few documents, some of which
are incomplete.  Cross references in mdoc, for example, do not generate
links in HTML or PDF documents.  It's possible to produce presentation
slides, too, but you have to do a little digging.  

> Ideally, if the same sort of collation could be done with HTML, that
> would be perfect.

Groff is not the ideal system for generating HTML.  You might like to
believe that eqn, tbl, and pic could be processed with grohtml and come
out lovely on the other side, but that goal remains over the horizon.
It's pretty rare just to find manpages rendered in proportional HTML
fonts.  

--jkl

Re: [groff] groff as the basis for comprehensive documentation?

[Nate Bargmann]

* On 2018 19 Apr 02:56 -0500, Ralph Corderoy wrote:
> > I know we're veering off topic for this list.
> 
> You must be new here.

Yes.  :-D

I do need to spend some time in the archives.

- Nate

-- 

"The optimist proclaims that we live in the best of all
possible worlds.  The pessimist fears this is true."

Web: http://www.n0nb.us  GPG key: D55A8819  GitHub: N0NB


signature.asc
Description: PGP signature

Re: [groff] groff as the basis for comprehensive documentation?

[Colin Watson]

On Thu, Apr 19, 2018 at 02:28:36AM +0200, Ingo Schwarze wrote:
> Nate Bargmann wrote on Wed, Apr 18, 2018 at 06:52:44PM -0500:
> > I was disappointed that unlike "man" that I find on Slackware or
> > Debian, I had to add an uninstalled man page into the db in order
> > for "mandoc" to open it.  Perhaps I missed an option, but the "man"
> > I'm familiar with is capable of opening a file simply by giving it
> > a path name.
> 
> That is non-standard behaviour even on GNU/Linux:
> 
>$ lsb_release -d
>   Description:Debian GNU/Linux 8.10 (jessie)
>$ dpkg-query -l man-db | tail -n 1
>   ii  man-db 2.7.0.2-5i386 on-line manual pager
>$ ls *.1
>   apropos.1  demandoc.1  man.1  man.options.1  mandoc.1  soelim.1
>$ man apropos.1# the man-db man(1) implementation
>   No manual entry for apropos.1

"man ./apropos.1", as Nate pointed out.  man-db's heuristic is that if
the page name contains a slash then it's surely a path name instead and
should be treated as such; I think that's a reasonable one.

> I think it is good that the '-l' ("local") option is required when
> giving a file name in the local directory, or an absolute path.

I agree that man should not try to guess the difference between a page
name and an unadorned file name in the local directory (with no slash),
but when it's an absolute path or a relative path containing a slash,
the situation seems quite clear.

> Besides, with the mandoc implementation of man(1), it is not
> absolutely required to update the database even after installing a
> new page into /usr/share/man/.

You can also leave out the qualifier clause there: this is true of the
man-db implementation too.  (It used to not be true, but I fixed that
long ago, back in 2002 or so.)

-- 
Colin Watson   [cjwat...@debian.org]

Re: [groff] groff as the basis for comprehensive documentation?

[Ralph Corderoy]

> I know we're veering off topic for this list.

You must be new here.

Re: [groff] groff as the basis for comprehensive documentation?

[Nate Bargmann]

* On 2018 18 Apr 19:30 -0500, Ingo Schwarze wrote:
> Hi Nate,
> 
> Nate Bargmann wrote on Wed, Apr 18, 2018 at 06:52:44PM -0500:
> 
> > After reading a bit about mdoc,
> 
> I'm sure here you mean mandoc(1), the program, not mdoc(7), the
> markup language.

Yes, indeed.
> 
> > I was disappointed that unlike "man" that I find on Slackware or
> > Debian, I had to add an uninstalled man page into the db in order
> > for "mandoc" to open it.  Perhaps I missed an option, but the "man"
> > I'm familiar with is capable of opening a file simply by giving it
> > a path name.
> 
> That is non-standard behaviour even on GNU/Linux:
> 
>$ lsb_release -d
>   Description:Debian GNU/Linux 8.10 (jessie)
>$ dpkg-query -l man-db | tail -n 1
>   ii  man-db 2.7.0.2-5i386 on-line manual pager
>$ ls *.1
>   apropos.1  demandoc.1  man.1  man.options.1  mandoc.1  soelim.1
>$ man apropos.1# the man-db man(1) implementation
>   No manual entry for apropos.1
>$ ./Bin/man apropos.1  # the mandoc man(1) implementation
>   man: No entry for apropos.1 in the manual.
> 
> However:
> 
>$ man -l apropos.1 | grep 'search manual'
>  apropos, whatis - search manual page databases
>$ ./Bin/man -l apropos.1 | grep 'search manual'
>  apropos, whatis - search manual page databases

Well, I am doing a simple 'man ./foo.1' in the project directory and I
get the page displayed through less.  Been doing it that way for years
on Debian and now Slackware.

The default man in Slackware complains there is no option '-l' available
and gives me the usage screen.

> As a matter of fact, the inspiration for the -l option came from
> Linux, and the person who asked me to implement it in mandoc several
> years ago was an Alpine Linux developer.  Classical BSD man(1) did
> not have the -l option, but i agree that it is useful, and i use it
> many times every day.
>
> I think it is good that the '-l' ("local") option is required when
> giving a file name in the local directory, or an absolute path.
> The program has many different ways to specify which page to display:
> 
>  * by full page name as in man(1)
>  * by whole word search as in whatis(1) = man -f
>  * by substring or regexp search as in apropos(1) = man -k
>  * by file name as in man -l

I now looked at mandoc's man(1) page again and the '-l' option is
explained as you have used it here.  Somehow I missed that the other day
and had tried '-m' and '-M' to no avail.  Then I managed to construct a
directory local .db and could read my test file.

Perhaps an example would be a benefit.  I often check the EXAMPLE
section of pages as I'm the sort that gets more out of working example
than even a good explanation.

> With so many different semantics of a command line, guessing
> what the user meant will only cause confusion.

One thing I have learned, a user will try all sorts of permutations the
developer never considered!  A user will also read the manual very
differently than the developer or its author.  I try to keep these
points in mind as I work with the project.

> Besides, with the mandoc implementation of man(1), it is not
> absolutely required to update the database even after installing a
> new page into /usr/share/man/.  The man(1) program will find the
> new page at once even if it is not yet in the database - but of
> course, apropos(1) cannot find it, so always keeping the database
> up to date still makes sense.  A good package manager will do that
> for you, so it is no hassle.

Thanks, Ingo.

I'll admit to not being a 'man' power user.  I use it to quickly pull up
a page and don't even think of 'apropos' or the other command line
options.  Like a lot of us, I'm probably lazy in all the wrong areas.
;-)

I know we're veering off topic for this list.

- Nate

-- 

"The optimist proclaims that we live in the best of all
possible worlds.  The pessimist fears this is true."

Web: http://www.n0nb.us  GPG key: D55A8819  GitHub: N0NB


signature.asc
Description: PGP signature

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi Nate,

Nate Bargmann wrote on Wed, Apr 18, 2018 at 06:52:44PM -0500:

> After reading a bit about mdoc,

I'm sure here you mean mandoc(1), the program, not mdoc(7), the
markup language.

> I was disappointed that unlike "man" that I find on Slackware or
> Debian, I had to add an uninstalled man page into the db in order
> for "mandoc" to open it.  Perhaps I missed an option, but the "man"
> I'm familiar with is capable of opening a file simply by giving it
> a path name.

That is non-standard behaviour even on GNU/Linux:

   $ lsb_release -d
  Description:Debian GNU/Linux 8.10 (jessie)
   $ dpkg-query -l man-db | tail -n 1
  ii  man-db 2.7.0.2-5i386 on-line manual pager
   $ ls *.1
  apropos.1  demandoc.1  man.1  man.options.1  mandoc.1  soelim.1
   $ man apropos.1# the man-db man(1) implementation
  No manual entry for apropos.1
   $ ./Bin/man apropos.1  # the mandoc man(1) implementation
  man: No entry for apropos.1 in the manual.

However:

   $ man -l apropos.1 | grep 'search manual'
 apropos, whatis - search manual page databases
   $ ./Bin/man -l apropos.1 | grep 'search manual'
 apropos, whatis - search manual page databases

As a matter of fact, the inspiration for the -l option came from
Linux, and the person who asked me to implement it in mandoc several
years ago was an Alpine Linux developer.  Classical BSD man(1) did
not have the -l option, but i agree that it is useful, and i use it
many times every day.

I think it is good that the '-l' ("local") option is required when
giving a file name in the local directory, or an absolute path.
The program has many different ways to specify which page to display:

 * by full page name as in man(1)
 * by whole word search as in whatis(1) = man -f
 * by substring or regexp search as in apropos(1) = man -k
 * by file name as in man -l

With so many different semantics of a command line, guessing
what the user meant will only cause confusion.


Besides, with the mandoc implementation of man(1), it is not
absolutely required to update the database even after installing a
new page into /usr/share/man/.  The man(1) program will find the
new page at once even if it is not yet in the database - but of
course, apropos(1) cannot find it, so always keeping the database
up to date still makes sense.  A good package manager will do that
for you, so it is no hassle.

Yours,
  Ingo

Re: [groff] groff as the basis for comprehensive documentation?

[Nate Bargmann]

* On 2018 18 Apr 12:26 -0500, Larry Kollar wrote:
> At work, we’re in the first stages of moving our writers over to a
> DITA-based CMS.

Well, you forced me to look up DITA.  ;-)

> Ingo’s mandoc solution is a good way to produce text/HTML output, and you
> can use groff for PDF. The only thing I’d take issue with is his assertion 
> that
> -mdoc is easier to write than legacy -man. No doubt that -mdoc’s semantic
> markup adds value, but I’ve always had trouble getting my head around the
> syntax.

After reading a bit about mdoc, I was disappointed that unlike "man"
that I find on Slackware or Debian, I had to add an uninstalled man page
into the db in order for "mandoc" to open it.  Perhaps I missed an
option, but the "man" I'm familiar with is capable of opening a file
simply by giving it a path name.  Fortunately, the "man" currently in
Slackware does read an mdoc formatted page just fine, so mdoc formatted
pages are not on island.

Another thing I have to consider is the ability of others to submit
patches based on familiarity.  As arcane as it is, most contributors
seem to have bumped against classic man macros but I think jumping to
mdoc would leave me as the sole maintainer.  Consequently, the project
will use the man macros until more distributions make mdoc the default.
Note that I'm not arguing against it, just saying that right now it
doesn't appear to be the right tool for us.

Deri's effort is quite in line with what I think I am looking for.
Using the .texi files for an introduction and even starting each
reference chapter and then including the man pages to make up the
references without duplicating the work, would seem to be an ideal way
to meld the strengths of each document system.

With that in mind, I'll continue to work on the man pages.  Thanks for
the good discussion thus far.

- Nate

-- 

"The optimist proclaims that we live in the best of all
possible worlds.  The pessimist fears this is true."

Web: http://www.n0nb.us  GPG key: D55A8819  GitHub: N0NB


signature.asc
Description: PGP signature

Re: [groff] groff as the basis for comprehensive documentation?

[Larry Kollar]

Nate Bargmann  wrote:

> I have long been involved with a project that has lacked good
> documentation for nearly all of its existence.  We've had documentation,
> but it isn't in a good format for generating man, HTML, or PDF versions.
> 
> Long ago I started with Docbook and then that got to a point no one else
> would touch it and I didn't want to either.  XML was the "wave of the
> future" but I didn't jump on that wagon.  …

At work, we’re in the first stages of moving our writers over to a DITA-based 
CMS.
It’s horribly complex (and I should know, I’m in the vanguard), but does have a
pretty good way of producing both good PDF and decent HTML. I love when
people go “ooh, topic-based, shiny!” and I point out manpages have been around
for decades. :-D

BUT… if you want manpages, write manpages. I have not found anything yet
that does a good job of automatically creating manpages from any other kind
of format. DITA’s reference topic might be a reasonable basis for a conversion,
but writing that descriptive title is still beyond any AI. It has to look 
natural to a
human reader, while containing all the keywords needed to find it in a search
(thus being both content and metadata).

Ingo’s mandoc solution is a good way to produce text/HTML output, and you
can use groff for PDF. The only thing I’d take issue with is his assertion that
-mdoc is easier to write than legacy -man. No doubt that -mdoc’s semantic
markup adds value, but I’ve always had trouble getting my head around the
syntax.

Larry

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

That would be excellent, thank you! =) I've included first-class support
for your gropdf macros which send device-specific commands in the
post-processing phase ... still blocked on an issue involving
panning-and-zooming (if you zoom in and the top-two corners of a page
extend past the top-left corner of the screen... well, you can't just
scroll into -345cms looking for where the mangified content went... :(

On 17 April 2018 at 04:05, ZEN  wrote:

> I am afraid it is at home, I am in Cornwall at the moment, will send it to
> you next week.
>
>
>
> On 16 April 2018 18:57:22 BST, John Gardner 
> wrote:
>>
>> Deri, that book looks like a fantastic example or something I can show to
>> demonstrate the capability of my Troff Renderer (which is currently blocked
>> on more mathematical-related nonsense...Do you have the Roff source for
>> that book, perchance?
>>
>>
>> On 17 April 2018 at 03:33,  wrote:
>>
>>> On Monday, 16 April 2018 00:14:05 BST Nate Bargmann wrote:
>>> > Thanks, Ingo, for that very informative reply.
>>> >
>>> > I did just start reading the mdoc man page after sending that mail.
>>> > Thanks for the additional resources.  I shall check them out as I
>>> > continue on with this aspect of the project.
>>> >
>>> > - Nate
>>>
>>>
>>> i have been looking at merging the groff.texi file and various groff man
>>> pages
>>> into one compendium pdf. The program which does the merging is still
>>> rather
>>> beta, and probably would have problems with other texinfo files, but you
>>> can
>>> view an example of the current state at:-
>>>
>>> http://chuzzlewit.co.uk/groff_book.pdf
>>>
>>> Chapter 6 (Preprocessors) is largely man pages. The pdf is built using
>>> groff.
>>>
>>> Is this the sort of thing you were envisaging.
>>>
>>> I am currently on holiday so minimal email.
>>>
>>> Cheers
>>>
>>> Deri
>>>
>>>
>>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>

Re: [groff] groff as the basis for comprehensive documentation?

[Nate Bargmann]

* On 2018 16 Apr 12:37 -0500, d...@chuzzlewit.myzen.co.uk wrote:
> i have been looking at merging the groff.texi file and various groff man 
> pages 
> into one compendium pdf. The program which does the merging is still rather 
> beta, and probably would have problems with other texinfo files, but you can 
> view an example of the current state at:-
> 
> http://chuzzlewit.co.uk/groff_book.pdf

I have just a minute for a quick glance and it looks very well done.  I
will spend some more time looking through it later.

> Chapter 6 (Preprocessors) is largely man pages. The pdf is built using
> groff.

The man pages integrate quite well, IMO.

> Is this the sort of thing you were envisaging.

For the most part, yes.  I'm still undecided on the Texinfo part, though
it may serve as the portion that ties everything together.  I have man
pages for utility programs of the project and will be writing man pages
for the C library.  Being able to collate this nicely would be a great
benefit.  Ideally, if the same sort of collation could be done with
HTML, that would be perfect.

> I am currently on holiday so minimal email.

I'm not in a hurry.  :-)  I have many other things to do first.

- Nate

-- 

"The optimist proclaims that we live in the best of all
possible worlds.  The pessimist fears this is true."

Web: http://www.n0nb.us  GPG key: D55A8819  GitHub: N0NB


signature.asc
Description: PGP signature

Re: [groff] groff as the basis for comprehensive documentation?

[ZEN]

I am afraid it is at home, I am in Cornwall at the moment, will send it to you 
next week.


On 16 April 2018 18:57:22 BST, John Gardner  wrote:
>Deri, that book looks like a fantastic example or something I can show
>to
>demonstrate the capability of my Troff Renderer (which is currently
>blocked
>on more mathematical-related nonsense...Do you have the Roff source for
>that book, perchance?
>
>
>On 17 April 2018 at 03:33,  wrote:
>
>> On Monday, 16 April 2018 00:14:05 BST Nate Bargmann wrote:
>> > Thanks, Ingo, for that very informative reply.
>> >
>> > I did just start reading the mdoc man page after sending that mail.
>> > Thanks for the additional resources.  I shall check them out as I
>> > continue on with this aspect of the project.
>> >
>> > - Nate
>>
>>
>> i have been looking at merging the groff.texi file and various groff
>man
>> pages
>> into one compendium pdf. The program which does the merging is still
>> rather
>> beta, and probably would have problems with other texinfo files, but
>you
>> can
>> view an example of the current state at:-
>>
>> http://chuzzlewit.co.uk/groff_book.pdf
>>
>> Chapter 6 (Preprocessors) is largely man pages. The pdf is built
>using
>> groff.
>>
>> Is this the sort of thing you were envisaging.
>>
>> I am currently on holiday so minimal email.
>>
>> Cheers
>>
>> Deri
>>
>>

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

Deri, that book looks like a fantastic example or something I can show to
demonstrate the capability of my Troff Renderer (which is currently blocked
on more mathematical-related nonsense...Do you have the Roff source for
that book, perchance?


On 17 April 2018 at 03:33,  wrote:

> On Monday, 16 April 2018 00:14:05 BST Nate Bargmann wrote:
> > Thanks, Ingo, for that very informative reply.
> >
> > I did just start reading the mdoc man page after sending that mail.
> > Thanks for the additional resources.  I shall check them out as I
> > continue on with this aspect of the project.
> >
> > - Nate
>
>
> i have been looking at merging the groff.texi file and various groff man
> pages
> into one compendium pdf. The program which does the merging is still
> rather
> beta, and probably would have problems with other texinfo files, but you
> can
> view an example of the current state at:-
>
> http://chuzzlewit.co.uk/groff_book.pdf
>
> Chapter 6 (Preprocessors) is largely man pages. The pdf is built using
> groff.
>
> Is this the sort of thing you were envisaging.
>
> I am currently on holiday so minimal email.
>
> Cheers
>
> Deri
>
>

Re: [groff] groff as the basis for comprehensive documentation?

[deri]

On Monday, 16 April 2018 00:14:05 BST Nate Bargmann wrote:
> Thanks, Ingo, for that very informative reply.
> 
> I did just start reading the mdoc man page after sending that mail.
> Thanks for the additional resources.  I shall check them out as I
> continue on with this aspect of the project.
> 
> - Nate


i have been looking at merging the groff.texi file and various groff man pages 
into one compendium pdf. The program which does the merging is still rather 
beta, and probably would have problems with other texinfo files, but you can 
view an example of the current state at:-

http://chuzzlewit.co.uk/groff_book.pdf

Chapter 6 (Preprocessors) is largely man pages. The pdf is built using groff.

Is this the sort of thing you were envisaging.

I am currently on holiday so minimal email.

Cheers

Deri

Re: [groff] groff as the basis for comprehensive documentation?

[Ralph Corderoy]

Hi John,

> I'm not sure whether to stay quiet or point out that you may have
> misread me...

No, I understood.  I was pointing out something I do instead of normal
fmt(1) formatting too.

> Roff commands. By contrast, `man` is minimalist and leaves formatting to

s/\./& /  :-)

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

I'm not sure whether to stay quiet or point out that you may have misread
me...

I'm referring to a select choice of words that just happens to neatly fall
against the 72-character limit... =) Here's the commit message I was
referring to:

Like man(7), mdoc(7) is a macro package for marking up computer manuals.
The main difference is that mdoc is semantic rather than presentational,
providing authors with a full DSL to abstract page markup from low-level
Roff commands. By contrast, `man` is minimalist and leaves formatting to
the author, who is expected to have a working amount of Roff knowledge.

Therefore the use of `mdoc` for marking up Node's manpage is a decidedly
better choice than bare `man`. Less room is left for error and mandoc(1)
offers very robust error-checking and linting.


I've been writing every commit-message like this for years and I got too
good at it, now I look completely mental... :-\

On 16 April 2018 at 23:56, Ralph Corderoy  wrote:

> Hi John,
>
> > Does anybody else here manage to line-wrap their commit messages at
> > *precisely* 72-characters without the aide of hyphenation or
> justification?
> > ;-) Or is it just me?
>
> I find myself sometimes breaking a line after a comma or full stop,
> without starting a new paragraph, if the line is still a good length.
> Rather than vim's `gwap', or fmt(1), say.
>
> It's convenient for later edits being isolated in their ripple effect.
> I think Bell Labs book authors often did this when using ed(1) as the
> edits were line based and the edit often wanted to work on a clause.
> Does anyone here know a fmt(1)-er that tries to do this?
>
> fmt(1) alternatives I know are
>
> https://en.wikipedia.org/wiki/Par_(command)
> http://search.cpan.org/~neilb/Text-Autoformat-1.74/lib/Text/
> Autoformat.pm
>
> The latter can easily be run from the command line,
> not just used as a module in Perl.
>
> --
> Cheers, Ralph.
> https://plus.google.com/+RalphCorderoy
>
>

Re: [groff] groff as the basis for comprehensive documentation?

[Ralph Corderoy]

Hi John,

> Does anybody else here manage to line-wrap their commit messages at
> *precisely* 72-characters without the aide of hyphenation or justification?
> ;-) Or is it just me?

I find myself sometimes breaking a line after a comma or full stop,
without starting a new paragraph, if the line is still a good length.
Rather than vim's `gwap', or fmt(1), say.

It's convenient for later edits being isolated in their ripple effect.
I think Bell Labs book authors often did this when using ed(1) as the
edits were line based and the edit often wanted to work on a clause.
Does anyone here know a fmt(1)-er that tries to do this?

fmt(1) alternatives I know are

https://en.wikipedia.org/wiki/Par_(command)
http://search.cpan.org/~neilb/Text-Autoformat-1.74/lib/Text/Autoformat.pm

The latter can easily be run from the command line,
not just used as a module in Perl.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [groff] groff as the basis for comprehensive documentation?

[Ralph Corderoy]

Hi Ingo,

> > https://wiki.archlinux.org/index.php/Man_page#Online_man_pages
> > points to https://manned.org/
> > 
> > I guess you're referring to
> > https://jlk.fjfi.cvut.cz/arch/manpages/about that they use for wiki
> > links
...
> > Odd that they don't explicitly reference it AFAICS.
>
> I have no idea what's going on there.  :-)

I've fiddled with that bit of the wiki page.  We'll see if it remains in
place.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [groff] groff as the basis for comprehensive documentation?

[John Gardner]

Sidenote: I recently refactored Node.js's manpage
 to use mdoc macros, advising
the project maintainers to stick to to mdoc whenever possible. I linked to
OpenBSD's mdoc reference and explained that mandoc itself isn't a
full-featured roff interpreter, so roff syntax should be avoided.

I'm glad they took notice of my utterly-inhuman justification abilities:
https://github.com/nodejs/node/pull/18559#issuecomment-364778689

Does anybody else here manage to line-wrap their commit messages at
*precisely* 72-characters without the aide of hyphenation or justification?
;-) Or is it just me?

On 16 April 2018 at 22:52, Ingo Schwarze  wrote:

> Hi Ralph,
>
> Ralph Corderoy wrote on Mon, Apr 16, 2018 at 12:43:19PM +0100:
> > Ingo Schwarze wrote:
>
> >> Debian, Ubuntu, Arch, Gentoo, Slackware, Homebrew, MacPorts, and
> >> pkgsrc provide packages that you can install.
>
> > Arch Linux's user repository, AUR, has
> > https://aur.archlinux.org/packages/mandoc/ but it seems the maintainer
> > has decided to have it conflict with groff:
>
> That is bad.  Thanks for noticing!
>
> There is more than one reason why i use both all the time:
> mandoc cannot do real typesetting; even for manual pages,
> i often do routine output comparisons; some manual pages -
> about 0.25% in the wild - still don't work with mandoc and
> require groff; ...
>
> > `...this package now
> > outright conflicts with groff.  mandoc provides the functionality of
> > both groff and man-db combined; as such, I consider it a full
> > replacement'.
>
> Uh oh.  They got that wrong.  :/
>
> > I've added a comment pointing out that groff might remain
> > useful.  :-)
>
> Thanks, absolutely!
>
> >> https://www.bsdcan.org/2018/schedule/events/958.en.html
>
> > That says
> >
> >  Both Debian and Arch Linux now also use the mandoc formatter for
> >  their official online manuals.
> >
> > but I had trouble finding Arch's.
> > https://wiki.archlinux.org/index.php/Man_page#Online_man_pages points to
> > https://manned.org/
> >
> > I guess you're referring to https://jlk.fjfi.cvut.cz/arch/manpages/about
> > that they use for wiki links, e.g.
> > https://jlk.fjfi.cvut.cz/arch/manpages/man/utf-8.7
>
> Yes, that's what i'm linking to from below "Documentation and help /
> online manual page portals" on http://mandoc.bsd.lv/ .
>
> > Odd that they don't explicitly reference it AFAICS.
>
> I have no idea what's going on there.  :-)
>
> Yours,
>   Ingo
>
>

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi Ralph,

Ralph Corderoy wrote on Mon, Apr 16, 2018 at 12:43:19PM +0100:
> Ingo Schwarze wrote:

>> Debian, Ubuntu, Arch, Gentoo, Slackware, Homebrew, MacPorts, and
>> pkgsrc provide packages that you can install.

> Arch Linux's user repository, AUR, has
> https://aur.archlinux.org/packages/mandoc/ but it seems the maintainer
> has decided to have it conflict with groff:

That is bad.  Thanks for noticing!

There is more than one reason why i use both all the time:
mandoc cannot do real typesetting; even for manual pages,
i often do routine output comparisons; some manual pages -
about 0.25% in the wild - still don't work with mandoc and
require groff; ...

> `...this package now
> outright conflicts with groff.  mandoc provides the functionality of
> both groff and man-db combined; as such, I consider it a full
> replacement'.

Uh oh.  They got that wrong.  :/

> I've added a comment pointing out that groff might remain
> useful.  :-)

Thanks, absolutely!

>> https://www.bsdcan.org/2018/schedule/events/958.en.html
 
> That says
> 
>  Both Debian and Arch Linux now also use the mandoc formatter for
>  their official online manuals.
> 
> but I had trouble finding Arch's.
> https://wiki.archlinux.org/index.php/Man_page#Online_man_pages points to
> https://manned.org/
> 
> I guess you're referring to https://jlk.fjfi.cvut.cz/arch/manpages/about
> that they use for wiki links, e.g.
> https://jlk.fjfi.cvut.cz/arch/manpages/man/utf-8.7

Yes, that's what i'm linking to from below "Documentation and help /
online manual page portals" on http://mandoc.bsd.lv/ .

> Odd that they don't explicitly reference it AFAICS.

I have no idea what's going on there.  :-)

Yours,
  Ingo

Re: [groff] groff as the basis for comprehensive documentation?

[Ralph Corderoy]

Hi Ingo,

> Debian, Ubuntu, Arch, Gentoo, Slackware, Homebrew, MacPorts, and
> pkgsrc provide packages that you can install.

Arch Linux's user repository, AUR, has
https://aur.archlinux.org/packages/mandoc/ but it seems the maintainer
has decided to have it conflict with groff: `...this package now
outright conflicts with groff.  mandoc provides the functionality of
both groff and man-db combined; as such, I consider it a full
replacement'.  I've added a comment pointing out that groff might remain
useful.  :-)

> https://www.bsdcan.org/2018/schedule/events/958.en.html

That says

 Both Debian and Arch Linux now also use the mandoc formatter for
 their official online manuals.

but I had trouble finding Arch's.
https://wiki.archlinux.org/index.php/Man_page#Online_man_pages points to
https://manned.org/

I guess you're referring to https://jlk.fjfi.cvut.cz/arch/manpages/about
that they use for wiki links, e.g.
https://jlk.fjfi.cvut.cz/arch/manpages/man/utf-8.7
Odd that they don't explicitly reference it AFAICS.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [groff] groff as the basis for comprehensive documentation?

[Nate Bargmann]

Thanks, Ingo, for that very informative reply.

I did just start reading the mdoc man page after sending that mail.
Thanks for the additional resources.  I shall check them out as I
continue on with this aspect of the project.

- Nate

-- 

"The optimist proclaims that we live in the best of all
possible worlds.  The pessimist fears this is true."

Web: http://www.n0nb.us  GPG key: D55A8819  GitHub: N0NB


signature.asc
Description: PGP signature

Re: [groff] groff as the basis for comprehensive documentation?

[Ingo Schwarze]

Hi Nate,

Nate Bargmann wrote on Sun, Apr 15, 2018 at 04:17:59PM -0500:

> I have long been involved with a project that has lacked good
> documentation for nearly all of its existence.  We've had documentation,
> but it isn't in a good format for generating man, HTML, or PDF versions.
[...]
> After letting this sit for some time, I realized that I had not
> investigated groff deeply

That is really strange.  :)

It has always been the central format for research and commercial
UNIX, it has always been used by all the BSD projects (actually
using GNU troff for about two decades), and the Linux man pages
project also uses it, so i would consider it the canonical solution
for documentation in general.

> But (there is always a 'but'), beyond the collection of nroff files I am
> creating for the man pages, it would be nice to be able to tie them all
> together into a whole.

For rendering manual pages, consider using mandoc(1) rather than
groff.  While it is not a typesetting system and consequently has
*much* weaker PostScript and PDF output than groff, it is slightly
better for terminal output and **much** better for HTML output.
The input languages are exactly the same, it is merely a different
parser and formatter implementation strictly optimized for manual
pages.

Also, both the command line and CGI tools are *much* more powerful
than the combination of groff with man-db, or with any other man(1)
implementation that runs on Linux.

Specifically, for the question you are asking, make sure you have
the man(1) implementation from the mandoc toolkit installed, then
use

  $ man -aks 2 ~.

to view a combined document containing all the section 2 manual
pages on a terminal or

  $ man -T pdf -aks 2 ~. > tmp.pdf

to create a single PDF document containing them all, or

  https://man.openbsd.org/?query=~.=1=2

for an online HTML / CGI version with hyperlinks; off course, you
can also create a hyperlinked offline HTML version, for example
with the catman(8) tool contained in the mandoc toolkit, but the
online version is certainly more useful.

All that is available out of the box, without installing any
packages whatsoever, on OpenBSD, Alpine Linux, and Void Linux.

FreeBSD, NetBSD, and illumos also contain all the required binaries
by default without installing any package, but they install less
powerful implementations of man(1) by default, but you can easily
symlink mandoc to man to get the mandoc implementation of man(1).

Debian, Ubuntu, Arch, Gentoo, Slackware, Homebrew, MacPorts, and
pkgsrc provide packages that you can install.  Building from
source is also tested on AIX and Solaris 9-11.

> I was hoping the mom macro package might have some sort of a facility
> for this idea,

While mom is certainly a very considerable option for a wide 
range of non-manual-page documents, it cannot be used with
manual pages at all.  But there is no need, mandoc already
provides all you are asking for, and more - for example,
building a PDF document on demand containing the pages
returned by a specific semantic search query - no matter
whether the query returns a handful, dozens, or hundreds
of pages.

> Regardless, I plan to continue with improving our project's collection
> of man pages and work from there and will likely remove the Texinfo
> files at some point.

Sounds like a reasonable plan - http://mandoc.bsd.lv/texi2mdoc/
may or may not help with the transition, by the way.  It is not
very actively maintained these days (though i might fix bugs
in it if any are reported), and the output certainly needs manual
postprocessing, but using it is very likely to save you quite
some work compared to writing everything from scratch.

Of course, make sure to use the semantic mdoc(7) language, not the
old, purely presentational man(7) language, or you won't have
semantic searching, and most of the hyperlinking won't be present,
either.  Also, writing man(7) is much harder than writing mdoc(7).

For documentation, see

  http://mandoc.bsd.lv/

For overviews of the wide variety of topics that are being worked
on regarding roff(7) as documentation format, see

  https://www.openbsd.org/papers/eurobsdcon2015-mandoc.pdf
  https://www.bsdcan.org/2018/schedule/events/958.en.html

Yours,
  Ingo