Re: [groff] More on stripping

[Peter Schaffter]

Bertrand --

On Mon, Mar 19, 2018, Bertrand Garrigues wrote:
> The commit 69c5a52a05efc39388b893d5b4d7cff549b6ab18 breaks the
> generation of mom pdf examples when building out of source: we
> must now pass mom source dir to -M instead of mom build dir
> (however if you build directly in the sources you will not see the
> problem).
> 
> The patch below fixes the problem, could you please test on your side
> too?

Patch tested at this end.  Example files are building fine both
when building in source and out of source.

-- 
Peter Schaffter
http://www.schaffter.ca

Re: [groff] More on stripping

[Bertrand Garrigues]

Hi Peter,

On Fri, Mar 16 2018 at 01:05:17 PM, Peter Schaffter  wrote:
> Ingo --
>
> On Thu, Mar 15, 2018, Ingo Schwarze wrote:
>> After a fierce fight resulting in the death of the vicious
>> git-version-gen dragon, i finally managed to test,
>
> A good fight, St. George. :)
>
>> ...that your patch looks good to code inspection and survives
>> the tests listed below on OpenBSD-current if you add the following
>> minor details:
>> 
>>  * remove "/contrib/mom/om.tmac" from .gitignore
>>  * in mom.am, also remove "  contrib/mom/om.tmac-u" from EXTRA_DIST
>>(or make dist will blow up)
>>  * in mom.am, also remove "  contrib/mom/om.tmac" from MOSTLYCLEANFILES
>>(or people may come to grief after spring cleaning)
>
> Thanks for catching these.
>  
>> The complete patch that i believe you want to commit is appended
>> at the end for comparison.
>
> Applied, tested, and pushed.

The commit 69c5a52a05efc39388b893d5b4d7cff549b6ab18 breaks the
generation of mom pdf examples when building out of source: we must now
pass mom source dir to -M instead of mom build dir (however if you build
directly in the sources you will not see the problem).

The patch below fixes the problem, could you please test on your side
too?

I've also removed the line:

  nodist_momtmac_DATA = $(MOMNORMALFILES)

this no longer makes any sense as we say just below:

  dist_momtmac_DATA = $(MOMNORMALFILES)

Regards,

Bertrand


diff --git a/contrib/mom/mom.am b/contrib/mom/mom.am
index a4a74318..ef0f9cf4 100644
--- a/contrib/mom/mom.am
+++ b/contrib/mom/mom.am
@@ -21,10 +21,9 @@
 # along with this program.  If not, see .
 
 mom_srcdir = $(top_srcdir)/contrib/mom
-mom_builddir = $(top_builddir)/contrib/mom
 
 # pdfmom command used to generated .pdf
-MOM_TFLAG = -M$(mom_builddir)
+MOM_TFLAG = -M$(mom_srcdir)
 MOM_KFLAG = -k -p -e -t
 MOMPDFMOM = \
   GROFF_COMMAND_PREFIX= \
@@ -40,7 +39,6 @@ MOMNORMALFILES = \
   contrib/mom/mom.tmac \
   contrib/mom/om.tmac
 momtmacdir = $(tmacdir)
-nodist_momtmac_DATA = $(MOMNORMALFILES)
 dist_momtmac_DATA = $(MOMNORMALFILES)
 
 # Files installed in htmldocdir/mom

Re: [groff] More on stripping

[Peter Schaffter]

Ingo --

On Thu, Mar 15, 2018, Ingo Schwarze wrote:
> After a fierce fight resulting in the death of the vicious
> git-version-gen dragon, i finally managed to test,

A good fight, St. George. :)

> ...that your patch looks good to code inspection and survives
> the tests listed below on OpenBSD-current if you add the following
> minor details:
> 
>  * remove "/contrib/mom/om.tmac" from .gitignore
>  * in mom.am, also remove "  contrib/mom/om.tmac-u" from EXTRA_DIST
>(or make dist will blow up)
>  * in mom.am, also remove "  contrib/mom/om.tmac" from MOSTLYCLEANFILES
>(or people may come to grief after spring cleaning)

Thanks for catching these.
 
> The complete patch that i believe you want to commit is appended
> at the end for comparison.

Applied, tested, and pushed.

-- 
Peter Schaffter
http://www.schaffter.ca

Re: [groff] More on stripping

[Ingo Schwarze]

Hi Peter,

Peter Schaffter wrote on Wed, Mar 14, 2018 at 07:36:36PM -0400:

> If om.tmac-u is removed from the sources and replaced
> with om.tmac (unstripped), the attached patch should give us
> what we want.

After a fierce fight resulting in the death of the vicious
git-version-gen dragon, i finally managed to test, and the result
is that that your patch looks good to code inspection and survives
the tests listed below on OpenBSD-current if you add the following
minor details:

 * remove "/contrib/mom/om.tmac" from .gitignore
 * in mom.am, also remove "  contrib/mom/om.tmac-u" from EXTRA_DIST
   (or make dist will blow up)
 * in mom.am, also remove "  contrib/mom/om.tmac" from MOSTLYCLEANFILES
   (or people may come to grief after spring cleaning)

The complete patch that i believe you want to commit is appended at
the end for comparison.

Tests done:
 * ./bootstrap
 * mkdir build; cd build; ../configure
 * make
 * make dist
 * comparison of all build logs before and after; all changes look sane
 * build and install OpenBSD package
 * compare package build logs before and after; all changes look sane
 * OpenBSD groff test suite still works

You may also want to test in-tree make and make dist before commit.

Yours,
  Ingo


diff --git a/.gitignore b/.gitignore
index d58329c5..e702a37d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -33,7 +33,6 @@ stamp-*
 /chem
 /contrib/hdtbl/hdmisc.tmac
 /contrib/hdtbl/hdtbl.tmac
-/contrib/mom/om.tmac
 /defs.h
 /eqn
 /eqn2graph
diff --git a/contrib/mom/mom.am b/contrib/mom/mom.am
index 12fb5215..a4a74318 100644
--- a/contrib/mom/mom.am
+++ b/contrib/mom/mom.am
@@ -36,14 +36,11 @@ man7_MANS += contrib/mom/groff_mom.7
 
 # Files installed in $(tmacdir).
 # MOMNORMALFILES are located in the source tree.
-# om.tmac is generated in the build tree from the unstripped file om.tmac-u 
with
-#  strip.sed
 MOMNORMALFILES = \
-  contrib/mom/mom.tmac
-MOMSTRIPFILES = \
+  contrib/mom/mom.tmac \
   contrib/mom/om.tmac
 momtmacdir = $(tmacdir)
-nodist_momtmac_DATA = $(MOMSTRIPFILES)
+nodist_momtmac_DATA = $(MOMNORMALFILES)
 dist_momtmac_DATA = $(MOMNORMALFILES)
 
 # Files installed in htmldocdir/mom
@@ -121,12 +118,10 @@ EXTRA_DIST += $(MOMHTMLDOCFILES) $(MOMEXAMPLEFILES) \
   contrib/mom/NEWS \
   contrib/mom/TODO \
   contrib/mom/copyright \
-  contrib/mom/groff_mom.7.man \
-  contrib/mom/om.tmac-u
+  contrib/mom/groff_mom.7.man
 
 MOSTLYCLEANFILES += \
   $(MOMPROCESSEDEXAMPLEFILES) \
-  contrib/mom/om.tmac \
   penguin.ps \
   penguin.pdf
 
@@ -136,7 +131,7 @@ SUFFIXES += .mom .pdf
$(GROFF_V)$(MKDIR_P) `dirname $@` \
&& LC_ALL=C $(MOMPDFMOM) $< >$@
 
-$(MOMPROCESSEDEXAMPLEFILES): $(MOMSTRIPFILES) $(MOMNORMALFILES) \
+$(MOMPROCESSEDEXAMPLEFILES): $(MOMNORMALFILES) \
   groff troff gropdf pdfmom penguin.ps penguin.pdf \
   gnu.eps font/devpdf/build_font_files
 
@@ -145,11 +140,6 @@ penguin.ps:
 penguin.pdf:
cp $(mom_srcdir)/examples/penguin.pdf $@
 
-$(MOMSTRIPFILES): $(top_srcdir)/tmac/strip.sed
-   @$(MKDIR_P) `dirname $@`
-   LANG=C LC_ALL=C \
-   sed -f $(top_srcdir)/tmac/strip.sed $(mom_srcdir)/`basename $@`-u > $@;
-
 install-data-hook: install_mom
 install_mom:
 if BUILD_PDFEXAMPLES
diff --git a/contrib/mom/om.tmac-u b/contrib/mom/om.tmac
similarity index 100%
rename from contrib/mom/om.tmac-u
rename to contrib/mom/om.tmac

Re: [groff] More on stripping

[Peter Schaffter]

On Tue, Mar 13, 2018, Bjarni Ingi Gislason wrote:
> On Tue, Mar 13, 2018 at 01:27:28AM +, Bjarni Ingi Gislason wrote:
> >[...] 
> >   Define a variable, for example "UNSTRIPPED_TMAC_FILES=...om.tmac-u"
> > (in the file mom.am (?))
> > 
> >   Add that to the target "install:" in the Makefile... .
> > 
> >   Other additions are appropriate, for example for uninstalling.
> 
>   This is wrong.
> 
>   It seems to be enough to add "...om.tmac-u" to the variable
> "MOMNORMALFILES" in "mom.am" to get it installed.

This results in om.tmac and om.tmac-u being installed.  We want only
om.tmac (not stripped) installed.  If om.tmac-u is removed from the
sources and replaced with om.tmac (unstripped), the attached patch
should give us what we want.

On the subject of strip.sed (which we don't want used on om.tmac),
even with Bjarni's suggested patches, it still removes the \" that
protects trailing spaces from constructions like

  .ds foo \
  bar \"

>From my tests, the modified version, below, results in fully
stripped files that keep the protecting \"

---modified strip.sed---
2 i\
.\\" This is a generated file, created by 'tmac/strip.sed' in groff's\
.\\" source code bundle from a file having '-u' appended to its name.
# strip comments, spaces, etc., after a line containing '%beginstrip%'
/%beginstrip%/,$ {
  s/^\.[ ]*/./
  s/^\.\\".*/./
  s/^\\#.*/./
  s/\([  ]*\)\\"[]*.\+/\1\\"/
  s/\\"[ ]*.\+//
  s/\\#.*/\\/
}
/^\.$/d
---modified strip.sed---

-- 
Peter Schaffter
http://www.schaffter.ca
diff --git a/contrib/mom/mom.am b/contrib/mom/mom.am
index 12fb521..8a96bdd 100644
--- a/contrib/mom/mom.am
+++ b/contrib/mom/mom.am
@@ -36,14 +36,11 @@ man7_MANS += contrib/mom/groff_mom.7
 
 # Files installed in $(tmacdir).
 # MOMNORMALFILES are located in the source tree.
-# om.tmac is generated in the build tree from the unstripped file om.tmac-u with
-#  strip.sed
 MOMNORMALFILES = \
-  contrib/mom/mom.tmac
-MOMSTRIPFILES = \
+  contrib/mom/mom.tmac \
   contrib/mom/om.tmac
 momtmacdir = $(tmacdir)
-nodist_momtmac_DATA = $(MOMSTRIPFILES)
+nodist_momtmac_DATA = $(MOMNORMALFILES)
 dist_momtmac_DATA = $(MOMNORMALFILES)
 
 # Files installed in htmldocdir/mom
@@ -136,7 +133,7 @@ SUFFIXES += .mom .pdf
 	$(GROFF_V)$(MKDIR_P) `dirname $@` \
 	&& LC_ALL=C $(MOMPDFMOM) $< >$@
 
-$(MOMPROCESSEDEXAMPLEFILES): $(MOMSTRIPFILES) $(MOMNORMALFILES) \
+$(MOMPROCESSEDEXAMPLEFILES): $(MOMNORMALFILES) \
   groff troff gropdf pdfmom penguin.ps penguin.pdf \
   gnu.eps font/devpdf/build_font_files
 
@@ -145,11 +142,6 @@ penguin.ps:
 penguin.pdf:
 	cp $(mom_srcdir)/examples/penguin.pdf $@
 
-$(MOMSTRIPFILES): $(top_srcdir)/tmac/strip.sed
-	@$(MKDIR_P) `dirname $@`
-	LANG=C LC_ALL=C \
-	sed -f $(top_srcdir)/tmac/strip.sed $(mom_srcdir)/`basename $@`-u > $@;
-
 install-data-hook: install_mom
 install_mom:
 if BUILD_PDFEXAMPLES

Re: [groff] More on stripping

[Ingo Schwarze]

Hello Doug,

Doug McIlroy wrote on Sun, Mar 11, 2018 at 11:29:45PM -0400:

> So I put together a -ms test with no preprocessors.
> The input was about 1 MB, comprising 30K lines
> of which 8000 were macro calls. I padded every line of
> s.tmac that did not end in \ or \c with a 32-character
> comment. Thus macro expansions were laden with comments.
> 
> The variance in running times pretty well swamped
> the difference between -ms padded and unpadded.
> The difference was at most a few percent.
> 
> With Peter preferring to distribute -mom unstripped,
> it's hard to find an argument for default stripping.
> Maybe mandoc doesn't want to give it up.

Quite to the contrary.

groff_mom(7) = contrib/mom/om.tmac-u has more than twice the size
of groff_mdoc(7) = tmac/doc*-u, and groff_man(7) = tmac/an-old.tmac
isn't even stripped in the first place.  So i really wouldn't see
the point in stripping manual page macros if even the comprehensive
and carefully documented mom package no longer wants to strip.
If i remember (and re-read) past discussions correctly, mom was
the chief argument for stripping, e.g.
  https://lists.gnu.org/archive/html/groff/2017-11/msg00105.html

If the argument is that groff_mom(7) is typically only run rarely,
on larger documents, after some editing is done, such that speed
doesn't matter as much as for manual pages which are often rendered
in real time when people interactively type man(1), that doesn't sound
very convincing to me either.  After all, practically all the operating
systems having large numbers of mdoc(7) manual pages use mandoc(1)
rather than groff(1) inside man(1), while those predominantly using
man(7) already have no stripping benefit right now, which applies to
most Linux distributions and some commercial UNIXes like Solaris 11.
Besides, quoting from
  https://lists.gnu.org/archive/html/groff/2017-11/msg00070.html
  I already did measurements two years ago and found no
  examples of manual pages where the time economy exceeds 20%.

I'm not saying we absolutely *must* stop stripping, but i would
certainly still like it, and i'm quite conviced that manual pages are
not the primary use case that requires stripping of macro sets.
Honestly, not even if hell freezes over and Michael Kerrisk switches
the Linux man pages project to mdoc(7).

Yours,
  Ingo

Re: [groff] More on stripping

[Bjarni Ingi Gislason]

On Tue, Mar 13, 2018 at 01:27:28AM +, Bjarni Ingi Gislason wrote:
>[...] 
>   Define a variable, for example "UNSTRIPPED_TMAC_FILES=...om.tmac-u"
> (in the file mom.am (?))
> 
>   Add that to the target "install:" in the Makefile... .
> 
>   Other additions are appropriate, for example for uninstalling.

  This is wrong.

  It seems to be enough to add "...om.tmac-u" to the variable
"MOMNORMALFILES" in "mom.am" to get it installed.

  Suggested patch is in the bug-groff list, number 53340.

-- 
Bjarni I. Gislason

Re: [groff] More on stripping

[Peter Schaffter]

On Tue, Mar 13, 2018, Bjarni Ingi Gislason wrote:
>   There is no need to stop stripping.
> 
>   Define a variable, for example "UNSTRIPPED_TMAC_FILES=...om.tmac-u"
> (in the file mom.am (?))
> 
>   Add that to the target "install:" in the Makefile... .

Unless I'm misreading, you're either suggesting to install both the
stripped and unstripped versions, or just om.tmac-u, which would
have to be renamed to om.tmac in order for groff to find it.  I'm
not seeing the point.

-- 
Peter Schaffter
http://www.schaffter.ca

Re: [groff] More on stripping

[Bjarni Ingi Gislason]

On Sun, Mar 11, 2018 at 12:26:45PM -0400, Peter Schaffter wrote:
>[...]
> Doug's points about the candle not being worth the penny--the
> performance gain of stripped macro files is minimal--and that
> removing comments for the sake of efficiency passes the cost onto
> users, who generally have to acquire the commented/indented files
> separately for debugging and tweaking, are both valid.
> 
> I'll wait to see if anyone else weighs in on this.  If not, I'll
> rename om.tmac-u to om.tmac and remove the strip.sed bits from
> mom.am.
> 

  See article nr. 47 from 5th November 2017 in the groff list ("What
does the "-u" in ".tmac-u" mean?").

  People are so busy writing statements, that they do not pay attention
to the task ("problem"(?)).

  It is not the stripping, or have I missed that "central" point(?).

  It is not about how many percents of time (which is _very_ relative)
is saved using stripped versus unstripped.

  There is no need to stop stripping.

  Define a variable, for example "UNSTRIPPED_TMAC_FILES=...om.tmac-u"
(in the file mom.am (?))

  Add that to the target "install:" in the Makefile... .

  Other additions are appropriate, for example for uninstalling.

###

"You must unlearn what you have learned".  Yoda in "Star Wars.  The
Empire Strikes back".

#

The only known cure for ignorance is education

E. C. McKenzie "14,000 Quips & Quotes for Writers & Speakers".
Greenwich House, New York.  1984.

#

At the rate we're going, the day may come when everybody has a college
degree and nobody has an education.

E. C. McKenzie "14,000 Quips & Quotes for Writers & Speakers".
Greenwich House, New York.  1984.

#

Education means developing the mind, not stuffing the memory.

E. C. McKenzie "14,000 Quips & Quotes for Writers & Speakers".
Greenwich House, New York.  1984.

#

  People are that animal on Earth, that causes the most harm (damage).

  People are thus the most harmful animal, which makes them the most
dangerous one.

###

"People will rather die than think."

  Not quite right.  They want to live, but:

People will rather cause harm (including get _other_ people killed)
than think.

###

-- 
Bjarni I. Gislason

Re: [groff] More on stripping

[Damian McGuckin]

On Sun, 11 Mar 2018, Doug McIlroy wrote:

The variance in running times pretty well swamped the difference between 
-ms padded and unpadded. The difference was at most a few percent.


I tried some hypothetic similar tests and got similar results. Even on a 
low power, slowish (firewall) CPU. Mind you, lots faster than a PDP-11.


With Peter preferring to distribute -mom unstripped, it's hard to find 
an argument for default stripping.


I would concur.

Regards - Damian

Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037
Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here
Views & opinions here are mine and not those of any past or present employer

Re: [groff] More on stripping

[Doug McIlroy]

The previous discussion that Branden cited offered no
significant data about the value of stripping.
I had given one example of a complicated scientific
paper with lots of pic, eqn, and tbl, typset with
the relatively simple -ms package. Stripping
made essentially no difference in the overall
run time.

Perhaps a more complicated macro package, especially
one that places comments within macro definitions,
would feel a greater effect--even more so if
preprocessors were not involved.

So I put together a -ms test with no preprocessors.
The input was about 1 MB, comprising 30K lines
of which 8000 were macro calls. I padded every line of
s.tmac that did not end in \ or \c with a 32-character
comment. Thus macro expansions were laden with comments.

The variance in running times pretty well swamped
the difference between -ms padded and unpadded.
The difference was at most a few percent.

With Peter preferring to distribute -mom unstripped,
it's hard to find an argument for default stripping.
Maybe mandoc doesn't want to give it up. If so, let
it have it and stew in its own complexity, while
all the rest get freed from meddling.

Doug

Re: [groff] More on stripping

[Steve Izma]

On Sun, Mar 11, 2018 at 12:26:45PM -0400, Peter Schaffter wrote:
> Subject: Re: [groff] More on stripping
> 
> Doug's points about the candle not being worth the penny--the
> performance gain of stripped macro files is minimal--and that
> removing comments for the sake of efficiency passes the cost onto
> users, who generally have to acquire the commented/indented files
> separately for debugging and tweaking, are both valid.

I agree. Let's forget the stripping. I find groff lightning fast
regardless of the size of the tmac files. And knowing the
thinking behind the code is essential for progress.

-- Steve

-- 
Steve Izma
-
Home: 35 Locust St., Kitchener, Ontario, Canada  N2H 1W6
E-mail: si...@golden.net  phone: 519-745-1313  cell: 519-998-2684

Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible,
you are, by definition, not smart enough to debug it.
-- Brian Kernighan

Re: [groff] More on stripping

[Peter Schaffter]

On Sat, Mar 10, 2018, G. Branden Robinson wrote:
> At 2018-03-10T12:02:08-0500, Peter Schaffter wrote:
> > I'm still strongly in favour of not stripping om.tmac, so providing
> > a fix would be counterproductive. :)
> > 
> > Unless there is a compelling reason to strip om.tmac, I'd like to
> > remove it from mom.am.  Comments?
> 
> Here's are some links to the last time we talked about this (for some
> reason the GNU archiver split the thread in two).
> 
> https://lists.gnu.org/archive/html/groff/2017-11/msg00055.html
> https://lists.gnu.org/archive/html/groff/2017-11/msg00104.html

Doug's points about the candle not being worth the penny--the
performance gain of stripped macro files is minimal--and that
removing comments for the sake of efficiency passes the cost onto
users, who generally have to acquire the commented/indented files
separately for debugging and tweaking, are both valid.

I'll wait to see if anyone else weighs in on this.  If not, I'll
rename om.tmac-u to om.tmac and remove the strip.sed bits from
mom.am.

-- 
Peter Schaffter
http://www.schaffter.ca

Re: [groff] More on stripping

[G. Branden Robinson]

At 2018-03-10T12:02:08-0500, Peter Schaffter wrote:
> I'm still strongly in favour of not stripping om.tmac, so providing
> a fix would be counterproductive. :)
> 
> Unless there is a compelling reason to strip om.tmac, I'd like to
> remove it from mom.am.  Comments?

Here's are some links to the last time we talked about this (for some
reason the GNU archiver split the thread in two).

https://lists.gnu.org/archive/html/groff/2017-11/msg00055.html
https://lists.gnu.org/archive/html/groff/2017-11/msg00104.html

-- 
Regards,
Branden


signature.asc
Description: PGP signature

Re: [groff] More on stripping

[Peter Schaffter]

On Sat, Mar 10, 2018, Bjarni Ingi Gislason wrote:
> On Fri, Mar 09, 2018 at 01:14:14PM -0500, Peter Schaffter wrote:
> > Another reason to stop stripping.  If we're going to recommend that
> > users put \" after a trailing space in strings in order to protect
> > against superfluous spaces, it's counter-productive to remove them
> > from om.tmac, especially since it's introducing errors.
> > 
> >[...]
> 
>   A suggested patch to keep the protection is in the patch #53314 on the
> bug-groff list.

The patch doesn't take into account definitions of the form

  .[da]s foo \
"bar \"

I'm still strongly in favour of not stripping om.tmac, so providing
a fix would be counterproductive. :)

Unless there is a compelling reason to strip om.tmac, I'd like to
remove it from mom.am.  Comments?

-- 
Peter Schaffter
http://www.schaffter.ca

Re: [groff] More on stripping

[Bjarni Ingi Gislason]

On Fri, Mar 09, 2018 at 01:14:14PM -0500, Peter Schaffter wrote:
> Another reason to stop stripping.  If we're going to recommend that
> users put \" after a trailing space in strings in order to protect
> against superfluous spaces, it's counter-productive to remove them
> from om.tmac, especially since it's introducing errors.
> 
>[...]

  A suggested patch to keep the protection is in the patch #53314 on the
bug-groff list.

-- 
Bjarni I. Gislason