Re: [HACKERS] port of INSTALL file generation to XSLT

2017-03-08 Thread Peter Eisentraut
On 3/1/17 20:23, Magnus Hagander wrote:
> On Wed, Mar 1, 2017 at 3:58 AM, Peter Eisentraut
>  > wrote:
> 
> On 2/28/17 08:57, Magnus Hagander wrote:
> > It appears we need pandoc 1.13 to get the good output.  This won't 
> be
> > available until Debian stretch.
> >
> > So for PG10, I propose the attached revised patch which keeps using 
> lynx
> > but uses xsltproc instead of jade.
> >
> >
> > It is available for people not using debian though? Might it be
> > worthwhile to make it dependent on the version of pandoc -- so use that
> > method if people have the newer pandoc and fall back to lynx if they 
> don't?
> 
> Well, this really only runs once every couple of months on borka and
> here or there for those building their own snapshot tarballs.  I don't
> think we need to cater to a wide audience here.  In fact, variety could
> be bad here:  We don't want to find out that a tarball was rolled with
> the wrong variant.
> 
> 
> Good point. Let 's just go for it then.

Committed that way.  Thanks.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] port of INSTALL file generation to XSLT

2017-03-01 Thread Magnus Hagander
On Wed, Mar 1, 2017 at 3:58 AM, Peter Eisentraut <
peter.eisentr...@2ndquadrant.com> wrote:

> On 2/28/17 08:57, Magnus Hagander wrote:
> > It appears we need pandoc 1.13 to get the good output.  This won't be
> > available until Debian stretch.
> >
> > So for PG10, I propose the attached revised patch which keeps using
> lynx
> > but uses xsltproc instead of jade.
> >
> >
> > It is available for people not using debian though? Might it be
> > worthwhile to make it dependent on the version of pandoc -- so use that
> > method if people have the newer pandoc and fall back to lynx if they
> don't?
>
> Well, this really only runs once every couple of months on borka and
> here or there for those building their own snapshot tarballs.  I don't
> think we need to cater to a wide audience here.  In fact, variety could
> be bad here:  We don't want to find out that a tarball was rolled with
> the wrong variant.
>

Good point. Let 's just go for it then.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: [HACKERS] port of INSTALL file generation to XSLT

2017-02-28 Thread Peter Eisentraut
On 2/28/17 08:57, Magnus Hagander wrote:
> It appears we need pandoc 1.13 to get the good output.  This won't be
> available until Debian stretch.
> 
> So for PG10, I propose the attached revised patch which keeps using lynx
> but uses xsltproc instead of jade.
> 
> 
> It is available for people not using debian though? Might it be
> worthwhile to make it dependent on the version of pandoc -- so use that
> method if people have the newer pandoc and fall back to lynx if they don't? 

Well, this really only runs once every couple of months on borka and
here or there for those building their own snapshot tarballs.  I don't
think we need to cater to a wide audience here.  In fact, variety could
be bad here:  We don't want to find out that a tarball was rolled with
the wrong variant.

The pandoc change can be revisited independently.  The main point right
now is to get away from the DSSSL toolchain.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] port of INSTALL file generation to XSLT

2017-02-28 Thread Magnus Hagander
On Mon, Feb 27, 2017 at 4:55 PM, Peter Eisentraut <
peter.eisentr...@2ndquadrant.com> wrote:

> On 12/31/16 07:33, Magnus Hagander wrote:
> > Borka being a standard debian jessie install has Pandoc
> > 1.12.4.2~dfsg-1+b14. Should be no problem installing that.
> >
> > I ran a "make INSTALL" on a jessie box, and it comes out with some
> > formatting still in the file, see attachment. That seems incorrect.
>
> It appears we need pandoc 1.13 to get the good output.  This won't be
> available until Debian stretch.
>
> So for PG10, I propose the attached revised patch which keeps using lynx
> but uses xsltproc instead of jade.
>
>
It is available for people not using debian though? Might it be worthwhile
to make it dependent on the version of pandoc -- so use that method if
people have the newer pandoc and fall back to lynx if they don't?

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: [HACKERS] port of INSTALL file generation to XSLT

2017-02-27 Thread Peter Eisentraut
On 12/31/16 07:33, Magnus Hagander wrote:
> Borka being a standard debian jessie install has Pandoc
> 1.12.4.2~dfsg-1+b14. Should be no problem installing that.
> 
> I ran a "make INSTALL" on a jessie box, and it comes out with some
> formatting still in the file, see attachment. That seems incorrect.

It appears we need pandoc 1.13 to get the good output.  This won't be
available until Debian stretch.

So for PG10, I propose the attached revised patch which keeps using lynx
but uses xsltproc instead of jade.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>From 081e0c8d4d47ab2628a9d11d6c4e0ed014180a20 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Mon, 27 Feb 2017 10:44:59 -0500
Subject: [PATCH v2] Create INSTALL file via XSLT

As before, create an INSTALL.html file for processing with lynx, but use
xsltproc and a new XSLT stylesheet instead of jade and DSSSL.

Replacing jade with xsltproc removes jade from the requirements for
distribution building.
---
 doc/src/sgml/.gitignore  |  1 +
 doc/src/sgml/Makefile| 35 --
 doc/src/sgml/stylesheet-text.xsl | 98 
 doc/src/sgml/stylesheet.dsl  | 43 --
 4 files changed, 120 insertions(+), 57 deletions(-)
 create mode 100644 doc/src/sgml/stylesheet-text.xsl

diff --git a/doc/src/sgml/.gitignore b/doc/src/sgml/.gitignore
index 2f0329c15f..cdeace2991 100644
--- a/doc/src/sgml/.gitignore
+++ b/doc/src/sgml/.gitignore
@@ -7,6 +7,7 @@
 /man-stamp
 # Other popular build targets
 /INSTALL
+/INSTALL.xml
 /postgres-US.pdf
 /postgres-A4.pdf
 /postgres.html
diff --git a/doc/src/sgml/Makefile b/doc/src/sgml/Makefile
index fe7ca65cd4..774d35de20 100644
--- a/doc/src/sgml/Makefile
+++ b/doc/src/sgml/Makefile
@@ -217,10 +217,9 @@ postgres.pdf:
 
 
 ##
-## Semi-automatic generation of some text files.
+## Generation of some text files.
 ##
 
-JADE.text = $(JADE) $(JADEFLAGS) $(SGMLINCLUDE) $(CATALOG) -d stylesheet.dsl -i output-text -t sgml
 ICONV = iconv
 LYNX = lynx
 
@@ -233,10 +232,15 @@ LYNX = lynx
 # locale support and is very picky about locale name spelling.  The
 # below has been finely tuned to run on FreeBSD and Linux/glibc.
 INSTALL: % : %.html
-	$(PERL) -p -e 's/ $@
+	$(PERL) -p -e 's, $@
 
-INSTALL.html: standalone-install.sgml installation.sgml version.sgml
-	$(JADE.text) -V nochunks standalone-install.sgml installation.sgml > $@
+INSTALL.html: %.html : stylesheet-text.xsl %.xml
+	$(XMLLINT) --noout --valid $*.xml
+	$(XSLTPROC) $(XSLTPROCFLAGS) $(XSLTPROC_HTML_FLAGS) $^ >$@
+
+INSTALL.xml: standalone-install.sgml installation.sgml version.sgml
+	$(OSX) -D. -x lower $(filter-out version.sgml,$^) >$@.tmp
+	$(call mangle-xml,chapter)
 
 
 ##
@@ -247,12 +251,15 @@ INSTALL.html: standalone-install.sgml installation.sgml version.sgml
 # if we try to do "make all" in a VPATH build without the explicit
 # $(srcdir) on the postgres.sgml dependency in this rule.  GNU make bug?
 postgres.xml: $(srcdir)/postgres.sgml $(ALMOSTALLSGML)
-	$(OSX) -D. -x lower -i include-xslt-index $< >postgres.xmltmp
-	$(PERL) -p -e 's/\[(aacute|acirc|aelig|agrave|amp|aring|atilde|auml|bull|copy|eacute|egrave|gt|iacute|lt|mdash|nbsp|ntilde|oacute|ocirc|oslash|ouml|pi|quot|scaron|uuml) *\]/\&\1;/gi;' \
-	   -e '$$_ .= qq{http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd";>\n} if $$. == 1;' \
-	   $@
-	rm postgres.xmltmp
-# ' hello Emacs
+	$(OSX) -D. -x lower -i include-xslt-index $< >$@.tmp
+	$(call mangle-xml,book)
+
+define mangle-xml
+$(PERL) -p -e 's/\[(aacute|acirc|aelig|agrave|amp|aring|atilde|auml|bull|copy|eacute|egrave|gt|iacute|lt|mdash|nbsp|ntilde|oacute|ocirc|oslash|ouml|pi|quot|scaron|uuml) *\]/\&\1;/gi;' \
+   -e '$$_ .= qq{http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd";>\n} if $$. == 1;' \
+  <$@.tmp > $@
+rm $@.tmp
+endef
 
 ifeq ($(STYLE),website)
 XSLTPROC_HTML_FLAGS += --param website.stylesheet 1
@@ -386,13 +393,13 @@ check-tabs:
 # This allows removing some files from the distribution tarballs while
 # keeping the dependencies satisfied.
 .SECONDARY: postgres.xml $(GENERATED_SGML) HTML.index
-.SECONDARY: INSTALL.html
+.SECONDARY: INSTALL.html INSTALL.xml
 

Re: [HACKERS] port of INSTALL file generation to XSLT

2016-12-31 Thread Magnus Hagander
On Sat, Dec 31, 2016 at 6:57 AM, Peter Eisentraut <
peter.eisentr...@2ndquadrant.com> wrote:

> A further step toward getting rid of the DSSSL tool chain requirement,
> here is a patch to change the generation of the text INSTALL file to use
> XLST and Pandoc.
>
> The change to Pandoc is not essential to this change, but it creates
> much better looking output and simplifies the locale/encoding handling
> over using Lynx.
>

Yeah, that seems a lot more clean than using Lynx.



>
> We'll need to get Pandoc installed on borka and check that that version
> works as well as the one I have been using.
>

Borka being a standard debian jessie install has Pandoc
1.12.4.2~dfsg-1+b14. Should be no problem installing that.

I ran a "make INSTALL" on a jessie box, and it comes out with some
formatting still in the file, see attachment. That seems incorrect.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


INSTALL
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] port of INSTALL file generation to XSLT

2016-12-30 Thread Peter Eisentraut
A further step toward getting rid of the DSSSL tool chain requirement,
here is a patch to change the generation of the text INSTALL file to use
XLST and Pandoc.

The change to Pandoc is not essential to this change, but it creates
much better looking output and simplifies the locale/encoding handling
over using Lynx.

We'll need to get Pandoc installed on borka and check that that version
works as well as the one I have been using.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
From 41fe676ae519e0601b67e5ea57cf32eeb266f40f Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Tue, 27 Dec 2016 12:00:00 -0500
Subject: [PATCH] Create INSTALL file via XSLT and Pandoc

Replace the old tool chain of jade and lynx with using xsltproc and
pandoc.  As before, we create an intermediate HTML file and convert that
to plain text.  Replacing jade with xsltproc removes jade from the
requirements for distribution building.  The change to pandoc is
incidental, but it creates better looking output and it avoids the
delicate locale/encoding issues of lynx because it always uses UTF-8 for
both input and output.
---
 doc/src/sgml/.gitignore  |  1 +
 doc/src/sgml/Makefile| 47 +++--
 doc/src/sgml/stylesheet-text.xsl | 89 
 3 files changed, 113 insertions(+), 24 deletions(-)
 create mode 100644 doc/src/sgml/stylesheet-text.xsl

diff --git a/doc/src/sgml/.gitignore b/doc/src/sgml/.gitignore
index 2f0329c15f..8197c0140d 100644
--- a/doc/src/sgml/.gitignore
+++ b/doc/src/sgml/.gitignore
@@ -21,6 +21,7 @@
 # Assorted byproducts from building the above
 /postgres.xml
 /INSTALL.html
+/INSTALL.xml
 /postgres-US.aux
 /postgres-US.log
 /postgres-US.out
diff --git a/doc/src/sgml/Makefile b/doc/src/sgml/Makefile
index fe7ca65cd4..0c18a8a84a 100644
--- a/doc/src/sgml/Makefile
+++ b/doc/src/sgml/Makefile
@@ -217,26 +217,22 @@ postgres.pdf:
 
 
 ##
-## Semi-automatic generation of some text files.
+## Generation of some text files
 ##
 
-JADE.text = $(JADE) $(JADEFLAGS) $(SGMLINCLUDE) $(CATALOG) -d stylesheet.dsl -i output-text -t sgml
 ICONV = iconv
-LYNX = lynx
-
-# The documentation may contain non-ASCII characters (mostly for
-# contributor names), which lynx converts to the encoding determined
-# by the current locale.  To get text output that is deterministic and
-# easily readable by everyone, we make lynx produce LATIN1 and then
-# convert that to ASCII with transliteration for the non-ASCII characters.
-# Official releases were historically built on FreeBSD, which has limited
-# locale support and is very picky about locale name spelling.  The
-# below has been finely tuned to run on FreeBSD and Linux/glibc.
+PANDOC = pandoc
+
 INSTALL: % : %.html
-	$(PERL) -p -e 's/ $@
+	$(PANDOC) $< -t plain | $(ICONV) -f utf8 -t us-ascii//TRANSLIT > $@
+
+INSTALL.html: %.html : stylesheet-text.xsl %.xml
+	$(XMLLINT) --noout --valid $*.xml
+	$(XSLTPROC) $(XSLTPROCFLAGS) $(XSLTPROC_HTML_FLAGS) $^ >$@
 
-INSTALL.html: standalone-install.sgml installation.sgml version.sgml
-	$(JADE.text) -V nochunks standalone-install.sgml installation.sgml > $@
+INSTALL.xml: standalone-install.sgml installation.sgml version.sgml
+	$(OSX) -D. -x lower $(filter-out version.sgml,$^) >$@.tmp
+	$(call mangle-xml,chapter)
 
 
 ##
@@ -247,12 +243,15 @@ INSTALL.html: standalone-install.sgml installation.sgml version.sgml
 # if we try to do "make all" in a VPATH build without the explicit
 # $(srcdir) on the postgres.sgml dependency in this rule.  GNU make bug?
 postgres.xml: $(srcdir)/postgres.sgml $(ALMOSTALLSGML)
-	$(OSX) -D. -x lower -i include-xslt-index $< >postgres.xmltmp
-	$(PERL) -p -e 's/\[(aacute|acirc|aelig|agrave|amp|aring|atilde|auml|bull|copy|eacute|egrave|gt|iacute|lt|mdash|nbsp|ntilde|oacute|ocirc|oslash|ouml|pi|quot|scaron|uuml) *\]/\&\1;/gi;' \
-	   -e '$$_ .= qq{http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd;>\n} if $$. == 1;' \
-	   $@
-	rm postgres.xmltmp
-# ' hello Emacs
+	$(OSX) -D. -x lower -i include-xslt-index $< >$@.tmp
+	$(call mangle-xml,book)
+
+define mangle-xml
+$(PERL) -p -e 's/\[(aacute|acirc|aelig|agrave|amp|aring|atilde|auml|bull|copy|eacute|egrave|gt|iacute|lt|mdash|nbsp|ntilde|oacute|ocirc|oslash|ouml|pi|quot|scaron|uuml) *\]/\&\1;/gi;' \
+   -e '$$_ .= qq{http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd;>\n} if $$. == 1;' \
+  <$@.tmp >$@
+rm $@.tmp
+endef
 
 ifeq ($(STYLE),website)
 XSLTPROC_HTML_FLAGS += --param website.stylesheet 1
@@ -386,13 +385,13 @@ check-tabs:
 # This allows removing some files from the distribution tarballs while
 # keeping the dependencies satisfied.
 .SECONDARY: postgres.xml $(GENERATED_SGML) HTML.index
-.SECONDARY: INSTALL.html
+.SECONDARY: INSTALL.html