Hi, On 2023-03-15 08:14:09 +0100, Peter Eisentraut wrote: > I have identified several open issues with the documentation build under > Meson (approximately in priority order): > > 1. Image files are not handled at all, so they don't show up in the final > product.
Hm. Somehow I thought I'd tackled that at some point. Ah. I got there for the PDF output, but didn't realize it's also an issue for the html output. For FO it sufficed to set the img.src.path param. For HTML that's not enough, because that just adjusts the link to the file - but we don't want to link to the source file. We actually solved this for the single-page html version - we just embed the svg. I wonder if we should just do that as well. Another way would be to emit the files into the desired place as part of the stylesheet. While it requires touching xslt, it does seems somewhat more elegant than just copying files around. I did implement that, curious what you think. > 2. Defaults to website stylesheet, no way to configure. This should be > adjusted to match the make build. Should we add a meson option? > 3. The various build targets and their combinations are mismatching and > incomplete. For example: > > Top-level GNUmakefile has these targets: > > - docs (builds html and man) > - html > - man > > (Those are the formats that are part of a distribution build.) > > doc/src/sgml/Makefile has these documented targets: > > - default target is html > - all (builds html and man, maps to top-level "docs") > - html > - man > - postgres-A4.pdf > - postgres-US.pdf > - check > > as well as (undocumented): > > - htmlhelp > - postgres.html > - postgres.txt > - epub > - postgres.epub > - postgres.info > > meson has the following documented targets: > > - docs (builds only html) > - alldocs (builds all formats, including obscure ones) > > as well as the following undocumented targets: > > - html > - man > - html_help [sic] renamed in the attached patch. > - postgres-A4.pdf > - postgres-US.pdf > - postgres.epub Note that these are actually named doc/src/sgml/{html,man,...}, not top-level targets. > - [info is not implemented at all] Would be easy to implement, but not sure it's worth doing. > - [didn't find an equivalent of check] That's probably worth doing - should it be run as an actual test, or be a target? > 4. There doesn't appear to be a way to install the documentation. > (There are also some open questions in the top-level meson.build about > the installation directories, but I suppose if we can't install them > then exactly where to install them hasn't been thought about too > much.) WIP patch for that attached. There's now install-doc-man install-doc-html run targets and a install-docs alias target. I did end up getting stuck when hacking on this, and ended up adding css support for nochunk and support for the website style for htmlhelp and nochunk, as well as obsoleting the need for copying the css files... But perhaps that's a bit too much. Greetings, Andres Freund
>From e604a5c5acdaa840ede0c83205c9799a9edb2630 Mon Sep 17 00:00:00 2001 From: Andres Freund <and...@anarazel.de> Date: Wed, 15 Mar 2023 15:53:14 -0700 Subject: [PATCH v1 1/5] meson: rename html_help target to htmlhelp Reported-by: Peter Eisentraut <peter.eisentr...@enterprisedb.com> Discussion: https://postgr.es/m/3fc3bb9b-f7f8-d442-35c1-ec82280c5...@enterprisedb.com --- doc/src/sgml/meson.build | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/src/sgml/meson.build b/doc/src/sgml/meson.build index 38f1b8e7b17..e6fe124c7bc 100644 --- a/doc/src/sgml/meson.build +++ b/doc/src/sgml/meson.build @@ -123,7 +123,7 @@ if xsltproc_bin.found() # build multi-page html docs as part of docs target docs += html - html_help = custom_target('html_help', + htmlhelp = custom_target('htmlhelp', input: ['stylesheet-hh.xsl', postgres_full_xml], output: 'htmlhelp', depfile: 'htmlhelp.d', @@ -131,7 +131,7 @@ if xsltproc_bin.found() command: [xsltproc, '--path', '@OUTDIR@', '-o', '@OUTDIR@/', xsltproc_flags, '@INPUT@'], build_by_default: false, ) - alldocs += html_help + alldocs += htmlhelp # single-page HTML -- 2.38.0
>From a739950d06678f28f08deb48d080939041dc5584 Mon Sep 17 00:00:00 2001 From: Andres Freund <and...@anarazel.de> Date: Wed, 15 Mar 2023 16:29:27 -0700 Subject: [PATCH v1 2/5] meson: make install_test_files more generic, rename to install_files Now it supports installing directories and directory contents as well. This will be used in a subsequent patch to install doc contents. --- meson.build | 5 ++- src/tools/install_files | 71 ++++++++++++++++++++++++++++++++++++ src/tools/install_test_files | 33 ----------------- 3 files changed, 74 insertions(+), 35 deletions(-) create mode 100644 src/tools/install_files delete mode 100644 src/tools/install_test_files diff --git a/meson.build b/meson.build index 2ebdf914c1b..b4b87f1a9fc 100644 --- a/meson.build +++ b/meson.build @@ -369,6 +369,8 @@ flex_cmd = [python, flex_wrapper, wget = find_program('wget', required: false, native: true) wget_flags = ['-O', '@OUTPUT0@', '--no-use-server-timestamps'] +install_files = files('src/tools/install_files') + ############################################################### @@ -2859,9 +2861,8 @@ testprep_targets += test_install_libs # command to install files used for tests, which aren't installed by default -install_test_files = files('src/tools/install_test_files') install_test_files_args = [ - install_test_files, + install_files, '--prefix', dir_prefix, '--install', contrib_data_dir, test_install_data, '--install', dir_lib_pkg, test_install_libs, diff --git a/src/tools/install_files b/src/tools/install_files new file mode 100644 index 00000000000..23ffd0aa2f7 --- /dev/null +++ b/src/tools/install_files @@ -0,0 +1,71 @@ +#!/usr/bin/env python3 + +# Helper to install additional files into the temporary installation +# for tests, beyond those that are installed by meson/ninja install. + +import argparse +import shutil +import os +from pathlib import PurePath + +parser = argparse.ArgumentParser() + +parser.add_argument('--destdir', type=str, + default=os.environ.get('DESTDIR', None)) +parser.add_argument('--prefix', type=str) +parser.add_argument('--install', type=str, nargs='+', + action='append', default=[]) +parser.add_argument('--install-dirs', type=str, nargs='+', + action='append', default=[]) +parser.add_argument('--install-dir-contents', type=str, nargs='+', + action='append', default=[]) + +args = parser.parse_args() + + +def error_exit(msg: str): + if os.path.isdir(src): + error_exit(f"{src} is a directory") + + +def create_target_dir(prefix: str, destdir: str, targetdir: str): + if not os.path.isabs(targetdir): + targetdir = os.path.join(prefix, targetdir) + + if destdir is not None: + # copy of meson's logic for joining destdir and install paths + targetdir = str(PurePath(destdir, *PurePath(targetdir).parts[1:])) + + os.makedirs(targetdir, exist_ok=True) + + return targetdir + + +def copy_files(targetdir: str, src_list: list): + for src in src_list: + shutil.copy2(src, targetdir) + + +def copy_dirs(targetdir: str, src_list: list, contents: bool): + for src in src_list: + if not os.path.isdir(src): + error_exit('{0} is not a directory'.format(src)) + + if contents: + target = targetdir + else: + target = os.path.join(targetdir, os.path.split(src)[1]) + shutil.copytree(src, target, dirs_exist_ok=True) + + +for installs in args.install: + targetdir = create_target_dir(args.prefix, args.destdir, installs[0]) + copy_files(targetdir, installs[1:]) + +for installs in args.install_dirs: + targetdir = create_target_dir(args.prefix, args.destdir, installs[0]) + copy_dirs(targetdir, installs[1:], contents=False) + +for installs in args.install_dir_contents: + targetdir = create_target_dir(args.prefix, args.destdir, installs[0]) + copy_dirs(targetdir, installs[1:], contents=True) diff --git a/src/tools/install_test_files b/src/tools/install_test_files deleted file mode 100644 index 8e0b36a74d1..00000000000 --- a/src/tools/install_test_files +++ /dev/null @@ -1,33 +0,0 @@ -#!/usr/bin/env python3 - -# Helper to install additional files into the temporary installation -# for tests, beyond those that are installed by meson/ninja install. - -import argparse -import shutil -import os -from pathlib import PurePath - -parser = argparse.ArgumentParser() - -parser.add_argument('--destdir', type=str, default=os.environ.get('DESTDIR', None)) -parser.add_argument('--prefix', type=str) -parser.add_argument('--install', type=str, nargs='+', action='append') - -args = parser.parse_args() - -def copy_files(prefix: str, destdir: str, targetdir: str, src_list: list): - if not os.path.isabs(targetdir): - targetdir = os.path.join(prefix, targetdir) - - if destdir is not None: - # copy of meson's logic for joining destdir and install paths - targetdir = str(PurePath(destdir, *PurePath(targetdir).parts[1:])) - - os.makedirs(targetdir, exist_ok=True) - - for src in src_list: - shutil.copy2(src, targetdir) - -for installs in args.install: - copy_files(args.prefix, args.destdir, installs[0], installs[1:]) -- 2.38.0
>From 1ca73641bd0def58bbaedd2c3a5b1ae95918cd32 Mon Sep 17 00:00:00 2001 From: Andres Freund <and...@anarazel.de> Date: Wed, 15 Mar 2023 16:43:40 -0700 Subject: [PATCH v1 3/5] wip: meson: add install-{docs,doc-html,doc-man} targets --- meson.build | 2 +- doc/src/sgml/meson.build | 27 ++++++++++++++++++++++++++- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/meson.build b/meson.build index b4b87f1a9fc..0ffeffa9edc 100644 --- a/meson.build +++ b/meson.build @@ -504,7 +504,7 @@ dir_man = get_option('mandir') # FIXME: These used to be separately configurable - worth adding? dir_doc = get_option('datadir') / 'doc' / 'postgresql' -dir_doc_html = dir_doc +dir_doc_html = dir_doc / 'html' dir_locale = get_option('localedir') diff --git a/doc/src/sgml/meson.build b/doc/src/sgml/meson.build index e6fe124c7bc..2da737c8af4 100644 --- a/doc/src/sgml/meson.build +++ b/doc/src/sgml/meson.build @@ -1,6 +1,7 @@ # Copyright (c) 2022-2023, PostgreSQL Global Development Group docs = [] +installdocs = [] alldocs = [] doc_generated = [] @@ -120,8 +121,16 @@ if xsltproc_bin.found() ) alldocs += html - # build multi-page html docs as part of docs target + install_doc_html = run_target('install-doc-html', + command: [ + python, install_files, '--prefix', dir_prefix, + '--install-dir-contents', dir_doc_html, html.full_path()], + depends: html) + + # build and install multi-page html docs as part of docs target docs += html + installdocs += install_doc_html + htmlhelp = custom_target('htmlhelp', input: ['stylesheet-hh.xsl', postgres_full_xml], @@ -208,6 +217,21 @@ if xsltproc_bin.found() build_by_default: false, ) alldocs += man + + mans = [] + foreach man_n : man.to_list() + mans += man_n.full_path() + endforeach + + install_doc_man = run_target('install-doc-man', + command: [ + python, install_files, '--prefix', dir_prefix, + '--install-dirs', dir_man, mans], + depends: man) + + # even though we don't want to build man pages as part of 'docs', we do want + # to install them as part of install-docs + installdocs += install_doc_man endif @@ -266,6 +290,7 @@ if docs.length() == 0 run_target('docs', command: [missing, 'xsltproc']) else alias_target('docs', docs) + alias_target('install-docs', installdocs) endif if alldocs.length() == 0 -- 2.38.0
>From 679973341b9fb131034e9d7a7acafe761ecf9ff8 Mon Sep 17 00:00:00 2001 From: Andres Freund <and...@anarazel.de> Date: Wed, 15 Mar 2023 16:45:14 -0700 Subject: [PATCH v1 4/5] docs: html: copy images to output as part of xslt build --- doc/src/sgml/Makefile | 2 -- doc/src/sgml/stylesheet-common.xsl | 22 ++++++++++++++++++++++ doc/src/sgml/stylesheet-hh.xsl | 6 ++++++ doc/src/sgml/stylesheet.xsl | 7 ++++++- 4 files changed, 34 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/Makefile b/doc/src/sgml/Makefile index b96c7cbf223..1b098f983ec 100644 --- a/doc/src/sgml/Makefile +++ b/doc/src/sgml/Makefile @@ -144,7 +144,6 @@ html: html-stamp html-stamp: stylesheet.xsl postgres-full.xml $(ALL_IMAGES) $(XSLTPROC) $(XMLINCLUDE) $(XSLTPROCFLAGS) $(XSLTPROC_HTML_FLAGS) $(wordlist 1,2,$^) - cp $(ALL_IMAGES) html/ cp $(srcdir)/stylesheet.css html/ touch $@ @@ -152,7 +151,6 @@ htmlhelp: htmlhelp-stamp htmlhelp-stamp: stylesheet-hh.xsl postgres-full.xml $(ALL_IMAGES) $(XSLTPROC) $(XMLINCLUDE) $(XSLTPROCFLAGS) $(wordlist 1,2,$^) - cp $(ALL_IMAGES) htmlhelp/ cp $(srcdir)/stylesheet.css htmlhelp/ touch $@ diff --git a/doc/src/sgml/stylesheet-common.xsl b/doc/src/sgml/stylesheet-common.xsl index 761484c7fef..d2928f86eb7 100644 --- a/doc/src/sgml/stylesheet-common.xsl +++ b/doc/src/sgml/stylesheet-common.xsl @@ -101,4 +101,26 @@ <xsl:apply-templates select="." mode="xref"/> </xsl:template> + +<!-- + Support for copying images to the output directory, so the output is self + contained. +--> +<xsl:template name="write-image"> + <xsl:variable name="input_filename"> + <xsl:value-of select="imagedata/@fileref"/> + </xsl:variable> + + <!-- references images directly, without images/ --> + <xsl:variable name="output_filename"> + <xsl:value-of select="concat($chunk.base.dir, substring-after($input_filename, '/'))"/> + </xsl:variable> + + <xsl:call-template name="write.chunk"> + <xsl:with-param name="filename" select="$output_filename"/> + <xsl:with-param name="content" select="document($input_filename)"/> + </xsl:call-template> +</xsl:template> + + </xsl:stylesheet> diff --git a/doc/src/sgml/stylesheet-hh.xsl b/doc/src/sgml/stylesheet-hh.xsl index 6f4b706dac6..568ccf36d2a 100644 --- a/doc/src/sgml/stylesheet-hh.xsl +++ b/doc/src/sgml/stylesheet-hh.xsl @@ -39,6 +39,12 @@ </xsl:template> +<xsl:template match="imageobject"> + <xsl:call-template name="write-image"/> + <!-- copy images to the output directory, so the output is self contained --> + <xsl:apply-templates select="imagedata"/> +</xsl:template> + <!-- strip directory name from image filerefs --> <xsl:template match="imagedata/@fileref"> <xsl:value-of select="substring-after(., '/')"/> diff --git a/doc/src/sgml/stylesheet.xsl b/doc/src/sgml/stylesheet.xsl index b6141303abd..f9163e7d946 100644 --- a/doc/src/sgml/stylesheet.xsl +++ b/doc/src/sgml/stylesheet.xsl @@ -29,7 +29,12 @@ </xsl:param> -<!-- strip directory name from image filerefs --> +<xsl:template match="imageobject"> + <xsl:call-template name="write-image"/> + <!-- copy images to the output directory, so the output is self contained --> + <xsl:apply-templates select="imagedata"/> +</xsl:template> + <xsl:template match="imagedata/@fileref"> <xsl:value-of select="substring-after(., '/')"/> </xsl:template> -- 2.38.0
>From f5e081b911797fa6897a883b0f1f3e35daf170ad Mon Sep 17 00:00:00 2001 From: Andres Freund <and...@anarazel.de> Date: Wed, 15 Mar 2023 20:49:47 -0700 Subject: [PATCH v1 5/5] wip: docs: copy or inline css --- doc/src/sgml/Makefile | 4 +--- doc/src/sgml/meson.build | 2 +- doc/src/sgml/stylesheet-common.xsl | 21 +++++++++++++++++++++ doc/src/sgml/stylesheet-hh.xsl | 10 +++++++++- doc/src/sgml/stylesheet-html-nochunk.xsl | 20 ++++++++++++++++++++ doc/src/sgml/stylesheet.xsl | 15 ++++++--------- 6 files changed, 58 insertions(+), 14 deletions(-) diff --git a/doc/src/sgml/Makefile b/doc/src/sgml/Makefile index 1b098f983ec..3f971f6eb33 100644 --- a/doc/src/sgml/Makefile +++ b/doc/src/sgml/Makefile @@ -53,7 +53,7 @@ else XSLTPROC = $(missing) xsltproc endif -override XSLTPROCFLAGS += --stringparam pg.version '$(VERSION)' +override XSLTPROCFLAGS += --xincludestyle --stringparam pg.version '$(VERSION)' GENERATED_SGML = version.sgml \ @@ -144,14 +144,12 @@ html: html-stamp html-stamp: stylesheet.xsl postgres-full.xml $(ALL_IMAGES) $(XSLTPROC) $(XMLINCLUDE) $(XSLTPROCFLAGS) $(XSLTPROC_HTML_FLAGS) $(wordlist 1,2,$^) - cp $(srcdir)/stylesheet.css html/ touch $@ htmlhelp: htmlhelp-stamp htmlhelp-stamp: stylesheet-hh.xsl postgres-full.xml $(ALL_IMAGES) $(XSLTPROC) $(XMLINCLUDE) $(XSLTPROCFLAGS) $(wordlist 1,2,$^) - cp $(srcdir)/stylesheet.css htmlhelp/ touch $@ # single-page HTML diff --git a/doc/src/sgml/meson.build b/doc/src/sgml/meson.build index 2da737c8af4..c581c7246f0 100644 --- a/doc/src/sgml/meson.build +++ b/doc/src/sgml/meson.build @@ -104,7 +104,7 @@ if xsltproc_bin.found() xsltproc_flags = [ '--nonet', '--stringparam', 'pg.version', pg_version, - '--param', 'website.stylesheet', '1' + '--xincludestyle', ] xsltproc = xmltools_wrapper + [ diff --git a/doc/src/sgml/stylesheet-common.xsl b/doc/src/sgml/stylesheet-common.xsl index d2928f86eb7..23dea8f29bf 100644 --- a/doc/src/sgml/stylesheet-common.xsl +++ b/doc/src/sgml/stylesheet-common.xsl @@ -41,6 +41,8 @@ <xsl:param name="variablelist.term.break.after">1</xsl:param> <xsl:param name="variablelist.term.separator"></xsl:param> <xsl:param name="xref.with.number.and.title" select="0"></xsl:param> +<!-- currently htmlhelp and other html have no common stylesheet --> +<xsl:param name="website.stylesheet" select="0"/> <!-- Change display of some elements --> @@ -122,5 +124,24 @@ </xsl:call-template> </xsl:template> +<xsl:template name="stylesheet-reference"> + <xsl:choose> + <xsl:when test="$website.stylesheet = 0">stylesheet.css + <xsl:variable name="styleout" select="concat($base.dir, 'stylesheet.css')"/> + <xsl:call-template name="write.chunk"> + <xsl:with-param name="filename" select="$styleout"/> + <xsl:with-param name="quiet" select="0"/> <!-- XXX: remove --> + <xsl:with-param name="omit-xml-declaration" select="'yes'"/> + <xsl:with-param name="content"> + <!-- xinclude is only processed in toplevel stylesheet --> + <xsl:call-template name="stylesheet-contents"/> + </xsl:with-param> + </xsl:call-template> + </xsl:when> + <xsl:otherwise> + https://www.postgresql.org/media/css/docs-complete.css + </xsl:otherwise> + </xsl:choose> +</xsl:template> </xsl:stylesheet> diff --git a/doc/src/sgml/stylesheet-hh.xsl b/doc/src/sgml/stylesheet-hh.xsl index 568ccf36d2a..6f6c6c64626 100644 --- a/doc/src/sgml/stylesheet-hh.xsl +++ b/doc/src/sgml/stylesheet-hh.xsl @@ -1,5 +1,6 @@ <?xml version='1.0'?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" + xmlns:xi="http://www.w3.org/2001/XInclude" version='1.0'> <xsl:import href="http://docbook.sourceforge.net/release/xsl/current/htmlhelp/htmlhelp.xsl"/> @@ -9,7 +10,6 @@ <xsl:param name="htmlhelp.use.hhk" select="'1'"/> <xsl:param name="base.dir" select="'htmlhelp/'"></xsl:param> -<xsl:param name="html.stylesheet" select="'stylesheet.css'"></xsl:param> <xsl:param name="use.id.as.filename" select="'1'"></xsl:param> <xsl:param name="manifest.in.base.dir" select="1"/> <xsl:param name="make.valid.html" select="1"></xsl:param> @@ -19,6 +19,14 @@ <xsl:param name="chunker.output.indent" select="'yes'"/> <xsl:param name="chunk.quietly" select="1"></xsl:param> +<xsl:param name="html.stylesheet"> + <xsl:call-template name="stylesheet-reference"/> +</xsl:param> + +<xsl:template name="stylesheet-contents"> + <xi:include href="stylesheet.css" parse="text"/> +</xsl:template> + <!-- Change display of some elements --> diff --git a/doc/src/sgml/stylesheet-html-nochunk.xsl b/doc/src/sgml/stylesheet-html-nochunk.xsl index 8167127b93a..560f6a0b0a7 100644 --- a/doc/src/sgml/stylesheet-html-nochunk.xsl +++ b/doc/src/sgml/stylesheet-html-nochunk.xsl @@ -1,5 +1,7 @@ <?xml version='1.0'?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" + xmlns:xi="http://www.w3.org/2001/XInclude" + xmlns="http://www.w3.org/1999/xhtml" version='1.0'> <xsl:import href="http://docbook.sourceforge.net/release/xsl/current/xhtml/docbook.xsl"/> @@ -7,6 +9,24 @@ <xsl:include href="stylesheet-html-common.xsl" /> <xsl:include href="stylesheet-speedup-xhtml.xsl" /> + +<xsl:param name="generate.css.header" select="1"/> +<xsl:template name="generate.css.headers"> + <xsl:choose> + <!-- inline css style sheet --> + <xsl:when test="$website.stylesheet = 0"> + <style type="text/css"> + <xi:include href="stylesheet.css" parse="text"/> + </style> + </xsl:when> + <!-- link to website --> + <xsl:otherwise> + <link rel="stylesheet" type="text/css" href="https://www.postgresql.org/media/css/docs-complete.css"/> + </xsl:otherwise> + </xsl:choose> +</xsl:template> + + <!-- embed SVG images into output file --> <xsl:template match="imagedata[@format='SVG']"> <xsl:variable name="filename"> diff --git a/doc/src/sgml/stylesheet.xsl b/doc/src/sgml/stylesheet.xsl index f9163e7d946..c551b9cd809 100644 --- a/doc/src/sgml/stylesheet.xsl +++ b/doc/src/sgml/stylesheet.xsl @@ -1,7 +1,8 @@ <?xml version='1.0'?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version='1.0' - xmlns="http://www.w3.org/1999/xhtml"> + xmlns="http://www.w3.org/1999/xhtml" + xmlns:xi="http://www.w3.org/2001/XInclude"> <xsl:import href="http://docbook.sourceforge.net/release/xsl/current/xhtml/chunk.xsl"/> <xsl:include href="stylesheet-common.xsl" /> @@ -17,17 +18,13 @@ <xsl:param name="chunk.quietly" select="1"></xsl:param> <xsl:param name="admon.style"></xsl:param> <!-- handled by CSS stylesheet --> -<xsl:param name="website.stylesheet" select="0"/> - <xsl:param name="html.stylesheet"> - <xsl:choose> - <xsl:when test="$website.stylesheet = 0">stylesheet.css</xsl:when> - <xsl:otherwise> - https://www.postgresql.org/media/css/docs-complete.css - </xsl:otherwise> - </xsl:choose> + <xsl:call-template name="stylesheet-reference"/> </xsl:param> +<xsl:template name="stylesheet-contents"> + <xi:include href="stylesheet.css" parse="text"/> +</xsl:template> <xsl:template match="imageobject"> <xsl:call-template name="write-image"/> -- 2.38.0