As most of you probably know, I'm hoping to get support for building BoostBook user documentation with BBv2, and I would like your help.
This message is long. Really long. It describes my "vision" of BoostBook documentation building with BBv2, along with implementation ideas. [Note: I'm CCing the Boost documentation list, but we should probably keep the discussion just on the jamboost list] === XSLT === XSLT is a language that describes a transformation from an XML document type into another document (or several documents). For our purposes, an XSLT transformation can be described by: 1) An XML document for input 2) An XSLT stylesheet 3) A set of stylesheet parameters (name/value pairs) 4) An output file or an output directory As I see it, we could have an "xslt" rule of the form: rule xslt target : source : stylesheet : parameters * If "target" is a normal file, then it's just a normal target in Jam. If "target" is a directory, it would have to be a pseudo-target because we can't generally guess at what the output files will be to build a target set. There is only a single source file, but like C++ source files it may include (via XIncludes) other XML files, and should depend on those files. We can probably restrict ourselves to XIncludes that look like this: <xi:include href="xml-file-we-depend-on"/> XSLT stylesheets are also XML files, but with a slightly different include mechanism. Stylesheet inclusion in XSLT takes one of these forms: <xsl:include href="xslt-file-we-depend-on"/> <xsl:import href="xslt-file-we-depend-on"/> Both XIncludes and XSL stylesheet inclusion the text in the href attribute can be either a local file (relative to the file including it; there is no search path) or a web address (e.g., http://www.etc.etc./). We should probably just ignore anything of the form "http://*"; and assume everything else is a file. XSL stylesheet parameters can probably be implemented via BBv2 features, e.g., xslt array-proposal.docbook : array.xml : docbook.xsl : <boost.generation.mode>standardese ; However, unlike compiler features that are properties of the compiler, stylesheet parameters are a property of the stylesheet so we can't easily maintain a list of all of the parameters for each stylesheet. We generally have to just "trust" the user. Perhaps an "xsl:" prefix to an arbitrary string? For instance, something like: xslt array-proposal.docbook : array.xml : docbook.xsl : <xsl:boost.generation.mode>standardese ; BBv2 would know nothing about "boost.generation.mode", but it would see the "xsl:" prefix and create the property (boost.generation.mode,standardese) for the target array-proposal.docbook. There are several XSLT processors available, so in its final form I would expect we'll want XSLT toolsets just like we have compiler toolsets. For now, however, I'm only recommending one XSLT processor (xsltproc). Here is the command line information for using xsltproc: xsltproc --xinclude <parameters> -o <target>? <stylesheet> <source> <parameters> is composed of strings "--stringparam feature value" for each property. <target> is the target (file or directory; doesn't matter) <stylesheet> is the stylesheet <source> is the source file So for the example above, the command line would be: xsltproc --xinclude --stringparam boost.generation.mode standardese \ -o array-proposal.docbook docbook.xsl array.xml (There's a little more information on the command line later, in the section "XML Catalogs") === BoostBook, DocBook, HTML, and Man pages === Given a BoostBook document we apply a series of XSLT transformations to get to end-user documentation (e.g., HTML or man pages). The transformations are: BoostBook file (.xml) --> DocBook file (.docbook) via stylesheet tools/boostbook/xsl/docbook.xsl DocBook (.docbook) --> single HTML file (.html) via stylesheet tools/boostbook/xsl/html-single.xsl DocBook (.docbook) --> HTML output directory via stylesheet tools/boostbook/xsl/html.xsl DocBook (.docbook) --> Man pages output directory via stylesheet tools/boostbook/xsl/man.xsl I expect that the Jamfile for these would look something like this: type.register BOOSTBOOK : xml bbk ; type.register DOCBOOK : docbook ; type.register HTML : html ; generators.register-standard boostbook.docbook : BOOSTBOOK : DOCBOOK ; generators.register-standard boostbook.html-single : HTML : DOCBOOK ; # ------- Most definitely broken attempt at a Jamfile -------- rule docbook ( target : source : parameters * ) { xslt $(target) : $(source) : $(BOOST_ROOT)/tools/boostbook/xsl/docbook.xsl : $(parameters) ; } rule html-single ( target : source : parameters * ) { xslt $(target) : $(source) : $(BOOST_ROOT)/tools/boostbook/xsl/html-single.xsl : $(parameters) ; } rule html ( target : source : parameters * ) { xslt $(target) : $(source) : $(BOOST_ROOT)/tools/boostbook/xsl/html.xsl : $(parameters) ; } rule man-pages ( target : source : parameters * ) { xslt $(target) : $(source) : $(BOOST_ROOT)/tools/boostbook/xsl/man.xsl : $(parameters) ; } # -----End most definitely broken attempt at a Jamfile ------ (The GNU make equivalent if some of these rules is at the end of this e-mail) The top-level target would reside in $(BOOST_ROOT)/doc/src and the Jamfile would probably contain: html-doc $(BOOST_ROOT)/doc/html : boost.xml ; A sample invocation: bjam html-doc xsl:boost.include.libraries="bind function ref" This would execute: xsltproc --xinclude --stringparam boost.include.libraries \ "bind function ref" -o boost.docbook \ $(BOOST_ROOT)/tools/boostbook/xsl/docbook.xsl boost.xml xsltproc --xinclude --stringparam boost.include.libraries \ "bind function ref" -o $(BOOST_ROOT)/doc/html \ $(BOOST_ROOT)/tools/boostbook/xsl/html.xsl boost.docbook === XML Catalogs === I mentioned previously that XSL stylesheet inclusions may use web addresses. When the XSLT processor needs the included stylesheets, it will download them first and then continue processing. Big bottleneck here (the DocBook HTML stylesheets alone are > 1MB) which can be solved by XML catalogs. The same thing goes for DTDs. XML catalogs remap remote web addresses (and, optionally, XML public identifiers like "-//Boost//DTD BoostBook XML V1.0//EN") to local files and directories. The XML catalog (just an XML file) is then passed to the XSLT processor, which does the remappings on-the-fly. XML catalogs can easily be generated when we know the local directories for: 1) The DocBook XSL stylesheets 2) The DocBook DTD 3) The BoostBook DTD (in $(BOOST_ROOT)/tools/boostbook/dtd) Ideally, the user could "configure" BBv2 with the locations of the first two (so that the user need not specify the directories each time), and BBv2 would build and use the XML catalog with whatever directory information it has, omitting the rest of the information so that the XSLT processor can download it as needed. For reference, the XML catalog I use (generated by the configure script at the end of this e-mail) is: <?xml version="1.0"?> <!DOCTYPE catalog PUBLIC "-//OASIS/DTD Entity Resolution XML Catalog V1.0//EN" "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd";> <catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> <rewriteURI uriStartString="http://www.oasis-open.org/docbook/xml/4.2/"; rewritePrefix="/home/gregod/Projects/share/docbook-dtd-4.2//"/> <rewriteURI uriStartString="http://docbook.sourceforge.net/release/xsl/current/"; rewritePrefix="/home/gregod/Projects/share/docbook-xsl-1.60.1//"/> <rewriteURI uriStartString="http://www.boost.org/tools/boostbook/dtd/"; rewritePrefix="/home/gregod/Projects/Boost/boost/tools/boostbook/dtd/"/> </catalog> The uriStartString attribute values are hardcoded values representing the DocBook XSL, DTD, and BoostBook DTD locations.The rewritePrefix attributes are the local directories for each. With xsltproc, the name of the XML catalog file is passed as an environment vaiable XML_CATALOG_FILES, so the command line would look like this when catalogs are used: XML_CATALOG_FILES=catalog.xml xsltproc <etc, etc> I have no idea how to accomplish this type of thing in BBv2 :( === Future issues: Doxygen, FO, PDF === At some point, there will be more potential steps in the documentation generation pipeline. Doxygen, for instance, can output XML, which can be collected into a single document (via an XSLT stylesheet) and then transformed into BoostBook (via an XSLT stylesheet). No need to worry about this now... The DocBook XSL stylesheet can also output XSL Formatting Object files (.fo), which are used for generating hardcopy versions of DocBook documents. However, FO->PS and FO->PDF conversion isn't done with XSLT, but with another external program (e.g., Apache's FOP). Again, no need to worry about this now, and it should be quite easy, but it's a hint that we may need yet another type of toolset in the future. Thanks, Doug === XML catalog configure script === #!/bin/sh if [ -z $2 ]; then echo "Syntax is ./configure <DocBook XSL path> <DocBook DTD path> [<FOP Path>]" echo "If you do not have the DocBook XSL Stylesheets or DTD, please download" echo "them from http://docbook.sourceforge.net/"; exit -1 fi DOCBOOK_XSL_PATH=$1 if [ ! -r $DOCBOOK_XSL_PATH/html/chunk.xsl ]; then echo "The DocBook XSL path given does not seem correct. Please check the" echo "path again and rerun configure. The DocBook XSL path given was: " echo " $DOCBOOK_XSL_PATH" exit -2 fi DOCBOOK_DTD_PATH=$2 if [ ! -r $DOCBOOK_DTD_PATH/docbookx.dtd ]; then echo "The DocBook DTD patch given does not seem correct. Please check the" echo "path again and rerun configure. The DocBook DTD path given was: " echo " $DOCBOOK_DTD_PATH" exit -3; fi BOOSTBOOK_XSL_PATH=`pwd` BOOSTBOOK_XSL_PATH="${BOOSTBOOK_XSL_PATH%/build*}/xsl" if [ ! -r "$BOOSTBOOK_XSL_PATH/docbook.xsl" ]; then echo "Please execute configure from the BoostBook build directory" exit -4; fi BOOSTBOOK_DTD_PATH=`pwd` BOOSTBOOK_DTD_PATH="${BOOSTBOOK_DTD_PATH%/build*}/dtd" if [ ! -r "$BOOSTBOOK_DTD_PATH/boostbook.dtd" ]; then echo "Please execute configure from the BoostBook build directory" exit -5; fi XSLTPROC=${XSLTPROC:-`which xsltproc`} if [ ! -x $XSLTPROC ]; then echo "Could not find the program xsltproc in the current PATH. Stopping..." exit -6 fi XMLLINT=${XMLLINT:-`which xmllint`} if [ ! -x $XMLLINT ]; then echo "Could not find the program xmllint in the current PATH." echo "This is not a problem per se, but you may not be able to validate" echo "XML files against a DTD." fi if [ -z $JAVA_HOME ]; then JAVA=${JAVA:-`which java`} JAVA_BIN_PATH=${JAVA%/*} JAVA_HOME=${JAVA_BIN_PATH%/bin} fi if [ ! -d $JAVA_HOME ]; then echo "Could not find Java. PDF generation will not be available." echo "If you have Java installed, set JAVA_HOME to its path and rerun " echo "configure." else FOP=${FOP:-$3} if [ ! -z $FOP ]; then if [ -d $FOP ]; then FOP=${FOP%/} FOP="$FOP/fop.sh" fi fi if [ -z $FOP ] || [ ! -x $FOP ]; then echo "Could not find FOP. PDF generation will not be available." echo "If you have FOP installed, pass the installation path as the second" echo "argument to configure, e.g.," echo " ./configure $DOCBOOK_XSL_PATH <FOP Path>" echo "" echo "If you do not have FOP and would like support for building PDF" echo "documentations, please download FOP from:" echo " http://xml.apache.org/fop/"; fi fi BOOSTBOOK_CATALOG="`pwd`/catalog.xml" cat > Makefile.def << . XSLTPROC=$XSLTPROC XMLLINT=$XMLLINT DOCBOOK_XSL_PATH=$DOCBOOK_XSL_PATH BOOSTBOOK_XSL_PATH=$BOOSTBOOK_XSL_PATH JAVA_HOME=$JAVA_HOME FOP=$FOP BOOSTBOOK_CATALOG=$BOOSTBOOK_CATALOG . cat > catalog.xml << . <?xml version="1.0"?> <!DOCTYPE catalog PUBLIC "-//OASIS/DTD Entity Resolution XML Catalog V1.0//EN" "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd";> <catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> <rewriteURI uriStartString="http://www.oasis-open.org/docbook/xml/4.2/"; rewritePrefix="$DOCBOOK_DTD_PATH/"/> <rewriteURI uriStartString="http://docbook.sourceforge.net/release/xsl/current/"; rewritePrefix="$DOCBOOK_XSL_PATH/"/> <rewriteURI uriStartString="http://www.boost.org/tools/boostbook/dtd/"; rewritePrefix="$BOOSTBOOK_DTD_PATH/"/> </catalog> . echo "BoostBook configuration is complete." === GNU make makefile === XSL=XML_CATALOG_FILES=$(BOOSTBOOK_CATALOG) $(XSLTPROC) --xinclude --nonet CHECK_DTD=XML_CATALOG_FILES=$(BOOSTBOOK_CATALOG) $(XMLLINT) --xinclude --noout --postvalid %.docbook: %.xml $(XSL) --stringparam boost.root $(BOOST_ROOT) -o $@ $(BOOSTBOOK_XSL_PATH)/docbook.xsl $< %.fo: %.docbook $(XSL) -o $@ $(BOOSTBOOK_XSL_PATH)/fo.xsl $< %.pdf: %.fo JAVA_HOME=$(JAVA_HOME) $(FOP) $< $@ %.html: %.docbook $(XSL) -o $@ $(BOOSTBOOK_XSL_PATH)/html-single.xsl $< [If you read this far, you deserve a cookie.] ------------------------------------------------------- This SF.net email is sponsored by:Crypto Challenge is now open! Get cracking and register here for some mind boggling fun and the chance of winning an Apple iPod: http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0031en _______________________________________________ Boost-docs mailing list [EMAIL PROTECTED] Unsubscribe and other administrative requests: https://lists.sourceforge.net/lists/listinfo/boost-docs
