Dear Gil, This is just to inform that I have managed to inject Erich Rexx script in my build chain so that my built documents should now be 100% the same as yours and the ones built by Erich or Rick. I could only test it on rexxref.pdf but that is a beast and if it works there the rest should be ok.
Removing the additional spaces shrunk rexxref.pdf with 10 pages and ooDialog with 182 pages ! ooDialog.pdf is now „only“ 1898 pages :-) I do not quite understand your remark about the TOC; your TOC is identical to the one built by the current Publican build chain, in fact the rexxref.pdf is identical at the byte level to the one from Sourceforge, I used a hex comparison to be sure. But maybe you are thinking of something else? Hälsningar/Regards/Grüsse, P.O. Jonsson oor...@jonases.se > Am 20.02.2020 um 20:00 schrieb Gil Barmwater <gbarmwa...@alum.rpi.edu>: > > Thanks P.O. for doing the compare! I'm glad to hear that you found no > differences in this very large document :-). > > There is still the issue of the Table Of Contents differences but I have > determined that they are actually generated and added by FOP and NOT by > xsltproc (surprise!). I think I may have found a parameter to control the > depth so a bit more testing before I recreate the package. Hopefully that > will happen soon. > > Gil > On 2/20/2020 8:24 AM, P.O. Jonsson wrote: >> Dear Gil, >> >> The Rexxref you sent was the latest version, 11980, so I could compare it >> also to the version we have on Sourceforge: >> >> 1: Your rexxref is BINARY IDENTICAL to the one on sourceforge! >> 2: I could compare your Rexxref to my own build and see that the textual >> content is identical to my builds BUT there is an important difference, >> namely where I have additional spaces (I did not implement Erichs filter) >> and you don’t, which goes to proof that your strategy was successful. The >> additional spaces in my build docs lead to shift in content eventually >> leading to renumbering of pages. >> >> I might have a go at injecting Erichs filter process in my build chain but >> only as an academic exercise, I think it is safe to say that your build >> change can replace Publican in the near future. Hats off! >> >> I could not include screenshots of the comparison so I included them in a >> document that I have attached. >> >> >> >> >> >> Hälsningar/Regards/Grüsse, >> P.O. Jonsson >> oor...@jonases.se <mailto:oor...@jonases.se> >> >> >> >>> Am 20.02.2020 um 02:02 schrieb Gil Barmwater <gbarmwa...@alum.rpi.edu >>> <mailto:gbarmwa...@alum.rpi.edu>>: >>> >>> It's at 11974, the same one that is packaged with the r11978 build of 2 >>> Feb. I will zip it up and send it to you directly. >>> On 2/19/2020 4:52 PM, P.O. Jonsson wrote: >>>> Dear Gil, >>>> >>>> This is great news! I will gladly go over your version of rexxref.pdf with >>>> a magnifying glass (Beyond Compare) and compare it with the version I have >>>> made using Publican. >>>> >>>> If you want to do some comparisons as well I have all the Docs in my >>>> Dropbox here: >>>> >>>> https://www.dropbox.com/sh/p66c7g01h4jz5ss/AAAZd_Q2yQddrTHagxPo_UiTa?dl=0 >>>> <https://www.dropbox.com/sh/p66c7g01h4jz5ss/AAAZd_Q2yQddrTHagxPo_UiTa?dl=0> >>>> >>>> In a folder ooRexxDocs >>>> >>>> In order to make the comparison meaningful we should be using the same >>>> build, please let me know which one you have done and I will rebuild mine >>>> using the same. >>>> >>>> Hälsningar/Regards/Grüsse, >>>> P.O. Jonsson >>>> oor...@jonases.se <mailto:oor...@jonases.se> >>>> >>>> >>>> >>>>> Am 19.02.2020 um 22:29 schrieb Gil Barmwater <gbarmwa...@alum.rpi.edu >>>>> <mailto:gbarmwa...@alum.rpi.edu>>: >>>>> >>>>> And we have success! I found the reason for my previous failure and a way >>>>> around it that only involved one change to a parameter in the stylesheet. >>>>> At P.O.'s suggestion, I also bumped up the heap space for FOP to 1536Mb >>>>> (my laptop couldn't support 2Gb). The resulting PDF appears almost >>>>> identical to the most recent version on Source Forge - same number of >>>>> pages, no extra lines in the examples, railroad diagrams are good - but I >>>>> get more entries in the Table Of Contents. This has been the case with >>>>> all the documents I've built so I suspect something has changed in the >>>>> DocBook stylesheets; the Publican process uses an old version I believe >>>>> as I see one in the windows-build-tools directory on SVN while my process >>>>> retrieves it from the web. I suspect that most folks would prefer the >>>>> "current" style of TOC so I will continue to investigate this issue. If >>>>> anyone is interested in going over the rexxref PDF I built with a fine >>>>> tooth comb to see if there are other issues, I will zip it up and put it >>>>> in my Dropbox. In the meantime, I will update the package files that I've >>>>> modified in order to make this work and zip them up into a package that >>>>> folks can download and try. Stay tuned... >>>>> >>>>> Gil B. >>>>> On 2/17/2020 11:59 AM, P.O. Jonsson wrote: >>>>>> Dear Gil, >>>>>> >>>>>> Have you tried an even higher value? When I built using Publican it >>>>>> balked at 950 kb (value set be Erich I think) for rexxref so I raised it >>>>>> to 2 GB and then it passed. It is worth a try, memory is not a >>>>>> bottleneck nowadays :-) >>>>>> >>>>>> Hälsningar/Regards/Grüsse, >>>>>> P.O. Jonsson >>>>>> oor...@jonases.se <mailto:oor...@jonases.se> >>>>>> >>>>>> >>>>>> >>>>>>> Am 17.02.2020 um 15:12 schrieb Gil Barmwater <gbarmwa...@alum.rpi.edu >>>>>>> <mailto:gbarmwa...@alum.rpi.edu>>: >>>>>>> >>>>>>> An update on my progress is long overdue but Real Life sometimes gets >>>>>>> in the way! >>>>>>> >>>>>>> I have "put the pieces together" and zipped them up along with two >>>>>>> files of documentation and have been able to take that package to >>>>>>> another computer, install it and successfully build the rxmath book. I >>>>>>> also researched the article on Java heap space and found a way to >>>>>>> specify a larger value - currently using 1GB - without having to change >>>>>>> the FOP package. Then, because I know that folks will want to build the >>>>>>> rexxref book right away, I decided to try it, mainly to see if 1GB >>>>>>> would be large enough. And, of course, it failed! But the problem was >>>>>>> not with FOP but rather with the xsltproc step. It seems that the >>>>>>> Publican stylesheet is looking for a piece of Perl code which is >>>>>>> obviously not present. So I'm back in debug mode, trying to determine >>>>>>> what tag rexxref is using that wasn't used by rxmath and then what I >>>>>>> can do about it. If I can get the rexxref book to build, I will make >>>>>>> the tool package available so we can find any other problems that may >>>>>>> be lurking. >>>>>>> >>>>>>> Gil B. >>>>>>> >>>>>>> On 1/30/2020 10:26 AM, Rony G. Flatscher wrote: >>>>>>>> Dear Gil: >>>>>>>> >>>>>>>> thank you *very* much for this interesting and informative update! >>>>>>>> Looking forward to your tooling! :-) >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> Ad "Java heap space": just skim over >>>>>>>> <https://alvinalexander.com/blog/post/java/java-xmx-xms-memory-heap-size-control >>>>>>>> >>>>>>>> <https://alvinalexander.com/blog/post/java/java-xmx-xms-memory-heap-size-control>>. >>>>>>>> >>>>>>>> Maybe helpful: there are two command line help information given by >>>>>>>> Java, one ("java --help") the >>>>>>>> default help, and another giving extended help ("java -X") which >>>>>>>> documents the switches for >>>>>>>> controlling the heap size Java should reserve. >>>>>>>> >>>>>>>> Best regards >>>>>>>> >>>>>>>> ---rony >>>>>>>> >>>>>>>> >>>>>>>> On 29.01.2020 21:38, Gil Barmwater wrote: >>>>>>>>> Previously I wrote: One other bit of good news is that the >>>>>>>>> combination of these patches and the >>>>>>>>> Common_Content sub-folder work-around are the only required changes >>>>>>>>> in order to use the XSLTPROC >>>>>>>>> and FOP tools to successfully build our documents. I will describe >>>>>>>>> that process in my next post. >>>>>>>>> ... >>>>>>>>> >>>>>>>>> So this is that next post but I am replying to Rony's post as I >>>>>>>>> wanted to also address the >>>>>>>>> questions that he raised. The process I came up with is very similar >>>>>>>>> to that used with the >>>>>>>>> Publican tools - run a transform tool, either Publican or XSLTPROC, >>>>>>>>> to create an XSL-FO file from >>>>>>>>> our Docbook/XML files and a (modified) Docbook stylesheet. Run an >>>>>>>>> ooRexx program written by Erich >>>>>>>>> to remove extra blank lines in the .fo file. Run FOP to create a PDF >>>>>>>>> from the (modified) .fo file. >>>>>>>>> But as always, the devil is in the details. >>>>>>>>> >>>>>>>>> I chose XSLTPROC as several web sites suggested it although other >>>>>>>>> tools like Xalan were mentioned >>>>>>>>> as well. I was attempting to follow some step by step directions for >>>>>>>>> building a PDF from Docbook >>>>>>>>> source but, of course, those web sites are never up to date and I had >>>>>>>>> to adapt the directions as I >>>>>>>>> encountered problems. I also wanted to minimize the number of changes >>>>>>>>> to our Publican process as >>>>>>>>> we are generally happy with the results it produces. So substituting >>>>>>>>> XSLTPROC for Publican as the >>>>>>>>> XSL transform tool seemed a good starting point. Likewise, I kept the >>>>>>>>> Publican stylesheet - an >>>>>>>>> override to the standard Docbook stylesheet - that we had further >>>>>>>>> modified but I was able to >>>>>>>>> eliminate a part of it as Docbook had corrected a problem that it was >>>>>>>>> fixing, something to do with >>>>>>>>> footnote spacing. And, of course, I used the most current versions of >>>>>>>>> the tools that were >>>>>>>>> available, both for XSLTPROC and FOP (ver. 2.4). >>>>>>>>> >>>>>>>>> Now I know that some folks are "chomping at the bit" to replicate >>>>>>>>> what I have done but before you >>>>>>>>> run off and start searching for the tools to download, let me give >>>>>>>>> you a list of the "pieces" that >>>>>>>>> are needed. First there is the XSLTPROC transform tool: this is >>>>>>>>> actually 4 packages(!) which need >>>>>>>>> to be downloaded, unzipped, and the executable folders (bin) added to >>>>>>>>> the path. Then of course >>>>>>>>> there is the FOP package which needs to be downloaded, unzipped and >>>>>>>>> the appropriate sub-folder >>>>>>>>> added to the path. In order to get the same "look" to the documents >>>>>>>>> as produced by Publican, you >>>>>>>>> need to add some special fonts - 2 packages - to your system. And >>>>>>>>> then there are the two Publican >>>>>>>>> stylesheets, one of which has been modified, and a configuration file >>>>>>>>> for FOP so that it can find >>>>>>>>> the graphic files to be included and use the special fonts that were >>>>>>>>> installed. Finally, you need >>>>>>>>> to retrieve the blank-stripping program by Erich from the SVN >>>>>>>>> repository. And once you have all >>>>>>>>> the "pieces" in place, you need to checkout the latest version of the >>>>>>>>> documents from SVN, copy the >>>>>>>>> "common" folder to the working copy for the book you will be building >>>>>>>>> and add the fop >>>>>>>>> configuration file to it. Then you can run xsltproc, the blank-line >>>>>>>>> stripping program and then >>>>>>>>> FOP. Piece of cake! >>>>>>>>> >>>>>>>>> Because the above might seem overwhelming(!), I have been developing >>>>>>>>> a "package" that simplifies >>>>>>>>> it to a large degree. If you were to use this package, it contains >>>>>>>>> all the "pieces" and a set of >>>>>>>>> CMD files to execute the process steps. It is designed to be unzipped >>>>>>>>> into a folder that will >>>>>>>>> become the working location for building one or more? documents. >>>>>>>>> After installing it, you would >>>>>>>>> need to install the fonts (included) and then you could build a >>>>>>>>> document. The first cmd file to be >>>>>>>>> run is DOCPATH which takes one argument - the path to the SVN working >>>>>>>>> copy of the documents. That >>>>>>>>> path is saved in an environment variable for use by the remaining >>>>>>>>> steps. Then you run DOCPREP >>>>>>>>> which also takes one argument - the name of the "book" you want to >>>>>>>>> build, e.g. rxmath. It takes >>>>>>>>> care of creating the "Common_Content" sub-folder and adding the FOP >>>>>>>>> configuration file to it as >>>>>>>>> well as saving the document name in another environment variable. >>>>>>>>> Next you run DOC2FO which runs >>>>>>>>> the transform step. And finally, FO2PDF which runs FOP. The .fo file, >>>>>>>>> the .pdf file and a .log >>>>>>>>> file containing all the (many) messages from FOP are placed in a >>>>>>>>> sub-directory named e.g. out-rxmath. >>>>>>>>> >>>>>>>>> The cmd files are written and have been tested on the rxmath "book". >>>>>>>>> I need to put the pieces >>>>>>>>> together and zip them up which is my next step. Then I will provide a >>>>>>>>> link so anyone interested >>>>>>>>> can download it and give it a try. Note that I have NOT tried this on >>>>>>>>> any other "books" so I >>>>>>>>> expect there will be issues with some of them. E.g. as P.O. noted in >>>>>>>>> a different thread and >>>>>>>>> mentioned by Erich as well, the Java heap space needs to be increased >>>>>>>>> for some of our documents. I >>>>>>>>> do not know how to do that <blush> but it was not necessary for the >>>>>>>>> rxmath book. Any other issues >>>>>>>>> should be "book-related", not process-related and can be fixed as >>>>>>>>> they are uncovered. And any >>>>>>>>> process issues or enhancements I am willing to investigate. >>>>>>>>> >>>>>>>>> If it is the consensus that I should run this process on "all" the >>>>>>>>> documents before I release it, >>>>>>>>> i.e. actually do a full test(!), I would be willing to do so. >>>>>>>>> >>>>>>>>> Your thoughts and comments are welcome. >>>>>>>>> >>>>>>>>> Gil B. >>>>>>>>> >>>>>>>>> On 1/7/2020 9:28 AM, Rony G. Flatscher wrote: >>>>>>>>>> Hi Gil, >>>>>>>>>> >>>>>>>>>> any chance for your next posting to get an idea of what you have >>>>>>>>>> done and come up to? Maybe with a >>>>>>>>>> bird eyes's view how you now would suggest to create the >>>>>>>>>> documentation according to your analysis, >>>>>>>>>> tests? >>>>>>>>>> >>>>>>>>>> Also, would you have already suggestions for the software to use, >>>>>>>>>> e.g. xsltproc (how about using >>>>>>>>>> Apache Xalan [1] for this), the FOP is probably Apache FOP [2]. >>>>>>>>>> >>>>>>>>>> Guessing that everyone has been waiting eagerly for your next >>>>>>>>>> insights and directions of how to >>>>>>>>>> duplicate your efforts to successfully create the documentation! :) >>>>>>>>>> >>>>>>>>>> ---rony >>>>>>>>>> >>>>>>>>>> [1] Apache Xalan Project:<https://xalan.apache.org/ >>>>>>>>>> <https://xalan.apache.org/>> >>>>>>>>>> [2] Apache FOP:<https://xmlgraphics.apache.org/fop/ >>>>>>>>>> <https://xmlgraphics.apache.org/fop/>>u >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 06.01.2020 20:07, Gil Barmwater wrote: >>>>>>>>>>> This thread is a continuation of the thread titled "Questions ad >>>>>>>>>>> generating the documentation >>>>>>>>>>> (publican, pandoc)" with a different Subject since Pandoc is no >>>>>>>>>>> longer being considered as an >>>>>>>>>>> alternative. >>>>>>>>>>> >>>>>>>>>>> To review, the ooRexx documentation is written in DocBook and has >>>>>>>>>>> been turned into PDFs and HTML >>>>>>>>>>> files using a system called Publican, originally developed by >>>>>>>>>>> RedHat. Publican is no longer >>>>>>>>>>> supported and works only occasionally under Windows 10. Under the >>>>>>>>>>> covers, Publican transforms the >>>>>>>>>>> DocBook XML into XSL-FO using xsltproc, probably the Perl bindings >>>>>>>>>>> based on comments by Erich, and >>>>>>>>>>> modified DocBook stylesheets. It then runs the FOP program to >>>>>>>>>>> convert the xsl-fo output into a PDF >>>>>>>>>>> file. In between those two steps, we run a Rexx program written by >>>>>>>>>>> Erich to remove extra blank >>>>>>>>>>> lines from the examples. >>>>>>>>>>> >>>>>>>>>>> The new process uses the latest XSLTPROC programs directly along >>>>>>>>>>> with the latest version of FOP. >>>>>>>>>>> However, Publican imposes some unique structure to the DocBook XML >>>>>>>>>>> which must be accounted for. >>>>>>>>>>> Publican has the concept of a "brand" which lets one define common >>>>>>>>>>> text and graphics that should >>>>>>>>>>> appear the same in all of a project's documentation. One denotes >>>>>>>>>>> those common text/graphic files >>>>>>>>>>> in the XML by preceding their names with "Common_Content/". As >>>>>>>>>>> Publican merges the various parts >>>>>>>>>>> of the document together so that it can be transformed by the >>>>>>>>>>> stylesheets, it resolves any >>>>>>>>>>> references to Common_Content so that the correct file is merged >>>>>>>>>>> into the complete source. As this >>>>>>>>>>> process is unique to Publican, we must account for it in order to >>>>>>>>>>> use XSLTPROC instead. >>>>>>>>>>> >>>>>>>>>>> One approach we could take would be to replace Common_Content/ with >>>>>>>>>>> either a relative or absolute >>>>>>>>>>> path to the location in our source tree where the files actually >>>>>>>>>>> are located. For the sake of this >>>>>>>>>>> discussion, I will assume the working copy of the documentation has >>>>>>>>>>> been checked out to a >>>>>>>>>>> directory named docs. Then the main xml file for the rxmath book >>>>>>>>>>> would be located at >>>>>>>>>>> docs\rxmath\en-US\rxmath.xml. And the files referenced by >>>>>>>>>>> Common_Content would be in >>>>>>>>>>> docs\oorexx\en-US\. The relative path would then be >>>>>>>>>>> ..\..\oorexx\en-US\. The only problem with >>>>>>>>>>> this approach is the number of places this would need to be >>>>>>>>>>> changed. My analysis shows over 140 >>>>>>>>>>> locations in over 50 files. >>>>>>>>>>> >>>>>>>>>>> A more expedient approach, and the one I would advocate, is to >>>>>>>>>>> create a "temporary" sub-directory >>>>>>>>>>> for the purpose of building the documentation and then to copy >>>>>>>>>>> everything from docs\oorexx\en-US\ >>>>>>>>>>> into it. So if one were going to build the rxmath book, one would >>>>>>>>>>> create >>>>>>>>>>> docs\rxmath\en-US\Common_Content\ and copy into it. This allows >>>>>>>>>>> XSLTPROC to locate the files that >>>>>>>>>>> need to be merged without having to make any changes to our source. >>>>>>>>>>> The disadvantage is that one >>>>>>>>>>> needs to do this for each book being built. It is however a simple >>>>>>>>>>> step that can be done either >>>>>>>>>>> with File Explorer or automated using the xcopy or robocopy >>>>>>>>>>> commands. >>>>>>>>>>> >>>>>>>>>>> Having gotten by the Common_Content issue, running XSLTPROC reveals >>>>>>>>>>> another problem caused by the >>>>>>>>>>> way Publican does the merge of the Common_Content files which I >>>>>>>>>>> will describe in the next posting. >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Oorexx-devel mailing list >>>>>>>> Oorexx-devel@lists.sourceforge.net >>>>>>>> <mailto:Oorexx-devel@lists.sourceforge.net> >>>>>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel >>>>>>>> <https://lists.sourceforge.net/lists/listinfo/oorexx-devel> >>>>>>> >>>>>>> -- >>>>>>> Gil Barmwater >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Oorexx-devel mailing list >>>>>>> Oorexx-devel@lists.sourceforge.net >>>>>>> <mailto:Oorexx-devel@lists.sourceforge.net> >>>>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel >>>>>>> <https://lists.sourceforge.net/lists/listinfo/oorexx-devel> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Oorexx-devel mailing list >>>>>> Oorexx-devel@lists.sourceforge.net >>>>>> <mailto:Oorexx-devel@lists.sourceforge.net> >>>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel >>>>>> <https://lists.sourceforge.net/lists/listinfo/oorexx-devel> >>>>> -- >>>>> Gil Barmwater >>>>> _______________________________________________ >>>>> Oorexx-devel mailing list >>>>> Oorexx-devel@lists.sourceforge.net >>>>> <mailto:Oorexx-devel@lists.sourceforge.net> >>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel >>>>> <https://lists.sourceforge.net/lists/listinfo/oorexx-devel> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Oorexx-devel mailing list >>>> Oorexx-devel@lists.sourceforge.net >>>> <mailto:Oorexx-devel@lists.sourceforge.net> >>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel >>>> <https://lists.sourceforge.net/lists/listinfo/oorexx-devel> >>> -- >>> Gil Barmwater >>> _______________________________________________ >>> Oorexx-devel mailing list >>> Oorexx-devel@lists.sourceforge.net >>> <mailto:Oorexx-devel@lists.sourceforge.net> >>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel >>> <https://lists.sourceforge.net/lists/listinfo/oorexx-devel> >> >> >> >> >> _______________________________________________ >> Oorexx-devel mailing list >> Oorexx-devel@lists.sourceforge.net >> <mailto:Oorexx-devel@lists.sourceforge.net> >> https://lists.sourceforge.net/lists/listinfo/oorexx-devel >> <https://lists.sourceforge.net/lists/listinfo/oorexx-devel> > -- > Gil Barmwater > _______________________________________________ > Oorexx-devel mailing list > Oorexx-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/oorexx-devel
_______________________________________________ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
