Re: [Firebird-docs] Request for testing gradle build to replace ant build
I noticed we went off-list, so I'm including the list again. On 06-03-2020 12:08, Mark Rotteveel wrote: On 2020-03-06 11:50, Norman Dunbar wrote: Excellent thanks. I've cloned that branch and managed to build the docs. One thing I notice in the PDF created with: ./gradelw docbookPdf is the front-matter pages have interesting Roman numbers. I click onto the Command Line Utilities in the main contents and the pages start numbering at ccxcvii :) (And I don't read Roman very well these days either, so I've no idea what page that should be! It's not a problem, or a complaint, just an observation that amused me. I'd recommend not building the entire base as a single PDF ;), but instead use ./gradlew docbookPdf --docId=firebird-commandline-utilities or maybe the id of the specific subdoc. However, I'll double check what the old behaviour was in this case. The old docbook used roman numerals in the page itself, but not in the PDF pagination (there it used the physical page number, not the printed page number). That the PDF pagination now uses the roman numerals is probably a result of the newer docbook-xsl version or Apache FOP, or possibly a result of the following thing. In the old build, the page numbers would reset for each book, but in the new build they are continuous within the set. This is also the reason the roman numeral is so high. This is because of a change in 1.77.1 with description: """ Robert Stayton: pagesetup.xsl Change initial page number for book from 1 to auto so front cover and title pages are sequential, and so that book inside set will continue numbering. """ As full set PDFs are not how we normally build the PDFs, I think it is fine to ignore this difference. Mark -- Mark Rotteveel ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
Hi Mark, Paul, I've been attempting to get Gradle working on my box to try out the new build. Finally found time. :) Of course, it's not yet mnerged, I assume it's still in your fork Mark? If so, I'll get a few attempts at building done soon. Then I'll start experimenting with asciidoc/asciidoctor. If you need any questions, I might be able to help - having just done an entire 400 page book in Asciidoctor, using asciidoctor-pdf as the generator. One thing you will find useful is this: // If the backend is PDF then do this. ifdef::backend-pdf[] [width=60%, cols="20%,40%,20%,20%", align = "center", caption="Table 9."] endif::[] // If the backend is html, then do this instead. ifdef::backend-html5[] [width=60%, cols="20%,40%,20%,20%", float = "center", caption="Table 9."] endif::[] .2 - ADC prescaler settings and frequencies. |=== // Insert Table Contents Here // |=== I found that when generating tables in a PDF and HTML, they need different commands to centre them on the page. It's a bit of a bind that the widths cannot be written then the ifdef embedded, so you have to duplicate the widths etc in both options. Captions are interesting. The above would give a table the caption of "Table 9.2 - ADC prescaler settings and frequencies" Asciidoctor has manual table/listing/etc numbering so moving or adding sections will require lots of updating of captions etc. A right major pain! (Ask me how I know!) Good luck and have fun with asciidoctor. (My current book is being written in Lyx!) Cheers, Norm. -- Norman Dunbar Dunbar IT Consultants Ltd Registered address: 27a Lidget Hill Pudsey West Yorkshire United Kingdom LS28 7LG Company Number: 05132767 ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
On 2020-03-04 23:14, Paul Vinkenoog wrote: Hi Mark, I think I have all the parts in place to replace the ant build(*). I have updated the instructions in the docbuilding-howto on my branch. Is it OK to merge this? Yes, by all means. Good, I'll merge it this weekend (or maybe earlier if I find the time). Then I'll start experimenting with asciidoc/asciidoctor. *: I haven't written a task to replace the old fo2at task (which generates an Area Tree), although I have the parts in place to create one if it is necessary. Nor did I create a replacement for the fofix, docs, zip and the various cleanXXX tasks. That's fine, we can always recreate them later as gradle tasks should the need arise. The area tree was a temporary thing anyway, at a time when I was also making some contributions to the FOP project in order to get our Firebird PDF's right. Cheers, Paul Vinkenoog PS: Apologies for my consistently slow replies! No problem. I appreciate the feedback! Mark ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
Hi Mark, > I think I have all the parts in place to replace the ant build(*). I > have updated the instructions in the docbuilding-howto on my branch. > > Is it OK to merge this? Yes, by all means. > *: I haven't written a task to replace the old fo2at task (which > generates an Area Tree), although I have the parts in place to create > one if it is necessary. Nor did I create a replacement for the fofix, > docs, zip and the various cleanXXX tasks. That's fine, we can always recreate them later as gradle tasks should the need arise. The area tree was a temporary thing anyway, at a time when I was also making some contributions to the FOP project in order to get our Firebird PDF's right. Cheers, Paul Vinkenoog PS: Apologies for my consistently slow replies! ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
I think I have all the parts in place to replace the ant build(*). I have updated the instructions in the docbuilding-howto on my branch. Is it OK to merge this? After merging, the old ant-based build will be gone. As some of the XSL changes are incompatible, it didn't make sense to keep it around. Manual cleanup of the tools, lib, inter and dist folders will be necessary after this (though it won't hurt to keep them). I considered writing a gradle task to delete these folders, but I thought it would be simpler/better to do this manually. Mark *: I haven't written a task to replace the old fo2at task (which generates an Area Tree), although I have the parts in place to create one if it is necessary. Nor did I create a replacement for the fofix, docs, zip and the various cleanXXX tasks. -- Mark Rotteveel ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
On 20-02-2020 10:02, Mark Rotteveel wrote: I built an equivalent for ttcmetrics so I could fix the issue with the Japanese rendering. I ran into problems with doing it from Ant (probably due to Java version mismatches) and decided it was easier to just do it from Gradle than spend time investigating exactly why Ant misbehaved. I will add equivalent tasks for TTF and T1 this weekend. I have committed changes for tasks to generate font metrics and updated the Japanese font-metrics. As an example: gradlew ttfMetrics --fontFile=C:\Windows\Fonts\msgothic.ttc --ttcName="MS Gothic" --fontName="MSGothic" --metricsFileName=msgothic.xml However, in the fop-userconfig.xml, there is a comment that says: """ Make sure msmincho.xml contains no entry for unicode 8203 (zero-width space), or the layout will go wrong for text containing that character. Whenever you generate a new msmincho.xml, see if it contains this element: or something similar with us <= 8203 <= ue. If you find such an element, cut out 8203 by replacing the element with two ranges, in this case: Note that the first element keeps the original value for gi. The gi of the second element is calculated as follows: gi(2) = gi(1) + us(2) - us(1) """ After some experimentation, I didn't removed the 8203 character, as I didn't see anything wrong with rendering, but it might be easy to miss. Any idea what I should be looking for? In addition, the formula to derive the gi seems to be wrong. As far as I can tell from the steps between earlier entries, gi is a running count of the numbers of characters, and the actual formula should be gi(2) = gi(1) + 1 + ue(1) - us(1) Which would also suggest that you can't go around and just excise characters without introducing off-by-n errors for later entries. Given the fact Japanese hasn't been kept up-to-date for some time (and for example the release notes only build by commenting out things), is it really worthwhile to invest more time in trying to get this 'right'? Or could this be something that we can leave until a new Japanese document needs to be released? Mark -- Mark Rotteveel ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
Mark Rotteveel wrote: > I built an equivalent for ttcmetrics so I could fix the issue with the > Japanese rendering. I ran into problems with doing it from Ant (probably > due to Java version mismatches) and decided it was easier to just do it > from Gradle than spend time investigating exactly why Ant misbehaved. I > will add equivalent tasks for TTF and T1 this weekend. Sound good, thanks! And indeed we should spend no more time on Ant (unless someone should come up with a very good reason that we didn't think of). Cheers, Paul ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
Mark Rotteveel wrote: > I had some problems with the Japanese rendering. I had to install the > Japanese Windows fonts, and I had to rebuild the font metrics files, and > rename the fonts from MSMincho to MS-Mincho and MSGothic to MS-Gothic. > Now it looks ok for me (I haven't committed those changes yet though). Funny, just this afternoon I had the same experience trying to build the Japanese docs. Then I looked at the "Using non-Western fonts" doc and realized what the problem must be, but ran out of time. Will try again tonight. Talking about metrics files, have you added gradle tasks for the three metrics targets (t1metrics, ttfmetrics and ttcmetrics) or did you build them with ant for now? People are going to need them, so if they're not already present they'll have to be added sooner or later. Cheers, Paul Vinkenoog ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
On 11-02-2020 23:09, Paul Vinkenoog wrote: However... before burning our bridges we should do some more testing, especially with non-Western script sets (Japanese, Russian). I had some problems with the Japanese rendering. I had to install the Japanese Windows fonts, and I had to rebuild the font metrics files, and rename the fonts from MSMincho to MS-Mincho and MSGothic to MS-Gothic. Now it looks ok for me (I haven't committed those changes yet though). I'll double check the Russian rendering, although at first glance it looks OK, but maybe those metrics files need to be rebuilt as well. Mark -- Mark Rotteveel ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
Hi Mark, > > I built several HTML and PDF targets successfully. But I noticed some > > structural > > differences in the PDF's. The ones built with gradle opened with the > > navigation > > bookmarks initially all collapsed, and the 'Table of Contents' bookmark > > entirely > > missing. > > I found the cause for both: docbook-xsl 1.79.1 uses XSL 1.1 bookmarks by > default, instead of FOP specific bookmark handling. Setting > xsl1.1.bookmarks to 0 fixed the problem. Fix is now committed. Thanks for the fixes; I built some PDF's and HTML and it all looks fine to me. Cheers, Paul ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
Mark Rotteveel wrote: > I had noticed the changes in margin, but I actually thought it looked > better ;). So do I, so let's keep them that way :-) Paul ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
On 12-02-2020 11:11, Mark Rotteveel wrote: On 11-02-2020 23:00, Paul Vinkenoog wrote: Hi Mark, I built several HTML and PDF targets successfully. But I noticed some structural differences in the PDF's. The ones built with gradle opened with the navigation bookmarks initially all collapsed, and the 'Table of Contents' bookmark entirely missing. I found the cause for both: docbook-xsl 1.79.1 uses XSL 1.1 bookmarks by default, instead of FOP specific bookmark handling. Setting xsl1.1.bookmarks to 0 fixed the problem. Fix is now committed. Digging further, it looks like the title page on same level, and inclusion of TOC in the bookmarks is done by our customization in src/docs/xsl/fo/fop1.xsl. Mark -- Mark Rotteveel ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
On 11-02-2020 23:00, Paul Vinkenoog wrote: Hi Mark, I built several HTML and PDF targets successfully. But I noticed some structural differences in the PDF's. The ones built with gradle opened with the navigation bookmarks initially all collapsed, and the 'Table of Contents' bookmark entirely missing. I found the cause for both: docbook-xsl 1.79.1 uses XSL 1.1 bookmarks by default, instead of FOP specific bookmark handling. Setting xsl1.1.bookmarks to 0 fixed the problem. Fix is now committed. Mark -- Mark Rotteveel ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
On 11-02-2020 23:00, Paul Vinkenoog wrote: Hi Mark, I'm currently working on a gradle build to replace the current ant-based build for the documentation. It is a work-in-progress, but I'd really like some other eyes on its output. So, could some of you please try it out, look at its output and report any problems you see compared to the existing output of the ant build, or with its usage? I built several HTML and PDF targets successfully. But I noticed some structural differences in the PDF's. The ones built with gradle opened with the navigation bookmarks initially all collapsed, and the 'Table of Contents' bookmark entirely missing. As far as I can tell, the reason for the initial collapse seems to be that the book title has a different level, which means all sections are collapsed under the title, while in the old version the title was on the same level as the sections, and sections were collapsed individually. As to the missing bookmark of the Table of Contents, I have no idea yet. Also, in the gradle builds the vertical margin before the section headers was smaller. But maybe that's just because you adjusted some XSL parameters in your copy? I had noticed the changes in margin, but I actually thought it looked better ;). In any case, I did not touch any XSL parameters, although I'll double check if maybe I skipped one. I'd sooner suspect the reason is upgrading to docbook-xsl 1.79.1, or possibly my removal of fo/fo-fix.xsl from fo.xsl (as far as I know those fixes are no longer necessary with FO 2.4.0). I'll experiment with that. Anyway, I'm too tired now to hunt down the source of there differences. I'll have another look at it tomorrow. Thanks! -- Mark Rotteveel ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
Hello Mark, > Due to incompatible changes with regard to how the stylesheets are > specified and resolved (specifically the imports), the Ant build doesn't > work. (...) > I'm wondering if I should invest time to 'fix' that or not. I think it > is probably better to switch-over and tackle any issues that occur > instead of allowing both to exist side-by-side. Fully agreed. Your gradle setup clearly works, so let's concentrate on ironing out the wrinkles and not waste time on keeping a soon-to-be- abandoned system up. However... before burning our bridges we should do some more testing, especially with non-Western script sets (Japanese, Russian). And I want my Table of Contents back, dammit! ;-) Cheers, Paul ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
Hi Mark, > I'm currently working on a gradle build to replace the current ant-based > build for the documentation. > > It is a work-in-progress, but I'd really like some other eyes on its > output. So, could some of you please try it out, look at its output and > report any problems you see compared to the existing output of the ant > build, or with its usage? I built several HTML and PDF targets successfully. But I noticed some structural differences in the PDF's. The ones built with gradle opened with the navigation bookmarks initially all collapsed, and the 'Table of Contents' bookmark entirely missing. Also, in the gradle builds the vertical margin before the section headers was smaller. But maybe that's just because you adjusted some XSL parameters in your copy? Anyway, I'm too tired now to hunt down the source of there differences. I'll have another look at it tomorrow. Cheers, Paul ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
Re: [Firebird-docs] Request for testing gradle build to replace ant build
I made some changes, the tasks are now generic and the base set name is specified with a commandline option: gradlew docbookHtml will build the entire firebirddocs set (firebirddocs is the default baseName) gradlew docbookHtml --baseName=refdocs --docId=fblangref25 will build the Firebird 2.5 language reference Same for docbookPdf and docbookMonohtml I decided to include docbook in the task name in preparation of supporting asciidoc in the future. Due to incompatible changes with regard to how the stylesheets are specified and resolved (specifically the imports), the Ant build doesn't work. Using ant requires using relative paths, while the gradle-based builds uses a catalog manager to resolve stylesheets by their URI. For example to make ant work again, it is necessary to replace href="http://docbook.sourceforge.net/release/xsl/current/fo/docbook.xsl"/> with I'm wondering if I should invest time to 'fix' that or not. I think it is probably better to switch-over and tackle any issues that occur instead of allowing both to exist side-by-side. Mark On 07-02-2020 18:26, Mark Rotteveel wrote: I'm currently working on a gradle build to replace the current ant-based build for the documentation. It is a work-in-progress, but I'd really like some other eyes on its output. So, could some of you please try it out, look at its output and report any problems you see compared to the existing output of the ant build, or with its usage? The gradle version used requires Java 8 or higher. You can find this work-in-progress in the branch gradle-conv-attempt-3 in my fork of the repository https://github.com/mrotteveel/firebird-documentation Some examples: Building docs can be done with gradlew firebirddocsHtml building German docs can be done with gradlew firebirddocsHtml --language=de building the nullguide as PDF can be done with gradlew firebirddocsPdf --docId=nullguide (on Linux or MacOS you may have to use ./gradlew, in PowerShell, you may have to use .\gradlew) There are separate tasks for release notes, papers and reference docs. Beware, the commandline parameters are per task, so if for example you want html and pdf in German in one execute, you must use: gradlew firebirddocsPdf --language=de firebirddocsHtml --language=de To see all tasks: gradlew tasks --group=documentation Help for a specific task (is pretty sparse though): gradlew help --task=rlsnotesHtml Some important changes: * The gradle build does not use the lib/ and tools/ folders (though the XML currently still contains relative references to the tools folder, which I want to fix once I'm done) * The ant-build used some conditional behaviour that drove me a bit crazy to replicate when implementing this, so I changed the following to ensure consistency ** No more special handling for firebirddocs English set (so output is in - for example - build/html-firebirddocs, and not in build/html, and the root firebirddocs.xml has been moved into the firebirddocs/ folder) ** Every base-set now has an XSLT file (eg html-firebirddocs.xsl) without falling back to html.xsl if such a file does not exist * I removed the foxon usage, because it caused errors with newer versions of Apache FOP, and it wasn't immediately clear to me what this was for * Build uses docbook XSL 1.79.1 (ant build uses 1.72.0) * Apache FOP 2.4 (ant build uses 0.95-1 I think) Some things that I haven't worked on yet: * Language specific FOP configuration * Rewriting the readme * ... not verified everything Important caveat: this is a work in progress. Naming of tasks and options isn't final yet. For example, I am still considering changing this to use a single task per output type instead of a task per (output type) x (base name), and configure the base name through a commandline option (eg gradlew html --baseName=rlsnotes instead of gradlew rlsnotesHtml) I made some changes to the ant build to make it consistent with the changes I made for the firebirddocs base name, but I haven't actually tested that yet. Mark -- Mark Rotteveel ___ Firebird-docs mailing list Firebird-docs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/firebird-docs
