Re: [Oorexx-devel] Old Process for Building the ooRexx Documentation (Publican)

[Jean Louis Faucher]

The notes and tools in my GitHub repository are for the old format of 
documentation, before the work made by David to switch to Publican.
Nothing you can reuse for the current version.


> On 1 Feb 2020, at 19:49, Gil Barmwater  wrote:
> 
> Hi Rony,
> 
> In looking at the documentation files on SF, I saw a number of .css files 
> which I assumed were needed/used by the HTML build process. Perhaps all they 
> need is some tweaking to fix the problems P.O. is seeing?
> 
> I've forgotten but does Jean-Louis' process produce the same railroad 
> diagrams that we are using now? The link you provided just leads to the diary 
> of his work.
> 
> I agree that color coding the examples would be nice but I believe that is a 
> "feature" of DocBook that may require a different transform program. This 
> link  describes 
> how to do something like this but I have not delved into the details so I 
> don't know how one would tell the highlighting program that the code was 
> ooRexx and then how to color it. Sounds like a project for someone to 
> undertake after we get the new process up and running. It appears that each 
> example would require as least some modification and, by my count, there are 
> more than 6400(!) instances of the  tag in our documents. I 
> did that research because I know why* we are getting extra blank lines in our 
> examples and I wanted to see how hard it would be to fix. Based on the number 
> of instances, I decided to use Erich's program to strip the blanks instead.
> Finally, I have been working exclusively on Windows 10 since Erich noted that 
> Publican was not stable on the platform and we therefore needed some other 
> toolchain in order to be able to reliably produce our documentation for 
> ooRexx. I try to keep my systems up to date and a "ver" command gives:
> 
> Microsoft Windows [Version 10.0.18363.592]
> 
> *The reason we get leading blanks in our examples is that we are coding them 
> wrong! Docbook says that the contents of the  element are 
> treated "verbatim" and, because Docbook is XML and not HTML, whitespace is 
> preserved, including line breaks. So when we code the tag on one line but 
> start the example on the next line, the first "character" of the example is a 
> line break(!) and we get a blank line in the output. Simply starting the 
> example code immediately after the tag solves the problem. There were several 
> examples in the "common" files which I fixed that way along with the DTD 
> changes. 
> Gil
> On 2/1/2020 8:59 AM, Rony G. Flatscher wrote:
>> On 31.01.2020 22:09, P.O. Jonsson wrote:
>>> many thanks for your feedback, I read your thread and I think you gave me 
>>> just the hint I needed
>>> to inject Erichs cleaning script in to my build toolchain (it works 
>>> slightly different to Erichs
>>> setup). I will let you know if it worked.
>>> 
>>> When I compare what you write with the process I see using Publican the 
>>> differences are not very
>>> big, except you use more up-to-date tools, I use Publican 3.0 and fop 1.1 
>>> and something called
>>> LIBXSLT, an extension written in Python (in 2008!) to replace XSLTPROC that 
>>> you have chosen, so I
>>> have good hope that your toolchain will produce very similar results to 
>>> what we are used to.
>>> 
>>> What about the html - are they needed at all? They do not render very well 
>>> in comparison to the
>>> PDFs and the output looks different on different browsers (surprise), but 
>>> the main deficiency I
>>> see is that some diagrams overwrite text, they are simply to large in 
>>> comparison. Skipping the
>>> html would cut the rendering time with approx 40%.
>> Well, having HTML renderings would allow one to place all of the ooRexx 
>> documentation on the web and
>> allow it to be indexed by search machines, so I would think it to be very 
>> useful and makes ooRexx
>> more researchable, more visible. If you succeed in making the documentation 
>> an automated task on
>> Jenkins, then it should not be a problem at all. Additionally, changes to 
>> the documentation do not
>> occur frequently, such that time stamped regenerations would be relatively 
>> scarce.
>> 
>> Ad formatting: this could be fixed by defining a CSS (as a separate text 
>> file, referred to by the
>> HTML renderings in the head's style element) with formatting definitions for 
>> the HTML version, such
>> that all browsers render it the same. This could also be used to control the 
>> size and floating rules
>> regarding the diagrams.
>> 
>> Ad rendering code samples: it would be great if future versions (once the 
>> documentation can be
>> reliably created again on all platforms) would syntax highlight the code, 
>> also it would be great if
>> the syntax diagrams could be generated the way Jean-Louis Faucher has been 
>> doing it [1]. Maybe we
>> could ask Jean-Louis to give a hand then?
>> 
>>> The Windows 10 platform is there and it is just rolling its 

Re: [Oorexx-devel] Old Process for Building the ooRexx Documentation (Publican)

[Gil Barmwater]

Hi Rony,

In looking at the documentation files on SF, I saw a number of .css 
files which I assumed were needed/used by the HTML build process. 
Perhaps all they need is some tweaking to fix the problems P.O. is seeing?


I've forgotten but does Jean-Louis' process produce the same railroad 
diagrams that we are using now? The link you provided just leads to the 
diary of his work.


I agree that color coding the examples would be nice but I believe that 
is a "feature" of DocBook that may require a different transform 
program. This link 
 describes 
how to do something like this but I have not delved into the details so 
I don't know how one would tell the highlighting program that the code 
was ooRexx and then how to color it. Sounds like a project for someone 
to undertake after we get the new process up and running. It appears 
that each example would require as least some modification and, by my 
count, there are more than 6400(!) instances of the  tag 
in our documents. I did that research because I know why* we are getting 
extra blank lines in our examples and I wanted to see how hard it would 
be to fix. Based on the number of instances, I decided to use Erich's 
program to strip the blanks instead.


Finally, I have been working exclusively on Windows 10 since Erich noted 
that Publican was not stable on the platform and we therefore needed 
some other toolchain in order to be able to reliably produce our 
documentation for ooRexx. I try to keep my systems up to date and a 
"ver" command gives:


Microsoft Windows [Version 10.0.18363.592]

*The reason we get leading blanks in our examples is that we are coding 
them wrong! Docbook says that the contents of the  
element are treated "verbatim" and, because Docbook is XML and not HTML, 
whitespace is preserved, including line breaks. So when we code the tag 
on one line but start the example on the next line, the first 
"character" of the example is a line break(!) and we get a blank line in 
the output. Simply starting the example code immediately after the tag 
solves the problem. There were several examples in the "common" files 
which I fixed that way along with the DTD changes.


Gil

On 2/1/2020 8:59 AM, Rony G. Flatscher wrote:

On 31.01.2020 22:09, P.O. Jonsson wrote:

many thanks for your feedback, I read your thread and I think you gave me just 
the hint I needed
to inject Erichs cleaning script in to my build toolchain (it works slightly 
different to Erichs
setup). I will let you know if it worked.

When I compare what you write with the process I see using Publican the 
differences are not very
big, except you use more up-to-date tools, I use Publican 3.0 and fop 1.1 and 
something called
LIBXSLT, an extension written in Python (in 2008!) to replace XSLTPROC that you 
have chosen, so I
have good hope that your toolchain will produce very similar results to what we 
are used to.

What about the html - are they needed at all? They do not render very well in 
comparison to the
PDFs and the output looks different on different browsers (surprise), but the 
main deficiency I
see is that some diagrams overwrite text, they are simply to large in 
comparison. Skipping the
html would cut the rendering time with approx 40%.

Well, having HTML renderings would allow one to place all of the ooRexx 
documentation on the web and
allow it to be indexed by search machines, so I would think it to be very 
useful and makes ooRexx
more researchable, more visible. If you succeed in making the documentation an 
automated task on
Jenkins, then it should not be a problem at all. Additionally, changes to the 
documentation do not
occur frequently, such that time stamped regenerations would be relatively 
scarce.

Ad formatting: this could be fixed by defining a CSS (as a separate text file, 
referred to by the
HTML renderings in the head's style element) with formatting definitions for 
the HTML version, such
that all browsers render it the same. This could also be used to control the 
size and floating rules
regarding the diagrams.

Ad rendering code samples: it would be great if future versions (once the 
documentation can be
reliably created again on all platforms) would syntax highlight the code, also 
it would be great if
the syntax diagrams could be generated the way Jean-Louis Faucher has been 
doing it [1]. Maybe we
could ask Jean-Louis to give a hand then?


The Windows 10 platform is there and it is just rolling its thumbs between 
Windows build, I would
say 23 hours out of 24 it is idling.

This would mean that creating the HTML documentation would not really impact 
the machine (assuming
it is one of your kindly provided computers for Jenkins and automated creation 
of ooRexx [2]), once
the ooRexx documentation can be created on that platform as well?

BTW, did you try whether the toolset you are using would work on Windows 10 in 
addition to Windows 7?

Thinking of it, Gil, what 

Re: [Oorexx-devel] Old Process for Building the ooRexx Documentation (Publican)

[Gil Barmwater]

Re: HTML - I have not attempted to build anything but PDFs at this 
point. I'm guessing that the transform step can produce HTML directly if 
given a different stylesheet but have not attempted that. The principal 
proponent for having an HTML version was our late Secretary/Treasurer 
Les Koehler as he preferred using them due to his eyesight issues. The 
other place they are referenced is from the RexxLA website. My vote 
would be to only build them for the formal release of ooRexx and not for 
intermediate development builds.


Gil

On 1/31/2020 4:09 PM, P.O. Jonsson wrote:

Dear Gil,

many thanks for your feedback, I read your thread and I think you gave 
me just the hint I needed to inject Erichs cleaning script in to my 
build toolchain (it works slightly different to Erichs setup). I will 
let you know if it worked.


When I compare what you write with the process I see using Publican 
the differences are not very big, except you use more up-to-date 
tools, I use Publican 3.0 and fop 1.1 and something called LIBXSLT, an 
extension written in Python (in 2008!) to replace XSLTPROC that you 
have chosen, so I have good hope that your toolchain will produce very 
similar results to what we are used to.


What about the html - are they needed at all? They do not render very 
well in comparison to the PDFs and the output looks different on 
different browsers (surprise), but the main deficiency I see is that 
some diagrams overwrite text, they are simply to large in comparison. 
Skipping the html would cut the rendering time with approx 40%.


The Windows 10 platform is there and it is just rolling its thumbs 
between Windows build, I would say 23 hours out of 24 it is idling.


H??lsningar/Regards/Gr??sse,
P.O. Jonsson
oor...@jonases.se 



Am 31.01.2020 um 20:04 schrieb Gil Barmwater >:


Hi P.O.,

Just wanted to comment on some of the points you raised. First of 
all, you are right - the "oorexx" directory is NOT a real document 
but just a repository for the Common_Content in the other documents.


As I noted in the other thread, I have not run my process on anything 
other than the readme article and the rxmath book so other than 
those, I have nothing to provide. If you mean the changed source, not 
the PDF output, the only changes were to the Common_Content files 
which Erich has already committed to trunk. If you have done an SVN 
update before running your process, you have used my modified files.


As per the files you had to change, the ooconsole book is just a 
skeleton for a project started by Mark Miesfeld before he passed away 
so I would not worry about it. Perhaps some day someone will resume 
working on that project and they can deal with the documentation 
issues then. The other three books look like they are trying to pick 
up something from an SVN checkout - working copy - of more than just 
the documents part of the tree. I'll leave it to the other developers 
to decide if that is a valid thing to do.


From my examination of Erich's program, it appears he expects it to 
run from the directory containing the .fo file(s) and it rewrites 
that file it place after removing the extra blank lines. It searches 
for and modifies the first/only .fo file it finds.


Thanks for providing a platform that will reliably build our 
documentation while we get the new process up and running!


Gil B.

On 1/29/2020 11:20 AM, P.O. Jonsson wrote:

Dear all,

I am starting a new thread so as not to pollute Gil??s thread. This 
is just until we have a permanent solution.


I now have a working installation on Windows 7 that can reliably 
build the documentation, both as pdf and html. All-in-all I have 16 
books, the only ones that fail are ???BuildMachine??? that it seems 
no-one worked on since 2012 and ???oorexx??? that contains the 
Common_Content used by by all other books (not a book in itself?).


From optical inspection I can not find any major differences with 
the document on Sourceforge. I intend to use also Gil's ???modded??? 
versions and see if this results in a different output. @Gil, can 
you make them available from revision 11973?


I had to make a few changes to some of the documents, I have 
documented the changes in a diff file (attached).


I also had to raise the limit for Java Heap space for ooDialog to 
build (2080 pages!). I have set 2048kb, possibly 1024 will be 
sufficient, it stalled on 2048 pages with 980kb.


I have not ??used Erichs pre-cleaning Rexx script until now, 
information on how to use it (and on what files) is welcome.


The bad news is that the build of some of the documents takes a LOT 
of time, rexxref 30 minutes and oodialog 40 minutes building pdf and 
html on a very powerful machine (14,8 Megaclauses).


The good news is that the build of all documents can be made in 
parallel! I can launch 16 builds at the same time and most documents 
finish in 3-15 minutes. So there is hope for putting this on the 

Re: [Oorexx-devel] Old Process for Building the ooRexx Documentation (Publican)

[Rony G. Flatscher]

On 31.01.2020 22:09, P.O. Jonsson wrote:
> many thanks for your feedback, I read your thread and I think you gave me 
> just the hint I needed
> to inject Erichs cleaning script in to my build toolchain (it works slightly 
> different to Erichs
> setup). I will let you know if it worked.
>
> When I compare what you write with the process I see using Publican the 
> differences are not very
> big, except you use more up-to-date tools, I use Publican 3.0 and fop 1.1 and 
> something called
> LIBXSLT, an extension written in Python (in 2008!) to replace XSLTPROC that 
> you have chosen, so I
> have good hope that your toolchain will produce very similar results to what 
> we are used to.
>
> What about the html - are they needed at all? They do not render very well in 
> comparison to the
> PDFs and the output looks different on different browsers (surprise), but the 
> main deficiency I
> see is that some diagrams overwrite text, they are simply to large in 
> comparison. Skipping the
> html would cut the rendering time with approx 40%.

Well, having HTML renderings would allow one to place all of the ooRexx 
documentation on the web and
allow it to be indexed by search machines, so I would think it to be very 
useful and makes ooRexx
more researchable, more visible. If you succeed in making the documentation an 
automated task on
Jenkins, then it should not be a problem at all. Additionally, changes to the 
documentation do not
occur frequently, such that time stamped regenerations would be relatively 
scarce.

Ad formatting: this could be fixed by defining a CSS (as a separate text file, 
referred to by the
HTML renderings in the head's style element) with formatting definitions for 
the HTML version, such
that all browsers render it the same. This could also be used to control the 
size and floating rules
regarding the diagrams.

Ad rendering code samples: it would be great if future versions (once the 
documentation can be
reliably created again on all platforms) would syntax highlight the code, also 
it would be great if
the syntax diagrams could be generated the way Jean-Louis Faucher has been 
doing it [1]. Maybe we
could ask Jean-Louis to give a hand then?

> The Windows 10 platform is there and it is just rolling its thumbs between 
> Windows build, I would
> say 23 hours out of 24 it is idling.

This would mean that creating the HTML documentation would not really impact 
the machine (assuming
it is one of your kindly provided computers for Jenkins and automated creation 
of ooRexx [2]), once
the ooRexx documentation can be created on that platform as well?

BTW, did you try whether the toolset you are using would work on Windows 10 in 
addition to Windows 7?

Thinking of it, Gil, what platform have you been using for your work on a 
"future-safe" generation
of the ooRexx documentation, Windows 7, 8 or Windows 10, or a different 
platform?

Please keep up your great work!

---rony

[1] Jean-Louis Faucher's diary on creating the syntax diagrams for his 
experimental extensions to
ooRexx: 


[2] P.O. Jonsson's presentation on the 2019 International Rexx Symposium, 
"Jenkins - what is it and
how is it used for NetRexx/ooRexx":





___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

Re: [Oorexx-devel] ooRexx 5 Beta install errors

[rvm--- via Oorexx-devel]

Thank you, your build installs on Mint 18.3 and 19.3 for me.
I can't use it as it is missing ncurses, but it proves the point.
The dev build that fails for me is for Ubuntu 16.04 on which Linux Mint 18 is 
based.
I'd be interested to hear from the people who are downloading 16.04 - what is 
it working on?

P.O. Jonsson   wrote the following on 30/01/20 21:43:23:
> Dear rm (?)
> 
> You are using an Ubuntu build maybe there is something there that does not 
> work?
> 
> I have built ooRexx 5.0.0 from source tonight for Linux Mint 19.1, 64 bit 
> version an installer specifically 
for Linux mint. Please try it out.
> 
> In my dropbox 
>  I 
> add various 
builds that are not official builds, feel free to have a look there, but I only 
add files here occasionally
> 
> Make sure to uninstall/remove the previous installation of ooRexx before 
> installing this version
> 
> 
> 
> 
> Hälsningar/Regards/Grüsse,
> P.O. Jonsson
> oor...@jonases.se
> 
> 
> 
> > Am 30.01.2020 um 13:06 schrieb rvm--- via Oorexx-devel 
> > :
> > 
> > I am still unable to install the latest builds.
> > 
> > on ooRexx-5.0.0-11972.ubuntu1604.x86_64.deb I get :
> > Removing oorexx (5.0.0) ...
> > Removing oorexx alternatives
> > Processing triggers for man-db (2.7.5-1) ...
> > Processing triggers for desktop-file-utils (0.22+linuxmint1) ...
> > Processing triggers for mime-support (3.59ubuntu1) ...
> > Selecting previously unselected package oorexx.
> > (Reading database ... 485989 files and directories currently installed.)
> > Preparing to unpack .../ooRexx-5.0.0-11972.ubuntu1604.x86_64.deb ...
> > Unpacking oorexx (5.0.0) ...
> > Setting up oorexx (5.0.0) ...
> > update-alternatives: error: alternative path /usr/bin/rexx-oorexx doesn't 
> > exist
> > dpkg: error processing package oorexx (--install):
> > subprocess installed post-installation script returned error exit status 2
> > Processing triggers for man-db (2.7.5-1) ...
> > Errors were encountered while processing:
> > oorexx
> > 
> > on re-installing ooRexx-5.0.0-11937.ubuntu1604.x86_64.deb :
> > Removing oorexx alternatives
> > Unpacking oorexx (5.0.0) over (5.0.0) ...
> > dpkg: warning: unable to delete old directory '/usr/local/share/man/man1': 
> > Directory not empty
> > dpkg: warning: unable to delete old directory '/usr/local/share/man': 
> > Directory not empty
> > dpkg: warning: unable to delete old directory '/usr/local/share': Directory 
> > not empty
> > dpkg: warning: unable to delete old directory '/usr/local/lib': Directory 
> > not empty
> > dpkg: warning: unable to delete old directory '/usr/local/include': 
> > Directory not empty
> > Setting up oorexx (5.0.0) ...
> > update-alternatives: using /usr/bin/rexx-oorexx to provide /usr/bin/rexx 
> > (rexx) in auto mode
> > Processing triggers for desktop-file-utils (0.22+linuxmint1) ...
> > Processing triggers for mime-support (3.59ubuntu1) ...
> > Processing triggers for man-db (2.7.5-1) ...
> > 
> > I have been unable to install any builds since November.
> > I'm running Linux Mint 18.3 and 19.1 on desktops and Thinkpads, same error 
> > on all.
> > There's nothing in syslog.
> > 
> > 
> > ___
> > Oorexx-devel mailing list
> > Oorexx-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/oorexx-devel
> 
Dear rm (?)You are using an Ubuntu build maybe there is something there that does not work?I have built ooRexx 5.0.0 from source tonight for Linux Mint 19.1, 64 bit version an installer specifically for Linux mint. Please try it out.In my dropbox I add various builds that are not official builds, feel free to have a look there, but I only add files here occasionallyMake sure to uninstall/remove the previous installation of ooRexx before installing this version___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel