Re: Alt-Design: Preliminary results FO tree build test
Keiron Liddle wrote:
The only questions I have at the moment:
- are markers handled properly, you mentioned something about that
earlier so maybe it is dealt with already
- what about arbitrary xml anywhere for extensions, is that still
possible (also instream-foreign-object but that is probably okay)
I know it is not a spec thing but it can enhance using FOP for many
users.
Marker handling is deferred until area tree construction. Not all of
the FOs that can have markers have been fitted with handling yet, but
the model for it at the FO tree building level is as follows (from
FoListBlock.java). The table-row comment, which I just noticed, is a
hang-over from another FO.
/** The number of markers on this FO. */
private int numMarkers = 0;
/** The number of list-items on this FO. */
private int numItems = 0;
/** The offset of 1st table-row within the children. */
private int firstItemOffset = -1;
while ((ev = xmlevents.expectStartElement
(FObjectNames.MARKER, XMLEvent.DISCARD_W_SPACE))
!= null) {
new FoMarker(getFOTree(), this, ev, stateFlags);
numMarkers++;
ev = xmlevents.getEndElement(xmlevents.DISCARD_EV, ev);
pool.surrenderEvent(ev);
}
There is no provision for extension elements, apart from the keeping
track of incoming elements with variant namespace declarations. In
terms of the inherent input validation of pull parsing, the checking of
foreign namespace elements could be inserted in the get/expect
processing of the FOs. The FO generation is already generalised (most
Fo? elements are nto named in the code, and are generated by the
makeFlowObject() method of FObjects, so generalising the validation of
foreign elements should be feasible. The semantics of such objects is
always going to be more of a problem.
Peter
--
Peter B. West [EMAIL PROTECTED] http://www.powerup.com.au/~pbwest/
Lord, to whom shall we go?
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, email: [EMAIL PROTECTED]
Re: Alt-Design: Preliminary results FO tree build test
Hi Peter, On Tue, 2002-12-10 at 03:30, Peter B. West wrote: Fop-devs, I have just run some quick test of property generation, to determine whether I was actually generating the property sets for the FOs. Although there are obviously still some bugs in property generation, the full property sets are being generated. I don't think that almost halving the memory usage of the FO tree build can be accounted for by unused variables in the maint code. Rather, it seems that the rewrite of FO tree building and property representation has achieved its design goal: significantly reduced memory usage. It must be somewhere, used variables maybe ;) It could well be mostly in the properties. Anyway that's great. The only questions I have at the moment: - are markers handled properly, you mentioned something about that earlier so maybe it is dealt with already - what about arbitrary xml anywhere for extensions, is that still possible (also instream-foreign-object but that is probably okay) I know it is not a spec thing but it can enhance using FOP for many users. To my surprise, it also runs faster, in spite of using an admittedly less efficient pull parsing method implemented over the top of SAX. Taking advantage of native pull parsing APIs when they become available (and the Neko pull parser is slated for release with Xerces soon) will increase the performance. Fast and low memory is good. Now if I can work out Forrest, I'll update the documentation. Peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: Alt-Design: Preliminary results FO tree build test
Peter B. West wrote: Keiron Liddle wrote: On Fri, 2002-12-06 at 03:47, Peter B. West wrote: If anyone else want to have a look I would be interested in the results. I am particularly interested in memory usage, which, prima facie, looks good. 9 megs total memory usage for 51 pages of FO tree sounds very encouraging to me, although I have called these preliminary because I am not certain that everything that needs to be created has been created. I don't know how much you know about it, but in HEAD Karen added whitespace handling that reduces the whitespace as it is processed (eg. when a block ends). I get about 17.5Mb for that document. I don't know what is using it all but there are lots of variables that are not being used. Unfortunately, since FOP was changed to trigger layout on the end element of a page-sequence, no complete FO tree is built in the current versions. There are, however, only two page sequences in the fo file, so the simple solution may be to remove the the smaller of these for the comparisons. Keiron, Using the current maint version, I made the following modifications: Forced GC for memory profiling Moved time calculation ahead of GC (it takes over 1.2 seconds) - in apps.StreamRenderer.java Commented out the call to render() Allowed page-sequence FO subtree to be added to the FO tree - in fo.FOTreeBuilder.java Compiled and run under j2sdk-1.4.1 and build.compiler=jikes. Obviously all of the renderer setup code is still present, including font setup, and as you say, there are no doubt variables which are initialised for the rendering. Results (fastest of three successive runs): [DEBUG] Input mode: [DEBUG] FO [DEBUG] fo input file: /home/pbw/public_html/xml/real-life-51-page-document.fo [DEBUG] Output mode: [DEBUG] pdf [DEBUG] output file: /tmp/51.pdf [DEBUG] OPTIONS [DEBUG] no user configuration file is used [default] [DEBUG] debug mode on [DEBUG] dump configuration [DEBUG] quiet mode on [INFO] Using org.apache.xerces.parsers.SAXParser as SAX2 Parser [INFO] base directory: file:/home/pbw/public_html/xml/ [INFO] FOP 0.20.5cvs [INFO] Using org.apache.xerces.parsers.SAXParser as SAX2 Parser [INFO] building formatting object tree [INFO] setting up fonts [INFO] Parsing of document complete, stopping renderer [DEBUG] Total time used: 17159ms [DEBUG] Pages rendered: 0 [DEBUG] Initial heap size: 497Kb [DEBUG] Current heap size: 16922Kb [DEBUG] Total memory used: 16425Kb The equivalent figures from my original post are: Initial heap size: 352Kb Current heap size: 8879Kb Total memory used: 8527Kb The comparable time is the elapsed time before preorder scan: 15960ms Fop-devs, I have just run some quick test of property generation, to determine whether I was actually generating the property sets for the FOs. Although there are obviously still some bugs in property generation, the full property sets are being generated. I don't think that almost halving the memory usage of the FO tree build can be accounted for by unused variables in the maint code. Rather, it seems that the rewrite of FO tree building and property representation has achieved its design goal: significantly reduced memory usage. To my surprise, it also runs faster, in spite of using an admittedly less efficient pull parsing method implemented over the top of SAX. Taking advantage of native pull parsing APIs when they become available (and the Neko pull parser is slated for release with Xerces soon) will increase the performance. Now if I can work out Forrest, I'll update the documentation. Peter -- Peter B. West [EMAIL PROTECTED] http://www.powerup.com.au/~pbwest/ Lord, to whom shall we go? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: Alt-Design: Preliminary results FO tree build test
Rhett Aultman wrote: I have a Solaris box (Sun Blade 100) here at home, but the sources should compile on Linux, I would think. I'm actually looking at integrating a jProf library into an MBean for the JBoss project, and I did all my preliminary work on my Solaris box (some of that research will be in the February issue of Java Developer's Journal, barring any catastrophes). I could run the tests over here, in theory. Maybe we should discuss the specifics of a test in private? Rhett, Please do. Peter -- Peter B. West [EMAIL PROTECTED] http://www.powerup.com.au/~pbwest/ Lord, to whom shall we go? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: Alt-Design: Preliminary results FO tree build test
On Fri, 2002-12-06 at 03:47, Peter B. West wrote: Rhett, Nerver having used it, I am not aware of its capabilities. As I don't develop in a Microsoft environment, and have no access to MS Visual C++, and I don't run in a Solaris environment, my options for trying this are limited. If anyone else want to have a look I would be interested in the results. I am particularly interested in memory usage, which, prima facie, looks good. 9 megs total memory usage for 51 pages of FO tree sounds very encouraging to me, although I have called these preliminary because I am not certain that everything that needs to be created has been created. I don't know how much you know about it, but in HEAD Karen added whitespace handling that reduces the whitespace as it is processed (eg. when a block ends). I get about 17.5Mb for that document. I don't know what is using it all but there are lots of variables that are not being used. Unfortunately, since FOP was changed to trigger layout on the end element of a page-sequence, no complete FO tree is built in the current versions. There are, however, only two page sequences in the fo file, so the simple solution may be to remove the the smaller of these for the comparisons. Peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: Alt-Design: Preliminary results FO tree build test
Keiron Liddle wrote: On Fri, 2002-12-06 at 03:47, Peter B. West wrote: Rhett, Nerver having used it, I am not aware of its capabilities. As I don't develop in a Microsoft environment, and have no access to MS Visual C++, and I don't run in a Solaris environment, my options for trying this are limited. If anyone else want to have a look I would be interested in the results. I am particularly interested in memory usage, which, prima facie, looks good. 9 megs total memory usage for 51 pages of FO tree sounds very encouraging to me, although I have called these preliminary because I am not certain that everything that needs to be created has been created. I don't know how much you know about it, but in HEAD Karen added whitespace handling that reduces the whitespace as it is processed (eg. when a block ends). I get about 17.5Mb for that document. I don't know what is using it all but there are lots of variables that are not being used. Unfortunately, since FOP was changed to trigger layout on the end element of a page-sequence, no complete FO tree is built in the current versions. There are, however, only two page sequences in the fo file, so the simple solution may be to remove the the smaller of these for the comparisons. Keiron, Using the current maint version, I made the following modifications: Forced GC for memory profiling Moved time calculation ahead of GC (it takes over 1.2 seconds) - in apps.StreamRenderer.java Commented out the call to render() Allowed page-sequence FO subtree to be added to the FO tree - in fo.FOTreeBuilder.java Compiled and run under j2sdk-1.4.1 and build.compiler=jikes. Obviously all of the renderer setup code is still present, including font setup, and as you say, there are no doubt variables which are initialised for the rendering. Results (fastest of three successive runs): [DEBUG] Input mode: [DEBUG] FO [DEBUG] fo input file: /home/pbw/public_html/xml/real-life-51-page-document.fo [DEBUG] Output mode: [DEBUG] pdf [DEBUG] output file: /tmp/51.pdf [DEBUG] OPTIONS [DEBUG] no user configuration file is used [default] [DEBUG] debug mode on [DEBUG] dump configuration [DEBUG] quiet mode on [INFO] Using org.apache.xerces.parsers.SAXParser as SAX2 Parser [INFO] base directory: file:/home/pbw/public_html/xml/ [INFO] FOP 0.20.5cvs [INFO] Using org.apache.xerces.parsers.SAXParser as SAX2 Parser [INFO] building formatting object tree [INFO] setting up fonts [INFO] Parsing of document complete, stopping renderer [DEBUG] Total time used: 17159ms [DEBUG] Pages rendered: 0 [DEBUG] Initial heap size: 497Kb [DEBUG] Current heap size: 16922Kb [DEBUG] Total memory used: 16425Kb The equivalent figures from my original post are: Initial heap size: 352Kb Current heap size: 8879Kb Total memory used: 8527Kb The comparable time is the elapsed time before preorder scan: 15960ms Peter -- Peter B. West [EMAIL PROTECTED] http://www.powerup.com.au/~pbwest/ Lord, to whom shall we go? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: Alt-Design: Preliminary results FO tree build test
Rhett, Nerver having used it, I am not aware of its capabilities. As I don't develop in a Microsoft environment, and have no access to MS Visual C++, and I don't run in a Solaris environment, my options for trying this are limited. If anyone else want to have a look I would be interested in the results. I am particularly interested in memory usage, which, prima facie, looks good. 9 megs total memory usage for 51 pages of FO tree sounds very encouraging to me, although I have called these preliminary because I am not certain that everything that needs to be created has been created. Unfortunately, since FOP was changed to trigger layout on the end element of a page-sequence, no complete FO tree is built in the current versions. There are, however, only two page sequences in the fo file, so the simple solution may be to remove the the smaller of these for the comparisons. Peter Rhett Aultman wrote: Hmmm...maybe we could use a JVM profiler like jProf (http://starship.python.net/crew/garyp/jProf.html) for this? -Original Message- From: Oleg Tkachenko [mailto:[EMAIL PROTECTED]] Peter B. West wrote: Herewith the preliminary results of running the alt.design FO tree build against the 51 page test document supplied by Bertrand. What is included in this measurement? I presume it's fo parsing, userconfig processing and fo tree buiding? Thanks for the file Bertrand. The file yields a tree of 11044 nodes. The pull buffer is set to 64, the maximum size of the FoXMLEvent pool is 74, and the total number of FoXMLEvent objects created is 78 (next event id -1). The sytem is a 256Mb PII laptop running Sun's Java 1.4.1 under Redhat 7.3. Elapsed time from the beginning of org.apache.fop.apps.Fop main to the end of the FO tree build is 15960. The preorder scan of the tree takes that to 16176. The test is run within a 'time' command with the results 18.02s real11.71s user 0.24s system It's rather interesting to compare it with the trunk code. But I believe we need real profiler software results to make any serious conclusions. -- Peter B. West [EMAIL PROTECTED] http://www.powerup.com.au/~pbwest/ Lord, to whom shall we go? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
RE: Alt-Design: Preliminary results FO tree build test
I have a Solaris box (Sun Blade 100) here at home, but the sources should compile on Linux, I would think. I'm actually looking at integrating a jProf library into an MBean for the JBoss project, and I did all my preliminary work on my Solaris box (some of that research will be in the February issue of Java Developer's Journal, barring any catastrophes). I could run the tests over here, in theory. Maybe we should discuss the specifics of a test in private? -Original Message- From: Peter B. West [mailto:[EMAIL PROTECTED]] Sent: Thu 12/5/2002 9:47 PM To: [EMAIL PROTECTED] Cc: Subject: Re: Alt-Design: Preliminary results FO tree build test Rhett, Nerver having used it, I am not aware of its capabilities. As I don't develop in a Microsoft environment, and have no access to MS Visual C++, and I don't run in a Solaris environment, my options for trying this are limited. If anyone else want to have a look I would be interested in the results. I am particularly interested in memory usage, which, prima facie, looks good. 9 megs total memory usage for 51 pages of FO tree sounds very encouraging to me, although I have called these preliminary because I am not certain that everything that needs to be created has been created. Unfortunately, since FOP was changed to trigger layout on the end element of a page-sequence, no complete FO tree is built in the current versions. There are, however, only two page sequences in the fo file, so the simple solution may be to remove the the smaller of these for the comparisons. Peter Rhett Aultman wrote: Hmmm...maybe we could use a JVM profiler like jProf (http://starship.python.net/crew/garyp/jProf.html) for this? -Original Message- From: Oleg Tkachenko [mailto:[EMAIL PROTECTED]] Peter B. West wrote: Herewith the preliminary results of running the alt.design FO tree build against the 51 page test document supplied by Bertrand. What is included in this measurement? I presume it's fo parsing, userconfig processing and fo tree buiding? Thanks for the file Bertrand. The file yields a tree of 11044 nodes. The pull buffer is set to 64, the maximum size of the FoXMLEvent pool is 74, and the total number of FoXMLEvent objects created is 78 (next event id -1). The sytem is a 256Mb PII laptop running Sun's Java 1.4.1 under Redhat 7.3. Elapsed time from the beginning of org.apache.fop.apps.Fop main to the end of the FO tree build is 15960. The preorder scan of the tree takes that to 16176. The test is run within a 'time' command with the results 18.02s real11.71s user 0.24s system It's rather interesting to compare it with the trunk code. But I believe we need real profiler software results to make any serious conclusions. -- Peter B. West [EMAIL PROTECTED] http://www.powerup.com.au/~pbwest/ Lord, to whom shall we go? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED] winmail.dat- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: Alt-Design: Preliminary results FO tree build test
Peter B. West wrote: Herewith the preliminary results of running the alt.design FO tree build against the 51 page test document supplied by Bertrand. What is included in this measurement? I presume it's fo parsing, userconfig processing and fo tree buiding? Thanks for the file Bertrand. The file yields a tree of 11044 nodes. The pull buffer is set to 64, the maximum size of the FoXMLEvent pool is 74, and the total number of FoXMLEvent objects created is 78 (next event id -1). The sytem is a 256Mb PII laptop running Sun's Java 1.4.1 under Redhat 7.3. Elapsed time from the beginning of org.apache.fop.apps.Fop main to the end of the FO tree build is 15960. The preorder scan of the tree takes that to 16176. The test is run within a 'time' command with the results 18.02s real11.71s user 0.24s system It's rather interesting to compare it with the trunk code. But I believe we need real profiler software results to make any serious conclusions. -- Oleg Tkachenko eXperanto team Multiconn Technologies, Israel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
RE: Alt-Design: Preliminary results FO tree build test
Hmmm...maybe we could use a JVM profiler like jProf (http://starship.python.net/crew/garyp/jProf.html) for this? -Original Message- From: Oleg Tkachenko [mailto:[EMAIL PROTECTED]] Sent: Tuesday, December 03, 2002 12:11 PM To: [EMAIL PROTECTED] Subject: Re: Alt-Design: Preliminary results FO tree build test Peter B. West wrote: Herewith the preliminary results of running the alt.design FO tree build against the 51 page test document supplied by Bertrand. What is included in this measurement? I presume it's fo parsing, userconfig processing and fo tree buiding? Thanks for the file Bertrand. The file yields a tree of 11044 nodes. The pull buffer is set to 64, the maximum size of the FoXMLEvent pool is 74, and the total number of FoXMLEvent objects created is 78 (next event id -1). The sytem is a 256Mb PII laptop running Sun's Java 1.4.1 under Redhat 7.3. Elapsed time from the beginning of org.apache.fop.apps.Fop main to the end of the FO tree build is 15960. The preorder scan of the tree takes that to 16176. The test is run within a 'time' command with the results 18.02s real11.71s user 0.24s system It's rather interesting to compare it with the trunk code. But I believe we need real profiler software results to make any serious conclusions. -- Oleg Tkachenko eXperanto team Multiconn Technologies, Israel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
Re: Alt-Design: Preliminary results FO tree build test
Oleg Tkachenko wrote: Peter B. West wrote: Herewith the preliminary results of running the alt.design FO tree build against the 51 page test document supplied by Bertrand. What is included in this measurement? I presume it's fo parsing, userconfig processing and fo tree buiding? Yes. It's rather interesting to compare it with the trunk code. But I believe we need real profiler software results to make any serious conclusions. It would be interesting. Peter -- Peter B. West [EMAIL PROTECTED] http://www.powerup.com.au/~pbwest/ Lord, to whom shall we go? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, email: [EMAIL PROTECTED]
