Vincent -
I agree this is a work-around and does distort the semantics of the
fo:page-sequence element. When I opened up the FOP 0.95 Source last week, it
became apparent that trying to interject where FOP starts its layout would be
time consuming. Currently the handlers are directly tied to the FO to only
start rendering when a page-sequence is closed.
I would like to point out that the patched code does not start rendering
earlier than this. It simply provides a method of continuing the rendering
without a page break. Again, this does not conform to the semantics of a
page-sequence -- in that -- a page-sequence should be a set of pages that start
and end on its own pages. However, there was no other way (at least that I
could see) within the confines of the time constraints I had to provide a
work-around for FOP to manage the memory correctly.
So, having said this, the patch allows RTF and PDF Rendering to work in
original fashion or in the new modified approach. The default behavior is to a
page-sequence not trigger a page break at the end of its content. To enable
your document to render as it did pre-patch the page-sequence is specified as:
<fo:page-sequence break-after="page">
... Contents ....
</fo:page-sequence>
I understand why you are not going to blend this into Trunk. Maybe some of the
work provided in this patch can be used to provide a new mechanism for
improving the memory management of FOP in which rendering is not tied to the
end of page-sequences.
Cheers.
Ben.
-----Original Message-----
From: Vincent Hennebert [mailto:vhenneb...@gmail.com]
Sent: Thursday, June 04, 2009 7:35 AM
To: fop-dev@xmlgraphics.apache.org
Cc: Chris Fanjoy; Jody Brownell; Glen Campbell
Subject: Re: Apache FOP 0.95 Patch
Hi Ben,
Thank you very much for your interest in FOP and your contribution. I've
opened a Bugzilla issue containing your patch so that it can easily be
referred to:
https://issues.apache.org/bugzilla/show_bug.cgi?id=47314
It is likely to interest other users who run into similar memory issues,
and the good thing of having made it against the 0.95 release is that it
won't be made obsolete by further changes in the code.
We are not going to apply this patch to the Trunk, though. This is
a workaround that, although quite clever, distorts too much the
semantics of the fo:page-sequence element. A page sequence really is
a self-contained set of typographical material, that should start and
end on its own pages (the common analogy is the chapter of a book).
Rather, at some point we will have to tackle that artificial limitation
of starting the layout only when the end of the page sequence has been
reached. It's possible to start earlier, like you have somehow proved
it. Also, we are planning to implement several layout options, providing
different trade-off between speed/memory consumption and typographical
quality. Eventually it should no longer be necessary to split the
document into several page sequences to avoid memory issues.
Still, meanwhile your patch may save the lives of users who need a
quick solution to that problem.
I hope you understand.
Thanks again,
Vincent
Ben Wuest wrote:
> Hi -
>
>
>
> We recently integrated Apache FOP 0.95 with our software to perform the
> rendering of RTF and PDF reports. This integration was very quick and
> provided great results. However, due to the large amounts of data that our
> software is required to handle, we began experiencing Out of Memory problems
> with FOP. We researched this and sent letters to the user community and
> determined that what we were experiencing OOM issues because each of our
> reports existed in one page-sequence. We came to the conclusions from the
> community response, Web Forums, and analysis of the Apache FOP code itself
> that FOP reads to the end of a page sequence and then begins to render.
> With the large amounts of data ( 40 Mb FO files ) we quickly ran into
> scalability issues with one page-sequence per report. At this point we
> divided up our reports into multiple page-sequences only to find that FOP
> starts a new page on every page sequence and this behavior can not be changed
> (through the means of alterin
g the FO file). Page breaking at unpredictable locations (sometimes leaving
half or ¾ pages empty) made the report presentation visually unacceptable.
>
>
>
> We have modified the Apache 0.95 code for PDF and RTF Rendering and would
> like to offer this patch back to the community (the attached SVN diff is from
> the 0.95 release baseline). Listed below is an overview of the
> modifications that have been made.
>
>
>
> 1. Page Sequence Changes
>
>
>
> The handling of the break-after attribute was added to the page-sequence.
> This can only be set to auto (meaning that no page break will occur after the
> page-sequence) or page (meaning that a page-break will occur after the
> page-sequence). There wasn't another suitable attribute to use (from the
> xsl-fo standard) so break-after was employed.
>
>
>
> 2. PDF Rendering
>
>
>
> In PDF rendering, the page-sequence handling was modified to consider the new
> attribute on a page sequence (described in #1 above). The rendering was
> modified to save the last page of a page sequence for the next one and track
> the available BPD. In this fashion, when the next page sequence starts it
> checks for an unfinished page on the previous page sequence and uses that to
> begin its Page Breaking Algorithm. This required some small changes in the
> Page Break algorithm to read available BPD from carry over pages.
>
>
>
> 3. RTF Rendering
>
>
>
> In RTF Rendering, the page-sequence handling was also modified to consider
> the new attribute on a page sequence (described in #1 above). The RTFHandler
> was modified to start a document Area for every page sequence and to release
> the memory on page-sequence once it was complete (it is our impression from
> JProfiler analysis that it currently does not do this). We modified the
> RTFElement class to allow for flags as to whether to write the prefix or
> suffix. This enables the rendering to actually write the RtfFile on every
> page-sequence without closing the document. In addition this means that
> RtfFile is flushed (all memory) on every page-sequence (or RtfSection in the
> context of the rendering).
>
>
>
>
>
> The result of this allows us to render 7000 page RTF and PDF reports that
> contain continuous tables of over 20,000 records seamlessly (without page
> breaks in the table that are not at an end of page) without OOM errors.
>
>
>
> Regards,
>
>
>
> Ben Wuest
>
>
>
