Appears to be solved.
I believe I've found where the memory went.
The PNGRenderer needs a lot of memory to transform large documents.
If I'm reading this right, from the results of JVisualVM, the memory use is the
pixel size.
The PNGRender stores every page as an image, which can be retrieved with
getPageImage(pageNumber).
That image translates to 72 pixels per inch.
For an 11 x 8.5 document that's 11 (792) x 8.5 (612) = 484704.
It appears the images contain those int[] arrays and use up that amount
(484704) x 4 + 16 bytes ( = 1938832)
I'm saving those images for print preview and loading them into a GUI window
with a zoom.
For the zoom I'm just resizing the image with Java's Graphics2D.drawImage.
If I redraw that to a larger size it gets a bit blurry. To help reduce that I
was sizing the initial image.
Fop generates a larger image if I set the methods on the FOUserAgent
(setSourceResolution(72), setTargetResolution(144)).
Plug that into the same calculation and it quadruples the memory use ((792 * 2
= 1584) * (612 * 2 = 1224)) * 4 + 16 = 7755280 bytes per page.
For a 100 page report that's a ton.
I am generating my own input (it's a custom report writer) so I know exactly
what fits on a page. I'm writing everything with hard page breaks and absolute
positioning.
I already had a memory problem just trying to create a PDF when I got around
150 pages so I solved that by breaking it up.
I now transform all output 10 pages at a time, creating multiple PDFs (in
memory) with xsl's initial-page-number then use pdfbox to put the pages
together.
The FO actually wasn't using much memory, just something Fop was doing when I
tried to use that FO to generate a parge PDF all at once.
So, I just save my FO files in an array and generate the PNG page images no
more than 10 at a time and I've limited memory use!
I'm using the PNG to split the process so I can generate output on one machine
(server) and display it on another (client).
If I only need to transform part of the document, and it's not taking much more
than the Graphics2D redraw, I can just call the render again for new page
requests so no more fuzzy images.
From: Georg Datterl [mailto:georg.datt...@geneon.de]
Sent: Thursday, May 19, 2011 12:22 AM
To: fop-dev@xmlgraphics.apache.org
Subject: AW: Fop Memory Use
Hi Eric,
That sounds interesting. If you run the transformer for each page and set a
breakpoint after the first run, there (IMHO) should not be a reference to any
fop object. Ignore the int[]s first, they are used everywhere. Concentrate on
the fop objects which should not be there. You could as well run your
transformation X times and then investigate all objects which exists exactly X
(or Y*X) times in memory. Those are probably accumulated over many runs and
crash your application sooner or later.
Regards,
Georg Datterl
-- Kontakt --
Georg Datterl
Geneon media solutions gmbh
Gutenstetter Straße 8a
90449 Nürnberg
HRB Nürnberg: 17193
Geschäftsführer: Yong-Harry Steiert
Tel.: 0911/36 78 88 - 26
Fax: 0911/36 78 88 - 20
www.geneon.de
Weitere Mitglieder der Willmy MediaGroup:
IRS Integrated Realization Services GmbH:www.irs-nbg.de
http://www.irs-nbg.de
Willmy PrintMedia GmbH: www.willmy.de
http://www.willmy.de
Willmy Consult Content GmbH: www.willmycc.de
http://www.willmycc.de
Von: Eric Douglas [mailto:edoug...@blockhouse.com]
Gesendet: Mittwoch, 18. Mai 2011 19:43
An: fop-dev@xmlgraphics.apache.org
Betreff: RE: Fop Memory Use
This test run isn't using SVG at all. The PDFRenderer is working, the
PNGRenderer runs out of memory, so it is using images but as output.
I already broke up the input to multiple FOs with multiple calls to the
transform to generate a large document by combining small documents using the
initial-page-number.
As the program runs it just keeps increasing memory use.
I tried running a profile with Java's VisualVM though I'm not sure what exactly
I'm looking at or what to do with it.
The number one item showing memory hog in the profiler, as of my last snapshot
was: class int[], live bytes 23,273,952 B, live objects 382, generations 10
After the program crashed with the profiler running I had an additional file
opened, Java2DRenderer.class, so I'm assuming it's doing something that breaks
PNGRenderer.
My class doesn't have any int[] references.
After that first reference the sizes drop off sharply in the profiler. The
next class reference is char[], then org.apache.fop.area.inline.SpaceArea.
From: Peter Hancock [mailto:peter.hanc...@gmail.com]
Sent: Wednesday, May 18, 2011 12:05 PM
To: fop-dev@xmlgraphics.apache.org
Subject: Re: Fop Memory Use
Hi Eric,
Does your document contain many large SVG's? If so take a look at Bugzilla
#46360. This issue was resolved in rev 997602 of FOP trunk.
Pete
On
