subject:"AW\: Fop Memory Use"

AW: Fop Memory Use

2011-05-18 Thread Georg Datterl

Hi Eric,

That sounds interesting. If you run the transformer for each page and set a 
breakpoint after the first run, there (IMHO) should not be a reference to any 
fop object. Ignore the int[]s first, they are used everywhere. Concentrate on 
the fop objects which should not be there. You could as well run your 
transformation X times and then investigate all objects which exists exactly X 
(or Y*X) times in memory. Those  are probably accumulated over many runs and 
crash your application sooner or later.

Regards,

Georg Datterl

-- Kontakt --

Georg Datterl

Geneon media solutions gmbh
Gutenstetter Straße 8a
90449 Nürnberg

HRB Nürnberg: 17193
Geschäftsführer: Yong-Harry Steiert

Tel.: 0911/36 78 88 - 26
Fax: 0911/36 78 88 - 20

www.geneon.de

Weitere Mitglieder der Willmy MediaGroup:

IRS Integrated Realization Services GmbH:
www.irs-nbg.de
Willmy PrintMedia GmbH:  www.willmy.de
Willmy Consult & Content GmbH:   
www.willmycc.de

Von: Eric Douglas [mailto:edoug...@blockhouse.com]
Gesendet: Mittwoch, 18. Mai 2011 19:43
An: fop-dev@xmlgraphics.apache.org
Betreff: RE: Fop Memory Use

This test run isn't using SVG at all.  The PDFRenderer is working, the 
PNGRenderer runs out of memory, so it is using images but as output.
I already broke up the input to multiple FOs with multiple calls to the 
transform to generate a large document by combining small documents using the 
initial-page-number.
As the program runs it just keeps increasing memory use.
I tried running a profile with Java's VisualVM though I'm not sure what exactly 
I'm looking at or what to do with it.
The number one item showing memory hog in the profiler, as of my last snapshot 
was: class int[], live bytes 23,273,952 B, live objects 382, generations 10
After the program crashed with the profiler running I had an additional file 
opened, Java2DRenderer.class, so I'm assuming it's doing something that breaks 
PNGRenderer.
My class doesn't have any int[] references.
After that first reference the sizes drop off sharply in the profiler.  The 
next class reference is char[], then org.apache.fop.area.inline.SpaceArea.



From: Peter Hancock [mailto:peter.hanc...@gmail.com]
Sent: Wednesday, May 18, 2011 12:05 PM
To: fop-dev@xmlgraphics.apache.org
Subject: Re: Fop Memory Use
Hi Eric,

Does your document contain many large SVG's?  If so take a look at  Bugzilla 
#46360.  This issue was resolved in rev 997602 of FOP trunk.

Pete



On Wed, May 18, 2011 at 5:10 PM, Adrian Cumiskey 
mailto:adrian.cumis...@gmail.com>> wrote:
Hi Eric,

Fop calculates layout in page sequence chunks, so try breaking up your pages 
into chunks of page sequences.  Pages should be available for garbage 
collection once the page sequence has been rendered.

Cheers, Adrian.

On May 18, 2011, at 7:24 AM, Michael Rubin 
mailto:mru...@thunderhead.com>> wrote:

Just a wild thought. But is there a way you could possibly get the JVM to 
garbage collect between each run? Maybe that might free the memory up?

Thanks.

-Mike

On 18/05/11 13:20, Eric Douglas wrote:

I am using Fop 1.0.
I tried using Fop to transform a single document.  When I got a little over 100 
pages my FO file was over 5 MB.  The transform crashed with a Java heap out of 
memory error.

I managed to break the input down, as I'm using embedded code generating the 
input programmatically, and the PDF output is a lot smaller.

So I'm currently transforming 10 pages at a time, setting the 
initial-page-number to the next sequence (1, 11, 21, etc).

Then I save all the generated PDFs in memory and merge them using pdfbox.  So 
far this is working great.

I tried to do the same thing with the PNGRenderer, just calling a method to 
transform 10 pages at a time and save the output images in an array.

The PNGRenderer is created locally in the method.  It should be getting 
released when the method ends but the java process never releases any memory.

I tested a 90 page report and the memory use was over 1 GB.  I tested on 
another machine where the memory limit is apparently lower and it crashed on 
page 24.

Everything about the method to render to PNG is the same as the method to 
render to PDF aside from the Renderer.
Is there a problem with this renderer or something I could need to do different?




Michael Rubin


Developer


Fehler! Es wurde kein Dateiname angegeben.

Fehler! Es wurde kein Dateiname angegeben.


 T

 F

 M

 E

 W


+44 20 8238 7400

+44 20 8238 7401



mru...@thunderhead.com
www.thunderhead.com




Thunderhead featured in The Sunday Times Profit Track 100 league table of 
companies with fastest-growing profits. Click 
here to read more.

Fehler! Es wurde kein Dateiname 
angegeben.

AW: Fop Memory Use

2011-05-18 Thread Georg Datterl

Hi Eric,

On a windows system it's quite straight-forward. Run your application in your 
IDE. Stop at breakpoint. Run VisualVM. You will get a list of running java 
processes, including your application. A few clicks will get you a memory 
profile and there you can see a tree structure, where each object in memory 
references it's "parent". There's probably one single root object you have to 
free to solve your memory issues.

Regards,

Georg Datterl

-- Kontakt --

Georg Datterl

Geneon media solutions gmbh
Gutenstetter Straße 8a
90449 Nürnberg

HRB Nürnberg: 17193
Geschäftsführer: Yong-Harry Steiert

Tel.: 0911/36 78 88 - 26
Fax: 0911/36 78 88 - 20

www.geneon.de<http://www.geneon.de>

Weitere Mitglieder der Willmy MediaGroup:

IRS Integrated Realization Services GmbH:
www.irs-nbg.de<http://www.irs-nbg.de>
Willmy PrintMedia GmbH:  www.willmy.de<http://www.willmy.de>
Willmy Consult & Content GmbH:   
www.willmycc.de<http://www.willmycc.de>

Von: Eric Douglas [mailto:edoug...@blockhouse.com]
Gesendet: Mittwoch, 18. Mai 2011 16:43
An: fop-dev@xmlgraphics.apache.org
Betreff: RE: Fop Memory Use

If there's no obvious answer as to what Fop might be doing wrong or what I 
might be doing wrong that would be my next guess, except I've never done that 
so I'd have to figure out how.


From: Georg Datterl [mailto:georg.datt...@geneon.de]
Sent: Wednesday, May 18, 2011 10:25 AM
To: fop-dev@xmlgraphics.apache.org
Subject: AW: Fop Memory Use
Hi Eric,

I'd run a debugger (available with modern JDKs). Set a breakpoint after the 
first run and memory profiling should tell you, which objects are still in 
memory and why.

Regards,

Georg Datterl

-- Kontakt --

Georg Datterl

Geneon media solutions gmbh
Gutenstetter Straße 8a
90449 Nürnberg

HRB Nürnberg: 17193
Geschäftsführer: Yong-Harry Steiert

Tel.: 0911/36 78 88 - 26
Fax: 0911/36 78 88 - 20

www.geneon.de<http://www.geneon.de>

Weitere Mitglieder der Willmy MediaGroup:

IRS Integrated Realization Services GmbH:
www.irs-nbg.de<http://www.irs-nbg.de>
Willmy PrintMedia GmbH:  www.willmy.de<http://www.willmy.de>
Willmy Consult & Content GmbH:   
www.willmycc.de<http://www.willmycc.de>

Von: Eric Douglas [mailto:edoug...@blockhouse.com]
Gesendet: Mittwoch, 18. Mai 2011 16:12
An: fop-dev@xmlgraphics.apache.org
Betreff: RE: Fop Memory Use

When I tested over 130 pages the PNG render crashed with this dump.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
 at java.lang.AbstractStringBuilder.(AbstractStringBuilder.java:45)
 at java.lang.StringBuilder.(StringBuilder.java:92)
 at org.apache.fop.area.inline.SpaceArea.(SpaceArea.java:43)
 at org.apache.fop.area.inline.TextArea.addSpace(TextArea.java:82)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager$TextAreaBuilder.addSpaces(TextLayoutManager.java:578)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager$TextAreaBuilder.setText(TextLayoutManager.java:497)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager$TextAreaBuilder.build(TextLayoutManager.java:442)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager$TextAreaBuilder.access$1(TextLayoutManager.java:435)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager.addAreaInfoAreas(TextLayoutManager.java:369)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager.addAreas(TextLayoutManager.java:297)
 at 
org.apache.fop.layoutmgr.inline.InlineLayoutManager.addAreas(InlineLayoutManager.java:479)
 at 
org.apache.fop.layoutmgr.inline.LineLayoutManager.addInlineArea(LineLayoutManager.java:1561)
 at 
org.apache.fop.layoutmgr.inline.LineLayoutManager.addAreas(LineLayoutManager.java:1430)
 at 
org.apache.fop.layoutmgr.BlockLayoutManager.addAreas(BlockLayoutManager.java:389)
 at 
org.apache.fop.layoutmgr.AreaAdditionUtil.addAreas(AreaAdditionUtil.java:121)
 at 
org.apache.fop.layoutmgr.BlockContainerLayoutManager$BlockContainerBreaker.addAreas(BlockContainerLayoutManager.java:939)
 at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:626)
 at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:497)
 at 
org.apache.fop.layoutmgr.BlockContainerLayoutManager$BlockContainerBreaker.addContainedAreas(BlockContainerLayoutManager.java:965)
 at 
org.apache.fop.layoutmgr.BlockContainerLayoutManager.addAreas(BlockContainerLayoutManager.java:1158)
 at 
org.apache.fop.layoutmgr.BlockLayoutManager.addAreas(BlockLayoutManager.java:389)
 at 
org.apache.fop.layoutmgr.AreaAdditionUtil.addAreas(AreaAdditionUtil.java:121)
 at 
org.apache.fop.layoutmgr.FlowLayoutManager.addAreas(FlowLayoutManager.java:342)
 at org.apache.fop.layoutmgr.PageBreaker.addAreas(PageBreaker.java:280)
 at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:626)
 at org.apache.fop.layoutmgr.AbstractBreaker

AW: Fop Memory Use

2011-05-18 Thread Georg Datterl

Hi Eric,

I'd run a debugger (available with modern JDKs). Set a breakpoint after the 
first run and memory profiling should tell you, which objects are still in 
memory and why.

Regards,

Georg Datterl

-- Kontakt --

Georg Datterl

Geneon media solutions gmbh
Gutenstetter Straße 8a
90449 Nürnberg

HRB Nürnberg: 17193
Geschäftsführer: Yong-Harry Steiert

Tel.: 0911/36 78 88 - 26
Fax: 0911/36 78 88 - 20

www.geneon.de

Weitere Mitglieder der Willmy MediaGroup:

IRS Integrated Realization Services GmbH:
www.irs-nbg.de
Willmy PrintMedia GmbH:  www.willmy.de
Willmy Consult & Content GmbH:   
www.willmycc.de

Von: Eric Douglas [mailto:edoug...@blockhouse.com]
Gesendet: Mittwoch, 18. Mai 2011 16:12
An: fop-dev@xmlgraphics.apache.org
Betreff: RE: Fop Memory Use

When I tested over 130 pages the PNG render crashed with this dump.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
 at java.lang.AbstractStringBuilder.(AbstractStringBuilder.java:45)
 at java.lang.StringBuilder.(StringBuilder.java:92)
 at org.apache.fop.area.inline.SpaceArea.(SpaceArea.java:43)
 at org.apache.fop.area.inline.TextArea.addSpace(TextArea.java:82)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager$TextAreaBuilder.addSpaces(TextLayoutManager.java:578)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager$TextAreaBuilder.setText(TextLayoutManager.java:497)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager$TextAreaBuilder.build(TextLayoutManager.java:442)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager$TextAreaBuilder.access$1(TextLayoutManager.java:435)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager.addAreaInfoAreas(TextLayoutManager.java:369)
 at 
org.apache.fop.layoutmgr.inline.TextLayoutManager.addAreas(TextLayoutManager.java:297)
 at 
org.apache.fop.layoutmgr.inline.InlineLayoutManager.addAreas(InlineLayoutManager.java:479)
 at 
org.apache.fop.layoutmgr.inline.LineLayoutManager.addInlineArea(LineLayoutManager.java:1561)
 at 
org.apache.fop.layoutmgr.inline.LineLayoutManager.addAreas(LineLayoutManager.java:1430)
 at 
org.apache.fop.layoutmgr.BlockLayoutManager.addAreas(BlockLayoutManager.java:389)
 at 
org.apache.fop.layoutmgr.AreaAdditionUtil.addAreas(AreaAdditionUtil.java:121)
 at 
org.apache.fop.layoutmgr.BlockContainerLayoutManager$BlockContainerBreaker.addAreas(BlockContainerLayoutManager.java:939)
 at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:626)
 at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:497)
 at 
org.apache.fop.layoutmgr.BlockContainerLayoutManager$BlockContainerBreaker.addContainedAreas(BlockContainerLayoutManager.java:965)
 at 
org.apache.fop.layoutmgr.BlockContainerLayoutManager.addAreas(BlockContainerLayoutManager.java:1158)
 at 
org.apache.fop.layoutmgr.BlockLayoutManager.addAreas(BlockLayoutManager.java:389)
 at 
org.apache.fop.layoutmgr.AreaAdditionUtil.addAreas(AreaAdditionUtil.java:121)
 at 
org.apache.fop.layoutmgr.FlowLayoutManager.addAreas(FlowLayoutManager.java:342)
 at org.apache.fop.layoutmgr.PageBreaker.addAreas(PageBreaker.java:280)
 at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:626)
 at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:497)
 at org.apache.fop.layoutmgr.PageBreaker.doPhase3(PageBreaker.java:308)
 at org.apache.fop.layoutmgr.AbstractBreaker.doLayout(AbstractBreaker.java:450)
 at org.apache.fop.layoutmgr.PageBreaker.doLayout(PageBreaker.java:85)
 at 
org.apache.fop.layoutmgr.PageSequenceLayoutManager.activateLayout(PageSequenceLayoutManager.java:107)
 at 
org.apache.fop.area.AreaTreeHandler.endPageSequence(AreaTreeHandler.java:238)
 at org.apache.fop.fo.pagination.PageSequence.endOfNode(PageSequence.java:120)

AW: Fop Memory Use

AW: Fop Memory Use

AW: Fop Memory Use

3 matches

Site Navigation

Mail list logo

Footer information