Re: Memory usage w/ FOP
Hi Philip,
I made a mailing for 14000 documents (roughly) i don't run it parallel,
buet sequencial. It takes me 2 hours and 15 mins now...
and this is not only a fop rendering i do, every document is reopened with
iText for some images and watermarks.
these documents are spread like this: 9000 of 2 pages, 3000 of 3 pages and
2000 of 4 and more pages.
my memory (on the taskmanager for the java.exe that does the whole process)
never goes higher then 36.000 Kb , and my cpu cycles are only used for 50%
so i would be able to riuns it twice @ the same time :D
if you want to be sure that your memory use won't jump to the sky make sure
you close every reference you don't need and garbage collect @ a regular
time...
i hope yur doing the same thing in java and if necessary i can help you
tweak the application
greetings,
Jochen Maes
ICT Development
KBC Securities (kbcsecurities.com)
Havenlaan 12 Avenue du Port SIF 8683
B-1080 Brussels
Belgium
Tel: +32 2 429 96 81
GSM: +32 496 57 90 99
E-mail : [EMAIL PROTECTED]
This message and any attachments hereto are for the named person's use
only. It may contain confidential, proprietary or legally privileged
information. You may not, directly or indirectly, use, disclose,
distribute, print, or copy any part of this message if you are not the
intended recipient. If you have received this e-mail message without being
the intended recipient, please notify KBC Securities promptly and delete
this e-mail. Any views expressed in this message are those of the
individual sender, except where the message states otherwise and the sender
is authorised to state them to be the views of KBC Securities. KBC
Securities reserves the right to monitor all e-mail communications through
its networks and any messages addressed to, received or sent by KBC
Securities or its employees are deemed to be professional in nature. The
sender or recipient of any messages to or of KBC Securities agrees that
those may be read by other employees of KBC Securities than the stated
recipient or sender in order to ensure the continuity of work-related
activities and allow supervision thereof. KBC Securities does not accept
liability for the correct and complete transmission of the information, nor
for any delay or interruption of the transmission, nor for damages arising
from the use of, or reliance on, the information.
"Philip S.
Constantinou"To: <[EMAIL PROTECTED]>
<[EMAIL PROTECTED]cc:
er.net> Subject: Re: Memory usage w/ FOP
24/06/2002
22:50
Please
respond to
fop-user
This really helps...
One of the things that I need to do is a mail merge creating thousands of
one page letters. If the pages are simple with no table of
contents/references/page numbers. Is it safe to assume that the FO
processor
will use roughly the same memory printing 10 letters as 1000?
- Original Message -
From: "J.Pietschmann" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Monday, June 24, 2002 1:44 PM
Subject: Re: Memory usage w/ FOP
> Philip S. Constantinou wrote:
> > Hi -
> > If I'm using an XSL Transformer and an XML ByteArrayOutputStream to
create a
> > FO formatted document which is rendered as PDF or printed, what is a
good
> > assumption to make about the memory utilization? Is the entire merged
FO
> > document s
Re: Memory usage w/ FOP
Philip S. Constantinou wrote: This really helps... One of the things that I need to do is a mail merge creating thousands of one page letters. If the pages are simple with no table of contents/references/page numbers. Is it safe to assume that the FO processor will use roughly the same memory printing 10 letters as 1000? It would be very hard to optimize momery consuption for one page documents. Apart from this, it depends whether you do it in parallel or sequentially. In case of the latter, FOP should always consume the same amount of memory if the documents are similar enough, regardless of complexity. If you do it in parallel, be prepared that 1000 parallel runs use 100 times the memory of 10 parallel runs. J.Pietschmann
Re: Memory usage w/ FOP
This really helps... One of the things that I need to do is a mail merge creating thousands of one page letters. If the pages are simple with no table of contents/references/page numbers. Is it safe to assume that the FO processor will use roughly the same memory printing 10 letters as 1000? - Original Message - From: "J.Pietschmann" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Monday, June 24, 2002 1:44 PM Subject: Re: Memory usage w/ FOP > Philip S. Constantinou wrote: > > Hi - > > If I'm using an XSL Transformer and an XML ByteArrayOutputStream to create a > > FO formatted document which is rendered as PDF or printed, what is a good > > assumption to make about the memory utilization? Is the entire merged FO > > document stored in memory or can the FO processor stream out the PDF or > > printed page? > > I'm not sure where the "XML ByteArrayOutputStream" > comes into play. > As for FOP itself: the processor has limited capabilities > for streaming, it also depends on the layout. > FOP starts reading the FO into an internal tree. After > the end of a page sequence, it starts rendering pages, > which can use a huge amount of memory. If you use forward > references to page numbers (like "page X of N"), every > page from the reference on is held in memory until the > element the reference points to is rendered. Elements > spanning multiple pages, in particular tables, are also > held in memory at least until the page it ends is completed. > Large images are also known as memory hogs. > > Some rules of thumb (roughly in this order) > - Don't use forward references. Put TOCs at the end. > - Use multiple page sequences. > - Avoid large images, scale them before feeding them to >FOP (unless print quality matters). For some formats, >in particular JPEG, the file size matters, not the >height*width, therefore choosing a higher compression >can help. > - Avoid large tables spanning multiple pages, use multiple >tables instead. > > HTH > J.Pietschmann > >
RE: Memory usage w/ FOP
> - Avoid large tables spanning multiple pages, use multiple >tables instead. very good tip!
Re: Memory usage w/ FOP
Philip S. Constantinou wrote: Hi - If I'm using an XSL Transformer and an XML ByteArrayOutputStream to create a FO formatted document which is rendered as PDF or printed, what is a good assumption to make about the memory utilization? Is the entire merged FO document stored in memory or can the FO processor stream out the PDF or printed page? I'm not sure where the "XML ByteArrayOutputStream" comes into play. As for FOP itself: the processor has limited capabilities for streaming, it also depends on the layout. FOP starts reading the FO into an internal tree. After the end of a page sequence, it starts rendering pages, which can use a huge amount of memory. If you use forward references to page numbers (like "page X of N"), every page from the reference on is held in memory until the element the reference points to is rendered. Elements spanning multiple pages, in particular tables, are also held in memory at least until the page it ends is completed. Large images are also known as memory hogs. Some rules of thumb (roughly in this order) - Don't use forward references. Put TOCs at the end. - Use multiple page sequences. - Avoid large images, scale them before feeding them to FOP (unless print quality matters). For some formats, in particular JPEG, the file size matters, not the height*width, therefore choosing a higher compression can help. - Avoid large tables spanning multiple pages, use multiple tables instead. HTH J.Pietschmann
Memory usage w/ FOP
Hi - If I'm using an XSL Transformer and an XML ByteArrayOutputStream to create a FO formatted document which is rendered as PDF or printed, what is a good assumption to make about the memory utilization? Is the entire merged FO document stored in memory or can the FO processor stream out the PDF or printed page? Are there any good rules of thumb? Thanks - Phil
