Re: Improving FOP Performance?
Ram Krishnamoorthi wrote: Currently, We are just invoking FOP command directly from a Shell Script which would call the FOP Command This means you have the overhead of starting a JVM process for each individual invoice to process, including all the warm-up costs. You should think about a custom Java program which processes a larger batch of files, or about a server based solution. I would like to have your opinion on the Performance that we observe here. The Test Machine we run FOP on has 2 CPUs and 4 GB RAM. Our Production Machine that we would be running FOP on soon has 4 CPUs though There is an absolute limit on memory which a single JVM can manage. Check the appropriate docs on how to set the memory limits for your JVM. Optimize the settings in order to get most out of the available physical RAM. FOP uses a single thread for rendering a single PDF. You can only take advantage of multiple CPUs by either running multiple JVMs in parallel, or by using a custom Java program (command line or server) which spawns multiple threads. For small input and output, and an insignificant amount of images, FOP is processor bound rather than I/O-bound. Including large, high resolution images or large user font files (CJK) might turn FOP into an I/O-bound process. In any case there shouldn't be much harm in starting a few more threads than there are processors, unless you systematically run out of JVM memory. Tables take up more memory..do they Impact Performance significantly? Probably not. I'm not aware of a systematic study though. (3) I am using xsl:for-each in 3-4 places in the Stylesheet to loop through nodes with same Tag instead of using 'Template Match'..does that matter? FOP doesn't execute the transformation, you use an XSLT processor for this purpose, usually the XSLT processor in the Java run time library. Generally, for all XSLT processor commonly in use currently, xsl:for-each is as fast as using xsl:apply-templates, and it may be faster by a tiny, unnoticeable amount. Depending on the structure f the input XML and the complexity of the transformation, using xsl:apply-templates may be recommended for better maintainability. BTW xsl:for-each is *not* a loop in the same sense as in common procedural programming languages. Caching a Stylesheet Check the docs for javax.xml.transform.TransformerFactory.new Templates and javax.xml.transform.TransformerFactory.Templates.new Transformer. I doubt you'll notice much of an improvement unless your stylesheets are somewhat complex and/or larger than a few hundred lines. Are their other means to Improve Performance? It depends where the bottleneck is. In your current setup, eliminating the JVM startup cost is most likely what will get you the most bang for the buck. System utilities like vmstat and iostat, some simple instrumentation, or a more advanced profiling will show you where possible problems hide. J.Pietschmann - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving FOP Performance?
Hi Ram, Please post the codes / send me the sample code files (.fo,.xsl) for us to study.. Thanks Eugene Ram Krishnamoorthi [EMAIL PROTECTED] 11.02.2005 20:36 Please respond to [EMAIL PROTECTED] To [EMAIL PROTECTED] cc Subject Improving FOP Performance? Hello, We are using FOP to convert to generate PDF Invoices for our Customers. We need to generate roughly around 25,000 Invoices Per Month and the Number would increase each Month. We generate PDF Docs from XML Input Files. We had Trial runs to check for Performance and Memory usage. Currently, We are just invoking FOP command directly from a Shell Script which would call the FOP Command repeatedly but making sure that not more than 2 (or sometimes 3) FOP Processes are running at any instant. I would like to have your opinion on the Performance that we observe here. The Test Machine we run FOP on has 2 CPUs and 4 GB RAM. Our Production Machine that we would be running FOP on soon has 4 CPUs though (1) In our case, An Invoice could have Several Individual Sections. The Number of Sections could vary from 10 to as high as 10,000 (depending on the data present in the XML Input). On an Average, an Invoice is around 10-20 Pages..while a small % of them exceed 500 Pages. Currently, We have a Page Sequence for each Section. We need to include only One Small Image in each Invoice. (2) There is a need to keep certain information together in the same page, so I have used Tables often..on an Average an Invoice could have around 50-60 Tables, But the Tables are not big (2 or 3 rows) and have only one or 2 columns utmost. I do understand Tables take up more memory..do they Impact Performance significantly? (3) I am using xsl:for-each in 3-4 places in the Stylesheet to loop through nodes with same Tag instead of using 'Template Match'..does that matter? At this Point, I see about 30,000 Pages being generated in an Hour if I run 2 threads. I have not embedded FOP in a Java Program Yet..Plan to do that..I have seen 1 or 2 messages mentioning that Caching a Stylesheet would help. Can someone share their experience by doing that? Are their other means to Improve Performance? Thanks for your assistance Ram __ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving FOP Performance? -
this would also be my suggestion. at the moment we are also using fop as a command line script. when finishing the webapplication i will create a java servlet that runs under tomcat. first tests show a perfomance boost of 50%. mainly because the JVM must not be started every time, this overhead is being saved using tomcat and fop as java-servlet. hope to see a few more suggestions... markus -Jesiolowski, Dominik [EMAIL PROTECTED] wrote: - To: '[EMAIL PROTECTED]' [EMAIL PROTECTED] From: Jesiolowski, Dominik [EMAIL PROTECTED] Date: 02/11/2005 07:13AM Subject: RE: [***SPAM***] - Improving FOP Performance? - Bayesian Filter d etected spam Hello, We are using FOP to convert to generate PDF Invoices for our Customers. We need to generate roughly around 25,000 Invoices Per Month and the Number would increase each Month. We generate PDF Docs from XML Input Files. We had Trial runs to check for Performance and Memory usage. Currently, We are just invoking FOP command directly from a Shell Script which would call the FOP Command repeatedly but making sure that not more than 2 (or sometimes 3) FOP Processes are running at any instant. So you are starting, and stopping JVM, each time you run fop. Consider writing java app that invokes fop processing in a loop. Regards Dominik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Improving FOP Performance?
Hello, We are using FOP to convert to generate PDF Invoices for our Customers. We need to generate roughly around 25,000 Invoices Per Month and the Number would increase each Month. We generate PDF Docs from XML Input Files. We had Trial runs to check for Performance and Memory usage. Currently, We are just invoking FOP command directly from a Shell Script which would call the FOP Command repeatedly but making sure that not more than 2 (or sometimes 3) FOP Processes are running at any instant. I would like to have your opinion on the Performance that we observe here. The Test Machine we run FOP on has 2 CPUs and 4 GB RAM. Our Production Machine that we would be running FOP on soon has 4 CPUs though (1) In our case, An Invoice could have Several Individual Sections. The Number of Sections could vary from 10 to as high as 10,000 (depending on the data present in the XML Input). On an Average, an Invoice is around 10-20 Pages..while a small % of them exceed 500 Pages. Currently, We have a Page Sequence for each Section. We need to include only One Small Image in each Invoice. (2) There is a need to keep certain information together in the same page, so I have used Tables often..on an Average an Invoice could have around 50-60 Tables, But the Tables are not big (2 or 3 rows) and have only one or 2 columns utmost. I do understand Tables take up more memory..do they Impact Performance significantly? (3) I am using xsl:for-each in 3-4 places in the Stylesheet to loop through nodes with same Tag instead of using 'Template Match'..does that matter? At this Point, I see about 30,000 Pages being generated in an Hour if I run 2 threads. I have not embedded FOP in a Java Program Yet..Plan to do that..I have seen 1 or 2 messages mentioning that Caching a Stylesheet would help. Can someone share their experience by doing that? Are their other means to Improve Performance? Thanks for your assistance Ram __ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [***SPAM***] - Improving FOP Performance? - Bayesian Filter d etected spam
Hello, We are using FOP to convert to generate PDF Invoices for our Customers. We need to generate roughly around 25,000 Invoices Per Month and the Number would increase each Month. We generate PDF Docs from XML Input Files. We had Trial runs to check for Performance and Memory usage. Currently, We are just invoking FOP command directly from a Shell Script which would call the FOP Command repeatedly but making sure that not more than 2 (or sometimes 3) FOP Processes are running at any instant. So you are starting, and stopping JVM, each time you run fop. Consider writing java app that invokes fop processing in a loop. Regards Dominik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Improving FOP Performance? -
I would second Dominik's idead to invoke FOP programatically.Manoj-"Jesiolowski, Dominik" [EMAIL PROTECTED] wrote: -To: "'[EMAIL PROTECTED]'" [EMAIL PROTECTED]From: "Jesiolowski, Dominik" [EMAIL PROTECTED]Date: 02/11/2005 07:13AMSubject: RE: [***SPAM***] - Improving FOP Performance? - Bayesian Filter d etected spam Hello, We are using FOP to convert to generate PDF Invoices for our Customers. We need to generate roughly around 25,000 Invoices Per Month and the Number would increase each Month. We generate PDF Docs from XML Input Files. We had Trial runs to check for Performance and Memory usage. Currently, We are just invoking FOP command directly from a Shell Script which would call the FOP Command repeatedly but making sure that not more than 2 (or sometimes 3) FOP Processes are running at any instant.So you are starting, and stopping JVM, each timeyou run fop. Consider writing java app that invokesfop processing in a loop.RegardsDominik-To unsubscribe, e-mail: [EMAIL PROTECTED]For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Improving FOP Performance? -
Yes a third also ... Also, you might want to considering using a servlet engine like tomcat to take care of multi-threading issues rather than trying to write your own thread code (unless of course you like writing multi-threaded code ...:) . You would then modify your existing code to invoke the servlet rather than your shell script ... that way you won't have to worry about restarting the jvm as tomcat will have it running constantly ... the rest of your code (which limits the # of simultaneous transactions to three ) can probably stay the same .. Thanks, -Riz Rizwan Virk CTO CambridgeDocs [EMAIL PROTECTED] personal: [EMAIL PROTECTED] From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Fri 2/11/2005 11:30 AM To: [EMAIL PROTECTED] Subject: RE: Improving FOP Performance? - I would second Dominik's idead to invoke FOP programatically. Manoj -Jesiolowski, Dominik [EMAIL PROTECTED] wrote: - To: '[EMAIL PROTECTED]' [EMAIL PROTECTED] From: Jesiolowski, Dominik [EMAIL PROTECTED] Date: 02/11/2005 07:13AM Subject: RE: [***SPAM***] - Improving FOP Performance? - Bayesian Filter d etected spam Hello, We are using FOP to convert to generate PDF Invoices for our Customers. We need to generate roughly around 25,000 Invoices Per Month and the Number would increase each Month. We generate PDF Docs from XML Input Files. We had Trial runs to check for Performance and Memory usage. Currently, We are just invoking FOP command directly from a Shell Script which would call the FOP Command repeatedly but making sure that not more than 2 (or sometimes 3) FOP Processes are running at any instant. So you are starting, and stopping JVM, each time you run fop. Consider writing java app that invokes fop processing in a loop. Regards Dominik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] winmail.dat- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Improving FOP Performance? -
If the requirements include: speed, efficiency, portability, and high reliability for multi-threading; then Ada with the Ravenscar profile for rate-monotonic scheduling is the best solution. Please visit http://www.adaic.org/ and http://www.stsc.hill.af.mil/crosstalk/2003/11/0311dobbing.html. Aonix has also done some work on real-time Java. www.aonix.com Bob Leif _ From: Riz Virk [mailto:[EMAIL PROTECTED] Sent: Friday, February 11, 2005 9:17 AM To: [EMAIL PROTECTED] Subject: RE: Improving FOP Performance? - Yes a third also ... Also, you might want to considering using a servlet engine like tomcat to take care of multi-threading issues rather than trying to write your own thread code (unless of course you like writing multi-threaded code ...:) . You would then modify your existing code to invoke the servlet rather than your shell script ... that way you won't have to worry about restarting the jvm as tomcat will have it running constantly ... the rest of your code (which limits the # of simultaneous transactions to three ) can probably stay the same .. Thanks, -Riz Rizwan Virk CTO CambridgeDocs [EMAIL PROTECTED] personal: [EMAIL PROTECTED] _ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Fri 2/11/2005 11:30 AM To: [EMAIL PROTECTED] Subject: RE: Improving FOP Performance? - I would second Dominik's idead to invoke FOP programatically. Manoj -Jesiolowski, Dominik [EMAIL PROTECTED] wrote: - To: '[EMAIL PROTECTED]' [EMAIL PROTECTED] From: Jesiolowski, Dominik [EMAIL PROTECTED] Date: 02/11/2005 07:13AM Subject: RE: [***SPAM***] - Improving FOP Performance? - Bayesian Filter d etected spam Hello, We are using FOP to convert to generate PDF Invoices for our Customers. We need to generate roughly around 25,000 Invoices Per Month and the Number would increase each Month. We generate PDF Docs from XML Input Files. We had Trial runs to check for Performance and Memory usage. Currently, We are just invoking FOP command directly from a Shell Script which would call the FOP Command repeatedly but making sure that not more than 2 (or sometimes 3) FOP Processes are running at any instant. So you are starting, and stopping JVM, each time you run fop. Consider writing java app that invokes fop processing in a loop. Regards Dominik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] attachment: winmail.dat- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
