Big/Huge XMLs
I havebig/huge XMLs, and I need to convertthem intoPDFs using FOP.Benchmarking the latest FOP gives poor results, both memory-wise andprocessing-wise. Its just taking too much time. The XML cannot really be broken down into chunks, as its all part of a report. And I need to process a lot of reports overnight, and I don't have a cluster at my disposal to distribute the load. Is there a way to speed up the processing time ? Best, Mohit Sharma
RE: Big/Huge XMLs
[I'll give J. a breather on this one] Assuming you've made certain that the bottleneck is in FOP and not your XSLT transformation, the only thing you can really do to help with very large reports is to break the PDFs into multiple page-sequences. IE start a new page-sequence every X nodes. If you have natural page break that you can use, no one will ever notice in the final output. Otherwise you may have awkward page breaks in your document. I have attached an XSLT stylesheet that we use to essentially break theXSL:FO into "chunks". Every 10 Investigator (=doctor) nodes we start a new page-sequence. Since this report has a page breakfor each new investigator anyway, the end result is no different. We increased our max PDF size on this report from 30 pages to 200 using this method, and seriously sped up rendering time for large reports. FYI - weare runningWeblogic on HP-UX with -hotspot, max-heap-size=512MB. For some reason, the Wintel JVM seems to perform a lot better than HP-UX on FOP, and in general. Note: as originally pointed out on this board, there is actually a slicker, and probably more efficient, way to do the node processing using a recurring template. ButI just found out how to do that and our bottleneck isn't in the XSLT so I haven't had any motiviation to go back and fix it. Here's the example: http://www.dpawson.co.uk/xsl/sect2/N4486.html#d4085e94I like Steve Tinney's solution to the grouping problem. Hope this helps, Matt -Original Message-From: Mohit Sharma [mailto:[EMAIL PROTECTED]Sent: Wednesday, May 21, 2003 6:34 PMTo: [EMAIL PROTECTED]Subject: Big/Huge XMLs I havebig/huge XMLs, and I need to convertthem intoPDFs using FOP.Benchmarking the latest FOP gives poor results, both memory-wise andprocessing-wise. Its just taking too much time. The XML cannot really be broken down into chunks, as its all part of a report. And I need to process a lot of reports overnight, and I don't have a cluster at my disposal to distribute the load. Is there a way to speed up the processing time ? Best, Mohit Sharma pvh_XmlToFo.xsl Description: pvh_XmlToFo.xsl - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Big/Huge XMLs
In addition to Matt's comments there is: http://xml.apache.org/fop/running.html#memory Also keep in mind that very high memory consumption often comes with a decrease in speed for Java. So reducing memory consumption may improve speed. Also avoid building a DOM for input. I you do that try changing to SAX event generation. On 22.05.2003 03:34:06 Mohit Sharma wrote: I have big/huge XMLs, and I need to convert them into PDFs using FOP. Benchmarking the latest FOP gives poor results, both memory-wise and processing-wise. Its just taking too much time. The XML cannot really be broken down into chunks, as its all part of a report. And I need to process a lot of reports overnight, and I don't have a cluster at my disposal to distribute the load. Is there a way to speed up the processing time ? Jeremias Maerki - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Graphic Problem
Hi, I wrote a program to retrieve some data from database and then generate a xml file by using JDOM. Then, I use FOP to generate the PDF file. It works fine but later I found out the image in the PDF file will not get refresh until I restart tomcat, even I tried to delete the PDF file and regenerate again, the problem still exists. If I delete the image from the folder and regenerate the PDF file, it get refreshed. If I put in the image which is not exist in the folder previously and regenerate the PDF file, it also get refreshed. The problem exists when I update the old image with a new image (same file name) and regenerate PDF, the image won't get refresh (i.e.: still showing the old image) until I restart tomcat. The tomcat I'm using is version 4.1.24. May I know is there any API that can allow me to force the image toget refreshed or this is a bug? Is it something wrong withmy tomcat configuration or fop? Is there any documentation or example that I can follow? I try to lookaround but couldn't get any.Thanks for any help...
Re: Graphic Problem
I've documented the behaviour recently, but the text hasn't made it to the main site, yet. Please follow the following URL for a solution: http://forrestbot.cocoondev.org/sites/xml-fop/graphics.html#caching I hope this helps. On 22.05.2003 09:13:07 Swee Yueng wrote: I wrote a program to retrieve some data from database and then generate a xml file by using JDOM. Then, I use FOP to generate the PDF file. It works fine but later I found out the image in the PDF file will not get refresh until I restart tomcat, even I tried to delete the PDF file and regenerate again, the problem still exists. If I delete the image from the folder and regenerate the PDF file, it get refreshed. If I put in the image which is not exist in the folder previously and regenerate the PDF file, it also get refreshed. The problem exists when I update the old image with a new image (same file name) and regenerate PDF, the image won't get refresh (i.e.: still showing the old image) until I restart tomcat. The tomcat I'm using is version 4.1.24. May I know is there any API that can allow me to force the image to get refreshed or this is a bug? Is it something wrong with my tomcat configuration or fop? Is there any documentation or example that I can follow? I try to look around but couldn't get any. Thanks for any help... Jeremias Maerki - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Graphic Problem
The Problem is located at FOPImageFactory. The m_urlMap caches the url of the Image to the produced FopImage (it doesn't check for last modified or something else, so you get the same image for the url until the image reference is cleared). Solution: in FOP 0.20.5rc2 you can call FopImageFactory.resetCache() in FOP 0.20.4 you have to implement it by yourself and recompile FOP ;( cu Torsten -Original Message-From: Swee Yueng [mailto:[EMAIL PROTECTED]Sent: Donnerstag, 22. Mai 2003 09:13To: [EMAIL PROTECTED]Subject: Graphic Problem Hi, I wrote a program to retrieve some data from database and then generate a xml file by using JDOM. Then, I use FOP to generate the PDF file. It works fine but later I found out the image in the PDF file will not get refresh until I restart tomcat, even I tried to delete the PDF file and regenerate again, the problem still exists. If I delete the image from the folder and regenerate the PDF file, it get refreshed. If I put in the image which is not exist in the folder previously and regenerate the PDF file, it also get refreshed. The problem exists when I update the old image with a new image (same file name) and regenerate PDF, the image won't get refresh (i.e.: still showing the old image) until I restart tomcat. The tomcat I'm using is version 4.1.24. May I know is there any API that can allow me to force the image toget refreshed or this is a bug? Is it something wrong withmy tomcat configuration or fop? Is there any documentation or example that I can follow? I try to lookaround but couldn't get any.Thanks for any help...
RE: Showing the name of the elements (user defined xml tag) in pd f using FOP
De : Zahidul Islam [mailto:[EMAIL PROTECTED] xsl:templates match=item_details xsl:apply-templates select=attributes/ /xsl:templates xsl:templates match=attributes xsl:apply-templates/ /xsl:templates If you use xsl:template match=attributes xsl:for-each select=* xsl:value-of select=name()/ xsl:text: /xsl:text xsl:apply-templates/ /xsl:for-each /xsl:template that should work better. Cheers, Dave. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
fo:leader and toc
Hi all,
i have a question concering the use of leaders in a toc to create some dots
between Chaptername and page number like:
1. Chapter1 .. 1
1.1 Subchapter 1.1 ... 2
and so on.
I have updated this day on fop0.20.5.rc3a and since then my leader doesn't
generate dots correctly.
Formerly i generated the dots with the following code.
fo:table-cell
fo:block text-align=left
fo:basic-link color=blue
internal-destination={concat($gfLinkId,generate-id())}
xsl:value-of select=@gp_bezeichnung/
/fo:basic-link
xsl:text /xsl:text
fo:leader leader-pattern=dots/
/fo:block
/fo:table-cell
fo:table-cell
fo:block text-align=right
fo:page-number-citation
ref-id={concat($gfLinkId,generate-id())}/
/fo:block
/fo:table-cell
The dots where generated up to the end of the table-cell.
Now the same code produces only 4 dots.
I have read in the spec the following:
If it is desired that the leader should stretch to fill all available space on
a line, the maximum length of the leader should be specified to be at least as
large as the column width
That's what i really want to do.
I have tested this with the actual fop-version, but the effect was that nothing
changed and only 4 dots are produced.
Thank you all in advance.
Mark
P.S.: The page-number alignment now works great, thanks to the developer who
does this :-))
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
diplaying text/image in particular (x,y) coordinate in FO
At first thanks for getting reply of my early request. I am now facing problem as i have to display all the text or images at there specific position determined by the x,y coordinate. For example, let i have to show the following : Name : Zahidul Islam My Image will Shown here Designation :Programmer here 'Name' will be at (x,y) = (200,100), 'Zahidul Islam' will be at (x,y) = (210,100), 'Designation' will be at (x,y) = (200,110), 'Programmer' will be at (x,y) = (215,110) and the image will be at (x,y) = (700,100). So is it possible to place thesedata at specified position if thecoordinates can be found from xml which is translated by xsl, xsl fo togenerate the pdf. waiting for reply. Do you Yahoo!? The New Yahoo! Search - Faster. Easier. Bingo.
RE: Tif image error
Title: RE: Tif image error Hello: I'm not sure, if you are on a windows box or not, but I have found that most windows imaging software can not handle the fax compression correctly and it inverts the colors. Try d-loading this browser plug-in to view your tiffs: http://www.alternatiff.com/ Hope this helps Rob -Original Message- From: Görz, Johannes [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 22, 2003 9:02 AM To: [EMAIL PROTECTED] Subject: Tif image error Hello, Im generating documents out of xml with FOP 0.20.5rc3. For images i installed JIMI and JAI like explained on the fop-website. The documents contain lots of images as .png and .tif. PNG works well, but i've got a problem with the TIFs. All of them appear in the generated PDF, but some of them are displayed with inverted colors. I compared the different image-files, but i found no common porperties which could be responsible for this strange behaviour. Neither the compression, the color-depth nor the resolution seem to have influence on that. The only difference i found was an error-message from fop, which occures every time a tif is processed: [INFO] Reverting to TIFF image handling through JAI: Error while loading image file:/C:/Projekte/xsl-fo/userproduction/nisu-38/input/28637964.tif : class org.apache.fop.image.TiffImage - unsupported samples per pixel value 3 [INFO] Reverting to TIFF image handling through JAI: Error while loading image file:/C:/Projekte/xsl-fo/userproduction/nisu-38/input/28638732.tif : class org.apache.fop.image.TiffImage - unsupported samples per pixel value 4 When the message occures with unsupported samples per pixel value 3 the image is displayed in the right way. With unsupported samples per pixel value 4 the result is inverted in the PDF file. Could anyone explain what this message exactly means? Did anyone have the same problems? Does anyone know how to solve this? Thanks for help... /johannes - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Big/Huge XMLs
Savino, Matt C wrote: We increased our max PDF size on this report from 30 pages to 200 Huh? What complications do you add to the layout to run out of memory at only *30* pages? I never had any problems until I got well past 1000 pages (using -mx128M, JDK 1.3.1) J.Pietschmann - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: diplaying text/image in particular (x,y) coordinate in FO
Zahidul Islam wrote: I am now facing problem as i have to display all the text or images at there specific position determined by the x,y coordinate. For example, let i have to show the following : Name : Zahidul Islam My Image will Shown here Designation : Programmer here 'Name' will be at (x,y) = (200,100), 'Zahidul Islam' will be at (x,y) = (210,100), 'Designation' will be at (x,y) = (200,110), 'Programmer' will be at (x,y) = (215,110) and the image will be at (x,y) = (700,100). Try an absolutely positioned block container: fo:block-container position=absolute top=200mm left=110mm height=10mm width=30mm fo:blockDesignation: pragrammer/fo:block fo:block-container IIRC the coordinates are counted from the top left of the page, but you'll surely find this out yourself. You must supply both height and width to the container. J.Pietschmann - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: fo:leader and toc
Mark Baier wrote: I have updated this day on fop0.20.5.rc3a and since then my leader doesn't generate dots correctly. See bug #19465. J.Pietschmann - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Nested Blocks Seem to Take Up Space
Using FOP 0.20.4. I have created a page-master with a certain width and height (768 x 1024) and no margins. I tried to fill this page with an image (PNG) - also of size 768 x 1024. If the image is enclosed in exactly one block - it works fine. However - if the image is enclosed inside of two blocks - a second page appears in my report that is completely blank. As can be seen in my example below - I tried setting all kinds of attributes on the block tags to 0px just to see if this would help the situation - but it didn't. Also - it seems that if I leave the width at 1024 - but shrink the height of the image to a maximum of 751 - the blank page will disappear. Any thoughts as to what is going on here? Thanks! A quick side question. The purpose of the outer block was originally to set up some default attributes for my document. I read somewhere that these could actually be placed inside fo:root instead. Unfortunately - I was unable to make this work. Can anyone confirm or deny that this should work? Do you know of any documentation on this? Thanks again for any help on this! Sample FO included below. Note: it refers to a .PNG file that was saved at 768 x 1024 (chart.png). I was cautioned away from including attachments - so I didn't include this. Hope that's not an issue. - fo:root xmlns:fo='http://www.w3.org/1999/XSL/Format' fo:layout-master-set !-- 1024 px x 768 px report, for Final Graphs Presentation. No margin. -- fo:simple-page-master master-name=FinalGraphsPage margin=0px page-height=768px page-width=1024px fo:region-body margin-left=0in margin-bottom=0in margin-right=0in margin-top=0in/ /fo:simple-page-master /fo:layout-master-set !-- Test Report -- !-- Final Graph #1 - Max Working Height=751 of 768-- fo:page-sequence xmlns:fo=http://www.w3.org/1999/XSL/Format; master-reference=FinalGraphsPage fo:flow flow-name=xsl-region-body fo:block space-before=0px space-after=0px margin=0px text-indent=0px padding=0px fo:block text-indent=0px padding=0px space-before=0px space-after=0px margin=0pxfo:external-graphic src='url(C:\chart.png)' width='1024px' height='768px' //fo:block /fo:block /fo:flow /fo:page-sequence /fo:root - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Big/Huge XMLs
30 pages is not our absolute max, but we have set a requirement that we have to be able to handle at least two concurrent reports, so we need some cushion. I have attached some FO for a report that runs around 50 pages, using only one page-sequence. I would really appreciate it if you coule run it through your FOP processor and tell me what kind of performance you see. It runs on our system, but two of these at once will come very close to an out of memory error. I have a feeling the large performance discrepancy may be because our report is one large table. Please let me know what you find. Below is the log output for a slightly larger (66 page) report also using only one page-sequence. I had to do some de-indefication to create the report I attached. But this log output is from the exact same report, just slightly different data. As you can see at -mx256m, we come close to an out of memory error on just this report. Once we implemented the page-sequence chunking though, full garbage collection knocks memory-in-use down to the range of 25-75MB depending on the report. This is a big improvement over the 215MB that you still see in use below even after a full GC. This was run on my Windows dev box (PIII 850Mhz 512MB ram) set to -mx256m. Memory performance and speed are always much worse on our production boxes( HP-UX, 2x550 RISC, 2GB ram) set to -mx512M -- especially on 2 or more concurrent reports. We're trying to talk our infrastructure police into letting us set up a Win2k server with Weblogic as a dedicated PDF report generator. No luck yet. [GC 41503K-39768K(216920K), 0.0604714 secs] [GC 41816K-40081K(216920K), 0.0734613 secs] (... FOP processing begins below ...) building formatting object tree setting up fonts [GC 42128K-40544K(216920K), 0.0598146 secs] [GC 42592K-41074K(216920K), 0.0308461 secs] (... many more partial garbage collections ...) [GC 113914K-112394K(216920K), 0.0217497 secs] [GC 114442K-112925K(216920K), 0.077 secs] [GC 114973K-113455K(216920K), 0.0222500 secs] [GC 115503K-113983K(216920K), 0.0219170 secs] (... FOP is done pre-processing, page generation begins ...) [1[GC 116031K-114464K(216920K), 0.0214575 secs] [GC 116512K-114883K(216920K), 0.0216715 secs] [GC 116931K-115305K(216920K), 0.0224520 secs] [GC 117353K-115722K(216920K), 0.0233072 secs] ] [2[GC 117770K-116113K(216920K), 0.0208135 secs] [GC 118161K-116531K(216920K), 0.0223947 secs] [GC 118579K-116951K(216920K), 0.0220651 secs] ][GC 118999K-117329K(216920K), 0.0211273 secs] [3[GC 119377K-117768K(216920K), 0.0220210 secs] [GC 119816K-118186K(216920K), 0.0219897 secs] [GC 120234K-118606K(216920K), 0.0223495 secs] (... pages 4-62 output and steady partial garbage collection ...) [63[GC 212110K-210967K(216920K), 0.0201827 secs] [GC 213011K-211792K(216920K), 0.0224984 secs] [GC 213370K-212929K(216920K), 0.0238419 secs] [GC 214977K-214223K(216920K), 0.0349902 secs] [GC 216271K-215075K(217464K), 0.0221023 secs] (... full garbage collection, at this point we know most memory in use is for FOP ...) (... IE - normal full garbage collection brings memory in use down to ~20MB ...) (... So at -mx256m, FOP is ~40MB away from an out of memory error ...) [Full GC 217123K-215866K(261888K), 5.5263608 secs] [GC 217207K-216127K(261888K), 1.4741779 secs] ] [64[GC 218175K-216477K(261888K), 0.0265551 secs] [GC 218525K-216896K(261888K), 0.0246568 secs] [GC 218944K-217318K(261888K), 0.0252940 secs] ][GC 219366K-217700K(261888K), 0.0236608 secs] [65[GC 219748K-218130K(261888K), 0.0244581 secs] [GC 220178K-218552K(261888K), 0.0248638 secs] [GC 220600K-218971K(261888K), 0.0303567 secs] ] [66[GC 221019K-219361K(261888K), 0.0230088 secs] ] Parsing of document complete, stopping renderer Initial heap size: 40908Kb Current heap size: 219991Kb Total memory used: 179083Kb Memory use is indicative; no GC was performed These figures should not be used comparatively Total time used: 66075ms Pages rendererd: 66 Avg render time: 1001ms/page -Original Message- From: J.Pietschmann [mailto:[EMAIL PROTECTED] Sent: Thursday, May 22, 2003 12:35 PM To: [EMAIL PROTECTED] Subject: Re: Big/Huge XMLs Savino, Matt C wrote: We increased our max PDF size on this report from 30 pages to 200 Huh? What complications do you add to the layout to run out of memory at only *30* pages? I never had any problems until I got well past 1000 pages (using -mx128M, JDK 1.3.1) J.Pietschmann - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] attachment: NAReportOutput.zip - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
