I do have a bit of experience with memory leaks (side effect of writing
high performance java code, unfortunately!)
I am pretty sure that this is not a memory leak, though. If this was a
memory leak, we would see calls happening multiple times before failure.
The failure happens on the very first call to create the buffered image:
BufferedImage image = new BufferedImage(rasterWidth, rasterHeight,
BufferedImage.TYPE_INT_ARGB);
Nothing is even getting put into the weakcache.
I think the core issue is that the raster data for the image is just too
big to store on the heap (without the heap being massive). It's
13152x13152x4 bytes (there is an alpha channel on this as well) - that's
almost 700MB - just for this one pattern.
Tilman wrote: "That
pattern has an XStep and YStep of 23438 although the image is 2148 x
440. Because of a matrix scale of 0.0673396 the image pattern size is
1578 x 1578 at 72 dpi. So at 1200 dpi the size would be about 26300 x
26300"
I am fairly familiar with PDF (I contributed the parsing library to iText
back in the day), but I'm not very familiar with rendering tile patterns,
so the above is not intuitive to me (yet). Is there some chance that
PdfBox is doing a lot more work than is necessary for this particular PDF?
Like is the matrix scale just absurdly bad in this file? For an image that
is originally 2148x440, it seems like requiring a 700MB raster should not
be necessary - but I do NOT have nearly enough experience with this area of
PDF...
Thanks,
- Kevin
Kevin Day
*trumpet**p| *480.961.6003 x1002
*e| *ke...@trumpetinc.com
www.trumpetinc.com <http://trumpetinc.com/>
LinkedIn <https://www.linkedin.com/company/trumpet-inc.>| Trumpet Blog
<http://trumpetinc.com/blog/>| Twitter <https://twitter.com/trumpetinc>
Proud to be Great Place To Work
<https://www.greatplacetowork.com/certified-company/7012667> Certified
2020-2021
On Fri, Nov 5, 2021 at 11:27 AM Tilman Hausherr <thaush...@t-online.de>
wrote:
> If you're experienced with memory leaks then it would be nice if you
> could search.
>
> Things I tried that didn't help:
> - calling graphics.setPaint(null) after operations (to "lose"
> TilingPaint objects)
> - disabling the (weak) cache of TilingPaint objects
> - adding finalize to see if the TilingPaint class gets finalized (yes)
> - adding finalize to see if the HighResolutionImageIcon class gets
> finalized (yes)
>
> Tilman
>
>
>
> Am 05.11.2021 um 06:37 schrieb Kevin Day:
> > Thanks for the analysis...
> >
> > So, how do we proceed? We are dealing with lots of different files from
> > lots of different sources, so custom tuning maxedge makes me nervous.
> >
> >
> > Is it worth trying BigBufferedImage? (I.e. would this even be accepted
> into
> > the code base?) "Effective Handling of Big Images in Java"
> >
> http://jobcardsystems.com/index.php/blog/29-effective-handling-of-big-images-in-java
> .
> > - license is Creative Commons CCO. If licensing is a problem, then
> > providing a hook to allow the BufferedImage creation to be overridden
> would
> > be needed.
> >
> >
> > Or should we be trying to profile to track down a memory leak?
> >
> > Awhile back, I dealt with native memory leaks in Java MappedByteBuffer
> > under windows, but this was a native resource leak, and that doesn't seem
> > like what we have here. And I don't know what the backing data store in
> > ByteBuffer actually is.
> >
> > K
> >
> >
> > On Thu, Nov 4, 2021, 9:52 PM Tilman Hausherr <thaush...@t-online.de>
> wrote:
> >
> >> Thanks for the files.
> >>
> >> I wonder if there is some memory leak, in PDFBox or in java itself. The
> >> H21 file has a pattern that contains an image (the company logo). That
> >> pattern has an XStep and YStep of 23438 although the image is 2148 x
> >> 440. Because of a matrix scale of 0.0673396 the image pattern size is
> >> 1578 x 1578 at 72 dpi. So at 1200 dpi the size would be about 26300 x
> >> 26300.
> >>
> >> You can limit this by setting pdfbox.rendering.tilingpaint.maxedge to a
> >> smaller value than the default of 3000. However this may have a bad
> >> influence on other files.
> >>
> >> I tried to show the H21 file in PDFDebugger at 1000% which would be 720
> >> dpi. This worked until the last image. But no problem when displaying
> >> the last image first. That is why I suspect a memory leak. I remember we
> >> noticed something similar years ago, BufferedImage objects that stayed
> >> despite nobody using them.
> >>
> >> Tilman
> >>
> >> Am 02.11.2021 um 20:06 schrieb Kevin Day:
> >>> Sorry it took so long to respond - I had to get permission to share
> >> these.
> >>> I did test with 2.0.24 and it did not help.
> >>>
> >>>
> >>> Here are some files that cause the problem:
> >>>
> >>>
> https://github.com/trumpetinc/boottest/blob/main/JrachvUniverse_H21.pdf
> >>>
> >>
> https://github.com/trumpetinc/boottest/blob/main/PP%20Opera%20Building_A53.pdf
> >>>
> >>> Here is some quick code that demonstrates the problem (update for your
> >>> printer name and PDF path):
> >>>
> >>>
> >>
> https://github.com/trumpetinc/boottest/blob/main/SimplifiedPdfBoxPdfPrinter.java
> >>>
> >>>
> >>> Maybe in TilingPaint#getImage(PageDrawer, PDTilingPattern,
> PDColorSpace,
> >>> PDColor, AffineTransform, Rectangle2D), the BufferedImage could
> >>> benefit from something similar to BigBufferedImage described in this
> >> post?
> >>> (
> >>>
> >>
> https://stackoverflow.com/questions/3854200/why-does-a-bufferedimage-require-so-much-memory-beyond-the-size-of-its-data-arra
> >>> ). Or maybe there is something in the scaling logic?
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Thu, Oct 28, 2021 at 11:12 AM Tilman Hausherr <
> thaush...@t-online.de>
> >>> wrote:
> >>>
> >>>> Huge buffered images can happen in patterns if the pattern matrix
> and/or
> >>>> the current transformation matrix result in a huge scale.
> >>>>
> >>>> You could retry with 2.0.24, but I'm pessimistic. Please share the PDF
> >>>> (upload to a sharehoster).
> >>>>
> >>>> Tilman
> >>>>
> >>>> Am 28.10.2021 um 01:17 schrieb Kevin Day:
> >>>>> Using PDFBox 2.0.20
> >>>>>
> >>>>> We use the following to print the PDF:
> >>>>>
> >>>>> HashPrintRequestAttributeSet attributes = new
> >>>>> HashPrintRequestAttributeSet();
> >>>>>
> >>>>> try(PDDocument doc = PDDocument.load(pdf,
> >>>>> MemoryUsageSetting.setupMixed(5000000L))){
> >>>>> job.setPageable(new PDFPageable(doc));
> >>>>> job.print(attributes);
> >>>>> }
> >>>>>
> >>>>> This works really well for thousands of PDFs, but we've run into one
> >>>>> particular PDF that causes an OutOfMemoryException.
> >>>>>
> >>>>>
> >>>>> The problem PDF is rendered (does not contain bitmaps).
> >>>>>
> >>>>> I've provided a huge amount of heap (over 1GB now), and it is still
> >>>> failing.
> >>>>> As near as I can tell from the stack trace (which I'll include
> below),
> >> it
> >>>>> seems like the problem is with creation of a huge buffered image.
> >>>>>
> >>>>> The physical page size in the PDF is 8.5 x 11".
> >>>>>
> >>>>> The PDFPrintable is configured as follows:
> >>>>>
> >>>>> Scaling is ACTUAL_SIZE
> >>>>> dpi is 0.0
> >>>>> subsamplingAllowed is false
> >>>>> renderingHints is null
> >>>>>
> >>>>>
> >>>>> Here is the stack trace:
> >>>>>
> >>>>> java.lang.OutOfMemoryError: Java heap space
> >>>>> at
> >>>>
> java.desktop/java.awt.image.DataBufferInt.<init>(DataBufferInt.java:75)
> >>>>> at
> >> java.desktop/java.awt.image.Raster.createPackedRaster(Raster.java:467)
> >>>>> at
> >>>>>
> >>
> java.desktop/java.awt.image.DirectColorModel.createCompatibleWritableRaster(DirectColorModel.java:1032)
> >>>>> at
> >>>>
> java.desktop/java.awt.image.BufferedImage.<init>(BufferedImage.java:333)
> >>>>> at
> >> org.apache.pdfbox.rendering.TilingPaint.getImage(TilingPaint.java:143)
> >>>>> at
> org.apache.pdfbox.rendering.TilingPaint.<init>(TilingPaint.java:103)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.rendering.TilingPaintFactory.create(TilingPaintFactory.java:60)
> >>>>> at
> org.apache.pdfbox.rendering.PageDrawer.getPaint(PageDrawer.java:351)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.rendering.PageDrawer.getNonStrokingPaint(PageDrawer.java:719)
> >>>>> at
> org.apache.pdfbox.rendering.PageDrawer.fillPath(PageDrawer.java:819)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.contentstream.operator.graphics.FillEvenOddRule.process(FillEvenOddRule.java:37)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:932)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:510)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:484)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.contentstream.PDFStreamEngine.showForm(PDFStreamEngine.java:187)
> >>>>> at
> >> org.apache.pdfbox.rendering.PageDrawer.showForm(PageDrawer.java:1462)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.contentstream.operator.graphics.DrawObject.process(DrawObject.java:86)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:932)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:510)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:484)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:156)
> >>>>> at
> org.apache.pdfbox.rendering.PageDrawer.drawPage(PageDrawer.java:271)
> >>>>> at
> >>>>>
> >>
> org.apache.pdfbox.rendering.PDFRenderer.renderPageToGraphics(PDFRenderer.java:431)
> >>>>> at
> org.apache.pdfbox.printing.PDFPrintable.print(PDFPrintable.java:246)
> >>>>> at
> >>>>>
> >>
> java.desktop/sun.print.RasterPrinterJob.printPage(RasterPrinterJob.java:2430)
> >>>>> at
> >>
> java.desktop/sun.print.RasterPrinterJob.print(RasterPrinterJob.java:1654)
> >>>>>
> >>>>> Next, I tried setting subsampling to true - this made no difference.
> >>>>>
> >>>>>
> >>>>> Next, I tried hard coding the dpi to 300f - this works with no
> >> problems.
> >>>>>
> >>>>> Next, I tried hard coding the dpi to 600f - this does not throw the
> >> heap
> >>>>> space error - but with strange results:
> >>>>> - I see "java.lang.OutOfMemoryError" on sys.err - it does NOT throw
> an
> >>>>> exception
> >>>>> - The pages come off the printer as blank pages
> >>>>>
> >>>>>
> >>>>> Does anyone have any suggestions on how we can print using the
> >> printer's
> >>>>> full resolution?
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: users-unsubscr...@pdfbox.apache.org
> >>>> For additional commands, e-mail: users-h...@pdfbox.apache.org
> >>>>
> >>>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscr...@pdfbox.apache.org
> >> For additional commands, e-mail: users-h...@pdfbox.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@pdfbox.apache.org
> For additional commands, e-mail: users-h...@pdfbox.apache.org
>
>
