[
https://issues.apache.org/jira/browse/PDFBOX-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17019006#comment-17019006
]
Tilman Hausherr commented on PDFBOX-4739:
-----------------------------------------
In the worst case one will need more than 1GB. There is an issue somewhere
about a file that needed 10GB to render at 96dpi. This wasn't because of an
image, it was a huge content stream.
My own rendering tests at at 96dpi and I use -Xmx8g and I have 16 virtual cores.
I suggest a flexible approach:
- update your jdk to a good one. In my initial test jdk13 used less RAM than
jdk8. If you must use jdk8, then use the latest from Oracle or from amazon.
- if this is a local system then get a PC with as much RAM as possible you can
afford, this is a one time cost that is probably minimal compared to the cost
of people.
- if this is a cloud system then make tests with many files (e.g. from
http://downloads.digitalcorpora.org/corpora/files/govdocs1/zipfiles/ ) to see
what is best, accept that your application may crash, and have a fallback
solution.
I'm still interested in files with an extreme memory footprint, to find out
whether this is a bug on our side.
> Memory issues when rendering pdf to image
> -----------------------------------------
>
> Key: PDFBOX-4739
> URL: https://issues.apache.org/jira/browse/PDFBOX-4739
> Project: PDFBox
> Issue Type: Bug
> Components: Rendering
> Affects Versions: 2.0.18
> Reporter: Lior Yaffe
> Priority: Blocker
> Labels: how-to
> Attachments: linkedinceoresume.pdf
>
>
> So I'm trying to write a web service which performs OCR on an input pdf files.
> The code is very simple - convert the pdf to tiff files using PDFBox, and
> then use tesseract on the tiff files to get text.
> code is very straight forward:
>
> {code:java}
> private List<ByteArrayOutputStream> convertPdfToTiff2() throws IOException {
> List<ByteArrayOutputStream> fileList = new ArrayList<>();
> PDDocument doc = PDDocument.load(this.bytes);
> doc.setResourceCache(new EmptyCache());
> try {
> PDFRenderer pdfRenderer = new PDFRenderer(doc);
> for (int page = 0; page < doc.getNumberOfPages(); ++page) {
> BufferedImage bufferedImage =
> pdfRenderer.renderImageWithDPI(page, 300, ImageType.RGB);
> calcImageSize(bufferedImage);
> ByteArrayOutputStream os = new ByteArrayOutputStream();
> ImageIO.write(bufferedImage, "tiff", os);
> os.flush();
> os.close();
> bufferedImage.flush();
> bufferedImage = null;
> fileList.add(os);
> }
> } finally {
> doc.close();
> }
> return fileList;
> }
> {code}
>
> I'm trying to run a sample test which runs this concurrent with 5-6 different
> threads, but the app is crashing very fast.
>
> I did some memory tests, and it seems that while the input file is around 70
> kb, the
> {code:java}
> pdfRenderer
> {code}
> object is around 300 MB!! no matter if i'm changing the DPI level, the object
> is still very large.
> in addition, only if I'm calling the GC I see the memory drops, even if I'm
> closing the doc object....
>
> Basically when I'm running my server with -Xmx6GB with 6 threads in
> concurrent, after 3 runs the service is crashing....what am I missing here?
>
> * I attached the input pdf file
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
