ds3v opened a new pull request #131:
URL: https://github.com/apache/pdfbox/pull/131
We are trying to use the PDFMergerUtility to merge a huge number of PDF
files(up to 50000 pages in summary), but are having an issue with large heap.
We have tried using setupTempFileOnly() but that doesn't seem to help.
We analyze heap dumps and found that main part of a heap is a current
page-buffers(arrays of 4096 bytes), that are referenced from ScratchFileBuffer.
The idea of this fix is to remove reference to page-buffer from
ScratchFileBuffer when COSStream completely processed.
Sample code:
```java
public class PdfBoxLargePdf
{
public static void main(String[] args) throws Exception
{
int fileCount = 2000;
List<Closeable> toBeClosed = new ArrayList<Closeable>(fileCount);
try
{
PDFMergerUtility utility = new PDFMergerUtility();
for (int i = 0; i < fileCount; i++)
{
FileInputStream fis = new FileInputStream(new
File("~/exchange/source.pdf"));
toBeClosed.add(fis);
utility.addSource(fis);
}
utility.setDestinationFileName("target/combined.pdf");
utility.mergeDocuments(MemoryUsageSetting.setupTempFileOnly());
}
finally
{
for (Closeable closeable : toBeClosed)
{
IOUtils.closeQuietly(closeable);
}
}
}
}
```
Sample uses pdfbox from 2.0 branch and runs with VM options “-Xmx1G
-XX:+HeapDumpOnOutOfMemoryError”.
In the branch 2.0 this code gets an OutOfMemoryError. Processed only 1058
source documents before OOM:
After fix this sample completed successfully. Processed all 2000 source
files:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
