[
https://issues.apache.org/jira/browse/PDFBOX-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702818#comment-16702818
]
Ben Manes commented on PDFBOX-4389:
-----------------------------------
Interesting. Locally it takes 19 seconds with a single thread. However, I have
multiple threads (4) that are independently processing the same PDF with their
own PDDocument instance. In that case it is taking 40 seconds each, with a loop
of 2000. There must be a shared lock even though it should be independent.
{code:java}
public static void main(String[] args) throws Exception {
var executor = Executors.newFixedThreadPool(4,
new ThreadFactoryBuilder().setDaemon(true).build());
for (int i = 0; i < 2_000; i++) {
int count = i;
executor.execute(Unchecked.runnable(() -> {
Stopwatch stopwatch = Stopwatch.createStarted();
var memoryUsageSetting = MemoryUsageSetting
.setupMixed(2_000_000)
.setTempDir(Files.createTempDir());
Path path = Paths.get("/Users/ben/Downloads/aggregate.pdf");
logger.error("Opening {} at {}", count, stopwatch);
try (var document = PDDocument.load(path.toFile(), memoryUsageSetting)) {
logger.error("Opened {} in {}", count, stopwatch);
}
logger.error("Finished {} in {}", count, stopwatch);
}));
}
executor.shutdown();
executor.awaitTermination(5, TimeUnit.MINUTES);
}
{code}
{code}
ERROR [2018-11-29 07:47:20,434]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Opening 2 at 12.24 ms
ERROR [2018-11-29 07:47:20,430]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Opening 3 at 8.471 ms
ERROR [2018-11-29 07:47:20,434]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Opening 1 at 12.20 ms
ERROR [2018-11-29 07:47:20,434]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Opening 0 at 12.22 ms
ERROR [2018-11-29 07:47:38,703]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Opened 1 in 18.28 s
ERROR [2018-11-29 07:47:39,506]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Opened 0 in 19.08 s
ERROR [2018-11-29 07:47:39,520]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Opened 3 in 19.10 s
ERROR [2018-11-29 07:47:55,654]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Opened 2 in 35.23 s
ERROR [2018-11-29 07:47:56,289]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Finished 0 in 35.87 s
ERROR [2018-11-29 07:47:56,290]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Opening 4 at 331.6 μs
ERROR [2018-11-29 07:48:07,024]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Finished 1 in 46.60 s
ERROR [2018-11-29 07:48:07,026]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Opening 5 at 364.1 μs
ERROR [2018-11-29 07:48:19,757]
[co.loaddocs.service.entity.type.pdf.PdfComponent] Finished 3 in 59.33 s
...
{code}
> Excessive load times for large pdfs
> -----------------------------------
>
> Key: PDFBOX-4389
> URL: https://issues.apache.org/jira/browse/PDFBOX-4389
> Project: PDFBox
> Issue Type: Bug
> Affects Versions: 2.0.12
> Environment: OpenJDK 10, Ubuntu
> pdfbox v2.0.12
> jbig2-imageio v3.0.2
> Reporter: Ben Manes
> Priority: Major
> Attachments: PdfComponent.java
>
>
> We render preview images for pdfs being uploaded. This is usually quite fast,
> as often these are short PDFs (e.g. shipments). One customer has a habit of
> uploading 6,000+ pages, which I believe is their historicals. This can take a
> while, though I am currently seeing over a minute per page:
> {{Processed page 940 / 1930 for pdf 1d2c0351-6c1f-4198-bd0b-6728927d7d00
> within f1816bb9-3da2-4b61-a3d2-3ca9c419598e in 1.443 min}}
> The operation is safely parallelized by reading the number of pages,
> enqueuing a task per page index, opening the pdf in the task, and rendering
> the page index. Each task creates a new {{MemoryUsageSetting}} at 2mb memory
> an unlimited disk. When monitoring this upload, which will take 32 hours at
> this rate, the active scratch files are over 500mb.
> {{$ du -h
> /tmp/cache_12639792278559363345/session_2059639776597126303/f1816bb9-3da2-4b61-a3d2-3ca9c419598e/component/pdf/pdfbox/1d2c0351-6c1f-4198-bd0b-6728927d7d00
> | cut -f1 | sort -u}}
> {{2.3G}}
> {{4.0K}}
> {{524M}}
> {{531M}}
> {{552M}}
> {{653M}}
> When polling the stack traces, the threads appear to be spending most of
> their time on expanding the temp file for the per-page task's loading of the
> pdf(s).
> Can you explain why this is so slow? My hope is that it could traverse to the
> page quickly, render it, and close. In this case I might try refactoring to
> pool the opened documents instead of loading anew, as previously the image
> rendering was performance problem (since {{KcmsServiceProvider}} is no longer
> available).
>
> ----
> java.lang.Thread.State: RUNNABLE
> at java.io.RandomAccessFile.setLength([email protected]/Native Method)
> at org.apache.pdfbox.io.ScratchFile.enlarge(ScratchFile.java:245)
> locked <0x00000006f6268cc0> (a java.lang.Object)
> at org.apache.pdfbox.io.ScratchFile.getNewPage(ScratchFile.java:167)
> locked <0x00000006f6268f10> (a java.util.BitSet)
> at org.apache.pdfbox.io.ScratchFileBuffer.addPage(ScratchFileBuffer.java:126)
> at
> org.apache.pdfbox.io.ScratchFileBuffer.ensureAvailableBytesInPage(ScratchFileBuffer.java:184)
> at org.apache.pdfbox.io.ScratchFileBuffer.write(ScratchFileBuffer.java:236)
> at
> org.apache.pdfbox.io.RandomAccessOutputStream.write(RandomAccessOutputStream.java:46)
> at org.apache.pdfbox.cos.COSStream$2.write(COSStream.java:279)
> at org.apache.pdfbox.pdfparser.COSParser.readValidStream(COSParser.java:1299)
> at org.apache.pdfbox.pdfparser.COSParser.parseCOSStream(COSParser.java:1127)
> at org.apache.pdfbox.pdfparser.COSParser.parseFileObject(COSParser.java:913)
> at
> org.apache.pdfbox.pdfparser.COSParser.parseObjectDynamically(COSParser.java:874)
> at
> org.apache.pdfbox.pdfparser.COSParser.parseObjectDynamically(COSParser.java:794)
> at org.apache.pdfbox.pdfparser.COSParser.parseDictObjects(COSParser.java:754)
> at org.apache.pdfbox.pdfparser.PDFParser.initialParse(PDFParser.java:185)
> at org.apache.pdfbox.pdfparser.PDFParser.parse(PDFParser.java:220)
> at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:1028)
> at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:949)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
