[
https://issues.apache.org/jira/browse/PDFBOX-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15388439#comment-15388439
]
Luis Filipe Nassif commented on PDFBOX-3429:
--------------------------------------------
Hi Tilman,
I was going to write a test code with only PDFBox, without Tika, thank you very
much. I've got same results with your code: ~100% cpu usage with v1.8.10 and
~80% with 2.0.2, with 12 threads (actually my system has 6 physical cores x 2
hiperthreading). I will attach the graphs. I modified a little bit your code to
used only one specific file (will attach it too), parsed 10000, but it was
chosen ramdonly, so I think there is nothing special with the file. Modified
code below:
{code}
public class BenchmarkPDFBox3429
{
public static final int JOB_COUNT = 10000;
public static final int THREAD_COUNT = 12;
public static void main(String[] args) throws IOException,
InterruptedException
{
java.util.logging.Logger.getLogger("org.apache").setLevel(java.util.logging.Level.OFF);
System.out.println("Benchmark is running");
System.out.printf("%s pdf strips with %s thread(s).", JOB_COUNT,
THREAD_COUNT);
long startTime = System.currentTimeMillis();
ExecutorService executor = Executors.newFixedThreadPool(THREAD_COUNT);
File file = new File("f:/teste/pdf2/000000000000B265.pdf");
for (int i = 0; i < JOB_COUNT; i++)
{
executor.execute(new Job(file));
}
executor.shutdown();
while (!executor.isTerminated())
{
Thread.sleep(1000);
}
long duration = System.currentTimeMillis() - startTime;
System.out.println(String.format("\nFinished all jobs. %s ms",
duration));
}
static class Job implements Runnable
{
private final File file;
Job(File file)
{
this.file = file;
}
@Override
public void run()
{
try
{
extractFromPdf(file);
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
}
}
public static String extractFromPdf(File file) throws IOException
{
PDDocument doc = null;
try
{
doc = PDDocument.load(file);
PDFTextStripper stripper = new PDFTextStripper();
return stripper.getText(doc);
}finally{
if(doc != null)
doc.close();
}
}
}
}
{code}
Also, I repeated the test with only 4 threads (1/3 of 12) and cpu usage was
around 33% (1/3 of 100%) as expected, so maybe the problem arises only in high
concurrent scenarios.
Finally, I repeated the test with 48 threads and got same results: cpu usage
only around 80%.
> Improve ExtractText Concurrency
> -------------------------------
>
> Key: PDFBOX-3429
> URL: https://issues.apache.org/jira/browse/PDFBOX-3429
> Project: PDFBox
> Issue Type: Improvement
> Components: Text extraction
> Affects Versions: 2.0.1
> Environment: Win7, jdk1.8.0_60 x64
> Reporter: Luis Filipe Nassif
> Priority: Minor
> Labels: optimization
> Attachments: cpu-pdfbox-2.0.1.png, cpu-pdfbox1.8.10.png
>
>
> While testing Tika 1.13, which uses PDFBox 2.0.1, from a multithreaded text
> extraction application, I noted cpu usage aroung 80% in my 6 core computer
> when processing a dataset of ~75 thousands of pdfs (18GB). It took 5min25sec
> to complete the text extraction. With Tika 1.10, which uses PDFBox 1.8.10,
> cpu usage stays aroung 100%. It took 4min37sec to complete. The dataset is
> read from a ramdrive, so there is no i/o bottleneck. I suspect there is some
> new synchronization code that blocks the threads for a non trivial amount of
> time, resulting in less cpu usage than before.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
