[ 
https://issues.apache.org/jira/browse/PDFBOX-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387997#comment-15387997
 ] 

Tilman Hausherr commented on PDFBOX-3429:
-----------------------------------------

It is at 100% with 4 threads on a 4 core system. But I don't have the same test 
as you do (I didn't use TIKA, only PDFBox alone), so this doesn't mean much.

Here's some code:
{code}
public class BenchmarkPDFBox3429
{
    public static final int JOB_COUNT = 500;
    public static final int THREAD_COUNT = 4;

    public static void main(String[] args) throws IOException, 
InterruptedException
    {
        System.out.println("Benchmark is running");
        System.out.printf("%s pdf strips with %s thread(s).", JOB_COUNT, 
THREAD_COUNT);
        long startTime = System.currentTimeMillis();

        ExecutorService executor = Executors.newFixedThreadPool(THREAD_COUNT);
        File[] list = new File("XXXXXXXXX").listFiles(new FilenameFilter()
        {
            @Override
            public boolean accept(File dir, String name)
            {
                return name.toLowerCase().endsWith(".pdf");
            }
        });
        for (int index = 0; index < JOB_COUNT; ++index)
        {
            executor.execute(new Job(list[index]));
        }

        executor.shutdown();
        while (!executor.isTerminated())
        {
            Thread.sleep(1000);
        }
        long duration = System.currentTimeMillis() - startTime;
        System.out.println(String.format("\nFinished all jobs. %s ms", 
duration));
    }

    static class Job implements Runnable
    {
        private final File file;

        Job(File file)
        {
            this.file = file;
        }

        @Override
        public void run()
        {
            try
            {
                extractFromPdf(file);
            }
            catch (IOException e)
            {
                e.printStackTrace();
            }
            finally
            {
            }
        }

        public static String extractFromPdf(File file) throws IOException
        {
            try (PDDocument doc = PDDocument.load(file))
            {
                PDFTextStripper stripper = new PDFTextStripper();
                return stripper.getText(doc);
            }
        }
    }

}
{code}


> Improve ExtractText Concurrency
> -------------------------------
>
>                 Key: PDFBOX-3429
>                 URL: https://issues.apache.org/jira/browse/PDFBOX-3429
>             Project: PDFBox
>          Issue Type: Improvement
>          Components: Text extraction
>    Affects Versions: 2.0.1
>         Environment: Win7, jdk1.8.0_60 x64
>            Reporter: Luis Filipe Nassif
>            Priority: Minor
>              Labels: optimization
>         Attachments: cpu-pdfbox-2.0.1.png, cpu-pdfbox1.8.10.png
>
>
> While testing Tika 1.13, which uses PDFBox 2.0.1, from a multithreaded text 
> extraction application, I noted cpu usage aroung 80% in my 6 core computer 
> when processing a dataset of ~75 thousands of pdfs (18GB). It took 5min25sec 
> to complete the text extraction. With Tika 1.10, which uses PDFBox 1.8.10, 
> cpu usage stays aroung 100%. It took 4min37sec to complete. The dataset is 
> read from a ramdrive, so there is no i/o bottleneck. I suspect there is some 
> new synchronization code that blocks the threads for a non trivial amount of 
> time, resulting in less cpu usage than before.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@pdfbox.apache.org
For additional commands, e-mail: dev-h...@pdfbox.apache.org

Reply via email to