[
https://issues.apache.org/jira/browse/TIKA-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18107587#comment-18107587
]
Tim Allison commented on TIKA-4835:
-----------------------------------
Looking like caching to disk is harmful on the regression server (at least),
and we were doing a lot more of it.
Our first fix brought caching from 10.5GB to 5.5GB, but we still have more to
do.
||Cell||Build / config||Temp writes (md2)||Wall (s)||Ratio vs 3.x||
|3.x|3.3.2, tika-batch, -J -t, 10 consumers, MD5 default|0.33 GB|245-287|1.00|
|4.x OLD|4.0.0-SNAPSHOT (pre TIKA-4828/29), production shape: per-client n=8,
default 256MB cache budget, MD5|10.5 GB|500|1.74-2.0|
|4.x FIXED|4.1.0-SNAPSHOT (TIKA-4828/29 merged), same production shape|5.5
GB|339|1.18-1.38|
|4.x FIXED + corrected config|shared server, 16g fork heap, 4GB cache budget
(-Dtika.pipes.cacheMemoryBudgetBytes), MD5|2.7 GB|282-307|0.98-1.25|
|4.x FIXED + corrected config + temp on tmpfs|as above, forked workers'
java.io.tmpdir on /dev/shm|~0|214-222|0.75-0.90|
|4.x OLD + temp on tmpfs|pre-fix jar, production shape, tmpdir on
/dev/shm|~0|263|0.92-1.07|
> Diagnose performance issues in 4.0.0
> ------------------------------------
>
> Key: TIKA-4835
> URL: https://issues.apache.org/jira/browse/TIKA-4835
> Project: Tika
> Issue Type: Task
> Reporter: Tim Allison
> Priority: Minor
>
> In both tika-server and tika-app, I'm seeing slower performance. This is
> somewhat explainable by the switch to pipes – we're trading robustness for
> speed.
> However, with some params on some docs on some vms, we're doing much better,
> and obv, tika-pipes in shared-server mode is much faster than the default
> isolated.
> I already opened a ticket to improve excess caching to disk, which wasn't
> great on a vm with a slowish drive. I also drafted this:
> https://tika.apache.org/docs/4.0.x/pipes/performance.html
> We should figure out if this is a change in how the parsers are working,
> something at the tika-level generally or something within pipes.
> There's more work to do here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
