[
https://issues.apache.org/jira/browse/TIKA-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822127#comment-16822127
]
Tim Allison commented on TIKA-2849:
-----------------------------------
With 20 loops of detection for our 147 zip-based files, there's quite an
improvement w streaming detection (time in seconds)
tika stream: 10
tika stream w file: 31
non tika stream:8
With 10 loops of parsing, the times are quite close:
tika stream: 129
tika stream w file: 130
non tika stream:133
And these are the times with in branch_1x before the merge:
With 20 loops of detection for our zip-based files, file based tika-stream is
roughly the same, but > 2x faster than detect with a TikaInputStream wrapped
InputStream
tika stream: 72
tika stream w file: 29
non tika stream:4 (NOTE THAT ZIP SUB-TYPE DETECTION IS SKIPPED!)*
With 10 loops of parsing, the times are all slightly slower in branch_1x:
tika stream: 166
tika stream w file: 143
non tika stream:170
*
{noformat}
else if (PackageParser.isZipArchive(type)
&& TikaInputStream.isTikaInputStream(input)) {
return detectZipFormat(tis);{noformat}
> TikaInputStream copies the input stream locally
> -----------------------------------------------
>
> Key: TIKA-2849
> URL: https://issues.apache.org/jira/browse/TIKA-2849
> Project: Tika
> Issue Type: Bug
> Affects Versions: 1.20
> Reporter: Boris Petrov
> Assignee: Tim Allison
> Priority: Major
>
> When doing "tika.detect(stream, name)" and the stream is a "TikaInputStream",
> execution gets to "TikaInputStream#getPath" which does a "Files.copy(in,
> path, REPLACE_EXISTING);" which is very, very bad. This input stream could
> be, as in our case, an input stream from a network file which is tens or
> hundreds of gigabytes large. Copying it locally is a huge waste of resources
> to say the least. Why does it do that and can I make it not do it? Or is this
> something that has to be fixed in Tika?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
