[ https://issues.apache.org/jira/browse/TIKA-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16364161#comment-16364161 ]
Nicholas DiPiazza edited comment on TIKA-2575 at 2/14/18 2:46 PM: ------------------------------------------------------------------ This sentence is the kicker > However, this may not prevent cases where we're currently loading large >chunks of the document into memory before parsing. In our code we have a "output-to-input-bytes ratio" where we kill if > a threshold, say 200% of the size is a reasonable value. Yet the thread in this snapshot !Selection_016.png! Note the dominating thread is using 500 MB! Thus the crash. was (Author: ndipiazza_gmail): This sentence is the kicker > However, this may not prevent cases where we're currently loading large >chunks of the document into memory before parsing. In our code we have a "output-to-input-bytes ratio" where we kill if > a threshold, say 200% of the size is a reasonable value. Yet the thread in this snapshot !screenshot-1.png! Note the dominating thread is using 500 MB! Thus the crash. > Provide a way to abort tika parses when tika input stream buffer grows passed > a certain threshold > ------------------------------------------------------------------------------------------------- > > Key: TIKA-2575 > URL: https://issues.apache.org/jira/browse/TIKA-2575 > Project: Tika > Issue Type: Bug > Components: parser > Reporter: Nicholas DiPiazza > Priority: Major > Attachments: Selection_016.png > > > Sometimes, for example, you use tika to parse an XLS file that isn't really > that big, maybe 60 MB. and suddenly the JVM heap size taken is >800Mb which > causes an OOM in my case. > Can we make an "abort threshold" where the tika parse will halt if parse > output bytes exceeds this value? > Or it is possible for users to already do this themselves by watching the > input stream as it grows somehow? > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)