[
https://issues.apache.org/jira/browse/TIKA-456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343193#comment-14343193
]
Tim Allison commented on TIKA-456:
----------------------------------
[~tpalsulich], until you pinged me on this, I regret that I hadn't appreciated
[~jukkaz]'s
[comment|https://issues.apache.org/jira/browse/TIKA-456?focusedCommentId=13409101&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13409101]
until now about the ForkParser's timeout on the server side. If we allow the
user to configure the {{Thread.sleep( x )}} in {{ForkServer}}'s {{run()}},
won't that meet the goal of this request?
I just ran some basic tests with {{ForkParser}} wrapping an {{EvilParser}}
against {{heavy_hang.evil}}, and the {{ForkParser}} timed out perfectly with a
TikaException 2 {{x}} the number of milliseconds for the {{Thread.sleep( x )}},
and it restarted as it should for the next document without any special user
intervention.
I completely agree with [~willp-bl] and [~grossws] that we can't kill a thread,
but we sometimes need to kill and restart a process.
> Support timeouts for parsers
> ----------------------------
>
> Key: TIKA-456
> URL: https://issues.apache.org/jira/browse/TIKA-456
> Project: Tika
> Issue Type: Improvement
> Components: parser
> Reporter: Ken Krugler
> Assignee: Chris A. Mattmann
>
> There are a number of reasons why Tika could hang while parsing. One common
> case is when a parser is fed an incomplete document, such as what happens
> when limiting the amount of data fetched during a web crawl.
> One solution is to create a TikaCallable that wraps the Tika parser, and
> then use this with a FutureTask. For example, when using a ParsedDatum POJO
> for the results of the parse operation, I do something like this:
> parser = new AutoDetectParser();
> Callable<ParsedDatum> c = new TikaCallable(parser, contenthandler,
> inputstream, metadata);
> FutureTask<ParsedDatum> task = new FutureTask<ParsedDatum>(c);
> Thread t = new Thread(task);
> t.start();
> ParsedDatum result = task.get(MAX_PARSE_DURATION, TimeUnit.SECONDS);
> And TikaCallable() looks like:
> class TikaCallable implements Callable<ParsedDatum> {
> public TikaCallable(Parser parser, ContentHandler handler, InputStream
> is, Metadata metadata) {
> _parser = parser;
> _handler = handler;
> _input = is;
> _metadata = metadata;
> ...
> }
> public ParsedDatum call() throws Exception {
> ....
> _parser.parse(_input, _handler, _metadata, new ParseContext());
> ....
> }
> }
> This seems like it would be generally useful, as I doubt that we'd ever be
> able to guarantee that none of the parsers being wrapped by Tika could ever
> hang.
> One idea is to create a TimeoutParser that wraps a regular Tika Parser. E.g.
> something like:
> Parser p = new TimeoutParser(new AutodetectParser(), 20, TimeUnit.SECONDS);
> Then the call to p.parse(...) would create a Callable (similar to the code
> above) and use the specified timeout when calling task.get().
> One minus with this approach is that it creates a new thread for each parse
> request, but I don't think the thread overhead is significant when compared
> to the typical parser operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
