Re: TikaIO concerns

Fri, 22 Sep 2017 14:21:06 -0700 

Hi,
On 22/09/17 22:02, Eugene Kirpichov wrote:

Sure - with hundreds of different file formats and the abundance of weird /
malformed / malicious files in the wild, it's quite expected that sometimes
the library will crash.

Some kinds of issues are easier to address than others. We can catch
exceptions and return a ParseResult representing a failure to parse this
document. Addressing freezes and native JVM process crashes is much harder
and probably not necessary in the first version.

Sergey - I think, the moment you introduce ParseResult into the code, other
changes I suggested will follow "by construction":
- There'll be 1 ParseResult per document, containing filename, content and
metadata, since per discussion above it probably doesn't make sense to
deliver these in separate PCollection elements
I was still harboring the hope that may be using a container bean like ParseResult (with the other changes you proposed) can somehow let us stream from Tika into the pipeline.
If it is 1 ParseResult per document then it means that until Tika has parsed all the document the pipeline will not see it.
I'm sorry if I may be starting to go in circles. But let me ask this. How can a Beam user write a Beam function which will ensure the Tika content pieces are seen ordered by the pipeline, without TikaIO ?
May be knowing that will help coming up with the idea how to generalize somehow with the help of TikaIO ? 


- Since you're returning a single value per document, there's no reason to
use a BoundedReader
- Likewise, there's no reason to use asynchronicity because you're not
delivering the result incrementally

I'd suggest to start the refactoring by removing the asynchronous codepath,
then converting from BoundedReader to ParDo or MapElements, then converting
from String to ParseResult.
This is a good plan, thanks, I guess at least for small documents it should work well (unless I've misunderstood a ParseResult idea) 

Thanks, Sergey


On Fri, Sep 22, 2017 at 12:10 PM Sergey Beryozkin <sberyoz...@gmail.com>
wrote:


Hi Tim, All
On 22/09/17 18:17, Allison, Timothy B. wrote:

Y, I think you have it right.


Tika library has a big problem with crashes and freezes


I wouldn't want to overstate it.  Crashes and freezes are exceedingly

rare, but when you are processing millions/billions of files in the wild
[1], they will happen.  We fix the problems or try to get our dependencies
to fix the problems when we can,

I only would like to add to this that IMHO it would be more correct to
state it's not a Tika library's 'fault' that the crashes might occur.
Tika does its best to get the latest libraries helping it to parse the
files, but indeed there will always be some file there that might use
some incomplete format specific tag etc which may cause the specific
parser to spin - but Tika will include the updated parser library asap.

And with Beam's help the crashes that can kill the Tika jobs completely
will probably become a history...

Cheers, Sergey

but given our past history, I have no reason to believe that these

problems won't happen again.


Thank you, again!

Best,

              Tim

[1] Stuff on the internet or ... some of our users are forensics

examiners dealing with broken/corrupted files


P.S./FTR  😊
1) We've gathered a TB of data from CommonCrawl and we run regression

tests against this TB (thank you, Rackspace for hosting our vm!) to try to
identify these problems.

2) We've started a fuzzing effort to try to identify problems.
3) We added "tika-batch" for robust single box fileshare/fileshare

processing for our low volume users

4) We're trying to get the message out.  Thank you for working with us!!!

-----Original Message-----
From: Eugene Kirpichov [mailto:kirpic...@google.com.INVALID]
Sent: Friday, September 22, 2017 12:48 PM
To: dev@beam.apache.org
Cc: d...@tika.apache.org
Subject: Re: TikaIO concerns

Hi Tim,
  From what you're saying it sounds like the Tika library has a big

problem with crashes and freezes, and when applying it at scale (eg. in the
context of Beam) requires explicitly addressing this problem, eg. accepting
the fact that in many realistic applications some documents will just need
to be skipped because they are unprocessable? This would be first example
of a Beam IO that has this concern, so I'd like to confirm that my
understanding is correct.


On Fri, Sep 22, 2017 at 9:34 AM Allison, Timothy B. <talli...@mitre.org>
wrote:


Reuven,

Thank you!  This suggests to me that it is a good idea to integrate
Tika with Beam so that people don't have to 1) (re)discover the need
to make their wrappers robust and then 2) have to reinvent these
wheels for robustness.

For kicks, see William Palmer's post on his toe-stubbing efforts with
Hadoop [1].  He and other Tika users independently have wound up
carrying out exactly your recommendation for 1) below.

We have a MockParser that you can get to simulate regular exceptions,
OOMs and permanent hangs by asking Tika to parse a <mock> xml [2].


However if processing the document causes the process to crash, then
it

will be retried.
Any ideas on how to get around this?

Thank you again.

Cheers,

             Tim

[1]
http://openpreservation.org/blog/2014/03/21/tika-ride-characterising-w
eb-content-nanite/
[2]


https://github.com/apache/tika/blob/master/tika-parsers/src/test/resources/test-documents/mock/example.xml

Reply via email to