Reuven, Thank you! This suggests to me that it is a good idea to integrate Tika with Beam so that people don't have to 1) (re)discover the need to make their wrappers robust and then 2) have to reinvent these wheels for robustness.
For kicks, see William Palmer's post on his toe-stubbing efforts with Hadoop [1]. He and other Tika users independently have wound up carrying out exactly your recommendation for 1) below. We have a MockParser that you can get to simulate regular exceptions, OOMs and permanent hangs by asking Tika to parse a <mock> xml [2]. > However if processing the document causes the process to crash, then it will > be retried. Any ideas on how to get around this? Thank you again. Cheers, Tim [1] http://openpreservation.org/blog/2014/03/21/tika-ride-characterising-web-content-nanite/ [2] https://github.com/apache/tika/blob/master/tika-parsers/src/test/resources/test-documents/mock/example.xml