Y. Please do open an issue. Thank you! On Sat, Mar 2, 2019 at 3:09 AM Cristian Vat <[email protected]> wrote:
> Not sure where this is between a bug and a performance issue.. > > I got a StackOverflowError while parsing a large PDF file using > ToHTMLContentHandler. Trace: > -- > java.lang.StackOverflowError: null > at java.base/java.util.HashMap.hash(HashMap.java:339) ~[na:na] > at java.base/java.util.HashMap.get(HashMap.java:552) ~[na:na] > at > org.apache.tika.sax.ToXMLContentHandler$ElementInfo.getPrefix(ToXMLContentHandler.java:54) > ~[tika-core-1.20.jar:1.20] > at > org.apache.tika.sax.ToXMLContentHandler$ElementInfo.getPrefix(ToXMLContentHandler.java:58) > ~[tika-core-1.20.jar:1.20] > ....about 1000 recursive calls... > at > org.apache.tika.sax.ToXMLContentHandler$ElementInfo.getPrefix(ToXMLContentHandler.java:58) > ~[tika-core-1.20.jar:1.20] > -- > > Error was received in a Spring Boot command-line app also doing other > processing. > I couldn't duplicate it with a standalone example, possibly standalone > it doesn't completely fill up the stack. > Also no error in standalone tika app running with GUI or as command-line. > > PDF File: "10.1007-s00268-016-3727-3.pdf" (also a DOI, might be more > sources/versions) > Wasn't sure if I should attach it here, it can be downloaded from > > https://www.researchgate.net/publication/309385633_Safety_of_Nonsteroidal_Anti-inflammatory_Drugs_in_Major_Gastrointestinal_Surgery_A_Prospective_Multicenter_Cohort_Study > Generated output has 4681 <meta> tags > Maximum tag depth of generated (X)HTML is 6 > > I then timed parsing with ToHTMLContentHandler versus directly with > ToXMLContentHandler. After a warmup of a few hundred parse calls times > were: > - ToHTMLContentHandler: avg 500 ms > - ToXMLContentHandler: avg 80-90 ms > > Profiling with YourKit showed a hotspot and very deep stack in > recursive calls on ToXMLContentHandler$ElementInfo.getPrefix(String) > in ToXMLContentHandler.java:58, same as was in the StackOverflowError > > Checking the code I found ToXMLContentHandler.endElement has a mention > and a fix of old similar issue TIKA-1070: > -- > // Reset the position in the tree, to avoid endless stack overflow > // chains (see TIKA-1070) > currentElement = currentElement.parent; > -- > > But ToHTMLContentHandler.endElement doesn't call super.endElement in > case of empty elements including the <meta> tag. Thus the > currentElement parents keep growing in this case? > > I created my own version of ToHTMLContentHandler where I called > super.endElement inside the EMPTY_ELEMENTS if and: > - no more StackOverflowError in the spring boot app > - parse times reduced to XML version one, so 5x speed improvement at least > - output is identical except additional "</meta>" closing tag. > > Questions: > - should I create an issue for this? Should I attach the PDF file to > issue? (not sure on rights). Shall I just include entire email text in > issue? > - should anybody be using ToHTMLContentHandler instead of > ToXMLContentHandler ? Not sure on the exact use-case since information > seems to be the same... > -- any way that ToHTMLContentHandler could be improved but without > emitting extra "</meta>" closing tag? > > Regards, > Cristian Vat >
