[
https://issues.apache.org/jira/browse/TIKA-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967786#comment-15967786
]
Md commented on TIKA-2326:
--------------------------
We have many files which are archived, for them, RecursiveParserWrapper is
doing the good job that's the reason I am using it.
Ok Thanks, I will use default handler instead.
I think I started using tika very early stage and at that time most of this
code is written. I did notice some difference with new AutoDetectParser() vs
HtmlParser() and that's the reason I was using it but Now it looks same. Thank
you for pointing it put.
Is it possible to add a timer in the parser? especially here
parser.parse(inputStream, handler, metadata, context);
Sometimes it's taking a long time and I would like to set a prespecified
timeout.
Thank you so much I really appriciate your help
> java.lang.OutOfMemoryError: Java heap space
> -------------------------------------------
>
> Key: TIKA-2326
> URL: https://issues.apache.org/jira/browse/TIKA-2326
> Project: Tika
> Issue Type: Bug
> Affects Versions: 1.8
> Environment: Ubuntu 16.04, java version "1.8.0_121"
> Reporter: Md
> Fix For: 1.13
>
> Attachments:
> 5d3e8154444263c73061d8804e15db3ammn0789_CLEAN_REVISED.docx
>
>
> I am using RecursiveParserWrapper with AutoDetectParser() and here is the
> part of my code which is doing parsing
>
> RecursiveParserWrapper parser = null;
> ContentHandlerFactory factory = new
> BasicContentHandlerFactory(BasicContentHandlerFactory.HANDLER_TYPE.TEXT, -1);
>
> if(fileNameWithPath.toLowerCase().contains(htmlInTxtFile.toLowerCase())) {
> parser = new RecursiveParserWrapper(new HtmlParser(), factory);
> } else {
> parser = new RecursiveParserWrapper(new AutoDetectParser(),
> factory);
> }
> // -1 for unlimited buffer
> ContentHandler handler = new BodyContentHandler(-1);
> ParseContext context = new ParseContext();
> parser.parse(inputStream, handler, metadata, context);
> Out of my 251000 files there is only one file where parser is unable to parse
> and proving out of memory error. Here goes the error message
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at org.apache.xmlbeans.impl.store.CharUtil.allocate(CharUtil.java:397)
> at org.apache.xmlbeans.impl.store.CharUtil.saveChars(CharUtil.java:506)
> at org.apache.xmlbeans.impl.store.CharUtil.saveChars(CharUtil.java:419)
> at org.apache.xmlbeans.impl.store.CharUtil.saveChars(CharUtil.java:489)
> at org.apache.xmlbeans.impl.store.Cur$CurLoadContext.text(Cur.java:2927)
> at
> org.apache.xmlbeans.impl.store.Cur$CurLoadContext.stripText(Cur.java:3130)
> at org.apache.xmlbeans.impl.store.Cur$CurLoadContext.text(Cur.java:3143)
> at
> org.apache.xmlbeans.impl.store.Locale$SaxHandler.characters(Locale.java:3291)
> at
> org.apache.xmlbeans.impl.piccolo.xml.Piccolo.reportCdata(Piccolo.java:992)
> at
> org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseXMLNS(PiccoloLexer.java:1290)
> at
> org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.parseXML(PiccoloLexer.java:1261)
> at
> org.apache.xmlbeans.impl.piccolo.xml.PiccoloLexer.yylex(PiccoloLexer.java:4812)
> at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.yylex(Piccolo.java:1290)
> at
> org.apache.xmlbeans.impl.piccolo.xml.Piccolo.yyparse(Piccolo.java:1400)
> at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.parse(Piccolo.java:714)
> at
> org.apache.xmlbeans.impl.store.Locale$SaxLoader.load(Locale.java:3479)
> at
> org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1277)
> at
> org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1264)
> at
> org.apache.xmlbeans.impl.schema.SchemaTypeLoaderBase.parse(SchemaTypeLoaderBase.java:345)
> at
> org.openxmlformats.schemas.wordprocessingml.x2006.main.DocumentDocument$Factory.parse(Unknown
> Source)
> at
> org.apache.poi.xwpf.usermodel.XWPFDocument.onDocumentRead(XWPFDocument.java:136)
> at org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:166)
> at
> org.apache.poi.xwpf.usermodel.XWPFDocument.<init>(XWPFDocument.java:118)
> at
> org.apache.poi.xwpf.extractor.XWPFWordExtractor.<init>(XWPFWordExtractor.java:59)
> at
> org.apache.poi.extractor.ExtractorFactory.createExtractor(ExtractorFactory.java:182)
> at
> org.apache.tika.parser.microsoft.ooxml.OOXMLExtractorFactory.parse(OOXMLExtractorFactory.java:86)
> at
> org.apache.tika.parser.microsoft.ooxml.OOXMLParser.parse(OOXMLParser.java:82)
> at
> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:281)
> at
> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:281)
> at
> org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:120)
> at
> org.apache.tika.parser.RecursiveParserWrapper.parse(RecursiveParserWrapper.java:130)
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
