[
https://issues.apache.org/jira/browse/CONNECTORS-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294642#comment-15294642
]
Mr.Keuz commented on CONNECTORS-1317:
-------------------------------------
It seems I found problem. Problem is in missed dependencies:
rome-1.0.jar
jdom-1.0.jar
Without this libraries I got next stacktraces:
FATAL 2016-05-21 04:55:16,633 (Worker thread '41') - Error tossed:
org/jdom/input/JDOMParseException
java.lang.NoClassDefFoundError: org/jdom/input/JDOMParseException
at com.sun.syndication.io.SyndFeedInput.<init>(SyndFeedInput.java:58)
at com.sun.syndication.io.SyndFeedInput.<init>(SyndFeedInput.java:48)
at org.apache.tika.parser.feed.FeedParser.parse(FeedParser.java:70)
at
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
at
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
at
org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:120)
at
org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:72)
at
org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbedded(ParsingEmbeddedDocumentExtractor.java:103)
at
org.apache.tika.parser.pkg.PackageParser.parseEntry(PackageParser.java:224)
at
org.apache.tika.parser.pkg.PackageParser.parse(PackageParser.java:181)
at
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
at
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
at
org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:120)
at
org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(TikaParser.java:48)
at
org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:226)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3221)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3072)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2706)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
at
org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:404)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
Caused by: java.lang.ClassNotFoundException: org.jdom.input.JDOMParseException
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 23 more
> Hang crawling on some ZIP documents
> -----------------------------------
>
> Key: CONNECTORS-1317
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1317
> Project: ManifoldCF
> Issue Type: Bug
> Components: File system connector
> Affects Versions: ManifoldCF 2.3
> Environment: Ubuntu 14.04 Linux 3.13.0-86-generic i686 i686
> java version "1.8.0_31"
> Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)
> DB: Postgres 9.5.1
> Reporter: Mr.Keuz
>
> I use ManifolCF as file crawler. But I found, that crawling process hangs on
> some zip files. Although some files parsing normally.
> Steps:
> 1. Run ManfoldCF by "example/start.sh" and Posgres as DB
> 2. Create manifold pipeline: File -> Tika -> Solr
> 3. Put zip file in folder (in attach below)
> 4. Run job
> Here zip file that should reproduce bug:
> "ManifoldCF_ISSUE_Dive.Into.Python.3.Mark.Pilgrim.2009.zip"
> https://yadi.sk/d/0uSdrR5GrsgmG
> Note:
> As I investigated (by strace) - crawler process tries to open and parse same
> zip file again and again (it seems from different workers threads). And It
> seems that document not removes from queue.
> I am newbie in ManifoldCF, so it is hard task to me to find problem in source
> code.
> I can send some additional info if needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
