Hallo.
Note that I specified the mime types on my solr output connection
Furthermore, I used the binary distribution, how cold I path it with tour fix?
I read on my job, that stuck up on 3 docs with:
WARN 2018-06-19T09:29:21,366 (Worker thread '14') - JCIFS: Possibly transient
exception detected on attempt 1 while getting share security: All pipe
instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563)
~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:663)
~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
~[jcifs-1.3.19.jar:?]
at
jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
~[jcifs-1.3.19.jar:?]
at
jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
~[jcifs-1.3.19.jar:?]
at
jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2946)
~[jcifs-1.3.19.jar:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
[mcf-jcifs-connector.jar:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
[mcf-jcifs-connector.jar:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
[mcf-jcifs-connector.jar:?]
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
[mcf-pull-agent.jar:?]
FATAL 2018-06-19T09:29:21,934 (Worker thread '12') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:21,982 (Worker thread '1') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:22,546 (Worker thread '24') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,416 (Worker thread '9') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,433 (Worker thread '45') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,459 (Worker thread '17') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,489 (Worker thread '26') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,507 (Worker thread '19') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,516 (Worker thread '13') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,556 (Worker thread '2') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,557 (Worker thread '41') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,597 (Worker thread '0') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,591 (Worker thread '16') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,593 (Worker thread '33') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,646 (Worker thread '25') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '7') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '23') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,698 (Worker thread '8') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,682 (Worker thread '48') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,686 (Worker thread '30') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,736 (Worker thread '46') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,739 (Worker thread '29') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,742 (Worker thread '3') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,797 (Worker thread '28') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,789 (Worker thread '21') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,791 (Worker thread '11') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,849 (Worker thread '42') - Error tossed: null
Thanks a lot for your great help
Da: Karl Wright <[email protected]>
Inviato: lunedì 18 giugno 2018 20:39
A: [email protected]
Oggetto: Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed:
null
Created CONNECTORS-1510 and committed a fix.
Karl
On Mon, Jun 18, 2018 at 2:33 PM Karl Wright
<[email protected]<mailto:[email protected]>> wrote:
It certainly is a particular file -- the mime type is null, and that's causing
this line to blow up:
final String lowerMimeType = mimeType.toLowerCase(Locale.ROOT);
That code was added a couple of revs back to address a different problem; it's
a trivial fix:
final String lowerMimeType = (mimeType !=
null)?mimeType.toLowerCase(Locale.ROOT):null;
(This is HttpPoster line 811)
Karl
On Mon, Jun 18, 2018 at 2:30 PM Steph van Schalkwyk
<[email protected]<mailto:[email protected]>> wrote:
Looks like a particular file may be causing this. Try to find the filanem it
crashes on and copy that to asmall crawl directory. Repeat crawl.
On Mon, Jun 18, 2018 at 11:34 AM, Bisonti Mario
<[email protected]<mailto:[email protected]>> wrote:
Hallo
I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3
I configured multiprocess-file-example
When I create a Job to scan a big Windows share (22000 docs word, pdf, etc,)
manifoldcf crash with the message:
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
[mcf-pull-agent.jar:?]
FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
java.lang.NullPointerException
at
org.apache.manifoldcf.agents.output.solr.HttpPoster.checkMimeTypeIndexable(HttpPoster.java:811)
~[?:?]
at
org.apache.manifoldcf.agents.output.solr.SolrConnector.checkMimeTypeIndexable(SolrConnector.java:534)
~[?:?]
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937)
~[mcf-agents.jar:?]
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864)
~[mcf-agents.jar:?]
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589)
~[mcf-agents.jar:?]
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273)
~[mcf-agents.jar:?]
at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029)
~[mcf-pull-agent.jar:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439)
~[?:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector$ProcessDocumentsFilter.accept(SharedDriveConnector.java:4874)
~[?:?]
at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]
at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]
at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]
at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318)
~[?:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:798)
~[?:?]
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
[mcf-pull-agent.jar:?]
If I use a smaller windows share it works.
Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it worked.
What could I do?
Thanks a lot
Mario
