Hallo Karl!
Now I found how to build (my first building … 😊 ) and now I am using:
/multiprocess-file-example-proprietary folder and I deployed into Tomcat .
I recreated the configuration that I used on binary version and I created the
same job
It works !!!
I see on manifold cf.lg the error:
WARN 2018-06-19T12:11:52,476 (Worker thread '5') - JCIFS: Possibly transient
exception detected on attempt 1 while getting share security: All pipe
instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563)
~[jcifs.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:663) ~[jcifs.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
~[jcifs.jar:?]
at
jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
~[jcifs.jar:?]
at
jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
~[jcifs.jar:?]
at
jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
~[jcifs.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
~[jcifs.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
~[jcifs.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2946) ~[jcifs.jar:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
[mcf-jcifs-connector.jar:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
[mcf-jcifs-connector.jar:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
[mcf-jcifs-connector.jar:?]
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
[mcf-pull-agent.jar:?]
WARN 2018-06-19T12:11:53,435 (Worker thread '3') - JCIFS: Possibly transient
exception detected on attempt 1 while getting share security: All pipe
instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563)
~[jcifs.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:663) ~[jcifs.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
~[jcifs.jar:?]
at
jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
~[jcifs.jar:?]
at
jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
~[jcifs.jar:?]
but it doesn’t stuck no more , so for me it isn’t a problem if they are only
warning!
Very very good!
Thanks for your help!
Mario
Da: Karl Wright <[email protected]>
Inviato: martedì 19 giugno 2018 09:52
A: [email protected]
Oggetto: Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed:
null
Hi Mario,
You cannot patch the binary. You must build from source to apply the patch.
The easiest way forward is to check out trunk directly (with svn) and build it.
The trunk svn URL is
https://svn.apache.org/repos/asf/manifoldcf/trunk<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ccea091a50fe347b3c2cc08d5d5b9888e%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=XzTfotr%2B8KS4wjHws3sKU79U9xgrWwfSAS4qK7Y%2F7NI%3D&reserved=0>
.
Karl
On Tue, Jun 19, 2018 at 3:35 AM Bisonti Mario
<[email protected]<mailto:[email protected]>> wrote:
Hallo.
Note that I specified the mime types on my solr output connection
Furthermore, I used the binary distribution, how cold I path it with tour fix?
I read on my job, that stuck up on 3 docs with:
WARN 2018-06-19T09:29:21,366 (Worker thread '14') - JCIFS: Possibly transient
exception detected on attempt 1 while getting share security: All pipe
instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563)
~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:663)
~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
~[jcifs-1.3.19.jar:?]
at
jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
~[jcifs-1.3.19.jar:?]
at
jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
~[jcifs-1.3.19.jar:?]
at
jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2946)
~[jcifs-1.3.19.jar:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
[mcf-jcifs-connector.jar:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
[mcf-jcifs-connector.jar:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
[mcf-jcifs-connector.jar:?]
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
[mcf-pull-agent.jar:?]
FATAL 2018-06-19T09:29:21,934 (Worker thread '12') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:21,982 (Worker thread '1') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:22,546 (Worker thread '24') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,416 (Worker thread '9') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,433 (Worker thread '45') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,459 (Worker thread '17') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,489 (Worker thread '26') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,507 (Worker thread '19') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,516 (Worker thread '13') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,556 (Worker thread '2') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,557 (Worker thread '41') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,597 (Worker thread '0') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,591 (Worker thread '16') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,593 (Worker thread '33') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,646 (Worker thread '25') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '7') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '23') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,698 (Worker thread '8') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,682 (Worker thread '48') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,686 (Worker thread '30') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,736 (Worker thread '46') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,739 (Worker thread '29') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,742 (Worker thread '3') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,797 (Worker thread '28') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,789 (Worker thread '21') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,791 (Worker thread '11') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,849 (Worker thread '42') - Error tossed: null
Thanks a lot for your great help
Da: Karl Wright <[email protected]<mailto:[email protected]>>
Inviato: lunedì 18 giugno 2018 20:39
A: [email protected]<mailto:[email protected]>
Oggetto: Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed:
null
Created CONNECTORS-1510 and committed a fix.
Karl
On Mon, Jun 18, 2018 at 2:33 PM Karl Wright
<[email protected]<mailto:[email protected]>> wrote:
It certainly is a particular file -- the mime type is null, and that's causing
this line to blow up:
final String lowerMimeType = mimeType.toLowerCase(Locale.ROOT);
That code was added a couple of revs back to address a different problem; it's
a trivial fix:
final String lowerMimeType = (mimeType !=
null)?mimeType.toLowerCase(Locale.ROOT):null;
(This is HttpPoster line 811)
Karl
On Mon, Jun 18, 2018 at 2:30 PM Steph van Schalkwyk
<[email protected]<mailto:[email protected]>> wrote:
Looks like a particular file may be causing this. Try to find the filanem it
crashes on and copy that to asmall crawl directory. Repeat crawl.
On Mon, Jun 18, 2018 at 11:34 AM, Bisonti Mario
<[email protected]<mailto:[email protected]>> wrote:
Hallo
I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3
I configured multiprocess-file-example
When I create a Job to scan a big Windows share (22000 docs word, pdf, etc,)
manifoldcf crash with the message:
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
[mcf-pull-agent.jar:?]
FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
java.lang.NullPointerException
at
org.apache.manifoldcf.agents.output.solr.HttpPoster.checkMimeTypeIndexable(HttpPoster.java:811)
~[?:?]
at
org.apache.manifoldcf.agents.output.solr.SolrConnector.checkMimeTypeIndexable(SolrConnector.java:534)
~[?:?]
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937)
~[mcf-agents.jar:?]
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864)
~[mcf-agents.jar:?]
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589)
~[mcf-agents.jar:?]
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273)
~[mcf-agents.jar:?]
at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029)
~[mcf-pull-agent.jar:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439)
~[?:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector$ProcessDocumentsFilter.accept(SharedDriveConnector.java:4874)
~[?:?]
at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]
at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]
at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]
at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318)
~[?:?]
at
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:798)
~[?:?]
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
[mcf-pull-agent.jar:?]
If I use a smaller windows share it works.
Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it worked.
What could I do?
Thanks a lot
Mario
