[
https://issues.apache.org/jira/browse/CONNECTORS-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14566286#comment-14566286
]
Arcadius Ahouansou commented on CONNECTORS-1191:
------------------------------------------------
{code}
WARN 2015-05-29 00:23:25,085 (Worker thread '19') - Server dropped connection
during indexing http://mysite.com/?view=desktop: Broken pipe
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at
org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:126)
at
org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:138)
at
org.apache.http.impl.io.SessionOutputBufferImpl.flush(SessionOutputBufferImpl.java:146)
at
org.apache.http.impl.io.ChunkedOutputStream.flush(ChunkedOutputStream.java:194)
at
org.apache.http.entity.mime.content.StringBody.writeTo(StringBody.java:176)
at
org.apache.manifoldcf.agents.output.solr.ModifiedHttpMultipart.doWriteTo(ModifiedHttpMultipart.java:211)
at
org.apache.manifoldcf.agents.output.solr.ModifiedHttpMultipart.writeTo(ModifiedHttpMultipart.java:229)
at
org.apache.manifoldcf.agents.output.solr.ModifiedMultipartEntity.writeTo(ModifiedMultipartEntity.java:187)
at
org.apache.http.impl.execchain.RequestEntityProxy.writeTo(RequestEntityProxy.java:123)
at
org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:158)
at
org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:162)
at
org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:237)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:122)
at
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
at
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:462)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:233)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:225)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
at
org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:901)
WARN 2015-05-29 00:23:25,086 (Worker thread '19') - Service interruption
reported for job 1430004534333 connection 'web': Server dropped connection
during indexing http://mysite.com/?view=x: Broken pipe
WARN 2015-05-29 00:36:00,122 (Worker thread '14') - Server dropped connection
during indexing http://mysite.com/?view=y: Broken pipe
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at
org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:126)
at
org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:138)
at
org.apache.http.impl.io.SessionOutputBufferImpl.flush(SessionOutputBufferImpl.java:146)
at
org.apache.http.impl.io.ChunkedOutputStream.flush(ChunkedOutputStream.java:194)
at
org.apache.http.entity.mime.content.StringBody.writeTo(StringBody.java:176)
at
org.apache.manifoldcf.agents.output.solr.ModifiedHttpMultipart.doWriteTo(ModifiedHttpMultipart.java:211)
at
org.apache.manifoldcf.agents.output.solr.ModifiedHttpMultipart.writeTo(ModifiedHttpMultipart.java:229)
at
org.apache.manifoldcf.agents.output.solr.ModifiedMultipartEntity.writeTo(ModifiedMultipartEntity.java:187)
at
org.apache.http.impl.execchain.RequestEntityProxy.writeTo(RequestEntityProxy.java:123)
at
org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:158)
at
org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:162)
at
org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:237)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:122)
at
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
at
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:462)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:233)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:225)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
at
org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:901)
WARN 2015-05-29 00:36:00,123 (Worker thread '14') - Service interruption
reported for job 1430004534323 connection 'web': Server dropped connection
during indexing http://mysite.com/?view=a: Broken pipe
ERROR 2015-05-29 00:36:19,666 (Worker thread '65') - Exception tossed:
Unexpected jobqueue status - record id 1430006793452, expecting active status,
saw 2
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected jobqueue
status - record id 1430006793452, expecting active status, saw 2
at
org.apache.manifoldcf.crawler.jobs.JobQueue.updateCompletedRecord(JobQueue.java:1019)
at
org.apache.manifoldcf.crawler.jobs.JobManager.markDocumentCompletedMultiple(JobManager.java:3260)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:661)
ERROR 2015-05-29 00:36:22,510 (Worker thread '84') - Exception tossed:
Unexpected jobqueue status - record id 1430352284513, expecting active status,
saw 2
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected jobqueue
status - record id 1430352284513, expecting active status, saw 2
at
org.apache.manifoldcf.crawler.jobs.JobQueue.updateCompletedRecord(JobQueue.java:1019)
at
org.apache.manifoldcf.crawler.jobs.JobManager.markDocumentCompletedMultiple(JobManager.java:3260)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:661)
ERROR 2015-05-29 00:38:03,439 (Worker thread '2') - Exception tossed:
Unexpected jobqueue status - record id 1430092859936, expecting active status,
saw 2
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected jobqueue
status - record id 1430092859936, expecting active status, saw 2
at
org.apache.manifoldcf.crawler.jobs.JobQueue.updateCompletedRecord(JobQueue.java:1019)
at
org.apache.manifoldcf.crawler.jobs.JobManager.markDocumentCompletedMultiple(JobManager.java:3260)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:661)
WARN 2015-05-29 00:38:03,820 (Worker thread '17') - Service interruption
reported for job 1430004534333 connection 'web': Job no longer active
WARN 2015-05-29 00:41:01,378 (Worker thread '11') - Server dropped connection
during indexing http://mysite.com/?view=b: Broken pipe
.....
..
...
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
at
org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:901)
WARN 2015-05-30 09:53:06,147 (Worker thread '51') - Service interruption
reported for job 1430004534328 connection 'web': Server dropped connection
during indexing http://mysite.com/job/?view=az: Broken pipe
ERROR 2015-05-30 09:53:55,386 (Worker thread '13') - Exception tossed:
Unexpected jobqueue status - record id 1430089523518, expecting active status,
saw 2
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected jobqueue
status - record id 1430089523518, expecting active status, saw 2
at
org.apache.manifoldcf.crawler.jobs.JobQueue.updateCompletedRecord(JobQueue.java:1019)
at
org.apache.manifoldcf.crawler.jobs.JobManager.markDocumentCompletedMultiple(JobManager.java:3260)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:661)
{code}
I am still seeing this.
- I have only one instance of MCF running.
- Running against MySQL
- Running against a single instance of Solr
- No Zookeeper installed
- The instance has been started using:
{code}
java -Xmx1024m -Duser.timezone=GMT -Dapp.name=manifoldcf -Djetty.host=127.0.0.1
-Djetty.send.server.version=false -classpath
/home/manifoldcf/apache-manifoldcf-2.2-dev/example/../lib/*:/home/manifoldcf/apache-manifoldcf-2.2-dev/example/../lib-proprietary/*:
org.apache.manifoldcf.jettyrunner.ManifoldCFJettyRunner
{code}
As a side note, I have seen a lot of "SELECT FOR UPDATE" in JobManager.java.
Maybe, is it a good idea to review them and use their atomic/lock-free variant
instead?
> ManifoldCFException: Unexpected job status encountered
> ------------------------------------------------------
>
> Key: CONNECTORS-1191
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1191
> Project: ManifoldCF
> Issue Type: Bug
> Components: Web connector
> Affects Versions: ManifoldCF 2.0.2
> Environment: - Debian 7.8 x86_64 GNU/Linux
> - Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
> - Server version: 5.5.41-0+wheezy1 (Debian)
> Reporter: Arcadius Ahouansou
> Assignee: Karl Wright
> Priority: Critical
> Fix For: ManifoldCF 1.9, ManifoldCF 2.1
>
> Attachments: CONNECTORS-1191.patch, manifoldcf2.0.2.log
>
>
> Hello.
> I am running the latest ManifoldCF 2.0.2 and my log is filled of
> {code}
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected job
> status encountered: 34
> at
> org.apache.manifoldcf.crawler.jobs.Jobs.returnJobToActive(Jobs.java:2073)
> at
> org.apache.manifoldcf.crawler.jobs.JobManager.resetJobs(JobManager.java:8261)
> at
> org.apache.manifoldcf.crawler.system.JobResetThread.run(JobResetThread.java:91)
> {code}
> I have attached full log for more detail.
> Note that I am running against MySQL.
> Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
