[
https://issues.apache.org/jira/browse/CONNECTORS-608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13555553#comment-13555553
]
David Morana commented on CONNECTORS-608:
-----------------------------------------
Hi Karl,
I only ever run one job at a time.
I reduced the JVMs down to one and reran the Profiles RSS Connector.
Well, Manifold ran even faster! It consumed over 20,000 profiles in
about 20
minutes.
And they all made it into the index in a timely manner!
I checked the log and there are index socket errors but I'm pretty sure
it's
choking on this link
https://c3qa.llan.ll.mit.edu/profiles/atom/profileType.do?typ=default; which
isn't even a real profile at all.
I don't know why I didn't see this before. It was always failing on
real
profiles before.
I'll have to exclude this from the crawl.
So, what's going on here? Are there just too many JVMs for my laptop?
Will
this happen on a real server? What's the solution here?
Thanks! I'm so relieved this is working....
> Solr connector gets socket timeouts on slow documents
> -----------------------------------------------------
>
> Key: CONNECTORS-608
> URL: https://issues.apache.org/jira/browse/CONNECTORS-608
> Project: ManifoldCF
> Issue Type: Bug
> Components: Lucene/SOLR connector
> Affects Versions: ManifoldCF 1.1
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 1.1
>
> Attachments: mcf-jetty-error.txt
>
>
> The Solr connector fails on some documents with the following exception.
> {code}
> ERROR 2013-01-11 11:13:59,372 (Worker thread '36') -
> Exception tossed: Repeated service interruptions - failure processing
> document: Software caused connection abort: recv failed
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated service
> interruptions - failure processing document: Software caused connection
> abort: recv failed
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:585)
> Caused by: java.net.SocketException: Software caused connection abort: recv
> failed
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(Unknown Source)
> at java.net.SocketInputStream.read(Unknown Source)
> at
> org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
> at
> org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
> at
> org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:61)
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
> at
> org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
> at
> org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
> at
> org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
> at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
> at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
> at
> org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:716)
> at
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:521)
> at
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> at
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> at
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:352)
> at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
> at
> org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
> at
> org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:742)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
