Re: leader/replica update error + client "expected mime type/stale state" error
On 9/21/2016 10:48 AM, Brent wrote: > Okay, I'll try to figure out how to making using to the latest feasible... my > situation is that I need to be able to talk to both DSE and Solr from the > same application, and 5.4.1 is the latest version that works with DSE. > > But this issue appears to be stemming from the Solr side. The client logs > its messages after the messages are logged in the Solr server logs. > > Just pretending that I'm using SolrJ 6.1.0, would there be any explanation > for my question? I can't say much about what client versions work with DSE. I don't know anything about it. I *do* know that CloudSolrClient is very sensitive to version discrepancies with SolrCloud, as already described. This would be the case with either DSE or plain Solr. The client log indicates that it sent its request to the .2 server, and got a 510 HTTP error, with an HTML response that doesn't really say what the problem was. The server-side messages from the .2 server seem to have no connection to the request made by the client. Those messages are from PeerSync -- they concern Solr working to ensure that the replica is in sync with the leader -- internal Solr operation, not a client request. If the messages indicate what I *think* they do, this should normally only happen on major events, like a restart or recovery after a server goes down and then comes back up. The server side messages from the .4 server indicate a low-level problem making an HTTP connection to the .2 server. There's no way to know from the log included why those requests were made. SolrJ is not talking to the .4 server according to the log you included, so those failed requests are probably unrelated to the request coming from your client. One thing you should do here is achieve perfect time sync between these machines, so you can be completely sure that the logs you are comparing are from the same timestamp. I haven't seen anything to indicate what OS you're running on, but if it's Linux or another UNIX-like OS, installing the ntp (or ntpd) package should take care of that. Thanks, Shawn
Re: leader/replica update error + client "expected mime type/stale state" error
Okay, I'll try to figure out how to making using to the latest feasible... my situation is that I need to be able to talk to both DSE and Solr from the same application, and 5.4.1 is the latest version that works with DSE. But this issue appears to be stemming from the Solr side. The client logs its messages after the messages are logged in the Solr server logs. Just pretending that I'm using SolrJ 6.1.0, would there be any explanation for my question? -- View this message in context: http://lucene.472066.n3.nabble.com/leader-replica-update-error-client-expected-mime-type-stale-state-error-tp4297053p4297205.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: leader/replica update error + client "expected mime type/stale state" error
On 9/20/2016 9:40 PM, Brent P wrote: > I'm running Solr Cloud 6.1.0, with a Java client using SolrJ 5.4.1. Don't do that. To successfully run mis-matched versions with SolrCloud, SolrJ should be *newer* than Solr. SolrCloud is evolving so rapidly that mixing SolrJ and Solr versions isn't recommended unless they're very close together within the same major version and SolrJ is newer. With such a wide version discrepancy, especially with a client that's older than the server, any errors you're seeing will not be surprising. If you have the same problem after upgrading SolrJ, then we can investigate more. SolrJ is an integral part of Solr itself, in addition to its status as a standalone client. The coupling between CloudSolrClient and the internal SolrCloud implementation is very tight. The version discrepancy probably would be no problem with HttpSolrClient, because the HTTP API evolves at a much slower pace than the Zookeeper/SolrCloud API. Thanks, Shawn
leader/replica update error + client "expected mime type/stale state" error
I'm running Solr Cloud 6.1.0, with a Java client using SolrJ 5.4.1. Every once in awhile, during a query, I get a pair of messages logged in the client from CloudSolrClient -- an error about a request failing, then a warning saying that it's retrying after a stale state error. For this test, the collection (test_collection) has one shard, with RF=2. There are two machines, 10.112.7.2 (replica) and 10.112.7.4 (leader). The client is on 10.112.7.4. Note that the system time on 10.112.7.4 is about 1 minute, 5-6 seconds ahead of the other machine. --- Leader (10.112.7.4) Solr log: --- 19:27:16.583 ERROR [c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica2] o.a.s.u.StreamingSolrClients error org.apache.http.NoHttpResponseException: 10.112.7.2:8983 failed to respond at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283) at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251) at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:311) at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:185) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$22(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 19:27:16.587 WARN [c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica2] o.a.s.u.p.DistributedUpdateProcessor Error sending update to http://10.112.7.2:8983/solr org.apache.http.NoHttpResponseException: 10.112.7.2:8983 failed to respond at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283) at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251) at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:311) at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:185) at
