Hello everyone, one or more of the nodes in the solrcloud go down randomly when we try to index data using solrj APIs. The nodes do recover. but when we try to index back, they go down again
Our configuration: 3 shards Solr 4.4. I see the following exceptions in the log file. <09/17/13 15:33:32:976|localhost-startStop-1-SendThread(10.68.129.119:9080)|INFO|org.apache.zookeeper.ClientCnxn|Socket connection established to 10.68.129.119/10.68.129.119:9080, initiating session| <09/17/13 15:33:32:978|localhost-startStop-1-SendThread(10.68.129.119:9080)|INFO|org.apache.zookeeper.ClientCnxn|Unable to reconnect to ZooKeeper service, session 0x34109f9474b0029 has expired, closing socket connection| <09/17/13 15:34:36:080|localhost-startStop-1-EventThread|ERROR|apache.solr.cloud.ZkController|There was a problem making a request to the leader:org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://solr02-prod.phneaz:8080/solr at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:431) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.ZkController.waitForLeaderToSeeDownState(ZkController.java:1421) at org.apache.solr.cloud.ZkController.registerAllCoresAsDown(ZkController.java:306) at org.apache.solr.cloud.ZkController.access$100(ZkController.java:86) at org.apache.solr.cloud.ZkController$1.command(ZkController.java:196) at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117) at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46) at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495) Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) We are also getting IOExcpetion in the client side. Adding chunk 122 Total Count 12422 org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://solr-prod.com:8443/solr/aq-collection at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:409) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181) at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:68) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:54) at com.billmelater.fraudworkstation.data.DataProvider.flushBatch(DataProvider.java:48) at com.billmelater.fraudworkstation.data.AQDBDataProvider.execute(AQDBDataProvider.java:114) at com.billmelater.fraudworkstation.data.AQDBDataProvider.main(AQDBDataProvider.java:244) Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) Your help is appreciated. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-node-goes-down-while-trying-to-index-records-tp4090610.html Sent from the Solr - User mailing list archive at Nabble.com.
