Re: Solr Live Nodes not updating immediately
Hi, tickTime in zookeeper was high. When i reduced it to 2000ms solr node status gets updated in 20s. Hence resolved my issue. Thanks for helping me. I have one more question. 1. Is it advisable to reduce the tickTime further. 2. Or whats the most appropriate tickTime which gives maximum performance and also solr node gets updated in lesser time. I hereby included my zoo.cfg configuration tickTime=2000 dataDir=/home/local/ranjith-1785/sources/solrcloud/zookeeper-3.4.5_Server1/zoodata clientPort = 2181 initLimit=5 syncLimit=2 maxClientCnxns=180 server.1=localhost:2888:3888 server.2=localhost:3000:4000 server.3=localhost:2500:3500 -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Live-Nodes-not-updating-immediately-tp4076560p4077467.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Live Nodes not updating immediately
On 7/11/2013 11:11 PM, Ranjith Venkatesan wrote: tickTime in zookeeper was high. When i reduced it to 2000ms solr node status gets updated in 20s. Hence resolved my issue. Thanks for helping me. I have one more question. 1. Is it advisable to reduce the tickTime further. 2. Or whats the most appropriate tickTime which gives maximum performance and also solr node gets updated in lesser time. I hereby included my zoo.cfg configuration tickTime=2000 dataDir=/home/local/ranjith-1785/sources/solrcloud/zookeeper-3.4.5_Server1/zoodata clientPort = 2181 initLimit=5 syncLimit=2 maxClientCnxns=180 server.1=localhost:2888:3888 server.2=localhost:3000:4000 server.3=localhost:2500:3500 Here's mine, comments removed. Except for dataDir, these are all default values found in the zookeeper download and on the zookeeper website: tickTime=2000 initLimit=10 syncLimit=5 dataDir=zoodata clientPort=2181 server.1=zoo1.REDACTED.com:2888:3888 server.2=zoo2.REDACTED.com:2888:3888 server.3=zoo3.REDACTED.com:2888:3888 http://zookeeper.apache.org/doc/r3.4.5/zookeeperStarted.html#sc_RunningReplicatedZooKeeper I hope your config is a dev install, because if all your zookeepers are running on the same server, you have no redundancy in the face of a server failure. Servers do fail, even if they have all the redundancy features you can buy. Thanks, Shawn
Re: Solr Live Nodes not updating immediately
What do you have your ZK Timeout set to (zkClientTimeout in solr.xml or command line if you override it)? A kill of the raw process is bad, but ZK should spot that using its heartbeat mechanism, so unless your timeout is very large, it should be detecting the node is no longer available, and then triggering a leadership election. We (still) use 4.3.0 (with some patches) and we do have some issues with Solr shutdowns not causing an election quickly enough for us, but that's a known issue within Solr/Jetty, and maybe causes 10-20s of outage, not 20 minutes! You say you have 3 machines, how many shards and how many ZKs, and are they embedded ZK or external? I think we need more info about the scenario. If you are running embedded ZK, then you are losing both a shard/replica and a ZK at the same time, which isn't ideal (we moved to external ZKs quite quickly, embedded just caused too many issues) but shouldn't be that catastrophic. Also does it only happen with a kill -9, what about a normal kill, and/or a normal shutdown of Jetty? On 9 July 2013 16:18, Shawn Heisey s...@elyograg.org wrote: We are going to use solr in production. There are chances that the machine itself might shutdown due to power failure or the network is disconnected due to manual intervention. We need to address those cases as well to build a robust system.. The latest version of Solr is 4.3.1, and 4.4 is right around the corner. Any chance you can test a nightly 4.4 build or a checkout of the lucene_solr_4_4 branch,ji so we can know whether you are running into the same problems with what will be released soon? No sense in fixing a problem that no longer exists. Thanks, Shawn
Re: Solr Live Nodes not updating immediately
My zkClientTimeout is set to 15000 by default. I am using external zookeeper-3.4.5 which is also running in 3 machines. I am using only one shard with replication factor being set to 3. Normal shutdown updates the solr state as soon as the node gets down.. I am facing issue with abrupt shut down(kill -9) or network problem. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Live-Nodes-not-updating-immediately-tp4076560p4076826.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr Live Nodes not updating immediately
Hi, I am new to solr. Currently i m using Solr-4.3.0. I had setup a solrcloud setup in 3 machines. If I kill a node running in any of the machine using /kill -9/, status of the killed node is not updating immediately in web console of solr. I takes hardly /20+ mins/ to mark that as Gone node. My question is 1. Why does it takes so much time to update the status of the inactive node. 2. And if the leader node itself is killed means, i cant able to use the service till the status of the node gets updated. Thanks in advance Ranjith Venkatesan -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Live-Nodes-not-updating-immediately-tp4076560.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Live Nodes not updating immediately
Something is wrong if it actually takes 20 minutes. - Mark On Jul 9, 2013, at 7:43 AM, Ranjith Venkatesan ranjit...@zohocorp.com wrote: Hi, I am new to solr. Currently i m using Solr-4.3.0. I had setup a solrcloud setup in 3 machines. If I kill a node running in any of the machine using /kill -9/, status of the killed node is not updating immediately in web console of solr. I takes hardly /20+ mins/ to mark that as Gone node. My question is 1. Why does it takes so much time to update the status of the inactive node. 2. And if the leader node itself is killed means, i cant able to use the service till the status of the node gets updated. Thanks in advance Ranjith Venkatesan -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Live-Nodes-not-updating-immediately-tp4076560.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Live Nodes not updating immediately
On 7/9/2013 5:43 AM, Ranjith Venkatesan wrote: I am new to solr. Currently i m using Solr-4.3.0. I had setup a solrcloud setup in 3 machines. If I kill a node running in any of the machine using /kill -9/, status of the killed node is not updating immediately in web console of solr. I takes hardly /20+ mins/ to mark that as Gone node. My question is 1. Why does it takes so much time to update the status of the inactive node. 2. And if the leader node itself is killed means, i cant able to use the service till the status of the node gets updated. As Mark said, something is very wrong if it takes 20 minutes for the cloud state to update. I'm wondering why you have done a kill -9 to stop Solr? If running a stop command (or a standard SIGTERM) doesn't properly shut the process down, then you may have some other underlying operating system issue that needs to be solved, and could be causing the node status problem. Thanks, Shawn
Re: Solr Live Nodes not updating immediately
The same scenario happens if network to any one of the machine is unavailable. (i.e if we manually disconnect network cable also, status of the node not gets updated immediately). Pls help me in this issue -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Live-Nodes-not-updating-immediately-tp4076560p4076621.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Live Nodes not updating immediately
We are going to use solr in production. There are chances that the machine itself might shutdown due to power failure or the network is disconnected due to manual intervention. We need to address those cases as well to build a robust system.. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Live-Nodes-not-updating-immediately-tp4076560p4076633.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Live Nodes not updating immediately
We are going to use solr in production. There are chances that the machine itself might shutdown due to power failure or the network is disconnected due to manual intervention. We need to address those cases as well to build a robust system.. The latest version of Solr is 4.3.1, and 4.4 is right around the corner. Any chance you can test a nightly 4.4 build or a checkout of the lucene_solr_4_4 branch,ji so we can know whether you are running into the same problems with what will be released soon? No sense in fixing a problem that no longer exists. Thanks, Shawn
