maxClientCnxns

2010-03-15 Thread Michael Bauland
Hi, I've got another question regarding the connection limit. I sometimes get the following warning in my zookeeper log file: 15:33:46,351 - WARN [NIOServerCxn.Factory:2181:nioservercnxn$fact...@226] - Too many connections from /A.B.C.D - max is 10 However, I don't have maxClientCnxns defined

Re: maxClientCnxns

2010-03-15 Thread Patrick Hunt
I'm afraid the docs are wrong, it's being fixed in 3.3.0: https://issues.apache.org/jira/browse/ZOOKEEPER-681 set the value to 0 if you want unlimited. Patrick Michael Bauland wrote: Hi, I've got another question regarding the connection limit. I sometimes get the following warning in my

Re: Managing multi-site clusters with Zookeeper

2010-03-15 Thread Benjamin Reed
it is a bit confusing but initLimit is the timer that is used when a follower connects to a leader. there may be some state transfers involved to bring the follower up to speed so we need to be able to allow a little extra time for the initial connection. after that we use syncLimit to figure

Re: Managing multi-site clusters with Zookeeper

2010-03-15 Thread Patrick Hunt
Michael Bauland wrote: - When I connect with a client to the Zookeeper ensemble I provide the three IP addresses of my three Zookeeper servers. Does the client then choose one of them arbitrarily or will it always try to connect to the first one first? I'm asking since I would like to have my

Re: Managing multi-site clusters with Zookeeper

2010-03-15 Thread Flavio Junqueira
On top of Ben's description, you probably need to set initLimit to several minutes to transfer 700MB (worst case). The value of syncLimit, however, does not need to be that large. -Flavio On Mar 15, 2010, at 7:24 PM, Benjamin Reed wrote: it is a bit confusing but initLimit is the timer

java heap size

2010-03-15 Thread Lei Zhang
Hi: I am puzzled by the Zookeeper Admin Guide's advice on choosing Java heap size: Set the Java heap size. This is very important to avoid swapping, which will seriously degrade ZooKeeper performance. To determine the correct value, use load tests, and make sure you are well below the usage limit

Re: java heap size

2010-03-15 Thread Ted Dunning
Your understanding is correct. But if you set a heap size nearly as big as your physical memory (or larger) then java may allocate that heap which will cause swapping. So swapping is definitely done by the OS, but it is the applications like Java that can cause the OS to do it. On Mon, Mar 15,

persistent storage and node recovery

2010-03-15 Thread Maxime Caron
Hi everybody, From what i understand Zookeeper consistency model work the same way as does Scalaris. Which is to keep the majority of the replica for an item UP. In Scalaris i f a single failed node does crash and recover, it simply start like a fresh new node and all data is lost. This is the

Re: persistent storage and node recovery

2010-03-15 Thread Maxime Caron
Thanks a lots it's much clearer now. When i say more replicas i don't mean the number of node but the number of copy of an item value. This was my misunderstanding because in Scalaris the item value is replicated when node join and leave the DHT. So this is all about the operation log so if a

Re: persistent storage and node recovery

2010-03-15 Thread Ted Dunning
I don't think that you have considered the impact of ordered updates here. On Mon, Mar 15, 2010 at 6:19 PM, Maxime Caron maxime.ca...@gmail.comwrote: So this is all about the operation log so if a node is in minority but have more recent committed value this node is in Veto over the other

Re: persistent storage and node recovery

2010-03-15 Thread Henry Robinson
The advantages of a DHT often include: 1. bounded size routes 2. load balancing 3. dynamic membership at the cost of only making very weak consistency guarantees. Typically a DHT is used for very read heavy workloads - such as CDNs - where the p2p approach is very scalable. But it's extremely

Re: persistent storage and node recovery

2010-03-15 Thread Ted Dunning
I like to say that the cost of now goes up dramatically with diameter. On Mon, Mar 15, 2010 at 7:50 PM, Henry Robinson he...@cloudera.com wrote: There is a fundamental tension between synchronicity of updates and scale.

Re: java heap size

2010-03-15 Thread Lei Zhang
Well JVM's default maximum heap size is 64M according to thishttp://forums.sun.com/thread.jspa?threadID=715348 . I think this is a doc bug.

Re: java heap size

2010-03-15 Thread Lei Zhang
Sorry I was too terse - I meant to say the default JVM heap size setting probably fits the need of most applications. Singling out this setting in Zookeeper Admin Guide is confusing rather than helping.