Re: zookeeper consistency model?

2010-04-29 Thread Flavio Junqueira
Hi Chen, Let's say that the value of a znode /test is initially v and client A writes value v' to znode /test. If the server that client B is connected to has not persisted the update operation of A, it will read v. If it submits sync before the read, client B will read v'. -Flavio On

Re: zookeeper consistency model?

2010-04-29 Thread Patrick Hunt
Hi Chen, this section of the docs has details on our guarantees, in particular see the note: http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkGuarantees Patrick On 04/29/2010 07:42 AM, Flavio Junqueira wrote: Hi Chen, Let's say that the value of a znode /test is

Re: Embedding ZK in another application

2010-04-29 Thread Vishal K
Hi, Well looks like FastLeaderElection.shutdown() is not invoked. This has been in 3.3.0. Should have checked on that earlier :-) On Thu, Apr 29, 2010 at 10:13 AM, Vishal K vishalm...@gmail.com wrote: Hi Ted, We want the application that embeds the ZK server to be running even after the ZK

Re: Embedding ZK in another application

2010-04-29 Thread Patrick Hunt
See the quorum tests for examples of how to do this, in particular take a look at org.apache.zookeeper.test.QuorumBase which is how we start/stop quorum servers in some of our tests. Our tests can run in fork once mode in junit. Meaning that all the tests (starting/stopping servers frequently)

FYI Apache Avro considering .NET support.

2010-04-29 Thread Patrick Hunt
For those windows based ZooKeeper developers, Avro is looking at .NET support http://bit.ly/9nThxD seems like it would be a great project (and we are planning to add Avro marshaling support at some point). Patrick

Re: zookeeper consistency model?

2010-04-29 Thread Ted Dunning
In general, the guarantee is that B will do exactly as you say it will read the new value or the old value. Your question depends on a definition of now that spans several machines. That is a dangerous concept and if your reasoning requires it, you are headed for trouble. On Thu, Apr 29,

Re: Misbehaving zk servers

2010-04-29 Thread Mahadev Konar
Hi Travis, How many clients did you have connected to this server? Usually the default is 8K file descriptors. Did you have clients more than that? Also, if clients fail to attach to a server, they will run off to another server. We do not do any blacklisting because we expect the server to

Re: Misbehaving zk servers

2010-04-29 Thread Patrick Hunt
Is there any good (simple/fast/bulletproof) way to monitor the FD use inside the jvm? If so we could stop accepting new client connections once we get close to the os imposed limit... The test would have to be a bulletproof one though - we wouldn't want to end up in some worse situation (where

Re: zookeeper consistency model?

2010-04-29 Thread Patrick Hunt
Ted, this is my browser homepage ;-) http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing Patrick On 04/29/2010 09:14 AM, Ted Dunning wrote: In general, the guarantee is that B will do exactly as you say it will read the new value or the old value. Your question depends on a

Re: Misbehaving zk servers

2010-04-29 Thread Travis Crawford
On Thu, Apr 29, 2010 at 9:49 AM, Patrick Hunt ph...@apache.org wrote: Is there any good (simple/fast/bulletproof) way to monitor the FD use inside the jvm? If so we could stop accepting new client connections once we get close to the os imposed limit... The test would have to be a bulletproof

Re: Misbehaving zk servers

2010-04-29 Thread Patrick Hunt
Did you find any bugs on java.sun.com related to those? ;-) That does sound like a good solution to me. We should stop accepting connections and log it to the log as well. We might also want to update the user docs and tell users to monitor the FD count as part of their monitoring regime. Is

Re: zookeeper consistency model?

2010-04-29 Thread Ted Dunning
I tend to say that the cost of Now scales very badly with the diameter of the Now that you are buying. A 5 cm Now is quite cheap. A 10 m Now is quite expensive and the cost of a 1000 km Now is more than you can imagine. On Thu, Apr 29, 2010 at 9:55 AM, Patrick Hunt ph...@apache.org wrote:

Re: Misbehaving zk servers

2010-04-29 Thread Travis Crawford
On Thu, Apr 29, 2010 at 10:24 AM, Patrick Hunt ph...@apache.org wrote: Did you find any bugs on java.sun.com related to those? ;-) That does sound like a good solution to me. We should stop accepting connections and log it to the log as well. We might also want to update the user docs and

Re: Embedding ZK in another application

2010-04-29 Thread Mahadev Konar
We do set that Chad but it doesn't seem to help on some systems (especially bsd)... Thanks mahadev On 4/29/10 11:22 AM, Chad Harrington chad.harring...@gmail.com wrote: On Thu, Apr 29, 2010 at 8:49 AM, Patrick Hunt ph...@apache.org wrote: This is not foolproof however. We found that in

Re: Embedding ZK in another application

2010-04-29 Thread Patrick Hunt
On 04/29/2010 11:22 AM, Chad Harrington wrote: On Thu, Apr 29, 2010 at 8:49 AM, Patrick Huntph...@apache.org wrote: This is not foolproof however. We found that in general this would work, however there were some infrequent cases where a restarted server would fail to initialize due to the

Re: Partially partitioned connectivity

2010-04-29 Thread Ted Dunning
Real ZK developers will answer shortly, but your scenario is still a bit ambiguous. The results will differ depending on the context of the partition. If the partition occurs after leader election and A is the leader, I don't think that operations will be impacted unless another leader election

Re: Partially partitioned connectivity

2010-04-29 Thread Mahadev Konar
Hi Kevin, I had the response set up but didn't hit send. Ted already answered your question, but to give you a more technical background assuming that you know a little bit more about transaction ids in ZooKeeper and server ids: If B and C are partitioned from each other, and A is the leader,

Re: Partially partitioned connectivity

2010-04-29 Thread Kevin Webb
On Thu, 29 Apr 2010 13:24:45 -0700 Mahadev Konar maha...@yahoo-inc.com wrote: Hi Kevin, I had the response set up but didn't hit send. Ted already answered your question, but to give you a more technical background assuming that you know a little bit more about transaction ids in ZooKeeper