This paper from Usenix might shed some light as well, esp around how things work and why we do what we do (i.e. majority quorums): http://static.usenix.org/event/usenix10/tech/full_papers/Hunt.pdf
Good Luck, Patrick On Mon, Jun 4, 2012 at 3:23 PM, Camille Fournier <[email protected]> wrote: >> However, the >> ZooKeeper client connects to a ZooKeeper server at random. This is another >> indication that ZooKeeper might be the wrong tool to use. > > You seed the client with the list of servers to connect to. If you > only seed it with one server, it will only connect to one server even > if there are others in the quorum, so this should not be an issue. > > As for your questions, you're definitely pushing the envelope on the > standard deployment model but ZK is probably the right tool for you to > use if you care about correctness in your distributed locking. Using > ZK for distributed locking is one of its key uses. Embedding it inside > an application is not common but not unheard of. The danger with > embedding ZK is that you now have another system that can cause ZK to > perform poorly due to GC, cpu contention, etc. Is there some reason > you're concerned with running ZK as a separate VM? It won't be highly > available, but it sounds like running a standalone ZK on a separate > server than your two system servers might be the best choice for you > right now. I would probably recommend this over trying to run an > embedded quorum. > > C > > On Mon, Jun 4, 2012 at 5:44 PM, David Nickerson < > [email protected]> wrote: > >> I work for a small software company and I have been tasked with researching >> a distributed lock manager for us to use. I would appreciate any feedback >> on whether or not ZooKeeper is the right choice. >> >> Our software solution utilizes multiple servers, but only the Java >> application called "Administrator" needs to acquire locks. The server that >> Administrator runs on is called the System Server, which presents a web >> interface to potentially thousands of users, and is highly multi-threaded. >> >> So far, we have used only one System Server, and we have needed >> locks only among threads in one application. However, we are looking to >> scale the number of System Servers, and therefore need a distributed lock >> manager to be shared among them. >> >> We don't want to set up another server just for ZooKeeper, and we don't >> want the System Server to run another instance of the JVM just for >> ZooKeeper, so we have decided to embed the ZooKeeper server in our >> Administrator application. >> >> If the System Server goes down for whatever reason, we no longer have a >> need for locks, so it's not a problem if ZooKeeper goes down too. However, >> if we have two System Servers, we would like to be able to fail over to the >> second server. However, ZooKeeper requires that a majority of the ZooKeeper >> servers be up in order to run. This requirement seems strange and >> worrisome. For the time being, we can just run an additional ZooKeeper >> server on another server to give ZooKeeper the redundancy it needs. >> >> Another issue is that we would like to have fast locks and reduced network >> traffic by being able to read from the local ZooKeeper server. However, the >> ZooKeeper client connects to a ZooKeeper server at random. This is another >> indication that ZooKeeper might be the wrong tool to use. >> >> An important feature that we need is deadlock detection. I had to write >> this myself, but I was able to do so using the interface that ZooKeeper >> provides. >> >> We also need to be able to administrate the ZooKeeper server from our >> program. For example, if the ZooKeeper server has an error or we need to >> shut down the System Server, then the program may need to shutdown the >> ZooKeeper server or restart it. Some code to do this might be: >> // To start the server >> QuorumPeerMain main; >> try { >> QuorumPeerConfig config = new QuorumPeerConfig(); >> config.parse("ZooKeeper files\\zoo.cfg"); >> main = new QuorumPeerMain(); >> main.runFromConfig(config); >> } catch (ConfigException e) { >> e.printStackTrace(); >> } catch (IOException e) { >> e.printStackTrace(); >> } >> // To shutdown the server >> main.quorumPeer.shutdown(); >> >> However, "main.quorumPeer" is protected. There is very little documentation >> on the Internet about embedding a ZooKeeper server. In fact, much of the >> server interface seems to be written with the mindset that the ZooKeeper >> server wouldn't be embedded in an application. >> >> So, in conclusion, I have two important questions. First, will ZooKeeper >> work for us, or are we trying to use it in a way that it's not designed >> for? And second, if ZooKeeper is a good choice, then is our idea of >> embedding the ZooKeeper server the right thing to do? >> >> Thanks for reading this, and I appreciate any thoughts or ideas. >>
