Hi Jonathan, Regarding sharing ZooKeeper servers among applications, we actually recommend doing that. We ask our users to have dedicated 3-5 node clusters with dedicated machines(/disk) for zookeeper and lets applications share those 3-5 nodes clusters.
But we do have have users (inside Yahoo!) wherein they want to run these clusters for themselves. The reasons being: - there usage of zookeeper (in terms of reads/writes per second and number of clients - 5K or so) is pretty high and they do not want to share the dedicated cluster with other properties inside Yahoo! - they also want to manage upgrades/administration of cluster on there own There might be more info on administering your own cluster at http://hadoop.apache.org/zookeeper/docs/r3.2.0/zookeeperAdmin.html Thanks mahadev On 7/17/09 12:48 PM, "Jonathan Gray" <[email protected]> wrote: > Hey guys, > > Been using ZK indirectly for a few months now in the HBase and Katta > realms. Both of these applications make it really easy so you don't > have to be involved much with managing your ZK cluster to support it. > > I'm now using ZK for a bunch of things internally, so now I'm manually > configuring, starting, and managing a cluster. > > What advice is there about whether I should be sharing a single cluster > between all my applications, or running separate ones for each use? > > I've been told that it's strongly recommended to run your ZK nodes > separately from the application using them (this is actually what we're > telling new users over in HBase, though a majority of installations will > likely co-host them with DataNodes and RegionServers). > > I don't have the resources to maintain a separate 3+ node ZK cluster for > each of my applications, so this is not really an option. I'm trying to > decide if I should have HBase running/managing it's own ZK cluster that > is co-located with some of the regionservers (there will be ample > memory, but ZK will not have a dedicated disk), or if I should be > pointing it to a dedicated 3 node ZK cluster. > > I would then also have Katta pointing at this same shared cluster (or a > separate cluster would be co-located with katta nodes). Same for my > application; could share nodes with the app servers or pointed at a > single ZK cluster. > > Trade-offs I should be aware of? Current best practices? > > Any help would be much appreciated. Thanks. > > Jonathan Gray
