Hi, AFAIK following are few cases where ZK is used in Hadoop & BookKeeper components.
Hadoop is using ZK to provides the high-availability of the Master process like, 1) HDFS is using ZK for making the NameNode highly available. Also, BKJM(uses BookKeeper as journal manager) which is a sub-module in the HDFS to store WAL(editlog transactions) http://hadoop.apache.org/docs/r2.3.0/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailabilityWithNFS.html 2) YARN is using ZK for making the ResourceManager highly available. Also, they are using it for storing the state of the RM in ZooKeeper(ZKRMStateStore: A ZooKeeper-based state-store implementation) http://hadoop.apache.org/docs/r2.4.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html#Configurations BookKeeper is using ZK for : 1) service discovery - to know the details about the running BK servers 2) metadata storage - keeps all the meta data about the ledgers(user data) 3) Auto-recovery - handle missing replicas to improve the fault-tolerance. http://zookeeper.apache.org/bookkeeper/ Regards, Rakesh -----Original Message----- From: mufc_fan [mailto:[email protected]] Sent: 18 August 2014 11:25 To: [email protected] Subject: zookeeper in hadoop I am new to zookeeper and from the documentation I learnt that "zookeeper is a sub-project of hadoop". so zookeeper must be used in hadoop. Can any one share their knowledge how zookeeper is used in hadoop with simple example. -- View this message in context: http://zookeeper-user.578899.n2.nabble.com/zookeeper-in-hadoop-tp7580179.html Sent from the zookeeper-user mailing list archive at Nabble.com.
