Elukey has submitted this change and it was merged. Change subject: HDFS Namenode automatic failover support - bug fixes. ......................................................................
HDFS Namenode automatic failover support - bug fixes. Added documentation to the README file, resolved an issue with the zookeeper_hosts variable. Bug: T129838 Change-Id: I6d716e9dd526835b72f76312bb5e95488d7b458c --- M README.md M manifests/hadoop/namenode.pp 2 files changed, 16 insertions(+), 4 deletions(-) Approvals: Elukey: Looks good to me, approved jenkins-bot: Verified diff --git a/README.md b/README.md index 7b87f00..bf78384 100644 --- a/README.md +++ b/README.md @@ -118,8 +118,11 @@ For detailed documentation, see the [CDH5 High Availability Guide](http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-High-Availability-Guide/cdh5hag_hdfs_ha_config.html). -This puppet module only supports Quorum-based HA storage using JournalNodes. -It does not support NFS based HA. +This puppet module supports Quorum-based HA storage using JournalNodes and HDFS Namenode +Automatic failover via Zookeeper. + +Namenode Automatic failover is enabled and configured automatically simply setting +```zookeeper_hosts``` and configuring the JournalNodes. Your JournalNodes will be automatically configured based on the value of ```$cdh::hadoop::journalnode_hosts```. When ```cdh::hadoop``` is included, @@ -221,6 +224,9 @@ sudo service hadoop-yarn-resourcemanager stop sudo service hadoop-hdfs-namenode stop +# If HDFS Automatic Failover is configured for the Master node: +sudo service hadoop-hdfs-zkfc stop + # On your hadoop worker nodes: sudo service hadoop-hdfs-datanode stop sudo service hadoop-yarn-nodemanager stop @@ -237,6 +243,9 @@ sudo service hadoop-hdfs-namenode start sudo service hadoop-yarn-resourcemanager start +# If HDFS Automatic Failover is configured for the Master node: +sudo service hadoop-hdfs-zkfc start + # Now that your primary NameNode is back up, and # JournalNodes have been initialized, bootstrap # your Standby NameNode(s). Run this command diff --git a/manifests/hadoop/namenode.pp b/manifests/hadoop/namenode.pp index bd95758..55f8160 100644 --- a/manifests/hadoop/namenode.pp +++ b/manifests/hadoop/namenode.pp @@ -64,13 +64,16 @@ # Create a znode in ZooKeeper inside of which the automatic failover # system stores its data. The command will create a znode in ZooKeeper # and it needs to be executed only when the znode is not present. - $zookeeper_hosts_string = join($::cdh::hadoop::zookeper_hosts, ',') + + # Catch-all if the zookeeper_hosts is not an array. + $zookeeper_hosts_string = inline_template('<%= Array(@zookeeper_hosts).join(",") %>') + exec { 'hadoop-hdfs-zkfc-init': command => '/usr/bin/hdfs zkfc -formatZK', user => 'hdfs', require => [ Service['hadoop-hdfs-namenode'], - File['/usr/lib/zookeeper/bin/zkCli.sh'] + Package['zookeeper'], ], unless => "/usr/lib/zookeeper/bin/zkCli.sh \ -server ${zookeeper_hosts_string} \ -- To view, visit https://gerrit.wikimedia.org/r/278748 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I6d716e9dd526835b72f76312bb5e95488d7b458c Gerrit-PatchSet: 2 Gerrit-Project: operations/puppet/cdh Gerrit-Branch: master Gerrit-Owner: Elukey <[email protected]> Gerrit-Reviewer: Elukey <[email protected]> Gerrit-Reviewer: Ottomata <[email protected]> Gerrit-Reviewer: jenkins-bot <> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
