Evans Ye created BIGTOP-1336:
--------------------------------
Summary: Puppet recipes failed to deploy kerberos enabled hadoop
cluster
Key: BIGTOP-1336
URL: https://issues.apache.org/jira/browse/BIGTOP-1336
Project: Bigtop
Issue Type: Bug
Components: Deployment
Affects Versions: 0.7.0
Reporter: Evans Ye
Here are some missing dependency setting in our puppet recipes in order to get
kerberos enabled on the hadoop cluster.
The first one is that kerberos principal for hdfs user hasn't been created
before formatting namenode, which cause the namenode formatting process failed.
The second one is that {{/etc/default/hadoop-hdfs-datanode}} doesn't get ready
before starting up datanodes and results in datanodes failed to startup.
The datanode error log:
{noformat}
2014-06-16 15:10:10,711 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode:
Exception in secureMain
java.lang.RuntimeException: Cannot start secure cluster without privileged
resources.
{noformat}
Here's the reproduce steps using
[vagrant-puppet|https://github.com/apache/bigtop/tree/master/bigtop-deploy/vm/vagrant-puppet]:
1.) Enable kerberos on the hadoop cluster.
{noformat}
$ vim bigtop-deploy/vm/vagrant-puppet/provision.sh
{noformat}
Add kerberos definitions.
{noformat}
cat > /bigtop-puppet/config/site.csv << EOF
hadoop_head_node,$1
hadoop_storage_dirs,/data/1,/data/2
bigtop_yumrepo_uri,http://bigtop.s3.amazonaws.com/releases/0.7.0/redhat/6/x86_64
jdk_package_name,java-1.7.0-openjdk-devel.x86_64
components,hadoop,hbase,yarn,mapred-app
hadoop_security,kerberos
hadoop_kerberos_domain,vagrant
hadoop_kerberos_realm,BIGTOP.ORG
hadoop_kerberos_kdc_server,bigtop1.vagrant
EOF
{noformat}
2.) Spin up the cluster.
{code}
$ ./startup.sh --cluster
{code}
3-1.) Get an error while formating namenode.
{noformat}
err: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Exec[namenode
format]/returns: change from notrun to 0 failed: /bin/bash -c 'yes Y | hdfs
namenode -format >> /var/lib/hadoop-hdfs/nn.format.log 2>&1' returned 1 instead
of one of [0] at /tmp/vagrant-puppet-2/modules-0/hadoop/manifests/init.pp:361
{noformat}
3-2.) Get an error while starting up datanodes.
{noformat}
err:
/Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Service[hadoop-hdfs-datanode]/ensure:
change from stopped to running failed: Could not start
Service[hadoop-hdfs-datanode]: Execution of '/sbin/service hadoop-hdfs-datanode
start' returned 1: at
/tmp/vagrant-puppet-2/modules-0/hadoop/manifests/init.pp:158
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
