[
https://issues.apache.org/jira/browse/BIGTOP-1336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040038#comment-14040038
]
Evans Ye edited comment on BIGTOP-1336 at 6/22/14 5:43 AM:
-----------------------------------------------------------
[~jayunit100], Yes you're right for the namenode part.
And there's another part regarding to datanode in this patch, if we do not
setup {{/etc/default/hadoop-hdfs-datanode}} before datanode started, following
FATAL error will show in datanode's log:
{noformat}
2014-06-16 15:10:10,711 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode:
Exception in secureMain
java.lang.RuntimeException: Cannot start secure cluster without privileged
resources.
{noformat}
Overall, this patch is mainly addressing issues to bring a kerberos hadoop
cluster up.
was (Author: evans_ye):
[~jayunit100], Yes you're right for the namenode part.
And there's another part regarding to datanode in this patch, if we do not
setup {{/etc/default/hadoop-hdfs-datanode}} before datanode started, following
FATAL error will show:
{noformat}
2014-06-16 15:10:10,711 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode:
Exception in secureMain
java.lang.RuntimeException: Cannot start secure cluster without privileged
resources.
{noformat}
Overall, this patch is mainly addressing issues to bring a kerberos hadoop
cluster up.
> Puppet recipes failed to deploy kerberos enabled hadoop cluster
> ---------------------------------------------------------------
>
> Key: BIGTOP-1336
> URL: https://issues.apache.org/jira/browse/BIGTOP-1336
> Project: Bigtop
> Issue Type: Bug
> Components: Deployment
> Affects Versions: 0.7.0
> Reporter: Evans Ye
> Fix For: 0.8.0
>
> Attachments: BIGTOP-1336.1.patch
>
>
> Here are some missing dependency setting in our puppet recipes in order to
> get kerberos enabled on the hadoop cluster.
> The first one is that kerberos principal for hdfs user hasn't been created
> before formatting namenode, which cause the namenode formatting process
> failed.
> The second one is that {{/etc/default/hadoop-hdfs-datanode}} doesn't get
> ready before starting up datanodes and results in datanodes failed to startup.
> The datanode error log:
> {noformat}
> 2014-06-16 15:10:10,711 FATAL
> org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
> java.lang.RuntimeException: Cannot start secure cluster without privileged
> resources.
> {noformat}
> Here's the reproduce steps using
> [vagrant-puppet|https://github.com/apache/bigtop/tree/master/bigtop-deploy/vm/vagrant-puppet]:
> 1.) Enable kerberos on the hadoop cluster.
> {noformat}
> $ vim bigtop-deploy/vm/vagrant-puppet/provision.sh
> {noformat}
> Add kerberos definitions.
> {noformat}
> cat > /bigtop-puppet/config/site.csv << EOF
> hadoop_head_node,$1
> hadoop_storage_dirs,/data/1,/data/2
> bigtop_yumrepo_uri,http://bigtop.s3.amazonaws.com/releases/0.7.0/redhat/6/x86_64
> jdk_package_name,java-1.7.0-openjdk-devel.x86_64
> components,hadoop,hbase,yarn,mapred-app
> hadoop_security,kerberos
> hadoop_kerberos_domain,vagrant
> hadoop_kerberos_realm,BIGTOP.ORG
> hadoop_kerberos_kdc_server,bigtop1.vagrant
> EOF
> {noformat}
> 2.) Spin up the cluster.
> {code}
> $ ./startup.sh --cluster
> {code}
> 3-1.) Get an error while formating namenode.
> {noformat}
> err: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Exec[namenode
> format]/returns: change from notrun to 0 failed: /bin/bash -c 'yes Y | hdfs
> namenode -format >> /var/lib/hadoop-hdfs/nn.format.log 2>&1' returned 1
> instead of one of [0] at
> /tmp/vagrant-puppet-2/modules-0/hadoop/manifests/init.pp:361
> {noformat}
> 3-2.) Get an error while starting up datanodes.
> {noformat}
> err:
> /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Service[hadoop-hdfs-datanode]/ensure:
> change from stopped to running failed: Could not start
> Service[hadoop-hdfs-datanode]: Execution of '/sbin/service
> hadoop-hdfs-datanode start' returned 1: at
> /tmp/vagrant-puppet-2/modules-0/hadoop/manifests/init.pp:158
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
