On Fri, Nov 28, 2014 at 08:06PM, Leidle, Rob wrote: > Hello all, I am trying to configure & install Bigtop 0.8.0 using the puppet > scripts on AmazonLinux on EC2. Thus far, almost everything has worked besides > one minor change I have made to the site.pp manifest. However, I am running > into a problem, it seems that the services such as proxy server or namenode > are not immediately starting. You can see the error below in the purple text > related to namenode. > > > info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: > Scheduling refresh of Service[hadoop-yarn-resourcemanager] > > info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: > Scheduling refresh of Service[hadoop-mapreduce-historyserver] > > info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: > Scheduling refresh of Service[hadoop-yarn-nodemanager] > > info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: > Scheduling refresh of Service[hadoop-yarn-proxyserver] > > debug: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: > The container Class[Hadoop::Common-yarn] will propagate my refresh event > > debug: Service[hadoop-yarn-proxyserver](provider=redhat): Executing > '/sbin/service hadoop-yarn-proxyserver status' > > debug: Service[hadoop-yarn-proxyserver](provider=redhat): Executing > '/sbin/service hadoop-yarn-proxyserver start' > > err: > /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[hadoop-yarn-proxyserver]/ensure: > change from stopped to running failed: Could not start > Service[hadoop-yarn-proxyserver]: Execution of '/sbin/service > hadoop-yarn-proxyserver start' returned 3: at > /mnt/var/lib/bootstrap-actions/1/bigtop-0.8.0/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:483 > > debug: Service[hadoop-yarn-proxyserver](provider=redhat): Executing > '/sbin/service hadoop-yarn-proxyserver status' > > debug: > /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[hadoop-yarn-proxyserver]: > Skipping restart; service is not running > > notice: > /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[hadoop-yarn-proxyserver]: > Triggered 'refresh' from 4 events > > debug: > /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[hadoop-yarn-proxyserver]: > The container Hadoop::Proxyserver[proxyserver] will propagate my refresh > event > > debug: Hadoop::Proxyserver[proxyserver]: The container > Class[Hadoop_head_node] will propagate my refresh event > > debug: Class[Hadoop::Common-yarn]: The container Stage[main] will propagate > my refresh event > > debug: Service[hadoop-hdfs-namenode](provider=redhat): Executing > '/sbin/service hadoop-hdfs-namenode status' > > debug: Service[hadoop-hdfs-namenode](provider=redhat): Executing > '/sbin/service hadoop-hdfs-namenode start' > > err: > /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-hdfs-namenode]/ensure: > change from stopped to running failed: Could not start > Service[hadoop-hdfs-namenode]: Execution of '/sbin/service > hadoop-hdfs-namenode start' returned 3: at > /mnt/var/lib/bootstrap-actions/1/bigtop-0.8.0/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:335 > > debug: Service[hadoop-hdfs-namenode](provider=redhat): Executing > '/sbin/service hadoop-hdfs-namenode status' > > debug: > /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-hdfs-namenode]: > Skipping restart; service is not running > > notice: > /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-hdfs-namenode]: > Triggered 'refresh' from 4 events > > debug: > /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-hdfs-namenode]: > The container Hadoop::Namenode[namenode] will propagate my refresh event > > debug: Hadoop::Namenode[namenode]: The container Class[Hadoop_head_node] will > propagate my refresh event > > notice: > /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Package[hadoop-hdfs-datanode]: > Dependency Service[hadoop-hdfs-namenode] has failures: true > > warning: > /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Package[hadoop-hdfs-datanode]: > Skipping because of failed dependencies > > notice: > /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mnt1/hdfs]: > Dependency Service[hadoop-hdfs-namenode] has failures: true > > warning: > /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mnt1/hdfs]: > Skipping because of failed dependencies > > notice: > /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mnt/hdfs]: > Dependency Service[hadoop-hdfs-namenode] has failures: true > > > The interesting part is that if I query namenode status eventually the > namenode will show up as started even though I have not taken any other > actions: > > > [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status > > Hadoop namenode is not running [FAILED] > > [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status > > Hadoop namenode is not running [FAILED] > > [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status > > Hadoop namenode is running [ OK ] > > The same goes for proxy server. The problem here is that all other > dependencies of namenode do not install (such as resource manager, etc). I > am using the latest release of AmazonLinux (2014.09) and this has puppet > 2.7.25-1. I am not sure what to do about this issue, has anyone else > experienced something like this? Should I just move to puppet 3.x and only > try to install out of the Bigtop trunk (0.9.0)?
ResourceManager isn't a dependency of namenode - it's the other way around. It's hard to say what's going on with your system without looking into particular daemon logs. I'd suggest you check them and investigate what's the trouble is. Also, there's a small issue BIGTOP-1522 with nodemanager recipes if you're installing a custom set of components, which might or not affect you Cos
signature.asc
Description: Digital signature
