Yes, I ran into Bigtop-1522 and figured out I needed to add mapred-app. Sorry, I wrote what I said in the previous email incorrectly, yes, resource manager does not install because the depdendency namenode does not install correctly. I will look more closely at the service logs to see if I can figure out why it isn¹t starting. The error code of ³3² indicates from the /etc/init.d/hadoop-hdfs-namenode script that this means it can¹t find the running process 5 seconds after starting it.
On 11/28/14, 4:14 PM, "Konstantin Boudnik" <[email protected]> wrote: >On Fri, Nov 28, 2014 at 08:06PM, Leidle, Rob wrote: >> Hello all, I am trying to configure & install Bigtop 0.8.0 using the >>puppet scripts on AmazonLinux on EC2. Thus far, almost everything has >>worked besides one minor change I have made to the site.pp manifest. >>However, I am running into a problem, it seems that the services such as >>proxy server or namenode are not immediately starting. You can see the >>error below in the purple text related to namenode. >> >> >> info: >>/Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: >>Scheduling refresh of Service[hadoop-yarn-resourcemanager] >> >> info: >>/Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: >>Scheduling refresh of Service[hadoop-mapreduce-historyserver] >> >> info: >>/Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: >>Scheduling refresh of Service[hadoop-yarn-nodemanager] >> >> info: >>/Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: >>Scheduling refresh of Service[hadoop-yarn-proxyserver] >> >> debug: >>/Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: >>The container Class[Hadoop::Common-yarn] will propagate my refresh event >> >> debug: Service[hadoop-yarn-proxyserver](provider=redhat): Executing >>'/sbin/service hadoop-yarn-proxyserver status' >> >> debug: Service[hadoop-yarn-proxyserver](provider=redhat): Executing >>'/sbin/service hadoop-yarn-proxyserver start' >> >> err: >>/Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[ha >>doop-yarn-proxyserver]/ensure: change from stopped to running failed: >>Could not start Service[hadoop-yarn-proxyserver]: Execution of >>'/sbin/service hadoop-yarn-proxyserver start' returned 3: at >>/mnt/var/lib/bootstrap-actions/1/bigtop-0.8.0/bigtop-deploy/puppet/module >>s/hadoop/manifests/init.pp:483 >> >> debug: Service[hadoop-yarn-proxyserver](provider=redhat): Executing >>'/sbin/service hadoop-yarn-proxyserver status' >> >> debug: >>/Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[ha >>doop-yarn-proxyserver]: Skipping restart; service is not running >> >> notice: >>/Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[ha >>doop-yarn-proxyserver]: Triggered 'refresh' from 4 events >> >> debug: >>/Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[ha >>doop-yarn-proxyserver]: The container Hadoop::Proxyserver[proxyserver] >>will propagate my refresh event >> >> debug: Hadoop::Proxyserver[proxyserver]: The container >>Class[Hadoop_head_node] will propagate my refresh event >> >> debug: Class[Hadoop::Common-yarn]: The container Stage[main] will >>propagate my refresh event >> >> debug: Service[hadoop-hdfs-namenode](provider=redhat): Executing >>'/sbin/service hadoop-hdfs-namenode status' >> >> debug: Service[hadoop-hdfs-namenode](provider=redhat): Executing >>'/sbin/service hadoop-hdfs-namenode start' >> >> err: >>/Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-h >>dfs-namenode]/ensure: change from stopped to running failed: Could not >>start Service[hadoop-hdfs-namenode]: Execution of '/sbin/service >>hadoop-hdfs-namenode start' returned 3: at >>/mnt/var/lib/bootstrap-actions/1/bigtop-0.8.0/bigtop-deploy/puppet/module >>s/hadoop/manifests/init.pp:335 >> >> debug: Service[hadoop-hdfs-namenode](provider=redhat): Executing >>'/sbin/service hadoop-hdfs-namenode status' >> >> debug: >>/Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-h >>dfs-namenode]: Skipping restart; service is not running >> >> notice: >>/Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-h >>dfs-namenode]: Triggered 'refresh' from 4 events >> >> debug: >>/Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-h >>dfs-namenode]: The container Hadoop::Namenode[namenode] will propagate >>my refresh event >> >> debug: Hadoop::Namenode[namenode]: The container >>Class[Hadoop_head_node] will propagate my refresh event >> >> notice: >>/Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Package[hadoop >>-hdfs-datanode]: Dependency Service[hadoop-hdfs-namenode] has failures: >>true >> >> warning: >>/Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Package[hadoop >>-hdfs-datanode]: Skipping because of failed dependencies >> >> notice: >>/Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mnt1/hdf >>s]: Dependency Service[hadoop-hdfs-namenode] has failures: true >> >> warning: >>/Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mnt1/hdf >>s]: Skipping because of failed dependencies >> >> notice: >>/Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mnt/hdfs >>]: Dependency Service[hadoop-hdfs-namenode] has failures: true >> >> >> The interesting part is that if I query namenode status eventually the >>namenode will show up as started even though I have not taken any other >>actions: >> >> >> [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode >>status >> >> Hadoop namenode is not running [FAILED] >> >> [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode >>status >> >> Hadoop namenode is not running [FAILED] >> >> [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode >>status >> >> Hadoop namenode is running [ OK ] >> >> The same goes for proxy server. The problem here is that all other >> dependencies of namenode do not install (such as resource manager, >>etc). I >> am using the latest release of AmazonLinux (2014.09) and this has puppet >> 2.7.25-1. I am not sure what to do about this issue, has anyone else >> experienced something like this? Should I just move to puppet 3.x and >>only >> try to install out of the Bigtop trunk (0.9.0)? > >ResourceManager isn't a dependency of namenode - it's the other way >around. >It's hard to say what's going on with your system without looking into >particular daemon logs. I'd suggest you check them and investigate what's >the >trouble is. > >Also, there's a small issue BIGTOP-1522 with nodemanager recipes if you're >installing a custom set of components, which might or not affect you > >Cos >
