Hi All,
I have a 3 node Hadoop-2.6.0 cluster with each Node having 32GB and 8 Cores.
yarn-site.xml
===========
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>24576</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>2048</value>
</property>
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>4</value>
</property>
mapred-site.xml
==============
<property>
<name>mapred.reduce.java.opts</name>
<value>-Xmx6144m -XX:-UseGCOverheadLimit -XX:+UseCompressedOops</value>
</property>
<property>
<name>mapreduce.map.log.level</name>
<value>ERROR</value>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>8192</value>
</property>
<property>
<name>mapreduce.map.memory.mb</name>
<value>4096</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.map.java.opts</name>
<value>-Xmx3072m</value>
</property>
<property>
<name>mapreduce.reduce.input.buffer.percent</name>
<value>0.7</value>
</property>
Hadoop shows:
[cid:[email protected]]
But still I keep getting the Heart beat issue.
>>> Invoking Main class now >>>
Main class : com.test.oozie.action.java.ShellCommandExecuter
Arguments :
root
pass
22
192.xxx.xxx.xxx
/tmp/test/WorkFlow1449055033241
shell14490550336199945.sh
ShellCommandExecuter Args Length :::6
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Q1: Is this Heat beat issue specific to Oozie ONLY?
Reason I am asking this is since my other MR jobs(SQOOP command fired
independently or HQLs) are getting executed successfully.
But a simple Java Action when triggered from Oozie hangs with Heart beat.
Q2: I read somewhere that the Heart beat issue comes in case we have limited
resources like in a single node Hadoop installation. There we can use fair
Scheduler.
Reference:
https://community.cloudera.com/t5/Batch-Processing-and-Workflow/Oozie-sqoop-action-in-CDH-5-2-Heart-beat-issue/td-p/22181
But I think the Heart beat issue should not be coming as I have a 3 Node Hadoop
cluster with ample resources.
Any pointers to the root cause would be of great help for me.
Thanks,
-Nirmal
________________________________
NOTE: This message may contain information that is confidential, proprietary,
privileged or otherwise protected by law. The message is intended solely for
the named addressee. If received in error, please destroy and notify the
sender. Any use of this email is prohibited when received in error. Impetus
does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors,
virus, interception or interference.
