[
https://issues.apache.org/jira/browse/OOZIE-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270805#comment-14270805
]
kaveen raajan v commented on OOZIE-1673:
----------------------------------------
I tried below solution it works perfectly for me.
1) Change the Hadoop schedule type from capacity scheduler to fair scheduler.
Because for small cluster each queue assign some memory size (2048MB) to
complete single map reduce job. If more than one map reduce job run in single
queue mean it met deadlock.
Solution: add below property to yarn-site.xml
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
</property>
<property>
<name>yarn.scheduler.fair.allocation.file</name>
<value>file:/%HADOOP_HOME%/etc/hadoop/fair-scheduler.xml</value>
</property>
2) By default Hadoop Total memory size was allot as 8GB.
So if we run two mapreduce program memory used by Hadoop get more than 8GB so
it met deadlock.
Solution: Increase the size of Total Memory of nodemanager using following
properties at yarn-site.xml
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>20960</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>1024</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>2048</value>
</property>
So If user try to run more than two mapreduce program mean he need to increase
nodemanager or he need to increase the size of total memory of Hadoop (note:
Increasing the size will reduce the system usage memory. Above property file
able to run 10 map reduce program concurrently.)
> "oozie.launcher.mapred.job.queue.name" as global property for oozie
> -------------------------------------------------------------------
>
> Key: OOZIE-1673
> URL: https://issues.apache.org/jira/browse/OOZIE-1673
> Project: Oozie
> Issue Type: New Feature
> Affects Versions: 4.0.0
> Reporter: Ostap
> Assignee: Ostap
>
> Currently, if not queue or a custom queue is set for an action, the launcher
> job will use the same queue.
> This leads to a deadlock situation in small clusters where all slots are
> taken by launcher jobs that cannot progress because there are not slots avail.
> That could be prevented by setting
> oozie.launcher.mapred.job.queue.name=launcherqueue.
> The problem is that we cannot specify the queue name on Oozie server. Each
> Oozie workflow job has to define its launcher job queue name
> (oozie.launcher.mapred.queue.name). If user didn’t specify this parameter in
> oozie’s job property, the “default” queue would be used for Launcher job,
> resulting in potential deadlock situation.
> We need to add a feature to Oozie so that it can take “oozie.launcher.xxx”
> parameter from oozie-site.xml so that user’s oozie job property doesn’t have
> to specify this parameter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
