Attila Sasvari created OOZIE-3007:
-------------------------------------
Summary: Ability to override default Map Reduce properties via
default action config file
Key: OOZIE-3007
URL: https://issues.apache.org/jira/browse/OOZIE-3007
Project: Oozie
Issue Type: Bug
Reporter: Attila Sasvari
According to https://oozie.apache.org/docs/4.2.0/AG_ActionConfiguration.html ,
it is possible to specify for default configuration parameters for Oozie
actions by the means of action configuration files.
However, this mechanism does not work for Map Reduce actions.
For example, assume we created {{/var/lib/oozie/action-conf/map-reduce.xml}}
(readable by Oozie) with the following content:
{code}
cat map-reduce.xml
<configuration>
<property>
<name>oozie.launcher.mapred.fairscheduler.pool</name>
<value>root.support.coe</value>
</property>
<property>
<name>oozie.launcher.mapred.job.queue.name</name>
<value>oozie_queue</value>
</property>
<property>
<name>mapreduce.job.queuename</name>
<value>oozie_queue</value>
</property>
<property>
<name>mapred.queue.name</name>
<value>oozie_queue</value>
</property>
</configuration>
{code}
and added the following to {{oozie-site.xml}}
{code}
<property>
<name>oozie.service.HadoopAccessorService.action.configurations</name>
<value>*=/var/lib/oozie/action-conf</value>
</property>
{code}
the mapper task started by Oozie will be put into the {{default}} queue instead
of {{oozie-launcher}}.
h5. Possible reason:
HadoopAccessorService's {{loadHadoopConfigs()}} loads default hadoop
configuration settings from hadoop config files.
{{JavaActionExecutor}} loads default action configuration from given files with
[createActionDefaultConf|https://github.com/apache/oozie/blob/84eb493642d35b8bb1e7527ef51b6ecfa40dd592/core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java#L458]
via HadoopAccessorService in
[setupActionConf|https://github.com/apache/oozie/blob/84eb493642d35b8bb1e7527ef51b6ecfa40dd592/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L389].
{code}
XConfiguration actionDefaults =
has.createActionDefaultConf(conf.get(HADOOP_YARN_RM), getType());
XConfiguration.injectDefaults(actionDefaults, actionConf);
{code}
However, {{actionConf}} already contains default value for
"{{mapreduce.job.queuename}}" so
[injectDefaults|https://github.com/apache/oozie/blob/84eb493642d35b8bb1e7527ef51b6ecfa40dd592/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L389]
wont add user specified "{{mapreduce.job.queuename}}" default queue (i.e.
{{oozie_queue}}).
I tested this behaviour with a 4.2.0 based Oozie, but the existing code did not
change here by OYA.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
