I had somewhat similar problem ,
If you want to apply a memory setting to the oozie launcher job then I
used these in the pipeline workflow xml
<property>
<name>oozie.launcher.mapreduce.map.memory.mb</name>
<value>8096</value>
</property>
<property>
<name>oozie.launcher.mapreduce.reduce.memory.mb</name>
<value>8096</value>
</property>
<property>
<name>oozie.launcher.mapreduce.child.java.opts</name>
<value>-Xmx8g</value>
</property>
When I wanted apply specific memory settings to the actual java actions
and it¹s mr run,
I made my java action class take a config and keep the map reduce related
params in it.
hadoop jar lib/yourjavaactionclasscontainsinthis.jar
com.x.y.JavaActionClass -libjars ${LIB_JARS} -conf yourconf.xml
And kept required values in this config
<property>
<name>mapreduce.job.reduces</name>
<value>49</value>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>4096</value>
</property>
<property>
<name>mapreduce.reduce.java.opts</name>
<value>-Xmx3276m</value>
</property>
<property>
<name>mapreduce.map.memory.mb</name>
<value>4096</value>
</property>
<property>
<name>mapreduce.map.java.opts</name>
<value>-Xmx3276m</value>
</property>
(actually you can invoke this command line and see without oozie it is
applying those settings ,then you are fine to integrate with oozie)
A snippet of my job defn
<main-class>com.x.y.z.parser.MyParser</main-class>
<java-opts>-Xmx8g -Xms8g</java-opts>
<arg>-conf</arg>
<arg>parse-conf.xml</arg>
<arg>-libjars</arg>
<arg>
hcatalog-core-0.12.0.2.0.11.0-1.jar,hive-metastore-0.12.0.2.0.11.0-1.jar,li
bfb303-0.9.0.jar,hive-exec-0.12.0.2.0.11.0-1.jar,jdo-api-3.0.1.jar,libthrif
t-0.9.0.jar,slf4j-api-1.6.6.jar
</arg>
<file>${workflowRoot}/lib/${JAR_FILE}#${JAR_FILE}</file>
<file>parse-conf.xml</file>
<capture-output/>
</java>
Thx and please feel free to let me know if any of it did not make sense or
you need more info
Then the actual map reduce job will obey these settings ,if you need any
more
On 11/5/14, 3:08 AM, "Jakub Stransky" <[email protected]> wrote:
>Hello experienced oozie users,
>
>I would like to adjust hadoop settings for given coordinator/workflow job,
>particullarily I am speaking about:
>yarn.app.mapreduce.am.resource.mb
>yarn.app.mapreduce.am.command-opts
>mapreduce.map.java.opts
>mapreduce.map.memory.mb
>
>Cluster defaults are to high and consume unreasonable memory amount for
>this particular workflow. I tried to put these settings to bundle.xml for
>a given coordinator, to coordinator
>
