Thanks Robert.I think I crossed past the issue by putting oozie.launcher.mapreduce.map.memory.mb property in the workflow.
On 9/5/14, 5:06 PM, "Robert Kanter" <[email protected]> wrote: >Hi Sathi, > >Is it the launcher job (i.e. your driver code) that's getting the OOM or >the MR job itself? >In general, the properties in <configuration> get applied only to the MR >job. If you want them to be applied to the launcher job, you should >prefix >the properties with "oozie.launcher" (e.g. >"oozie.launcher.mapreduce.map.memory.mb"). > > >Also, with the Java action, if you want the <configuration> properties >from >the workflow XML to get passed to your driver for launching the MR job, >you >have to load an XML file prepared by Oozie and use that for your MR job's >JobConf. It's not currently documented, but you can look at OOZIE-1969 ><https://issues.apache.org/jira/browse/OOZIE-1696> for details. > >- Robert > > > >On Fri, Sep 5, 2014 at 4:41 PM, Sathi Chowdhury ><[email protected] >> wrote: > >> >> >Trying to run a java action (this is the driver class for running a map >> >reduce job) >> >> Here is my workflow xml >> <action name ="loaddata"> >> <java> >> <job-tracker>${jobTracker}</job-tracker> >> <name-node>${nameNode}</name-node> >> <job-xml>${workflowRoot}/hive-site.xml</job-xml> >> <configuration> >> <property> >> <name>mapred.job.queue.name</name> >> <value>${queueName}</value> >> </property> >> <property> >> <name>oozie.action.max.output.data</name> >> <value>100000</value> >> </property> >> <property> >> <name>mapreduce.reduce.memory.mb</name> >> <value>4096</value> >> </property> >> <property> >> <name>mapreduce.reduce.java.opts</name> >> <value>-Xms4g -Xmx4g</value> >> </property> >> <property> >> <name>mapreduce.map.memory.mb</name> >> <value>4096</value> >> </property> >> <property> >> <name>mapreduce.map.java.opts</name> >> <value>-Xmx4g -Xms4g</value> >> </property> >> <property> >> <name>oozie.launcher.mapred.child.java.opts</name> >> <value>-Xmx2048m</value> >> </property> >> <property> >> <name>hcat.metastore.maxpartitions</name> >> <value>200000</value> >> </property> >> </configuration> >> <main-class>x.y.z.ParseXTables</main-class> >> <java-opts>-Xmx4g -Xms4g</java-opts> >> <arg>-conf</arg> >> <arg>defaultconf.xml</arg> >> <file>${workflowRoot}/hive-site.xml</file> >> <file>lib/${EXEC}#${EXEC}</file> >> <file>${workflowRoot}/defaultconf.xml</file> >> <capture-output/> >> </java> >> <ok to="mark-daily-job-success" /> >> <error to="fail" /> >> </action> >> >> >> >> I am getting this error in the attempt log, I am thinking this happens >> when oozie launcher tries to run >> >> From the job history the job_1409808477902_0089_conf.xml shows >> mapreduce.map.memory.mb 2151 >> >> >> Wondering why did it not get propagated from the oozie workflow setting >> and should have been 4096. >> >> Not sure what else I have missied. >> Will appreciate any clue on this. >> Thanks >> Sathi >> >> >> >>
