[
https://issues.apache.org/jira/browse/OOZIE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Cseh updated OOZIE-2687:
------------------------------
Description:
It was possible to manipulate the MapperLauncher's environment through
properties like:
# mapreduce.map.memory.mb
# mapreduce.map.cpu.vcores
# mapred.child.env
# mapred.child.java.opts
# mapred.job.queue.name - ability to set launcher queue
E.g. We were using mapred.child.env to pass SPARK_HOME to the LauncherMapper
and make PySpark work.
Fixing OOZIE-2596 added a hack. We should decide how we support or break
compatibility and how we allow the manipulation of the Launcher environment.
Verify if the new launcher section in global applies to actions in
sub-workflows as well. It did not use to work before and was only fixed in
OOZIE-2030. It would be good to have that testcase
(TestSubWorkflowActionExecutor. testParentGlobalConf) updated with the new
launcher section as well.
was:
It was possible to manipulate the MapperLauncher's environment through
properties like:
# mapreduce.map.memory.mb
# mapreduce.map.cpu.vcores
# mapred.child.env
# mapred.child.java.opts
# mapred.job.queue.name - ability to set launcher queue
E.g. We were using mapred.child.env to pass SPARK_HOME to the LauncherMapper
and make PySpark work.
Fixing OOZIE-2596 added a hack. We should decide how we support or break
compatibility and how we allow the manipulation of the Launcher environment.
> OYA: Figure out how to handle MR-specific properties
> ----------------------------------------------------
>
> Key: OOZIE-2687
> URL: https://issues.apache.org/jira/browse/OOZIE-2687
> Project: Oozie
> Issue Type: Sub-task
> Reporter: Peter Cseh
> Assignee: Peter Bacsko
>
> It was possible to manipulate the MapperLauncher's environment through
> properties like:
> # mapreduce.map.memory.mb
> # mapreduce.map.cpu.vcores
> # mapred.child.env
> # mapred.child.java.opts
> # mapred.job.queue.name - ability to set launcher queue
> E.g. We were using mapred.child.env to pass SPARK_HOME to the LauncherMapper
> and make PySpark work.
> Fixing OOZIE-2596 added a hack. We should decide how we support or break
> compatibility and how we allow the manipulation of the Launcher environment.
> Verify if the new launcher section in global applies to actions in
> sub-workflows as well. It did not use to work before and was only fixed in
> OOZIE-2030. It would be good to have that testcase
> (TestSubWorkflowActionExecutor. testParentGlobalConf) updated with the new
> launcher section as well.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
