[
https://issues.apache.org/jira/browse/MAPREDUCE-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167894#comment-13167894
]
Hitesh Shah commented on MAPREDUCE-3426:
----------------------------------------
Summary of changes:
LocalContainerLauncher changes:
- in runSubTask(), set the correct config values for task specific
information as well as the local dirs setting so that the output is written to
the correct dir ( which the process has write permissions to )
- cleaned up commented out code.
Added flags to be able to profile map and reduce tasks with separate profiling
params.
MRAppMaster changes:
- fixed disabling of speculative execution if job is uberized.
- Moved uber decision into new function
- hardcoded max reduce count supported to 1 as multiple reduces are not
supported.
- using MR AM vm size as memory bound check for uber decision
RMCommunicator change:
- for MR-3421
JobStatus/JobReport changes:
- Print out uber choice on client side to inform user whether running job in
uber or non-uber mode
Comments changes:
- Removed comments on race condition in local launcher for "MapTask can't
send TA_UPDATE before TA_CONTAINER_LAUNCHED"
- the state machine has a write lock, also the launched event is sent
first followed by any updates if any from the task itself.
- renameMapOutputForReduce is still required. Output files are generated
into top level dir and not a task specific dir.
> uber-jobs tried to write outputs into wrong dir
> -----------------------------------------------
>
> Key: MAPREDUCE-3426
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3426
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Hitesh Shah
> Fix For: 0.23.1
>
> Attachments: MR-3426.1.patch, MR-3426.2.patch
>
>
> Incorrect setup of the uber tasks causes tasks to try to write intermidiate
> outputs into dirs that the user does not have permissions to write to on a
> secure cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
