[
https://issues.apache.org/jira/browse/MAPREDUCE-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated MAPREDUCE-4813:
----------------------------------
Attachment: MAPREDUCE-4813-2.patch
Thanks for the review, Bobby. Updated the patch to address the review comments:
* I went ahead and fixed the JOB_START in KILLED race since it's a one-line fix
to ignore that event.
* I noticed that JOB_TASK_COMPLETED also needs to be handled in the FAIL_ABORT
state.
* The commit cancel wait loop was busted, as it needs to recompute {{now}}
within the loop to function properly.
* Standardized on StringUtils.stringifyException to report committer exceptions
in diagnostics. I had previously preserved the existing behavior, but I agree
we should be consistent.
* Updated IOException to Exception within the try block of committer calls to
make sure we don't "leak" exceptions and deadlock.
> AM timing out during job commit
> -------------------------------
>
> Key: MAPREDUCE-4813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4813
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Priority: Critical
> Attachments: JobImplStateMachine.pdf, MAPREDUCE-4813-2.patch,
> MAPREDUCE-4813-2.patch, MAPREDUCE-4813-2.patch, MAPREDUCE-4813-2.patch,
> MAPREDUCE-4813.patch, MAPREDUCE-4813.patch, MAPREDUCE-4813.patch
>
>
> The AM calls the output committer's {{commitJob}} method synchronously during
> JobImpl state transitions, which means the JobImpl write lock is held the
> entire time the job is being committed. Holding the write lock prevents the
> RM allocator thread from heartbeating to the RM. Therefore if committing the
> job takes too long (e.g.: the job has tons of files to commit and/or the
> namenode is bogged down) then the AM appears to be unresponsive to the RM and
> the RM kills the AM attempt.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
