[
https://issues.apache.org/jira/browse/MAPREDUCE-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13662166#comment-13662166
]
Carlo Curino commented on MAPREDUCE-5176:
-----------------------------------------
I like the idea Alejandro, especially as more and more AMs will come into
existence, providing some of this in the common layer of YARN might be
interesting, and maybe help factor out some of the basic mechanisms. We do
something in this direction for the checkpoint service itself. We posted it in
MAPREDUCE-5197, but we are internally are experiencing reusing it for other
AMs, so maybe you are right this belongs in some "common" part of the codebase.
I would like to gather some more opinions on this, and try to build consensus
before starting to shuffle this patches around.
Thoughts on the idea of having some place in YARN (or whatever other common
place) to put these annotations and maybe the basic common checkpoint service?
> Preemptable annotations (to support preemption in MR)
> -----------------------------------------------------
>
> Key: MAPREDUCE-5176
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5176
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Attachments: MAPREDUCE-5176.1.patch, MAPREDUCE-5176.patch
>
>
> Proposing a patch that introduces a new annotation @Preemptable that
> represents to the framework property of user-supplied classes (e.g., Reducer,
> OutputCommiter). The intended semantics is that a tagged class is safe to be
> preempted between invocations.
> (this is in spirit similar to the Output Contracts of [Nephele/PACT |
> https://stratosphere.eu/sites/default/files/papers/ComparingMapReduceAndPACTs_11.pdf])
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
