[
https://issues.apache.org/jira/browse/MAPREDUCE-5196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13867301#comment-13867301
]
Carlo Curino commented on MAPREDUCE-5196:
-----------------------------------------
Jason, it was not intentional I guess, we wanted to get "feedback" from the AM
about preemption to flow back to the Task as often as possible, and not just on
statusUpdates...
Hence folding the two calls, and have the optional "please preempt" message as
response. My understanding of ping vs statusUpdate was a keep-alive vs explicit
progress
reporting (I thought it was mostly to avoid overwhelming the AM with excessive
progress reporting). I thought the AM could see the lack of semantic progress
anyway and deal
with it.
I think the two alternatives are going back to ping() and statusUpdate() and
have both providing responses to have the preemption infos propagated, or
changes in a "no-progress"
statusUpdates handling/wiring to make sure to disambiguate the case you
mention. If you have a fix in mind just go ahead, I am happy to review it to
make sure the "preemption" aspects
are consistent with our intended semantics. Makes sense?
> CheckpointAMPreemptionPolicy implements preemption in MR AM via checkpointing
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5196
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5196
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mr-am, mrv2
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Fix For: 3.0.0
>
> Attachments: MAPREDUCE-5196.1.patch, MAPREDUCE-5196.2.patch,
> MAPREDUCE-5196.3.patch, MAPREDUCE-5196.patch, MAPREDUCE-5196.patch
>
>
> This JIRA tracks a checkpoint-based AM preemption policy. The policy handles
> propagation of the preemption requests received from the RM to the
> appropriate tasks, and bookeeping of checkpoints. Actual checkpointing of the
> task state is handled in upcoming JIRAs.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
