[
https://issues.apache.org/jira/browse/YARN-568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13651550#comment-13651550
]
Carlo Curino commented on YARN-568:
-----------------------------------
Sandy, I ran, in a small 10 machines cluster, the FS patched with your latest
edit.
I confirmed the previous correct behavior we had in our patch, i.e., the
scheduler logged preemption and killing actions.
I also ran it with our "preemptable MR" with checkpointing as an AM, and the
preemption messages were propagated to the AM
correctly (and checkpointing triggered).
(Also we independently tested that the PreemptionContract, side of YARN-45 was
exercised properly by the CapacityScheduler, so
also YARN-567 which is now rolled into this seem in good shape).
So I would say, this patch is good to go for me.
As a separate consideration, I think we should work on good defaults parameters
and good indications on how to tune this preemption policy.
> FairScheduler: support for work-preserving preemption
> ------------------------------------------------------
>
> Key: YARN-568
> URL: https://issues.apache.org/jira/browse/YARN-568
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Attachments: YARN-568-1.patch, YARN-568-2.patch, YARN-568-2.patch,
> YARN-568.patch, YARN-568.patch
>
>
> In the attached patch, we modified the FairScheduler to substitute its
> preemption-by-killling with a work-preserving version of preemption (followed
> by killing if the AMs do not respond quickly enough). This should allows to
> run preemption checking more often, but kill less often (proper tuning to be
> investigated). Depends on YARN-567 and YARN-45, is related to YARN-569.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
