[
https://issues.apache.org/jira/browse/TEZ-1742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14203328#comment-14203328
]
Bikas Saha commented on TEZ-1742:
---------------------------------
Good catch. The latest addendum patch fixes the bug and adds a unit test for
that method. Please review.
This jiras original patch contains not just the percentage improvement but also
bug fixes where incorrect priorities were being compared and we could end up
preempting the wrong tasks. The addendum patch has a test enhancement for that
bug fix. It also contains a fix to not trigger task preemption in back-to-back
cycles so that we dont preempt before the RM can react to the preemption. The
existing behavior of fewer preemptions can be enabled by setting a sufficiently
low value of the preemptionPercentage, if needed. Hence, I feel that this patch
is relevant for branch-0.5 and should make its way there. Proportional
preemptions is a general improvement, that helps out of order scheduling, but
is unrelated to it and can help in any other case where scheduling constraints
need to be enforced quickly. e.g. losing many tasks on a rack or a re-execution
of all reducers because of a correlated outage.
> Improve response time of internal preemption
> --------------------------------------------
>
> Key: TEZ-1742
> URL: https://issues.apache.org/jira/browse/TEZ-1742
> Project: Apache Tez
> Issue Type: Task
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Fix For: 0.5.3
>
> Attachments: TEZ-1742.1.patch, TEZ-1742.2.patch, TEZ-1742.3.patch,
> TEZ-1742.4.patch, TEZ-1742.4.patch.addendum, TEZ-1742.addendum.2.patch
>
>
> Tez YARN Task Scheduler currently preempts 1 running task at a time when a
> higher priority task is waiting and there are no available resources. When a
> large number of higher priority tasks are pending then it can take a long
> time to preempt the required number of lower priority tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
