[
https://issues.apache.org/jira/browse/TEZ-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16293297#comment-16293297
]
Jason Lowe commented on TEZ-3770:
---------------------------------
Thanks for kicking the tires on the patch, Eric!
The patch doesn't modify the code for the default scheduler,
YarnTaskSchedulerService, so I can't readily explain things if this patch makes
that scheduler unable to run that workload. Is it able to run that workload
without this patch?
I could see that behavior occurring if you also had pulled in the patch for
TEZ-394, as it's a known problem causing massive preemptions for certain DAG
topologies when combining TEZ-394 with YarnTaskSchedulerService.
> DAG-aware YARN task scheduler
> -----------------------------
>
> Key: TEZ-3770
> URL: https://issues.apache.org/jira/browse/TEZ-3770
> Project: Apache Tez
> Issue Type: New Feature
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: TEZ-3770.001.patch
>
>
> There are cases where priority alone does not convey the relationship between
> tasks, and this can cause problems when scheduling or preempting tasks. If
> the YARN task scheduler was aware of the relationship between tasks then it
> could make smarter decisions when trying to assign tasks to containers or
> preempt running tasks to schedule pending tasks.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
