[
https://issues.apache.org/jira/browse/TEZ-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034325#comment-15034325
]
Bikas Saha commented on TEZ-2914:
---------------------------------
At least 2 ways to do this.
1) Make the change in the yarn task scheduler - now that we have external
facing id's we could buffer requests by vertexId and add them to the actual
scheduling based on allowed concurrency. Cons would be that this would make
this a user land feature. The buffering has to be done before the request
reaches AMRMCient as otherwise, the combination of YARN requests and internal
concurrency limit would be difficult to coordinate.
2) Make the change in the DAGScheduler. The intent of the dag scheduler was
supposed to be the central choke point where resource sharing and priority
between vertices overall could be applied. Requests that go to it from attempts
could be buffered after reaching the max desired concurrency. This would allow
vertex concurrency limit, overall dag concurrency limit and other combinations.
> Ability to limit vertex concurrency
> -----------------------------------
>
> Key: TEZ-2914
> URL: https://issues.apache.org/jira/browse/TEZ-2914
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jonathan Eagles
> Assignee: Bikas Saha
>
> Add equivalent functionality of *MAPREDUCE-5583. Ability to limit running map
> and reduce tasks* in tez
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
