[
https://issues.apache.org/jira/browse/TEZ-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14979285#comment-14979285
]
Hitesh Shah commented on TEZ-2914:
----------------------------------
\cc [~bikassaha] [~sseth]
If the intention is to ensure good job completion time while at the same time
not hogging too many resources on a cluster, a cluster resource cap might be
something to think about. Once this is in place, the Tez scheduler could make
smarter decisions on when/what to schedule.
If this is due to tasks making calls to other services/endpoints which get
overloaded if there are many tasks running concurrently, then yes, a cap on the
concurrent tasks being executed makes sense. Have you considered how we can
augment the VertexManager to enforce this cap? This might be a good time to
also figure out how to modularize the VertexManager so that the user configured
VM could be based on a set of building blocks ( each block providing a
different feature ) instead of a monolith.
> Ability to limit vertex concurrency
> -----------------------------------
>
> Key: TEZ-2914
> URL: https://issues.apache.org/jira/browse/TEZ-2914
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jonathan Eagles
>
> Add equivalent functionality of *MAPREDUCE-5583. Ability to limit running map
> and reduce tasks* in tez
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
