[
https://issues.apache.org/jira/browse/TEZ-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14979050#comment-14979050
]
Jason Lowe commented on TEZ-2914:
---------------------------------
If we're trying to port the capability of MAPREDUCE-5583 then it would be more
of a per-vertex concurrent cap rather than an overall concurrent task cap. The
main use case for MAPREDUCE-5583 is to avoid having the job hammer some service
"too hard," and I would assume that typically that service would be hit by one
vertex rather than every vertex in the DAG. Therefore it would be wasteful to
place the cap across all tasks rather than on the one vertex where it matters.
The problem then becomes how does a user specify these limits. Pig will be
interested in trying to map these properties to the Tez job. I'm assuming it
will simply apply the map property to all root vertices and the reduce property
to any non-root vertex. That should make it run similarly to how it did in
MapReduce, and I think we'd only need support for per-vertex task concurrency
limits in Tez.
> Ability to limit vertex concurrency
> -----------------------------------
>
> Key: TEZ-2914
> URL: https://issues.apache.org/jira/browse/TEZ-2914
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jonathan Eagles
>
> Add equivalent functionality of *MAPREDUCE-5583. Ability to limit running map
> and reduce tasks* in tez
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
