[
https://issues.apache.org/jira/browse/TEZ-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200257#comment-15200257
]
Jason Lowe commented on TEZ-3168:
---------------------------------
Couple of points after a quick review:
Looking at the queue capacity could be very wrong in cases where the user
limits only allow the user a tiny fraction of the queue. The Tez AM will think
it has access to a lot more than it really does.
Apps can be moved between queues, so if someone moved the Tez AM from one queue
to another it could be looking at the wrong queue when it makes decisions.
It's unfortunate that the root queue metrics aren't conveyed in the metrics
returned from getYarnClusterMetrics. They are tracked in the RM ClusterMetrics
but for some reason not conveyed to the client. That would be cheaper for both
the RM and the Tez AM since both could avoid looping over root-level queues.
But that would couple this with a pending YARN change.
> Provide a more predictable approach for total resource guidance for
> wave/split calculation
> -------------------------------------------------------------------------------------------
>
> Key: TEZ-3168
> URL: https://issues.apache.org/jira/browse/TEZ-3168
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Hitesh Shah
> Assignee: Hitesh Shah
> Attachments: TEZ-3168.wip.patch
>
>
> Currently, Tez uses headroom for checking total available resources. This is
> flaky as it ends up causing the split count to be determined by a point in
> time lookup at what is available in the cluster. A better approach would be
> either the queue size or even cluster size to get a more predictable count.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
