[
https://issues.apache.org/jira/browse/YARN-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15865986#comment-15865986
]
Varun Saxena commented on YARN-6148:
------------------------------------
I fully understand the use case you are pointing out. Yes, the flaws with
respect to headroom won't be as problematic as it would be with blacklisting in
the scenario pointed above.
If extra headroom is reported, it will just lead to more reducers being ramped
up(as AM will think there are enough resources for map tasks) which may impact
assigning of resources to maps. But post MAPREDUCE-6302 this wont lead to
forever hanging of jobs. If we send headroom for each label separately we may
be able to ramp up/down reducers more effectively. However, headroom is used at
multiple places and there would be other areas to look at.
In principle I am fine with sending node count per label. This isn't
necessarily a use case for us as of now but in general there are use cases as
pointed out in discussion above as well, so we can handle it this way if there
is a general agreement with adding a new field in Allocate Response.
I will wait for feedback from Wangda on this and then we can proceed.
> NM node count reported to AM in Allocate Response should consider requested
> node label partitions.
> --------------------------------------------------------------------------------------------------
>
> Key: YARN-6148
> URL: https://issues.apache.org/jira/browse/YARN-6148
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Attachments: YARN-6148.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
