Sandy Ryza commented on YARN-1197:

bq. I think this assumes cluster is quite idle, I understand the low latency 
could be achieved, but it's not guaranteed since we don't support 
oversubscribing, etc.
If the cluster is fully contended we certainly won't get this performance.  But 
as long as there is a decent chunk of space, which is common in many settings, 
we can.  The cluster doesn't need to be fully idle by any means.

More broadly, just because YARN is not good at hitting sub-second latencies 
doesn't mean that it isn't a design goal.  I strongly oppose any argument that 
uses the current slowness of YARN as a justification for why we should make 
architectural decisions that could compromise latencies.

That said, I still don't have a strong grasp on the kind of complexity we're 
introducing in the AM, so would like to try to understand that before arguing 
against you further.

Is the main problem we're grappling still the one Meng brought up here:
I.e. that an AM can receive an increase from the RM, then issue a decrease to 
the NM, and then use its increase to get resources it doesn't deserve?

Or is the idea that, even if we didn't have this JIRA, NMClient is too 
complicated, and we'd like to reduce that?

> Support changing resources of an allocated container
> ----------------------------------------------------
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>         Attachments: YARN-1197 old-design-docs-patches-for-reference.zip, 
> YARN-1197_Design.pdf
> The current YARN resource management logic assumes resource allocated to a 
> container is fixed during the lifetime of it. When users want to change a 
> resource 
> of an allocated container the only way is releasing it and allocating a new 
> container with expected size.
> Allowing run-time changing resources of an allocated container will give us 
> better control of resource usage in application side

This message was sent by Atlassian JIRA

Reply via email to