Wangda Tan commented on YARN-1197:

bq. To clarify: with proper tuning, we can currently get low hundreds of 
milliseconds without adding any new scheduler features. With the new scheduler 
feature I'm imagining, we'd only be limited by the RPC + scheduler time, so we 
could get 10s of milliseconds with proper tuning.
I think this assumes cluster is quite idle, I understand the low latency could 
be achieved, but it's not guaranteed since we don't support oversubscribing, 
etc. If you assume the cluster is very idle, one solution might be holding more 
resource at the beginning instead of increasing. In real environment, I think 
the expectation of delay should still be seconds level.

>From YARN's perspective, (b) handles most of logic within YARN daemons 
>(instead of AM), we don't need to consider inconsistency status between RM/AM 
>when doing recovery, that is really what I prefer :). I'm not against of doing 
>(a), but I prefer to do that when we have solid foundation for fast 
>scheduling. I'm not sure if there's any resource management platform in 
>production supports that, but some research papers such as Sparrow uses quite 
>different protocol/approach than YARN. I expect there're still some TODO items 
>for YARN to get guaranteed fast scheduling.

> Support changing resources of an allocated container
> ----------------------------------------------------
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>         Attachments: YARN-1197 old-design-docs-patches-for-reference.zip, 
> YARN-1197_Design.pdf
> The current YARN resource management logic assumes resource allocated to a 
> container is fixed during the lifetime of it. When users want to change a 
> resource 
> of an allocated container the only way is releasing it and allocating a new 
> container with expected size.
> Allowing run-time changing resources of an allocated container will give us 
> better control of resource usage in application side

This message was sent by Atlassian JIRA

Reply via email to