Alejandro Abdelnur commented on YARN-1039:

A few general comments:

* We need to have a flag to indicate an AM is long lived.
* We need to have another flag to indicate that the containers requested by an 
AM will be long lived, it must be set at application creation time and all 
containers of the app will be considered long lived. This is because the RM 
does not keep track of individual container requests.
* We need to sort out the delegation token story for long lived AMs and long 
lived containers. Delegation token story means, how do you renew and get new 
tokens indefinitely without having kerberos credentials at hand (a current 
requirement for renewing/getting-new delegation tokens). I believe [~adhoot] 
was discussing this already, I cannot find the JIRA.
* Having a long enum of flag to indicated optional qualities of the requested 
containers has been discussed in the past (in the context of some JIRAs related 
to Llama) and it has been discarded as it would mean divergence on the features 
different schedulers support.

> Add parameter for YARN resource requests to indicate "long lived"
> -----------------------------------------------------------------
>                 Key: YARN-1039
>                 URL: https://issues.apache.org/jira/browse/YARN-1039
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 3.0.0, 2.1.1-beta
>            Reporter: Steve Loughran
>            Assignee: Craig Welch
>         Attachments: YARN-1039.1.patch, YARN-1039.2.patch, YARN-1039.3.patch
> A container request could support a new parameter "long-lived". This could be 
> used by a scheduler that would know not to host the service on a transient 
> (cloud: spot priced) node.
> Schedulers could also decide whether or not to allocate multiple long-lived 
> containers on the same node

This message was sent by Atlassian JIRA

Reply via email to