Steve Loughran commented on YARN-1039:

bq. However, this can be done either based on historical information (previous 
waves of this task type or previous execution of the same job) or on 
application level knowledge.

Historical information is generally the best estimate, though if the input data 
is different, so can duration.

Maybe a simple enum as "short-lived", "session", and "service": services 
provide no termination, session = "a few hours to a few days (i.e within the 
lifespan of kerberos tokens). 

> Add parameter for YARN resource requests to indicate "long lived"
> -----------------------------------------------------------------
>                 Key: YARN-1039
>                 URL: https://issues.apache.org/jira/browse/YARN-1039
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 3.0.0, 2.1.1-beta
>            Reporter: Steve Loughran
>            Assignee: Craig Welch
>         Attachments: YARN-1039.1.patch, YARN-1039.2.patch, YARN-1039.3.patch
> A container request could support a new parameter "long-lived". This could be 
> used by a scheduler that would know not to host the service on a transient 
> (cloud: spot priced) node.
> Schedulers could also decide whether or not to allocate multiple long-lived 
> containers on the same node

This message was sent by Atlassian JIRA

Reply via email to