Steve Loughran commented on YARN-1039:

I've always envisaged the flag could switch on some different policies, though 
with container-preservation across restarts, labels, log aggregation and 
windows for failure tracking, much of that is dealt with.

Otherwise, the longevity flag could be of use in
# RM UI. There's no "percentage done" any more, more "live/not-live". This 
already causes confusion for our slider users.
# placement: do you want 100% of a node capacity to be for long-lived stuff, at 
the expense of being able to run anything short-lived there?
# pre-emption. The cost of pre-emption may be higher, but at the same time 
long-lived containers are the ones you may want to pre-empt the most, because 
the scheduler knows they won't go away any time soon.

The easy target is the UI, as that doesn't need scheduling changes, and the 
current "percentage done" view doesn't work. Something to indicate live/not 
live makes more sense (though not red/green unless you don't want colour blind 
people using your app)

> Add parameter for YARN resource requests to indicate "long lived"
> -----------------------------------------------------------------
>                 Key: YARN-1039
>                 URL: https://issues.apache.org/jira/browse/YARN-1039
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 3.0.0, 2.1.1-beta
>            Reporter: Steve Loughran
>            Assignee: Craig Welch
>         Attachments: YARN-1039.1.patch, YARN-1039.2.patch, YARN-1039.3.patch
> A container request could support a new parameter "long-lived". This could be 
> used by a scheduler that would know not to host the service on a transient 
> (cloud: spot priced) node.
> Schedulers could also decide whether or not to allocate multiple long-lived 
> containers on the same node

This message was sent by Atlassian JIRA

Reply via email to