[
https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096938#comment-15096938
]
Nathan Roberts commented on YARN-1011:
--------------------------------------
Thanks for the update [~kasha]. I have a few questions but I'll start at a high
level since the answers may clear up others.
bq. On SchedulerNode#allocateContainer, mark the container OPPORTUNISTIC if
allocating this container would take the allocation over the advertised
capacity. And, keep track of opportunistic containers in a list separate from
launchedContainers.
It sounds like any container from any queue (assuming CapacityScheduler here
but similar constructs will exist in other schedulers) could be marked
OPPORTUNISTIC. Does this mean a queue with SLA sensitive jobs could get
OPPORTUNISTIC containers? I just don't see how these jobs will be able to meet
their SLAs with the uncertainty as to the resources they'll actually get, and
the likelihood they'll get shot by the NM. This is precisely the reason there
is the ability to designate queues as preemptable since SLA sensitive jobs just
don't do well in preemtable queues.
> [Umbrella] Schedule containers based on utilization of currently allocated
> containers
> -------------------------------------------------------------------------------------
>
> Key: YARN-1011
> URL: https://issues.apache.org/jira/browse/YARN-1011
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Arun C Murthy
> Attachments: yarn-1011-design-v0.pdf, yarn-1011-design-v1.pdf,
> yarn-1011-design-v2.pdf
>
>
> Currently RM allocates containers and assumes resources allocated are
> utilized.
> RM can, and should, get to a point where it measures utilization of allocated
> containers and, if appropriate, allocate more (speculative?) containers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
