[
https://issues.apache.org/jira/browse/YARN-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962152#comment-13962152
]
Thomas Graves commented on YARN-1769:
-------------------------------------
Thanks for the review Jason.
- LeafQueue.findNodeToUnreserve is adjusting the headroom when it unreserves,
but I don't see other unreservations doing a similar calculation. Wondering if
this fixup is something that should have been in completedContainer or needs to
be done elsewhere? I could easily be missing something here but asking just in
case other unreservation situations also need to have the headroom fixed.
You have to adjust the headroom otherwise we will have subtracted it twice.
When you allocate a container (reservation or normal), it is subtracted from
your headroom. Normally then when it goes to schedule a reserved resource, it
goes into assignReservedContainer which makes sure it doesn't subtract it
twice. In our case we first unreserve so we have to add that headroom back,
then we schedule and the normal schedule path will handle subtracting it again.
Also note that it is possible to unreserve a container with size greater then
what we are now going to take. There are a bunch of issues with headroom, most
listed under YARN-1198. I think we should leave any other fixups to that jira.
- In LeafQueue.findNodeToUnreserve, isn't it kinda bad if the app thinks it
has reservations on the node but the scheduler doesn't know about it? Wondering
if the bookkeeping is messed up at that point therefore something a bit more
than debug is an appropriate log level and if further fixup is needed.
I put this in just as a precaution. I'll change this it log an error. I
don't think I want to try to fix it up as that might just cause more issues.
I addressed the rest of the comments. I'll upload a new patch shortly.
> CapacityScheduler: Improve reservations
> ----------------------------------------
>
> Key: YARN-1769
> URL: https://issues.apache.org/jira/browse/YARN-1769
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
> Affects Versions: 2.3.0
> Reporter: Thomas Graves
> Assignee: Thomas Graves
> Attachments: YARN-1769.patch, YARN-1769.patch, YARN-1769.patch,
> YARN-1769.patch, YARN-1769.patch, YARN-1769.patch, YARN-1769.patch
>
>
> Currently the CapacityScheduler uses reservations in order to handle requests
> for large containers and the fact there might not currently be enough space
> available on a single host.
> The current algorithm for reservations is to reserve as many containers as
> currently required and then it will start to reserve more above that after a
> certain number of re-reservations (currently biased against larger
> containers). Anytime it hits the limit of number reserved it stops looking
> at any other nodes. This results in potentially missing nodes that have
> enough space to fullfill the request.
> The other place for improvement is currently reservations count against your
> queue capacity. If you have reservations you could hit the various limits
> which would then stop you from looking further at that node.
> The above 2 cases can cause an application requesting a larger container to
> take a long time to gets it resources.
> We could improve upon both of those by simply continuing to look at incoming
> nodes to see if we could potentially swap out a reservation for an actual
> allocation.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
