Thanks, makes sense now. On Feb 19, 2014 9:35 PM, "Jian He" <[email protected]> wrote:
> Yes, in the scenario you mentioned, the scheduler will take away the 10% > from queue B and give it back to queue A > > > On Wed, Feb 19, 2014 at 5:35 PM, Alex Nastetsky <[email protected]>wrote: > >> Thanks Jian. >> >> In what sense is the "capacity" resources "guaranteed"? >> >> Let's say there are two queues at the root level, A and B. Both have >> capacity of 50% and max capacity of 70%. If queue A is only currently using >> 40% and queue B is using 50% and needs more, then it will be able to borrow >> 10% from queue A, so now queue A is using 40% and queue B is using 60%. But >> what happens if queue A now needs 50%? Will the scheduler take away the 10% >> from queue B and give it back to queue A even if queue B needs it? If not, >> it would seem that the scheduler is reneging on its guarantee. >> >> >> On Wed, Feb 19, 2014 at 6:01 PM, Jian He <[email protected]> wrote: >> >>> Hi Alex >>> >>> You can find good explanation from here: >>> http://hortonworks.com/blog/understanding-apache-hadoops-capacity-scheduler/ >>> Short term: Capacity is the soft limit that queue is guaranteed for such >>> an amount of resource. For the purpose of necessary elasticity, queue can >>> go beyond capacity limit but can not go beyond Max-Capacity limit which is >>> the hard limit. >>> >>> Jian >>> >>> >>> On Wed, Feb 19, 2014 at 11:08 AM, Alex Nastetsky <[email protected] >>> > wrote: >>> >>>> Please help me understand how "capacity" and "maximum-capacity" on a >>>> queue work in the Capacity Scheduler. >>>> >>>> My understanding is that a queue is allocated "capacity" amount of >>>> resources, and if it needs more, it can "stretch" up to "maximum-capacity" >>>> resources. >>>> >>>> But if that's the case, why do we need "capacity" at all? It seems like >>>> "maximum-capacity" is the true limit and "capacity" is ignored? >>>> >>>> Thanks in advance, >>>> Alex. >>>> >>> >>> >>> CONFIDENTIALITY NOTICE >>> NOTICE: This message is intended for the use of the individual or entity >>> to which it is addressed and may contain information that is confidential, >>> privileged and exempt from disclosure under applicable law. If the reader >>> of this message is not the intended recipient, you are hereby notified that >>> any printing, copying, dissemination, distribution, disclosure or >>> forwarding of this communication is strictly prohibited. If you have >>> received this communication in error, please contact the sender immediately >>> and delete it from your system. Thank You. >> >> >> > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
