Jian He commented on YARN-1707:

Hi Carlo, thanks for your work !  I looked at the patch, some comments and 
- to simplify, we can use getNumApplications() method
disposableLeafQueue.getApplications().size() > 0
        || disposableLeafQueue.pendingApplications.size() > 0
- PlanQueue.java 80 column limit
- why “newQueue.changeCapacity(sesConf.getCapacity());” is inside the check and 
“queue.setMaxCapacity(sesConf.getMaxCapacity());” is outside the check
- CapacityScheduler#getReservationQueueNames seems getting the child 
reservation queues of the given plan queue. We can use the 
planQueue#childQueues directly
- DynamicQueueConf, how about calling it QueueEntitlement to be consistent ?
- CapacityScheduler#parseQueue method, I think we can simplify the condition 
for isReservableQueue flag something like this:
    boolean isReservableQueue = conf.isReservableQueue(fullQueueName);
    if (isReservableQueue) {
      ParentQueue parentQueue = 
          new PlanQueue(csContext, queueName, parent,
      queue = hook.hook(parentQueue);
    } else if ((childQueueNames == null || childQueueNames.length == 0))
- just to simplify, this log msg may be put after previous “qiter.remove();” to 
avoid the removed boolean flag.
    if (LOG.isDebugEnabled()) {
      LOG.debug("updateChildQueues (action: remove queue): " + removed + " "
          + getChildQueuesToPrint());
- we can add a new reinitialize in ReservationQueue which does all these 
          schedulerContext.getResourceCalculator(), ses, this,
      ses.reinitialize(ses, clusterResource);
      ((ReservationQueue) ses).setMaxApplications(this
      ((ReservationQueue) ses).setMaxApplicationsPerUser(this
- IIUC, right now, queueName here is for the planQueue(inherits parentQueue), 
and the reservationID is for the reservationQueue(inherits from leafQueue). I 
think if we can get the proper reservationQueueName(leafQueue) upfront and pass 
it as the queueName parameter into this method, we can avoid some if/else 
condition changes inside this method and the method signature. 
  private synchronized void addApplication(ApplicationId applicationId,
    String queueName, String user, boolean isAppRecovering,
    ReservationId reservationID)

> Making the CapacityScheduler more dynamic
> -----------------------------------------
>                 Key: YARN-1707
>                 URL: https://issues.apache.org/jira/browse/YARN-1707
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Carlo Curino
>            Assignee: Carlo Curino
>              Labels: capacity-scheduler
>         Attachments: YARN-1707.2.patch, YARN-1707.3.patch, YARN-1707.4.patch, 
> YARN-1707.patch
> The CapacityScheduler is a rather static at the moment, and refreshqueue 
> provides a rather heavy-handed way to reconfigure it. Moving towards 
> long-running services (tracked in YARN-896) and to enable more advanced 
> admission control and resource parcelling we need to make the 
> CapacityScheduler more dynamic. This is instrumental to the umbrella jira 
> YARN-1051.
> Concretely this require the following changes:
> * create queues dynamically
> * destroy queues dynamically
> * dynamically change queue parameters (e.g., capacity) 
> * modify refreshqueue validation to enforce sum(child.getCapacity())<= 100% 
> instead of ==100%
> We limit this to LeafQueues. 

This message was sent by Atlassian JIRA

Reply via email to