MENG DING commented on YARN-4165:

Hi, [~leftnoteasy]

Please correct me if I am wrong. According to the current logic, in 
{{shouldAllocOrReserveNewContainer}}, the starvation number will grow with 
re-reservations. Based on the use case in this ticket, I think the 
re-reservations will increase for the first reserved container, and to the 
point that the scheduler will try to reserve on another host. Isn't it?

starvation =
          (int) ((application.getReReservations(priority) / 
              (float) reservedContainers) * (1.0f - (Math.min(
                  nodeFactor, application.getCSLeafQueue()

> An outstanding container request makes all nodes to be reserved causing all 
> jobs pending
> ----------------------------------------------------------------------------------------
>                 Key: YARN-4165
>                 URL: https://issues.apache.org/jira/browse/YARN-4165
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacity scheduler, resourcemanager, scheduler
>    Affects Versions: 2.7.1
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
> We have a long running service in YARN, it has a outstanding container 
> request that YARN cannot satisfy (require more memory that nodemanager can 
> supply). Then YARN reserves all nodes for this application, when I submit 
> other jobs (require relative small memory that nodemanager can supply), all 
> jobs are pending because YARN skips scheduling containers on the nodes that 
> have been reserved.

This message was sent by Atlassian JIRA

Reply via email to