gu-chi created YARN-3730:

             Summary: scheduler reserve more resource than required
                 Key: YARN-3730
             Project: Hadoop YARN
          Issue Type: Bug
          Components: scheduler
            Reporter: gu-chi

Using capacity scheduler, environment is 3 NM and each has 9 vcores, I ran a 
spark task with 4 executors and each executor 5 cores, as suspected, only 1 
executor not able to start and will be reserved, but actually more containers 
are reserved. This way, I can not run some other smaller tasks. As I checked 
the capacity scheduler, the 'needContainers' method in has a 
computation of 'starvation', this cause the scenario of more container reserved 
than required, any idea or suggestion on this?

This message was sent by Atlassian JIRA

Reply via email to