Wangda Tan created YARN-3983: -------------------------------- Summary: Make CapacityScheduler to easier extend application allocation logic Key: YARN-3983 URL: https://issues.apache.org/jira/browse/YARN-3983 Project: Hadoop YARN Issue Type: Bug Reporter: Wangda Tan Assignee: Wangda Tan
While working on YARN-1651 (resource allocation for increasing container), I found it is very hard to extend existing CapacityScheduler resource allocation logic to support different types of resource allocation. For example, there's a lot of differences between increasing a container and allocating a container: - Increasing a container doesn't need to check locality delay. - Increasing a container doesn't need to build/modify a resource request tree (ANY->RACK/HOST). - Increasing a container doesn't need to check allocation/reservation starvation (see {{shouldAllocOrReserveNewContainer}}). - After increasing a container is approved by scheduler, it need to update an existing container token instead of creating new container. And there're lots of similarities when allocating different types of resources. - User-limit/queue-limit will be enforced for both of them. - Both of them needs resource reservation logic. (Maybe continuous reservation looking is needed for both of them). The purpose of this JIRA is to make easier extending CapacityScheduler resource allocation logic to support different types of resource allocation, make common code reusable, and also better code organization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)