Hi, Why does YARN block an entire resource once a container has been allocated in the node? A job might use different quantities of resources at different points of time and allocating a pre-determined set of resources might be counter-productive. For example, if the resources requested by the job are 1VCORE and 4GB of RAM, the entire 4GB is blocked (other Jobs/processes can not access this memory) irrespective of whether or not the job uses the complete 4GB at a given point of time.
This might be a problem with I/O bound tasks wherein the CPU utilization is close to 1% but 1VCORE is used nevertheless since the hdfs-site.xml dictates a minimum VCORE of 1. This VCORE could have been given to a CPU intensive job wherein the 1VCORE allocation actually makes sense. The YARN-1197 <https://issues.apache.org/jira/browse/YARN-1197> Jira shows support for dynamic containers but I believe the container sizes are not increased/decreased fractionally. Thanks, Muthu
