Zihan Li created GOBBLIN-1823:
---------------------------------
Summary: Improving Container Calculation and Allocation Methodology
Key: GOBBLIN-1823
URL: https://issues.apache.org/jira/browse/GOBBLIN-1823
Project: Apache Gobblin
Issue Type: Improvement
Reporter: Zihan Li
When Yarn allocates "ghost containers" without calling the
onContainerAllocated() method and when the container is eventually released,
onContainersCompleted() is called, container numbers mismatches can occur.
In the onContainerAllocated() method, we add the container to the containerMap
using the container ID as the key, and increase the count for the specific tag.
In the onContainersCompleted() method, we remove the container from the
containerMap and decrease the count. However, in some cases, we find that the
containerMap does not contain the ID, and we ignore this while still decreasing
the number of the allocated tag. We do this because sometimes
onContainersCompleted() is called before onContainerAllocated() for the same
container.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
