Bilwa S T created YARN-9233: ------------------------------- Summary: RM may report allocated container which is killed (but not acquired by AM ) to AM which can cause spark AM confused Key: YARN-9233 URL: https://issues.apache.org/jira/browse/YARN-9233 Project: Hadoop YARN Issue Type: Bug Reporter: Bilwa S T
After the RM kills an allocated (Allocated state) container for various reasons, it will go through the state transition process to the FINISHED state just like other state containers. Currently RM doesn't consider if container is acquired by the AM. Hence All the containers transitioned to FINISH state are added to justFinishedContainers list. Therefore the container that is not obtained by the AM and is killed by the rm will also return through the AM heartbeat. So AM re-applies for more resources than needed which would eventually cause number of containers to exceed the maximum limit -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org