Bilwa S T created YARN-9233:
-------------------------------
Summary: RM may report allocated container which is killed (but
not acquired by AM ) to AM which can cause spark AM confused
Key: YARN-9233
URL: https://issues.apache.org/jira/browse/YARN-9233
Project: Hadoop YARN
Issue Type: Bug
Reporter: Bilwa S T
After the RM kills an allocated (Allocated state) container for various
reasons, it will go through the state transition process to the FINISHED state
just like other state containers. Currently RM doesn't consider if container is
acquired by the AM. Hence All the containers transitioned to FINISH state are
added to justFinishedContainers list. Therefore the container that is not
obtained by the AM and is killed by the rm will also return through the AM
heartbeat. So AM re-applies for more resources than needed which would
eventually cause number of containers to exceed the maximum limit
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
