Marco Gaido created YARN-8385:
---------------------------------
Summary: Clean local directories when a container is killed
Key: YARN-8385
URL: https://issues.apache.org/jira/browse/YARN-8385
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Marco Gaido
In long running applications, it may happen that many containers are created
and killed. A use case is Spark Thrift Server when dynamic allocation is
enabled. A lot of containers are killed and the application keeps running
indefinitely.
Currently, YARN seems to remove the local directories only when the whole
application terminates. In the scenario described above, this can cause serious
resource leakages. Please, check
https://issues.apache.org/jira/browse/SPARK-22575.
I think YARN should clean up all the local directories of a container when it
is killed and not when the whole application terminates.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
