[
https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004747#comment-14004747
]
Mingyu Kim edited comment on SPARK-1860 at 5/21/14 3:10 PM:
------------------------------------------------------------
[~aash], is there a reliable way to check "folder is owned by a running
application"? I thought that's not possible, so I was just going to have the
second if statement, which means folder for running applications that just
haven't been active for TTS will also get wiped out, assuming that executor is
writing out something to either stdout or stderr when it runs some computation.
This also means that if you have a long-running inactive application, the
application should send a "heartbeat" by running a trivial computation once
every while.
Any suggestions?
was (Author: mkim):
[~aash], is there a reliable way to check "folder is owned by a running
application". I thought that's not possible, so I was just going to have the
second if statement, which means folder for running applications that just
haven't been active for TTS will also get wiped out, assuming that executor is
writing out something to either stdout or stderr when it runs some computation.
This also means that if you have a long-running inactive application, the
application should send a "heartbeat" by running a trivial computation once
every while.
Any suggestions?
> Standalone Worker cleanup should not clean up running applications
> ------------------------------------------------------------------
>
> Key: SPARK-1860
> URL: https://issues.apache.org/jira/browse/SPARK-1860
> Project: Spark
> Issue Type: Bug
> Components: Deploy
> Affects Versions: 1.0.0
> Reporter: Aaron Davidson
> Priority: Critical
> Fix For: 1.1.0
>
>
> The default values of the standalone worker cleanup code cleanup all
> application data every 7 days. This includes jars that were added to any
> applications that happen to be running for longer than 7 days, hitting
> streaming jobs especially hard.
> Applications should not be cleaned up if they're still running. Until then,
> this behavior should not be enabled by default.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
