[
https://issues.apache.org/jira/browse/SPARK-18968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-18968.
-------------------------------
Resolution: Duplicate
Without any more detail, this is a duplicate
> .sparkStaging quickly fill up HDFS
> ----------------------------------
>
> Key: SPARK-18968
> URL: https://issues.apache.org/jira/browse/SPARK-18968
> Project: Spark
> Issue Type: Bug
> Components: DStreams
> Affects Versions: 1.6.2
> Reporter: Chen He
>
> We are running streaming jobs using spark. Even the
> "spark.yarn.preserve.staging.files" is set to b "false", HDFS is quickly been
> filled up.
> Also find people ask mail-list similar question but no further response.
> http://apache-spark-user-list.1001560.n3.nabble.com/HDFS-folder-sparkStaging-not-deleted-and-filled-up-HDFS-in-yarn-mode-td7851.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
