Prabhu Joseph created SPARK-51678:
-------------------------------------
Summary: Cleanup Spark RDD Checkpoint directory at job completion
Key: SPARK-51678
URL: https://issues.apache.org/jira/browse/SPARK-51678
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 4.1.0
Reporter: Prabhu Joseph
Spark RDD checkpointing is only used within the same application that performed
the checkpoint and not outside of it. Currently, the checkpointed RDD is not
cleaned up, leading to unnecessary storage usage in HDFS and Cloud storage.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
