nchammas commented on a change in pull request #31742:
URL: https://github.com/apache/spark/pull/31742#discussion_r587740705
##########
File path: core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala
##########
@@ -351,6 +351,10 @@ class ContextCleanerSuite extends ContextCleanerSuiteBase {
case _ => false
}, askStorageEndpoints = true).isEmpty)
}
+
+ test("SPARK-33000: shutdown cleans up checkpointed data when configured to
do so") {
+ // TODO: How do I write a test for this?
+ }
Review comment:
I am not sure what would be an appropriate test to add here. I can
directly call the `cleanupOnShutdown()` method, but that doesn't prove that
Spark will do the right thing when it shuts down.
Ideally, I would write some sort of acceptance test that launches a tiny
Spark application and checks the checkpoint directory afterwards. But that
seems very bulky and I don't know if there is an existing pattern in the
codebase that I can follow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
