srowen commented on a change in pull request #33251:
URL: https://github.com/apache/spark/pull/33251#discussion_r667131611
##########
File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
##########
@@ -2010,10 +2012,12 @@ private[spark] object BlockManager {
val filePath = file.path()
+ val deleter: DownloadFileDeleter = file.deleter()
+
def cleanUp(): Unit = {
logDebug(s"Clean up file $filePath")
- if (!file.delete()) {
+ if (!deleter.delete()) {
Review comment:
I'm probably missing something obvious but I'm not clear why the
interface changes when things get deleted. It seems like it happens in the same
place everywhere?
##########
File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
##########
@@ -255,7 +255,9 @@ private[spark] class BlockManager(
// specified memory threshold. Files will be deleted automatically based on
weak reference.
// Exposed for test
private[storage] val remoteBlockTempFileManager =
- new BlockManager.RemoteBlockDownloadFileManager(this)
+ new BlockManager.RemoteBlockDownloadFileManager(
Review comment:
I wasn't clear why this had to change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
