LuciferYang commented on a change in pull request #33267:
URL: https://github.com/apache/spark/pull/33267#discussion_r666631560
##########
File path:
core/src/main/scala/org/apache/spark/storage/DiskBlockObjectWriter.scala
##########
@@ -288,3 +288,19 @@ private[spark] class DiskBlockObjectWriter(
bs.flush()
}
}
+
+private[spark] object DiskBlockObjectHelper extends Logging {
+
+ /**
+ * Reverts writes that haven't been committed yet and delete the object file
hold by the
+ * DiskBlockObjectWriter.
+ */
+ def deleteAbnormalDiskBlockObjectFile(writer: DiskBlockObjectWriter): Unit =
{
Review comment:
@Ngone51 That's a good question, but I think it's a little strange for
an `OutputStream( DiskBlockObjectWriter extends OutputStream)` to delete
itself.
For example, `o.a.h.fs.FileSystem` can `create` a `FSDataOutputStream`,
although the `FSDataOutputStream` has the capabilities of `write data` and
`close itself`, but we still need to use the `o.a.h.fs.FileSystem` to delete
the file.
Of course, The `java.io.File` API has the ability to delete itself, the
`java.io.File` can `createNewFile` and the `File` instance can `delete` itself,
but it still needs to be wrap as `FileOutputStream` to write data, and
`FileOutputStream` cannot delete itself too.
At present, I choose the former because `DiskBlockObjectWriter` is a
`OutputStream` rather than a `File` , but I also accept it as a member
function of `DiskBlockObjectWriter`.
Do you think `keep it as a member function of DiskBlockObjectWriter` is
better?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
