LuciferYang opened a new pull request #33628:
URL: https://github.com/apache/spark/pull/33628
### What changes were proposed in this pull request?
We always do file truncate operation before delete a write failed file held
by `DiskBlockObjectWriter`, a typical process is as follows:
```
if (!success) {
// This code path only happens if an exception was thrown above before we
set success;
// close our stuff and let the exception be thrown further
writer.revertPartialWritesAndClose()
if (file.exists()) {
if (!file.delete()) {
logWarning(s"Error deleting ${file}")
}
}
}
```
The `revertPartialWritesAndClose` method will reverts writes that haven't
been committed yet, but it doesn't seem necessary in the current scene.
So this pr add a new method to `DiskBlockObjectWriter` named
`deleteHeldFile()`, the new method just revert write metrics and delete the
write failed file.
### Why are the changes needed?
Avoid unnecessary file operations.
### Does this PR introduce _any_ user-facing change?
Add a new method to `DiskBlockObjectWriter` named `deleteHeldFile().
### How was this patch tested?
Pass the Jenkins or GitHub Action
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
