This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push:
new 9770016b180b [SPARK-47236][CORE] Fix `deleteRecursivelyUsingJavaIO` to
skip non-existing file input
9770016b180b is described below
commit 9770016b180b0477060777d3739a2bfaabc6fcb3
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Thu Feb 29 19:08:15 2024 -0800
[SPARK-47236][CORE] Fix `deleteRecursivelyUsingJavaIO` to skip non-existing
file input
### What changes were proposed in this pull request?
This PR aims to fix `deleteRecursivelyUsingJavaIO` to skip non-existing
file input.
### Why are the changes needed?
`deleteRecursivelyUsingJavaIO` is a fallback of
`deleteRecursivelyUsingUnixNative`.
We should have identical capability. Currently, it fails.
```
[info] java.nio.file.NoSuchFileException:
/Users/dongjoon/APACHE/spark-merge/target/tmp/spark-e264d853-42c0-44a2-9a30-22049522b04f
[info] at
java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
[info] at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
[info] at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
[info] at
java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
[info] at
java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:148)
[info] at java.base/java.nio.file.Files.readAttributes(Files.java:1851)
[info] at
org.apache.spark.network.util.JavaUtils.deleteRecursivelyUsingJavaIO(JavaUtils.java:126)
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
This is difficult to test this `private static` Java method. I tested this
with #45344 .
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #45346 from dongjoon-hyun/SPARK-47236.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
(cherry picked from commit 1cd7bab5c5c2bd8d595b131c88e6576486dbf123)
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
common/utils/src/main/java/org/apache/spark/network/util/JavaUtils.java | 1 +
1 file changed, 1 insertion(+)
diff --git
a/common/utils/src/main/java/org/apache/spark/network/util/JavaUtils.java
b/common/utils/src/main/java/org/apache/spark/network/util/JavaUtils.java
index bbe764b8366c..d6603dcbee1a 100644
--- a/common/utils/src/main/java/org/apache/spark/network/util/JavaUtils.java
+++ b/common/utils/src/main/java/org/apache/spark/network/util/JavaUtils.java
@@ -120,6 +120,7 @@ public class JavaUtils {
private static void deleteRecursivelyUsingJavaIO(
File file,
FilenameFilter filter) throws IOException {
+ if (!file.exists()) return;
BasicFileAttributes fileAttributes =
Files.readAttributes(file.toPath(), BasicFileAttributes.class);
if (fileAttributes.isDirectory() && !isSymlink(file)) {
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
