(celeborn) branch main updated: [CELEBORN-1449] Fix JavaUtils#deleteRecursivelyUsingJavaIO to skip non-existing file input

[csy]

This is an automated email from the ASF dual-hosted git repository.

csy pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git


The following commit(s) were added to refs/heads/main by this push:
     new 9e5f7e5a4 [CELEBORN-1449] Fix JavaUtils#deleteRecursivelyUsingJavaIO 
to skip non-existing file input
9e5f7e5a4 is described below

commit 9e5f7e5a4da70d77a6b5f7a5745718a678b56d8f
Author: SteNicholas <programg...@163.com>
AuthorDate: Wed Jun 5 14:04:36 2024 +0800

    [CELEBORN-1449] Fix JavaUtils#deleteRecursivelyUsingJavaIO to skip 
non-existing file input
    
    ### What changes were proposed in this pull request?
    
    Fix `JavaUtils#deleteRecursivelyUsingJavaIO` to skip non-existing file 
input. Meanwhile, reduce multiple file attribute calls of 
`JavaUtils#deleteRecursivelyUsingJavaIO`.
    
    ### Why are the changes needed?
    
    `deleteRecursivelyUsingJavaIO` is a fallback of 
`deleteRecursivelyUsingUnixNative` in `JavaUtils`. We should have identical 
capability for `JavaUtils#deleteRecursivelyUsingJavaIO` which should skip 
non-existing file input. Meanwhile, `JavaUtils#deleteRecursivelyUsingJavaIO` 
method performs multiple file attribute calls.
    
    Backport:
    
    - https://github.com/apache/spark/pull/36636
    - https://github.com/apache/spark/pull/45346
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    GA.
    
    Closes #2543 from SteNicholas/CELEBORN-1449.
    
    Authored-by: SteNicholas <programg...@163.com>
    Signed-off-by: Shaoyun Chen <c...@apache.org>
---
 .../main/java/org/apache/celeborn/common/util/JavaUtils.java   | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git 
a/common/src/main/java/org/apache/celeborn/common/util/JavaUtils.java 
b/common/src/main/java/org/apache/celeborn/common/util/JavaUtils.java
index d4ce0bf2e..258a3fd40 100644
--- a/common/src/main/java/org/apache/celeborn/common/util/JavaUtils.java
+++ b/common/src/main/java/org/apache/celeborn/common/util/JavaUtils.java
@@ -22,6 +22,8 @@ import java.net.InetAddress;
 import java.nio.ByteBuffer;
 import java.nio.channels.ReadableByteChannel;
 import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.attribute.BasicFileAttributes;
 import java.util.Locale;
 import java.util.Map;
 import java.util.Random;
@@ -132,7 +134,10 @@ public class JavaUtils {
 
   private static void deleteRecursivelyUsingJavaIO(File file, FilenameFilter 
filter)
       throws IOException {
-    if (file.isDirectory() && !isSymlink(file)) {
+    if (!file.exists()) return;
+    BasicFileAttributes fileAttributes =
+        Files.readAttributes(file.toPath(), BasicFileAttributes.class);
+    if (fileAttributes.isDirectory() && !isSymlink(file)) {
       IOException savedIOException = null;
       for (File child : listFilesSafely(file, filter)) {
         try {
@@ -148,7 +153,8 @@ public class JavaUtils {
     }
 
     // Delete file only when it's a normal file or an empty directory.
-    if (file.isFile() || (file.isDirectory() && listFilesSafely(file, 
null).length == 0)) {
+    if (fileAttributes.isRegularFile()
+        || (fileAttributes.isDirectory() && listFilesSafely(file, null).length 
== 0)) {
       boolean deleted = file.delete();
       // Delete can also fail if the file simply did not exist.
       if (!deleted && file.exists()) {