JingsongLi commented on code in PR #11182:
URL: https://github.com/apache/paimon/pull/11182#discussion_r4583089037


##########
paimon-core/src/main/java/org/apache/paimon/operation/FileStoreCommitImpl.java:
##########
@@ -757,16 +763,76 @@ public void abort(List<CommitMessage> commitMessages) {
             for (DataFileMeta file : 
commitMessage.compactIncrement().compactAfter()) {
                 file.collectFiles(dataPathFactory).stream()
                         .filter(path -> !compactInputPaths.contains(path))
-                        .forEach(fileIO::deleteQuietly);
+                        .forEach(filesToDelete::add);
             }
 
             List<IndexFileMeta> indexFilesToDelete = new ArrayList<>();
             
indexFilesToDelete.addAll(commitMessage.newFilesIncrement().newIndexFiles());
             
indexFilesToDelete.addAll(commitMessage.compactIncrement().newIndexFiles());
             for (IndexFileMeta file : indexFilesToDelete) {
-                fileIO.deleteQuietly(indexPathFactory.toPath(file));
+                filesToDelete.add(indexPathFactory.toPath(file));
             }
         }
+        deleteConcurrently(filesToDelete);
+    }
+
+    /**
+     * Attempts every delete and waits for all of them even when the caller is 
interrupted. The
+     * first unchecked failure in input order is rethrown as is, later ones 
are suppressed into it.
+     * Do not call this from a {@link FileOperationThreadPool} thread, it may 
wait for itself.
+     */
+    private void deleteConcurrently(List<Path> files) {
+        if (files.isEmpty()) {
+            return;
+        }
+
+        ExecutorService executor =
+                
FileOperationThreadPool.getExecutorService(options.fileOperationThreadNum());
+        boolean interrupted = false;
+        // Run each delete with the caller's context class loader, as the 
sequential abort did.
+        ClassLoader classLoader = 
Thread.currentThread().getContextClassLoader();
+        List<FutureTask<?>> deletes = new ArrayList<>(files.size());
+        for (Path file : files) {
+            FutureTask<?> delete =
+                    new FutureTask<>(
+                            () -> {
+                                Thread thread = Thread.currentThread();
+                                ClassLoader previous = 
thread.getContextClassLoader();
+                                thread.setContextClassLoader(classLoader);
+                                try {
+                                    fileIO.deleteQuietly(file);
+                                } finally {
+                                    thread.setContextClassLoader(previous);
+                                }
+                            },
+                            null);
+            // Clear the flag first, otherwise the bounded executor skips its 
permit and the bound.
+            interrupted |= Thread.interrupted();
+            executor.execute(delete);

Review Comment:
   [P2] Preserve the delete bound when interrupted while acquiring a permit
   
   Clearing the interrupt flag before `execute` does not cover an interrupt 
arriving while `SemaphoredDelegatingExecutor.execute` is blocked in 
`Semaphore.acquire()`. That executor catches the interruption and dispatches 
the command without a permit. A table limit below the shared pool size is a 
normal configuration, so cancellation can bypass the advertised maximum 
concurrent file operations.
   
   I reproduced this through public `abort` using four real owned file paths, 
`file-operation.thread-num=1`, and a shared pool with at least four threads. 
Holding the first delete, waiting until the caller is blocked in semaphore 
acquisition, then interrupting it once produces two simultaneous deletes; the 
identical no-interrupt control produces one. Both finish all deletes and 
preserve the interrupt flag.
   
   Please make bounded submission continue waiting for a permit under 
interruption (while remembering and restoring the flag), or use a fixed number 
of delete workers, and cover interruption during submission. The existing tests 
interrupt before submission or after every delete has already been submitted.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to