JingsongLi commented on code in PR #11102:
URL: https://github.com/apache/paimon/pull/11102#discussion_r4581664852


##########
paimon-core/src/main/java/org/apache/paimon/operation/FileStoreCommitImpl.java:
##########
@@ -549,7 +762,7 @@ private int overwritePartition(
         }
 
         try {
-            boolean skipOverwrite = false;
+            boolean skipOverwrite = committable instanceof 
CompactRecoveryCommittable;

Review Comment:
   [P1] Apply stale-compaction handling to OVERWRITE recovery too
   
   This marker enables overwrite recovery, but its later COMPACT call invokes 
`tryCommit` without setting `committingCompactRecovery` or catching 
`StaleCompactRecoveryException` as `commit()` now does. The liveness prechecks 
do not catch every LSM conflict: a concurrent writer can publish an overlapping 
level-1 file while the recovered compaction's original level-0 input is still 
live.
   
   I reproduced this by publishing only A's OVERWRITE phase, then having 
another user APPEND and compact a distinct same-key file to level 1. A's 
pending output is also level 1. `filterCommitted` returns A's recovery on each 
restart, and two fresh `overwritePartition` recovery attempts both throw `LSM 
conflicts detected! Give up committing.` The equivalent APPEND recovery control 
returns zero snapshots through the new stale-recovery handler. 
`TableCommitImpl` with overwrite configured dispatches the marker directly to 
this affected path.
   
   Please share the recovery flag/catch handling between both entry paths and 
add a deterministic OVERWRITE overlap regression. Otherwise the new recovery 
route can still leave a batch overwrite job in a persistent restart loop.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to