JingsongLi commented on code in PR #11102:
URL: https://github.com/apache/paimon/pull/11102#discussion_r4581664852
##########
paimon-core/src/main/java/org/apache/paimon/operation/FileStoreCommitImpl.java:
##########
@@ -549,7 +762,7 @@ private int overwritePartition(
}
try {
- boolean skipOverwrite = false;
+ boolean skipOverwrite = committable instanceof
CompactRecoveryCommittable;
Review Comment:
[P1] Apply stale-compaction handling to OVERWRITE recovery too
This marker enables overwrite recovery, but its later COMPACT call invokes
`tryCommit` without setting `committingCompactRecovery` or catching
`StaleCompactRecoveryException` as `commit()` now does. The liveness prechecks
do not catch every LSM conflict: a concurrent writer can publish an overlapping
level-1 file while the recovered compaction's original level-0 input is still
live.
I reproduced this by publishing only A's OVERWRITE phase, then having
another user APPEND and compact a distinct same-key file to level 1. A's
pending output is also level 1. `filterCommitted` returns A's recovery on each
restart, and two fresh `overwritePartition` recovery attempts both throw `LSM
conflicts detected! Give up committing.` The equivalent APPEND recovery control
returns zero snapshots through the new stale-recovery handler.
`TableCommitImpl` with overwrite configured dispatches the marker directly to
this affected path.
Please share the recovery flag/catch handling between both entry paths and
add a deterministic OVERWRITE overlap regression. Otherwise the new recovery
route can still leave a batch overwrite job in a persistent restart loop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]