JingsongLi commented on code in PR #10625:
URL: https://github.com/apache/paimon/pull/10625#discussion_r4374819956


##########
paimon-core/src/main/java/org/apache/paimon/utils/TagManager.java:
##########
@@ -132,11 +172,204 @@ public void createTag(
             List<TagCallback> callbacks,
             boolean ignoreIfExists) {
         checkArgument(!StringUtils.isNullOrWhitespaceOnly(tagName), "Tag name 
shouldn't be blank.");
-        if (tagExists(tagName)) {
-            checkArgument(ignoreIfExists, "Tag '%s' already exists.", tagName);
+        // The existence check and metadata write must not race with snapshot 
expiration, which
+        // deletes files based on a tag set collected under the same lock.
+        ReentrantLock coordinationLock = coordinationLock();
+        coordinationLock.lock();
+        try {
+            if (tagExists(tagName)) {
+                checkArgument(ignoreIfExists, "Tag '%s' already exists.", 
tagName);
+                if (awaitSettledTag(tagName)) {
+                    return;
+                }
+            }
+            Runnable beforeCommit = beforeTagCommit;
+            if (beforeCommit != null) {
+                beforeCommit.run();
+            }
+            ensureSnapshotRetained(snapshot, tagName, false);
+            String owner = UUID.randomUUID().toString();
+            createPendingTagMarker(tagName, owner);
+            try {
+                createOrReplaceTag(snapshot, tagName, timeRetained, callbacks, 
true);
+            } finally {
+                fileIO.deleteQuietly(pendingTagMarkerPath(tagName, owner));

Review Comment:
   [P1] Keep the pending fence until publication is validated or rollback is 
confirmed
   
   This finally clears the pending marker even when the tag remains 
unvalidated. `overwriteFileUtf8` can persist the complete tag and then throw on 
close; its existing catch leaves that file because the outcome is unknown, so 
post-write retention validation never runs. Alternatively, validation can fail 
while `deleteQuietly(tagPath)` fails to remove the file. In both cases this 
finally removes the fence, and another JVM's `createTag(..., 
ignoreIfExists=true)` sees the tag with no pending marker and returns success.
   
   I reproduced both cases while actual expiration was paused after its final 
tag scan: the retry exited successfully, then expiration completed with the tag 
still present but its snapshot and delta-manifest list deleted. The first case 
throws only after a real successful tag write; the second returns false only 
for the tag's deletion. Keep the pending state until publication has passed 
validation or rollback is confirmed. An unresolved write/delete outcome should 
remain fenced (as the documented crashed-creator case does), or be resolved 
safely before acknowledging retries.



##########
paimon-core/src/main/java/org/apache/paimon/utils/TagManager.java:
##########
@@ -181,6 +416,15 @@ private void createOrReplaceTag(
                     e);
         }
 
+        if (rollbackIfSnapshotExpired) {
+            try {
+                ensureSnapshotRetained(snapshot, tagName, true);
+            } catch (RuntimeException e) {
+                fileIO.deleteQuietly(tagPath);

Review Comment:
   [P1] Roll back only the publication owned by this creator
   
   Two creator JVMs can both pass the initial absence check, so the per-name 
pending markers do not make this deletion owned. With expiration paused after 
its final scan, let B pass the absence check and wait; A publishes `same` for 
fenced snapshot 2 and pauses after the write. B then publishes `same` for 
retained snapshot 4, passes validation and returns success. When A resumes, its 
snapshot-2 validation fails and this deletion removes B's already acknowledged 
snapshot-4 tag.
   
   An actual-expirer/two-creator JDK 8 reproduction confirms the tag points to 
snapshot 4 after B succeeds, then disappears when A rolls back, although 
snapshot 4 remains valid. Removing only this new deletion in a scratch control 
preserves B's tag under the identical schedule, so this is additional damage 
beyond the baseline's overwrite race. Please give same-name publication 
cross-process ownership and make rollback conditional on that ownership; a 
separate read-before-delete check still races with another publication. Add the 
overlapping-creator case alongside the existing waiter test.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to