JingsongLi commented on code in PR #10853:
URL: https://github.com/apache/paimon/pull/10853#discussion_r4479161963
##########
paimon-python/pypaimon/execution/python/write/file_store_commit.py:
##########
@@ -674,6 +675,10 @@ def _try_commit_once(self, retry_result:
Optional[RetryResult], commit_kind: str
None,
base_data_files=base_data_files,
)
+ except FileSyncError:
+ # A visible snapshot does not settle a known fsync failure.
Preserve
+ # its files, but do not retry and acknowledge it as a duplicate.
+ raise
Review Comment:
[P1] Preserve checkpoint identity when recovering a visible publication. On
a local append table I used
`new_stream_write_builder().with_commit_user('recovering-job')`, prepared one
row for checkpoint 1, and injected `EIO` only on the snapshot directory sync
after rename. `commit(messages, 1)` raised `FileSyncError` with snapshot 1
visible. After restoring fsync, replaying the same failed checkpoint published
snapshot 2 and a full scan returned the row twice. A fresh call starts with no
`CommitFailRetryResult`, so the duplicate-check path never runs. Please carry
an explicit published-but-durability-uncertain outcome into recovery and cover
replay after this failure. The checkpoint must not be published again or
treated as durable solely because its snapshot is readable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
