lilei1128 commented on code in PR #9535:
URL: https://github.com/apache/paimon/pull/9535#discussion_r3910087679
##########
paimon-core/src/main/java/org/apache/paimon/table/source/DataTableStreamScan.java:
##########
@@ -315,6 +315,19 @@ public Long watermark() {
@Override
public void restore(@Nullable Long nextSnapshotId) {
Review Comment:
Similar possible situations need to be taken into consideration, for example:
A previous asynchronous compaction may expire and delete the snapshot files
that the source is about to read.
Possible sequence:
The source selects snapshot N for reading
-> a previous asynchronous compaction commits a newer snapshot
-> snapshot N or its manifest files are deleted
-> the source continues reading snapshot N
-> the read fails with OutOfRangeException or FileNotFoundException
This is a read-time race and can happen even without a Flink failover.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
