lilei1128 commented on code in PR #9535:
URL: https://github.com/apache/paimon/pull/9535#discussion_r3910184680
##########
paimon-core/src/main/java/org/apache/paimon/table/source/DataTableStreamScan.java:
##########
@@ -315,6 +315,19 @@ public Long watermark() {
@Override
public void restore(@Nullable Long nextSnapshotId) {
Review Comment:
I think we can treat it as a source-and-sink recovery problem, follows:
detect expired snapshot(include `normal read` or `checkpoint restore`)
-> discard expired pending splits
-> build a baseline from the latest valid snapshot
-> notify all compaction writers
-> rebuild writer state
-> ignore in-flight splits covered by the new baseline
-> continue from the next snapshot
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
