JingsongLi commented on code in PR #11040:
URL: https://github.com/apache/paimon/pull/11040#discussion_r4522781443
##########
paimon-core/src/main/java/org/apache/paimon/table/source/AbstractDataTableScan.java:
##########
@@ -539,7 +543,7 @@ protected StartingScanner createStartingScanner(boolean
isStreaming) {
snapshotManager,
changelogManager,
startupMillis,
- options.changelogLifecycleDecoupled())
+ readRetainedChangelog())
Review Comment:
**[P1] Resume a valid checkpoint without resolving the original timestamp**
I reproduced this with a real PK `INPUT` changelog table on `2e687236b`.
After two expirations, changelogs 1–3 and snapshots 5–8 were readable, and ID 4
had expired. With `scan.timestamp-millis` set to snapshot 5's timestamp, I ran
`newStreamScan().restore(7L)` and then `plan()`. The scan failed with `Cannot
determine the starting snapshot ... missing for IDs 4 to 4`. The unchanged
baseline plans snapshot 7 and advances to 8.
`DataTableStreamScan.planWithoutAuth` initializes the timestamp starter
before using the restored `nextSnapshotId`, so a production job can no longer
restart even though its saved position is readable. Could we resolve timestamp
startup only for a fresh scan, and add a restart regression with the checkpoint
after the gap? Spark's AvailableNow also eagerly calls `initOffset`; please
cover that restart entry point as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]