JingsongLi commented on code in PR #11040:
URL: https://github.com/apache/paimon/pull/11040#discussion_r4522781443


##########
paimon-core/src/main/java/org/apache/paimon/table/source/AbstractDataTableScan.java:
##########
@@ -539,7 +543,7 @@ protected StartingScanner createStartingScanner(boolean 
isStreaming) {
                                 snapshotManager,
                                 changelogManager,
                                 startupMillis,
-                                options.changelogLifecycleDecoupled())
+                                readRetainedChangelog())

Review Comment:
   **[P1] Resume a valid checkpoint without resolving the original timestamp**
   
   I reproduced this with a real PK `INPUT` changelog table on `2e687236b`. 
After two expirations, changelogs 1–3 and snapshots 5–8 were readable, and ID 4 
had expired. With `scan.timestamp-millis` set to snapshot 5's timestamp, I ran 
`newStreamScan().restore(7L)` and then `plan()`. The scan failed with `Cannot 
determine the starting snapshot ... missing for IDs 4 to 4`. The unchanged 
baseline plans snapshot 7 and advances to 8.
   
   `DataTableStreamScan.planWithoutAuth` initializes the timestamp starter 
before using the restored `nextSnapshotId`, so a production job can no longer 
restart even though its saved position is readable. Could we resolve timestamp 
startup only for a fresh scan, and add a restart regression with the checkpoint 
after the gap? Spark's AvailableNow also eagerly calls `initOffset`; please 
cover that restart entry point as well.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to