LsomeYeah opened a new issue, #9308: URL: https://github.com/apache/paimon/issues/9308
### Problem Spark Structured Streaming cannot advance past an initial full snapshot that produces no splits. Every trigger restores the same full snapshot, so later appended snapshots are never reached. ### Reproduction 1. Create a Paimon table, insert one row, then make the current snapshot empty with `INSERT OVERWRITE ... WHERE false`. 2. Start a Spark streaming read with the default `latest-full` mode. 3. Append one row in a new snapshot. 4. The query still reads nothing. `DataTableStreamScan` advances its checkpoint to the next snapshot, but `StreamHelper` creates an offset only from the last returned split. An empty full plan therefore loses the advanced cursor, and the next trigger restores the same full snapshot again. Expected: persist progress for a completed zero-split full snapshot and continue with later delta snapshots. This also occurs without `consumer-id`. A fix must distinguish a truly completed empty snapshot from `ReadMinRows`, which may intentionally defer a non-empty plan. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
