wangyong9999 opened a new issue, #299: URL: https://github.com/apache/paimon-cpp/issues/299
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon-cpp/issues) and found nothing similar. ### Paimon-cpp version main (ded7cfb8) ### Minimal reproduce step 1. Create a primary key table with one bucket and commit three times (snapshots 1 to 3), for example `(k1, v1), (k2, v2)`, then `(k3, v3)`, then `(k1, v1b)`. 2. Scan it with a `Cache` set on the `ScanContext` and `scan.manifest-entry-cache.max-snapshots = 2`, so the live manifest entries of snapshot 3 are cached for bucket 0. 3. Roll the table back and commit twice more, for example from Flink: ```sql CALL sys.rollback_to(`table` => 'default.t', snapshot_id => 1); INSERT INTO t VALUES ('k4', 'v4'); INSERT INTO t VALUES ('k1', 'v1c'); ``` The rollback deletes `snapshot-2`, `snapshot-3` and the data files only they referenced; the two inserts create a new `snapshot-2` and `snapshot-3`. 4. Scan again through the same `Cache`. ### What doesn't meet your expectations? The second scan plans from the cached entries of the deleted snapshot 3. Reading `k1` or `k3` fails with `Not exist: File .../bucket-0/data-....parquet not exists`, `k4` returns no row although it was committed, and only `k2` is correct. The table recovers once a fourth snapshot is committed and the cached ids fall behind. `FileStoreScan::ReadManifestEntriesWithCache` treats `cached->snapshot_id == snapshot.Id()` as a hit (`file_store_scan.cpp:343`). Nothing else identifies the snapshot, so a snapshot id that is reused after a rollback (or after dropping and recreating a table at the same path) matches entries that belong to a different snapshot. With the default `max-snapshots = 0` the cache is off; any deployment that enables it for point lookups is exposed. ### Anything else? Storing the snapshot's delta manifest list name next to the cached entries and requiring it to match on a hit is enough: the name carries a UUID, so a rewritten snapshot never matches, and a mismatch falls through to the existing rebuild path. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
