JingsongLi commented on PR #10199: URL: https://github.com/apache/paimon/pull/10199#issuecomment-5950767138
[P2] Refresh Spark's cached table after an ambiguously successful rollback (`RollbackToAsLatestProcedure.java:161–165`) The new failure path preserves the protection tag correctly, but it rethrows before `BaseProcedure.modifyPaimonTable` reaches `refreshSparkCache`. If a post-commit callback throws, the rollback snapshot is already durable while `CACHE TABLE` continues serving the pre-rollback data. I reproduced this through real Spark SQL using the PR's `FailingRollbackCallback`: 1. Write snapshot 1 with `(1, 'original')`, then overwrite with snapshot 2 containing `(2, 'replacement')`. 2. Run `CACHE TABLE T` and read the replacement row. 3. Set `failRollbackCommit = true`, then call `rollback_to_as_latest(..., snapshot_id => 1)`. The call reports the injected post-commit callback error, and snapshot 3 has nevertheless been published. 4. `SELECT * FROM T` still returns `(2, 'replacement')`. After `UNCACHE TABLE T`, the identical query returns `(1, 'original')` from the persisted rollback. The shared helper's success-only refresh predates this PR, but this new procedure introduces the public rollback failure path that reaches it; the issue is in this wrapper's handling of the already-published state. The repository's MSCK partial-failure tests similarly require cached reads to reflect durable changes even when the command reports an error. Please refresh/invalidate cached plans after an entered rollback with an uncertain outcome, including this exception path, and preserve the original rollback exception if refreshing also fails. Add a `CACHE TABLE` regression alongside the existing post-commit callback tests. The successful rollback path already refreshes correctly. Verified on both Spark 3.5 and Spark 4.1: the cached-vs-persisted mismatch reproduces on each. All three original rollback tests pass on both with normal Maven checks; additional real CALL checks for exclusive/missing arguments, tag rollback after snapshot expiration, snapshot-id fallback through a retained tag, and successful cache refresh also pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
