JingsongLi commented on PR #10928: URL: https://github.com/apache/paimon/pull/10928#issuecomment-6569173790
Requirement fit: SUPPORTED. I did not find an introduced correctness regression in the configuration forwarding. On b221e7e9e9, the 7 common configuration cases, 8 Flink procedure/action cases and 8 Spark procedure cases pass locally with the normal JDK 8 Maven checks. The current upstream CI run is green as well. One nonblocking operational boundary would be useful to document and test: a dedicated procedure argument can make expiration decoupled while ordinary readers remain coupled. I reproduced this with a real primary-key table, input changelogs, four commits, and persisted `snapshot.num-retained.max=4` plus explicit `changelog.num-retained.max=4`. Applying `retain_max=1` through the shared helper expires three snapshots and correctly retains changelog metadata/files; the saved split still reads row 1. However, a freshly loaded ordinary reader restored at snapshot 1 plans no rows, while a control reader with `changelog.num-retained.max=5` reads row 1. `DataTableStreamScan` derives discovery from the reader's options, and `NextSnapshotFetcher` does not consult long-lived changelogs when those options are coupled. This is an existing configuration/discovery limitation exposed by operation-local overrides, not new data loss introduced by this patch: the old helper deleted that history outright. Please clarify that a reader must itself use a decoupled retention configuration in this situation, including dedicated snapshot overrides, just as your description already explains for dynamic options. A regression covering the retained-files/read-configuration distinction would make that boundary explicit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
