gengliangwang opened a new pull request, #59150: URL: https://github.com/apache/spark/pull/59150
### What changes were proposed in this pull request? Backport #59119 to `branch-4.3` by cherry-picking `a32c06b2cfe0275b5e6643810dbff8c91b74e5c6`. Add `TableCatalog.changelogStateOptionKeys()`, filter catalog-level changelog options, and reuse changelog metadata for reads with the same context and state options. Preserve bounded CDC reads during refresh and temporary-view access, and reload open-ended reads through the changelog API. Backport adjustments: - Resolve the `views.scala` import conflict by adding `ChangelogTable` and retaining only `StaticSQLConf`; branch-4.3 has no corresponding `SQLConf` usage. - Mark the new API `@since 4.3.1`, matching the target branch's `4.3.1-SNAPSHOT` version. ### Why are the changes needed? Bring the [SPARK-59841](https://issues.apache.org/jira/browse/SPARK-59841) fixes to branch-4.3. Reads that differ only in scan options should reuse changelog metadata, while different ranges or state options must remain separate. Bounded CDC reads must retain their original range during refresh and temporary-view access. ### Does this PR introduce _any_ user-facing change? Yes, the same behavior changes as #59119. `loadChangelog` receives only options declared by `changelogStateOptionKeys()`, which defaults to `tableStateOptionKeys()`. Connectors must declare options required for metadata loading; scan-only options remain available to scan builders. ### How was this patch tested? - Compared the added and removed lines in all changed files against the original commit. The only difference is the branch-specific `@since` annotation. - Passed `git diff --check`, added-source ASCII and line-length checks, and error-conditions JSON parsing. - Included the original regression tests. Local build and test suites were not rerun for this backport. The original PR reports 323 passing tests plus focused reruns; see #59119 for its validation details and dependency workaround. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
