cshuo opened a new pull request, #19997: URL: https://github.com/apache/hudi/pull/19997
### Describe the issue this Pull Request addresses RFC-112 was claimed in #19993. Spark 4.1 introduced Spark Declarative Pipelines, but Hudi does not yet define or validate an end-to-end SDP source and target contract. Users must assemble equivalent layered pipelines with separate Spark jobs, writer calls, checkpoints, and external orchestration. This RFC proposes first-class Hudi integration with SDP while keeping the existing ownership boundary: SDP manages dataset declarations, dependencies, checkpoints, and execution; Hudi retains table storage, commit, upsert, incremental-read, and table-service semantics. ### Summary and Changelog - Add RFC-112 covering Hudi-backed SDP materialized views, streaming tables, keyed upserts, and basic Hudi-to-Hudi incremental chaining. - Define the proposed catalog, batch refresh, streaming sink, configuration, failure, rollout, and test contracts. - Link RFC-112 from the RFC index. ### Impact Documentation-only proposal. This pull request does not change runtime behavior, public APIs, configuration handling, or performance. Implementation will follow separately after community review. ### Risk Level None. This pull request only adds design documentation. ### Documentation Update Adds RFC-112 and links it from `rfc/README.md`. ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
