cshuo opened a new pull request, #19997: URL: https://github.com/apache/hudi/pull/19997
### Describe the issue this Pull Request addresses RFC-112 was reserved in #19993. Spark 4.1 introduced Spark Declarative Pipelines, but Hudi does not yet define or validate an end-to-end SDP source and target contract. Users must assemble equivalent layered pipelines with separate Spark jobs, writer calls, checkpoints, and external orchestration. This RFC specifies the Hudi integration contract while SDP retains orchestration and flow execution. Review identified catalog, append-write, and sink-failure prerequisites tracked in #20369, #20370, #20371, and #20400. ### Summary and Changelog - Add RFC-112 and link it from the RFC index. - Require timeline-preserving truncate together with checkpoint-generation isolation, and document the remaining non-atomic refresh window. - Define catalog creation/reconciliation, streaming configuration transport, retry behavior, downstream reset recovery, and table-service expectations. - Require query-scoped embedded timeline-service and write-client ownership, reuse across micro-batches, and cleanup on completion, stop, failure, and reset (#20400). - Specify Connect server setup, explicit business-column projection, a real CLI/Connect test harness, and a phased roadmap through fan-in, DSv2, and Auto CDC. ### Impact Documentation-only proposal; this PR changes no runtime behavior, public APIs, storage format, configuration defaults, or performance. The RFC explicitly identifies shared behavior changes that future implementation PRs must document and regression-test. ### Risk Level None. This pull request only adds design documentation. ### Documentation Update Adds RFC-112 at `rfc/rfc-112/rfc-112.md` and links it from `rfc/README.md`. Validation: checked the revised contract against the review comments and relevant Hudi/Spark source paths; checked Markdown fences, relative links, shell example syntax, and `git diff --check`; validated this description with `scripts/pr_compliance.py`. Runtime tests were not added or run because this PR contains no implementation. The RFC lists required implementation acceptance tests. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable — not applicable to this documentation-only PR; implementation tests are specified in the RFC. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
