alexanderbianchi opened a new pull request, #26535: URL: https://github.com/apache/datafusion/pull/26535
## Which issue does this PR close? Related to #22676: this adds reference resolution only, without materialization. ## Rationale for this change Substrait producers can represent a shared subtree once in `Plan.relations` and refer to it from multiple inputs using `ReferenceRel`. The consumer currently rejects plans with multiple relation trees and does not handle reference relations. ## What changes are included in this PR? - Accept one output root alongside referenced relation trees, preserving support for a single bare relation. - Resolve `ReferenceRel.subtree_ordinal` against the enclosing plan and inline the target independently at each occurrence, including forward and nested references. - Add `DefaultSubstraitConsumer::new_with_plan` and a `SubstraitConsumer::consume_reference` hook for custom consumers. - Reject invalid ordinals, cycles, and unsupported output-root layouts. Reference traversal is cleaned up on errors and cancellation. ## What is the testing strategy for this PR? Public API integration tests cover repeated references, forward and nested references, invalid graphs, output-root validation, ignoring a referenced root's output names, and reuse after a failed conversion. A volatile counter UDF verifies that two references execute the body twice rather than share cached results. Validation passed: - `cargo fmt --all --check` - `cargo clippy --all-targets --all-features -- -D warnings` - The contributor guide's extended workspace test command, including all 536 SQL logic test files. - Substrait: 68 unit tests and 234 integration tests passed (6 ignored). ## Are there any user-facing changes? `from_substrait_plan` can consume plans with referenced subtrees. References are inlined; they do not imply materialization. The additional consumer APIs are additive. Multiple output roots and referenced trees containing outer references remain unsupported. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
