viirya opened a new pull request, #26035: URL: https://github.com/apache/datafusion/pull/26035
## Which issue does this PR close? Closes #26034. ## Rationale for this change Substrait roundtripping loses pushed-down TableScan offsets and limits when no parent Limit exists. A Projection between the scan and a parent Limit also bypasses the existing offset compensation, changing query results. ## What changes are included in this PR? - Encode a scan's skip/fetch in a FetchRel wrapping its ReadRel. - Serialize each parent Limit's own offset without adding the scan offset again. - Add execution-based roundtrip coverage for 96 combinations of scan offsets, fetch counts, optional projections, and parent limits, including zero limits and offsets beyond the input. ## What is the testing strategy for this PR? Verified the offset-only regression against upstream 60477917bd9eadd1a299185db6a8a08ccae7a4c7: the original scan returns [2, 3, 4, 5], while the roundtripped scan incorrectly returns all six rows. The new regression test passes with the fix. Passed: - cargo fmt --all - cargo clippy --all-targets --all-features -- -D warnings - All Substrait integration tests: 226 passed, 6 ignored. - Required extended workspace tests: 12,387 passed, 0 failed, 8 ignored; all 526 SQLLogicTests files completed. - Contributor lint suite through formatting, its feature-specific Clippy check, TOML formatting, license headers, typos, documentation formatting and generation, workflow checks, large-file checks, Markdown links, and security audit. - Rust documentation check. Environment limitations: - Circular dependency check cannot resolve dev/depcheck's locked cargo 0.100.0 through the required Databricks registry proxy, which currently exposes versions through 0.99.0. The unused-dependency checker also reports this metadata-resolution failure. - HTML documentation build prerequisites cargo-depgraph and Graphviz are unavailable. No relevant local Substrait producer benchmark is defined. ## Are there any user-facing changes? Substrait roundtrips preserve pushed-down table-scan offsets and limits, including across projections and nested limits. No public API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
