sakshichitnis27 opened a new pull request, #7511: URL: https://github.com/apache/datafusion-comet/pull/7511
## Which issue does this PR close? Closes #6977. ## Rationale for this change The top-level and nested field-major struct paths repeat the same type dispatch and null handling. Keeping one field-processing implementation reduces the places where a type or null-handling change must be made. ## What changes are included in this PR? - Collect top-level struct row addresses, sizes, and validity once, then pass them to the existing nested field-major implementation. - Use `process_field!` for Binary, Utf8, and Decimal128 fields. - Add tests for variable-length fields, decimals, nulls, lists, and a nonzero row range across top-level and nested structs. ## How are these changes tested? - `cd native && cargo test -p datafusion-comet-shuffle --lib` — 210 passed with stable Rust 1.99.1. - `cd native && cargo clippy -p datafusion-comet-shuffle --lib --tests -- -D warnings` - `rustfmt --check native/shuffle/src/spark_unsafe/row.rs` - `cd native && cargo bench --profile dev -p datafusion-comet-shuffle --bench row_columnar -- fields_20/rows_10000 --sample-size 15 --warm-up-time 1 --measurement-time 3` — before/after, 10,000 rows and 20 fields: flat 26.3 → 13.5 ms; nested 17.0 → 17.0 ms; deeply nested 18.3 → 18.1 ms. Spark/JVM integration suites were not run locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
