andygrove opened a new pull request, #7647: URL: https://github.com/apache/datafusion-comet/pull/7647
## Which issue does this PR close? Backport of #7084 to `branch-1.2`. Closes #7063 on this branch. ## Rationale for this change The native Murmur3 and xxHash64 kernels have no arm for `Null`, `Interval(YearMonth)` or `Duration(Microsecond)`, so `hash(a, null)`, `xxhash64(a, null)` and native round-robin shuffle over those types fail with `Unsupported data type in hasher`. Hashing a null struct that has a `NullType` field also panics (`NullArray data should not contain a null buffer`). `branch-1.2` has the same kernels. See #7084 and #7063 for the full analysis. ## What changes are included in this PR? Cherry-pick of f1778b986b (`git cherry-pick -x`) with no conflicts and no adaptations. It adds the missing arms to `create_hashes_internal!`, makes `child_with_parent_nulls` return a `Null` child unchanged, and includes the source PR's unit tests and SQL file tests. The diff against the source commit is identical apart from line numbers and blob hashes. Other release branches: `branch-1.2` is the newest release branch, so there is no newer one to port to. ## How are these changes tested? On this branch, with JDK 17 and the default Spark profile only: the 88 `hash_funcs` tests in `datafusion-comet-spark-expr` and the `struct_nulls` tests in `datafusion-comet-common` pass, `cargo check --all-targets` on both crates and `cargo fmt --check` are clean, and `./mvnw test` ran `CometSqlFileTestSuite` for `hash_null_type.sql`, `round_robin_null_and_intervals.sql`, `get_array_struct_fields.sql` and `get_struct_field.sql` against a native library built from this branch (4 tests, 0 failures). With the native sources (`native/common`, `native/spark-expr`) restored to `branch-1.2`'s version, all four SQL file tests fail (`Unsupported data type in hasher: Null` and the `NullArray` null-buffer panic), so the branch has the bug and the tests catch it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
