andygrove opened a new pull request, #4476: URL: https://github.com/apache/datafusion-comet/pull/4476
## Which issue does this PR close? Closes #. ## Rationale for this change Continuation of the per-category expression audit. Same pattern as #4475 (conditional), #4474 (misc), #4473 (collection), #4470 (json), #4469 (struct), using the updated `audit-comet-expression` skill in #4468. ## What changes are included in this PR? ### Support-doc audit notes Add per-version audit sub-bullets to `crc32`, `hash`, `md5`, `sha`, `sha1`, `sha2`, and `xxhash64`. `sha` is a registry alias of `Sha1`. Spark 4.0 only adds the `DefaultStringProducingExpression` trait and the `nullIntolerant: Boolean` field refactor on the four `String`-producing expressions (`Md5`, `Sha1`, `Sha2`, `Crc32`); no runtime behaviour change across the category. ### Support-level consistency fixes (in `hash.scala`) - Refactor `HashUtils` to return reasons (`unsupportedReasonFor`, `supportLevelForChildren`, `unsupportedReasons`) instead of calling `withInfo` from inside the helper. The recursive type check no longer side-effects on the expression tree at type-check time, which the audit skill calls out as the canonical antipattern. - `CometXxHash64`, `CometMurmur3Hash`, `CometSha1`, `CometSha2`: override `getSupportLevel` and `getUnsupportedReasons` so the unsupported-child-type and (for `Sha2`) the non-foldable-`numBits` restrictions reach both the dispatcher's EXPLAIN message and the compatibility doc generator. ### Tracking issues filed for follow-up None. The `TimeType` gap (Spark 4.0+) is covered by the existing #4418 EPIC; the `DecimalType`-precision-18 gap is a documented semantic difference (Spark hashes via Java `BigDecimal`), already declared by the new `HashUtils.unsupportedReasons`. ### Audit process Audited directly using the `audit-comet-expression` skill (4 Spark versions per #4468). Four serde objects plus the shared `HashUtils` helper. ## How are these changes tested? - `./mvnw test -Dsuites="org.apache.comet.CometHashExpressionSuite" -Dtest=none` (37 tests pass) - `make core` succeeds with the serde refactor. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
