nat-openai opened a new pull request, #6831: URL: https://github.com/apache/datafusion-comet/pull/6831
## Which issue does this PR close? No linked issue. This fixes two operand-handling bugs in native date and timestamp truncation. ## Rationale for this change - With native truncation opted in, `trunc(DATE '2024-06-15', fmt)` fails when `fmt` is a column, even when every format is valid. It should use each row's format, returning June 1 for `month` and January 1 for `year`. - `TimestampTruncExpr` evaluates both a timestamp and a format, but exposes only the timestamp to expression-tree traversal. A rewrite that changes column positions therefore leaves the format at its old position, which can select the wrong column or fail during evaluation. ## What changes are included in this PR? - Expand a scalar DATE to the format column's length and reuse the existing batch truncation kernel. Array inputs retain their existing buffers. - Expose both timestamp operands through `children()` and replace both in `with_new_children()`, preserving the session timezone and overflow policy. - Add regression tests for mixed formats, pre-epoch dates, sliced dictionary formats, null dates, empty batches and timestamp column remapping. Extend the SQL fixture with explicit native-execution checks. Nonliteral DATE formats retain their existing incompatibility opt-in. This does not change the kernels' handling of invalid or null format values, expression admission, or timestamp truncation semantics. ## How are these changes tested? - Before the production fix, all four new Rust regressions failed and 42 existing truncation tests passed. With the fix, all 46 truncation tests pass. - All 1,171 expression unit tests pass. - All 34 selected SQL cases pass on Spark 4.1.3 with JDK 17, including the updated DATE fixture and existing timestamp, timezone and overflow coverage. The native library was freshly built from this candidate; the packaged library has the same SHA-256. - The native build, root Maven package build, workspace all-target Clippy with warnings denied, Rust formatting, Maven Spotless and diff whitespace checks pass. - Independent correctness, performance and scope/simplicity reviews found no actionable issues. No performance improvement is claimed; existing batch kernels are unchanged. AI assistance was used to implement and review this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
