andygrove opened a new pull request, #4716: URL: https://github.com/apache/datafusion-comet/pull/4716
## Which issue does this PR close? No dedicated issue. `array_prepend` was previously listed as planned (🔜) in the expression reference. ## Rationale for this change `array_prepend` is a `RuntimeReplaceable` expression in Spark. In every version that defines it (3.5.8, 4.0.1, 4.1.1) the analyzer rewrites it to `ArrayInsert(arr, Literal(1), elem)` before Comet's serde runs, and that replacement is identical across all three versions. Comet already implements `ArrayInsert` as `Compatible`, so `array_prepend` already executes natively end to end. No new Scala serde, protobuf, or Rust code is required. This PR locks that in with a regression test and corrects the documented support status. ## What changes are included in this PR? - Add a Comet SQL file test at `spark/src/test/resources/sql-tests/expressions/array/array_prepend.sql`. It is guarded with `MinSparkVersion: 3.5` because `array_prepend` was added in Spark 3.5.0 and does not exist in 3.4. Coverage includes column/literal/mixed arguments, NULL array yielding NULL, NULL element prepended, empty array, the int/string/boolean/double (including NaN)/long/multibyte-UTF8 element types, and the three cases Spark's own `DataFrameFunctionsSuite` exercises: type coercion (`array_prepend(array(1, 2), 1.23D)` -> `[1.23, 1.0, 2.0]`), nested-array elements, and binary elements. - Mark `array_prepend` as supported (✅) in `docs/source/user-guide/latest/expressions.md`. - Record the cross-version audit in `docs/source/contributor-guide/expression-audits/array_funcs.md`. The `implement-comet-expression` skill was used to scaffold this work, including the `audit-comet-expression` cross-version comparison against Spark 3.4.3, 3.5.8, 4.0.1, and 4.1.1. ## How are these changes tested? New SQL file test `array_prepend.sql`, run via `./mvnw test -Dsuites="org.apache.comet.CometSqlFileTestSuite array_prepend" -Dtest=none`. The framework runs each query under both Spark and Comet and asserts the results match and that the expression runs natively. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
