peterxcli opened a new pull request, #5725: URL: https://github.com/apache/datafusion-comet/pull/5725
## Which issue does this PR close? Closes #5687. ## Rationale for this change `concat_ws` with string-array arguments currently uses Spark codegen dispatch, or falls back to Spark when dispatch is disabled. DataFusion Spark 55, already used by Comet, provides the required native implementation. ## What changes are included in this PR? Wire `concat_ws` to `SparkConcatWs` and serialize its explicit string return type to bypass DataFusion's string-only signature. Remove the array argument fallback restriction. Retain the existing all-foldable fallback and NULL-separator shortcut. Update native and Spark regressions for ordered mixed arguments, multiple arrays, nulls, empty arrays and strings, Unicode, and column separators. The Scala regression disables codegen dispatch to require native execution. Update expression support and audit documentation. The `implement-comet-expression` skill was used to scaffold the implementation workflow. ## How are these changes tested? - Native build and JVM package build passed. - Rust registration regression: 1 test passed. - Spark 4.1: the concat_ws SQL-file case passed all 16 queries; the Scala native-path regression passed with codegen dispatch disabled and native split enabled for its split inputs. - Workspace Clippy with `-D warnings`, Cargo formatting, Spotless, and Scala style checks passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
