rich7420 opened a new pull request, #5954: URL: https://github.com/apache/datafusion-comet/pull/5954
## Which issue does this PR close? Closes #5911. ## Rationale for this change An `ARRAY<STRING>` field currently prevents Spark-to-Comet conversion even though the Arrow list and string writers already support it. This blocks otherwise eligible native operators above configured Spark sources. ## What changes are included in this PR? Admit binary string arrays through the existing type check, including nullable elements and placement inside supported structs. Reuse the existing readers and writers, and document the supported conversion types. Add coverage for schema restrictions, JSON/Parquet conversion and native shuffle, plus row and columnar reader tests for list offsets, UTF-8 values, batching, buffer ownership and cleanup. ## How are these changes tested? The two new `CometExecSuite` tests and nine string-array cases in `CometArrowStreamSuite` pass in the local verification run on Spark 4.1.3 / JDK 21 after rebasing onto main, using a native library rebuilt with Rust 1.98.1. Formatting and Scalastyle passed. Cross-version fork CI passed before rebasing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
