mohitgurav20 commented on issue #6251:
URL:
https://github.com/apache/datafusion-comet/issues/6251#issuecomment-5852758497
Hi @andygrove, I would like to take this up and implement the fix!
### Root Cause & Proposed Fix
The issue occurs because Java Arrow's `ArrayImporter` indexes struct
children by field name during JVM vector import, causing duplicate struct field
names (e.g., from `arrays_zip(a, a)`) to collapse and throw
`java.lang.IllegalStateException: ArrowArray struct has 2 children (expected
1)`.
Following the established pattern in `CometCreateNamedStruct`:
1. **SerDe Validation**: Update `CometArraysZip.getSupportLevel` in
`spark/src/main/scala/org/apache/comet/serde/arrays.scala` to evaluate
`expr.names` and check for duplicate field names:
```scala
val names = expr.names.map(_.eval(EmptyRow))
if (names.length != names.distinct.length) {
return Unsupported(Some("`arrays_zip` with duplicate field names is not
supported"))
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
