andygrove opened a new issue, #2199:
URL: https://github.com/apache/datafusion-comet/issues/2199
### What is the problem the feature request solves?
In `CometExecRule` we currently have a lot of restrictions on supporting
nested complex types:
```
case ArrayType(ArrayType(_, _), _) => false // TODO: nested array is not
supported
case ArrayType(MapType(_, _, _), _) => false // TODO: map array element
is not supported
case ArrayType(elementType, _) =>
supportedShuffleDataType(elementType)
case MapType(MapType(_, _, _), _, _) => false // TODO: nested map is not
supported
case MapType(_, MapType(_, _, _), _) => false
case MapType(StructType(_), _, _) => false // TODO: struct map key/value
is not supported
case MapType(_, StructType(_), _) => false
case MapType(ArrayType(_, _), _, _) => false // TODO: array map
key/value is not supported
case MapType(_, ArrayType(_, _), _) => false
case MapType(keyType, valueType, _) =>
supportedShuffleDataType(keyType) &&
supportedShuffleDataType(valueType)
```
We apply the same checks both to native and columnar shuffle, which doesn't
seem correct to me. I wonder if some of these limitations existed early on in
complex type support and can be removed now.
### Describe the potential solution
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
