andygrove opened a new issue, #1612:
URL: https://github.com/apache/datafusion-comet/issues/1612
### Describe the bug
I tried testing complex type support from the Spark shell, by setting
`COMET_PARQUET_SCAN_IMPL=native_datafusion` but I could not read a Parquet file
containing structs and arrays.
The error message says "Unsupported schema" which isn't a great user
experience. It would be better if we could show the specific reason why the
schema isn't supported (in this case, I had not enabled
`COMET_SCAN_ALLOW_INCOMPATIBLE`).
```
scala> spark.sql("select * from t1").show
25/04/05 09:36:50 WARN CometSparkSessionExtensions$CometExecRule: Comet
cannot execute some parts of this plan natively (set
spark.comet.explainFallback.enabled=false to disable this logging):
CollectLimit [COMET: CollectLimit is not supported]
+- Project [COMET: Project is not native because the following children are
not native (Scan parquet )]
+- Scan parquet [COMET: Unsupported schema
StructType(StructField(c0,BooleanType,true),StructField(c1,ByteType,true),...
```
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
