andygrove opened a new issue, #385:
URL: https://github.com/apache/datafusion-comet/issues/385
### What is the problem the feature request solves?
As a user, I do not think I have a way to see why my query stage did not run
natively.
We store information in the plan with the reasons why we don't support
specific operators and expressions and this information is available
programatically via `ExtendedExplainInfo.generateExtendedInfo` but this only
seems to be accessed from tests.
My workaround for now is to add this code fragment at the end of
`CometSparkSessionExtensions.CometExecRule#apply`:
```scala
if (!isCometNative(finalPlan)) {
val str = new ExtendedExplainInfo().generateExtendedInfo(finalPlan)
logInfo(s"Cannot run plan natively:\n\t- ${str.split("\n").mkString("\n\t-
")}")
}
```
Side note: It would be nice to store the reasons as `Set[String]` rather
than newline-delimited string containing multiple reasons.
Example output:
```
Cannot run plan natively:
- LocalTableScan is not supported
- WriteFiles is not supported
- Execute InsertIntoHadoopFsRelationCommand is not supported
Cannot run plan natively:
- Execute CreateViewCommand is not supported
Cannot run plan natively:
- Comet does not guarantee correct results for cast from StringType to
TimestampType with timezone Some(UTC) and evalMode LEGACY (Not all valid
formats are supported). To enable all incompatible casts, set
spark.comet.cast.allowIncompatible=true
```
It would be nice to enable this if a config is set. Perhaps
`spark.comet.explain=VERBOSE`.
### Describe the potential solution
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
