andygrove opened a new issue, #7621: URL: https://github.com/apache/datafusion-comet/issues/7621
### What is the problem the feature request solves? A query that fails on a Comet Native Executor cluster reaches Spark as a `CometNativeException` carrying the cluster's message, not as the exception Spark would raise. For example, `id div k` with ANSI mode on and `k = 0` raises `SparkArithmeticException` with the `DIVIDE_BY_ZERO` error class in Spark, but on the cluster it surfaces as a `CometNativeException`. Ballista keeps only the Debug string of the failing task's error, so the driver has nothing structured to rebuild the exception from. ### Describe the potential solution Have the cluster return a structured error for a failed query (the Comet error with its Spark error class and parameters), and have the driver rebuild Spark's exception from it through `SparkErrorConverter`, as `CometExecIterator` does for Comet on Spark. ### Additional context Raised in review of #7544, which lists this under Current limitations in `native_executors.md`. Part of #7288. ### Willingness to contribute Yes. I can contribute this feature independently. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
