felixloesing opened a new pull request, #12279:
URL: https://github.com/apache/gluten/pull/12279
## What changes are proposed in this pull request?
Adds `spark.gluten.sql.columnar.failOnFallback` (default false). When
enabled, the query throws a GlutenException listing the operators and reasons
whenever any operator falls back to Spark instead of running natively.
We are rolling out Gluten to only queries that have been determined to be
fully offloadable and use this config to guard against query changes after
migration.
GlutenFallbackReporter now drives both its log output and the SQL UI summary
through a shared GlutenExplainUtils.visitFallbackNodes helper, so all three
views (logs, UI, failOnFallback) agree on what counts as a fallback. As a side
effect, untagged vanilla operators that were previously silent in the reporter
logs are now reported with the reason "Gluten does not touch it or does not
support it", matching what the SQL UI has always shown.
## How was this patch tested?
- GlutenFallbackSuite: new fail job on fallback when failOnFallback is
enabled test covers both default-off (no throw) and enabled (throws
GlutenException mentioning the config key).
- Existing fallback logging and event tests still pass.
- Manual: run a query with an unsupported operator under failOnFallback=true
and confirm the job fails with a useful operator/reason list.
- This has been rolled out internally
## Was this patch authored or co-authored using generative AI tooling?
Co-authored with Claude Opus 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
