karuppayya opened a new pull request, #4519:
URL: https://github.com/apache/datafusion-comet/pull/4519
## Which issue does this PR close?
Closes #4518 .
## Rationale for this change
Introduces a stage-aware fallback that reverts stages to Spark row-based
execution when the number of C2R transitions exceeds a threshold.
## What changes are included in this PR?
- `RevertNativeForTransitionHeavyStages` — `postColumnarTransitions` rule
that counts C2R transitions per stage and reverts via `CometExec.originalPlan`
when threshold exceeded
- Configs: `spark.comet.exec.transitionRevert.enabled` (default: true),
`spark.comet.exec.transitionRevert.maxTransitions` (default: 2, min: 2)
- Works for both AQE and non-AQE paths
-
## How are these changes tested?
Unit tests added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
