malinjawi commented on code in PR #12215:
URL: https://github.com/apache/gluten/pull/12215#discussion_r3720331012
##########
gluten-delta/src/main/scala/org/apache/gluten/extension/DeltaPostTransformRules.scala:
##########
@@ -90,6 +99,25 @@ object DeltaPostTransformRules {
child.copy(output = p.output)
}
+ /**
+ * Native DELETE/UPDATE/MERGE DV support can deliberately keep the target
row-index scan in Spark.
+ * Keep only the contiguous scan-adjacent filter/project chain in Spark,
avoiding an isolated
+ * native island without propagating fallback through joins, exchanges, or
aggregations. The
+ * BitmapAggregator and the rest of the DML plan remain eligible for native
execution.
+ */
+ val keepDmlRowIndexFallbackSubtreeOnSpark: Rule[SparkPlan] = (plan:
SparkPlan) =>
Review Comment:
@felipepessoto I think That check would disable this rule exactly when it's
needed as it fires when the DML target scan did not offload, so the plan has a
`FileSourceScanExec` with a fallback tag and possibly no `DeltaScanTransformer`
at all.
So I added the equivalent short-circuit in the latest commit instead such
that the rule exits early unless the plan contains a tagged DML row-index
fallback scan for the same condition it rewrites on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
