viirya commented on code in PR #1120:
URL: https://github.com/apache/datafusion-comet/pull/1120#discussion_r1857303548
##########
spark/src/main/scala/org/apache/comet/CometSparkSessionExtensions.scala:
##########
@@ -1029,12 +1029,20 @@ class CometSparkSessionExtensions
var firstNativeOp = true
newPlan.transformDown {
case op: CometNativeExec =>
- if (firstNativeOp) {
+ val newPlan = if (firstNativeOp) {
firstNativeOp = false
op.convertBlock()
} else {
op
}
+
+ // If reaching leaf node, reset `firstNativeOp` to true
+ // because it will start a new block in next iteration.
+ if (op.children.isEmpty) {
+ firstNativeOp = true
+ }
+
+ newPlan
Review Comment:
Here this is the main diff that we don't have in `main` branch. Because
native scan now is a real "native" operator, once we encounter it when
transforming down, we need to reset `firstNativeOp` flag.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
