PHILO-HE commented on code in PR #10545:
URL:
https://github.com/apache/incubator-gluten/pull/10545#discussion_r2332772067
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/WholeStageTransformer.scala:
##########
@@ -446,10 +446,11 @@ case class WholeStageTransformer(child: SparkPlan,
materializeInput: Boolean = f
/**
* the whole stage contains NO [[LeafTransformSupport]]. this the
default case for:
- * 1. SCAN with clickhouse backend (check
ColumnarCollapseTransformStages#separateScanRDD())
- * 2. test case where query plan is constructed from simple
dataframes (e.g.
- * GlutenDataFrameAggregateSuite) in these cases, separate RDDs
takes care of SCAN as a
- * result, genFinalStageIterator rather than genFirstStageIterator
will be invoked
+ * 1. SCAN with clickhouse backend (check
Review Comment:
@beliefer, I suggest to refine the comments for better readability as
follows. Thanks.
```diff
diff --git
a/gluten-substrait/src/main/scala/org/apache/gluten/execution/WholeStageTransformer.scala
b/gluten-substrait/src/main/scala/org/apache/gluten/execution/WholeStageTransformer.scala
index 0c5e1b58b..588ba4567 100644
---
a/gluten-substrait/src/main/scala/org/apache/gluten/execution/WholeStageTransformer.scala
+++
b/gluten-substrait/src/main/scala/org/apache/gluten/execution/WholeStageTransformer.scala
@@ -438,11 +438,14 @@ case class WholeStageTransformer(child: SparkPlan,
materializeInput: Boolean = f
} else {
/**
- * the whole stage contains NO [[LeafTransformSupport]]. this the
default case for:
- * 1. SCAN with clickhouse backend (check
ColumnarCollapseTransformStages#separateScanRDD())
- * 2. test case where query plan is constructed from simple
dataframes (e.g.
- * GlutenDataFrameAggregateSuite) in these cases, separate RDDs
takes care of SCAN as a
- * result, genFinalStageIterator rather than
genFirstStageIterator will be invoked
+ * The whole stage contains NO [[LeafTransformSupport]]. This is the
default case for:
+ * - SCAN of clickhouse backend. See
+ *
BackendsApiManager.getSettings.excludeScanExecFromCollapsedStage.
+ * - Test case where query plan is constructed from simple
DataFrames, e.g.
+ * GlutenDataFrameAggregateSuite.
+ *
+ * In these cases, separate RDDs take care of SCAN. As a result,
genFinalStageIterator rather
+ * than genFirstStageIterator will be invoked.
*/
new WholeStageZippedPartitionsRDD(
sparkContext,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
