zhztheplayer commented on code in PR #9145:
URL: https://github.com/apache/incubator-gluten/pull/9145#discussion_r2016227274
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ColumnarCollectLimitBaseExec.scala:
##########
@@ -33,15 +32,11 @@ abstract class ColumnarCollectLimitBaseExec(
override def outputPartitioning: Partitioning = SinglePartition
override protected def doValidateInternal(): ValidationResult = {
-
- if (
- (childPlan.supportsColumnar && GlutenConfig.get.enablePreferColumnar) &&
- BackendsApiManager.getSettings.supportColumnarShuffleExec() &&
- SparkShimLoader.getSparkShims.isColumnarLimitExecSupported()
- ) {
- return ValidationResult.succeeded
+ if (!SparkShimLoader.getSparkShims.isColumnarLimitExecSupported()) {
+ return ValidationResult.failed(
+ "Columnar collect-limit is unsupported under the current Spark
version")
Review Comment:
It's expected if it only involves performance degradation or so. Because
`doValidate()` is not expected to be called with offloaded child operators or
not. If this kind of joint-optimization is required, let's consider adding an
individual rule with `injectPostTransform` that matches on certain patterns in
the plan node tree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
