ArnavBalyan commented on code in PR #9145:
URL: https://github.com/apache/incubator-gluten/pull/9145#discussion_r2016156347
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ColumnarCollectLimitBaseExec.scala:
##########
@@ -33,15 +32,11 @@ abstract class ColumnarCollectLimitBaseExec(
override def outputPartitioning: Partitioning = SinglePartition
override protected def doValidateInternal(): ValidationResult = {
-
- if (
- (childPlan.supportsColumnar && GlutenConfig.get.enablePreferColumnar) &&
- BackendsApiManager.getSettings.supportColumnarShuffleExec() &&
- SparkShimLoader.getSparkShims.isColumnarLimitExecSupported()
- ) {
- return ValidationResult.succeeded
+ if (!SparkShimLoader.getSparkShims.isColumnarLimitExecSupported()) {
+ return ValidationResult.failed(
+ "Columnar collect-limit is unsupported under the current Spark
version")
Review Comment:
We may need the child operator to support columnar since this operator
implementation relies on child columnar execution. Anything without it may
invoke an R2C overhead between collect limit and child operator. Is this
expected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
