zhztheplayer commented on code in PR #7360:
URL: https://github.com/apache/incubator-gluten/pull/7360#discussion_r1802558974
##########
backends-velox/src/main/java/org/apache/gluten/columnarbatch/VeloxColumnarBatches.java:
##########
@@ -45,6 +45,14 @@ public static void checkNonVeloxBatch(ColumnarBatch batch) {
String.format("Comprehensive batch type is already %s",
COMPREHENSIVE_TYPE_VELOX));
}
+ public static Boolean isVeloxBatch(ColumnarBatch batch) {
+ if (!ColumnarBatches.isLightBatch(batch)) {
+ return false;
+ }
+ final String comprehensiveType =
ColumnarBatches.getComprehensiveLightBatchType(batch);
+ return Objects.equals(comprehensiveType, COMPREHENSIVE_TYPE_VELOX);
+ }
Review Comment:
Let's remove this API as it's for implicit transitions which were deprecated.
##########
gluten-substrait/src/main/scala/org/apache/gluten/expression/ExpressionUtils.scala:
##########
@@ -31,4 +34,15 @@ object ExpressionUtils {
1 + childrenDepth.max
}
}
+
+ class FallbackComplexExpressions(threshold: Int) extends Validator {
+ override def validate(plan: SparkPlan): Validator.OutCome = {
+ if (plan.expressions.exists(e =>
ExpressionUtils.getExpressionTreeDepth(e) > threshold)) {
+ return fail(
+ s"Disabled because at least one present expression exceeded depth
threshold: " +
+ s"${plan.nodeName}")
+ }
+ pass()
+ }
+ }
Review Comment:
nit: We can keep this as a private class in `Validators.scala`, then add an
API `def isComplexExpression(): Boolean` in `ExpressionUtils.scala`, then let
`FallbackComplexExpressions` call this API to generate validation outcome.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
