luis4a0 commented on code in PR #13103:
URL: https://github.com/apache/gluten/pull/13103#discussion_r4536974852


##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -152,6 +152,19 @@ bool SubstraitToVeloxPlanValidator::validateRound(
     return false;
   }
 
+  const auto& valueType = params[0]->type();
+  if (valueType->isDecimal()) {
+    const auto& scale = arguments[1].value().literal();

Review Comment:
   This is the same private-call invariant: `validateScalarFunction` creates 
`params` in a 1:1 loop over `scalarFunction.arguments()` and returns before 
calling `validateRound` if an argument cannot be validated. The converter 
contract supplies one non-null expression per argument. Thus two Substrait 
arguments imply at least two converted parameters; no independently incomplete 
vector can reach this function.



##########
gluten-substrait/src/main/scala/org/apache/gluten/expression/DecimalRoundTransformer.scala:
##########
@@ -20,15 +20,16 @@ import org.apache.gluten.backendsapi.BackendsApiManager
 import org.apache.gluten.exception.GlutenNotSupportException
 
 import org.apache.spark.sql.catalyst.expressions._
-import org.apache.spark.sql.types.{DataType, DecimalType}
+import org.apache.spark.sql.types.{DataType, DecimalType, IntegerType}
 
 case class DecimalRoundTransformer(
     substraitExprName: String,
     child: ExpressionTransformer,
     original: Round)
   extends BinaryExpressionTransformer {
 
-  val toScale: Int = original.scale.eval(EmptyRow).asInstanceOf[Int]
+  private val scaleValue = original.scale.eval(EmptyRow)
+  val toScale: Int = if (scaleValue == null) 0 else 
scaleValue.asInstanceOf[Int]

Review Comment:
   An invalid foldable scale is an error in Spark as well, not a case where 
fallback makes the query succeed. Spark `RoundBase` evaluates 
`scale.eval(EmptyRow)` into `_scale`, and decimal result typing reads `_scale` 
during planning. Converting that error into an unsupported-expression fallback 
would only defer the same Spark error and change its timing. The API regression 
intentionally verifies that the invalid cast throws while the generic Scala 
validator does not impose backend-specific eligibility. No catch is needed here.



##########
gluten-ut/common/src/test/scala/org/apache/spark/sql/GlutenDecimalRoundSuiteBase.scala:
##########
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql
+
+import org.apache.gluten.config.GlutenConfig
+import org.apache.gluten.execution.ProjectExecTransformer
+
+import org.apache.spark.sql.catalyst.expressions.{Literal, Round}
+import org.apache.spark.sql.catalyst.optimizer.{ConstantFolding, 
NullPropagation}
+import org.apache.spark.sql.internal.SQLConf
+import org.apache.spark.sql.types._
+
+import scala.collection.JavaConverters._
+
+abstract class GlutenDecimalRoundSuiteBase extends GlutenSQLTestsTrait {
+  private def withInput(dataType: DecimalType, values: Seq[String])(f: => 
Unit): Unit = {
+    val schema = new StructType().add("value", dataType)
+    val rows = values.map(value => Row(new java.math.BigDecimal(value))) :+ 
Row(null)
+    withSQLConf(
+      SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "false",
+      GlutenConfig.GLUTEN_ANSI_FALLBACK_ENABLED.key -> "false") {
+      withTempPath {
+        path =>
+          spark.createDataFrame(rows.asJava, 
schema).coalesce(1).write.parquet(path.toString)
+          withTempView("decimal_round_input") {
+            
spark.read.parquet(path.toString).createOrReplaceTempView("decimal_round_input")
+            f
+          }
+      }
+    }
+  }
+
+  private def assertExecution(df: DataFrame, native: Boolean): Unit = {
+    val plan = df.queryExecution.executedPlan
+    val locations = plan.collect {
+      case node if node.expressions.exists(_.exists(_.isInstanceOf[Round])) =>
+        node.isInstanceOf[ProjectExecTransformer]
+    }
+    assert(
+      locations.nonEmpty && locations.forall(_ == native),
+      s"Unexpected ROUND execution:\n$plan")
+  }
+
+  private def checkRound(scale: String, native: Boolean = true): Unit = {
+    val query = s"SELECT round(value, $scale) FROM decimal_round_input"
+    withSQLConf(GlutenConfig.GLUTEN_ENABLED.key -> "false") {
+      val vanilla = sql(query)
+      val expectedSchema = vanilla.schema
+      val expected = vanilla.collect().toSeq
+      withSQLConf(GlutenConfig.GLUTEN_ENABLED.key -> "true") {
+        val df = sql(query)
+        assertExecution(df, native)
+        assert(df.schema == expectedSchema)
+        checkAnswer(df, expected)
+      }
+    }
+  }
+
+  testGluten("decimal ROUND preserves schemas and HALF_UP results across 
storage boundaries") {
+    val cases = Seq(
+      (DecimalType(3, 1), Seq("2.5", "-2.5", "0")),
+      (DecimalType(18, 0), Seq("999999999999999999", "-999999999999999999")),
+      (DecimalType(19, 0), Seq("9999999999999999999", "-9999999999999999999")),
+      (DecimalType(38, 1), Seq("9" * 37 + ".9", "-" + "9" * 37 + ".9")),
+      (DecimalType(38, 38), Seq("0.6", "-0.6", "0.5", "-0.5"))
+    )
+    for ((dataType, values) <- cases) {
+      withInput(dataType, values) {
+        for (ansi <- Seq("false", "true")) {
+          withSQLConf(SQLConf.ANSI_ENABLED.key -> ansi) {
+            Seq(-400, -39, -1, 0, 1, 38, 400).foreach(scale => 
checkRound(scale.toString))
+            checkRound("-1 + 0")
+          }
+        }
+      }
+    }
+  }
+
+  testGluten("decimal ROUND handles the 38-digit HALF_UP tie and precision 
overflow") {
+    withInput(DecimalType(38, 0), Seq("4" + "9" * 37, "-4" + "9" * 37)) {
+      checkRound("-38")
+    }
+    for ((value, scale) <- Seq(("9" * 38, -1), ("5" + "0" * 37, -38), ("-5" + 
"0" * 37, -38))) {
+      withInput(DecimalType(38, 0), Seq(value)) {
+        for (ansi <- Seq("false", "true")) {
+          withSQLConf(SQLConf.ANSI_ENABLED.key -> ansi) {
+            val query = s"SELECT round(value, $scale) FROM decimal_round_input"
+            withSQLConf(GlutenConfig.GLUTEN_ENABLED.key -> "false") {
+              val error = intercept[Exception](sql(query).collect())
+              
assert(Iterator.iterate[Throwable](error)(_.getCause).takeWhile(_ != 
null).exists {
+                cause =>
+                  Option(cause.getMessage).exists {
+                    message =>
+                      message.contains("NUMERIC_VALUE_OUT_OF_RANGE") ||
+                      
message.contains("DECIMAL_PRECISION_EXCEEDS_MAX_PRECISION")
+                  }
+              })
+            }
+            val df = sql(query)
+            assertExecution(df, native = true)
+            val error = intercept[Exception](df.collect())
+            assert(Iterator.iterate[Throwable](error)(_.getCause).takeWhile(_ 
!= null).exists {
+              cause => Option(cause.getMessage).exists(_.contains("Decimal 
overflow"))
+            })

Review Comment:
   There is no more specific stable Spark error class exposed through the JNI 
`GlutenException`; the generic native code is `INVALID_ARGUMENT`, which would 
match many unrelated failures. The message token is the registered native 
function contract: `DecimalRound.cpp` emits `Status::UserError("Decimal 
overflow in {}.", name_)`, and Velox’s own `DecimalRoundTest` asserts `"Decimal 
overflow in decimal_round."`. Keeping the focused substring verifies that this 
exact overflow path was reached without coupling the Gluten test to the full 
wrapper/stack-trace text.



##########
gluten-substrait/src/main/scala/org/apache/gluten/expression/DecimalRoundTransformer.scala:
##########
@@ -43,5 +44,6 @@ case class DecimalRoundTransformer(
   }
 
   override def left: ExpressionTransformer = child
-  override def right: ExpressionTransformer = LiteralTransformer(toScale)
+  override def right: ExpressionTransformer =
+    LiteralTransformer(Literal.create(scaleValue, IntegerType))

Review Comment:
   This eager evaluation matches Spark’s own `RoundBase` contract. Spark stores 
`scale.eval(EmptyRow)` in `scaleV`/`_scale`, and decimal `dataType` depends on 
`_scale`; a foldable ANSI cast failure therefore surfaces during Spark planning 
too. Falling back cannot produce a successful Spark plan for that input and 
would only move the same error. The current behavior is intentional.



##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -141,7 +141,7 @@ bool SubstraitToVeloxPlanValidator::flattenDualLevel(const 
TypePtr& type, std::v
 
 bool SubstraitToVeloxPlanValidator::validateRound(
     const ::substrait::Expression::ScalarFunction& scalarFunction,
-    const RowTypePtr& inputType) {
+    const std::vector<core::TypedExprPtr>& params) {

Review Comment:
   Verified against the private caller contract. `validateScalarFunction` 
iterates the Substrait arguments and appends exactly one converted, non-null 
expression to `params` for each argument; if nested validation fails, it 
returns before dispatching to `validateRound`. Therefore `params.size() == 
arguments.size()` here, and the existing `arguments.size() < 2` check 
guarantees `params[0]` exists. The header documents this invariant, so an 
additional size check would only duplicate an unreachable condition.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to