[GitHub] [spark] gengliangwang commented on a diff in pull request #36525: [SPARK-39166] Provide runtime error query context for binary arithmetic when WSCG is off
gengliangwang commented on code in PR #36525:
URL: https://github.com/apache/spark/pull/36525#discussion_r871525344
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala:
##
@@ -418,9 +428,11 @@ case class Multiply(
protected override def nullSafeEval(input1: Any, input2: Any): Any =
dataType match {
case _: IntegerType if failOnError =>
- MathUtils.multiplyExact(input1.asInstanceOf[Int],
input2.asInstanceOf[Int], origin.context)
+ MathUtils.multiplyExact(input1.asInstanceOf[Int],
input2.asInstanceOf[Int],
+queryContext)
Review Comment:
```suggestion
MathUtils.multiplyExact(input1.asInstanceOf[Int],
input2.asInstanceOf[Int], queryContext)
```
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala:
##
@@ -418,9 +428,11 @@ case class Multiply(
protected override def nullSafeEval(input1: Any, input2: Any): Any =
dataType match {
case _: IntegerType if failOnError =>
- MathUtils.multiplyExact(input1.asInstanceOf[Int],
input2.asInstanceOf[Int], origin.context)
+ MathUtils.multiplyExact(input1.asInstanceOf[Int],
input2.asInstanceOf[Int],
+queryContext)
case _: LongType if failOnError =>
- MathUtils.multiplyExact(input1.asInstanceOf[Long],
input2.asInstanceOf[Long], origin.context)
+ MathUtils.multiplyExact(input1.asInstanceOf[Long],
input2.asInstanceOf[Long],
+queryContext)
Review Comment:
```suggestion
MathUtils.multiplyExact(input1.asInstanceOf[Long],
input2.asInstanceOf[Long], queryContext)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on a diff in pull request #36525: [SPARK-39166] Provide runtime error query context for binary arithmetic when WSCG is off
gengliangwang commented on code in PR #36525: URL: https://github.com/apache/spark/pull/36525#discussion_r871524691 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala: ## @@ -381,9 +389,11 @@ case class Subtract( case _: YearMonthIntervalType => MathUtils.subtractExact(input1.asInstanceOf[Int], input2.asInstanceOf[Int]) case _: IntegerType if failOnError => - MathUtils.subtractExact(input1.asInstanceOf[Int], input2.asInstanceOf[Int], origin.context) + MathUtils.subtractExact(input1.asInstanceOf[Int], input2.asInstanceOf[Int], +queryContext) case _: LongType if failOnError => - MathUtils.subtractExact(input1.asInstanceOf[Long], input2.asInstanceOf[Long], origin.context) + MathUtils.subtractExact(input1.asInstanceOf[Long], input2.asInstanceOf[Long], +queryContext) Review Comment: ```suggestion MathUtils.subtractExact(input1.asInstanceOf[Long], input2.asInstanceOf[Long], queryContext) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on a diff in pull request #36525: [SPARK-39166] Provide runtime error query context for binary arithmetic when WSCG is off
gengliangwang commented on code in PR #36525: URL: https://github.com/apache/spark/pull/36525#discussion_r871524308 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala: ## @@ -381,9 +389,11 @@ case class Subtract( case _: YearMonthIntervalType => MathUtils.subtractExact(input1.asInstanceOf[Int], input2.asInstanceOf[Int]) case _: IntegerType if failOnError => - MathUtils.subtractExact(input1.asInstanceOf[Int], input2.asInstanceOf[Int], origin.context) + MathUtils.subtractExact(input1.asInstanceOf[Int], input2.asInstanceOf[Int], +queryContext) Review Comment: ```suggestion MathUtils.subtractExact(input1.asInstanceOf[Int], input2.asInstanceOf[Int], queryContext) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on a diff in pull request #36525: [SPARK-39166] Provide runtime error query context for binary arithmetic when WSCG is off
gengliangwang commented on code in PR #36525:
URL: https://github.com/apache/spark/pull/36525#discussion_r871497091
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala:
##
@@ -588,6 +588,28 @@ abstract class UnaryExpression extends Expression with
UnaryLike[Expression] {
}
}
+/**
+ * An expression with SQL query context. The context string can be serialized
from the Driver
+ * to executors. It will also be kept after rule transforms.
+ */
+trait SupportQueryContext extends Expression with Serializable {
+ protected var queryContext: String = initQueryContext()
+
+ def initQueryContext(): String
+
+ // Note: Even though query contexts are serialized to executors, it will be
regenerated from an
+ // empty "Origin" during rule transforms since "Origin"s are not
serialized to executors
+ // for better performance. Thus, we need to copy the original query
context during
+ // transforms. The query context string is considered as a "tag" on
the expression here.
+ override def copyTagsFrom(other: Expression): Unit = {
Review Comment:
@cloud-fan If you find this weird, we can consider changing the method name
as `copyTagsAndContextFrom`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
