WangGuangxin commented on code in PR #5626:
URL: https://github.com/apache/incubator-gluten/pull/5626#discussion_r1635597416
##########
gluten-core/src/main/scala/org/apache/gluten/utils/PullOutProjectHelper.scala:
##########
@@ -143,8 +144,39 @@ trait PullOutProjectHelper {
ae.copy(aggregateFunction = newAggFunc, filter = newFilter)
}
+ private def needPreComputeRangeFrameBoundary(bound: Expression): Boolean = {
+ bound match {
+ case _: PreComputeRangeFrameBound => false
+ case _ if !bound.foldable => false
+ case _ => true
+ }
+ }
+
+ private def preComputeRangeFrameBoundary(
+ bound: Expression,
+ orderSpec: SortOrder,
+ expressionMap: mutable.HashMap[Expression, NamedExpression]): Expression
= {
+ bound match {
+ case _: PreComputeRangeFrameBound => bound
+ case _ if !bound.foldable => bound
+ case _ if bound.foldable =>
+ val a = expressionMap
+ .getOrElseUpdate(
+ bound.canonicalized,
+ Alias(Add(orderSpec.child, bound), generatePreAliasName)())
Review Comment:
Sorry for late response, all UT passed now, the failed one seems unrelated.
Please help review again @PHILO-HE @ulysses-you
The reason why it was failed is because sort column `l_discount` is a
decimal, and when we do `Add(sort_col, offset)`, the precision of decimal type
will change, so that the data type check failed in Spark
(https://github.com/apache/spark/blob/82a84ede6a47232fe3af86672ceea97f703b3e8a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala#L83)
In fact, we only support `ByteType | ShortType | IntegerType | LongType |
DateType` by design, which is checked in
https://github.com/apache/incubator-gluten/pull/5626/files#diff-68925b6a5ee67a942a976971ae6727a45a0b166d200d8d983454169e7c3bd1f6R316,
but even when WindowExec fallback by this check, the `PullOutPreProjection`
rule will still be executed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
