cloud-fan commented on a change in pull request #35207:
URL: https://github.com/apache/spark/pull/35207#discussion_r785699720
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala
##########
@@ -50,6 +50,8 @@ trait InvokeLike extends Expression with NonSQLExpression
with ImplicitCastInput
def propagateNull: Boolean
+ override def foldable: Boolean =
+ children.forall(_.foldable) && !dataType.isInstanceOf[ObjectType]
Review comment:
Good catch and it's a bug we must fix first. It turns out that we never
respect `BoundFunction.isDeterministic` which can cause many issues when doing
project collpase, filter pushdown, etc. and leads to wrong query result.
I think we need to add a `deterministic` flag into `InvokeLike` and set it
when use it to call DS v2 funciton. @sunchao can you help to fix this issue?
also cc @huaxingao as ds v2 function was released in 3.2.0 and we might want
to fix it in 3.2.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
