aokolnychyi commented on code in PR #57357:
URL: https://github.com/apache/spark/pull/57357#discussion_r3626348125
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -7864,6 +7864,20 @@ object SQLConf {
.createWithDefault(false)
}
+ val LEGACY_ALLOW_NON_DETERMINISTIC_V2_FILTER_PUSHDOWN =
Review Comment:
What if the filter is used in multiple places and then it gets pushed in one
connector scan and not the other? How do we ensure the pushed filter behaves
exactly the same as the one we keep on the Spark side? Is it even safe to push
down undeterministic conditions into any connectors?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
