cloud-fan commented on code in PR #47313:
URL: https://github.com/apache/spark/pull/47313#discussion_r1675480931
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala:
##########
@@ -315,7 +315,11 @@ private[hive] trait HiveStrategies {
// hive table scan operator to be used for partition pruning.
val partitionKeyIds = AttributeSet(relation.partitionCols)
val normalizedFilters = DataSourceStrategy.normalizeExprs(
- filters.filter(_.deterministic), relation.output)
+ filters.filter { f =>
+ f.deterministic &&
+ !SubqueryExpression.hasSubquery(f) &&
+ !f.exists(_.isInstanceOf[PythonUDF])
Review Comment:
ditto, I think at planning phase we don't need to care about PythonUDF
anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
