cloud-fan commented on a change in pull request #31646:
URL: https://github.com/apache/spark/pull/31646#discussion_r591439100
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -863,7 +863,9 @@ object SQLConf {
.doc("The threshold of set size for InSet predicate when pruning
partitions through Hive " +
"Metastore. When the set size exceeds the threshold, we rewrite the
InSet predicate " +
"to be greater than or equal to the minimum value in set and less than
or equal to the " +
- "maximum value in set. Larger values may cause Hive Metastore stack
overflow.")
+ "maximum value in set. Larger values may cause Hive Metastore stack
overflow. But for " +
+ "the predicate of Not InSet which values exceeds the threshold, we
won't push it to " +
Review comment:
`But for InSet inside Not with values exceeding ...`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
