dongjoon-hyun commented on a change in pull request #32583:
URL: https://github.com/apache/spark/pull/32583#discussion_r735204796
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -1008,6 +1008,16 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val HIVE_METASTORE_PARTITION_PRUNING_FAST_FALLBACK =
+ buildConf("spark.sql.hive.metastorePartitionPruningFastFallback")
+ .doc("When this config is enabled, if the predicates are not supported
by Hive or Spark " +
+ "does fallback due to encountering MetaException from the metastore, "
+
+ "Spark will instead prune partitions by getting the partition names
first " +
+ "and then evaluate the filter expressions on the client side.")
Review comment:
Is this correct for the predicates with `TimeZoneAwareExpression`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
