Github user kmanamcheri commented on a diff in the pull request:
https://github.com/apache/spark/pull/22614#discussion_r223429117
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientSuite.scala
---
@@ -79,12 +82,30 @@ class HiveClientSuite(version: String)
client = init(true)
}
- test(s"getPartitionsByFilter returns all partitions when
$tryDirectSqlKey=false") {
- val client = init(false)
- val filteredPartitions =
client.getPartitionsByFilter(client.getTable("default", "test"),
- Seq(attr("ds") === 20170101))
+ test(s"getPartitionsByFilter returns all partitions when
$partPruningFallbackKey=true") {
+
withSQLConf(SQLConf.HIVE_METASTORE_PARTITION_PRUNING_FALLBACK_ENABLED.key ->
"true",
+ SQLConf.HIVE_METASTORE_PARTITION_PRUNING.key -> "true") {
+ val client = init(false)
+ // tryDirectSql = false and a non-string partition filter will
always fail. This condition
+ // is used to test if the fallback works
+ val filteredPartitions =
client.getPartitionsByFilter(client.getTable("default", "test"),
+ Seq(attr("ds") === 20170101))
- assert(filteredPartitions.size == testPartitionCount)
+ assert(filteredPartitions.size == testPartitionCount)
+ }
+ }
+
+ test(s"getPartitionsByFilter should throw an exception if
$partPruningFallbackKey=false") {
--- End diff --
Hmm.. The behavior does not depend on the value of tryDirectSqlKey though.
It solely only depends on if pruning is enabled and if pruning.fallback is
enabled.
The reason we set tryDirectSqlKey to false is to generate a consistent way
for pruning to fail. Only setting tryDirectSql to false does not throw an
exception.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
