[
https://issues.apache.org/jira/browse/SPARK-36137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-36137:
------------------------------------
Assignee: Apache Spark
> HiveShim always fallback to getAllPartitionsOf regardless of whether
> directSQL is enabled in remote HMS
> -------------------------------------------------------------------------------------------------------
>
> Key: SPARK-36137
> URL: https://issues.apache.org/jira/browse/SPARK-36137
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: Chao Sun
> Assignee: Apache Spark
> Priority: Major
>
> At the moment {{getPartitionsByFilter}} in Hive shim only fallback to use
> {{getAllPartitionsOf}} when {{hive.metastore.try.direct.sql}} is enabled in
> the remote HMS. However, in certain cases the remote HMS will fallback to use
> ORM (which only support string type for partition columns) to query the
> underlying RDBMS even if this config is set to true, and Spark will not be
> able to recover from the error and will just fail the query.
> For instance, we encountered this bug HIVE-21497 in HMS running Hive 3.1.2,
> and Spark was not able to pushdown filter for {{date}} column.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
