Wechar created SPARK-46203:
------------------------------
Summary: Support get partitions by names in partition filter
Key: SPARK-46203
URL: https://issues.apache.org/jira/browse/SPARK-46203
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.5.0
Reporter: Wechar
{{Hive#getPartitionsByFilter}} api is low-performance api and would cause Hive
MetaStore backend DBS suffer heavy load if there are many calls to tables
containing many partitions.
In this path, we introduce a new switch that enable filtering partitions in
Spark side, and then get target partitions by the high performance API
{{{}Hive#getPartitionsByNames}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
