This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 3fc46b038532 [SPARK-60935][SQL] Remove unused `Shim.getAllPartitions`
from `HiveShim`
3fc46b038532 is described below
commit 3fc46b038532b221c4ff4f56af6dc2c5c7925492
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu Nov 5 11:20:07 2026 -0800
[SPARK-60935][SQL] Remove unused `Shim.getAllPartitions` from `HiveShim`
### What changes were proposed in this pull request?
This PR removes the unused `Shim.getAllPartitions(hive, table)` abstract
method and its only implementation `Shim_v2_0.getAllPartitions` from
`HiveShim.scala`.
### Why are the changes needed?
Nothing calls `Shim.getAllPartitions`. `Shim` is a `private[client] sealed
abstract class`, so it cannot be used outside the
`org.apache.spark.sql.hive.client` package, and no test uses it either.
`Shim_v2_0.getPartitionsByFilter` falls back to
`hive.getAllPartitionsOf(table)` directly rather than through this method.
History of the method:
- SPARK-22214 (08b204fd2c7, 2017-10-06, first released in
[v2.3.0](https://github.com/apache/spark/releases/tag/v2.3.0), tagged
2018-02-22) changed `HiveClientImpl.getPartitions()` to call
`client.getPartitions` instead of `shim.getAllPartitions` when the partition
spec is empty. That removed the last caller outside the shim.
- After that, only `Shim_v0_12.getPartitionsByFilter` used the method,
because Hive 0.12 cannot push the filter down to the metastore (HIVE-4888) and
had to fetch all partitions.
- SPARK-45358 (122e9f8a746, 2023-09-30, first released in
[v4.0.0](https://github.com/apache/spark/releases/tag/v4.0.0), tagged
2025-05-19) removed the Hive shims older than 2.0.0. It moved the
implementation to `Shim_v2_0` but dropped that only caller, so the method has
had no callers since.
### Does this PR introduce _any_ user-facing change?
No. The removed method is internal (`private[client]`) and has no callers.
### How was this patch tested?
Pass the CIs. Also verified locally:
```
build/sbt -Phive hive/Test/compile
build/sbt -Phive hive/scalastyle
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Sonnet 5.5
Closes #60136 from dongjoon-hyun/SPARK-60935.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala | 7 -------
1 file changed, 7 deletions(-)
diff --git
a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
index 445cc80e625c..c4d83bd563ec 100644
--- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
+++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
@@ -99,8 +99,6 @@ private[client] sealed abstract class Shim {
def setDataLocation(table: Table, loc: String): Unit
- def getAllPartitions(hive: Hive, table: Table): Seq[Partition]
-
def getPartitionsByFilter(
hive: Hive,
table: Table,
@@ -336,11 +334,6 @@ private[client] class Shim_v2_0 extends Shim with Logging {
hive.createPartitions(addPartitionDesc)
}
- override def getAllPartitions(hive: Hive, table: Table): Seq[Partition] = {
- recordHiveCall()
- hive.getAllPartitionsOf(table).asScala.toSeq
- }
-
override def getPartitionsByFilter(
hive: Hive,
table: Table,
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]