This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 7420db7 [SPARK-57845] Exclude `org.roaringbitmap` dependency
7420db7 is described below
commit 7420db7ac1043971a97d836809d1490516299cf5
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Jul 1 08:16:04 2026 -0700
[SPARK-57845] Exclude `org.roaringbitmap` dependency
### What changes were proposed in this pull request?
This PR excludes the transitive `org.roaringbitmap:RoaringBitmap`
dependency from the shaded operator jar by adding `exclude group:
"org.roaringbitmap"` to the Spark dependency declarations in the
`spark-operator` and `spark-submission-worker` modules.
### Why are the changes needed?
`RoaringBitmap` is pulled in transitively via
`org.apache.spark:spark-network-common`, where Spark uses it for shuffle map
status tracking (`HighlyCompressedMapStatus`). The operator only builds
driver/submission specs and does not run the shuffle/executor runtime, so this
library is unused. It belongs to the same category of shuffle/storage runtime
libraries that are already excluded (`org.lz4`, `org.rocksdb`,
`org.fusesource.leveldbjni`, `org.xerial.snappy`, `com.github.luben`). Rem [...]
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.8
Closes #728 from dongjoon-hyun/SPARK-57845.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
spark-operator/build.gradle | 1 +
spark-submission-worker/build.gradle | 1 +
2 files changed, 2 insertions(+)
diff --git a/spark-operator/build.gradle b/spark-operator/build.gradle
index 760a66b..1759197 100644
--- a/spark-operator/build.gradle
+++ b/spark-operator/build.gradle
@@ -49,6 +49,7 @@ dependencies {
exclude group: "org.checkerframework"
exclude group: "org.fusesource.leveldbjni"
exclude group: "org.lz4"
+ exclude group: "org.roaringbitmap"
exclude group: "org.rocksdb"
exclude group: "org.slf4j"
exclude group: "org.xerial.snappy"
diff --git a/spark-submission-worker/build.gradle
b/spark-submission-worker/build.gradle
index e3e6358..4907e89 100644
--- a/spark-submission-worker/build.gradle
+++ b/spark-submission-worker/build.gradle
@@ -38,6 +38,7 @@ dependencies {
exclude group: "org.checkerframework"
exclude group: "org.fusesource.leveldbjni"
exclude group: "org.lz4"
+ exclude group: "org.roaringbitmap"
exclude group: "org.rocksdb"
exclude group: "org.slf4j"
exclude group: "org.xerial.snappy"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
