This is an automated email from the ASF dual-hosted git repository.
mchades pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new eff1373938 [#11468] fix(lance): remove lance-core from lance-common
runtime deps to reduce binary size (#11477)
eff1373938 is described below
commit eff137393817f62cf44b6b84a55d5519024e40ae
Author: Qi Yu <[email protected]>
AuthorDate: Mon Jun 8 15:31:10 2026 +0800
[#11468] fix(lance): remove lance-core from lance-common runtime deps to
reduce binary size (#11477)
### What changes were proposed in this pull request?
`lance-common/build.gradle.kts` had `implementation(libs.lance)` which
pulled `lance-core-6.0.0.jar` (a fat jar bundling three-platform native
JNI libraries) into every downstream module's runtime classpath,
including `lance-rest-server`.
This PR:
- Removes `implementation(libs.lance)` from `lance-common`
- Adds `implementation(libs.arrow.vector)` (the only Arrow dependency
actually used by `lance-common`'s source code — `ArrowUtils` and
`LanceDataTypeConverter`)
- Registers `arrow-vector` in the version catalog at version `18.3.0`
(the version previously provided transitively by `lance-core`)
Impact on `lance-rest-server` runtime classpath:
| Artifact | Before | After |
|----------|--------|-------|
| `lance-core-6.0.0.jar` (fat jar, 3-platform native libs) | 164 MB |
removed |
| `arrow-dataset-18.3.0.jar` (transitive of lance-core) | 74 MB |
removed |
| `arrow-vector-18.3.0.jar` | — | +2.1 MB |
| **Net reduction** | | **≈ −236 MB** |
### Why are the changes needed?
`lance-core-6.0.0.jar` is a fat jar that embeds three platform native
libraries (`linux-x86-64`, `linux-aarch64`, `darwin-aarch64`), each ~150
MB unstripped. The `lance-rest-server` is a pure REST layer (Jersey +
Gravitino backend); it never calls any `org.lance.*` JNI classes
directly. The only reason `lance-core` was on its classpath was the
unnecessary `implementation(libs.lance)` in `lance-common`.
Fix: #11468
### Does this PR introduce _any_ user-facing change?
No API changes. The `gravitino-lance-rest-server` distribution package
will be significantly smaller (~236 MB reduction in the lib directory).
### How was this patch tested?
- `./gradlew :lance:lance-common:compileJava
:lance:lance-rest-server:compileJava` — compiles clean
- `./gradlew :lance:lance-common:test :lance:lance-rest-server:test
-PskipITs` — unit tests pass
- Verified `lance-core` and `arrow-dataset` are no longer in
`:lance:lance-rest-server` runtimeClasspath; `arrow-vector` is present
---------
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
---
gradle/libs.versions.toml | 2 ++
lance/lance-common/build.gradle.kts | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml
index 233da7623d..0d9023129c 100644
--- a/gradle/libs.versions.toml
+++ b/gradle/libs.versions.toml
@@ -30,6 +30,7 @@ guava = "32.1.3-jre"
lombok = "1.18.20"
slf4j = "2.0.16"
log4j = "2.25.4"
+arrow = "18.3.0"
lance = "6.0.0"
lance-namespace = "0.7.5"
delta-kernel = "3.3.0"
@@ -196,6 +197,7 @@ log4j-api = { group = "org.apache.logging.log4j", name =
"log4j-api", version.re
log4j-core = { group = "org.apache.logging.log4j", name = "log4j-core",
version.ref = "log4j" }
log4j-12-api = { group = "org.apache.logging.log4j", name = "log4j-1.2-api",
version.ref = "log4j" }
log4j-layout-template-json = { group = "org.apache.logging.log4j", name =
"log4j-layout-template-json", version.ref = "log4j" }
+arrow-vector = { group = "org.apache.arrow", name = "arrow-vector",
version.ref = "arrow" }
lance = { group = "org.lance", name = "lance-core", version.ref = "lance" }
lance-namespace-core = { group = "org.lance", name = "lance-namespace-core",
version.ref = "lance-namespace" }
delta-kernel = { group = "io.delta", name = "delta-kernel-api", version.ref =
"delta-kernel" }
diff --git a/lance/lance-common/build.gradle.kts
b/lance/lance-common/build.gradle.kts
index 886fdbccd1..618dc6a733 100644
--- a/lance/lance-common/build.gradle.kts
+++ b/lance/lance-common/build.gradle.kts
@@ -41,7 +41,7 @@ dependencies {
implementation(libs.jackson.datatype.jdk8)
implementation(libs.jackson.datatype.jsr310)
implementation(libs.jackson.jaxrs.json.provider)
- implementation(libs.lance)
+ implementation(libs.arrow.vector)
implementation(libs.lance.namespace.core) {
exclude(group = "org.lance", module = "lance-core")
exclude(group = "com.google.guava", module = "guava") // provided by
gravitino
