This is an automated email from the ASF dual-hosted git repository.
jshao pushed a commit to branch branch-lance-namepspace-dev
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/branch-lance-namepspace-dev by
this push:
new f6d49d95e5 [#8891] feat(lance-rest-server): Add standalone and
auxiliary mode support for Lance REST service (#8903)
f6d49d95e5 is described below
commit f6d49d95e58947ec27fef0ea5bc14850637421ee
Author: Junda Yang <[email protected]>
AuthorDate: Thu Oct 23 23:54:43 2025 -0700
[#8891] feat(lance-rest-server): Add standalone and auxiliary mode support
for Lance REST service (#8903)
### What changes were proposed in this pull request?
1. Created service discovery file
(META-INF/services/org.apache.gravitino.auxiliary.GravitinoAuxiliaryService)
2. Added unit tests (TestLanceConfig.java)
### Why are the changes needed?
Fix: #8891
To enable flexible deployment of Lance REST service:
Standalone mode (independent JVM):
- Lance needs to connect to Gravitino server via HTTP to access
catalog/schema/table metadata
- GRAVITINO_URI and GRAVITINO_METALAKE configs specify the connection
parameters
- Use case: Deploy Lance REST separately for isolation or scaling
Auxiliary mode (embedded in Gravitino server):
- Lance runs in the same JVM as Gravitino, enabling direct memory access
(zero overhead)
- META-INF/services file enables automatic service discovery via Java
ServiceLoader
- Use case: Simplified deployment with shared resources and unified
monitoring
### Does this PR introduce _any_ user-facing change?
Yes - New configuration properties
For Standalone Mode:
```
# Required: Gravitino server URI
gravitino.lance-rest.gravitino-uri=http://gravitino-server:8090
# Required: Metalake name
gravitino.lance-rest.gravitino-metalake=production
```
For Auxiliary Mode:
```
# Enable Lance REST as auxiliary service
gravitino.auxService.names=iceberg-rest,lance-rest
# Classpath for Lance REST JAR
gravitino.lance-rest.classpath=lance/lance-rest-server/build/libs
```
Deployment modes supported:
Standalone: Lance REST runs independently, connects to Gravitino via
HTTP
Auxiliary: Lance REST runs embedded in Gravitino, uses direct memory
access
No breaking changes - all changes are additive and backward compatible.
### How was this patch tested?
1. unit test
2. manual test
**Standalone mode test:**
```
junda@java(junda.devpod-us-or) ~/gravitino/distribution/package/bin
% ./gravitino.sh start
Gravitino Server is running[PID:3907790]
junda@java(junda.devpod-us-or) ~
% curl http://localhost:8090/api/version
{"code":0,"version":{"version":"1.1.0-SNAPSHOT","compileDate":"23/10/2025
21:03:17","gitCommit":"35bd819975badea2493b4dbd9d0de68425229945"}}%
junda@java(junda.devpod-us-or) ~/gravitino/distribution/package/bin
% ./gravitino-lance-rest-server.sh start
GravitinoLanceRESTServer is running[PID:3930096]
junda@java(junda.devpod-us-or) ~
% curl "http://localhost:9101/lance/v1/namespace/root/list";
{"id":"root","delimiter":"$","namespaces":["default"]}%
```
**Auxiliary mode test:**
```
junda@java(junda.devpod-us-or) ~/gravitino/distribution/package
% cd /home/user/gravitino && cat >>
distribution/package/conf/gravitino.conf << 'EOF'
# ===== Lance REST Auxiliary Service =====
gravitino.auxService.names = iceberg-rest,lance-rest
gravitino.lance-rest.classpath =
/home/user/gravitino/lance/lance-rest-server/build/libs
gravitino.lance-rest.catalog-name = lance_catalog
EOF
junda@java(junda.devpod-us-or) ~/gravitino/distribution/package
% ./bin/gravitino.sh start
Gravitino Server is running[PID:274396]
junda@java(junda.devpod-us-or) ~/gravitino/distribution/package
% tail -50 logs/gravitino-server.log | grep -i lance | grep -i started
2025-10-23 21:34:41.495 INFO [main]
[org.apache.gravitino.server.web.JettyServer.start(JettyServer.java:202)] -
lance-rest web server started on host 0.0.0.0 port 9101.
2025-10-23 21:34:41.495 INFO [main]
[org.apache.gravitino.lance.LanceRESTService.serviceStart(LanceRESTService.java:77)]
- Lance REST service started
```
---
lance/lance-common/build.gradle.kts | 1 +
.../gravitino/lance/common/config/LanceConfig.java | 8 ++
.../lance/common/config/TestLanceConfig.java | 114 +++++++++++++++++++++
...e.gravitino.auxiliary.GravitinoAuxiliaryService | 19 ++++
4 files changed, 142 insertions(+)
diff --git a/lance/lance-common/build.gradle.kts
b/lance/lance-common/build.gradle.kts
index 4e91dd6c5c..27057950f3 100644
--- a/lance/lance-common/build.gradle.kts
+++ b/lance/lance-common/build.gradle.kts
@@ -42,6 +42,7 @@ dependencies {
implementation(libs.lance.namespace.core)
implementation(libs.slf4j.api)
+ testImplementation(project(":server-common"))
testImplementation(libs.junit.jupiter.api)
testRuntimeOnly(libs.junit.jupiter.engine)
}
diff --git
a/lance/lance-common/src/main/java/org/apache/gravitino/lance/common/config/LanceConfig.java
b/lance/lance-common/src/main/java/org/apache/gravitino/lance/common/config/LanceConfig.java
index dfe863953e..b6614c87ee 100644
---
a/lance/lance-common/src/main/java/org/apache/gravitino/lance/common/config/LanceConfig.java
+++
b/lance/lance-common/src/main/java/org/apache/gravitino/lance/common/config/LanceConfig.java
@@ -80,6 +80,14 @@ public class LanceConfig extends Config implements
OverwriteDefaultConfig {
return get(CATALOG_NAME);
}
+ public String getNamespaceUri() {
+ return get(NAMESPACE_URI);
+ }
+
+ public String getGravitinoMetalake() {
+ return get(METALAKE_NAME);
+ }
+
@Override
public Map<String, String> getOverwriteDefaultConfig() {
return ImmutableMap.of(
diff --git
a/lance/lance-common/src/test/java/org/apache/gravitino/lance/common/config/TestLanceConfig.java
b/lance/lance-common/src/test/java/org/apache/gravitino/lance/common/config/TestLanceConfig.java
new file mode 100644
index 0000000000..176634f309
--- /dev/null
+++
b/lance/lance-common/src/test/java/org/apache/gravitino/lance/common/config/TestLanceConfig.java
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.lance.common.config;
+
+import com.google.common.collect.ImmutableMap;
+import java.util.Map;
+import org.apache.gravitino.server.web.JettyServerConfig;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+public class TestLanceConfig {
+ @Test
+ public void testLoadLanceConfig() {
+ Map<String, String> properties =
+ ImmutableMap.of("gravitino.lance-rest.catalog-name", "test_catalog");
+
+ LanceConfig lanceConfig = new LanceConfig();
+ lanceConfig.loadFromMap(properties, k ->
k.startsWith("gravitino.lance-rest."));
+ Assertions.assertEquals("test_catalog", lanceConfig.getCatalogName());
+
+ LanceConfig lanceConfig2 = new LanceConfig(properties);
+ Assertions.assertEquals("test_catalog", lanceConfig2.getCatalogName());
+ }
+
+ @Test
+ public void testDefaultCatalogName() {
+ // Test default catalog name when not specified
+ Map<String, String> properties = ImmutableMap.of();
+ LanceConfig lanceConfig = new LanceConfig(properties);
+ Assertions.assertEquals("default", lanceConfig.getCatalogName());
+ }
+
+ @Test
+ public void testLanceHttpPort() {
+ Map<String, String> properties = ImmutableMap.of();
+ LanceConfig lanceConfig = new LanceConfig(properties);
+ JettyServerConfig jettyServerConfig =
JettyServerConfig.fromConfig(lanceConfig);
+ Assertions.assertEquals(
+ LanceConfig.DEFAULT_LANCE_REST_SERVICE_HTTP_PORT,
jettyServerConfig.getHttpPort());
+ Assertions.assertEquals(
+ LanceConfig.DEFAULT_LANCE_REST_SERVICE_HTTPS_PORT,
jettyServerConfig.getHttpsPort());
+
+ properties =
+ ImmutableMap.of(
+ JettyServerConfig.WEBSERVER_HTTP_PORT.getKey(),
+ "9101",
+ JettyServerConfig.WEBSERVER_HTTPS_PORT.getKey(),
+ "9533");
+ lanceConfig = new LanceConfig(properties);
+ jettyServerConfig = JettyServerConfig.fromConfig(lanceConfig);
+ Assertions.assertEquals(9101, jettyServerConfig.getHttpPort());
+ Assertions.assertEquals(9533, jettyServerConfig.getHttpsPort());
+ }
+
+ @Test
+ public void testGravitinoUriAndMetalake() {
+ // Test default values
+ Map<String, String> properties = ImmutableMap.of();
+ LanceConfig lanceConfig = new LanceConfig(properties);
+ Assertions.assertEquals("http://localhost:8090";,
lanceConfig.getNamespaceUri());
+ Assertions.assertNull(lanceConfig.getGravitinoMetalake()); // No default,
must be configured
+
+ // Test custom values
+ properties =
+ ImmutableMap.of(
+ LanceConfig.NAMESPACE_URI.getKey(),
+ "http://gravitino-server:8090";,
+ LanceConfig.METALAKE_NAME.getKey(),
+ "production");
+ lanceConfig = new LanceConfig(properties);
+ Assertions.assertEquals("http://gravitino-server:8090";,
lanceConfig.getNamespaceUri());
+ Assertions.assertEquals("production", lanceConfig.getGravitinoMetalake());
+ }
+
+ @Test
+ public void testCompleteConfiguration() {
+ // Test all configurations together for auxiliary mode
+ Map<String, String> properties =
+ ImmutableMap.<String, String>builder()
+ .put(LanceConfig.CATALOG_NAME.getKey(), "lance_catalog")
+ .put(LanceConfig.NAMESPACE_URI.getKey(),
"http://gravitino-prod:8090";)
+ .put(LanceConfig.METALAKE_NAME.getKey(), "production")
+ .put(LanceConfig.NAMESPACE_BACKEND.getKey(), "gravitino")
+ .put(JettyServerConfig.WEBSERVER_HTTP_PORT.getKey(), "9101")
+ .build();
+
+ LanceConfig lanceConfig = new LanceConfig(properties);
+
+ // Verify all config values
+ Assertions.assertEquals("lance_catalog", lanceConfig.getCatalogName());
+ Assertions.assertEquals("http://gravitino-prod:8090";,
lanceConfig.getNamespaceUri());
+ Assertions.assertEquals("production", lanceConfig.getGravitinoMetalake());
+
+ JettyServerConfig jettyConfig = JettyServerConfig.fromConfig(lanceConfig);
+ Assertions.assertEquals(9101, jettyConfig.getHttpPort());
+ }
+}
diff --git
a/lance/lance-rest-server/src/main/resources/META-INF/services/org.apache.gravitino.auxiliary.GravitinoAuxiliaryService
b/lance/lance-rest-server/src/main/resources/META-INF/services/org.apache.gravitino.auxiliary.GravitinoAuxiliaryService
new file mode 100644
index 0000000000..97c3f356df
--- /dev/null
+++
b/lance/lance-rest-server/src/main/resources/META-INF/services/org.apache.gravitino.auxiliary.GravitinoAuxiliaryService
@@ -0,0 +1,19 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+org.apache.gravitino.lance.LanceRESTService
