This is an automated email from the ASF dual-hosted git repository.
FANNG1 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new a1ba08b2e7 [#11900] feat(iceberg): Support HDFS Kerberos auth for IRC
JDBC catalog (#11901)
a1ba08b2e7 is described below
commit a1ba08b2e7bd6896e4508607275be40729b7fda0
Author: MaSai <[email protected]>
AuthorDate: Thu Jul 9 08:54:14 2026 +0800
[#11900] feat(iceberg): Support HDFS Kerberos auth for IRC JDBC catalog
(#11901)
### What changes were proposed in this pull request?
This PR adds Kerberos authentication support for the Iceberg REST
Catalog (IRC) when using a **JDBC** catalog backend with an HDFS
warehouse.
- Add `ClosableJdbcCatalog` wrapping Iceberg `JdbcCatalog`, implementing
`SupportsKerberos` and `Closeable` (aligned with `ClosableHiveCatalog`).
- Update `IcebergCatalogUtil.loadJdbcCatalog` to use
`ClosableJdbcCatalog`, configure HDFS security settings, and initialize
Kerberos login.
- Make `JdbcCatalogWithMetadataLocationSupport` extend
`ClosableJdbcCatalog`.
- Add unit tests (`TestClosableJdbcCatalog`) and integration test
(`IcebergRestKerberosJdbcCatalogIT`).
- Update IRC documentation and
`gravitino-iceberg-rest-server.conf.template` with Kerberos
configuration examples.
Fix: #11900
### Why are the changes needed?
The Hive catalog backend already supports HDFS Kerberos via
`ClosableHiveCatalog`. JDBC-backed IRC deployments on Kerberized HDFS
clusters currently lack equivalent support, causing warehouse access
failures.
### Does this PR introduce _any_ user-facing change?
Yes. IRC with JDBC backend now supports Kerberos authentication for HDFS
warehouse access when configured:
```
gravitino.iceberg-rest.authentication.type=kerberos
gravitino.iceberg-rest.authentication.kerberos.principal=xxx@REALM
gravitino.iceberg-rest.authentication.kerberos.keytab-uri=file:///path/to/xxx.keytab
gravitino.iceberg-rest.hadoop.security.authentication=kerberos
```
JDBC metadata store authentication still uses `jdbc-user` and
`jdbc-password`.
### How was this patch tested?
- UT: `./gradlew :iceberg:iceberg-common:test --tests
"org.apache.gravitino.iceberg.common.TestClosableJdbcCatalog" --tests
"org.apache.gravitino.iceberg.common.utils.TestIcebergCatalogUtil"
-PskipITs`
- IT: `./gradlew :iceberg:iceberg-rest-server:test --tests
"org.apache.gravitino.iceberg.integration.test.IcebergRestKerberosJdbcCatalogIT"
-PskipDockerTests=false`
- `./gradlew :iceberg:iceberg-common:spotlessApply
:iceberg:iceberg-rest-server:spotlessApply`
---------
Co-authored-by: Cursor <[email protected]>
---
conf/gravitino-iceberg-rest-server.conf.template | 18 ++
docs/iceberg-rest-service.md | 68 +++++-
docs/lakehouse-iceberg-catalog.md | 26 +--
.../iceberg/common/ClosableHiveCatalog.java | 78 ++-----
.../iceberg/common/ClosableJdbcCatalog.java | 108 +++++++++
.../iceberg/common/utils/IcebergCatalogUtil.java | 18 +-
.../iceberg/common/utils/KerberosCatalogUtils.java | 175 ++++++++++++++
.../JdbcCatalogWithMetadataLocationSupport.java | 3 +-
.../iceberg/common/TestClosableJdbcCatalog.java | 134 +++++++++++
.../common/utils/TestIcebergCatalogUtil.java | 26 ++-
.../common/utils/TestKerberosCatalogUtils.java | 64 ++++++
.../integration/test/IcebergRESTServiceBaseIT.java | 9 +
.../integration/test/IcebergRESTServiceIT.java | 8 +-
.../test/IcebergRestKerberosHiveCatalogIT.java | 125 ++++------
...KerberosHiveWithUserImpersonationCatalogIT.java | 7 +-
.../test/IcebergRestKerberosJdbcCatalogIT.java | 133 +++++++++++
...erberosJdbcWithUserImpersonationCatalogIT.java} | 22 +-
.../test/IcebergRestKerberosTestEnv.java | 253 +++++++++++++++++++++
.../util/IcebergRESTServerManagerForDeploy.java | 93 +++++---
19 files changed, 1144 insertions(+), 224 deletions(-)
diff --git a/conf/gravitino-iceberg-rest-server.conf.template
b/conf/gravitino-iceberg-rest-server.conf.template
index 3aed55216a..62df320496 100644
--- a/conf/gravitino-iceberg-rest-server.conf.template
+++ b/conf/gravitino-iceberg-rest-server.conf.template
@@ -63,3 +63,21 @@ gravitino.iceberg-rest.warehouse = /tmp
# gravitino.iceberg-rest.s3-secret-access-key = xxx
# gravitino.iceberg-rest.s3-endpoint = http://192.168.215.4:9010
# gravitino.iceberg-rest.s3-region = xxx
+
+# THE CONFIGURATION EXAMPLE FOR JDBC CATALOG BACKEND WITH KERBEROS-SECURED
HDFS WAREHOUSE
+# JDBC username/password authenticates the metadata store; Kerberos
authenticates HDFS access.
+
+# gravitino.iceberg-rest.catalog-backend = jdbc
+# gravitino.iceberg-rest.jdbc-driver = org.postgresql.Driver
+# gravitino.iceberg-rest.uri = jdbc:postgresql://127.0.0.1:5432/iceberg
+# gravitino.iceberg-rest.jdbc-user = iceberg
+# gravitino.iceberg-rest.jdbc-password = secret
+# gravitino.iceberg-rest.jdbc-initialize = true
+# gravitino.iceberg-rest.warehouse =
hdfs://127.0.0.1:9000/user/hive/warehouse-jdbc
+# gravitino.iceberg-rest.authentication.type = kerberos
+# gravitino.iceberg-rest.authentication.kerberos.principal =
[email protected]
+# gravitino.iceberg-rest.authentication.kerberos.keytab-uri =
file:///etc/security/keytabs/iceberg.keytab
+# gravitino.iceberg-rest.hadoop.security.authentication = kerberos
+# gravitino.iceberg-rest.dfs.namenode.kerberos.principal =
hdfs/[email protected]
+#
+# For Hive catalog Kerberos configuration, see docs/iceberg-rest-service.md
(Backend Authentication).
diff --git a/docs/iceberg-rest-service.md b/docs/iceberg-rest-service.md
index 66900d8824..52e63bbf65 100644
--- a/docs/iceberg-rest-service.md
+++ b/docs/iceberg-rest-service.md
@@ -163,6 +163,8 @@ The Gravitino Iceberg REST catalog service uses the memory
catalog backend by de
| `gravitino.iceberg-rest.warehouse` |
The warehouse directory of the Hive catalog, such as
`/user/hive/warehouse-hive/`.
| (none)
| Yes | 0.2.0 |
| `gravitino.iceberg-rest.catalog-backend-name` |
The catalog backend name passed to underlying Iceberg catalog backend. Catalog
name in JDBC backend is used to isolate namespace and tables. | `hive` for Hive
backend, `jdbc` for JDBC backend, `memory` for memory backend | No |
0.5.2 |
+For Kerberos-secured Hive Metastore and HDFS, see [Backend
Authentication](#backend-authentication).
+
#### JDBC Backend Configuration
| Configuration item | Description
| Default value
| Required | Since Version |
@@ -180,6 +182,8 @@ The Gravitino Iceberg REST catalog service uses the memory
catalog backend by de
If you have a JDBC Iceberg catalog prior, you must set `catalog-backend-name`
to keep consistent with your Jdbc Iceberg catalog name to operate the prior
namespace and tables.
+Use `gravitino.iceberg-rest.jdbc-user` and
`gravitino.iceberg-rest.jdbc-password` to authenticate the JDBC catalog
metadata store. When the warehouse is on Kerberos-secured HDFS, set
`gravitino.iceberg-rest.authentication.type` to `kerberos` and configure the
Kerberos properties in [Backend Authentication](#backend-authentication) to
access warehouse paths.
+
:::caution
Download the corresponding JDBC driver to the `iceberg-rest-server/libs`
directory.
If you are using multiple JDBC catalog backends, setting `jdbc-initialize` to
true may not take effect for RDBMS like `Mysql`, you should create Iceberg meta
tables explicitly.
@@ -428,18 +432,62 @@ Refer to [HTTPS
Configuration](./security/how-to-use-https.md#apache-iceberg-res
#### Backend Authentication
-For JDBC backend, you can use the `gravitino.iceberg-rest.jdbc-user` and
`gravitino.iceberg-rest.jdbc-password` to authenticate the JDBC connection. For
Hive backend, you can use the `gravitino.iceberg-rest.authentication.type` to
specify the authentication type, and use the
`gravitino.iceberg-rest.authentication.kerberos.principal` and
`gravitino.iceberg-rest.authentication.kerberos.keytab-uri` to authenticate the
Kerberos connection.
+For the JDBC catalog backend, use `gravitino.iceberg-rest.jdbc-user` and
`gravitino.iceberg-rest.jdbc-password` to authenticate the JDBC metadata store
connection. Use `gravitino.iceberg-rest.authentication.type` to specify how the
catalog backend accesses the warehouse storage. When the warehouse is on HDFS,
set it to `kerberos` or `simple`, and configure
`gravitino.iceberg-rest.authentication.kerberos.principal` and
`gravitino.iceberg-rest.authentication.kerberos.keytab-uri` for Kerber [...]
+
+For the Hive catalog backend, `gravitino.iceberg-rest.authentication.type`
controls both Hive Metastore and HDFS access. When using Kerberos, also
configure `gravitino.iceberg-rest.hive.metastore.sasl.enabled` and related Hive
Metastore Kerberos properties.
+
The detailed configuration items are as follows:
-| Configuration item |
Description
| Default value | Required
[...]
-|---------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
-| `gravitino.iceberg-rest.authentication.type` |
The type of authentication for Iceberg rest catalog backend. This configuration
only applicable for Hive backend, and only supports `Kerberos`, `simple`
currently. As for JDBC backend, only username/password authentication was
supported now. | `simple` | No
[...]
-| `gravitino.iceberg-rest.authentication.impersonation-enable` |
Whether to enable impersonation for the Iceberg catalog
| `false` | No
[...]
-| `gravitino.iceberg-rest.hive.metastore.sasl.enabled` |
Whether to enable SASL authentication protocol when connect to Kerberos Hive
metastore.
| `false` | No, This value should be true in most case(Some will
use SSL protocol, but it rather rare) if the value of
`gravitino.iceberg-rest.authentication.type [...]
-| `gravitino.iceberg-rest.authentication.kerberos.principal` |
The principal of the Kerberos authentication
| (none) | required if the value of
`gravitino.iceberg-rest.authentication.type` is Kerberos.
[...]
-| `gravitino.iceberg-rest.authentication.kerberos.keytab-uri` |
The URI of The keytab for the Kerberos authentication.
| (none) | required if the value of
`gravitino.iceberg-rest.authentication.type` is Kerberos.
[...]
-| `gravitino.iceberg-rest.authentication.kerberos.check-interval-sec` |
The check interval of Kerberos credential for Iceberg catalog.
| 60 | No
[...]
-| `gravitino.iceberg-rest.authentication.kerberos.keytab-fetch-timeout-sec` |
The fetch timeout of retrieving Kerberos keytab from
`authentication.kerberos.keytab-uri`.
| 60 | No
[...]
+| Configuration item
| Description
| Default value | Required
[...]
+|
----------------------------------------------------------------------------- |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| ----------------- |
------------------------------------------------------------------------------------------------------------------------------------------
[...]
+| `gravitino.iceberg-rest.authentication.type`
| The authentication type for HDFS warehouse access. Supports `kerberos` and
`simple` for Hive and JDBC catalog backends.
| `simple` | No
[...]
+| `gravitino.iceberg-rest.authentication.impersonation-enable`
| Whether to enable impersonation for the Iceberg catalog.
| `false` | No
[...]
+| `gravitino.iceberg-rest.hive.metastore.sasl.enabled`
| Whether to enable SASL when connecting to a Kerberos Hive Metastore.
| `false` | No. Set to true in most cases when
`gravitino.iceberg-rest.authentication.type` is `kerberos` (some deployments
use SSL instead, but that [...]
+| `gravitino.iceberg-rest.authentication.kerberos.principal`
| The principal for Kerberos authentication.
| (none) | Yes, if
`gravitino.iceberg-rest.authentication.type` is `kerberos`.
[...]
+| `gravitino.iceberg-rest.authentication.kerberos.keytab-uri`
| The URI of the keytab for Kerberos authentication.
| (none) | Yes, if
`gravitino.iceberg-rest.authentication.type` is `kerberos`.
[...]
+| `gravitino.iceberg-rest.authentication.kerberos.check-interval-sec`
| The check interval of Kerberos credential for the Iceberg catalog.
| 60 | No
[...]
+| `gravitino.iceberg-rest.authentication.kerberos.keytab-fetch-timeout-sec`
| The fetch timeout for retrieving Kerberos keytab from
`authentication.kerberos.keytab-uri`.
| 60 | No
[...]
+
+Kerberos applies to **HDFS warehouse** access.
+
+**Hive backend with Kerberos** — authenticates to both Hive Metastore and HDFS:
+
+```text
+gravitino.iceberg-rest.catalog-backend = hive
+gravitino.iceberg-rest.uri = thrift://127.0.0.1:9083
+gravitino.iceberg-rest.warehouse =
hdfs://127.0.0.1:9000/user/hive/warehouse-hive
+
+gravitino.iceberg-rest.authentication.type = kerberos
+gravitino.iceberg-rest.authentication.kerberos.principal = [email protected]
+gravitino.iceberg-rest.authentication.kerberos.keytab-uri =
file:///etc/security/keytabs/iceberg.keytab
+
+gravitino.iceberg-rest.hive.metastore.sasl.enabled = true
+gravitino.iceberg-rest.hive.metastore.kerberos.principal =
hive/[email protected]
+
+gravitino.iceberg-rest.hadoop.security.authentication = kerberos
+gravitino.iceberg-rest.dfs.namenode.kerberos.principal =
hdfs/[email protected]
+```
+
+**JDBC backend with Kerberos-secured HDFS warehouse** — JDBC credentials
authenticate the metadata store; Kerberos authenticates HDFS access only:
+
+```text
+gravitino.iceberg-rest.catalog-backend = jdbc
+gravitino.iceberg-rest.jdbc-driver = org.postgresql.Driver
+gravitino.iceberg-rest.uri = jdbc:postgresql://127.0.0.1:5432/iceberg
+gravitino.iceberg-rest.jdbc-user = iceberg
+gravitino.iceberg-rest.jdbc-password = secret
+gravitino.iceberg-rest.jdbc-initialize = true
+gravitino.iceberg-rest.warehouse =
hdfs://127.0.0.1:9000/user/hive/warehouse-jdbc
+
+gravitino.iceberg-rest.authentication.type = kerberos
+gravitino.iceberg-rest.authentication.kerberos.principal = [email protected]
+gravitino.iceberg-rest.authentication.kerberos.keytab-uri =
file:///etc/security/keytabs/iceberg.keytab
+
+gravitino.iceberg-rest.hadoop.security.authentication = kerberos
+gravitino.iceberg-rest.dfs.namenode.kerberos.principal =
hdfs/[email protected]
+```
+
+Replace `host.example.com` and `EXAMPLE.COM` with your Hive/HDFS host name and
Kerberos realm. Hive Metastore SASL properties are not required for the JDBC
backend.
#### Credential Vending
diff --git a/docs/lakehouse-iceberg-catalog.md
b/docs/lakehouse-iceberg-catalog.md
index fdc3eca7f5..0badff0441 100644
--- a/docs/lakehouse-iceberg-catalog.md
+++ b/docs/lakehouse-iceberg-catalog.md
@@ -38,7 +38,7 @@ Mixing Iceberg JARs from different versions on the client
classpath is not compa
- Supports DDL operations for Iceberg schemas and tables.
- Doesn't support snapshot or table management operations.
- Supports multi storage, including S3, GCS, ADLS, OSS and HDFS.
-- Supports Kerberos or simple authentication for Iceberg catalog with Hive
backend.
+- Supports Kerberos or simple authentication for warehouse storage access on
HDFS for Hive and JDBC catalog backends.
- Supports table metadata cache.
### Catalog Properties
@@ -60,7 +60,7 @@ If you are using the Gravitino with Spark, you can pass the
Spark Iceberg connec
#### JDBC Backend
-If you are using JDBC backend, you must provide properties like `jdbc-user`,
`jdbc-password` and `jdbc-driver`.
+If you are using JDBC backend, you must provide properties like `jdbc-user`,
`jdbc-password` and `jdbc-driver`. Use `jdbc-user` and `jdbc-password` for the
JDBC metadata store connection. When the warehouse is on Kerberos-secured HDFS,
set `authentication.type` to `kerberos` and configure the Kerberos properties
in [Catalog Backend Security](#catalog-backend-security).
| Property name | Description
| Default value | Required |
Since Version |
|-------------------|---------------------------------------------------------------------------------------------------------|---------------|----------|---------------|
@@ -228,17 +228,17 @@ Please set the `warehouse` parameter to
`{storage_prefix}://{bucket_name}/${pref
#### Catalog Backend Security
-Users can use the following properties to configure the security of the
catalog backend if needed. For example, if you are using a Kerberos Hive
catalog backend, you must set `authentication.type` to `Kerberos` and provide
`authentication.kerberos.principal` and `authentication.kerberos.keytab-uri`.
-
-| Property name | Description
| Default value |
Required
| Since Version |
-|----------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------|
-| `authentication.type` | The type of
authentication for Iceberg catalog backend. This configuration only applicable
for for Hive backend, and only supports `Kerberos`, `simple` currently. As for
JDBC backend, only username/password authentication was supported now. |
`simple` | No
| 0.6.0-incubating |
-| `authentication.impersonation-enable` | Whether to enable
impersonation for the Iceberg catalog
| `false`
| No
| 0.6.0-incubating |
-| `hive.metastore.sasl.enabled` | Whether to enable SASL
authentication protocol when connect to Kerberos Hive metastore. This is a raw
Hive configuration
| `false` |
No, This value should be true in most case(Some will use SSL protocol, but it
rather rare) if the value of `gravitino.iceberg-rest.authentication.type` is
Kerberos. | 0.6.0-incubating |
-| `authentication.kerberos.principal` | The principal of the
Kerberos authentication
| (none) |
required if the value of `authentication.type` is Kerberos.
| 0.6.0-incubating |
-| `authentication.kerberos.keytab-uri` | The URI of The keytab
for the Kerberos authentication.
| (none) |
required if the value of `authentication.type` is Kerberos.
| 0.6.0-incubating |
-| `authentication.kerberos.check-interval-sec` | The check interval of
Kerberos credential for Iceberg catalog.
| 60 | No
| 0.6.0-incubating |
-| `authentication.kerberos.keytab-fetch-timeout-sec` | The fetch timeout of
retrieving Kerberos keytab from `authentication.kerberos.keytab-uri`.
| 60 |
No
| 0.6.0-incubating |
+Users can use the following properties to configure warehouse storage security
when needed. For example, if you are using a Hive or JDBC catalog backend with
a Kerberos-secured HDFS warehouse, set `authentication.type` to `kerberos` and
provide `authentication.kerberos.principal` and
`authentication.kerberos.keytab-uri`. For JDBC backend, JDBC username/password
authentication for the metadata store is configured separately via `jdbc-user`
and `jdbc-password`.
+
+| Property name | Description
| Default
value | Required
[...]
+| ------------------------------------------------------ |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| ----------------- |
------------------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
+| `authentication.type` | The authentication
type for HDFS warehouse access. Supports `kerberos` and `simple` for Hive and
JDBC catalog backends.
|
`simple` | No
[...]
+| `authentication.impersonation-enable` | Whether to enable
impersonation for the Iceberg catalog.
|
`false` | No
[...]
+| `hive.metastore.sasl.enabled` | Whether to enable
SASL when connecting to a Kerberos Hive Metastore. This is a raw Hive
configuration.
| `false` | No. Set to true in most cases when `authentication.type`
is `kerberos` (some deployments use SSL instead, but that is rare).
[...]
+| `authentication.kerberos.principal` | The principal for
Kerberos authentication.
| (none)
| Yes, if `authentication.type` is `kerberos`.
[...]
+| `authentication.kerberos.keytab-uri` | The URI of the
keytab for Kerberos authentication.
|
(none) | Yes, if `authentication.type` is `kerberos`.
[...]
+| `authentication.kerberos.check-interval-sec` | The check interval
of Kerberos credential for the Iceberg catalog.
| 60
| No
[...]
+| `authentication.kerberos.keytab-fetch-timeout-sec` | The fetch timeout
for retrieving Kerberos keytab from `authentication.kerberos.keytab-uri`.
| 60
| No
[...]
#### Table Metadata Cache
diff --git
a/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/ClosableHiveCatalog.java
b/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/ClosableHiveCatalog.java
index 6fd8dcd7d3..b93eacd2ba 100644
---
a/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/ClosableHiveCatalog.java
+++
b/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/ClosableHiveCatalog.java
@@ -22,7 +22,6 @@ package org.apache.gravitino.iceberg.common;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.Lists;
import java.io.Closeable;
-import java.io.File;
import java.io.IOException;
import java.lang.reflect.Field;
import java.security.PrivilegedExceptionAction;
@@ -31,14 +30,12 @@ import java.util.Map;
import java.util.concurrent.ScheduledExecutorService;
import lombok.Getter;
import org.apache.commons.lang3.reflect.FieldUtils;
-import org.apache.gravitino.catalog.hadoop.auth.KerberosAuthUtils;
import org.apache.gravitino.catalog.hadoop.auth.KerberosClient;
import org.apache.gravitino.iceberg.common.authentication.AuthenticationConfig;
import org.apache.gravitino.iceberg.common.authentication.SupportsKerberos;
-import
org.apache.gravitino.iceberg.common.authentication.kerberos.KerberosConfig;
import
org.apache.gravitino.iceberg.common.utils.CaffeineSchedulerExtractorUtils;
import org.apache.gravitino.iceberg.common.utils.IcebergHiveCachedClientPool;
-import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.gravitino.iceberg.common.utils.KerberosCatalogUtils;
import org.apache.hadoop.hive.metastore.IMetaStoreClient;
import org.apache.hadoop.hive.thrift.DelegationTokenIdentifier;
import org.apache.hadoop.security.UserGroupInformation;
@@ -83,7 +80,7 @@ public class ClosableHiveCatalog extends HiveCatalog
implements Closeable, Suppo
AuthenticationConfig authenticationConfig = new
AuthenticationConfig(properties);
if (authenticationConfig.isKerberosAuth()) {
- this.kerberosClient = initKerberosClient();
+ this.kerberosClient =
KerberosCatalogUtils.initKerberosClient(properties, getConf(), name());
}
try {
@@ -95,13 +92,7 @@ public class ClosableHiveCatalog extends HiveCatalog
implements Closeable, Suppo
@Override
public void close() throws IOException {
- if (kerberosClient != null) {
- try {
- kerberosClient.close();
- } catch (Exception e) {
- LOGGER.warn("Failed to close KerberosClient", e);
- }
- }
+ KerberosCatalogUtils.closeKerberosClient(kerberosClient, LOGGER);
// Do clean up work here. We need a mechanism to close the HiveCatalog;
however, HiveCatalog
// doesn't implement the Closeable interface.
@@ -124,24 +115,19 @@ public class ClosableHiveCatalog extends HiveCatalog
implements Closeable, Suppo
@Override
public <R> R doKerberosOperations(Executable<R> executable) throws Throwable
{
- Map<String, String> properties = this.properties();
- AuthenticationConfig authenticationConfig = new
AuthenticationConfig(properties);
-
- final String finalPrincipalName;
- String proxyKerberosPrincipalName =
PrincipalUtils.getCurrentPrincipal().getName();
-
- if (!proxyKerberosPrincipalName.contains("@")) {
- finalPrincipalName =
- String.format("%s@%s", proxyKerberosPrincipalName,
kerberosClient.getRealm());
- } else {
- finalPrincipalName = proxyKerberosPrincipalName;
+ AuthenticationConfig authenticationConfig = new
AuthenticationConfig(this.properties());
+ if (!authenticationConfig.isKerberosAuth()) {
+ return executable.execute();
+ }
+ if (kerberosClient == null) {
+ throw new IllegalStateException(
+ "Kerberos is configured but KerberosClient is not initialized");
}
+ String finalPrincipalName =
KerberosCatalogUtils.resolveProxyPrincipal(kerberosClient);
UserGroupInformation realUser =
- authenticationConfig.isImpersonationEnabled()
- ? UserGroupInformation.createProxyUser(
- finalPrincipalName, kerberosClient.getLoginUser())
- : kerberosClient.getLoginUser();
+ KerberosCatalogUtils.createRealUser(
+ authenticationConfig, kerberosClient, finalPrincipalName);
try {
ClientPool<IMetaStoreClient, TException> newClientPool =
(ClientPool<IMetaStoreClient, TException>)
FieldUtils.readField(this, "clients", true);
@@ -166,18 +152,7 @@ public class ClosableHiveCatalog extends HiveCatalog
implements Closeable, Suppo
throw new RuntimeException(
"Failed to get delegation token for principal: " +
finalPrincipalName, e);
}
- return realUser.doAs(
- (PrivilegedExceptionAction<R>)
- () -> {
- try {
- return executable.execute();
- } catch (Throwable e) {
- if (RuntimeException.class.isAssignableFrom(e.getClass())) {
- throw (RuntimeException) e;
- }
- throw new RuntimeException("Failed to invoke method", e);
- }
- });
+ return KerberosCatalogUtils.executeAs(realUser, executable);
}
private ClientPool<IMetaStoreClient, TException> resetIcebergHiveClientPool()
@@ -272,29 +247,4 @@ public class ClosableHiveCatalog extends HiveCatalog
implements Closeable, Suppo
LOGGER.warn("Failed to close HiveCatalog internal client pool", e);
}
}
-
- private KerberosClient initKerberosClient() {
- try {
- KerberosConfig kerberosConfig = new KerberosConfig(this.properties());
- KerberosClient kerberosClient =
- KerberosClient.builder(kerberosConfig.getPrincipalName(),
this.getConf())
- .loginMode(KerberosAuthUtils.LoginMode.CURRENT_USER)
- .checkIntervalSec(kerberosConfig.getCheckIntervalSec())
- .build();
- // catalog_uuid always exists for Gravitino managed catalogs, `0` is
just a fallback value.
- String catalogUUID = properties().getOrDefault("catalog_uuid", "0");
- File keytabFile =
- new File(String.format(KerberosConfig.GRAVITINO_KEYTAB_FORMAT,
catalogUUID));
- KerberosAuthUtils.saveKeytabFromUri(
- kerberosConfig.getKeytab(),
- keytabFile,
- kerberosConfig.getFetchTimeoutSec(),
- false,
- this.getConf());
- kerberosClient.login(keytabFile.getAbsolutePath());
- return kerberosClient;
- } catch (IOException e) {
- throw new RuntimeException("Failed to login with kerberos", e);
- }
- }
}
diff --git
a/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/ClosableJdbcCatalog.java
b/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/ClosableJdbcCatalog.java
new file mode 100644
index 0000000000..6e2dd5c07e
--- /dev/null
+++
b/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/ClosableJdbcCatalog.java
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.iceberg.common;
+
+import java.io.Closeable;
+import java.util.Map;
+import java.util.function.Function;
+import javax.annotation.Nullable;
+import org.apache.gravitino.catalog.hadoop.auth.KerberosClient;
+import org.apache.gravitino.iceberg.common.authentication.AuthenticationConfig;
+import org.apache.gravitino.iceberg.common.authentication.SupportsKerberos;
+import org.apache.gravitino.iceberg.common.utils.KerberosCatalogUtils;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.iceberg.io.FileIO;
+import org.apache.iceberg.jdbc.JdbcCatalog;
+import org.apache.iceberg.jdbc.JdbcClientPool;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * ClosableJdbcCatalog is a wrapper class to wrap Iceberg JdbcCatalog to do
some clean-up work like
+ * closing resources and supporting Kerberos authentication for HDFS access.
+ */
+public class ClosableJdbcCatalog extends JdbcCatalog implements Closeable,
SupportsKerberos {
+
+ private static final Logger LOGGER =
LoggerFactory.getLogger(ClosableJdbcCatalog.class);
+
+ @Nullable private Configuration hadoopConf;
+
+ private KerberosClient kerberosClient;
+
+ public ClosableJdbcCatalog() {
+ super();
+ }
+
+ public ClosableJdbcCatalog(
+ Function<Map<String, String>, FileIO> ioBuilder,
+ Function<Map<String, String>, JdbcClientPool> clientPoolBuilder,
+ boolean initializeCatalogTables) {
+ super(ioBuilder, clientPoolBuilder, initializeCatalogTables);
+ }
+
+ @Override
+ public void setConf(Object conf) {
+ super.setConf(conf);
+ this.hadoopConf = (Configuration) conf;
+ }
+
+ /**
+ * Initialize the ClosableJdbcCatalog with the given input name and
properties.
+ *
+ * <p>Note: This method can only be called once as it will create new client
pools.
+ *
+ * @param inputName name of the catalog
+ * @param properties properties for the catalog
+ */
+ @Override
+ public void initialize(String inputName, Map<String, String> properties) {
+ super.initialize(inputName, properties);
+
+ AuthenticationConfig authenticationConfig = new
AuthenticationConfig(properties);
+ if (authenticationConfig.isKerberosAuth()) {
+ try {
+ this.kerberosClient =
+ KerberosCatalogUtils.initKerberosClient(properties, hadoopConf,
name());
+ } catch (RuntimeException e) {
+ try {
+ close();
+ } catch (Exception closeException) {
+ e.addSuppressed(closeException);
+ }
+ throw e;
+ }
+ }
+ }
+
+ @Override
+ public void close() {
+ KerberosCatalogUtils.closeKerberosClient(kerberosClient, LOGGER);
+ try {
+ super.close();
+ } catch (Exception e) {
+ LOGGER.warn("Failed to close JdbcCatalog", e);
+ }
+ }
+
+ @Override
+ public <R> R doKerberosOperations(Executable<R> executable) throws Throwable
{
+ return KerberosCatalogUtils.doKerberosOperations(this.properties(),
kerberosClient, executable);
+ }
+}
diff --git
a/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/utils/IcebergCatalogUtil.java
b/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/utils/IcebergCatalogUtil.java
index 0d2d5462d0..a5587964e5 100644
---
a/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/utils/IcebergCatalogUtil.java
+++
b/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/utils/IcebergCatalogUtil.java
@@ -32,6 +32,7 @@ import
org.apache.gravitino.catalog.lakehouse.iceberg.IcebergCatalogBackend;
import org.apache.gravitino.catalog.lakehouse.iceberg.IcebergConstants;
import org.apache.gravitino.exceptions.ConnectionFailedException;
import org.apache.gravitino.iceberg.common.ClosableHiveCatalog;
+import org.apache.gravitino.iceberg.common.ClosableJdbcCatalog;
import org.apache.gravitino.iceberg.common.IcebergConfig;
import org.apache.gravitino.iceberg.common.authentication.AuthenticationConfig;
import
org.apache.gravitino.iceberg.common.rest.auth.UserPrincipalForwardingAuthManager;
@@ -108,7 +109,7 @@ public class IcebergCatalogUtil {
} catch (ClassNotFoundException e) {
throw new IllegalArgumentException("Couldn't load jdbc driver " +
driverClassName);
}
- JdbcCatalog jdbcCatalog =
+ ClosableJdbcCatalog jdbcCatalog =
new JdbcCatalogWithMetadataLocationSupport(
icebergConfig.get(IcebergConfig.JDBC_INIT_TABLES));
@@ -123,9 +124,20 @@ public class IcebergCatalogUtil {
HdfsConfiguration hdfsConfiguration = new HdfsConfiguration();
properties.forEach(hdfsConfiguration::set);
- jdbcCatalog.setConf(hdfsConfiguration);
+ AuthenticationConfig authenticationConfig = new
AuthenticationConfig(properties);
try {
- jdbcCatalog.initialize(icebergCatalogName, properties);
+ if (authenticationConfig.isSimpleAuth()) {
+ jdbcCatalog.setConf(hdfsConfiguration);
+ jdbcCatalog.initialize(icebergCatalogName, properties);
+ } else if (authenticationConfig.isKerberosAuth()) {
+ hdfsConfiguration.set(HADOOP_SECURITY_AUTHORIZATION, "true");
+ hdfsConfiguration.set(HADOOP_SECURITY_AUTHENTICATION, "kerberos");
+ jdbcCatalog.setConf(hdfsConfiguration);
+ jdbcCatalog.initialize(icebergCatalogName, properties);
+ } else {
+ throw new UnsupportedOperationException(
+ "Unsupported authentication method: " +
authenticationConfig.getAuthType());
+ }
} catch (UncheckedSQLException e) {
if (e.getCause() instanceof SQLException
&& e.getCause().getMessage().contains("Access denied")) {
diff --git
a/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/utils/KerberosCatalogUtils.java
b/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/utils/KerberosCatalogUtils.java
new file mode 100644
index 0000000000..d7749914e3
--- /dev/null
+++
b/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/utils/KerberosCatalogUtils.java
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.iceberg.common.utils;
+
+import java.io.File;
+import java.io.IOException;
+import java.security.PrivilegedExceptionAction;
+import java.util.Map;
+import javax.annotation.Nullable;
+import org.apache.gravitino.catalog.hadoop.auth.KerberosAuthUtils;
+import org.apache.gravitino.catalog.hadoop.auth.KerberosClient;
+import org.apache.gravitino.iceberg.common.authentication.AuthenticationConfig;
+import org.apache.gravitino.iceberg.common.authentication.SupportsKerberos;
+import
org.apache.gravitino.iceberg.common.authentication.kerberos.KerberosConfig;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.slf4j.Logger;
+
+/** Shared Kerberos initialization and execution helpers for Iceberg closable
catalogs. */
+public final class KerberosCatalogUtils {
+
+ private KerberosCatalogUtils() {}
+
+ /**
+ * Initializes and logs in a {@link KerberosClient} for the given catalog.
+ *
+ * @param properties Catalog properties containing Kerberos configuration.
+ * @param hadoopConf Hadoop configuration used for keytab download and login.
+ * @param catalogName Catalog name used to derive the local keytab path.
+ * @return The initialized and logged-in Kerberos client.
+ */
+ public static KerberosClient initKerberosClient(
+ Map<String, String> properties, Configuration hadoopConf, String
catalogName) {
+ try {
+ KerberosConfig kerberosConfig = new KerberosConfig(properties);
+ KerberosClient kerberosClient =
+ KerberosClient.builder(kerberosConfig.getPrincipalName(), hadoopConf)
+ .loginMode(KerberosAuthUtils.LoginMode.CURRENT_USER)
+ .checkIntervalSec(kerberosConfig.getCheckIntervalSec())
+ .build();
+ File keytabFile =
+ new File(String.format(KerberosConfig.GRAVITINO_KEYTAB_FORMAT,
catalogName));
+ KerberosAuthUtils.saveKeytabFromUri(
+ kerberosConfig.getKeytab(),
+ keytabFile,
+ kerberosConfig.getFetchTimeoutSec(),
+ false,
+ hadoopConf);
+ kerberosClient.login(keytabFile.getAbsolutePath());
+ return kerberosClient;
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to login with kerberos", e);
+ }
+ }
+
+ /**
+ * Closes a Kerberos client and logs failures without rethrowing.
+ *
+ * @param kerberosClient The Kerberos client to close, or null.
+ * @param logger Logger used to record close failures.
+ */
+ public static void closeKerberosClient(@Nullable KerberosClient
kerberosClient, Logger logger) {
+ if (kerberosClient != null) {
+ try {
+ kerberosClient.close();
+ } catch (Exception e) {
+ logger.warn("Failed to close KerberosClient", e);
+ }
+ }
+ }
+
+ /**
+ * Executes catalog operations under Kerberos when configured, otherwise
runs directly.
+ *
+ * @param properties Catalog properties.
+ * @param kerberosClient Initialized Kerberos client, or null when Kerberos
is not configured.
+ * @param executable Operation to execute.
+ * @param <R> Result type.
+ * @return Operation result.
+ * @throws Throwable If the operation fails.
+ */
+ public static <R> R doKerberosOperations(
+ Map<String, String> properties,
+ @Nullable KerberosClient kerberosClient,
+ SupportsKerberos.Executable<R> executable)
+ throws Throwable {
+ AuthenticationConfig authenticationConfig = new
AuthenticationConfig(properties);
+ if (!authenticationConfig.isKerberosAuth()) {
+ return executable.execute();
+ }
+ if (kerberosClient == null) {
+ throw new IllegalStateException(
+ "Kerberos is configured but KerberosClient is not initialized");
+ }
+
+ String finalPrincipalName = resolveProxyPrincipal(kerberosClient);
+ UserGroupInformation realUser =
+ createRealUser(authenticationConfig, kerberosClient,
finalPrincipalName);
+ return executeAs(realUser, executable);
+ }
+
+ /**
+ * Resolves the proxy Kerberos principal for the current subject.
+ *
+ * @param kerberosClient Kerberos client used to obtain the realm when
needed.
+ * @return Fully qualified Kerberos principal name.
+ */
+ public static String resolveProxyPrincipal(KerberosClient kerberosClient) {
+ String proxyKerberosPrincipalName =
PrincipalUtils.getCurrentPrincipal().getName();
+ if (!proxyKerberosPrincipalName.contains("@")) {
+ return String.format("%s@%s", proxyKerberosPrincipalName,
kerberosClient.getRealm());
+ }
+ return proxyKerberosPrincipalName;
+ }
+
+ /**
+ * Creates the effective UGI for catalog operations, optionally as a proxy
user.
+ *
+ * @param authenticationConfig Authentication configuration.
+ * @param kerberosClient Kerberos client providing the login user.
+ * @param finalPrincipalName Proxy principal name when impersonation is
enabled.
+ * @return UGI used to execute catalog operations.
+ */
+ public static UserGroupInformation createRealUser(
+ AuthenticationConfig authenticationConfig,
+ KerberosClient kerberosClient,
+ String finalPrincipalName) {
+ return authenticationConfig.isImpersonationEnabled()
+ ? UserGroupInformation.createProxyUser(finalPrincipalName,
kerberosClient.getLoginUser())
+ : kerberosClient.getLoginUser();
+ }
+
+ /**
+ * Executes an operation as the given UGI.
+ *
+ * @param userGroupInformation UGI to run as.
+ * @param executable Operation to execute.
+ * @param <R> Result type.
+ * @return Operation result.
+ * @throws Throwable If the operation fails.
+ */
+ public static <R> R executeAs(
+ UserGroupInformation userGroupInformation,
SupportsKerberos.Executable<R> executable)
+ throws Throwable {
+ return userGroupInformation.doAs(
+ (PrivilegedExceptionAction<R>)
+ () -> {
+ try {
+ return executable.execute();
+ } catch (Throwable e) {
+ if (RuntimeException.class.isAssignableFrom(e.getClass())) {
+ throw (RuntimeException) e;
+ }
+ throw new RuntimeException("Failed to invoke method", e);
+ }
+ });
+ }
+}
diff --git
a/iceberg/iceberg-common/src/main/java/org/apache/iceberg/jdbc/JdbcCatalogWithMetadataLocationSupport.java
b/iceberg/iceberg-common/src/main/java/org/apache/iceberg/jdbc/JdbcCatalogWithMetadataLocationSupport.java
index e3f51ef5c1..d1b17c2ced 100644
---
a/iceberg/iceberg-common/src/main/java/org/apache/iceberg/jdbc/JdbcCatalogWithMetadataLocationSupport.java
+++
b/iceberg/iceberg-common/src/main/java/org/apache/iceberg/jdbc/JdbcCatalogWithMetadataLocationSupport.java
@@ -23,6 +23,7 @@ import com.google.common.base.Preconditions;
import java.sql.SQLException;
import java.util.Map;
import org.apache.commons.lang3.reflect.FieldUtils;
+import org.apache.gravitino.iceberg.common.ClosableJdbcCatalog;
import org.apache.gravitino.iceberg.common.cache.SupportsMetadataLocation;
import org.apache.iceberg.MetastoreRegisterTableUtils;
import org.apache.iceberg.Table;
@@ -33,7 +34,7 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
// Use Iceberg package to reuse JdbcUtil related classes.
-public class JdbcCatalogWithMetadataLocationSupport extends JdbcCatalog
+public class JdbcCatalogWithMetadataLocationSupport extends ClosableJdbcCatalog
implements SupportsMetadataLocation {
private static final Logger LOG =
LoggerFactory.getLogger(JdbcCatalogWithMetadataLocationSupport.class);
diff --git
a/iceberg/iceberg-common/src/test/java/org/apache/gravitino/iceberg/common/TestClosableJdbcCatalog.java
b/iceberg/iceberg-common/src/test/java/org/apache/gravitino/iceberg/common/TestClosableJdbcCatalog.java
new file mode 100644
index 0000000000..4e386fa5fa
--- /dev/null
+++
b/iceberg/iceberg-common/src/test/java/org/apache/gravitino/iceberg/common/TestClosableJdbcCatalog.java
@@ -0,0 +1,134 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.iceberg.common;
+
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.Map;
+import org.apache.gravitino.catalog.lakehouse.iceberg.IcebergConstants;
+import org.apache.gravitino.iceberg.common.authentication.AuthenticationConfig;
+import
org.apache.gravitino.iceberg.common.authentication.kerberos.KerberosConfig;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hdfs.HdfsConfiguration;
+import org.apache.iceberg.CatalogProperties;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+public class TestClosableJdbcCatalog {
+
+ @TempDir private Path warehouse;
+
+ @Test
+ void testSimpleInit() {
+ ClosableJdbcCatalog catalog = new ClosableJdbcCatalog();
+ Configuration conf = new HdfsConfiguration();
+ catalog.setConf(conf);
+ catalog.initialize("test", newJdbcCatalogProperties());
+
+ Assertions.assertDoesNotThrow(catalog::close);
+ }
+
+ @Test
+ void testKerberosInitNoConf() {
+ ClosableJdbcCatalog catalog = new ClosableJdbcCatalog();
+ Map<String, String> properties = newJdbcCatalogProperties();
+ properties.put(AuthenticationConfig.AUTH_TYPE_KEY, "kerberos");
+ properties.put(KerberosConfig.PRINCIPAL_KEY, "cli@HADOOPKRB");
+ properties.put(KerberosConfig.KET_TAB_URI_KEY, "/tmp/missing.keytab");
+
+ Assertions.assertThrows(RuntimeException.class, () ->
catalog.initialize("test", properties));
+ }
+
+ @Test
+ void testKerberosInitBadKeytab() {
+ ClosableJdbcCatalog catalog = new ClosableJdbcCatalog();
+ Configuration conf = new HdfsConfiguration();
+ catalog.setConf(conf);
+
+ Map<String, String> properties = newJdbcCatalogProperties();
+ properties.put(AuthenticationConfig.AUTH_TYPE_KEY, "kerberos");
+ properties.put(KerberosConfig.PRINCIPAL_KEY, "cli@HADOOPKRB");
+ properties.put(KerberosConfig.KET_TAB_URI_KEY, "/tmp/missing.keytab");
+
+ RuntimeException exception =
+ Assertions.assertThrows(
+ RuntimeException.class, () -> catalog.initialize("test",
properties));
+ Assertions.assertTrue(exception.getMessage().contains("Failed to login
with kerberos"));
+ }
+
+ @Test
+ void testSimpleOps() throws Throwable {
+ ClosableJdbcCatalog catalog = new ClosableJdbcCatalog();
+ catalog.setConf(new HdfsConfiguration());
+ catalog.initialize("test", newJdbcCatalogProperties());
+
+ Assertions.assertEquals("ok", catalog.doKerberosOperations(() -> "ok"));
+ }
+
+ @Test
+ void testImpersonationConfig() {
+ Map<String, String> properties = newJdbcCatalogProperties();
+ properties.put(AuthenticationConfig.IMPERSONATION_ENABLE_KEY, "true");
+ Assertions.assertTrue(new
AuthenticationConfig(properties).isImpersonationEnabled());
+
+ properties.put(AuthenticationConfig.IMPERSONATION_ENABLE_KEY, "false");
+ Assertions.assertFalse(new
AuthenticationConfig(properties).isImpersonationEnabled());
+ }
+
+ @Test
+ void testOpsNoImpersonation() throws Throwable {
+ ClosableJdbcCatalog catalog = new ClosableJdbcCatalog();
+ catalog.setConf(new HdfsConfiguration());
+
+ Map<String, String> properties = newJdbcCatalogProperties();
+ properties.put(AuthenticationConfig.IMPERSONATION_ENABLE_KEY, "false");
+ catalog.initialize("test", properties);
+
+ Assertions.assertEquals("ok", catalog.doKerberosOperations(() -> "ok"));
+ }
+
+ @Test
+ void testOpsNoClient() {
+ ClosableJdbcCatalog catalog = new ClosableJdbcCatalog();
+ Configuration conf = new HdfsConfiguration();
+ catalog.setConf(conf);
+
+ Map<String, String> properties = newJdbcCatalogProperties();
+ properties.put(AuthenticationConfig.AUTH_TYPE_KEY, "kerberos");
+ properties.put(KerberosConfig.PRINCIPAL_KEY, "cli@HADOOPKRB");
+ properties.put(KerberosConfig.KET_TAB_URI_KEY, "/tmp/missing.keytab");
+
+ Assertions.assertThrows(RuntimeException.class, () ->
catalog.initialize("test", properties));
+ Assertions.assertThrows(
+ IllegalStateException.class, () -> catalog.doKerberosOperations(() ->
"ok"));
+ }
+
+ private Map<String, String> newJdbcCatalogProperties() {
+ Map<String, String> properties = new HashMap<>();
+ properties.put(CatalogProperties.URI, "jdbc:sqlite::memory:");
+ properties.put(CatalogProperties.WAREHOUSE_LOCATION, warehouse.toString());
+ properties.put(IcebergConstants.GRAVITINO_JDBC_DRIVER, "org.sqlite.JDBC");
+ properties.put(IcebergConstants.ICEBERG_JDBC_USER, "test");
+ properties.put(IcebergConstants.ICEBERG_JDBC_PASSWORD, "test");
+ properties.put(IcebergConstants.ICEBERG_JDBC_INITIALIZE, "true");
+ return properties;
+ }
+}
diff --git
a/iceberg/iceberg-common/src/test/java/org/apache/gravitino/iceberg/common/utils/TestIcebergCatalogUtil.java
b/iceberg/iceberg-common/src/test/java/org/apache/gravitino/iceberg/common/utils/TestIcebergCatalogUtil.java
index 0bf3745538..1b302f4437 100644
---
a/iceberg/iceberg-common/src/test/java/org/apache/gravitino/iceberg/common/utils/TestIcebergCatalogUtil.java
+++
b/iceberg/iceberg-common/src/test/java/org/apache/gravitino/iceberg/common/utils/TestIcebergCatalogUtil.java
@@ -24,7 +24,9 @@ import java.util.HashMap;
import java.util.Map;
import org.apache.gravitino.catalog.lakehouse.iceberg.IcebergCatalogBackend;
import org.apache.gravitino.catalog.lakehouse.iceberg.IcebergConstants;
+import org.apache.gravitino.iceberg.common.ClosableJdbcCatalog;
import org.apache.gravitino.iceberg.common.IcebergConfig;
+import org.apache.gravitino.iceberg.common.authentication.AuthenticationConfig;
import org.apache.iceberg.CatalogProperties;
import org.apache.iceberg.Schema;
import org.apache.iceberg.catalog.Catalog;
@@ -33,7 +35,6 @@ import org.apache.iceberg.catalog.TableIdentifier;
import org.apache.iceberg.exceptions.NoSuchNamespaceException;
import org.apache.iceberg.hive.HiveCatalog;
import org.apache.iceberg.inmemory.InMemoryCatalog;
-import org.apache.iceberg.jdbc.JdbcCatalog;
import org.apache.iceberg.jdbc.JdbcCatalogWithMetadataLocationSupport;
import org.apache.iceberg.types.Types;
import org.junit.jupiter.api.Assertions;
@@ -80,7 +81,7 @@ public class TestIcebergCatalogUtil {
catalog =
IcebergCatalogUtil.loadCatalogBackend(
IcebergCatalogBackend.JDBC, new IcebergConfig(properties));
- Assertions.assertTrue(catalog instanceof JdbcCatalog);
+ Assertions.assertInstanceOf(ClosableJdbcCatalog.class, catalog);
Assertions.assertThrowsExactly(
IllegalArgumentException.class,
@@ -89,6 +90,27 @@ public class TestIcebergCatalogUtil {
});
}
+ @Test
+ void testJdbcBadAuth() {
+ Map<String, String> properties = new HashMap<>();
+ properties.put(CatalogProperties.URI, "jdbc:sqlite::memory:");
+ properties.put(CatalogProperties.WAREHOUSE_LOCATION, warehouse.toString());
+ properties.put(IcebergConstants.GRAVITINO_JDBC_DRIVER, "org.sqlite.JDBC");
+ properties.put(IcebergConstants.ICEBERG_JDBC_USER, "test");
+ properties.put(IcebergConstants.ICEBERG_JDBC_PASSWORD, "test");
+ properties.put(IcebergConstants.ICEBERG_JDBC_INITIALIZE, "true");
+ properties.put(AuthenticationConfig.AUTH_TYPE_KEY, "oauth");
+
+ UnsupportedOperationException exception =
+ Assertions.assertThrows(
+ UnsupportedOperationException.class,
+ () ->
+ IcebergCatalogUtil.loadCatalogBackend(
+ IcebergCatalogBackend.JDBC, new
IcebergConfig(properties)));
+ Assertions.assertTrue(
+ exception.getMessage().contains("Unsupported authentication method:
oauth"));
+ }
+
@Test
void testJdbcCatalogDefaultSchemaVersionIsV1() {
Map<String, String> properties = new HashMap<>();
diff --git
a/iceberg/iceberg-common/src/test/java/org/apache/gravitino/iceberg/common/utils/TestKerberosCatalogUtils.java
b/iceberg/iceberg-common/src/test/java/org/apache/gravitino/iceberg/common/utils/TestKerberosCatalogUtils.java
new file mode 100644
index 0000000000..7798cf490f
--- /dev/null
+++
b/iceberg/iceberg-common/src/test/java/org/apache/gravitino/iceberg/common/utils/TestKerberosCatalogUtils.java
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.iceberg.common.utils;
+
+import java.util.HashMap;
+import java.util.Map;
+import org.apache.gravitino.catalog.hadoop.auth.KerberosClient;
+import org.apache.gravitino.iceberg.common.authentication.AuthenticationConfig;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.mockito.Mockito;
+
+public class TestKerberosCatalogUtils {
+
+ @Test
+ void testOpsSimpleAuth() throws Throwable {
+ Map<String, String> properties = new HashMap<>();
+ properties.put(AuthenticationConfig.AUTH_TYPE_KEY, "simple");
+
+ String result = KerberosCatalogUtils.doKerberosOperations(properties,
null, () -> "ok");
+
+ Assertions.assertEquals("ok", result);
+ }
+
+ @Test
+ void testOpsNoClient() {
+ Map<String, String> properties = new HashMap<>();
+ properties.put(AuthenticationConfig.AUTH_TYPE_KEY, "kerberos");
+
+ IllegalStateException exception =
+ Assertions.assertThrows(
+ IllegalStateException.class,
+ () -> KerberosCatalogUtils.doKerberosOperations(properties, null,
() -> "ok"));
+
+ Assertions.assertTrue(
+ exception.getMessage().contains("Kerberos is configured but
KerberosClient is not"));
+ }
+
+ @Test
+ void testProxyPrincipalRealm() {
+ KerberosClient kerberosClient = Mockito.mock(KerberosClient.class);
+ Mockito.when(kerberosClient.getRealm()).thenReturn("EXAMPLE.COM");
+
+ String principal =
KerberosCatalogUtils.resolveProxyPrincipal(kerberosClient);
+
+ Assertions.assertTrue(principal.endsWith("@EXAMPLE.COM"));
+ }
+}
diff --git
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRESTServiceBaseIT.java
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRESTServiceBaseIT.java
index ab32e165d6..b48db1c6e4 100644
---
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRESTServiceBaseIT.java
+++
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRESTServiceBaseIT.java
@@ -76,9 +76,13 @@ public abstract class IcebergRESTServiceBaseIT {
void stopIcebergTestEnv() throws Exception {
stopSparkEnv();
icebergRESTServerManager.stopIcebergRESTServer();
+ afterIcebergRESTServerStopped();
LOG.info("Gravitino and Spark env stopped,{}", catalogType);
}
+ /** Hook invoked after the Iceberg REST server is stopped during test
teardown. */
+ protected void afterIcebergRESTServerStopped() {}
+
boolean catalogTypeNotMemory() {
return !catalogType.equals(IcebergCatalogBackend.MEMORY);
}
@@ -152,9 +156,14 @@ public abstract class IcebergRESTServiceBaseIT {
"spark.sql.catalog.rest.header.X-Iceberg-Access-Delegation",
"vended-credentials");
}
+ customizeSparkConf(sparkConf);
+
sparkSession =
SparkSession.builder().master("local[1]").config(sparkConf).getOrCreate();
}
+ /** Hook to customize Spark configuration before the test Spark session is
created. */
+ protected void customizeSparkConf(SparkConf sparkConf) {}
+
private void stopSparkEnv() {
if (sparkSession != null) {
sparkSession.close();
diff --git
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRESTServiceIT.java
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRESTServiceIT.java
index 9109f9734a..4f6e4c6590 100644
---
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRESTServiceIT.java
+++
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRESTServiceIT.java
@@ -93,7 +93,13 @@ public abstract class IcebergRESTServiceIT extends
IcebergRESTServiceBaseIT {
}
@AfterAll
- void cleanup() {
+ @Override
+ void stopIcebergTestEnv() throws Exception {
+ cleanup();
+ super.stopIcebergTestEnv();
+ }
+
+ private void cleanup() {
purgeAllIcebergTestNamespaces();
}
diff --git
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosHiveCatalogIT.java
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosHiveCatalogIT.java
index 36969f0e31..0b494d0861 100644
---
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosHiveCatalogIT.java
+++
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosHiveCatalogIT.java
@@ -18,31 +18,24 @@
*/
package org.apache.gravitino.iceberg.integration.test;
-import java.io.File;
-import java.lang.reflect.Method;
-import java.nio.charset.StandardCharsets;
-import java.nio.file.Files;
import java.util.HashMap;
import java.util.Map;
-import java.util.Objects;
-import org.apache.commons.io.FileUtils;
import org.apache.gravitino.catalog.lakehouse.iceberg.IcebergCatalogBackend;
import org.apache.gravitino.iceberg.common.IcebergConfig;
import org.apache.gravitino.integration.test.container.HiveContainer;
import org.apache.gravitino.integration.test.util.GravitinoITUtils;
-import org.apache.gravitino.integration.test.util.ITUtils;
+import org.apache.spark.SparkConf;
import org.junit.jupiter.api.Tag;
+import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.TestInstance.Lifecycle;
-import org.junit.jupiter.api.condition.EnabledIf;
+import org.junit.jupiter.api.condition.DisabledIf;
@Tag("gravitino-docker-test")
@TestInstance(Lifecycle.PER_CLASS)
-@EnabledIf("isEmbedded")
public class IcebergRestKerberosHiveCatalogIT extends IcebergRESTHiveCatalogIT
{
protected static final String HIVE_METASTORE_CLIENT_PRINCIPAL =
"cli@HADOOPKRB";
- protected static final String HIVE_METASTORE_CLIENT_KEYTAB =
"/client.keytab";
protected static String tempDir;
@@ -51,58 +44,12 @@ public class IcebergRestKerberosHiveCatalogIT extends
IcebergRESTHiveCatalogIT {
}
void initEnv() {
- containerSuite.startKerberosHiveContainer();
- try {
-
- // Init kerberos configurations;
- File baseDir = new File(System.getProperty("java.io.tmpdir"));
- File file = Files.createTempDirectory(baseDir.toPath(), "test").toFile();
- file.deleteOnExit();
- tempDir = file.getAbsolutePath();
-
- HiveContainer kerberosHiveContainer =
containerSuite.getKerberosHiveContainer();
- kerberosHiveContainer
- .getContainer()
- .copyFileFromContainer("/etc/admin.keytab", tempDir +
HIVE_METASTORE_CLIENT_KEYTAB);
-
- String tmpKrb5Path = tempDir + "/krb5.conf_tmp";
- String krb5Path = tempDir + "/krb5.conf";
-
kerberosHiveContainer.getContainer().copyFileFromContainer("/etc/krb5.conf",
tmpKrb5Path);
-
- // Modify the krb5.conf and change the kdc and admin_server to the
container IP
- String ip =
containerSuite.getKerberosHiveContainer().getContainerIpAddress();
- String content = FileUtils.readFileToString(new File(tmpKrb5Path),
StandardCharsets.UTF_8);
- content = content.replace("kdc = localhost:88", "kdc = " + ip + ":88");
- content = content.replace("admin_server = localhost", "admin_server = "
+ ip + ":749");
- FileUtils.write(new File(krb5Path), content, StandardCharsets.UTF_8);
-
- LOG.info("Kerberos kdc config:\n{}, path: {}", content, krb5Path);
- System.setProperty("java.security.krb5.conf", krb5Path);
- System.setProperty("sun.security.krb5.debug", "true");
- System.setProperty("java.security.krb5.realm", "HADOOPKRB");
- System.setProperty("java.security.krb5.kdc", ip);
-
- refreshKerberosConfig();
- resetDefaultRealm();
-
- // Give cli@HADOOPKRB permission to access the hdfs
- containerSuite
- .getKerberosHiveContainer()
- .executeInContainer("hadoop", "fs", "-chown", "-R", "cli",
"/user/hive/");
-
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
+ tempDir = IcebergRestKerberosTestEnv.init(containerSuite);
}
- void resetDefaultRealm() {
- try {
- String kerberosNameClass =
"org.apache.hadoop.security.authentication.util.KerberosName";
- Class<?> cl = Class.forName(kerberosNameClass);
- cl.getMethod("resetDefaultRealm").invoke(null);
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
+ @Override
+ protected void afterIcebergRESTServerStopped() {
+ IcebergRestKerberosTestEnv.reset();
}
@Override
@@ -115,7 +62,7 @@ public class IcebergRestKerberosHiveCatalogIT extends
IcebergRESTHiveCatalogIT {
HIVE_METASTORE_CLIENT_PRINCIPAL);
configMap.put(
"gravitino.iceberg-rest.authentication.kerberos.keytab-uri",
- tempDir + HIVE_METASTORE_CLIENT_KEYTAB);
+ tempDir + IcebergRestKerberosTestEnv.CLIENT_KEYTAB);
configMap.put("gravitino.iceberg-rest.hive.metastore.sasl.enabled",
"true");
configMap.put(
"gravitino.iceberg-rest.hive.metastore.kerberos.principal",
@@ -150,28 +97,44 @@ public class IcebergRestKerberosHiveCatalogIT extends
IcebergRESTHiveCatalogIT {
return configMap;
}
- protected static boolean isEmbedded() {
- String mode =
- System.getProperty(ITUtils.TEST_MODE) == null
- ? ITUtils.EMBEDDED_TEST_MODE
- : System.getProperty(ITUtils.TEST_MODE);
+ @Override
+ protected void customizeSparkConf(SparkConf sparkConf) {
+ IcebergRestKerberosTestEnv.configureSparkKerberos(
+ sparkConf,
+ HIVE_METASTORE_CLIENT_PRINCIPAL,
+ containerSuite.getKerberosHiveContainer().getHostName());
+ }
- return Objects.equals(mode, ITUtils.EMBEDDED_TEST_MODE);
+ // Spark writes data files to HDFS directly in these tests; keep them in
embedded mode only.
+ @Test
+ @DisabledIf(
+
"org.apache.gravitino.iceberg.integration.test.IcebergRestKerberosTestEnv#isDeployMode")
+ @Override
+ void testDML() {
+ super.testDML();
}
- protected static void refreshKerberosConfig() {
- Class<?> classRef;
- try {
- if (System.getProperty("java.vendor").contains("IBM")) {
- classRef = Class.forName("com.ibm.security.krb5.internal.Config");
- } else {
- classRef = Class.forName("sun.security.krb5.Config");
- }
-
- Method refreshMethod = classRef.getMethod("refresh");
- refreshMethod.invoke(null);
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
+ @Test
+ @DisabledIf(
+
"org.apache.gravitino.iceberg.integration.test.IcebergRestKerberosTestEnv#isDeployMode")
+ @Override
+ void testRegisterTable() {
+ super.testRegisterTable();
+ }
+
+ @Test
+ @DisabledIf(
+
"org.apache.gravitino.iceberg.integration.test.IcebergRestKerberosTestEnv#isDeployMode")
+ @Override
+ void testSnapshot() {
+ super.testSnapshot();
+ }
+
+ @Test
+ @DisabledIf(
+
"org.apache.gravitino.iceberg.integration.test.IcebergRestKerberosTestEnv#isDeployMode")
+ @Override
+ void testRegisterTableOverwrite() throws Exception {
+ super.testRegisterTableOverwrite();
}
}
diff --git
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosHiveWithUserImpersonationCatalogIT.java
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosHiveWithUserImpersonationCatalogIT.java
index 28a7ee4dcc..f4209305e2 100644
---
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosHiveWithUserImpersonationCatalogIT.java
+++
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosHiveWithUserImpersonationCatalogIT.java
@@ -29,11 +29,9 @@ import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.TestInstance.Lifecycle;
-import org.junit.jupiter.api.condition.EnabledIf;
@Tag("gravitino-docker-test")
@TestInstance(Lifecycle.PER_CLASS)
-@EnabledIf("isEmbedded")
public class IcebergRestKerberosHiveWithUserImpersonationCatalogIT
extends IcebergRestKerberosHiveCatalogIT {
@@ -95,6 +93,11 @@ public class
IcebergRestKerberosHiveWithUserImpersonationCatalogIT
"spark.sql.catalog.rest.header.X-Iceberg-Access-Delegation",
"vended-credentials");
}
+ IcebergRestKerberosTestEnv.configureSparkKerberos(
+ sparkConf,
+ HIVE_METASTORE_CLIENT_PRINCIPAL,
+ containerSuite.getKerberosHiveContainer().getHostName());
+
sparkSession =
SparkSession.builder().master("local[1]").config(sparkConf).getOrCreate();
}
diff --git
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosJdbcCatalogIT.java
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosJdbcCatalogIT.java
new file mode 100644
index 0000000000..a97f7f59c9
--- /dev/null
+++
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosJdbcCatalogIT.java
@@ -0,0 +1,133 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.iceberg.integration.test;
+
+import java.util.HashMap;
+import java.util.Map;
+import org.apache.gravitino.catalog.lakehouse.iceberg.IcebergCatalogBackend;
+import org.apache.gravitino.catalog.lakehouse.iceberg.IcebergConstants;
+import org.apache.gravitino.iceberg.common.IcebergConfig;
+import org.apache.gravitino.integration.test.container.ContainerSuite;
+import org.apache.gravitino.integration.test.container.HiveContainer;
+import org.apache.gravitino.integration.test.util.GravitinoITUtils;
+import org.apache.spark.SparkConf;
+import org.junit.jupiter.api.Tag;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.TestInstance;
+import org.junit.jupiter.api.TestInstance.Lifecycle;
+import org.junit.jupiter.api.condition.DisabledIf;
+
+@Tag("gravitino-docker-test")
+@TestInstance(Lifecycle.PER_CLASS)
+public class IcebergRestKerberosJdbcCatalogIT extends IcebergRESTServiceIT {
+
+ private static final ContainerSuite containerSuite =
ContainerSuite.getInstance();
+
+ protected static final String HDFS_CLIENT_PRINCIPAL = "cli@HADOOPKRB";
+
+ private static String tempDir;
+
+ public IcebergRestKerberosJdbcCatalogIT() {
+ catalogType = IcebergCatalogBackend.JDBC;
+ }
+
+ @Override
+ void initEnv() {
+ tempDir = IcebergRestKerberosTestEnv.init(containerSuite);
+ }
+
+ @Override
+ protected void afterIcebergRESTServerStopped() {
+ IcebergRestKerberosTestEnv.reset();
+ }
+
+ @Override
+ Map<String, String> getCatalogConfig() {
+ Map<String, String> configMap = new HashMap<>();
+
+ configMap.put("gravitino.iceberg-rest.authentication.type", "kerberos");
+ configMap.put(
+ "gravitino.iceberg-rest.authentication.kerberos.principal",
HDFS_CLIENT_PRINCIPAL);
+ configMap.put(
+ "gravitino.iceberg-rest.authentication.kerberos.keytab-uri",
+ tempDir + IcebergRestKerberosTestEnv.CLIENT_KEYTAB);
+ configMap.put("gravitino.iceberg-rest.hadoop.security.authentication",
"kerberos");
+
configMap.put("gravitino.iceberg-rest.authentication.impersonation-enable",
"false");
+ configMap.put(
+ "gravitino.iceberg-rest.dfs.namenode.kerberos.principal",
+ "hdfs/_HOST@HADOOPKRB"
+ .replace("_HOST",
containerSuite.getKerberosHiveContainer().getHostName()));
+
+ configMap.put(
+ IcebergConfig.ICEBERG_CONFIG_PREFIX +
IcebergConfig.CATALOG_BACKEND.getKey(),
+ IcebergCatalogBackend.JDBC.toString().toLowerCase());
+ configMap.put(
+ IcebergConfig.ICEBERG_CONFIG_PREFIX +
IcebergConfig.JDBC_DRIVER.getKey(),
+ "org.sqlite.JDBC");
+ configMap.put(
+ IcebergConfig.ICEBERG_CONFIG_PREFIX +
IcebergConfig.CATALOG_URI.getKey(),
+ "jdbc:sqlite::memory:");
+ configMap.put(
+ IcebergConfig.ICEBERG_CONFIG_PREFIX +
IcebergConstants.ICEBERG_JDBC_USER, "iceberg");
+ configMap.put(
+ IcebergConfig.ICEBERG_CONFIG_PREFIX +
IcebergConstants.ICEBERG_JDBC_PASSWORD, "iceberg");
+ configMap.put(
+ IcebergConfig.ICEBERG_CONFIG_PREFIX +
IcebergConfig.JDBC_INIT_TABLES.getKey(), "true");
+ configMap.put(IcebergConfig.ICEBERG_CONFIG_PREFIX + "jdbc.schema-version",
"V1");
+ configMap.put(
+ IcebergConfig.ICEBERG_CONFIG_PREFIX +
IcebergConfig.CATALOG_WAREHOUSE.getKey(),
+ GravitinoITUtils.genRandomName(
+ String.format(
+ "hdfs://%s:%d/user/hive/warehouse-jdbc-kerberos",
+
containerSuite.getKerberosHiveContainer().getContainerIpAddress(),
+ HiveContainer.HDFS_DEFAULTFS_PORT)));
+ return configMap;
+ }
+
+ @Override
+ protected void customizeSparkConf(SparkConf sparkConf) {
+ IcebergRestKerberosTestEnv.configureSparkKerberos(
+ sparkConf, HDFS_CLIENT_PRINCIPAL,
containerSuite.getKerberosHiveContainer().getHostName());
+ }
+
+ // Spark writes data files to HDFS directly in these tests; keep them in
embedded mode only.
+ @Test
+ @DisabledIf(
+
"org.apache.gravitino.iceberg.integration.test.IcebergRestKerberosTestEnv#isDeployMode")
+ @Override
+ void testDML() {
+ super.testDML();
+ }
+
+ @Test
+ @DisabledIf(
+
"org.apache.gravitino.iceberg.integration.test.IcebergRestKerberosTestEnv#isDeployMode")
+ @Override
+ void testRegisterTable() {
+ super.testRegisterTable();
+ }
+
+ @Test
+ @DisabledIf(
+
"org.apache.gravitino.iceberg.integration.test.IcebergRestKerberosTestEnv#isDeployMode")
+ @Override
+ void testSnapshot() {
+ super.testSnapshot();
+ }
+}
diff --git
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosHiveWithUserImpersonationCatalogIT.java
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosJdbcWithUserImpersonationCatalogIT.java
similarity index 87%
copy from
iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosHiveWithUserImpersonationCatalogIT.java
copy to
iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosJdbcWithUserImpersonationCatalogIT.java
index 28a7ee4dcc..72d2b0b038 100644
---
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosHiveWithUserImpersonationCatalogIT.java
+++
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosJdbcWithUserImpersonationCatalogIT.java
@@ -21,6 +21,7 @@ package org.apache.gravitino.iceberg.integration.test;
import java.util.HashMap;
import java.util.Map;
import javax.annotation.Nullable;
+import org.apache.gravitino.integration.test.container.ContainerSuite;
import org.apache.spark.SparkConf;
import org.apache.spark.sql.SparkSession;
import org.junit.jupiter.api.BeforeAll;
@@ -29,27 +30,21 @@ import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.TestInstance.Lifecycle;
-import org.junit.jupiter.api.condition.EnabledIf;
@Tag("gravitino-docker-test")
@TestInstance(Lifecycle.PER_CLASS)
-@EnabledIf("isEmbedded")
-public class IcebergRestKerberosHiveWithUserImpersonationCatalogIT
- extends IcebergRestKerberosHiveCatalogIT {
+public class IcebergRestKerberosJdbcWithUserImpersonationCatalogIT
+ extends IcebergRestKerberosJdbcCatalogIT {
private static final String NORMAL_USER = "normal";
- public IcebergRestKerberosHiveWithUserImpersonationCatalogIT() {
- super();
- }
+ private static final ContainerSuite containerSuite =
ContainerSuite.getInstance();
@BeforeAll
void prepareSQLContext() {
-
// Change the ownership of /user/hive to normal user for user
impersonation test. If we do not
- // change the ownership, the normal user will not have the permission to
create table in Hive
- // as the /user/hive is owned by user `cli`, please see what's done in
`initEnv` method in
- // superclass.
+ // change the ownership, the normal user will not have the permission to
create table on HDFS
+ // warehouse as the /user/hive is owned by user `cli`.
containerSuite
.getKerberosHiveContainer()
.executeInContainer("hadoop", "fs", "-chown", "-R", NORMAL_USER,
"/user/hive/");
@@ -95,6 +90,9 @@ public class
IcebergRestKerberosHiveWithUserImpersonationCatalogIT
"spark.sql.catalog.rest.header.X-Iceberg-Access-Delegation",
"vended-credentials");
}
+ IcebergRestKerberosTestEnv.configureSparkKerberos(
+ sparkConf, HDFS_CLIENT_PRINCIPAL,
containerSuite.getKerberosHiveContainer().getHostName());
+
sparkSession =
SparkSession.builder().master("local[1]").config(sparkConf).getOrCreate();
}
@@ -106,7 +104,7 @@ public class
IcebergRestKerberosHiveWithUserImpersonationCatalogIT
parentNamespace = "iceberg_rest.nested.table_test";
separator = ".";
} else {
- parentNamespace = "iceberg_rest_with_kerberos_impersonation_table_test";
+ parentNamespace =
"iceberg_rest_with_kerberos_jdbc_impersonation_table_test";
separator = "_";
}
diff --git
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosTestEnv.java
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosTestEnv.java
new file mode 100644
index 0000000000..abf24cd04b
--- /dev/null
+++
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRestKerberosTestEnv.java
@@ -0,0 +1,253 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.iceberg.integration.test;
+
+import java.io.File;
+import java.io.IOException;
+import java.lang.reflect.Method;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.util.HashMap;
+import java.util.Map;
+import org.apache.commons.io.FileUtils;
+import org.apache.gravitino.integration.test.container.ContainerSuite;
+import org.apache.gravitino.integration.test.container.HiveContainer;
+import org.apache.gravitino.integration.test.util.ITUtils;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.spark.SparkConf;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Shared Kerberos test environment setup for Iceberg REST integration tests.
*/
+public final class IcebergRestKerberosTestEnv {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(IcebergRestKerberosTestEnv.class);
+
+ private static final String KRB5_CONF_PROPERTY = "java.security.krb5.conf";
+ private static final String KRB5_DEBUG_PROPERTY = "sun.security.krb5.debug";
+ private static final String KRB5_REALM_PROPERTY = "java.security.krb5.realm";
+ private static final String KRB5_KDC_PROPERTY = "java.security.krb5.kdc";
+
+ private static final String[] MANAGED_SYSTEM_PROPERTIES = {
+ KRB5_CONF_PROPERTY, KRB5_DEBUG_PROPERTY, KRB5_REALM_PROPERTY,
KRB5_KDC_PROPERTY
+ };
+
+ /** Local path suffix for the copied client keytab under the temp directory.
*/
+ public static final String CLIENT_KEYTAB = "/client.keytab";
+
+ private static String tempDir;
+
+ /**
+ * Configures Spark to authenticate to Kerberos-protected HDFS.
+ *
+ * <p>In deploy mode the Iceberg REST server runs in a separate JVM, so
Spark must perform its own
+ * Kerberos login to write data files to HDFS during DML tests.
+ *
+ * @param sparkConf Spark configuration to update
+ * @param clientPrincipal Kerberos principal for the Spark client
+ * @param hdfsHostName HDFS namenode host name used to build the namenode
Kerberos principal
+ */
+ public static void configureSparkKerberos(
+ SparkConf sparkConf, String clientPrincipal, String hdfsHostName) {
+ if (tempDir == null) {
+ throw new IllegalStateException(
+ "Kerberos test environment is not initialized, call init() first");
+ }
+ String keytabPath = tempDir + CLIENT_KEYTAB;
+ String hdfsPrincipal = "hdfs/_HOST@HADOOPKRB".replace("_HOST",
hdfsHostName);
+ try {
+ Configuration configuration = new Configuration();
+ configuration.set("hadoop.security.authentication", "kerberos");
+ configuration.set("dfs.namenode.kerberos.principal", hdfsPrincipal);
+ UserGroupInformation.setConfiguration(configuration);
+ if (!UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.loginUserFromKeytab(clientPrincipal, keytabPath);
+ }
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to login Spark client with Kerberos",
e);
+ }
+
+ sparkConf
+ .set("spark.hadoop.security.authentication", "kerberos")
+ .set("spark.hadoop.dfs.namenode.kerberos.principal", hdfsPrincipal)
+ .set("spark.kerberos.keytab", keytabPath)
+ .set("spark.kerberos.principal", clientPrincipal);
+
+ String krb5Path = System.getProperty(KRB5_CONF_PROPERTY);
+ if (krb5Path != null) {
+ StringBuilder opts = new StringBuilder();
+ opts.append(String.format("-Djava.security.krb5.conf=%s", krb5Path));
+ String realm = System.getProperty(KRB5_REALM_PROPERTY);
+ if (realm != null) {
+ opts.append(String.format(" -Djava.security.krb5.realm=%s", realm));
+ }
+ String kdc = System.getProperty(KRB5_KDC_PROPERTY);
+ if (kdc != null) {
+ opts.append(String.format(" -Djava.security.krb5.kdc=%s", kdc));
+ }
+ sparkConf.set("spark.driver.extraJavaOptions", opts.toString());
+ sparkConf.set("spark.executor.extraJavaOptions", opts.toString());
+ }
+ }
+
+ private static final String KEYTAB_CONTAINER_PATH = "/etc/admin.keytab";
+
+ private static Map<String, String> savedSystemProperties;
+ private static int initRefCount;
+
+ private IcebergRestKerberosTestEnv() {}
+
+ /** Returns whether integration tests are running in deploy mode. */
+ public static boolean isDeployMode() {
+ String mode =
+ System.getProperty(ITUtils.TEST_MODE) == null
+ ? ITUtils.EMBEDDED_TEST_MODE
+ : System.getProperty(ITUtils.TEST_MODE);
+ return ITUtils.DEPLOY_TEST_MODE.equals(mode);
+ }
+
+ /**
+ * Starts the Kerberos Hive container and configures the JVM krb5 settings
for tests.
+ *
+ * @param containerSuite shared docker container suite
+ * @return temp directory containing krb5.conf and client keytab
+ */
+ public static String init(ContainerSuite containerSuite) {
+ if (initRefCount == 0) {
+ savedSystemProperties = saveSystemProperties(MANAGED_SYSTEM_PROPERTIES);
+ }
+ initRefCount++;
+
+ containerSuite.startKerberosHiveContainer();
+ try {
+ File baseDir = new File(System.getProperty("java.io.tmpdir"));
+ File file = Files.createTempDirectory(baseDir.toPath(), "test").toFile();
+ file.deleteOnExit();
+ String tempDir = file.getAbsolutePath();
+
+ HiveContainer kerberosHiveContainer =
containerSuite.getKerberosHiveContainer();
+ kerberosHiveContainer
+ .getContainer()
+ .copyFileFromContainer(KEYTAB_CONTAINER_PATH, tempDir +
CLIENT_KEYTAB);
+
+ String tmpKrb5Path = tempDir + "/krb5.conf_tmp";
+ String krb5Path = tempDir + "/krb5.conf";
+
kerberosHiveContainer.getContainer().copyFileFromContainer("/etc/krb5.conf",
tmpKrb5Path);
+
+ String ip = kerberosHiveContainer.getContainerIpAddress();
+ String content = FileUtils.readFileToString(new File(tmpKrb5Path),
StandardCharsets.UTF_8);
+ content = content.replace("kdc = localhost:88", "kdc = " + ip + ":88");
+ content = content.replace("admin_server = localhost", "admin_server = "
+ ip + ":749");
+ FileUtils.write(new File(krb5Path), content, StandardCharsets.UTF_8);
+
+ LOG.info("Kerberos kdc config:\n{}, path: {}", content, krb5Path);
+ System.setProperty(KRB5_CONF_PROPERTY, krb5Path);
+ if (Boolean.getBoolean(KRB5_DEBUG_PROPERTY)) {
+ System.setProperty(KRB5_DEBUG_PROPERTY, "true");
+ } else {
+ System.clearProperty(KRB5_DEBUG_PROPERTY);
+ }
+ System.setProperty(KRB5_REALM_PROPERTY, "HADOOPKRB");
+ System.setProperty(KRB5_KDC_PROPERTY, ip);
+
+ refreshKerberosConfig();
+ resetDefaultRealm();
+
+ kerberosHiveContainer.executeInContainer(
+ "hadoop", "fs", "-chown", "-R", "cli", "/user/hive/");
+
+ IcebergRestKerberosTestEnv.tempDir = tempDir;
+ return tempDir;
+ } catch (Exception e) {
+ reset();
+ throw new RuntimeException(e);
+ }
+ }
+
+ /** Restores JVM Kerberos system properties changed by {@link
#init(ContainerSuite)}. */
+ public static void reset() {
+ if (initRefCount == 0) {
+ return;
+ }
+ initRefCount--;
+ if (initRefCount > 0) {
+ return;
+ }
+
+ if (savedSystemProperties != null) {
+ restoreSystemProperties(savedSystemProperties);
+ savedSystemProperties = null;
+ tempDir = null;
+ try {
+ refreshKerberosConfig();
+ resetDefaultRealm();
+ } catch (RuntimeException e) {
+ LOG.warn("Failed to refresh Kerberos configuration after test
cleanup", e);
+ }
+ }
+ }
+
+ /** Refreshes the JVM Kerberos configuration after updating krb5.conf. */
+ public static void refreshKerberosConfig() {
+ Class<?> classRef;
+ try {
+ if (System.getProperty("java.vendor").contains("IBM")) {
+ classRef = Class.forName("com.ibm.security.krb5.internal.Config");
+ } else {
+ classRef = Class.forName("sun.security.krb5.Config");
+ }
+
+ Method refreshMethod = classRef.getMethod("refresh");
+ refreshMethod.invoke(null);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ /** Resets Hadoop KerberosName default realm after JVM krb5 properties
change. */
+ public static void resetDefaultRealm() {
+ try {
+ String kerberosNameClass =
"org.apache.hadoop.security.authentication.util.KerberosName";
+ Class<?> cl = Class.forName(kerberosNameClass);
+ cl.getMethod("resetDefaultRealm").invoke(null);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ private static Map<String, String> saveSystemProperties(String... keys) {
+ Map<String, String> saved = new HashMap<>();
+ for (String key : keys) {
+ saved.put(key, System.getProperty(key));
+ }
+ return saved;
+ }
+
+ private static void restoreSystemProperties(Map<String, String> saved) {
+ saved.forEach(
+ (key, value) -> {
+ if (value == null) {
+ System.clearProperty(key);
+ } else {
+ System.setProperty(key, value);
+ }
+ });
+ }
+}
diff --git
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/util/IcebergRESTServerManagerForDeploy.java
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/util/IcebergRESTServerManagerForDeploy.java
index c283a19d2a..57d4d7ecd2 100644
---
a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/util/IcebergRESTServerManagerForDeploy.java
+++
b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/util/IcebergRESTServerManagerForDeploy.java
@@ -23,6 +23,7 @@ import java.io.File;
import java.nio.charset.StandardCharsets;
import java.nio.file.Path;
import java.nio.file.Paths;
+import java.util.Map;
import java.util.Optional;
import java.util.UUID;
import java.util.concurrent.Future;
@@ -35,10 +36,16 @@ import
org.apache.gravitino.integration.test.util.ProcessData.TypesOfData;
public class IcebergRESTServerManagerForDeploy extends
IcebergRESTServerManager {
private static final String SCRIPT_NAME = "gravitino-iceberg-rest-server.sh";
- private Path icebergRESTServerHome;
+ private static final String KRB5_CONF_PLACEHOLDER =
+ "#JAVA_OPTS+=\" -Djava.security.krb5.conf=/etc/krb5.conf\"";
+ private static final String KRB5_CONF_PROPERTY = "java.security.krb5.conf";
+ private static final String KRB5_REALM_PROPERTY = "java.security.krb5.realm";
+ private static final String KRB5_KDC_PROPERTY = "java.security.krb5.kdc";
private static final String SQLITE_DRIVER_DOWNLOAD_URL =
"https://repo1.maven.org/maven2/org/xerial/sqlite-jdbc/3.42.0.0/sqlite-jdbc-3.42.0.0.jar";;
+ private final Path icebergRESTServerHome;
+
public IcebergRESTServerManagerForDeploy() {
String gravitinoRootDir = System.getenv("GRAVITINO_ROOT_DIR");
this.icebergRESTServerHome = Paths.get(gravitinoRootDir, "distribution",
"package");
@@ -56,48 +63,64 @@ public class IcebergRESTServerManagerForDeploy extends
IcebergRESTServerManager
Paths.get(icebergRESTServerHome.toString(), "iceberg-rest-server",
"libs").toString());
String gravitinoRestStartShell = icebergRESTServerHome.toString() +
"/bin/" + SCRIPT_NAME;
- String krb5Path = System.getProperty("java.security.krb5.conf");
+ String startShell = gravitinoRestStartShell;
+ String krb5Path = System.getProperty(KRB5_CONF_PROPERTY);
if (krb5Path != null) {
LOG.info("java.security.krb5.conf: {}", krb5Path);
- String modifiedGravitinoStartShell =
- String.format(
- "%s/bin/gravitino-iceberg-rest-server_%s.sh",
- icebergRESTServerHome.toString(), UUID.randomUUID());
- // Replace '/etc/krb5.conf' with the one in the test
- try {
- String content =
- FileUtils.readFileToString(new File(gravitinoRestStartShell),
StandardCharsets.UTF_8);
- content =
- content.replace(
- "#JAVA_OPTS+=\" -Djava.security.krb5.conf=/etc/krb5.conf\"",
- String.format("JAVA_OPTS+=\" -Djava.security.krb5.conf=%s\"",
krb5Path));
- File tmp = new File(modifiedGravitinoStartShell);
- FileUtils.write(tmp, content, StandardCharsets.UTF_8);
- tmp.setExecutable(true);
- LOG.info("modifiedGravitinoStartShell content: \n{}", content);
- CommandExecutor.executeCommandLocalHost(
- modifiedGravitinoStartShell + " start", false,
ProcessData.TypesOfData.OUTPUT);
- } catch (Exception e) {
- LOG.error("Can replace /etc/krb5.conf with real kerberos
configuration", e);
- }
- } else {
- String cmd = String.format("%s/bin/%s start",
icebergRESTServerHome.toString(), SCRIPT_NAME);
- CommandExecutor.executeCommandLocalHost(
- cmd,
- false,
- ProcessData.TypesOfData.OUTPUT,
- ImmutableMap.of("GRAVITINO_HOME", icebergRESTServerHome.toString()));
+ startShell = prepareKerberosStartShell(gravitinoRestStartShell,
krb5Path);
}
+
+ CommandExecutor.executeCommandLocalHost(
+ startShell + " start", false, ProcessData.TypesOfData.OUTPUT,
deployEnvironment());
return Optional.empty();
}
@Override
public void doStopIcebergRESTServer() {
String cmd = String.format("%s/bin/%s stop",
icebergRESTServerHome.toString(), SCRIPT_NAME);
- CommandExecutor.executeCommandLocalHost(
- cmd,
- false,
- TypesOfData.ERROR,
- ImmutableMap.of("GRAVITINO_HOME", icebergRESTServerHome.toString()));
+ CommandExecutor.executeCommandLocalHost(cmd, false, TypesOfData.ERROR,
deployEnvironment());
+ }
+
+ private Map<String, String> deployEnvironment() {
+ return ImmutableMap.of("GRAVITINO_HOME", icebergRESTServerHome.toString());
+ }
+
+ private String prepareKerberosStartShell(String gravitinoRestStartShell,
String krb5Path)
+ throws Exception {
+ String modifiedGravitinoStartShell =
+ String.format(
+ "%s/bin/gravitino-iceberg-rest-server_%s.sh",
+ icebergRESTServerHome.toString(), UUID.randomUUID());
+ String content =
+ FileUtils.readFileToString(new File(gravitinoRestStartShell),
StandardCharsets.UTF_8);
+ if (!content.contains(KRB5_CONF_PLACEHOLDER)) {
+ throw new IllegalStateException(
+ String.format(
+ "Failed to patch Kerberos options in %s: missing placeholder %s",
+ gravitinoRestStartShell, KRB5_CONF_PLACEHOLDER));
+ }
+ content =
+ content.replace(
+ KRB5_CONF_PLACEHOLDER,
+ String.format("JAVA_OPTS+=\"%s\"",
buildKerberosJavaOpts(krb5Path)));
+ File tmp = new File(modifiedGravitinoStartShell);
+ FileUtils.write(tmp, content, StandardCharsets.UTF_8);
+ tmp.setExecutable(true);
+ LOG.info("modifiedGravitinoStartShell content: \n{}", content);
+ return modifiedGravitinoStartShell;
+ }
+
+ private String buildKerberosJavaOpts(String krb5Path) {
+ StringBuilder opts = new StringBuilder();
+ opts.append(String.format(" -Djava.security.krb5.conf=%s", krb5Path));
+ String realm = System.getProperty(KRB5_REALM_PROPERTY);
+ if (realm != null) {
+ opts.append(String.format(" -Djava.security.krb5.realm=%s", realm));
+ }
+ String kdc = System.getProperty(KRB5_KDC_PROPERTY);
+ if (kdc != null) {
+ opts.append(String.format(" -Djava.security.krb5.kdc=%s", kdc));
+ }
+ return opts.toString();
}
}
