This is an automated email from the ASF dual-hosted git repository.
adutra pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/polaris.git
The following commit(s) were added to refs/heads/main by this push:
new 854c8b033 Add documentation for the NoSQL persistence (#3818)
854c8b033 is described below
commit 854c8b0337df9cb6b3d9032c8fa6bd6fb3d9581e
Author: Alexandre Dutra <[email protected]>
AuthorDate: Thu Feb 19 18:54:55 2026 +0100
Add documentation for the NoSQL persistence (#3818)
This PR adds documentation for the NoSQL persistence.
It also splits the Metastores existing page into a new section.
It also improves the Admin Tool a little by enhancing the bootstrap
examples with more realistic commands that target real backends.
---
site/content/in-dev/unreleased/admin-tool.md | 104 +++++++++++++-----
.../content/in-dev/unreleased/metastores/_index.md | 47 ++++++++
.../in-dev/unreleased/metastores/nosql-mongodb.md | 120 +++++++++++++++++++++
.../relational-jdbc.md} | 58 +++++++---
4 files changed, 290 insertions(+), 39 deletions(-)
diff --git a/site/content/in-dev/unreleased/admin-tool.md
b/site/content/in-dev/unreleased/admin-tool.md
index a999ed670..cf91530ed 100644
--- a/site/content/in-dev/unreleased/admin-tool.md
+++ b/site/content/in-dev/unreleased/admin-tool.md
@@ -22,33 +22,30 @@ type: docs
weight: 300
---
-Polaris includes a tool for administrators to manage the metastore.
+Polaris includes a tool for administrators to manage the [metastore]({{% ref
"metastores" %}}).
-The tool must be built with the necessary JDBC drivers to access the metastore
database. For
-example, to build the tool with support for Postgres, run the following:
+The tool is available as a Docker image: `apache/polaris-admin-tool`. It can
also be downloaded as part of the [binary distribution]({{% ref
"getting-started/binary-distribution" %}}).
-```shell
-./gradlew \
- :polaris-admin:assemble \
- :polaris-admin:quarkusAppPartsBuild --rerun \
- -Dquarkus.container-image.build=true
-```
-
-The above command will generate:
-
-- One Fast-JAR in `runtime/admin/build/quarkus-app/quarkus-run.jar`
-- Two Docker images named `apache/polaris-admin-tool:latest` and
`apache/polaris-admin-tool:<version>`
+{{< alert note >}}
+The tool must be built with the necessary database drivers to access the
metastore database.
+The default build includes drivers for the PostgreSQL and NoSQL (MongoDB)
backends.
+{{< /alert >}}
## Usage
Please make sure the admin tool and Polaris server are with the same version
before using it.
To run the standalone JAR, use the following command:
+If you downloaded the [binary distribution]({{% ref
"getting-started/binary-distribution" %}}), you
+can run the admin tool as follows:
+
```shell
-java -jar runtime/admin/build/quarkus-app/quarkus-run.jar --help
+java -jar polaris-bin-<version>/admin/quarkus-run.jar --help
```
-To run the Docker image, use the following command:
+Make sure to replace `<version>` with the actual version of Polaris you are
using.
+
+To run the Docker image instead, use the following command:
```shell
docker run apache/polaris-admin-tool:latest --help
@@ -87,8 +84,17 @@ for the Polaris server. This command is idempotent and can
be run multiple times
issues. If a realm is already bootstrapped, running the `bootstrap` command
again will not have any
effect on that realm.
+If you have downloaded the [binary distribution]({{% ref
"getting-started/binary-distribution" %}}),
+you can run the `bootstrap` command as follows:
+
```shell
-java -jar runtime/admin/build/quarkus-app/quarkus-run.jar bootstrap --help
+java -jar polaris-bin-<version>/admin/quarkus-run.jar bootstrap --help
+```
+
+You can also use the Docker image to run the `bootstrap` command:
+
+```shell
+docker run apache/polaris-admin-tool:latest bootstrap --help
```
The basic usage of the `bootstrap` command is outlined below:
@@ -117,12 +123,34 @@ File Input Options:
```
For example, to bootstrap the `realm1` realm and create its root principal
credential with the
-client ID `admin` and client secret `admin`, you can run the following command:
+client ID `admin` and client secret `admin`, you can run the following
commands:
-```shell
-java -jar runtime/admin/build/quarkus-app/quarkus-run.jar bootstrap -r realm1
-c realm1,admin,admin
+Example for the PostgreSQL backend:
+
+```bash
+docker run --rm -it \
+ --env="polaris.persistence.type=relational-jdbc" \
+ --env="quarkus.datasource.username=<your-username>" \
+ --env="quarkus.datasource.password=<your-password>" \
+ --env="quarkus.datasource.jdbc.url=<jdbc-url-of-postgres>" \
+ apache/polaris-admin-tool:latest bootstrap -r realm1 -c realm1,admin,admin
+```
+
+Example for the NoSQL (MongoDB) backend:
+
+```bash
+docker run --rm -it \
+ --env="polaris.persistence.type=nosql" \
+ --env="polaris.persistence.nosql.backend=MongoDb" \
+ --env="quarkus.mongodb.database=polaris" \
+ --env="quarkus.mongodb.connection-string=<mongodb-connection-string>" \
+ apache/polaris-admin-tool:latest bootstrap -r realm1 -c realm1,admin,admin
```
+As you can see, the Polaris Admin Tool must be executed with appropriate
configuration to connect to the same database used by the Polaris server. The
configuration can be done via environment variables (as above) or system
properties.
+
+To know which configuration options you should use, read the [Metastores]({{%
ref "metastores" %}}) section and the documentation of the specific metastore
backend you are using.
+
## Purging Realms and Principal Credentials
The `purge` command is used to remove realms and principal credentials from
the Polaris server.
@@ -133,8 +161,16 @@ This includes all entities (catalogs, namespaces, tables,
views, roles), all pri
credentials, grants, and any other data associated with the realms.
{{< /alert >}}
+If you have downloaded the [binary distribution]({{% ref
"getting-started/binary-distribution" %}}), you can run the `purge` command as
follows:
+
```shell
-java -jar runtime/admin/build/quarkus-app/quarkus-run.jar purge --help
+java -jar polaris-bin-<version>/admin/quarkus-run.jar purge --help
+```
+
+You can also use the Docker image to run the `purge` command:
+
+```shell
+docker run apache/polaris-admin-tool:latest purge --help
```
The basic usage of the `purge` command is outlined below:
@@ -147,8 +183,28 @@ Purge realms and all associated entities.
-V, --version Print version information and exit.
```
-For example, to purge the `realm1` realm, you can run the following command:
+For example, to purge the `realm1` realm, you can run the following commands:
-```shell
-java -jar runtime/admin/build/quarkus-app/quarkus-run.jar purge -r realm1
+Example for the PostgreSQL backend:
+
+```bash
+docker run --rm -it \
+ --env="polaris.persistence.type=relational-jdbc" \
+ --env="quarkus.datasource.username=<your-username>" \
+ --env="quarkus.datasource.password=<your-password>" \
+ --env="quarkus.datasource.jdbc.url=<jdbc-url-of-postgres>" \
+ apache/polaris-admin-tool:latest purge -r realm1
```
+
+Example for the NoSQL (MongoDB) backend:
+
+```bash
+docker run --rm -it \
+ --env="polaris.persistence.type=nosql" \
+ --env="polaris.persistence.nosql.backend=MongoDb" \
+ --env="quarkus.mongodb.database=polaris" \
+ --env="quarkus.mongodb.connection-string=<mongodb-connection-string>" \
+ apache/polaris-admin-tool:latest purge -r realm1
+```
+
+Again, the Polaris Admin Tool must be executed with appropriate configuration
to connect to the same database used by the Polaris server. The configuration
can be done via environment variables (as above) or system properties.
diff --git a/site/content/in-dev/unreleased/metastores/_index.md
b/site/content/in-dev/unreleased/metastores/_index.md
new file mode 100644
index 000000000..aa279101a
--- /dev/null
+++ b/site/content/in-dev/unreleased/metastores/_index.md
@@ -0,0 +1,47 @@
+---
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+title: Metastores
+type: docs
+weight: 700
+---
+
+
+A metastore is the persistence backend that stores and manages catalog
metadata. This includes:
+
+- Polaris [entities](../entities): catalogs, namespaces, tables, views,
principals, roles, etc.
+- Polaris [RBAC data](../managing-security/access-control): authorization and
access control information (when using the built-in authorizer).
+- Polaris [policies](../policy): policy configurations for data governance and
security.
+
+Polaris supports three metastore implementations and persistence backends:
+
+| Backend | Type | Support Level | Description
|
+|-------------------|-------------------|---------------|-------------------------------------------------------------------------------------------------|
+| In-Memory | `in-memory` | Test only | Data is stored in
memory and lost when pods restart. Suitable for development and testing only. |
+| PostgreSQL (JDBC) | `relational-jdbc` | Ready to use | Data is stored in a
PostgreSQL database using JDBC. Recommended for production. |
+| MongoDB (NoSQL) | `nosql` | Experimental | Data is stored in a
MongoDB database. Currently in beta. |
+
+{{< alert warning >}}
+The default `in-memory` backend is **not suitable for production**. Data will
be lost when the server restarts!
+{{< /alert >}}
+
+This section explains how to configure and use Polaris with the following
backends:
+
+- [Relational JDBC](relational-jdbc)
+- [NoSQL MongoDB](nosql-mongodb)
diff --git a/site/content/in-dev/unreleased/metastores/nosql-mongodb.md
b/site/content/in-dev/unreleased/metastores/nosql-mongodb.md
new file mode 100644
index 000000000..ee84bc6f7
--- /dev/null
+++ b/site/content/in-dev/unreleased/metastores/nosql-mongodb.md
@@ -0,0 +1,120 @@
+---
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+title: NoSQL MongoDB
+type: docs
+weight: 200
+---
+
+{{< alert note >}}
+The MongoDB backend is currently in **beta**.
+{{< /alert >}}
+
+This implementation uses MongoDB as the persistence backend and leverages the
Quarkus MongoDB extension for connection management. Configuration can be done
through environment variables or JVM -D flags at startup.
+
+## Basic Configuration
+
+Using environment variables:
+
+```properties
+POLARIS_PERSISTENCE_TYPE=nosql
+POLARIS_PERSISTENCE_NOSQL_BACKEND=MongoDb
+QUARKUS_MONGODB_DATABASE=polaris
+QUARKUS_MONGODB_CONNECTION_STRING=mongodb://<username>:<password>@<host>:<port>
+```
+
+Using properties file:
+
+```properties
+polaris.persistence.type=nosql
+polaris.persistence.nosql.backend=MongoDb
+quarkus.mongodb.database=polaris
+quarkus.mongodb.connection-string=mongodb://<username>:<password>@<host>:<port>
+```
+
+## MongoDB Atlas Configuration
+
+For MongoDB Atlas deployments, use the `mongodb+srv://` connection string
format:
+
+```properties
+polaris.persistence.type=nosql
+polaris.persistence.nosql.backend=MongoDb
+quarkus.mongodb.database=polaris
+quarkus.mongodb.connection-string=mongodb+srv://<username>:<password>@<cluster>.mongodb.net/?retryWrites=true&w=majority
+```
+
+## MongoDB Replica Set Configuration
+
+For replica set deployments:
+
+```properties
+polaris.persistence.type=nosql
+polaris.persistence.nosql.backend=MongoDb
+quarkus.mongodb.database=polaris
+quarkus.mongodb.connection-string=mongodb://<username>:<password>@<host1>:<port1>,<host2>:<port2>,<host3>:<port3>/?replicaSet=<rs-name>&authSource=admin
+```
+
+## Advanced MongoDB Configuration
+
+Additional MongoDB settings can be configured via Quarkus MongoDB properties:
+
+```properties
+quarkus.mongodb.connect-timeout=10S
+quarkus.mongodb.read-timeout=30S
+quarkus.mongodb.server-selection-timeout=30S
+quarkus.mongodb.write-concern.safe=true
+quarkus.mongodb.read-preference=primaryPreferred
+```
+
+For TLS/SSL connections, add `tls=true` to the connection string:
+
+```properties
+quarkus.mongodb.connection-string=mongodb://<username>:<password>@<host>:<port>/?tls=true
+```
+
+For more details on available MongoDB configuration options, please refer to
the [Quarkus MongoDB configuration
reference](https://quarkus.io/guides/mongodb#configuration-reference).
+
+## Bootstrapping Polaris
+
+Before using Polaris with the MongoDB backend, you must bootstrap the
metastore to create the necessary collections and initial realm. This is done
using the [Admin Tool]({{% ref "../admin-tool" %}}).
+
+Using Docker:
+
+```bash
+docker run --rm -it \
+ --env="polaris.persistence.type=nosql" \
+ --env="polaris.persistence.nosql.backend=MongoDb" \
+ --env="quarkus.mongodb.database=polaris" \
+
--env="quarkus.mongodb.connection-string=mongodb://<username>:<password>@<host>:<port>"
\
+ apache/polaris-admin-tool:latest bootstrap -r <realm-name> -c
<realm-name>,<client-id>,<client-secret>
+```
+
+Using the standalone JAR:
+
+```bash
+java \
+ -Dpolaris.persistence.type=nosql \
+ -Dpolaris.persistence.nosql.backend=MongoDb \
+ -Dquarkus.mongodb.database=polaris \
+
-Dquarkus.mongodb.connection-string=mongodb://<username>:<password>@<host>:<port>
\
+ -jar polaris-admin-tool.jar bootstrap -r <realm-name> -c
<realm-name>,<client-id>,<client-secret>
+```
+
+For more details on the bootstrap command and other administrative operations,
see the [Admin Tool]({{% ref "../admin-tool" %}}) documentation.
+
diff --git a/site/content/in-dev/unreleased/metastores.md
b/site/content/in-dev/unreleased/metastores/relational-jdbc.md
similarity index 65%
rename from site/content/in-dev/unreleased/metastores.md
rename to site/content/in-dev/unreleased/metastores/relational-jdbc.md
index 6013db68b..037cb0faf 100644
--- a/site/content/in-dev/unreleased/metastores.md
+++ b/site/content/in-dev/unreleased/metastores/relational-jdbc.md
@@ -17,41 +17,40 @@
# specific language governing permissions and limitations
# under the License.
#
-title: Metastores
+title: Relational JDBC
type: docs
-weight: 700
+weight: 100
---
-This page explains how to configure and use Polaris metastores with the
recommended Relational JDBC backend.
-
-## Relational JDBC
This implementation leverages Quarkus for datasource management and supports
configuration through
environment variables or JVM -D flags at startup. For more information, refer
to the [Quarkus configuration
reference](https://quarkus.io/guides/config-reference#env-file).
We have 2 options for configuring the persistence backend:
-### 1. Relational JDBC metastore with username and password
+## 1. Relational JDBC metastore with username and password
-using environment variables:
-```
+Using environment variables:
+
+```properties
POLARIS_PERSISTENCE_TYPE=relational-jdbc
QUARKUS_DATASOURCE_USERNAME=<your-username>
QUARKUS_DATASOURCE_PASSWORD=<your-password>
QUARKUS_DATASOURCE_JDBC_URL=<jdbc-url-of-postgres>
```
-using properties file:
-```
+Using properties file:
+
+```properties
polaris.persistence.type=relational-jdbc
quarkus.datasource.jdbc.username=<your-username>
quarkus.datasource.jdbc.password=<your-password>
quarkus.datasource.jdbc.jdbc-url=<jdbc-url-of-postgres>
```
-### 2. AWS Aurora PostgreSQL metastore using IAM AWS authentication
+## 2. AWS Aurora PostgreSQL metastore using IAM AWS authentication
-```
+```properties
polaris.persistence.type=relational-jdbc
quarkus.datasource.jdbc.url=jdbc:postgresql://polaris-cluster.cluster-xyz.us-east-1.rds.amazonaws.com:6160/polaris
quarkus.datasource.jdbc.additional-jdbc-properties.wrapperPlugins=iam
@@ -66,10 +65,39 @@ quarkus.rds.credentials-provider.aws.username=dbusername
quarkus.rds.credentials-provider.aws.hostname=polaris-cluster.cluster-xyz.us-east-1.rds.amazonaws.com
quarkus.rds.credentials-provider.aws.port=6160
```
-This is the basic configuration. For more details, please refer to the
[Quarkus plugin
documentation](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-rds.html#_configuration_reference)
+
+This is the basic configuration. For more details, please refer to the
[Quarkus plugin
documentation](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-rds.html#_configuration_reference).
The Relational JDBC metastore currently relies on a Quarkus-managed datasource
and supports only PostgresSQL and H2 databases. At this time, official
documentation is provided exclusively for usage with PostgreSQL.
Please refer to the documentation here:
-[Configure data sources in Quarkus](https://quarkus.io/guides/datasource)
+[Configure data sources in Quarkus](https://quarkus.io/guides/datasource).
+
+Additionally, the retries can be configured via
`polaris.persistence.relational.jdbc.*` properties; please refer to the
[Configuring Polaris]({{% ref "../configuration" %}}) section.
+
+## Bootstrapping Polaris
+
+Before using Polaris with the Relational JDBC backend, you must bootstrap the
metastore to create the necessary schema and initial realm. This is done using
the [Admin Tool]({{% ref "../admin-tool" %}}).
+
+Using Docker:
+
+```bash
+docker run --rm -it \
+ --env="polaris.persistence.type=relational-jdbc" \
+ --env="quarkus.datasource.username=<your-username>" \
+ --env="quarkus.datasource.password=<your-password>" \
+ --env="quarkus.datasource.jdbc.url=<jdbc-url-of-postgres>" \
+ apache/polaris-admin-tool:latest bootstrap -r <realm-name> -c
<realm-name>,<client-id>,<client-secret>
+```
+
+Using the standalone JAR:
+
+```bash
+java \
+ -Dpolaris.persistence.type=relational-jdbc \
+ -Dquarkus.datasource.username=<your-username> \
+ -Dquarkus.datasource.password=<your-password> \
+ -Dquarkus.datasource.jdbc.url=<jdbc-url-of-postgres> \
+ -jar polaris-admin-tool.jar bootstrap -r <realm-name> -c
<realm-name>,<client-id>,<client-secret>
+```
-Additionally, the retries can be configured via
`polaris.persistence.relational.jdbc.*` properties please ref
[configuration]({{% ref "configuration" %}})
+For more details on the bootstrap command and other administrative operations,
see the [Admin Tool]({{% ref "../admin-tool" %}}) documentation.
