snazy commented on code in PR #4812: URL: https://github.com/apache/polaris/pull/4812#discussion_r3481182244
########## site/content/in-dev/unreleased/metastores/relational-jdbc.md: ########## @@ -25,40 +25,99 @@ weight: 100 This implementation leverages Quarkus for datasource management and supports configuration through environment variables or JVM -D flags at startup. For more information, refer to the [Quarkus configuration reference](https://quarkus.io/guides/config-reference#env-file). -We have 2 options for configuring the persistence backend: +## Named Datasources and Runtime Activation + +Polaris uses Quarkus [named datasources](https://quarkus.io/guides/datasource#multiple-datasources) to +allow selecting which database connection to use at runtime without rebuilding the application. + +Two named datasources are built into the Polaris distribution: + +| Name | Database | Suitable for | +|--------------|-----------------------------|-----------------| +| `postgresql` | PostgreSQL (or CockroachDB) | Production use | +| `h2` | H2 | Testing only | + +The `polaris.persistence.relational.jdbc.datasource` property selects which datasource Polaris +activates. Polaris automatically activates the selected datasource and deactivates all others — +**do not set `quarkus.datasource.*.active` manually**. + +### Using custom datasources + +You can also define a custom named datasource (for example, to use a different JDBC driver) by +configuring its Quarkus properties and pointing `polaris.persistence.relational.jdbc.datasource` at +it: + +```properties +polaris.persistence.type=relational-jdbc +polaris.persistence.relational.jdbc.datasource=mydb + +# Register the custom datasource with Quarkus +quarkus.datasource.mydb.db-kind=postgresql +quarkus.datasource.mydb.jdbc.url=jdbc:postgresql://myhost:5432/polaris +quarkus.datasource.mydb.username=<your-username> +quarkus.datasource.mydb.password=<your-password> +``` + +{{< alert warning >}} +The property `quarkus.datasource.<name>.db-kind` is a build-time property! If you intend to use +custom datasources, you must rebuild the Polaris server image. +{{< /alert >}} + +### CockroachDB + +CockroachDB uses the PostgreSQL wire protocol and JDBC driver, so it works with the built-in +`postgresql` named datasource. However, Polaris uses CockroachDB-specific schema DDL, so you must +set `polaris.persistence.relational.jdbc.database-type=cockroachdb` explicitly — auto-detection via +JDBC metadata is not reliable in all deployment configurations. + +```properties +polaris.persistence.type=relational-jdbc +polaris.persistence.relational.jdbc.datasource=postgresql +polaris.persistence.relational.jdbc.database-type=cockroachdb + +quarkus.datasource.postgresql.jdbc.url=jdbc:postgresql://<cockroachdb-host>:26257/<db-name>?sslmode=disable +quarkus.datasource.postgresql.username=<your-username> +quarkus.datasource.postgresql.password=<your-password> +``` + +We have 2 options for configuring the persistence backend with the built-in datasources: ## 1. Relational JDBC metastore with username and password Using environment variables: ```properties POLARIS_PERSISTENCE_TYPE=relational-jdbc +POLARIS_PERSISTENCE_RELATIONAL_JDBC_DATASOURCE=postgresql -QUARKUS_DATASOURCE_USERNAME=<your-username> -QUARKUS_DATASOURCE_PASSWORD=<your-password> -QUARKUS_DATASOURCE_JDBC_URL=<jdbc-url-of-postgres> +QUARKUS_DATASOURCE_POSTGRESQL_USERNAME=<your-username> +QUARKUS_DATASOURCE_POSTGRESQL_PASSWORD=<your-password> +QUARKUS_DATASOURCE_POSTGRESQL_JDBC_URL=<jdbc-url-of-postgres> ``` Using properties file: ```properties polaris.persistence.type=relational-jdbc -quarkus.datasource.jdbc.username=<your-username> -quarkus.datasource.jdbc.password=<your-password> -quarkus.datasource.jdbc.jdbc-url=<jdbc-url-of-postgres> +polaris.persistence.relational.jdbc.datasource=postgresql + +quarkus.datasource.postgresql.jdbc.username=<your-username> +quarkus.datasource.postgresql.jdbc.password=<your-password> +quarkus.datasource.postgresql.jdbc.jdbc-url=<jdbc-url-of-postgres> Review Comment: ```suggestion quarkus.datasource.postgresql.username quarkus.datasource.postgresql.password quarkus.datasource.postgresql.jdbc.url ``` ########## runtime/service/build.gradle.kts: ########## @@ -143,7 +148,10 @@ dependencies { testImplementation("io.quarkus:quarkus-junit-mockito") testImplementation("io.quarkus:quarkus-rest-client") testImplementation("io.quarkus:quarkus-rest-client-jackson") - testImplementation("io.quarkus:quarkus-jdbc-h2") + testImplementation("io.quarkus:quarkus-agroal") + testImplementation("io.quarkus:quarkus-jdbc-h2") { + exclude(group = "org.locationtech.jts", module = "jts-core") Review Comment: Same here ########## runtime/common/src/main/java/org/apache/polaris/quarkus/common/config/jdbc/QuarkusRelationalJdbcConfiguration.java: ########## @@ -19,7 +19,34 @@ package org.apache.polaris.quarkus.common.config.jdbc; import io.smallrye.config.ConfigMapping; +import io.smallrye.config.WithName; +import java.util.Optional; import org.apache.polaris.persistence.relational.jdbc.RelationalJdbcConfiguration; @ConfigMapping(prefix = "polaris.persistence.relational.jdbc") -public interface QuarkusRelationalJdbcConfiguration extends RelationalJdbcConfiguration {} +public interface QuarkusRelationalJdbcConfiguration extends RelationalJdbcConfiguration { + + /** The maximum number of retries before giving up the operation. */ Review Comment: Maybe follow-up work in the docs tooling. ########## runtime/service/build.gradle.kts: ########## @@ -65,7 +65,12 @@ dependencies { implementation("io.quarkus:quarkus-security") implementation("io.quarkus:quarkus-smallrye-context-propagation") implementation("io.quarkus:quarkus-smallrye-fault-tolerance") + + // JDBC backends runtimeOnly("io.quarkus:quarkus-jdbc-postgresql") + runtimeOnly("io.quarkus:quarkus-jdbc-h2") { + exclude(group = "org.locationtech.jts", module = "jts-core") Review Comment: Not excluding the dependency here? ########## runtime/defaults/src/main/resources/application-test.properties: ########## @@ -20,12 +20,20 @@ # Configuration common to ALL unit tests (executed with "test" profile – Gradle "test" tasks). # Per-test specific configuration should use QuarkusTestProfile +# Use H2 as the default persistence backend for unit tests +# (note: the test must be configured to use relational-jdbc persistence type). +polaris.persistence.relational.jdbc.datasource=h2 +# Use in-memory persistence backend for unit tests by default. +polaris.persistence.type=in-memory +quarkus.datasource.active=false +quarkus.datasource.devservices.enabled=false +quarkus.datasource.postgresql.devservices.enabled=false +quarkus.datasource.h2.devservices.enabled=false + # For unit tests, the default HTTP and management *test-ports* (8081 and 9001 respectively) must be Review Comment: Guess the port numbers are wrong (not from this PR though) ########## runtime/admin/src/main/resources/application.properties: ########## @@ -31,7 +31,12 @@ quarkus.container-image.registry=docker.io quarkus.container-image.group=apache quarkus.container-image.name=polaris-admin-tool quarkus.container-image.additional-tags=latest -quarkus.datasource.db-kind=postgresql + +# Named datasources for relational persistence backends. +# Polaris ships with many built-in datasources; by using named datasources, users can configure +# at runtime which datasources to use for relational persistence backends. +quarkus.datasource.postgresql.db-kind=postgresql Review Comment: Not sure H2 helps - at least not as long it's only in-memory. ########## CHANGELOG.md: ########## @@ -42,6 +44,7 @@ request adding CHANGELOG notes for breaking (!) changes and possibly other secti - Names containing control (invisible) characters - Names with leading or trailing whitespace - Names containing any of these characters: <code>/\:*?"<>|#+`</code> +- Due to the introduction of dynamic datasource activation, the default (PostgreSQL) datasource is now unused. If you had any custom configuration for that datasource, it should be migrated from `quarkus.datasource.*` to `quarkus.datasource.postgresql.*`. The same is valid for environment variables: `QUARKUS_DATASOURCE_*` should be replaced with `QUARKUS_DATASOURCE_POSTGRESQL_*`. Review Comment: Now that 1.6 is branches, I guess this is no longer a concern ;) ########## runtime/server/distribution/LICENSE: ########## @@ -1781,6 +1782,21 @@ License: BSD 2-Clause -------------------------------------------------------------------------------- +This product bundles H2 JDBC Driver. + +* Maven group:artifact IDs: com.h2database:h2 + +Project URL: https://www.h2database.com/ +License: Eclipse Public License 1.0 - https://www.eclipse.org/org/documents/epl-1.0/ Review Comment: Slight correction: H2 is dual-licensed: E**P**L 1.0 and MPL 2.0 - both are category "B" and fine to be included in a binary convenience artifact. ########## runtime/defaults/src/main/resources/application-test.properties: ########## @@ -20,12 +20,20 @@ # Configuration common to ALL unit tests (executed with "test" profile – Gradle "test" tasks). # Per-test specific configuration should use QuarkusTestProfile +# Use H2 as the default persistence backend for unit tests +# (note: the test must be configured to use relational-jdbc persistence type). +polaris.persistence.relational.jdbc.datasource=h2 +# Use in-memory persistence backend for unit tests by default. +polaris.persistence.type=in-memory +quarkus.datasource.active=false +quarkus.datasource.devservices.enabled=false +quarkus.datasource.postgresql.devservices.enabled=false +quarkus.datasource.h2.devservices.enabled=false + # For unit tests, the default HTTP and management *test-ports* (8081 and 9001 respectively) must be # overridden to zero, so that Quarkus will automatically select an available port. quarkus.http.test-port=0 -quarkus.management.test-port=0 Review Comment: Accidental removal? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
