dimas-b commented on code in PR #2762:
URL: https://github.com/apache/polaris/pull/2762#discussion_r2411296837
##########
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/DatabaseType.java:
##########
@@ -52,26 +48,15 @@ public static DatabaseType fromDisplayName(String
displayName) {
* Open an InputStream that contains data from an init script. This stream
should be closed by the
* caller.
*/
- public InputStream openInitScriptResource(@Nonnull SchemaOptions
schemaOptions) {
- if (schemaOptions.schemaFile() != null) {
- try {
- return new FileInputStream(schemaOptions.schemaFile());
- } catch (IOException e) {
- throw new IllegalArgumentException("Unable to load file " +
schemaOptions.schemaFile(), e);
- }
- } else {
- final String schemaSuffix;
- switch (schemaOptions.schemaVersion()) {
- case null -> schemaSuffix = "schema-v3.sql";
- case 1 -> schemaSuffix = "schema-v1.sql";
- case 2 -> schemaSuffix = "schema-v2.sql";
- case 3 -> schemaSuffix = "schema-v3.sql";
- default ->
- throw new IllegalArgumentException(
- "Unknown schema version " + schemaOptions.schemaVersion());
- }
- ClassLoader classLoader = DatasourceOperations.class.getClassLoader();
- return classLoader.getResourceAsStream(this.getDisplayName() + "/" +
schemaSuffix);
+ public InputStream openInitScriptResource(int schemaVersion) {
+ final String schemaSuffix;
+ switch (schemaVersion) {
+ case 1 -> schemaSuffix = "schema-v1.sql";
+ case 2 -> schemaSuffix = "schema-v2.sql";
+ case 3 -> schemaSuffix = "schema-v3.sql";
Review Comment:
nit: could be a string format call with a `Preconditions` check (simpler)
##########
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/DatabaseType.java:
##########
@@ -52,26 +48,15 @@ public static DatabaseType fromDisplayName(String
displayName) {
* Open an InputStream that contains data from an init script. This stream
should be closed by the
* caller.
*/
- public InputStream openInitScriptResource(@Nonnull SchemaOptions
schemaOptions) {
- if (schemaOptions.schemaFile() != null) {
- try {
- return new FileInputStream(schemaOptions.schemaFile());
- } catch (IOException e) {
- throw new IllegalArgumentException("Unable to load file " +
schemaOptions.schemaFile(), e);
- }
- } else {
- final String schemaSuffix;
- switch (schemaOptions.schemaVersion()) {
- case null -> schemaSuffix = "schema-v3.sql";
- case 1 -> schemaSuffix = "schema-v1.sql";
- case 2 -> schemaSuffix = "schema-v2.sql";
- case 3 -> schemaSuffix = "schema-v3.sql";
- default ->
- throw new IllegalArgumentException(
- "Unknown schema version " + schemaOptions.schemaVersion());
- }
- ClassLoader classLoader = DatasourceOperations.class.getClassLoader();
- return classLoader.getResourceAsStream(this.getDisplayName() + "/" +
schemaSuffix);
+ public InputStream openInitScriptResource(int schemaVersion) {
+ final String schemaSuffix;
+ switch (schemaVersion) {
+ case 1 -> schemaSuffix = "schema-v1.sql";
+ case 2 -> schemaSuffix = "schema-v2.sql";
+ case 3 -> schemaSuffix = "schema-v3.sql";
+ default -> throw new IllegalArgumentException("Unknown schema version "
+ schemaVersion);
}
+ ClassLoader classLoader = DatasourceOperations.class.getClassLoader();
+ return classLoader.getResourceAsStream(this.getDisplayName() + "/" +
schemaSuffix);
Review Comment:
nit: `getClass().getResource(shortName)` is preferable IMHO
##########
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcMetaStoreManagerFactory.java:
##########
@@ -154,12 +154,27 @@ public synchronized Map<String, PrincipalSecretsResult>
bootstrapRealms(
RealmContext realmContext = () -> realm;
if (!metaStoreManagerMap.containsKey(realm)) {
DatasourceOperations datasourceOperations = getDatasourceOperations();
+ int currentSchemaVersion =
+ JdbcBasePersistenceImpl.loadSchemaVersion(
+ datasourceOperations,
+ configurationStore.getConfiguration(
+ realmContext,
BehaviorChangeConfiguration.SCHEMA_VERSION_FALL_BACK_ON_DNE));
+ int requestedSchemaVersion =
JdbcBootstrapUtils.getRequestedSchemaVersion(bootstrapOptions);
+ int effectiveSchemaVersion =
+ JdbcBootstrapUtils.getRealmBootstrapSchemaVersion(
+ currentSchemaVersion,
+ requestedSchemaVersion,
+
JdbcBasePersistenceImpl.entityTableExists(datasourceOperations));
+ LOGGER.info(
+ "Effective schema version: {} for bootstrapping realm: {}",
+ effectiveSchemaVersion,
+ realm);
try {
// Run the set-up script to create the tables.
datasourceOperations.executeScript(
datasourceOperations
.getDatabaseType()
- .openInitScriptResource(bootstrapOptions.schemaOptions()));
+ .openInitScriptResource(effectiveSchemaVersion));
Review Comment:
`openInitScriptResource` no longer supports plain files... so why keep files
in `SchemaOptions`?
##########
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBootstrapUtils.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.persistence.relational.jdbc;
+
+import java.util.Optional;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import org.apache.polaris.core.persistence.bootstrap.BootstrapOptions;
+import org.apache.polaris.core.persistence.bootstrap.SchemaOptions;
+
+public class JdbcBootstrapUtils {
+
+ // Define a pattern to find 'v' followed by one or more digits (\d+)
+ private static final Pattern pattern = Pattern.compile("(v\\d+)");
+
+ private JdbcBootstrapUtils() {}
+
+ /**
+ * Determines the correct schema version to use for bootstrapping a realm.
+ *
+ * @param currentSchemaVersion The current version of the database schema.
+ * @param requiredSchemaVersion The requested schema version (-1 for
auto-detection).
+ * @param hasAlreadyBootstrappedRealms Flag indicating if any realms already
exist.
+ * @return The calculated bootstrap schema version.
+ * @throws IllegalStateException if the combination of parameters represents
an invalid state.
+ */
+ public static int getRealmBootstrapSchemaVersion(
+ int currentSchemaVersion, int requiredSchemaVersion, boolean
hasAlreadyBootstrappedRealms) {
+
+ // If versions already match, no change is needed.
+ if (currentSchemaVersion == requiredSchemaVersion) {
+ return requiredSchemaVersion;
+ }
+
+ // Handle fresh installations where no schema version is recorded (version
0).
+ if (currentSchemaVersion == 0) {
+ if (hasAlreadyBootstrappedRealms) {
+ // System was bootstrapped with v1 before schema versioning was
introduced.
+ if (requiredSchemaVersion == -1 || requiredSchemaVersion == 1) {
+ return 1;
+ }
+ } else {
+ // A truly fresh start. Default to v3 for auto-detection, otherwise
use the specified
+ // version.
+ return requiredSchemaVersion == -1 ? 3 : requiredSchemaVersion;
+ }
+ }
+
+ // Handle auto-detection on an existing installation (current version > 0).
+ if (requiredSchemaVersion == -1) {
+ // Use the current version if realms already exist; otherwise, use v3
for the new realm.
+ return hasAlreadyBootstrappedRealms ? currentSchemaVersion : 3;
+ }
+
+ // Any other combination is an unhandled or invalid migration path.
+ throw new IllegalStateException(
+ String.format(
+ "Cannot determine bootstrap schema version. Current: %d, Required:
%d, Bootstrapped: %b",
+ currentSchemaVersion, requiredSchemaVersion,
hasAlreadyBootstrappedRealms));
+ }
+
+ /**
+ * Extracts the requested schema version from the provided BootstrapOptions.
+ *
+ * @param bootstrapOptions: The bootstrap options containing schema
information from which to
+ * extract the version.
+ * @return The requested schema version, or -1 if not specified.
+ */
+ public static int getRequestedSchemaVersion(BootstrapOptions
bootstrapOptions) {
+ SchemaOptions schemaOptions = bootstrapOptions.schemaOptions();
+ if (schemaOptions != null) {
+ Optional<Integer> version = schemaOptions.schemaVersion();
+ if (version.isPresent()) {
+ return version.get();
+ }
+ Optional<String> schemaFile = schemaOptions.schemaFile();
+ if (schemaFile.isPresent()) {
+ Matcher matcher = pattern.matcher(schemaFile.get());
Review Comment:
If we have to rely on `schemaFile()` to know the schema version, I propose
to remove `schemaFile()` from `SchemaOptions` and keep only the (optional)
number.
What is the use case for `schemaFile()`? From my POV is it rather cryptic
for end users. I think allowing only built-in SQL scripts (resources) is
preferable.
##########
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBootstrapUtils.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.polaris.persistence.relational.jdbc;
+
+import java.util.Optional;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import org.apache.polaris.core.persistence.bootstrap.BootstrapOptions;
+import org.apache.polaris.core.persistence.bootstrap.SchemaOptions;
+
+public class JdbcBootstrapUtils {
+
+ // Define a pattern to find 'v' followed by one or more digits (\d+)
+ private static final Pattern pattern = Pattern.compile("(v\\d+)");
+
+ private JdbcBootstrapUtils() {}
+
+ /**
+ * Determines the correct schema version to use for bootstrapping a realm.
+ *
+ * @param currentSchemaVersion The current version of the database schema.
+ * @param requiredSchemaVersion The requested schema version (-1 for
auto-detection).
+ * @param hasAlreadyBootstrappedRealms Flag indicating if any realms already
exist.
+ * @return The calculated bootstrap schema version.
+ * @throws IllegalStateException if the combination of parameters represents
an invalid state.
+ */
+ public static int getRealmBootstrapSchemaVersion(
+ int currentSchemaVersion, int requiredSchemaVersion, boolean
hasAlreadyBootstrappedRealms) {
Review Comment:
Having the `hasAlreadyBootstrappedRealms` parameter make the logic in this
method's body hard to follow as it depends on external factors... Can we fold
`hasAlreadyBootstrappedRealms` into this method?
##########
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/DatasourceOperations.java:
##########
@@ -396,7 +400,9 @@ public boolean isConstraintViolation(SQLException e) {
}
public boolean isRelationDoesNotExist(SQLException e) {
- return RELATION_DOES_NOT_EXIST.equals(e.getSQLState());
+ return (RELATION_DOES_NOT_EXIST.equals(e.getSQLState())
+ && databaseType == DatabaseType.POSTGRES)
+ || (H2_RELATION_DOES_NOT_EXIST.equals(e.getSQLState()) && databaseType
== DatabaseType.H2);
Review Comment:
nit: this feels a bit out of place in this PR, but the change LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
