dimas-b commented on code in PR #1532:
URL: https://github.com/apache/polaris/pull/1532#discussion_r2080644065
##########
quarkus/service/src/test/java/org/apache/polaris/service/quarkus/admin/PolarisOverlappingTableTest.java:
##########
@@ -73,10 +74,25 @@ private int createTable(TestServices services, String
location) {
static Stream<Arguments> testTableLocationRestrictions() {
Map<String, Object> laxServices =
- Map.of("ALLOW_UNSTRUCTURED_TABLE_LOCATION", "true",
"ALLOW_TABLE_LOCATION_OVERLAP", "true");
+ Map.of(
+ "ALLOW_UNSTRUCTURED_TABLE_LOCATION",
+ "true",
+ "ALLOW_TABLE_LOCATION_OVERLAP",
+ "true",
+ "ALLOW_INSECURE_STORAGE_TYPES_ACCEPTING_SECURITY_RISKS",
Review Comment:
It is set in `application-it.properties`... could we also set it in
`-test.properties` and avoid repeating here and in many other tests?
##########
polaris-core/src/main/java/org/apache/polaris/core/config/FeatureConfiguration.java:
##########
@@ -234,4 +233,38 @@ public static void enforceFeatureEnabledOrThrow(
.description("If true, the policy-store endpoints are enabled")
.defaultValue(true)
.buildFeatureConfiguration();
+
+ public static final FeatureConfiguration<Boolean>
ALLOW_SPECIFYING_FILE_IO_IMPL =
+ PolarisConfiguration.<Boolean>builder()
+ .key("ALLOW_SPECIFYING_FILE_IO_IMPL")
+ .description(
+ "Config key for whether to allow setting the FILE_IO_IMPL using
catalog properties. "
+ + "Must only be enabled in dev/test environments, never in
production systems.")
Review Comment:
IMHO, "never" is too strong. I do not think we should presume to know all
factors Polaris admins have to consider. I'd change to `It should not normally
be enabled in production systems.`
##########
service/common/src/main/java/org/apache/polaris/service/catalog/validation/IcebergPropertiesValidation.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.catalog.validation;
+
+import static
org.apache.polaris.core.config.FeatureConfiguration.ALLOW_INSECURE_STORAGE_TYPES_ACCEPTING_SECURITY_RISKS;
+import static
org.apache.polaris.core.config.FeatureConfiguration.ALLOW_SPECIFYING_FILE_IO_IMPL;
+import static
org.apache.polaris.core.config.FeatureConfiguration.SUPPORTED_CATALOG_STORAGE_TYPES;
+
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Nullable;
+import java.util.Map;
+import org.apache.iceberg.CatalogProperties;
+import org.apache.iceberg.exceptions.ValidationException;
+import org.apache.polaris.core.context.CallContext;
+import org.apache.polaris.core.storage.PolarisStorageConfigurationInfo;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class IcebergPropertiesValidation {
+ private static final Logger LOGGER =
LoggerFactory.getLogger(IcebergPropertiesValidation.class);
+
+ public static void validateIcebergProperties(
+ @Nonnull CallContext callContext, @Nonnull Map<String, String>
properties) {
+ validateIcebergProperties(callContext, properties, null);
+ }
+
+ public static String validateIcebergProperties(
Review Comment:
Since the purpose of this method is to validate and return only the FileIO
class name, how about `determineFileIoClassName(...)`
##########
service/common/src/main/java/org/apache/polaris/service/catalog/validation/IcebergPropertiesValidation.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.catalog.validation;
+
+import static
org.apache.polaris.core.config.FeatureConfiguration.ALLOW_INSECURE_STORAGE_TYPES_ACCEPTING_SECURITY_RISKS;
+import static
org.apache.polaris.core.config.FeatureConfiguration.ALLOW_SPECIFYING_FILE_IO_IMPL;
+import static
org.apache.polaris.core.config.FeatureConfiguration.SUPPORTED_CATALOG_STORAGE_TYPES;
+
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Nullable;
+import java.util.Map;
+import org.apache.iceberg.CatalogProperties;
+import org.apache.iceberg.exceptions.ValidationException;
+import org.apache.polaris.core.context.CallContext;
+import org.apache.polaris.core.storage.PolarisStorageConfigurationInfo;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class IcebergPropertiesValidation {
+ private static final Logger LOGGER =
LoggerFactory.getLogger(IcebergPropertiesValidation.class);
+
+ public static void validateIcebergProperties(
+ @Nonnull CallContext callContext, @Nonnull Map<String, String>
properties) {
+ validateIcebergProperties(callContext, properties, null);
+ }
+
+ public static String validateIcebergProperties(
+ @Nonnull CallContext callContext,
+ @Nonnull Map<String, String> properties,
+ @Nullable PolarisStorageConfigurationInfo storageConfigurationInfo) {
+ var ctx = callContext.getPolarisCallContext();
+ var configStore = ctx.getConfigurationStore();
+
+ var ioImpl = properties.get(CatalogProperties.FILE_IO_IMPL);
+ if (ioImpl != null) {
+ if (!configStore.getConfiguration(ctx, ALLOW_SPECIFYING_FILE_IO_IMPL)) {
+ throw new ValidationException(
+ "Cannot set property '%s' to '%s' for this catalog.",
+ CatalogProperties.FILE_IO_IMPL, ioImpl);
+ }
+ LOGGER.debug(
+ "Allowing overriding ioImplClassName to {} for storageConfiguration
{}",
+ ioImpl,
+ storageConfigurationInfo);
+ } else if (storageConfigurationInfo != null) {
+ ioImpl = storageConfigurationInfo.getFileIoImplClassName();
+ LOGGER.debug(
+ "Resolved ioImplClassName {} from storageConfiguration {}",
+ ioImpl,
+ storageConfigurationInfo);
+ }
+
+ if (ioImpl != null) {
+ var storageType = StorageTypeFileIO.fromFileIoImplementation(ioImpl);
+ if (storageType.validateAllowedStorageType()
+ && !configStore
+ .getConfiguration(ctx, SUPPORTED_CATALOG_STORAGE_TYPES)
+ .contains(storageType.name())) {
+ throw new ValidationException(
+ "File IO implementation '%s', as storage type '%s' is not
supported",
+ ioImpl, storageType);
+ }
+
+ if (!storageType.safe()
+ && !configStore.getConfiguration(
+ ctx, ALLOW_INSECURE_STORAGE_TYPES_ACCEPTING_SECURITY_RISKS)) {
+ throw new ValidationException(
+ "File IO implementation '%s' (storage type '%s') is considered
insecure and must not be used",
+ ioImpl, storageType);
+ }
+ }
+
+ return ioImpl;
+ }
+
+ public static boolean safeStorageType(String name) {
+ return StorageTypeFileIO.valueOf(name).safe();
Review Comment:
This is called from only one place and not in this file... Why not inline?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@polaris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
