FANNG1 commented on code in PR #6036:
URL: https://github.com/apache/gravitino/pull/6036#discussion_r1904795429
##########
spark-connector/spark-common/src/main/java/org/apache/gravitino/spark/connector/iceberg/IcebergPropertiesConverter.java:
##########
@@ -49,7 +49,21 @@ public Map<String, String>
toSparkCatalogProperties(Map<String, String> properti
Preconditions.checkArgument(
StringUtils.isNotBlank(catalogBackend),
String.format("%s should not be empty",
IcebergConstants.CATALOG_BACKEND));
- all.put(IcebergPropertiesConstants.ICEBERG_CATALOG_TYPE, catalogBackend);
+ if (catalogBackend.equalsIgnoreCase(
+ IcebergPropertiesConstants.ICEBERG_CATALOG_BACKEND_CUSTOM)) {
+ String catalogBackendImpl =
all.remove(IcebergConstants.CATALOG_BACKEND_IMPL);
+ Preconditions.checkArgument(
+ StringUtils.isNotBlank(catalogBackendImpl),
+ String.format(
+ "%s should not be empty when %s is %s",
+ IcebergConstants.CATALOG_BACKEND_IMPL,
+ IcebergConstants.CATALOG_BACKEND,
+ IcebergPropertiesConstants.ICEBERG_CATALOG_BACKEND_CUSTOM));
+ all.put(IcebergPropertiesConstants.ICEBERG_CATALOG_IMPL,
catalogBackendImpl);
Review Comment:
The reason why defining `IcebergPropertiesConstants.ICEBERG_CATALOG_IMPL`
not using `CatalogProperties.CATALOG_IMPL` directly is to define the properties
used by Gravitino centrally. so the other developers could find and use them
easily.
##########
spark-connector/spark-common/src/main/java/org/apache/gravitino/spark/connector/iceberg/IcebergPropertiesConstants.java:
##########
@@ -58,6 +59,8 @@ public class IcebergPropertiesConstants {
static final String GRAVITINO_ICEBERG_CATALOG_BACKEND_REST = "rest";
+ public static final String ICEBERG_CATALOG_BACKEND_CUSTOM = "custom";
Review Comment:
We have already defined `custom` catalog backend in `IcebergCatalogBackend`,
could you do a refactor to moving `IcebergCatalogBackend` to
`catalogs/catalog-common` module, so the spark connector could reuse the class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
