SinghAsDev commented on a change in pull request #4011:
URL: https://github.com/apache/iceberg/pull/4011#discussion_r814339181
##########
File path: core/src/main/java/org/apache/iceberg/hadoop/HadoopCatalog.java
##########
@@ -98,6 +98,8 @@ public HadoopCatalog() {
@Override
public void initialize(String name, Map<String, String> properties) {
+ super.initialize(name, properties);
Review comment:
Added
##########
File path: core/src/main/java/org/apache/iceberg/util/PropertyUtil.java
##########
@@ -70,4 +72,16 @@ public static String propertyAsString(Map<String, String>
properties,
}
return defaultValue;
}
+
+ public static Map<String, String> propertiesWithPrefix(Map<String, String>
properties,
+ String prefix) {
Review comment:
Thanks, that's a good point.
##########
File path: core/src/test/java/org/apache/iceberg/hadoop/TestHadoopCatalog.java
##########
@@ -547,4 +548,60 @@ private static void addVersionsToTable(Table table) {
table.newAppend().appendFile(dataFile1).commit();
table.newAppend().appendFile(dataFile2).commit();
}
+
Review comment:
thanks, added
##########
File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java
##########
@@ -214,6 +224,32 @@ private Transaction newReplaceTableTransaction(boolean
orCreate) {
return Transactions.replaceTableTransaction(identifier.toString(),
ops, metadata);
}
}
+
+ /**
+ * Get default table properties set at Catalog level through catalog
properties.
+ *
+ * @return default table properties specified in catalog properties
+ */
+ private Map<String, String> tableDefaultProperties() {
+ if (catalogProps == null || catalogProps.isEmpty()) {
+ return Collections.emptyMap();
+ }
+
+ return PropertyUtil.propertiesWithPrefix(catalogProps,
CatalogProperties.TABLE_DEFAULT_PREFIX);
+ }
+
+ /**
+ * Get table properties that are enforced at Catalog level through catalog
properties.
+ *
+ * @return default table properties enforced through catalog properties
+ */
+ private Map<String, String> tableOverrideProperties() {
+ if (catalogProps == null || catalogProps.isEmpty()) {
+ return Collections.emptyMap();
+ }
+
+ return PropertyUtil.propertiesWithPrefix(catalogProps,
CatalogProperties.TABLE_OVERRIDE_PREFIX);
+ }
Review comment:
The null and empty checks were pushed down to
`PropertyUtil.propertiesWithPrefix`, so these are not needed. Removed. These
functions can be inlined too, however, keeping them here with java doc makes it
a bit more readable. If there are strong opinion on this, I can inline these
functions too.
##########
File path:
hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java
##########
@@ -468,4 +469,84 @@ public void testUUIDinTableProperties() throws Exception {
catalog.dropTable(tableIdentifier);
}
}
+
+ @Test
+ public void testTablePropsDefaultsWithoutConflict() {
+ Schema schema = new Schema(
+ required(1, "id", Types.IntegerType.get(), "unique ID")
+ );
+ TableIdentifier tableIdent = TableIdentifier.of(DB_NAME, "tbl");
+
+ ImmutableMap<String, String> catalogProps =
ImmutableMap.of("table-default.key3", "value3",
+ "table-override.key4", "value4");
+ HiveCatalog hiveCatalog = (HiveCatalog)
CatalogUtil.loadCatalog(HiveCatalog.class.getName(),
+ CatalogUtil.ICEBERG_CATALOG_TYPE_HIVE, catalogProps, hiveConf);
+
+ try {
+ Table table = hiveCatalog.buildTable(tableIdent, schema)
+ .withProperty("key1", "value1")
+ .withProperty("key2", "value2")
+ .create();
+
+ Assert.assertEquals("value1", table.properties().get("key1"));
+ Assert.assertEquals("value2", table.properties().get("key2"));
+ Assert.assertEquals("value3", table.properties().get("key3"));
+ Assert.assertEquals("value4", table.properties().get("key4"));
+ } finally {
+ hiveCatalog.dropTable(tableIdent);
+ }
+ }
+
+ @Test
+ public void testTablePropsOverrideCatalogDefaultProps() {
+ Schema schema = new Schema(
+ required(1, "id", Types.IntegerType.get(), "unique ID")
+ );
+ TableIdentifier tableIdent = TableIdentifier.of(DB_NAME, "tbl");
+
+ ImmutableMap<String, String> catalogProps =
ImmutableMap.of("table-default.key3", "value3");
+ HiveCatalog hiveCatalog = (HiveCatalog)
CatalogUtil.loadCatalog(HiveCatalog.class.getName(),
+ CatalogUtil.ICEBERG_CATALOG_TYPE_HIVE, catalogProps, hiveConf);
+
+ try {
+ Table table = hiveCatalog.buildTable(tableIdent, schema)
+ .withProperty("key1", "value1")
+ .withProperty("key2", "value2")
+ .withProperty("key3", "value31")
+ .create();
+
+ Assert.assertEquals("value1", table.properties().get("key1"));
+ Assert.assertEquals("value2", table.properties().get("key2"));
+ Assert.assertEquals("value31", table.properties().get("key3"));
+ } finally {
+ hiveCatalog.dropTable(tableIdent);
+ }
+ }
+
+ @Test
+ public void testCatalogOverridePropsOverrideTableDefaults() {
Review comment:
I initially kept separate tests to make test failures communicate which
specific test failed. But, we can do that with assert messages as well, so
adopted this suggestion and added assert messages to explain failures.
##########
File path: core/src/main/java/org/apache/iceberg/util/PropertyUtil.java
##########
@@ -70,4 +72,16 @@ public static String propertyAsString(Map<String, String>
properties,
}
return defaultValue;
}
+
+ public static Map<String, String> propertiesWithPrefix(Map<String, String>
properties,
Review comment:
Added java doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
