linliu-code commented on code in PR #13424:
URL: https://github.com/apache/hudi/pull/13424#discussion_r2302396356
##########
hudi-aws/src/main/java/org/apache/hudi/config/GlueCatalogSyncClientConfig.java:
##########
@@ -97,4 +103,21 @@ public class GlueCatalogSyncClientConfig extends
HoodieConfig {
.markAdvanced()
.withDocumentation("Glue sync may fail if the Glue table exists with
partitions differing from the Hoodie table or if schema evolution is not
supported by Glue."
+ "Enabling this configuration will drop and create the table to
match the Hoodie config");
+
+ public static final ConfigProperty<String> GLUE_SYNC_DATABASE_NAME =
ConfigProperty
+ .key(GLUE_CLIENT_PROPERTY_PREFIX + "database_name")
+ .noDefaultValue()
+ .withInferFunction(cfg ->
Option.ofNullable(cfg.getString(META_SYNC_DATABASE_NAME.key()))
+ .or(() -> Option.of(cfg.getStringOrDefault(DATABASE_NAME,
DATABASE_NAME.defaultValue()))))
Review Comment:
`DATABASE_NAME.defaultValue()`: the default value is null, therefore, will
`Option.of()` should throw NPE. Will it be expected?
##########
hudi-aws/src/test/java/org/apache/hudi/aws/sync/TestAWSGlueSyncClient.java:
##########
@@ -259,6 +267,50 @@ void testUpdateTableProperties() throws
ExecutionException, InterruptedException
assertThrows(HoodieGlueSyncException.class, () ->
awsGlueSyncClient.updateTableProperties(tableName, newTableProperties));
}
+ @Test
+ void testTableAndDatabaseName() {
+ assertEquals(GlueTestUtil.DB_NAME, awsGlueSyncClient.getDatabaseName());
+ assertEquals(GlueTestUtil.TABLE_NAME, awsGlueSyncClient.getTableName());
+
+ String dbName = "test_db1";
+ String tableName = "test_table1";
+ Properties properties = new Properties();
+
properties.setProperty(GlueCatalogSyncClientConfig.GLUE_SYNC_DATABASE_NAME.key(),
dbName);
+
properties.setProperty(GlueCatalogSyncClientConfig.GLUE_SYNC_TABLE_NAME.key(),
tableName);
+
+ HiveSyncConfig hiveSyncConfig = new HiveSyncConfig(properties);
+ awsGlueSyncClient = new AWSGlueCatalogSyncClient(mockAwsGlue,
hiveSyncConfig, GlueTestUtil.getMetaClient());
+ assertEquals(dbName, awsGlueSyncClient.getDatabaseName());
+ assertEquals(tableName, awsGlueSyncClient.getTableName());
+
+ dbName = "test_db2";
+ tableName = "test_table2";
+ properties = new Properties();
+ properties.setProperty(META_SYNC_DATABASE_NAME.key(), dbName);
+ properties.setProperty(META_SYNC_TABLE_NAME.key(), tableName);
+
+ hiveSyncConfig = new HiveSyncConfig(properties);
+ awsGlueSyncClient = new AWSGlueCatalogSyncClient(mockAwsGlue,
hiveSyncConfig, GlueTestUtil.getMetaClient());
+ assertEquals(dbName, awsGlueSyncClient.getDatabaseName());
+ assertEquals(tableName, awsGlueSyncClient.getTableName());
+
+ dbName = "test_db3";
+ tableName = "test_table3";
+ properties = new Properties();
+ properties.setProperty(DATABASE_NAME.key(), dbName);
+ properties.setProperty(HOODIE_TABLE_NAME_KEY, tableName);
+
+ hiveSyncConfig = new HiveSyncConfig(properties);
+ awsGlueSyncClient = new AWSGlueCatalogSyncClient(mockAwsGlue,
hiveSyncConfig, GlueTestUtil.getMetaClient());
+ assertEquals(dbName, awsGlueSyncClient.getDatabaseName());
+ assertEquals(tableName, awsGlueSyncClient.getTableName());
+
+ hiveSyncConfig = new HiveSyncConfig(new Properties());
+ awsGlueSyncClient = new AWSGlueCatalogSyncClient(mockAwsGlue,
hiveSyncConfig, GlueTestUtil.getMetaClient());
+ assertEquals(META_SYNC_DATABASE_NAME.defaultValue(),
awsGlueSyncClient.getDatabaseName());
+ assertEquals(META_SYNC_TABLE_NAME.defaultValue(),
awsGlueSyncClient.getTableName());
Review Comment:
for the dabase or table the value could be null. When it happen, should it
be safer to throw instead of return null?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
