parisni commented on code in PR #8743:
URL: https://github.com/apache/hudi/pull/8743#discussion_r1464900587
##########
hudi-aws/src/main/java/org/apache/hudi/aws/sync/AWSGlueCatalogSyncClient.java:
##########
@@ -432,6 +443,120 @@ public void createTable(String tableName,
}
}
+ /**
+ * This will manage partitions indexes. Users can activate/deactivate them
on existing tables.
+ * Removing index definition, will result in dropping the index.
+ * <p>
+ * reference doc for partition indexes:
+ *
https://docs.aws.amazon.com/glue/latest/dg/partition-indexes.html#partition-index-getpartitions
+ *
+ * @param tableName
+ */
+ public void managePartitionIndexes(String tableName) throws
ExecutionException, InterruptedException {
+ if (!config.getBooleanOrDefault(META_SYNC_PARTITION_INDEX_FIELDS_ENABLE)) {
+ // deactivate indexing if enabled
+ if (getPartitionIndexEnable(tableName)) {
+ LOG.warn("Deactivating partition indexing");
Review Comment:
yes. The suggestion to use moto to mock aws glue is great. However it does
not support partition index right now. So moto should be considered as a basis
for IT in the hudi-aws, but not in this PR.
BTW I tested this and provided a python script for people to try this out
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
