codope commented on code in PR #12653:
URL: https://github.com/apache/hudi/pull/12653#discussion_r1928026570
##########
hudi-common/src/main/java/org/apache/hudi/metadata/MetadataPartitionType.java:
##########
@@ -471,6 +471,59 @@ public static MetadataPartitionType
fromPartitionPath(String partitionPath) {
throw new IllegalArgumentException("No MetadataPartitionType for partition
path: " + partitionPath);
}
+ /**
+ * Given metadata config and table config, determine whether a new secondary
index definition is required.
+ */
+ public static boolean
isNewSecondaryIndexDefinitionRequired(HoodieMetadataConfig metadataConfig,
HoodieTableMetaClient dataMetaClient) {
+ String secondaryIndexColumn = metadataConfig.getSecondaryIndexColumn();
+ if (StringUtils.isNullOrEmpty(secondaryIndexColumn)) {
+ return false;
+ }
+ // check the index definition already exists or not for this column
+ List<HoodieIndexDefinition> indexDefinitions =
getIndexDefinitions(secondaryIndexColumn, PARTITION_NAME_SECONDARY_INDEX,
dataMetaClient);
+ return indexDefinitions.isEmpty();
+ }
+
+ /**
+ * Given metadata config and table config, determine whether a new
expression index definition is required.
+ */
+ public static boolean
isNewExpressionIndexDefinitionRequired(HoodieMetadataConfig metadataConfig,
HoodieTableMetaClient dataMetaClient) {
+ String expressionIndexColumn = metadataConfig.getExpressionIndexColumn();
+ if (StringUtils.isNullOrEmpty(expressionIndexColumn)) {
+ return false;
+ }
+
+ // check that expr is present in index options
+ Map<String, String> expressionIndexOptions =
metadataConfig.getExpressionIndexOptions();
+ if (expressionIndexOptions.isEmpty()) {
+ return false;
+ }
+
+ // get all index definitions for this column and index type
+ // check if none of the index definitions has index function matching the
expression
+ List<HoodieIndexDefinition> indexDefinitions =
getIndexDefinitions(expressionIndexColumn, PARTITION_NAME_EXPRESSION_INDEX,
dataMetaClient);
+ return indexDefinitions.isEmpty()
+ || indexDefinitions.stream().noneMatch(indexDefinition ->
indexDefinition.getIndexFunction().equals(expressionIndexOptions.get(HoodieExpressionIndex.EXPRESSION_OPTION)));
Review Comment:
Yes we could, but i think it's better to keep the object equals check dumb
and compare specific attributes based on index type at the place where
filtering is required. That way, developer knows what exactly is being
compared. For instance, there could be more than one expr indexes on the same
column, however sec index is 1-1 based on column name. So, equality of index
definition differs by index type, and it is easy to remember when dev reads the
filtering code inline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
