codope commented on code in PR #12653:
URL: https://github.com/apache/hudi/pull/12653#discussion_r1922212709
##########
hudi-common/src/main/java/org/apache/hudi/metadata/MetadataPartitionType.java:
##########
@@ -471,6 +476,47 @@ public static MetadataPartitionType
fromPartitionPath(String partitionPath) {
throw new IllegalArgumentException("No MetadataPartitionType for partition
path: " + partitionPath);
}
+ /**
+ * Given metadata config and table config, determine whether a new secondary
index definition is required.
+ */
+ public static boolean
isNewSecondaryIndexDefinitionRequired(HoodieMetadataConfig metadataConfig,
HoodieTableMetaClient dataMetaClient) {
+ String secondaryIndexColumn = metadataConfig.getSecondaryIndexColumn();
+ if (StringUtils.isNullOrEmpty(secondaryIndexColumn)) {
+ return false;
+ }
+ // check the index definition already exists or not for this column
+ return dataMetaClient.getIndexMetadata().isEmpty() ||
!isIndexDefinitionPresentForColumn(secondaryIndexColumn,
PARTITION_NAME_SECONDARY_INDEX, dataMetaClient);
+ }
+
+ /**
+ * Given metadata config and table config, determine whether a new
expression index definition is required.
+ */
+ public static boolean
isNewExpressionIndexDefinitionRequired(HoodieMetadataConfig metadataConfig,
HoodieTableMetaClient dataMetaClient) {
+ String expressionIndexColumn = metadataConfig.getExpressionIndexColumn();
+ if (StringUtils.isNullOrEmpty(expressionIndexColumn)) {
+ return false;
+ }
+
+ // check that expr is present in index options
+ Map<String, String> expressionIndexOptions =
metadataConfig.getExpressionIndexOptions();
+ if (expressionIndexOptions.isEmpty()) {
+ return false;
+ }
+
+ String expression =
expressionIndexOptions.get(HoodieExpressionIndex.EXPRESSION_OPTION);
+ if (StringUtils.isNullOrEmpty(expression)) {
+ return false;
+ }
+
+ // check the index definition already exists or not for this expression
+ return dataMetaClient.getIndexMetadata().isEmpty() ||
!isIndexDefinitionPresentForColumn(expressionIndexColumn,
PARTITION_NAME_EXPRESSION_INDEX_PREFIX, dataMetaClient);
+ }
+
+ private static boolean isIndexDefinitionPresentForColumn(String
indexedColumn, String indexType, HoodieTableMetaClient dataMetaClient) {
+ return dataMetaClient.getIndexMetadata().isPresent() &&
dataMetaClient.getIndexMetadata().get().getIndexDefinitions().values().stream()
+ .anyMatch(indexDefinition ->
indexDefinition.getSourceFields().contains(indexedColumn) &&
indexDefinition.getIndexType().equals(indexType));
Review Comment:
I have changed the logic. Now, we first get all the index definitions for
the same indexed column and index type. Then check for expression. Those
methods are still in this enum. PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
