codope commented on code in PR #5293:
URL: https://github.com/apache/hudi/pull/5293#discussion_r847898428
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -379,21 +379,24 @@ protected <T extends SpecificRecordBase> void
initializeIfNeeded(HoodieTableMeta
}
// if metadata table exists, then check if any of the enabled partition
types needs to be initialized
- Set<String> inflightAndCompletedPartitions =
getInflightAndCompletedMetadataPartitions(dataMetaClient.getTableConfig());
- List<MetadataPartitionType> partitionsToInit =
this.enabledPartitionTypes.stream()
- .filter(p ->
!inflightAndCompletedPartitions.contains(p.getPartitionPath()) &&
!MetadataPartitionType.FILES.equals(p))
- .collect(Collectors.toList());
+ // NOTE: It needs to be guarded by async index config because if that is
enabled then initialization happens through the index scheduler.
+ if (!dataWriteConfig.isMetadataAsyncIndex()) {
Review Comment:
It should work. I mean the bloom_filters partition should be initialized and
updated by the regular writer. I also tested the following scenario. Let me
know if this is not what you meant.
1. First deltacommit by regular writer with default configs (so only files
partition created).
2. indexing commit by indexer only for column_stats -> now there is both
files and column_stats partition.
3. Second deltacommit by regular writer with all indexes enabled (i.e. both
column_stats and bloom_filters) -> now this initializes the bloom_filter
partition as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
