nsivabalan commented on code in PR #5293:
URL: https://github.com/apache/hudi/pull/5293#discussion_r847686429
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -379,21 +379,24 @@ protected <T extends SpecificRecordBase> void
initializeIfNeeded(HoodieTableMeta
}
// if metadata table exists, then check if any of the enabled partition
types needs to be initialized
- Set<String> inflightAndCompletedPartitions =
getInflightAndCompletedMetadataPartitions(dataMetaClient.getTableConfig());
- List<MetadataPartitionType> partitionsToInit =
this.enabledPartitionTypes.stream()
- .filter(p ->
!inflightAndCompletedPartitions.contains(p.getPartitionPath()) &&
!MetadataPartitionType.FILES.equals(p))
- .collect(Collectors.toList());
+ // NOTE: It needs to be guarded by async index config because if that is
enabled then initialization happens through the index scheduler.
+ if (!dataWriteConfig.isMetadataAsyncIndex()) {
Review Comment:
except FILES partition, wrt every other MDT partition, either it will be
synchronous or it will be be async. guess the fix ensures that right.
But tell me if this is feasible. not required to be fixed in this patch.
enable FILES via regular writer.
enable col stats via async indexer.
after sometime, enable bloom filter synchronously via regular writer? ->
From the looks of the change, guess this is not feasible I guess. or am I
missing something
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
