nsivabalan commented on code in PR #12529:
URL: https://github.com/apache/hudi/pull/12529#discussion_r1894047267
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -477,7 +482,16 @@ private void initializeFromFilesystem(String
initializationTime, List<MetadataPa
HoodieData<HoodieRecord> records =
fileGroupCountAndRecordsPair.getValue();
bulkCommit(instantTimeForPartition, partitionName, records,
fileGroupCount);
metadataMetaClient.reloadActiveTimeline();
-
+ if (partitionType == COLUMN_STATS) {
+ // initialize Col Stats
+ // if col stats, lets also update list of columns indexed if changed.
+ updateColumnsToIndexWithColStats(columnsToIndex);
Review Comment:
essentially, for regular commits, both mdt write and data table write has to
complete and we update the index defn.
but for initialization, its a special commit, once mdt commit is completed,
readers are eligible to read the data. and hence its better to keep the index
defn updated.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -477,7 +482,16 @@ private void initializeFromFilesystem(String
initializationTime, List<MetadataPa
HoodieData<HoodieRecord> records =
fileGroupCountAndRecordsPair.getValue();
bulkCommit(instantTimeForPartition, partitionName, records,
fileGroupCount);
metadataMetaClient.reloadActiveTimeline();
-
+ if (partitionType == COLUMN_STATS) {
+ // initialize Col Stats
+ // if col stats, lets also update list of columns indexed if changed.
+ updateColumnsToIndexWithColStats(columnsToIndex);
Review Comment:
essentially, for regular commits, both mdt write and data table write has to
complete and we update the index defn.
but for initialization, its a special commit, once mdt commit is completed,
readers are eligible to read the data. and hence its better to keep the index
defn in sync.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
