manojpec commented on a change in pull request #4352:
URL: https://github.com/apache/hudi/pull/4352#discussion_r796236590
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java
##########
@@ -145,22 +151,82 @@
ValidationUtils.checkArgument(!this.metadataWriteConfig.isMetadataTableEnabled(),
"File listing cannot be used for Metadata Table");
- initRegistry();
this.dataMetaClient =
HoodieTableMetaClient.builder().setConf(hadoopConf).setBasePath(dataWriteConfig.getBasePath()).build();
+ enablePartitions();
+ initRegistry();
initialize(engineContext, actionMetadata, inflightInstantTimestamp);
initTableMetadata();
} else {
enabled = false;
- this.metrics = Option.empty();
}
}
public HoodieBackedTableMetadataWriter(Configuration hadoopConf,
HoodieWriteConfig writeConfig,
- HoodieEngineContext engineContext) {
+ HoodieEngineContext engineContext) {
this(hadoopConf, writeConfig, engineContext, Option.empty(),
Option.empty());
}
+ /**
+ * Enable metadata table partitions based on config.
+ */
+ private void enablePartitions() {
+ final HoodieMetadataConfig metadataConfig =
dataWriteConfig.getMetadataConfig();
+ boolean isBootstrapCompleted = false;
+ Option<HoodieTableMetaClient> metaClient = Option.empty();
+ try {
+ isBootstrapCompleted = dataMetaClient.getFs().exists(new
Path(metadataWriteConfig.getBasePath(), HoodieTableMetaClient.METAFOLDER_NAME));
+ if (isBootstrapCompleted) {
+ metaClient =
Option.of(HoodieTableMetaClient.builder().setConf(hadoopConf.get())
+ .setBasePath(metadataWriteConfig.getBasePath()).build());
+ }
+ } catch (IOException e) {
+ throw new HoodieException("Failed to enable metadata partitions!", e);
+ }
+
+ enablePartition(MetadataPartitionType.FILES, metadataConfig, metaClient,
isBootstrapCompleted);
Review comment:
This is to align with async indexing project where users can enable
certain features/index which will inturn enable these partitions under metadata
table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
