nsivabalan commented on a change in pull request #3836:
URL: https://github.com/apache/hudi/pull/3836#discussion_r745261515
##########
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java
##########
@@ -83,22 +83,31 @@ public SparkRDDWriteClient(HoodieEngineContext context,
HoodieWriteConfig client
@Deprecated
public SparkRDDWriteClient(HoodieEngineContext context, HoodieWriteConfig
writeConfig, boolean rollbackPending) {
- super(context, writeConfig);
+ this(context, writeConfig, Option.empty());
}
@Deprecated
public SparkRDDWriteClient(HoodieEngineContext context, HoodieWriteConfig
writeConfig, boolean rollbackPending,
Option<EmbeddedTimelineService> timelineService) {
- super(context, writeConfig, timelineService);
+ this(context, writeConfig, timelineService);
}
public SparkRDDWriteClient(HoodieEngineContext context, HoodieWriteConfig
writeConfig,
Option<EmbeddedTimelineService> timelineService) {
super(context, writeConfig, timelineService);
+ bootstrapMetadataTable(Option.empty());
+ }
+
+ private void bootstrapMetadataTable(Option<String>
instantInProgressTimestamp) {
Review comment:
not sure if we can call it as bootstrap metadata table. this is
essentially initializing metadata table. internal impl could do bootstrap if
applicable.
##########
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java
##########
@@ -95,10 +95,19 @@ public SparkRDDWriteClient(HoodieEngineContext context,
HoodieWriteConfig writeC
public SparkRDDWriteClient(HoodieEngineContext context, HoodieWriteConfig
writeConfig,
Option<EmbeddedTimelineService> timelineService) {
super(context, writeConfig, timelineService);
+ bootstrapMetadataTable();
+ }
+
+ private void bootstrapMetadataTable() {
if (config.isMetadataTableEnabled()) {
- // If the metadata table does not exist, it should be bootstrapped here
- // TODO: Check if we can remove this requirement - auto bootstrap on
commit
-
SparkHoodieBackedTableMetadataWriter.create(context.getHadoopConf().get(),
config, context);
+ // Defer bootstrap if upgrade / downgrade is pending
+ HoodieTableMetaClient metaClient = createMetaClient(true);
+ UpgradeDowngrade upgradeDowngrade = new UpgradeDowngrade(
+ metaClient, config, context,
SparkUpgradeDowngradeHelper.getInstance());
+ if
(!upgradeDowngrade.needsUpgradeOrDowngrade(HoodieTableVersion.current())) {
Review comment:
thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
