codope commented on code in PR #12288:
URL: https://github.com/apache/hudi/pull/12288#discussion_r1852295680
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/EightToSevenDowngradeHandler.java:
##########
@@ -79,33 +107,46 @@ public Map<ConfigProperty, String>
downgrade(HoodieWriteConfig config, HoodieEng
UpgradeDowngradeUtils.syncCompactionRequestedFileToAuxiliaryFolder(table);
HoodieTableMetaClient metaClient =
HoodieTableMetaClient.builder().setConf(context.getStorageConf().newInstance()).setBasePath(config.getBasePath()).build();
- List<HoodieInstant> instants =
metaClient.getActiveTimeline().getInstants();
+ List<HoodieInstant> instants = new ArrayList<>();
+ try {
+ // We need to move all the instants - not just completed ones.
+ instants =
metaClient.scanHoodieInstantsFromFileSystem(metaClient.getTimelinePath(),
+ ActiveTimelineV2.VALID_EXTENSIONS_IN_ACTIVE_TIMELINE, false);
+ } catch (IOException ioe) {
+ LOG.error("Failed to get instants from filesystem", ioe);
+ throw new HoodieIOException("Failed to get instants from filesystem",
ioe);
+ }
+
if (!instants.isEmpty()) {
InstantFileNameGenerator instantFileNameGenerator =
metaClient.getInstantFileNameGenerator();
CommitMetadataSerDeV2 commitMetadataSerDeV2 = new
CommitMetadataSerDeV2();
CommitMetadataSerDeV1 commitMetadataSerDeV1 = new
CommitMetadataSerDeV1();
ActiveTimelineV1 activeTimelineV1 = new ActiveTimelineV1(metaClient);
- String tmpFilePrefix = "temp_commit_file_for_eight_to_seven_downgrade_";
context.map(instants, instant -> {
String fileName = instantFileNameGenerator.getFileName(instant);
+ // Rename the metadata file name from the
${instant_time}_${completion_time}.action[.state] format in version 1.x to the
${instant_time}.action[.state] format in version 0.x.
+ StoragePath fromPath = new
StoragePath(TIMELINE_LAYOUT_V2.getTimelinePath(metaClient.getBasePath()),
fileName);
Review Comment:
Don't we also need to handle timeline in `SevenToEightUpgradeHandler` i.e.
move the instants to `.hoodie/timeline`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
