codope commented on code in PR #12288:
URL: https://github.com/apache/hudi/pull/12288#discussion_r1852493612
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/EightToSevenDowngradeHandler.java:
##########
@@ -79,33 +107,46 @@ public Map<ConfigProperty, String>
downgrade(HoodieWriteConfig config, HoodieEng
UpgradeDowngradeUtils.syncCompactionRequestedFileToAuxiliaryFolder(table);
HoodieTableMetaClient metaClient =
HoodieTableMetaClient.builder().setConf(context.getStorageConf().newInstance()).setBasePath(config.getBasePath()).build();
- List<HoodieInstant> instants =
metaClient.getActiveTimeline().getInstants();
+ List<HoodieInstant> instants = new ArrayList<>();
+ try {
+ // We need to move all the instants - not just completed ones.
+ instants =
metaClient.scanHoodieInstantsFromFileSystem(metaClient.getTimelinePath(),
+ ActiveTimelineV2.VALID_EXTENSIONS_IN_ACTIVE_TIMELINE, false);
+ } catch (IOException ioe) {
+ LOG.error("Failed to get instants from filesystem", ioe);
+ throw new HoodieIOException("Failed to get instants from filesystem",
ioe);
+ }
+
if (!instants.isEmpty()) {
InstantFileNameGenerator instantFileNameGenerator =
metaClient.getInstantFileNameGenerator();
CommitMetadataSerDeV2 commitMetadataSerDeV2 = new
CommitMetadataSerDeV2();
CommitMetadataSerDeV1 commitMetadataSerDeV1 = new
CommitMetadataSerDeV1();
ActiveTimelineV1 activeTimelineV1 = new ActiveTimelineV1(metaClient);
- String tmpFilePrefix = "temp_commit_file_for_eight_to_seven_downgrade_";
context.map(instants, instant -> {
String fileName = instantFileNameGenerator.getFileName(instant);
+ // Rename the metadata file name from the
${instant_time}_${completion_time}.action[.state] format in version 1.x to the
${instant_time}.action[.state] format in version 0.x.
+ StoragePath fromPath = new
StoragePath(TIMELINE_LAYOUT_V2.getTimelinePath(metaClient.getBasePath()),
fileName);
Review Comment:
I have handled it in my branch -
https://github.com/codope/hudi/commit/bdb38a8e260556ae487e1261a76621642e350ced
You can either use this commit or i can fix it after this PR gets merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
