rahil-c commented on code in PR #13642:
URL: https://github.com/apache/hudi/pull/13642#discussion_r2254812184
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/UpgradeDowngrade.java:
##########
@@ -275,4 +303,42 @@ protected Pair<Map<ConfigProperty, String>,
List<ConfigProperty>> downgrade(Hood
throw new HoodieUpgradeDowngradeException(fromVersion.versionCode(),
toVersion.versionCode(), false);
}
}
+
+ /**
+ * Checks if any handlers in the upgrade/downgrade path require running
rollback and compaction before starting process.
+ *
+ * @param fromVersion the current table version
+ * @param toVersion the target table version
+ */
+ private void performRollbackAndCompactionIfRequired(HoodieTableVersion
fromVersion, HoodieTableVersion toVersion, boolean isUpgrade) {
Review Comment:
I think I originally tried this recommendation from @yihua and saw several
failures in the respective test upgrade/downgrade classes. Hence why I
restricted this to run for only the specific upgrade/downgrade handlers that
were calling it.
##########
hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/table/upgrade/TestUpgradeDowngrade.java:
##########
@@ -153,6 +156,42 @@ public void testAutoUpgradeDisabled(HoodieTableVersion
originalVersion) throws E
LOG.info("Auto-upgrade disabled test passed for version {}",
originalVersion);
}
+ @ParameterizedTest
+ @MethodSource("metadataTableCorruptionTestVersionPairs")
+ public void testMetadataTableUpgradeDowngradeFailure(HoodieTableVersion
fromVersion, HoodieTableVersion toVersion) throws Exception {
+ boolean isUpgrade = fromVersion.lesserThan(toVersion);
+ String operation = isUpgrade ? "upgrade" : "downgrade";
+ LOG.info("Testing metadata table failure during {} from version {} to {}",
operation, fromVersion, toVersion);
+
+ HoodieTableMetaClient originalMetaClient = loadFixtureTable(fromVersion);
+ assertEquals(fromVersion,
originalMetaClient.getTableConfig().getTableVersion(),
+ "Fixture table should be at expected version");
+
+ HoodieWriteConfig cfg = createWriteConfig(originalMetaClient, true);
+
+ String metadataTablePath = HoodieTableMetadata.getMetadataTableBasePath(
+ originalMetaClient.getBasePath().toString());
+ StoragePath metadataHoodiePath = new StoragePath(metadataTablePath,
HoodieTableMetaClient.METAFOLDER_NAME);
+ StoragePath propsPath = new StoragePath(metadataHoodiePath,
HoodieTableConfig.HOODIE_PROPERTIES_FILE);
+ StoragePath backupPropsPath = new StoragePath(metadataHoodiePath,
HoodieTableConfig.HOODIE_PROPERTIES_FILE_BACKUP);
+
+ String corruptedContent =
"CORRUPTED_INVALID_CONTENT\n\nTHIS_IS_NOT_VALID_PROPERTIES_FORMAT";
+ try (OutputStream propsOut =
originalMetaClient.getStorage().create(propsPath, true);
+ OutputStream backupOut =
originalMetaClient.getStorage().create(backupPropsPath, true)) {
+ propsOut.write(corruptedContent.getBytes());
+ backupOut.write(corruptedContent.getBytes());
+ }
+
+ HoodieUpgradeDowngradeException exception = assertThrows(
+ HoodieUpgradeDowngradeException.class,
+ () -> new UpgradeDowngrade(originalMetaClient, cfg, context(),
SparkUpgradeDowngradeHelper.getInstance())
+ .run(toVersion, null)
+ );
+
+ LOG.info("Successfully caught expected exception during {} from {} to {}:
{}",
+ operation, fromVersion, toVersion, exception.getMessage());
Review Comment:
Can do this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
