hudi-agent commented on code in PR #18572:
URL: https://github.com/apache/hudi/pull/18572#discussion_r3291894529
##########
hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/functional/TestHoodieBackedMetadata.java:
##########
@@ -4395,4 +4396,114 @@ private static Properties
getDisabledRowWriterProperties() {
properties.setProperty("hoodie.datasource.write.row.writer.enable",
String.valueOf(false));
return properties;
}
+
+ /**
+ * Verifies that {@code deleteMdtIfNecessaryBeforeRestore} returns true and
deletes the MDT when
+ * the restore target is older than the oldest completed MDT compaction.
+ */
+ @Test
+ public void
testDeleteMdtIfNecessaryBeforeRestoreDeletesMdtForOlderTimestamp() throws
Exception {
+ init(HoodieTableType.COPY_ON_WRITE);
+ HoodieWriteConfig cfg = getWriteConfigBuilder(true, true, false)
+ .withMetadataConfig(HoodieMetadataConfig.newBuilder()
+ .enable(true)
+ .enableMetrics(false)
+ .build())
+ .build();
+
+ try (SparkRDDWriteClient client = getHoodieWriteClient(cfg)) {
+ String firstCommitTime = WriteClientTestUtils.createNewInstantTime();
+ List<HoodieRecord> records = dataGen.generateInserts(firstCommitTime,
100);
+ WriteClientTestUtils.startCommitWithTime(client, firstCommitTime);
+ assertNoWriteErrors(client.insert(jsc.parallelize(records, 1),
firstCommitTime).collect());
+
+ String secondCommitTime = WriteClientTestUtils.createNewInstantTime();
+ records = dataGen.generateInserts(secondCommitTime, 100);
+ WriteClientTestUtils.startCommitWithTime(client, secondCommitTime);
+ assertNoWriteErrors(client.insert(jsc.parallelize(records, 1),
secondCommitTime).collect());
+
+ // Explicitly compact the MDT so there is at least one completed
compaction.
+ metadataWriter =
SparkHoodieBackedTableMetadataWriter.create(storageConf, cfg, context);
+ Properties mdtProps = ((SparkHoodieBackedTableMetadataWriter)
metadataWriter).getWriteConfig().getProps();
+ mdtProps.setProperty(INLINE_COMPACT_NUM_DELTA_COMMITS.key(), "1");
+ HoodieWriteConfig mdtWriteConfig =
HoodieWriteConfig.newBuilder().withProperties(mdtProps).build();
+ try (SparkRDDWriteClient mdtClient = new SparkRDDWriteClient(context,
mdtWriteConfig, Option.empty())) {
+ Option<String> compactionInstOpt =
mdtClient.scheduleCompaction(Option.empty());
+ assertTrue(compactionInstOpt.isPresent(), "MDT compaction should
schedule after 2 data commits");
+ mdtClient.commitCompaction(compactionInstOpt.get(),
mdtClient.compact(compactionInstOpt.get()), Option.empty());
+ }
+
+ HoodieTableMetaClient mdtMetaClient =
HoodieTableMetaClient.builder().setConf(storageConf)
+
.setBasePath(HoodieTableMetadata.getMetadataTableBasePath(metaClient.getBasePath())).build();
+ assertTrue(mdtMetaClient.getActiveTimeline()
+
.getTimelineOfActions(CollectionUtils.createSet(COMMIT_ACTION)).countInstants()
>= 1,
+ "Expected at least 1 MDT compaction");
+
+ // firstCommitTime is before any MDT compaction —
deleteMdtIfNecessaryBeforeRestore must
+ // detect the inconsistency, delete the MDT, and return true.
+ boolean mdtDeleted =
client.deleteMdtIfNecessaryBeforeRestore(firstCommitTime);
Review Comment:
🤖 nit:
`HoodieTableMetadata.getMetadataTableBasePath(metaClient.getBasePath())` is
computed twice here and again in the second test — could you extract it to a
`String mdtBasePath` local variable near the top of each test body? It would
make the assertions a bit easier to scan.
<sub><i>- AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
