wombatu-kun commented on code in PR #19426:
URL: https://github.com/apache/hudi/pull/19426#discussion_r3688293186
##########
hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java:
##########
@@ -271,7 +272,40 @@ public Option<String>
getMetastoreLocation(IMetaStoreClient client, String datab
throw new HoodieHiveSyncException("Failed to get the metastore location
from the table " + tableName, e);
}
}
-
+
+ @ParameterizedTest
+ @MethodSource("syncMode")
+ void testForceRecreateTable(String syncMode) throws Exception {
+ hiveSyncProps.setProperty(HIVE_SYNC_MODE.key(), syncMode);
+ String instantTime = "100";
+ HiveTestUtil.createCOWTable(instantTime, 1, true);
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+ assertTrue(hiveClient.tableExists(HiveTestUtil.TABLE_NAME));
+
+ // simulate manual drift on the metastore table that a fully-synced
incremental sync would never touch
+ ddlExecutor.runSQL("ALTER TABLE `" + HiveTestUtil.TABLE_NAME + "` SET
TBLPROPERTIES ('drift_marker'='true')");
+ IMetaStoreClient client = IMetaStoreClientUtil.getMSC(getHiveConf());
+ assertTrue(client.getTable(HiveTestUtil.DB_NAME,
HiveTestUtil.TABLE_NAME).getParameters().containsKey("drift_marker"));
+
+ // nothing changed on the Hoodie timeline, so a normal sync is a no-op and
leaves the drift behind
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+ client.reconnect();
+ assertTrue(client.getTable(HiveTestUtil.DB_NAME,
HiveTestUtil.TABLE_NAME).getParameters().containsKey("drift_marker"),
+ "a no-op incremental sync must not touch the table");
+
+ // forcing recreation drops and rebuilds the table from scratch, even
though nothing else changed
+ hiveSyncProps.setProperty(META_SYNC_FORCE_RECREATE_TABLE.key(), "true");
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+ client.reconnect();
+ assertFalse(client.getTable(HiveTestUtil.DB_NAME,
HiveTestUtil.TABLE_NAME).getParameters().containsKey("drift_marker"),
+ "force-recreate must drop and recreate the table even when the
incremental sync would otherwise be a no-op");
+ assertEquals(instantTime,
hiveClient.getLastCommitTimeSynced(HiveTestUtil.TABLE_NAME).get());
Review Comment:
testForceRecreateTable asserts only that the drift marker is gone, so a
recreate that drops the partitions or fails to republish hudi_writer_version
would still pass. Add assertions on hiveClient.getAllPartitions and on the
HOODIE_WRITER_VERSION table parameter after the force recreate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
