yihua commented on code in PR #19426:
URL: https://github.com/apache/hudi/pull/19426#discussion_r3688644565
##########
hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java:
##########
@@ -271,7 +272,40 @@ public Option<String>
getMetastoreLocation(IMetaStoreClient client, String datab
throw new HoodieHiveSyncException("Failed to get the metastore location
from the table " + tableName, e);
}
}
-
+
+ @ParameterizedTest
+ @MethodSource("syncMode")
+ void testForceRecreateTable(String syncMode) throws Exception {
+ hiveSyncProps.setProperty(HIVE_SYNC_MODE.key(), syncMode);
+ String instantTime = "100";
+ HiveTestUtil.createCOWTable(instantTime, 1, true);
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+ assertTrue(hiveClient.tableExists(HiveTestUtil.TABLE_NAME));
+
+ // simulate manual drift on the metastore table that a fully-synced
incremental sync would never touch
+ ddlExecutor.runSQL("ALTER TABLE `" + HiveTestUtil.TABLE_NAME + "` SET
TBLPROPERTIES ('drift_marker'='true')");
+ IMetaStoreClient client = IMetaStoreClientUtil.getMSC(getHiveConf());
+ assertTrue(client.getTable(HiveTestUtil.DB_NAME,
HiveTestUtil.TABLE_NAME).getParameters().containsKey("drift_marker"));
+
+ // nothing changed on the Hoodie timeline, so a normal sync is a no-op and
leaves the drift behind
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+ client.reconnect();
+ assertTrue(client.getTable(HiveTestUtil.DB_NAME,
HiveTestUtil.TABLE_NAME).getParameters().containsKey("drift_marker"),
+ "a no-op incremental sync must not touch the table");
+
+ // forcing recreation drops and rebuilds the table from scratch, even
though nothing else changed
+ hiveSyncProps.setProperty(META_SYNC_FORCE_RECREATE_TABLE.key(), "true");
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+ client.reconnect();
+ assertFalse(client.getTable(HiveTestUtil.DB_NAME,
HiveTestUtil.TABLE_NAME).getParameters().containsKey("drift_marker"),
+ "force-recreate must drop and recreate the table even when the
incremental sync would otherwise be a no-op");
+ assertEquals(instantTime,
hiveClient.getLastCommitTimeSynced(HiveTestUtil.TABLE_NAME).get());
Review Comment:
Done, added an assertion on the HOODIE_WRITER_VERSION table parameter.
Verified it fails without the fix and passes with it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
