yihua commented on code in PR #20354:
URL: https://github.com/apache/hudi/pull/20354#discussion_r4288865818
##########
hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java:
##########
@@ -1141,6 +1141,36 @@ public void testSyncWithSchema(String syncMode, String
enablePushDown) throws Ex
"The last commit that was synced should be updated in the
TBLPROPERTIES");
}
+ @ParameterizedTest
+ @MethodSource("syncMode")
+ public void testSyncWithNullTypeFields(String syncMode) throws Exception {
+ hiveSyncProps.setProperty(HIVE_SYNC_MODE.key(), syncMode);
+ String commitTime = "100";
+ HiveTestUtil.createCOWTableWithSchema(commitTime,
HoodieSchema.parse("{\"type\": \"record\", \"name\": \"User\", "
+ + "\"namespace\": \"example.avro\", \"fields\": ["
+ + "{\"name\": \"name\", \"type\": \"string\"},"
+ + "{\"name\": \"clientPromotion\", \"type\": \"null\", \"default\":
null},"
+ + "{\"name\": \"address\", \"type\": {\"type\": \"record\", \"name\":
\"Address\", \"fields\": ["
+ + "{\"name\": \"city\", \"type\": \"string\"},"
+ + "{\"name\": \"nestedPromotion\", \"type\": \"null\", \"default\":
null}]}}]}"));
+
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+ assertEquals(commitTime,
hiveClient.getLastCommitTimeSynced(HiveTestUtil.TABLE_NAME).get(),
+ "The last commit that was synced should be updated in the
TBLPROPERTIES");
+ Map<String, String> hiveSchema =
hiveClient.getMetastoreSchema(HiveTestUtil.TABLE_NAME);
+ assertEquals("string", hiveSchema.get("name").toLowerCase(Locale.ROOT));
+ assertFalse(hiveSchema.containsKey("clientpromotion"), "null type field
should not be synced");
Review Comment:
Does the case matter here, `clientpromotion` vs `clientPromotion`? Could we
use lowercase only?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
