nsivabalan commented on code in PR #13831:
URL: https://github.com/apache/hudi/pull/13831#discussion_r2320674001
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java:
##########
@@ -1862,14 +1862,14 @@ public void testPayloadClassUpdate() throws Exception {
assertEquals(metaClient.getTableConfig().getPayloadClass(),
DefaultHoodieRecordPayload.class.getName());
//now create one more deltaStreamer instance and update payload class
- cfg = TestHelpers.makeConfig(dataSetBasePath,
WriteOperationType.BULK_INSERT,
+ HoodieDeltaStreamer.Config updatedConfig =
TestHelpers.makeConfig(dataSetBasePath, WriteOperationType.BULK_INSERT,
Collections.singletonList(SqlQueryBasedTransformer.class.getName()),
PROPS_FILENAME_TEST_SOURCE, false,
true, true, DummyAvroPayload.class.getName(), "MERGE_ON_READ");
- new HoodieDeltaStreamer(cfg, jsc, fs, hiveServer.getHiveConf());
-
- // NOTE: Payload class cannot be updated.
- metaClient = HoodieTableMetaClient.reload(metaClient);
- assertEquals(metaClient.getTableConfig().getPayloadClass(),
DefaultHoodieRecordPayload.class.getName());
+ Exception e = assertThrows(HoodieException.class, () -> {
+ new HoodieDeltaStreamer(updatedConfig, jsc, fs,
hiveServer.getHiveConf());
+ }, "Should error out when payload class is switched");
+ assertTrue(e.getMessage().contains("Config conflict(key"));
+
assertTrue(e.getMessage().contains(HoodieTableConfig.PAYLOAD_CLASS_NAME.key()));
Review Comment:
can we enhance this for merge mode and merge strategy id as well?
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/HoodieStreamer.java:
##########
@@ -747,6 +747,7 @@ public StreamSyncService(Config cfg,
HoodieSparkEngineContext hoodieSparkContext
cfg.baseFileFormat = "PARQUET"; // default for backward
compatibility
}
Map<String, String> propsToValidate = new HashMap<>();
+
properties.get().setProperty(HoodieTableConfig.PAYLOAD_CLASS_NAME.key(),
cfg.payloadClassName);
Review Comment:
can we also add other merge properties to this list.
`hoodie.write.record.merge.mode`
and `hoodie.write.record.merge.strategy.id`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
