nsivabalan commented on code in PR #5543:
URL: https://github.com/apache/hudi/pull/5543#discussion_r871257552
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java:
##########
@@ -829,6 +831,25 @@ private HoodieWriteConfig getHoodieClientConfig(Schema
schema) {
return config;
}
+ private Schema getSchemaForWriteConfig(Schema targetSchema) {
+ Schema newWriteSchema = targetSchema;
+ try {
+ if (targetSchema != null) {
+ // check if targetSchema is equal to NULL schema
+ if (SchemaCompatibility.checkReaderWriterCompatibility(targetSchema,
InputBatch.NULL_SCHEMA).getType() ==
SchemaCompatibility.SchemaCompatibilityType.COMPATIBLE
+ &&
SchemaCompatibility.checkReaderWriterCompatibility(InputBatch.NULL_SCHEMA,
targetSchema).getType() ==
SchemaCompatibility.SchemaCompatibilityType.COMPATIBLE) {
+ // target schema is null. fetch schema from commit metadata and use
it
+ HoodieTableMetaClient meta =
HoodieTableMetaClient.builder().setConf(new
Configuration(fs.getConf())).setBasePath(cfg.targetBasePath).setPayloadClassName(cfg.payloadClassName).build();
Review Comment:
yeah. I thought about that. this happens only incase of empty batches from
the source, on which case anyways there is not much load on the system in
general. So, resolved to using a local one. If the scope of usage of metaclient
increases in future, we can add it as instance variable. wdyt.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
