the-other-tim-brown commented on code in PR #9743:
URL: https://github.com/apache/hudi/pull/9743#discussion_r1351220251
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/StreamSync.java:
##########
@@ -661,6 +652,35 @@ private Pair<SchemaProvider, Pair<String,
JavaRDD<HoodieRecord>>> fetchFromSourc
return Pair.of(schemaProvider, Pair.of(checkpointStr, records));
}
+ /**
+ * Apply schema reconcile and schema evolution rules(schema on read) and
generate new target schema provider.
+ *
+ * @param incomingSchema schema of the source data
+ * @param sourceSchemaProvider Source schema provider.
+ * @return the SchemaProvider that can be used as writer schema.
+ */
+ private SchemaProvider getDeducedSchemaProvider(Schema incomingSchema,
SchemaProvider sourceSchemaProvider) {
+ Option<Schema> latestTableSchemaOpt =
UtilHelpers.getLatestTableSchema(hoodieSparkContext.jsc(), fs,
cfg.targetBasePath);
Review Comment:
this UtilHelpers is also creating a new MetaClient so we should avoid that
call
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
