danny0405 commented on code in PR #7687:
URL: https://github.com/apache/hudi/pull/7687#discussion_r1125417796
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/debezium/DebeziumSource.java:
##########
@@ -86,21 +90,28 @@ public DebeziumSource(TypedProperties props,
JavaSparkContext sparkContext,
deserializerClassName =
props.getString(DataSourceWriteOptions.KAFKA_AVRO_VALUE_DESERIALIZER_CLASS().key(),
DataSourceWriteOptions.KAFKA_AVRO_VALUE_DESERIALIZER_CLASS().defaultValue());
+ // Currently, debezium source requires Confluent/Kafka schema-registry to
fetch the latest schema.
+ if (schemaProvider == null || !(schemaProvider instanceof
SchemaRegistryProvider)) {
+ schemaRegistryProvider = new SchemaRegistryProvider(props, sparkContext);
+ schemaProvider = schemaRegistryProvider;
+ } else {
+ schemaRegistryProvider = (SchemaRegistryProvider) schemaProvider;
+ }
+
try {
props.put(NATIVE_KAFKA_VALUE_DESERIALIZER_PROP,
Class.forName(deserializerClassName).getName());
+ if
(deserializerClassName.equals(KafkaAvroSchemaDeserializer.class.getName())) {
+ if (schemaProvider == null) {
+ throw new HoodieIOException("SchemaProvider has to be set to use
KafkaAvroSchemaDeserializer");
+ }
+ props.put(KAFKA_AVRO_VALUE_DESERIALIZER_SCHEMA,
schemaProvider.getSourceSchema().toString());
+ }
Review Comment:
Let's try to add some tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
