[
https://issues.apache.org/jira/browse/HUDI-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17373786#comment-17373786
]
ASF GitHub Bot commented on HUDI-2069:
--------------------------------------
nsivabalan commented on a change in pull request #3172:
URL: https://github.com/apache/hudi/pull/3172#discussion_r663248903
##########
File path:
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala
##########
@@ -381,7 +381,6 @@ object DataSourceWriteOptions {
val DEFAULT_ASYNC_COMPACT_ENABLE_OPT_VAL = "true"
// Avro Kafka Source configs
- val KAFKA_AVRO_VALUE_DESERIALIZER =
"hoodie.deltastreamer.source.kafka.value.deserializer.class"
- // Schema to be used in custom kakfa deserializer
- val KAFKA_AVRO_VALUE_DESERIALIZER_SCHEMA =
"hoodie.deltastreamer.source.kafka.value.deserializer.schema"
+ val KAFKA_AVRO_VALUE_DESERIALIZER_OPT_KEY =
"hoodie.deltastreamer.source.kafka.value.deserializer.class"
Review comment:
nit: can we reuse the property prefix variable here as well?
KAFKA_AVRO_VALUE_DESERIALIZER_PROPERTY_PREFIX
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/AvroKafkaSource.java
##########
@@ -60,7 +66,7 @@ public AvroKafkaSource(TypedProperties props,
JavaSparkContext sparkContext, Spa
super(props, sparkContext, sparkSession, schemaProvider);
props.put(NATIVE_KAFKA_KEY_DESERIALIZER_PROP, StringDeserializer.class);
- String deserializerClassName =
props.getString(DataSourceWriteOptions.KAFKA_AVRO_VALUE_DESERIALIZER(), "");
+ String deserializerClassName =
props.getString(KAFKA_AVRO_VALUE_DESERIALIZER_OPT_KEY(),
DEFAULT_KAFKA_AVRO_VALUE_DESERIALIZER_OPT_VAL());
Review comment:
I am not sure I get you. Can you clarify please.
As of now, it looks fine, since the default value is empty string. I am
talking about when we make the new deser class as default. We can come back to
revisit this when we fix the default value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Fix KafkaAvroSchemaDeserializer to not rely on reflection
> ---------------------------------------------------------
>
> Key: HUDI-2069
> URL: https://issues.apache.org/jira/browse/HUDI-2069
> Project: Apache Hudi
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: sivabalan narayanan
> Priority: Major
> Labels: pull-request-available
>
> h1. Fix KafkaAvroSchemaDeserializer to not rely on reflection
>
> Instantiation is failing since jssc is required to instantiate
> schemaprovider.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
