[
https://issues.apache.org/jira/browse/SPARK-31961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132725#comment-17132725
]
Gunjan Kumar commented on SPARK-31961:
--------------------------------------
spark will not be tied to kafka version as these property keys never changes in
any kafka version.
In the current scenarion, suppose i want to set poll timeout but i dont know
the property name for same then i have to go to kafka doc search for that
property and paste it in Option().
This is not how a programmer should code :)
> Add a class in spark with all Kafka configuration key available as string
> -------------------------------------------------------------------------
>
> Key: SPARK-31961
> URL: https://issues.apache.org/jira/browse/SPARK-31961
> Project: Spark
> Issue Type: New Feature
> Components: SQL, Structured Streaming
> Affects Versions: 2.4.6
> Reporter: Gunjan Kumar
> Priority: Minor
> Labels: kafka, sql, structured-streaming
>
> Add a class in spark with all Kafka configuration key available as string.
> see the highligted class which i want.
> eg:-
> Current code:-
> val df_cluster1 = spark
> .read
> .format("kafka")
> .option("kafka.bootstrap.servers","cluster1_host:cluster1_port)
> .option("subscribe", "topic1")
> Expected code:-
> val df_cluster1 = spark
> .read
> .format("kafka")
> .option(*KafkaConstantClass*.KAFKA_BOOTSTRAP_SERVERS,"cluster1_host:cluster1_port)
> .option(*KafkaConstantClass*.subscribe, "topic1")
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
