[
https://issues.apache.org/jira/browse/FLINK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17545390#comment-17545390
]
Salva edited comment on FLINK-27872 at 6/2/22 10:02 AM:
--------------------------------------------------------
Based on a quick experiment, it seems that using a custom subscriber requires
to create your own fork for these two classes:
- `KafkaSourceBuilder`
- `KafkaSource`
The rest of the connector seems reusable, but that is still far from ideal.
was (Author: JIRAUSER287051):
Based on a quick experiment, it seems that using a custom subscriber requires
to create your own fork for these two:
- `KafkaSourceBuilder`
-`KafkaSource`
The rest of the connector seems reusable, but that is still far from ideal
(although better than what I originally thought).
> Allow KafkaBuilder to set arbitrary subscribers
> -----------------------------------------------
>
> Key: FLINK-27872
> URL: https://issues.apache.org/jira/browse/FLINK-27872
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kafka
> Reporter: Salva
> Priority: Major
>
> Currently,
> [KafkaBuilder|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java]
> has two setters:
> *
> [setTopics|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L157]
> *
> [setTopicPattern|https://github.com/apache/flink/blob/586715f23ef49939ab74e4736c58d71c643a64ba/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java#L168]
> which under the hood instantiate the corresponding (concrete) subscribers.
> This covers the most common needs, I agree, but it might fall short in some
> cases. Why not add a more generic setter:
> * {{setSubscriber (???)}}
> Otherwise, how can users read from kafka in combination with custom
> subscribing logic? Without looking much into it, it seems that they basically
> cannot, at least without having to replicate some parts of the connector,
> which seems rather inconvenient.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
