Allen Wang created FLINK-11319:
----------------------------------
Summary: Allow usage of custom implementation of Kafka Producer
and Consumer in source and sink
Key: FLINK-11319
URL: https://issues.apache.org/jira/browse/FLINK-11319
Project: Flink
Issue Type: Improvement
Components: Kafka Connector
Reporter: Allen Wang
We use our own implementation of Kafka producer and consumer in our cloud
environment for better integration with our infrastructure. The {{Consumer}}
and {{Producer}} interfaces are properly implemented, but the implementation
does not extend {{KafkaConsumer}} or {{KafkaProducer}}. Instead, it wraps and
decorates the instance of the default Kafka implementation.
I propose the following changes to make it easy to hook up our own
implementation with Flink.
* Refer to {{Consumer}} and {{Producer}} interface, not {{KafkaConsumer}} or
{{KafkaProducer}} in {{FlinkKafkaInternalProducer}} and {{KafkaConsumerThread}}
* Add {{ConsumerBuilder}} and {{ProducerBuilder}} interface with the following
definition
{code:java}
// ProducerBuilder
Producer<K, V> build(Properties properties)
// ConsumerBuilder
Consumer<byte[], byte[]> build(Properties properties){code}
* Add new constructors in {{FlinkKafkaProducer}} and {{FlinkKafkaConsumer}} to
pass in the {{ProducerBuilder}} and {{ConsumerBuilder}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
