Robert Metzger created FLINK-4789:
-------------------------------------
Summary: Avoid Kafka partition discovery on restore and share
consumer instance for discovery and data consumption
Key: FLINK-4789
URL: https://issues.apache.org/jira/browse/FLINK-4789
Project: Flink
Issue Type: Improvement
Components: Kafka Connector
Affects Versions: 1.2.0
Reporter: Robert Metzger
As part of FLINK-4379, the Kafka partition discovery was moved from the
Constructor to the open() method. This is in general a good change, as outlined
in FLINK-4155, as it allows us to detect new partitions and topics based on
regex on the fly.
However, currently the partitions are discovered on restore as well.
Also, the {{FlinkKafkaConsumer09.getKafkaPartitions()}} is creating a separate
{{KafkaConsumer}} just for the partition discovery.
Since the partition discovery happens on the task managers now, we can use the
regular {{KafkaConsumer}} instance, which is used for data retrieval as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
