Robert Metzger created FLINK-1935:
-------------------------------------
Summary: Reimplement PersistentKafkaSource using high level Kafka
API
Key: FLINK-1935
URL: https://issues.apache.org/jira/browse/FLINK-1935
Project: Flink
Issue Type: Improvement
Components: Kafka Connector, Streaming
Affects Versions: 0.9
Reporter: Robert Metzger
Assignee: Robert Metzger
Fix For: 0.9
The current PersistentKafkaSource in Flink has some limitations that I seek to
overcome by reimplementing it using Kafka's high level API (and manually
committing the offsets to ZK).
This approach only works when the offsets are committed to ZK directly.
The current PersistentKafkaSource does not integrate with existing Kafka tools
(for example for monitoring the "lag"). All the communication with Zookeeper is
implemented manually in our current code. This is prone to errors and
inefficiencies.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
