Saisai Shao created SPARK-5718:
----------------------------------
Summary: Change to native offset management for
ReliableKafkaReceiver
Key: SPARK-5718
URL: https://issues.apache.org/jira/browse/SPARK-5718
Project: Spark
Issue Type: Improvement
Components: Streaming
Affects Versions: 1.3.0
Reporter: Saisai Shao
Kafka 0.8.2 supports native offsets management instead of ZK, this will get
better performance, for now in ReliableKafkaReceiver, we rely on ZK to manage
the offsets, this potentially will be a bottleneck if the injection rate is
high (once per 200ms by default), so here in order to get better performance as
well as keeping consistent with Kafka, add native offset management for
ReliableKafkaReceiver.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
