Jake Maes created SAMZA-1481:
--------------------------------
Summary: KafkaSystemConsumer does not properly track highwatermark
in some cases.
Key: SAMZA-1481
URL: https://issues.apache.org/jira/browse/SAMZA-1481
Project: Samza
Issue Type: Bug
Reporter: Jake Maes
Assignee: Jake Maes
Fix For: 0.14.0
The KafkaSystemConsumer currently only updates the isAtHighWatermark when it
receives a payload from Kafka.
This does not work when:
1. The topic is empty
2. The oldest or newest offset changes on the broker
3. An invalid initial offset is used for the consumer and the Kafka auto reset
policy kicks in.
The isAtHighWatermark flag is crucial for topics that we bootstrap (checkpoint,
changelog, coordinator) because we poll infinitely until we're at head of
stream. And if this flag is wrong, we poll indefinitely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
