Tzu-Li (Gordon) Tai created FLINK-7195:
------------------------------------------
Summary: FlinkKafkaConsumer should not respect fetched partitions
to filter restored partition states
Key: FLINK-7195
URL: https://issues.apache.org/jira/browse/FLINK-7195
Project: Flink
Issue Type: Bug
Components: Kafka Connector
Affects Versions: 1.3.1
Reporter: Tzu-Li (Gordon) Tai
Assignee: Tzu-Li (Gordon) Tai
Priority: Blocker
Fix For: 1.3.2
This issue is a re-appearance of FLINK-6006. On restore, we should not respect
any fetched partitions list from Kafka and perform any filtering of the
restored partition states. There are corner cases where, due to Kafka broker
downtime, some partitions may be missing in the fetched partition list. To be
more precise, we actually should not require fetching partitions on restore.
We've stepped on our own foot again and reintroduced this bug in
https://github.com/apache/flink/pull/3378/commits/ed68fedbe90db03823d75a020510ad3c344fa73e.
The previous test for this behavior was too implementation specific, and
therefore the leak in catching this on different internal implementations.
We should have a proper unit test for this that does not rely on the internal
implementations and test only on public abstractions of
{{FlinkKafkaConsumerBase}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
