[
https://issues.apache.org/jira/browse/FLINK-7195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091262#comment-16091262
]
ASF GitHub Bot commented on FLINK-7195:
---------------------------------------
Github user tzulitai commented on the issue:
https://github.com/apache/flink/pull/4344
For this change, I think we also need to verify how the consumers behave
when some restored partition is no longer reachable. (since previously, no
longer reachable partitions will be filtered out on restore, but that has a bad
side effect of dropping state).
> FlinkKafkaConsumer should not respect fetched partitions to filter restored
> partition states
> --------------------------------------------------------------------------------------------
>
> Key: FLINK-7195
> URL: https://issues.apache.org/jira/browse/FLINK-7195
> Project: Flink
> Issue Type: Bug
> Components: Kafka Connector
> Affects Versions: 1.3.1
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Tzu-Li (Gordon) Tai
> Priority: Blocker
> Fix For: 1.3.2
>
>
> This issue is a re-appearance of FLINK-6006. On restore, we should not
> respect any fetched partitions list from Kafka and perform any filtering of
> the restored partition states. There are corner cases where, due to Kafka
> broker downtime, some partitions may be missing in the fetched partition
> list. To be more precise, we actually should not require fetching partitions
> on restore.
> We've stepped on our own foot again and reintroduced this bug in
> https://github.com/apache/flink/pull/3378/commits/ed68fedbe90db03823d75a020510ad3c344fa73e.
> The previous test for this behavior was too implementation specific, and
> therefore the leak in catching this on different internal implementations.
> We should have a proper unit test for this that does not rely on the internal
> implementations and test only on public abstractions of
> {{FlinkKafkaConsumerBase}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
