[
https://issues.apache.org/jira/browse/FLINK-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542145#comment-15542145
]
Tzu-Li (Gordon) Tai edited comment on FLINK-4722 at 10/3/16 11:00 AM:
----------------------------------------------------------------------
Hi [~sudhanshulenka],
I think this is expected behaviour.
Internally, each source subtask of {{FlinkKafkaConsumer09}} uses
{{KafkaConsumer#assign()}} (no consumer group functionality) instead of
{{KafkaConsumer#subscribe()}} (has consumer group functionality) to read from
topic partitions. So that's why all 3 FlinkKafkaConsumer09 are getting all
records.
Right now, we need to internally use {{KafkaConsumer#assign()}} because
partition-to-subtask assignment must be determinate to achieve exactly-once
guarantees using Flink's checkpointing mechanism.
was (Author: tzulitai):
Hi [~sudhanshulenka],
I think this is expected behaviour.
Internally, each source subtask of {{FlinkKafkaConsumer09}} uses
{{KafkaConsumer#assign()}} (no consumer group functionality) instead of
{{KafkaConsumer#subscribe()}} (has consumer group functionality). So that's why
all 3 FlinkKafkaConsumer09 are getting all records.
Right now, we need to internally use {{KafkaConsumer#assign()}} because
partition-to-subtask assignment must be determinate to achieve exactly-once
guarantees using Flink's checkpointing mechanism.
> Consumer group concept not working properly with FlinkKafkaConsumer09
> -----------------------------------------------------------------------
>
> Key: FLINK-4722
> URL: https://issues.apache.org/jira/browse/FLINK-4722
> Project: Flink
> Issue Type: Bug
> Components: Kafka Connector
> Affects Versions: 1.1.2
> Reporter: Sudhanshu Sekhar Lenka
>
> When Kafka one Topic has 3 partition and 3 FlinkKafkaConsumer09 connected to
> that same topic using "group.id" ,"myGroup" property . Still flink consumer
> get all data which are push to each 3 partition . While it work properly
> with normal java consumer. each consumer get specific data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
