[
https://issues.apache.org/jira/browse/FLINK-12548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041932#comment-17041932
]
Seth Wiesman commented on FLINK-12548:
--------------------------------------
When you say multiple instances, does that mean multiple slots running on one
TM part of a single job or multiple jobs running on one cluster consuming from
the same topic?
Is the missing information in the kafka-consumer-groups report breaking
something or just unexpected?
As Alijoscha mentioned, the way Flink works with Kafka differently than most
everything else. My biggest concern is reporting back information that might
diverge from what is in Flink. While it might not be immediately obvious to
most users why certain fields are not reported I think that is more honest then
showing information that differs from what the running job is actually doing.
> FlinkKafkaConsumer issues configuring underlying KafkaConsumer
> --------------------------------------------------------------
>
> Key: FLINK-12548
> URL: https://issues.apache.org/jira/browse/FLINK-12548
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: 1.8.0
> Reporter: Leonid Ilyevsky
> Assignee: Hugo Da Cruz Louro
> Priority: Major
> Attachments: KafkaError.txt
>
>
> I observe two issues that are possibly related, or at least belong to the
> same part of the code.
> I am running multiple instances of pipelines (by setting parallelism) under
> Flink cluster.
> First problem I see is the "Error registering AppInfo mbean", see attached
> stack trace [^KafkaError.txt] . From the trace I understand that it tries to
> use consumer group id as the bean name; obviously this fails as I have
> multiple instances using the same group id under the same task manager.
> Second problem is, the client id is not set at all (I do provide the
> "client.id" property when invoking FlinkKafkaConsumer constructor). This
> creates problems tracking consumers on Kafka side. On the other hand, I
> obviously do not want to set the same client id on all instances - preferably
> Flink should add maybe some UUID suffix to it or something to make it unique.
> The same unique name could be used for that mbean, this way solving the first
> problem.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
