[
https://issues.apache.org/jira/browse/BEAM-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729133#comment-16729133
]
Xu Mingmin commented on BEAM-6285:
----------------------------------
Right, it's the randomized group_id. In my case, both
\{{ConsumerConfig.GROUP_ID_CONFIG}} and \{{ConsumerConfig.CLIENT_ID_CONFIG}}
are used together for auth/auth, maybe a subset of \{{ConsumerProperties}} is
required.
> option to disable offsetConsumer in KafkaIO
> -------------------------------------------
>
> Key: BEAM-6285
> URL: https://issues.apache.org/jira/browse/BEAM-6285
> Project: Beam
> Issue Type: Improvement
> Components: io-java-kafka
> Reporter: Xu Mingmin
> Assignee: Xu Mingmin
> Priority: Major
>
> Add an option in KafkaIO.read() to avoid using `offsetConsumer`.
> *What's the problem:*
> When security is enabled, function
> `KafkaUnboundedReader.updateLatestOffsets()` always fails with lots of WARN
> log.
> *What's the cause:*
> `offsetConsumer` uses a mocked `ConsumerConfig.GROUP_ID_CONFIG`, and it
> doesn't work as expected when security is enabled. In our case,
> `ConsumerConfig.GROUP_ID_CONFIG` is the key to validate whether this consumer
> is approved.
> *What's the solution:*
> Add an option to disable `offsetConsumer` by developers;
> *What's the side effect:*
> With `offsetConsumer` disabled, backlog is unknown.
>
> [~rangadi] any comments?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
