[
https://issues.apache.org/jira/browse/KAFKA-13365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjin Lee updated KAFKA-13365:
--------------------------------
Description:
As of present, MirrorMaker 2 (aka MM2) 's client configurtaion feature has some
problems:
# The replication-level client configuration works only to the common
properties like {{bootstrap.servers}}, {{security.protocol}}, ssl, sasl, etc;
that is, a configuration like {{'A→B.producer.batch.size'}} is ignored.
## Also, which admin client is affected by the replication-level configuration
like A→B.admin.retry.backoff.ms is unclear; MM2 uses two admin clients for both
upstream and downstream clusters, respectively.
# MM2 is based on Kafka Connect framework's connector; Since MM2 Connectors
({{MirrorSourceConnector}}, {{MirrorCheckpointConnector}}, and
{{MirrorHeartbeatConnector}}) are source connectors, they use producer instance
created by Kafka Connector, which uses {{'producer.override.\{property-name}'}}
in connector configuration; But, {{'target.producer.\{property-name}'}} are not
automatically applied to {{'producer.override.\{property-name}'}} so not
actually applied to producer instance.
# MM2 requires to define the {{'bootstrap.servers'}} of the clusters in
cluster-level, like {{'A.bootstrap.servers'}} or {{'B.bootstrap.servers'}}; but
it also allows to override them in cluster-level and replication-level configs,
like {{'A.producer.bootstrap.servers'}} or
{{'A→B.consumer.bootstrap.servers'}}; actually these configurations are not
used but, it would be better to ignore it and give a warning.
was:
As of present, the producer-specific options or consumer-only settings are not
configurable in MirrorMaker 2, dislike to the documentation.
{{us-east.producer.acks = all // ignored}}
{{us-west.consumer.max.poll.interval.ms = 120000 // also ignored}}
> Improve MirrorMaker2's client configuration
> -------------------------------------------
>
> Key: KAFKA-13365
> URL: https://issues.apache.org/jira/browse/KAFKA-13365
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
> Reporter: Dongjin Lee
> Assignee: Dongjin Lee
> Priority: Critical
> Labels: needs-kip
>
> As of present, MirrorMaker 2 (aka MM2) 's client configurtaion feature has
> some problems:
> # The replication-level client configuration works only to the common
> properties like {{bootstrap.servers}}, {{security.protocol}}, ssl, sasl, etc;
> that is, a configuration like {{'A→B.producer.batch.size'}} is ignored.
> ## Also, which admin client is affected by the replication-level
> configuration like A→B.admin.retry.backoff.ms is unclear; MM2 uses two admin
> clients for both upstream and downstream clusters, respectively.
> # MM2 is based on Kafka Connect framework's connector; Since MM2 Connectors
> ({{MirrorSourceConnector}}, {{MirrorCheckpointConnector}}, and
> {{MirrorHeartbeatConnector}}) are source connectors, they use producer
> instance created by Kafka Connector, which uses
> {{'producer.override.\{property-name}'}} in connector configuration; But,
> {{'target.producer.\{property-name}'}} are not automatically applied to
> {{'producer.override.\{property-name}'}} so not actually applied to producer
> instance.
> # MM2 requires to define the {{'bootstrap.servers'}} of the clusters in
> cluster-level, like {{'A.bootstrap.servers'}} or {{'B.bootstrap.servers'}};
> but it also allows to override them in cluster-level and replication-level
> configs, like {{'A.producer.bootstrap.servers'}} or
> {{'A→B.consumer.bootstrap.servers'}}; actually these configurations are not
> used but, it would be better to ignore it and give a warning.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
