[
https://issues.apache.org/jira/browse/KAFKA-17851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17891894#comment-17891894
]
Greg Harris commented on KAFKA-17851:
-------------------------------------
[~firebook] Thanks for the feature request.
I don't think I fully understand the motivation for this feature and the
proposed use-case. Adding a third cluster to these replication flows would make
them strictly less reliable, because now functionality will be offline when the
source, target, and MM2 are online, but the cluster hosting internal topics is
offline. If this one cluster hosts internal topics for multiple replication
flows, it becomes a single-point-of-failure for all of those replication flows
as well.
The contents of these internal topics is also only relevant when one or both
clusters is online. For example, the contents of the Offset Syncs topic is only
relevant for running the MirrorCheckpointTask, which requires that both
clusters be online to read and write offsets.
So while I understand that you could put these topics elsewhere, I don't
understand when it would be beneficial.
> Save internal topic to a separate kafka cluster
> -----------------------------------------------
>
> Key: KAFKA-17851
> URL: https://issues.apache.org/jira/browse/KAFKA-17851
> Project: Kafka
> Issue Type: Improvement
> Components: mirrormaker
> Reporter: Zhijian Chen
> Priority: Major
>
> For the mm2-offset-syncs topic there is a parameter to control its storage
> cluster: offset-syncs.topic.location, but this parameter can only control
> whether it is in the source or target cluster.
> For checkpoints topic and heartbeat topic, there is no parameter to control
> their storage clusters.
> In a multi-cluster backup scenario, we would like to store these internal
> topics in a separate kafka cluster, which has the advantage of being easy to
> manage, and we can deploy this separate cluster across three server rooms so
> that we don't have to worry about the failure of this separate cluster, which
> also solves the problem of the existing scenario, where the internal topics
> cannot be accessed due to the failure of the source or target clusters.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
