[
https://issues.apache.org/jira/browse/KAFKA-17200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Egerton updated KAFKA-17200:
----------------------------------
Labels: needs-kip (was: )
> Enable MM2 to replicate topics ending in "internal" suffix
> ----------------------------------------------------------
>
> Key: KAFKA-17200
> URL: https://issues.apache.org/jira/browse/KAFKA-17200
> Project: Kafka
> Issue Type: Improvement
> Components: mirrormaker
> Reporter: Patrik Márton
> Assignee: Patrik Márton
> Priority: Minor
> Labels: needs-kip
>
> In the current Mirror Maker 2 implementation, topics ending in ".internal" or
> "-internal" cannot be replicated as they are considered connect/mm2 internal
> topics.
> In some cases, users have business topics ending in ".internal" or
> "-internal" that are excluded from the replication for the same reason. This
> is because of two things:
> (1) The ReplicationPolicy interface excludes all topics ending in ".internal"
> or "-internal" from the replication, as they are considered internal connect
> topics:
> {code:java}
> /** Internal topics are never replicated. */
> default boolean isInternalTopic(String topic) {
> boolean isKafkaInternalTopic = topic.startsWith("__") ||
> topic.startsWith(".");
> boolean isDefaultConnectTopic = topic.endsWith("-internal") ||
> topic.endsWith(".internal");
> return isMM2InternalTopic(topic) || isKafkaInternalTopic ||
> isDefaultConnectTopic;
> } {code}
> (2) The DefaultTopicFilter has the following default exclude regular
> expression:
> {code:java}
> ".*[\\-\\.]internal, .*\\.replica, __.*" {code}
> The goal of this ticket is to enable the replication of such business topics,
> while making sure that kafka internal topics are not replicated.
> *Solution 1:* The DefaultTopicFilter can be configured to have a different
> exclude list with more specific regex to exclude all kafka internal topics,
> but include business internal topics, eg.:
> {code:java}
> "mm2.*[\\-\\.]internal, .*\\.replica, __.*"; {code}
> As these topics are explicitly excluded in the ReplicationPolicy interface
> too, we can have a new replication policy from DefaultReplicationPolicy, that
> preserves the same behavior, but overrides the isInternalTopic() method in a
> way that topics ending with the internal suffix are not considered internal
> topics. Obviously in this case it would be important to set up the topic
> filter correctly to avoid replicating kafka internal topics.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
