Hi all,
I'd like you to review and comment on this new KIP on this email thread.

https://cwiki.apache.org/confluence/spaces/KAFKA/pages/451974696/KIP-1378+Allow+Consumer+Group+Renaming+in+MirrorMaker+2+Checkpoint+Connector

MirrorMaker 2 (MM2) provides pluggable topic renaming across clusters
via the `ReplicationPolicy` interface (e.g.,
`DefaultReplicationPolicy` prefixes topic names with the source
cluster alias `<source>.<topic>`, while `IdentityReplicationPolicy`
keeps them identical).

However, consumer groups do not have an equivalent pluggable naming
mechanism. Today, `MirrorCheckpointConnector` and
`MirrorCheckpointTask` mirror consumer group offsets to the target
cluster preserving the exact source consumer group ID (`group.id`).

When multiple primary clusters replicate into a single disaster
recovery (DR) or analytical target cluster, consumer group names from
different source clusters can easily collide if identical group IDs
were used independently on each source.

This KIP proposes introducing a pluggable `GroupMirroringPolicy`
interface (analogous to `ReplicationPolicy`) to allow operators to
customize consumer group names created and tracked on the target
cluster.

ciao,
Edoardo

Reply via email to