Hi Patrik, Thanks for the KIP!
Your motivation for this KIP is reasonable, because it is definitely possible for the ".internal" suffix to collide with real topics. It would have been nice if the original design included some mm2-specific namespace like "mm2.internal" to lessen the likelihood of a collision. However, this is a problem that has numerous existing workarounds: * Use a custom ReplicationPolicy and override the methods (for existing workloads/mirror makers) * Use non-conflicting user topic names (for new user topics) * Use the replication.policy.separator to use a non-conflicting separator character (for new mirror maker setups) And the feature as-described has significant risks attached: * May allow replication cycles and runaway replication * Mirrors internal topics that are unusable on the destination cluster While these risks can be accounted for if a user is attentive (e.g. when they're writing their own ReplicationPolicy) it is not a risk-free configuration that composes well with other out-of-the-box configurations. For example, someone may expect to take their existing configuration, turn on this new option, and expect reasonable behavior, which isn't always guaranteed. If you're still interested in this feature, please reference the existing workarounds and include them as rejected alternatives so we can know where the existing solutions fall short. We'd also have to figure out if and how the risks I mentioned could be mitigated. Thanks, Greg On Tue, Jul 30, 2024 at 5:49 AM Patrik Marton <pmar...@cloudera.com.invalid> wrote: > Hi Team, > > I would like to start a discussion on KIP-1074: Make the replication of > internal topics configurable > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1074%3A+Make+the+replication+of+internal+topics+configurable > > > > The goal of this KIP is to make it easier to replicate topics that seem > internal (for example ending in .internal suffix), but are just normal > business topics created by the user. > > I appreciate any feedback and recommendations! > > Thanks! > Patrik >
