Hello Marcin, actually KIP-382 includes LegacyReplicationPolicy (not yet released) which is similar to your PrefixlessReplicationPolicy and the MigrationReplicationPolicy mentioned. I don't think these are hacky, though maybe the documentation could better address what to do when a method doesn't necessarily apply cleanly.
Frankly, I anticipated needing to make changes to MM2 itself to accommodate these sort of use-cases -- the fact that you got this working as-is is encouraging :) Re the proposal, I think we could ship one or more of these replication policies with MM2 to support prefixless replication. There may be slightly different implementations depending on the use-case, i.e. "migration" from an old cluster vs active/passive w/o renaming topics vs mimicking MM1 with LegacyReplicationPolicy. Maybe they are all identical and we just need one? I like that use-cases are driving these requests rather than us try to enumerate all possible ReplicationPolicy implementations. Given a particular use-case and a successful application of a custom ReplicationPolicy, I for one would support including it in Apache Kafka. Ryanne On Wed, Mar 11, 2020, 5:26 AM Marcin Wieloch <marcin.wiel...@gmail.com> wrote: > Dear All, > > I would like to raise a feature request for ‘prefixless’ topic > mirroring with MirrorMaker2 (see detailed description below). > > As after quite a few tries I am not 100% sure, could you please > confirm that by now > the ‘prefixless’ mirroring is not supported and that my feature > request is valid? I would then file it in JIRA. > > Best regards, > > Marcin > > > Feature Request > > We would like to use MirrorMaker2 as a DRP solution for a simple setup > with two sites working in active-passive configuration > where all topics are always mirrored in one direction only and the > downstream topics keep their original names, i.e., without being > prefixed by MirrorMaker2. > > After a few rounds of 'trial and error' we came up with a customized > replication policy which, although hacky, seems to fulfill our need; > it is presented in stackoverflow here: > https://stackoverflow.com/a/60619233 > and source code is available here: > > https://github.com/mawileo/mm2-prefixless-policy/blob/feature/prefixless/src/main/java/ch/mawileo/kafka/mm2/PrefixlessReplicationPolicy.java > > This solution is 'hacky' in that it violates specification for the > topicSource and upstreamTopic methods as expressed in javadoc of the > ReplicationPolicy interface. > > That makes us suspect that MirrorMaker2 has been never meant to work > in such a configuration and that our solution may stop > working with future releases of Kafka. > > We would like then to ask for making the possibility of 'prefixless' > topic mirroring in a simplified two-cluster setup a standard > feature of Kafka Mirror Maker 2. >
