Hey Henry, this was done with MM1 at LinkedIn at one point, but it requires support for shallow iteration in KafkaConsumer, which was removed from Apache Kafka a long time ago. Given recent talk of breaking changes in Kafka 3.0, this might be a good time to revisit this.
Ryanne On Thu, May 7, 2020, 2:46 AM Henry Cai <h...@pinterest.com> wrote: > I saw this feature mentioned in the cloudera blog post: > https://blog.cloudera.com/a-look-inside-kafka-mirrormaker-2/ > > High Throughput Identity Mirroring > The batch does not need to be decompressed and compressed and deserialized > and serialized if nothing had to be changed. Identity mirroring can have a > much higher throughput than the traditional approach. This is another > feature that will be coming soon in MM2. > > Is this feature already implemented in MM2? Is there a KIP or JIRA > associated with the feature? >
