Awesome, this will be a huge advancement. I also want to point out that this KIP implements MirrorSinkConnector as well, finally, which is a very often requested missing feature in my experience.
Ryanne On Fri, Aug 21, 2020, 9:45 AM Ning Zhang <ning2008w...@gmail.com> wrote: > Hello, I wrote a KIP about MirrorMaker2 Exactly-once Semantics (EOS) > https://cwiki.apache.org/confluence/display/KAFKA/KIP-656%3A+MirrorMaker2+Exactly-once+Semantics > At the high-level, it resembles the idea of how HDFS Sink Connector > achieves EOS across clusters by managing and storing the consumer offsets > in an external persistent storage, but also leverages the current Kafka EOS > guarantee within a single cluster. I have done some experiments especially > for the failure cases and I am very appreciated for comments and feedback > on this KIP from bigger audience. >
