MirrorMaker acts as a consumer+producer. So it will consume from the source topic and produce to the destination topic. That means that the destination partition is chosen using the same technique as the normal producer:
* if the source record has a key, the key will be hashed and the hash will be used to choose a partition. If the source partition was chosen using some different hashing algorithm or a custom partitioner, then you may end up writing to a different destination. * if the source record does NOT have a key, then the destination partition will be randomly chosen. MirrorMaker supports custom message handlers. You can use those to map the source partition to the destination partition, which will allow you to avoid the above two problems. Here's an example of how to do it. https://github.com/gwenshap/kafka-examples/tree/master/MirrorMakerHandler -James Sent from my iPhone > On Jun 29, 2017, at 9:57 AM, Tom Bentley <t.j.bent...@gmail.com> wrote: > > I believe so. You need to be careful that the mirror maker producer doesn't > reorder messages; in particular if retries > 0 then > max.in.flight.requests.per.connection must be 1. If > retries=0 then it doesn't matter what max.in.flight.requests.per.connection > is. > > > >> On 29 June 2017 at 05:52, Sunil Parmar <sunilosu...@gmail.com> wrote: >> >> Is it possible to configure mirror maker using message handler to preserve >> the order of messages in each topic partition. In the particular use case >> we're looking at both source and destination topics have same number of >> partitions. >> >> Sunil >>
