> I am trying out kafka mirroring. It seems to me that I can start a kafka > server (broker) to mirror one cluster only. So in order to mirror N source > clusters to one aggregate cluster, I will need to start N brokers with each > mirror one source cluster. Is this correct? >
That's right. Also, you may want to take a look at the new mirroring mechanism which will help address your scenario. It was recently added (in trunk) so there's not much by way of documentation apart from the code itself. It is pretty simple: - Start up your mirror cluster - Run kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config cluster1_consumer.properties --consumer.config cluster2_consumer.properties --producer.config mirror_producer.properties --whitelist=".*" I'll update that mirroring wiki soon, since the embedded consumer approach is being deprecated in favor of the stand-alone mirroring tool. > Also, is there a plan to enable non-random partitioning in the build-in > producer for mirroring? > > This would be a good feature to have - can you file a jira to track it? I'd have to think through how best to implement it (right now the partitioning key type needs to be embedded in the code. The mirroring tool instantiates a producer directly and assumes random partitioning.) Thanks, Joel
