Thanks, Joey, filed jira https://issues.apache.org/jira/browse/KAFKA-333
On Wed, Apr 18, 2012 at 12:55 PM, Joel Koshy <jjkosh...@gmail.com> wrote: > > I am trying out kafka mirroring. It seems to me that I can start a kafka > > server (broker) to mirror one cluster only. So in order to mirror N > source > > clusters to one aggregate cluster, I will need to start N brokers with > each > > mirror one source cluster. Is this correct? > > > > That's right. Also, you may want to take a look at the new mirroring > mechanism which will help address your scenario. It was recently added (in > trunk) so there's not much by way of documentation apart from the code > itself. It is pretty simple: > - Start up your mirror cluster > - Run kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config > cluster1_consumer.properties --consumer.config cluster2_consumer.properties > --producer.config mirror_producer.properties --whitelist=".*" > > I'll update that mirroring wiki soon, since the embedded consumer approach > is being deprecated in favor of the stand-alone mirroring tool. > > > > Also, is there a plan to enable non-random partitioning in the build-in > > producer for mirroring? > > > > > This would be a good feature to have - can you file a jira to track it? I'd > have to think through how best to implement it (right now the partitioning > key type needs to be embedded in the code. The mirroring tool instantiates > a producer directly and assumes random partitioning.) > > > Thanks, > > Joel >
