>> However, when I try to submit three properties files, I get a usage error (see below).
The wiki doesn't clearly state this, but it is written for Kafka v0.7 (to be released soon). The setup instructions do not work with 0.6, since in that version, the mirror didn't use a producer to write the data, it merely used internal Kafka APIs to write to the local broker. We will update the wiki to reflect that, for now, would it work for you to try this out on trunk ? Another option is using RC7 from branch kafka/branches/0.7. Thanks, Neha On Tue, Dec 13, 2011 at 9:07 AM, Carey, Lynda <lynda.ca...@six3systems.com>wrote: > So, I've read all the documents that were suggested and now I'm trying to > follow the instructions for "How to set up a Mirror" on this site: > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring. The > instructions state that you provide the broker with the 3 properties files. > However, when I try to submit three properties files, I get a usage error > (see below). > > [kafka %] JMX_PORT=8888 bin/kafka-server-start.sh > config/mirror-server.properties config/mirror-consumer.properties > config/mirror-producer.properties > > USAGE: java [options] KafkaServer server.properties [consumer.properties > > > I am using kafka-0.6, downloaded from > http://sna-projects.com/kafka/downloads.php > > > _______________________________ > Lynda Carey, Software Engineer > Six3 Systems, Enterprise Systems Division > 301-206-6000 (Office) > 410-262-4942(Cell) > > This e-mail message is for the sole use of the intended recipient(s) and > may contain Six3 Systems Private or Six3 Systems Proprietary information. > Any unauthorized review, use, disclosure, or distribution is prohibited. If > you are not an intended recipient, please contact the sender by reply > e-mail and destroy all copies of the original message. > > > ________________________________________ > From: Jay Kreps [jay.kr...@gmail.com] > Sent: Monday, December 12, 2011 11:49 PM > To: kafka-users@incubator.apache.org > Subject: Re: Kafak Mirroring Questions > > Hi Olivier, > > No the mirrored cluster is essentially a completely different cluster. It > may have a different number of partitions or servers and there is no > correspondence between offsets. > > -Jay > > On Mon, Dec 12, 2011 at 8:35 PM, Olivier Pomel <o...@datadoghq.com> wrote: > > > One more question: will the offsets for individual messages in the > > master and mirror always be the same? In other words, would a failover > > be completely transparent to consumers that may persist state linked > > to specific offsets? > > Thanks, > > Olivier. > > > > On Mon, Dec 12, 2011 at 8:18 PM, Jun Rao <jun...@gmail.com> wrote: > > > You can find more information in the patch of this jira: > > > https://issues.apache.org/jira/browse/KAFKA-199 > > > > > > Thanks, > > > > > > Jun > > > > > > On Mon, Dec 12, 2011 at 11:57 AM, Carey, Lynda > > > <lynda.ca...@six3systems.com>wrote: > > > > > >> Hi! I've been tasked with setting up a kafka cluster and mirroring. > > I've > > >> set up a small cluster of just two machines. I am running zookeeper, > > two > > >> brokers, 1 producer and 3 consumers. Each consumer is in it's own > group > > >> and all are reading from the same topic. So, with this setup, I'm > using > > >> simple console producer that reads from the command line and writes to > > >> kafka. The consumer read the messages fine. > > >> > > >> Now, I need to set up a mirroring cluster. I've read the wiki > document > > >> about kafka mirroring and it leaves me a little confused. I > understand > > >> that the mirror cluster uses an embedded consumer to read from the > > source > > >> cluster and writes the messages to kafka on the mirrored cluster. > > What's > > >> not clear to me is what else needs to be established on the mirrored > > >> cluster? Do I only need one embedded consumer and producer on the > > mirror > > >> to get all the messages (regardless of topic/broker/etc)? of do I > need > > a > > >> pair of embedded consumer and producer for each topic? What is > reading > > the > > >> messages on the mirrored cluster? -- do I need to deploy the same > > consumers > > >> there as on the source cluster? Is there any other documentation > > regarding? > > >> > > >> Any information you can give me would be awesome. I'm just not > getting > > it > > >> from the documentation alone. > > >> > > >> Thanks > > >> _______________________________ > > >> Lynda Carey, Software Engineer > > >> Six3 Systems, Enterprise Systems Division > > >> > > >> This e-mail message is for the sole use of the intended recipient(s) > and > > >> may contain Six3 Systems Private or Six3 Systems Proprietary > > information. > > >> Any unauthorized review, use, disclosure, or distribution is > > prohibited. If > > >> you are not an intended recipient, please contact the sender by reply > > >> e-mail and destroy all copies of the original message. > > >> > > >> > > >
