Hi Olivier, No the mirrored cluster is essentially a completely different cluster. It may have a different number of partitions or servers and there is no correspondence between offsets.
-Jay On Mon, Dec 12, 2011 at 8:35 PM, Olivier Pomel <o...@datadoghq.com> wrote: > One more question: will the offsets for individual messages in the > master and mirror always be the same? In other words, would a failover > be completely transparent to consumers that may persist state linked > to specific offsets? > Thanks, > Olivier. > > On Mon, Dec 12, 2011 at 8:18 PM, Jun Rao <jun...@gmail.com> wrote: > > You can find more information in the patch of this jira: > > https://issues.apache.org/jira/browse/KAFKA-199 > > > > Thanks, > > > > Jun > > > > On Mon, Dec 12, 2011 at 11:57 AM, Carey, Lynda > > <lynda.ca...@six3systems.com>wrote: > > > >> Hi! I've been tasked with setting up a kafka cluster and mirroring. > I've > >> set up a small cluster of just two machines. I am running zookeeper, > two > >> brokers, 1 producer and 3 consumers. Each consumer is in it's own group > >> and all are reading from the same topic. So, with this setup, I'm using > >> simple console producer that reads from the command line and writes to > >> kafka. The consumer read the messages fine. > >> > >> Now, I need to set up a mirroring cluster. I've read the wiki document > >> about kafka mirroring and it leaves me a little confused. I understand > >> that the mirror cluster uses an embedded consumer to read from the > source > >> cluster and writes the messages to kafka on the mirrored cluster. > What's > >> not clear to me is what else needs to be established on the mirrored > >> cluster? Do I only need one embedded consumer and producer on the > mirror > >> to get all the messages (regardless of topic/broker/etc)? of do I need > a > >> pair of embedded consumer and producer for each topic? What is reading > the > >> messages on the mirrored cluster? -- do I need to deploy the same > consumers > >> there as on the source cluster? Is there any other documentation > regarding? > >> > >> Any information you can give me would be awesome. I'm just not getting > it > >> from the documentation alone. > >> > >> Thanks > >> _______________________________ > >> Lynda Carey, Software Engineer > >> Six3 Systems, Enterprise Systems Division > >> > >> This e-mail message is for the sole use of the intended recipient(s) and > >> may contain Six3 Systems Private or Six3 Systems Proprietary > information. > >> Any unauthorized review, use, disclosure, or distribution is > prohibited. If > >> you are not an intended recipient, please contact the sender by reply > >> e-mail and destroy all copies of the original message. > >> > >> >
