Thanks for the superb document Joel. I may not have communicated very well, but even with mirrormaker, the mirroring is achieved through a consumer. This assumes that the message is persisted on the source Kafka broker. If the source broker dies before flushing the messages to disk, won't we lose those messages? I am sure I'm missing something here.
Thanks! -neelesh On Tue, May 22, 2012 at 10:46 AM, Joel Koshy <jjkosh...@gmail.com> wrote: > Thanks! In the meanwhile, do you think simply modifying the producer to > > log the same message to multiple brokers (which are not part of a > cluster) > > is a reasonably good solution to reduce the probability of lost > messages? I > > have seen the mirroring documentation, but it sounded like the mirroring > > consumer was embedded inside the broker, and hence a dead broker will > mean > > losing unflushed messages. > > > > The embedded consumer approach is being deprecated. I added a new mirroring > doc outlining the new approach which uses a separate tool to do mirroring. > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring+(MirrorMaker) > > However, mirroring is orthogonal to your use case. i.e., it does not quite > address the ability to react to producer failures (e.g., during a bounce of > a broker) which will be available in 0.8. > > Thanks, > > Joel > > > On Fri, May 18, 2012 at 3:26 PM, Neha Narkhede <neha.narkh...@gmail.com > > >wrote: > > > > > Hi Neelesh, > > > > > > Thank you for your interest in Kafka ! > > > > > > We are shooting for end of summer this year for the 0.8 release. You > can > > > track the progress of this feature here - > > > https://issues.apache.org/jira/browse/KAFKA-50 > > > > > > Thanks, > > > Neha > > > > > > > > > On Fri, May 18, 2012 at 3:21 PM, Neelesh <neele...@gmail.com> wrote: > > > > > > > Hi, > > > > I am intrigued by the design of Kafka and pushing for using it in > our > > > > org. We have many use cases where kafka fits the bill. Is there a > > > timeframe > > > > you are looking at for Kafka 0.8 to be released?I am especially > > > interested > > > > in long polling, (KAFKA-48) and thinking of writing a more > > sophisticated > > > > PHP client which can do producer side load balancing, and possible > > > > something like IronCount for the consumer side. > > > > > > > > Thanks! > > > > -neelesh > > > > > > > > > >
