That is correct - as I mentioned it does not provide any ability to react to producer failures which can happen if you bounce the source brokers or if a source broker dies before flushing to disk.
Thanks, Joel On Tue, May 22, 2012 at 4:48 PM, Neelesh <neele...@gmail.com> wrote: > Thanks for the superb document Joel. I may not have communicated very well, > but even with mirrormaker, the mirroring is achieved through a consumer. > This assumes that the message is persisted on the source Kafka broker. If > the source broker dies before flushing the messages to disk, won't we lose > those messages? I am sure I'm missing something here. > > Thanks! > -neelesh > > On Tue, May 22, 2012 at 10:46 AM, Joel Koshy <jjkosh...@gmail.com> wrote: > > > Thanks! In the meanwhile, do you think simply modifying the producer to > > > log the same message to multiple brokers (which are not part of a > > cluster) > > > is a reasonably good solution to reduce the probability of lost > > messages? I > > > have seen the mirroring documentation, but it sounded like the > mirroring > > > consumer was embedded inside the broker, and hence a dead broker will > > mean > > > losing unflushed messages. > > > > > > > The embedded consumer approach is being deprecated. I added a new > mirroring > > doc outlining the new approach which uses a separate tool to do > mirroring. > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring+(MirrorMaker) > > > > However, mirroring is orthogonal to your use case. i.e., it does not > quite > > address the ability to react to producer failures (e.g., during a bounce > of > > a broker) which will be available in 0.8. > > > > Thanks, > > > > Joel > > > > > > On Fri, May 18, 2012 at 3:26 PM, Neha Narkhede <neha.narkh...@gmail.com > > > >wrote: > > > > > > > Hi Neelesh, > > > > > > > > Thank you for your interest in Kafka ! > > > > > > > > We are shooting for end of summer this year for the 0.8 release. You > > can > > > > track the progress of this feature here - > > > > https://issues.apache.org/jira/browse/KAFKA-50 > > > > > > > > Thanks, > > > > Neha > > > > > > > > > > > > On Fri, May 18, 2012 at 3:21 PM, Neelesh <neele...@gmail.com> wrote: > > > > > > > > > Hi, > > > > > I am intrigued by the design of Kafka and pushing for using it in > > our > > > > > org. We have many use cases where kafka fits the bill. Is there a > > > > timeframe > > > > > you are looking at for Kafka 0.8 to be released?I am especially > > > > interested > > > > > in long polling, (KAFKA-48) and thinking of writing a more > > > sophisticated > > > > > PHP client which can do producer side load balancing, and possible > > > > > something like IronCount for the consumer side. > > > > > > > > > > Thanks! > > > > > -neelesh > > > > > > > > > > > > > > >
