>>for redundancy, we were planning to simply rsync the kafka message logs to a second machine periodically. or do you see any obvious problems with that?
Rsync approach has several problems and could be a lossy solution. We moved away from that by replacing a legacy system with Kafka. We recommend you setup your redundant cluster using the mirroring approach, which is much more reliable and real-time than rsync. I think 0.6 has a stripped down version of mirroring, where you cannot control the mirroring for specific topics. Thanks, Neha On Fri, Nov 4, 2011 at 9:12 AM, Tim Lossen <t...@lossen.de> wrote: > > interesting. is this already available in 0.6? > > for redundancy, we were planning to simply rsync the kafka > message logs to a second machine periodically. or do you > see any obvious problems with that? > > cheers > tim > > On 2011-11-04, at 17:07 , Neha Narkhede wrote: > > We have a mirroring feature where you can setup 2 clusters, one to be a > > mirror of the other. At LinkedIn, we have a production Kafka cluster, and > > an analytics Kafka cluster that mirrors the production one in real time. We > > still haven't updated the documentation to describe this in detail. > > -- > http://tim.lossen.de > > >
