I should add in our custom producers we buffer in RAM if required, so Kafka can 
be restarted etc. But I would never code streaming to disk now. I would just 
run a Kafka instance on the same node. 

Philip

On Nov 28, 2013, at 4:08 PM, Philip O'Toole <phi...@loggly.com> wrote:

> By FS I guess you mean file system. 
> 
> In that case, if one is that concerned, why not run a single Kafka broker on 
> the same machine, and connect to it over localhost? And disable ZK mode too, 
> perhaps. 
> 
> I may be missing something, but I never fully understand why people try 
> really hard to build a stream-to-disk backup approach, when they might be 
> able to couple tightly to Kafka, which, well, just streams to disk. 
> 
> Philip
> 
> On Nov 28, 2013, at 3:58 PM, Otis Gospodnetic <otis.gospodne...@gmail.com> 
> wrote:
> 
>> Hi,
>> 
>> We've done this at Sematext, where we use Kafka in all 3 products/services
>> you see in my signature.  When we fail to push a message into Kafka we
>> store it in the FS and from there we can process it later.
>> 
>> Otis
>> --
>> Performance Monitoring * Log Analytics * Search Analytics
>> Solr & Elasticsearch Support * http://sematext.com/
>> 
>> 
>> On Thu, Nov 28, 2013 at 9:37 AM, Demian Berjman <dberj...@despegar.com>wrote:
>> 
>>> Hi.
>>> 
>>> Anyone has build a retry system (durable) for the producer in case the
>>> kakfa cluster is down? We have certain messages that must be sent because
>>> there are after a transaction that cannot be undo.
>>> 
>>> We could set the property "message.send.max.retries", but if the producer
>>> goes down, we lost the messages.
>>> 
>>> Thanks,
>>> 

Reply via email to