Using Kafka for Event Sourcing

2015-01-11 Thread Yann Simon
Hi, after having read http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying, I am considering Kafka for an application build around CQRS and Event Sourcing. Disclaimer: I read the documentation but do not have any

new to kafka

2015-01-11 Thread Alpesh Desai
I am fairly new to Kafka and working on building a stream processing system using Kafka and Storm. I am looking for couple of references: 1. Reference architecture for Kafka hardware setup that will provide complete redundancy and fault tolerance 2. Benchmarking tool for Kafka setup -Alpesh

Re: Using Kafka for Event Sourcing

2015-01-11 Thread Jay Kreps
Hey Yann, Yes, you can just make the retention infinite which will disable any deletion. What you describe with compaction might work, but wasn't exactly the intention. This type of event logging can work two ways: you can log the command or you can log the result of the command. In databases