Oli, Kafka is only a partial solution. As I describe here ( http://www.slideshare.net/chris.e.richardson/hacksummit-2016-eventdriven-microservices-events-on-the-outside-on-the-inside-and-at-the-core/57) an event store is a hybrid of a database and a message broker. It is a database because it provides an API for inserting events for an entity and retrieving them by the entity's primary key. It is a message broker because it provides an API for subscribing to events. Kafka clearly satisfies the latter but not the former.
Just my two cents. Chris -- Microservices application platform http://eventuate.io On Tue, May 17, 2016 at 12:18 AM, Olivier Lalonde <olalo...@gmail.com> wrote: > Hi all, > > I am considering adopting an "event sourcing" architecture for a system I > am developing and Kafka seems like a good choice of store for events. > > For those who aren't aware, this architecture style consists in storing all > state changes of the system as an ordered log of events and building > derivative views as needed for easier querying (using a SQL database for > example). Those views must be completely derived from the event log alone > so that the log effectively becomes a "single source of truth". > > I was wondering if anyone else is using Kafka for that purpose and more > specifically: > > 1) Can Kafka store messages permanently? > > 2) Let's say I throw away my derived view and want to re-build it from > scratch, is it possible to consume messages from a topic from its very > first message and once it has caught up, listen for new messages like it > would normally do? > > 2) Does it support transactions? Let's say I want to push 3 messages > atomically but the producer process crashes after sending only 2 messages, > is it possible to "rollback" the first 2 messages (e.g. "all or nothing" > semantics)? > > 3) Does it support request/response style semantics or can they be > simulated? My system's primary interface with the outside world is an HTTP > API so it would be nice if I could publish an event and wait for all the > internal services which need to process the event to be "done" > processing before returning a response. > > PS: I'm a Node.js/Go developer so when possible please avoid Java centric > terminology. > > Thanks! > > - Oli > > -- > - Oli > > Olivier Lalonde > http://www.syskall.com <-- connect with me! >
