Re: Event Sourcing with Kafka Streams and processing order of a re-entrant pipeline

2021-02-25 Thread Davide Icardi
Thank you John for the explanation! I confirm that introducing a full integration I have reproduced the problem. We have reviewed our pipelines using your suggestion (ValueTransformer and a state store) and now it seems to work correctly! If someone is interested here is the improved version of

Re: Event Sourcing with Kafka Streams and processing order of a re-entrant pipeline

2021-01-31 Thread John Roesler
Hi David, Thank you for the question. If I can confirm, it looks like the "operations" topic is the only input to the topology, and the topology reads the "operations" topic joined with the "account" table and generates a "movements" stream. It reads (and aggregates) the "movements" stream to

Event Sourcing with Kafka Streams and processing order of a re-entrant pipeline

2021-01-31 Thread Davide Icardi
I'm working on a project where I want to use Kafka Streams for Event Sourcing. General idea is that I have a "commands" topic/KStream, an "events" topic/KStream and a "snapshots" topic/KTable. Snapshots contains the current state of the entities. Commands are validated using the "snapshots" and