I've been thinking about how Akka Persistence could possibly be improved for CQRS and *persistent* stream or log processing more generally.
Here's my stream (no pun intended) of thought: Currently, PersistentActor is focussed on persisting state changes for an Actor (persistenceId) but why couldn't that be extended to just persisting a stream of events (or more generally messages) with the option to skip the reconstruction of state, i.e. PersistentStreamProducer. Similarly, PersistentView could be extended to consume a persistent stream of messages (keeping track of where it was up to, with the option to start again if needed), i.e. PersistentStreamConsumer. It should be possible to label these persistent streams with something other than a persistenceId (if appropriate), i.e. something like a topic. And finally, the most important for CQRS, PersistentStreamConsumer shouldn't need to be tied to a real PersistentStreamProducer. It should be possible to create (perhaps with a DSL in Scala or separately in the store itself, or elsewhere) virtual streams (projections) based on PersistentStreamProducer(s), PSP types, message types, etc. Greg Young's Event Store would seem to be a great store for facilitating most (if not all) of this, with its projections functionality. Kafka would seem to be an even more scalable solution but (AFAIK) currently lacks the ability to create virtual streams (i.e. virtual logs). I think this approach is somewhat different from the current Akka Streams, as far as I understand them, which seem to be "run-time only" streams. Perhaps these actors would be better named PersistentLogProducer and PersistentLogConsumer (or just LogProducer and LogConsumer). Or they could just be traits. In that case PersistentActor would be "Actor with LogProducer" and PersistentView would be "Actor with LogConsumer." Perhaps this can (almost) already be done with PersistentActor and PersistentView? I'd be very interested to know if the current PersistentViews could somehow use "virtual" persistenceIds resulting from projections created in Event Store. I assume the current journals create "topics" for each PersistentActor persistenceId and the PersistenceViews just "look them up." So if it was possible to create a projection in Event Store with a synthetic persistenceId then perhaps a PersistenceView could just find it, believing it to be a journal for a real PersistentActor. Your thoughts? I note there is already: 1. An Akka client for Event Store (EventStore.JVM) 2. An early Event Store plugin for Akka Persistence 3. An Akka consumer for Kafka (akka-kafka) -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
