On Tue, Aug 19, 2014 at 8:51 AM, Martin Krasser <[email protected]> wrote:
> Hi Ashley, > > thanks for bringing up these questions. Here are some general comments: > > as you already mentioned (in different words) akka-persistence is > currently optimized around write models rather than read models (= Q in > CQRS) i.e it is optimized for fast, scalable persistence and recovery of > stateful actors (= PersistentActor). > > For full CQRS support, the discussions so far (in several other threads) > make the assumption that both write and read models are backed by the same > backend store (assuming read models are maintained by PersistentView actor, > receiving a stream of events from synthetic or physical "topics"). > That is not my view of it, at all. PersistentView is a way to replicate the events to the read side, which typically will store a denormalized representation optimized for queries. That query store is typically not the same as the event store, because the requirements are very different. Some simple read models may keep this representation in-memory but that is not what I see as the most common case. /Patrik > This is a severe limitation, IMO. As Greg already mentioned elsewhere, > some read models may be best backed by a graph database, for example. > Although a graph database may be good for backing certain read models, it > may have limitations for fast logging of events (something where Kafka or > Cassandra are very good at). Consequently, it definitely makes sense to > have different backend stores for write and read models. > > If akka-persistence should have support for CQRS in the future, its > design/API should be extended to allow different backend stores for write > and read models (of course, a provider may choose to use the same backend > store to serve both which may be a reasonable default). This way > PersistentActors log events to one backend store and PersistentViews (or > whatever consumer) generate read models from the other backend store. Data > transfer between these backend stores can be implementation-specific for > optimization purposes. For example > > - Cassandra (for logging events) => Spark (to batch-process logged events) > => Graph database XY (to store events processed with Spark), or > - Kafka (for logging events) => Spark Streaming (to stream-process logged > events) => Database XY (to store events processed with Spark Streaming) > - ... > > These are just two examples how read model backend stores can be populated > in a highly scalable way (both in batch and streaming mode). Assuming > akka-persistence provides an additional plugin API for storage backends on > the read model side (XY in the examples above) a wide range of CQRS > applications could be covered with whatever scalability and/or ordering > requirements needed by the respective applications. In case you want to > read more about it, take a look at akka-analytics > <https://github.com/krasserm/akka-analytics> (it is very much work in > progress as I'm waiting for Spark to upgrade to Akka 2.3 and Kafka to Scala > 2.11) > > WDYT? > > Cheers, > Martin > > > On 19.08.14 04:52, Ashley Aitken wrote: > > > I'm keen to hear other people's thoughts on the choice of an event store > for Akka Persistence for doing CQRS. > > As mentioned in my other post, I believe that Akka Persistence only > provides part of the story for CQRS (but a very important part) and that > other stores will most likely be needed for query models (both SQL and > NOSQL stores). > > Since they are project specific I would like to focus here on what store > is "best" for Akka Persistence for CQRS. > > Right now my leading contenders are Kafka and Event Store (but I haven't > thought too much about Cassandra or MongoDB etc). My knowledge of all of > these is limited so please excuse and correct me if any of my statements > are wrong. > > > KAFKA: Apache Kafka is publish-subscribe messaging rethought as a > distributed commit log. > Persistent topics for publishing and subscribing > Highly scalable and distributed > Need to manually create topics for projections > Each topic has own copy of events > Writing to multiple topics is not atomic > Allows logs to be kept for different amounts of time > Battle tested technology from LinkedIn > Not generally used a lifetime store for events > > http://kafka.apache.org > https://github.com/krasserm/akka-persistence-kafka/ > > > EVENT STORE: The open-source, functional database with Complex Event > Processing in JavaScript. > Built specifically for storing and projecting events > Store event once and create virtual projection streams > Journal plugin at early stage in development > Projections are still beta but finalising soon > JSON serialisation (which has +ve and -ve points) > Javascript for projection stream specification > Atom interface helps with debugging > Not as distributed or scalable? > Includes temporal criteria for streams > > http://geteventstore.com > https://github.com/EventStore/EventStore.Akka.Persistence > > Personally, I like the potential Kafka has to be the event store / log > for CQRS but also the store for logs in big data processing and analytics. > However, the fact that events need to be manually replicated to different > topics and problems that would be caused if this wasn't consistent is a > worry. > > On the other hand, Event Store has been specifically designed and built > for event store and projection processing by a leader in the field of CQRS. > However, it uses a unique set of technologies and I am not sure of it has > been battle tested by many or its long term viability. > > What do others think? What are your thoughts of and has your experience > been with other stores? > > > MONGODB: ? > > > CASSANDRA: ? > > > As mentioned, I can definitely see the use of the last two for query > models in addition to one of the event persistence and projection stream > store but have not really considered them for the latter myself. > > Of course, enormous kudos and no disrespect to any of these fantastic > free and open-source projects > > Thanks in advance for sharing any thoughts / experiences. > > Cheers, > Ashley. > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > > > -- > Martin Krasser > > blog: http://krasserm.blogspot.com > code: http://github.com/krasserm > twitter: http://twitter.com/mrt1nz > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Patrik Nordwall Typesafe <http://typesafe.com/> - Reactive apps on the JVM Twitter: @patriknw -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
