Thanks for that, I'm not very familiar with the Akka streams API yet, so any feedback is very welcome.
I guess the cluster singleton could work for my use case, it could just communicate to sharded workers that do the data normalization so that it does not become a bottleneck. On Friday, November 20, 2015 at 6:21:34 PM UTC, Patrik Nordwall wrote: > > > > 20 nov. 2015 kl. 18:50 skrev Bruno Filippone <[email protected] > <javascript:>>: > > > What you want to do is process all the events coming from the > PersistenceQuery stream and create a view that can be queried, for example: > transform all the events into normalized data and insert them into > ElasticSearch. > > > Right > > > A quick example would look like the following (using > EventStoreReadJournal): > > val queries = PersistenceQuery(system).readJournalFor > [EventStoreReadJournal](EventStoreReadJournal.Identifier) > > queries.allPersistenceIds() > > .runForeach { persistenceId => > > > queries.eventsByPersistenceId(persistenceId).map(_.event).runForEach(elasticSearchIndexer > > ! _) > > } > > > You should use mapAsync + ask or some of the other facilities in akka > streams to integrate with actors without loosing the backpressure to the > stream. The problem with foreach + tell is that if elasticSearchIndexer is > slower than the stream you will fill up the mailbox snd eventually OOME. > > > Hope this helps. > > On a side note, one of the things I'm struggling with instead is how to > process the event stream if I have a cluster of nodes on my read side > without having all the nodes processing all the events coming from the > events source, potentially I would like to have each node indexing a subset > of the events into my ElasticSearch database. > > Does anyone have any guidance on how to achieve this? > > The problem is that I have no way to chop the stream up in my read nodes > based on my cluster shards for example, the PersistentView played well with > cluster sharding, but what about PersistenceQuery? > > > Perhaps you can use a cluster singleton for the allPersistenceIds query, > which delegates to a sharded entity actor for each persistenceId? > > Thanks for sharing your thoughts about this. > > Cheers, > Patrik > > > > On Sunday, November 8, 2015 at 5:36:57 PM UTC, Adam Dohnal wrote: >> >> Hello, >> >> recently I read through new akka-persistence-query-experimental module >> and would like to see some tutorials and examples to see how it can work. >> Documentation is really abstract for me. Unfortunately, I'm not able to >> find any activator templates nor tutorials. >> >> What I understand is how can I get a stream of events for my aggregate >> root. But how can I construct my read model. Should I subscribe to that >> event stream and for each event I should update ma read model and store it >> to read database? I think I am little confused. >> >> Can you guys share your experience with that module? Some blog posts, >> mini-projects on github etc... to show how can it be used to implement >> query side of CQRS? Thank you >> > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected] > <javascript:>. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
