I am seeking advice from those more experienced about some ideas for dealing with dependent processing. The use case is simple though I have to be a bit coy because of NDAs. I have some ideas myself but was hoping some of you with more experience than I could throw out some suggestions.
I have an entity that ranges in the millions of them, we can call this entity A (the actual type is irrelevant). When entity A is updated it requires that entity B (the sorting entity) be updated depending upon the sort of fields from all the As in each B. Now to keep everything in synch and not be dependent on outside forces we update all the As in a batch every minute to get updates from external systems. So at each minute 1 to 10 million As could be updated and we never know where within that range. Currently we are not using an actor system. We have an endpoint that hits a play server which fetches all the data to update the As and then we wait for all As to be updated then feed those updated values into the Bs to sort and categorize them. We cant split out the sort because the order is dependent on every other A in the B entity. We cant do away with the sort either. I want to move this to an actor system. What I cant do is resort the list every time I get a single UpdatedA message. If I did, the B would be perpetually sorting and getting behind. I also need to keep all the As in synch in B so I cant update them and wait to sort them on a one minute interval. I have to collect a batch of A updates and do the sort all at once. I also cant do the entire update process of updating As from the external events in the B actor because that would mean the load isn't distributed and that would be prohibitive. Sorting a 10 million entry list is about 4 seconds, sorting and updating all 10 million would be over a minute and not possible. I had thought of possibly using distributed pub-sub and having the system collect A update messages and then periodically sending itself a message to update the As in the B and sort them. Another idea (though I have no clue how to do this) would be to collect A updates and then if we stop getting them for a few seconds, do an update and sort ay that time. Can anyone think of any other way they might suggest architecting a solution to this problem with actors? Thanks in advance. -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
