If you want to have an actor per domain object handled by akka-sharding for read/write and after each write update the indexes affected by that object for an in-memory map reduce engine which again one exist per akka-shard node.
That is...very do-able, you can save yourself sometime and use a map reduce API (a jar dependency if you will) for the in-memory representation and reduce part. search != map reduce (in this case means similar but not as deep as searching) It depends on what you want to do and how far you want to get. Don't try to re-invent the wheel, re-use some non-distributed in-memory map reduce API and make it distributed with akka-sharding and some hacks and then you have something new, usable and useful. HTH, Guido. On Thursday, March 31, 2016 at 9:46:38 PM UTC+1, Ryan Tanner wrote: > > I don't want to rain on anyone's parade, but I think there's a reason that > products like Solr and Elasticsearch are complicated: search is hard! > Getting it right at scale is *really tough*. > > Personally, I would gladly trade the operational complexity of ES/Solr > over reimplementing them myself. > > On Wednesday, March 30, 2016 at 12:42:53 AM UTC-6, Chelios wrote: >> >> Hey guys >> >> I've got an Eventsource based application (Not CQRS - Read and write are >> both on the write side). The state of all the entities/aggregates/actor are >> stored in memory because the data is not going to go above 120GB and I've >> have a machine with 265GB RAM. >> >> *Problem:* >> Suppose I have a million Products where each *Product* is an Actor >> supervised by *ProductSupervisorActor* and I want to perform the >> following query: >> *Query*: Find all the products where the *product description* matches >> some user input. >> >> I'm wondering if I could get away with just querying the state of the >> million actors and aggregating the result into one >> *SearchRequestHandlerActor* instead of using a search database like SOLR >> ? I've used SOLR before and it's super fast but I'm just trying to reduce >> the complexity in my application. If the state is already in memory may be >> I can just find a way to query it instead of introducing another moving >> part (SOLR) into the system that I have to manage and make sure that the >> data is synchronized. >> >> I would really like to find a solution to perform the above query >> efficiently by just using Actors with paging. If I can achieve this then I >> can have *ProductActor*s running anywhere in a cluster and the search >> would work just fine. Instead, if I was using SOLR I would have to shard or >> partition the database which just another hassle. >> >> RIght now I've got a *ProductSearchRequestHandlerActor *which*, *on >> initilization, accepts *totalNumberOfMessagesExpect: Long* and accepts >> messages of type *Option[ProductState]* until the >> *totalNumberOfMessagesExpect* is reached. *I have not implemented paging >> yet.* >> >> I just wanted to get your opinion or ideas on how I can achieve this >> efficiently or any tips or I'm being silly for trying this because there is >> no central index ? >> >> Chel >> > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
