Hi Alex, sorry, this fell through the cracks: creating a source of sources is not a problem, you can flatMapConcat them and each one will only be materialized once its turn has come. This means that if the Source is not bound to a running DB query but instead initializes the query after materialization then the whole stream becomes completely lazy.
Regards, Roland > 30 nov 2015 kl. 03:38 skrev Alex <[email protected]>: > > Hello there, > > I have been assigned the task of rewriting an HTTP api service, initially > written in Spray/Observables, to Akka HTTP/Akka Streams. > While things have gone smoothly for the most part, I have currently reached a > roadblock. > The problem is the following: > There is an API call for fetching a list of categories and the items that > belong to those respective categories. The goal is to return a Source of > Chunked JSON for the following route: > > path("categories") { > get { > encodeResponse { > complete( > HttpResponse(entity = > HttpEntity.Chunked( > ContentTypes.`application/json`, > Database.Get.categories.via(asChunkStreamPart) > ) > ) > ) > } > } > > Let's pretend that I am returning Plain text rather than JSON in order to > simplify things: > > The initial Database query returns vector of KeyPair (String, > List[Strings]), corresponding to Category Name and the Id's of the items that > fall under that category. > > I began by wrapping this Observable into a Source[String, List[String]) - > Database.Get.categories: Source[String, List[String] > > Next, I need to the content of each item of that list. Now this is where it > gets tricky, I would like for EACH item to ONLY be fetched as there is demand > from the HTTP Response. > > Initially I opted for rewriting def getItem(id: String): Observable[String] > as Source[String, Unit] but in order to avoid having a Source of Sources I > wrapped it as Future[String] instead. > > This lead to the following signature: Source[Key, List[ Future[String], Unit] > ( So each row has a list of Futures which one invoked will retrieve the item > data) > > My question is, is my design fundamentally flawed or am I overlooking > something? Would it make it easier to flatten everything if the DB queries > for the items were Sources themselves? > > I need to have the entire list flatten with the Key, so that the method > signature becomes Source[String, Unit]. > However, if i am not mistaken, if I were to use flatMapConcat I would be > consuming the entire source ( therefore performing all DB queries for all the > items) correct? > > Just to emphasize, I would want to be able to do this: > val dbResult = Database.Get.categories // let's assume there are two > categories with 2 items each : [ "categoy1", "itemCat1A", "itemCat1B", > "categoy2", "itemCat2A", "itemCat2B"] > > dbResult.take(1) // "categoy1" > dbResult.drop(1) // "itemCat1A" // Fires getItem and return's its string > result once available > dbResult.drop(2) // "itemCat1B" // Fires getItem and return's its string > result once available > dbResult.drop(3) // "categoy2" > dbResult.drop(4) // "itemCat2A" // Fires getItem and return's its string > result once available > > > If anyone could provide some input it would be highly appreciated! > > Thanks for reading. > > > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/> > >>>>>>>>>> Check the FAQ: > >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html> > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > >>>>>>>>>> <https://groups.google.com/group/akka-user> > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To post to this group, send email to [email protected] > <mailto:[email protected]>. > Visit this group at http://groups.google.com/group/akka-user > <http://groups.google.com/group/akka-user>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. Dr. Roland Kuhn Akka Tech Lead Typesafe <http://typesafe.com/> – Reactive apps on the JVM. twitter: @rolandkuhn <http://twitter.com/#!/rolandkuhn> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
