On Fri, Dec 9, 2016 at 7:31 PM, <[email protected]> wrote: > I thought I needed to use a GraphStageLogic because I need to do the > following: > > request url for page of data and push > if response has next token, grab another page and push > continue until no next token > complete the stream > > I wasn't sure how to accomplish that logic in a Source. > > I figured out a solution by using getAsyncCallback > > val pushCallback = getAsyncCallback[Seq[String]] { seq => > push(out, seq) > } > > > Then on my future callback I call pushCallback.invoke(seq) > > > It seems to work just fine but not sure that materializing a source in a > GraphStageLogic is the best approach. > > You could just as well emit the whole Source:
val yourCustomSource: Source[Source[String]] val finalSource: Source[String] = yourCustomSource.flatMapConcat(identity) This is what I meant by composing with flatMapConcat. It will be way faster than materializing the Source on your own. -Endre > > If you have some example you can point me at I'm all ears! > > > On Friday, December 9, 2016 at 1:27:31 AM UTC-7, drewhk wrote: >> >> >> >> On Thu, Dec 8, 2016 at 9:21 PM, <[email protected]> wrote: >> >>> Hi, >>> >>> I'm creating a Source via GraphStageLogic which makes calls to another >>> api, which happens to return a Source. However I'm unsure how to deal with >>> Source/Futures in a GraphStageLogic. It seems that I want my shape to look >>> like >>> >>> val shape: SourceShape[Seq[String]] = SourceShape(out) >>> >>> >>> but I get back a Source[ByteString, NotUsed] which I can covert to >>> Future[Seq[String]] >>> via runWith(Sink.seq). So should I make my shape >>> >>> val shape: SourceShape[Future[Seq[String]]] = SourceShape(out) >>> >>> >>> But then I need to materialize the Source inside the onPull(), which >>> doesn't seem right. Is there a better way to handle this situation? >>> >> >> You likely need to combine your stage with a flatMapConcat(). Remember >> that you don't need to implement all your logic inside a GraphStage, you >> can create Sources just by using existing combinators like mapAsync or >> flatMapConcat. In fact, if you need some custom operation that is not >> covered by the built-in combinators, it is usually a good idea to minimize >> the custom part and combine it with the existing combinators to achieve >> what you want. >> >> >>> >>> Thanks! >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/c >>> urrent/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/grou >>> p/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ > current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
