In fetchLikesAndUpdateProfile, you can broadcast the input to
profileLikesSink and also to a flow the performs a fold to count all
elements passed through and update the profile.
On Saturday, November 5, 2016 at 11:43:36 PM UTC-3, Eugene Dzhurinsky wrote:
>
> Okay, perhaps the simple source snippet will make it clear:
>
> type OptProfile = Option[FullProfile]
> type LikesAndCount = (Int, Stream[Profile])
>
> val src: Source[Int, NotUsed] = Source[Int](Conf.startId() to Conf.
> endId())
> val fetchProfileFlow: Flow[Int, OptProfile, NotUsed] = Flow.
> fromFunction(Profile.extractFullProfile)
> val fetchLikesFlow: Flow[Int, LikesAndCount, NotUsed] = Flow.
> fromFunction(Likes.extractUserList)
> val profileDataSink: Sink[ByteString, Future[IOResult]] = FileIO.
> toPath(new File(base, "profiles").toPath)
>
> val fetchLikesAndUpdateProfile = Flow[(OptProfile,
> LikesAndCount)].flatMapConcat
> {
> case (Some(profile), (likesExpected, stream)) ⇒ GraphDSL.create() {
> implicit builder ⇒
> import GraphDSL.Implicits._
>
> val in = builder.add(Source(stream))
> val profileLikesSink = builder.add(FileIO.toPath(new File(base,
> s"likes_${profile.id}").toPath))
> in ~> Flow.fromFunction[Profile.Profile, ByteString](p ⇒
> ByteString(s"${p.id}:${p.username}\n")) ~> profileLikesSink
> // update profile here with the numbers of records and emit it
> Source.single(profile).shape
> }
> }
>
> RunnableGraph.fromGraph(
> GraphDSL.create() {
> implicit builder ⇒
> import GraphDSL.Implicits._
>
> val inlet = builder.add(Broadcast[Int](2))
> val merge = builder.add(Zip[OptProfile, LikesAndCount])
>
> src ~> inlet.in
> inlet.out(0) ~> fetchProfileFlow ~> merge.in0
> inlet.out(1) ~> fetchLikesFlow ~> merge.in1
> merge.out ~> fetchLikesAndUpdateProfile ~>
> Flow.fromFunction[Profile.FullProfile, ByteString](p ⇒
> ByteString(s"$p\n")) ~>
> profileDataSink
> ClosedShape
> }
> ).run()
>
>
> So far it is not clear how would I write *fetchLikesAndUpdateProfile* in
> a way that it will
> - create a sink for storing list of fetched data (every profile has the
> associated file named after the profile ID)
> - how to retrieve the number of stored records in
> *fetchLikesAndUpdateProfile* and update the property in the *FullProfile*
> object.
>
> Thanks!
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
