I think this question has a higher chance of getting an answer if asked in
the Spark community/mailing list.

The linked project seems to be more about feeding data from Akka Streams
into spark than running Akka Streams inside of Spark.

-- 
Johan
Akka Team

On Fri, Mar 17, 2017 at 8:49 PM, Kyrylo Stokoz <k.sto...@gmail.com> wrote:

>
> Hi All,
>
> I`m trying to figure out how one should use Sources within Spark Jobs
> keeping all benefits of Spark.
> Consider following snippet:
>
> val items: Seq[String] = Seq("a", "b", "c")
> sparkSession.sparkContext.parallelize(items, 10).flatMap { item =>
>   val subItems: Source[String, _] = f(item)
>   // (1)
>   ???
> }.map { subItem =>
>   f1(subItem)
> }.reduce(_ + _)
>
>
> There is this project https://github.com/lloydmeta/sparkka-streams which
> tries to bridge akka streams and Spark Streaming.
> I also created an IteratorSinkStage which materialized into Iterator
> similar as InputStreamSinkStage which is materialized into InputStream, but
> i`m not sure if this is best solution for this problem.
>
> What would be the best way to work with akka Source within Spark
> environment?
>
> Regards,
> Kyrylo
>
>
>
>
>
>
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>>>>>>>>>      Read the docs: http://akka.io/docs/
>>>>>>>>>>      Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>      Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Reply via email to