Re: [akka-user] Akka Actor usage in Apache Spark and Apache Flink

[Konrad Malawski]

Another pretty big Akka based project (as in, really using Akka a lot) is
Apache OpenWhisk, the open source "amazon lambda";
Check it out as well.

https://github.com/apache?q=openwhisk

On Mon, Feb 19, 2018 at 2:32 PM, Konrad “ktoso” Malawski <
konrad.malaw...@lightbend.com> wrote:

> Best to ask on a Flink forum ;-)
>
> --
> Cheers,
> Konrad 'ktoso ' Malawski
> Akka  @ Lightbend 
>
> On February 19, 2018 at 14:32:15, Unmesh Joshi (unmeshjo...@gmail.com)
> wrote:
>
> Thanks Konrad.
> What I was interested in was what use cases both Spark and Flink used
> Actors for specifically. Are there any notes/docs around those that anyone
> knows of?
>
>
>
> On Monday, 19 February 2018 07:33:43 UTC+5:30, Konrad Malawski wrote:
>>
>> Hi there,
>> There's been some confusion about how Spark used Akka, so let's clear
>> that up first:
>> It wasn't really core to any of the operations or core of Spark, and was
>> used only to orchestrate some things,
>> it was using raw Akka Remoting (which is discouraged, use Cluster instead
>> due to the resilience benefits it gives).
>>
>> Users would sometimes want to combine Akka based things with their Spark
>> jobs, but since Spark did not have Akka as a focus-on
>> thing, but merely a small piece of code they tagged along, and they had
>> internal mechanisms to do similar things as they used Akka for (just a few
>> simple rpc-style calls),
>> it was better for everyone to remove the dependency -- as Spark was using
>> a rather old version, and people wanted to use recent versions of Akka --
>> the usual dependency mess you get with big projects in other words.
>>
>> Thus Akka was removed from Spark, allowing users to bring their own Akka
>> based libraries into Spark apps at-will, which is better for both projects
>> and users as well.
>> Shedding dependencies is generally a good thing, and I think Spark did
>> the right thing there. You'll also notice how Akka tries to avoid any kind
>> of external dependency that we're able to avoid -- for the same reasons,
>> avoiding dependency hell for end users.
>>
>> Hope this helps,
>>
>> On Sun, Feb 18, 2018 at 8:58 PM, Unmesh Joshi  wrote:
>>
>>> Hi,
>>>
>>> Apache Spark used to use Akka Actors (which is replaced by Netty now)
>>> and Apache Flink uses Actors. I am trying to understand the use case of
>>> Actor usage in both these frameworks. Any pointers for help?
>>>
>>> Thanks,
>>> Unmesh
>>> --
>>> >> Read the docs: http://akka.io/docs/
>>> >> Check the FAQ: http://doc.akka.io/docs/akka/c
>>> urrent/additional/faq.html
>>> >> Search the archives: https://groups.google.com/grou
>>> p/akka-user
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to akka-user+...@googlegroups.com.
>>> To post to this group, send email to akka...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Cheers,
>> Konrad 'ktoso' Malawski
>> Akka  @ Typesafe 
>>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>


-- 
Cheers,
Konrad 'ktoso' Malawski
Akka  @ Lightbend 

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka Actor usage in Apache Spark and Apache Flink

[Konrad “ktoso” Malawski]

Best to ask on a Flink forum ;-)

-- 
Cheers,
Konrad 'ktoso ' Malawski
Akka  @ Lightbend 

On February 19, 2018 at 14:32:15, Unmesh Joshi (unmeshjo...@gmail.com)
wrote:

Thanks Konrad.
What I was interested in was what use cases both Spark and Flink used
Actors for specifically. Are there any notes/docs around those that anyone
knows of?



On Monday, 19 February 2018 07:33:43 UTC+5:30, Konrad Malawski wrote:
>
> Hi there,
> There's been some confusion about how Spark used Akka, so let's clear that
> up first:
> It wasn't really core to any of the operations or core of Spark, and was
> used only to orchestrate some things,
> it was using raw Akka Remoting (which is discouraged, use Cluster instead
> due to the resilience benefits it gives).
>
> Users would sometimes want to combine Akka based things with their Spark
> jobs, but since Spark did not have Akka as a focus-on
> thing, but merely a small piece of code they tagged along, and they had
> internal mechanisms to do similar things as they used Akka for (just a few
> simple rpc-style calls),
> it was better for everyone to remove the dependency -- as Spark was using
> a rather old version, and people wanted to use recent versions of Akka --
> the usual dependency mess you get with big projects in other words.
>
> Thus Akka was removed from Spark, allowing users to bring their own Akka
> based libraries into Spark apps at-will, which is better for both projects
> and users as well.
> Shedding dependencies is generally a good thing, and I think Spark did the
> right thing there. You'll also notice how Akka tries to avoid any kind of
> external dependency that we're able to avoid -- for the same reasons,
> avoiding dependency hell for end users.
>
> Hope this helps,
>
> On Sun, Feb 18, 2018 at 8:58 PM, Unmesh Joshi  wrote:
>
>> Hi,
>>
>> Apache Spark used to use Akka Actors (which is replaced by Netty now) and
>> Apache Flink uses Actors. I am trying to understand the use case of Actor
>> usage in both these frameworks. Any pointers for help?
>>
>> Thanks,
>> Unmesh
>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: http://doc.akka.io/docs/akka/
>> current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+...@googlegroups.com.
>> To post to this group, send email to akka...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Cheers,
> Konrad 'ktoso' Malawski
> Akka  @ Typesafe 
>
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
"Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka Actor usage in Apache Spark and Apache Flink

[Unmesh Joshi]

Thanks for sharing. Trying to find and understand the use cases where
Actors are used in other open source projects.

On Mon, Feb 19, 2018 at 11:04 AM, Konrad Malawski <
konrad.malaw...@lightbend.com> wrote:

> Another pretty big Akka based project (as in, really using Akka a lot) is
> Apache OpenWhisk, the open source "amazon lambda";
> Check it out as well.
>
> https://github.com/apache?q=openwhisk
>
> On Mon, Feb 19, 2018 at 2:32 PM, Konrad “ktoso” Malawski <
> konrad.malaw...@lightbend.com> wrote:
>
>> Best to ask on a Flink forum ;-)
>>
>> --
>> Cheers,
>> Konrad 'ktoso ' Malawski
>> Akka  @ Lightbend 
>>
>> On February 19, 2018 at 14:32:15, Unmesh Joshi (unmeshjo...@gmail.com)
>> wrote:
>>
>> Thanks Konrad.
>> What I was interested in was what use cases both Spark and Flink used
>> Actors for specifically. Are there any notes/docs around those that anyone
>> knows of?
>>
>>
>>
>> On Monday, 19 February 2018 07:33:43 UTC+5:30, Konrad Malawski wrote:
>>>
>>> Hi there,
>>> There's been some confusion about how Spark used Akka, so let's clear
>>> that up first:
>>> It wasn't really core to any of the operations or core of Spark, and was
>>> used only to orchestrate some things,
>>> it was using raw Akka Remoting (which is discouraged, use Cluster
>>> instead due to the resilience benefits it gives).
>>>
>>> Users would sometimes want to combine Akka based things with their Spark
>>> jobs, but since Spark did not have Akka as a focus-on
>>> thing, but merely a small piece of code they tagged along, and they had
>>> internal mechanisms to do similar things as they used Akka for (just a few
>>> simple rpc-style calls),
>>> it was better for everyone to remove the dependency -- as Spark was
>>> using a rather old version, and people wanted to use recent versions of
>>> Akka -- the usual dependency mess you get with big projects in other words.
>>>
>>> Thus Akka was removed from Spark, allowing users to bring their own Akka
>>> based libraries into Spark apps at-will, which is better for both projects
>>> and users as well.
>>> Shedding dependencies is generally a good thing, and I think Spark did
>>> the right thing there. You'll also notice how Akka tries to avoid any kind
>>> of external dependency that we're able to avoid -- for the same reasons,
>>> avoiding dependency hell for end users.
>>>
>>> Hope this helps,
>>>
>>> On Sun, Feb 18, 2018 at 8:58 PM, Unmesh Joshi 
>>> wrote:
>>>
 Hi,

 Apache Spark used to use Akka Actors (which is replaced by Netty now)
 and Apache Flink uses Actors. I am trying to understand the use case of
 Actor usage in both these frameworks. Any pointers for help?

 Thanks,
 Unmesh
 --
 >> Read the docs: http://akka.io/docs/
 >> Check the FAQ: http://doc.akka.io/docs/akka/c
 urrent/additional/faq.html
 >> Search the archives: https://groups.google.com/grou
 p/akka-user
 ---
 You received this message because you are subscribed to the Google
 Groups "Akka User List" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to akka-user+...@googlegroups.com.
 To post to this group, send email to akka...@googlegroups.com.
 Visit this group at https://groups.google.com/group/akka-user.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>>>
>>> --
>>> Cheers,
>>> Konrad 'ktoso' Malawski
>>> Akka  @ Typesafe 
>>>
>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: http://doc.akka.io/docs/akka/c
>> urrent/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+unsubscr...@googlegroups.com.
>> To post to this group, send email to akka-user@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>
>
> --
> Cheers,
> Konrad 'ktoso' Malawski
> Akka  @ Lightbend 
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit 

Re: [akka-user] Akka Actor usage in Apache Spark and Apache Flink

[Unmesh Joshi]

Thanks Konrad.
What I was interested in was what use cases both Spark and Flink used 
Actors for specifically. Are there any notes/docs around those that anyone 
knows of?



On Monday, 19 February 2018 07:33:43 UTC+5:30, Konrad Malawski wrote:
>
> Hi there,
> There's been some confusion about how Spark used Akka, so let's clear that 
> up first:
> It wasn't really core to any of the operations or core of Spark, and was 
> used only to orchestrate some things,
> it was using raw Akka Remoting (which is discouraged, use Cluster instead 
> due to the resilience benefits it gives).
>
> Users would sometimes want to combine Akka based things with their Spark 
> jobs, but since Spark did not have Akka as a focus-on
> thing, but merely a small piece of code they tagged along, and they had 
> internal mechanisms to do similar things as they used Akka for (just a few 
> simple rpc-style calls),
> it was better for everyone to remove the dependency -- as Spark was using 
> a rather old version, and people wanted to use recent versions of Akka -- 
> the usual dependency mess you get with big projects in other words.
>
> Thus Akka was removed from Spark, allowing users to bring their own Akka 
> based libraries into Spark apps at-will, which is better for both projects 
> and users as well.
> Shedding dependencies is generally a good thing, and I think Spark did the 
> right thing there. You'll also notice how Akka tries to avoid any kind of 
> external dependency that we're able to avoid -- for the same reasons, 
> avoiding dependency hell for end users.
>
> Hope this helps,
>
> On Sun, Feb 18, 2018 at 8:58 PM, Unmesh Joshi  > wrote:
>
>> Hi,
>>
>> Apache Spark used to use Akka Actors (which is replaced by Netty now) and 
>> Apache Flink uses Actors. I am trying to understand the use case of Actor 
>> usage in both these frameworks. Any pointers for help?
>>
>> Thanks,
>> Unmesh
>>
>> -- 
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Cheers,
> Konrad 'ktoso' Malawski
> Akka  @ Typesafe 
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka Actor usage in Apache Spark and Apache Flink

[Konrad Malawski]

Hi there,
There's been some confusion about how Spark used Akka, so let's clear that
up first:
It wasn't really core to any of the operations or core of Spark, and was
used only to orchestrate some things,
it was using raw Akka Remoting (which is discouraged, use Cluster instead
due to the resilience benefits it gives).

Users would sometimes want to combine Akka based things with their Spark
jobs, but since Spark did not have Akka as a focus-on
thing, but merely a small piece of code they tagged along, and they had
internal mechanisms to do similar things as they used Akka for (just a few
simple rpc-style calls),
it was better for everyone to remove the dependency -- as Spark was using a
rather old version, and people wanted to use recent versions of Akka -- the
usual dependency mess you get with big projects in other words.

Thus Akka was removed from Spark, allowing users to bring their own Akka
based libraries into Spark apps at-will, which is better for both projects
and users as well.
Shedding dependencies is generally a good thing, and I think Spark did the
right thing there. You'll also notice how Akka tries to avoid any kind of
external dependency that we're able to avoid -- for the same reasons,
avoiding dependency hell for end users.

Hope this helps,

On Sun, Feb 18, 2018 at 8:58 PM, Unmesh Joshi  wrote:

> Hi,
>
> Apache Spark used to use Akka Actors (which is replaced by Netty now) and
> Apache Flink uses Actors. I am trying to understand the use case of Actor
> usage in both these frameworks. Any pointers for help?
>
> Thanks,
> Unmesh
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Cheers,
Konrad 'ktoso' Malawski
Akka  @ Typesafe 

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.