Re: [akka-user] Dispatcher configuration ?

2017-05-27 Thread Vishnu Vardhan 
Patrik,

Thanks. I believe the issue is that i am using artery with docker

https://groups.google.com/forum/#!msg/akka-user/Hp-Y2cHJwp8/Rz7bdti3BQAJ

Also, i think it clearly reinforced in my mind the need to design so we can 
fail. Getting these failures helped me internalize this a little more.

Best Regards
Vishnu



On Friday, May 26, 2017 at 7:25:33 AM UTC-7, Patrik Nordwall wrote:
>
> The most important is that you put the blocking actors/task on a dedicated 
> dispatcher, and don't run them on the default dispatcher.
>
> http://doc.akka.io/docs/akka/current/scala/general/actor-systems.html#blocking-needs-careful-management
>
> /Patrik
>
> On Thu, May 25, 2017 at 5:52 PM, Vishnu Vardhan  > wrote:
>
>> Hi,
>>
>> I posted earlier here 
>> https://groups.google.com/forum/#!topic/akka-user/7jwQC7vuYQs but i 
>> wanted to reframe what i think is going on and seek some advice.
>>
>> The code is available on GitHub if anybody is interested. 
>> https://github.com/vardhanv/cosbench_ng
>>
>> Essentially, i am writing an S3 performance tester. Because it is a 
>> performance tester it needs to scale.. 
>>
>> My architecture is
>> - An akka cluster, with a singleton consistent hashing router
>> - Many slaves running on other servers (default is 2 slave workers per 
>> host)
>> - An akka streams implementation that is routing jobs to the slaves 
>> (gets/PUTs) etc
>> - Slave workers that receive the commands and forward them to 
>>- a stateless S3 scala object that does blocking S3 PUTs - but is 
>> protected by a custom thread pool executor with 40 threads
>>
>> My default test setup is "one server that also has two slaves, and one 
>> agent that hosts two more slaves - for a total of 4 slaves".
>>
>> All this in docker containers, so that deployment is simple.
>>
>> I believe that the inconsistent behavior i am seeing with startup, 
>> message passing, unexpected shutdowns are to do with how these various 
>> pieces and their threads are getting scheduled.
>>
>> I am wondering if my hypothesis is true, and what can i do to protect 
>> these components from each other (cluster message passing, from stream 
>> messages, from the blocking calls)
>>
>> Best Regards
>> Vishnu
>>
>>
>>
>> -- 
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
>
> Patrik Nordwall
> Akka Tech Lead
> Lightbend  -  Reactive apps on the JVM
> Twitter: @patriknw
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Dispatcher configuration ?

2017-05-26 Thread Patrik Nordwall 
The most important is that you put the blocking actors/task on a dedicated
dispatcher, and don't run them on the default dispatcher.
http://doc.akka.io/docs/akka/current/scala/general/actor-systems.html#blocking-needs-careful-management

/Patrik

On Thu, May 25, 2017 at 5:52 PM, Vishnu Vardhan  wrote:

> Hi,
>
> I posted earlier here https://groups.google.com/forum/#!topic/akka-user/
> 7jwQC7vuYQs but i wanted to reframe what i think is going on and seek
> some advice.
>
> The code is available on GitHub if anybody is interested. https://github.
> com/vardhanv/cosbench_ng
>
> Essentially, i am writing an S3 performance tester. Because it is a
> performance tester it needs to scale..
>
> My architecture is
> - An akka cluster, with a singleton consistent hashing router
> - Many slaves running on other servers (default is 2 slave workers per
> host)
> - An akka streams implementation that is routing jobs to the slaves
> (gets/PUTs) etc
> - Slave workers that receive the commands and forward them to
>- a stateless S3 scala object that does blocking S3 PUTs - but is
> protected by a custom thread pool executor with 40 threads
>
> My default test setup is "one server that also has two slaves, and one
> agent that hosts two more slaves - for a total of 4 slaves".
>
> All this in docker containers, so that deployment is simple.
>
> I believe that the inconsistent behavior i am seeing with startup, message
> passing, unexpected shutdowns are to do with how these various pieces and
> their threads are getting scheduled.
>
> I am wondering if my hypothesis is true, and what can i do to protect
> these components from each other (cluster message passing, from stream
> messages, from the blocking calls)
>
> Best Regards
> Vishnu
>
>
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>



-- 

Patrik Nordwall
Akka Tech Lead
Lightbend  -  Reactive apps on the JVM
Twitter: @patriknw

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Dispatcher configuration ?

2017-05-25 Thread Vishnu Vardhan 
Hi,

I posted earlier here 
https://groups.google.com/forum/#!topic/akka-user/7jwQC7vuYQs but i wanted 
to reframe what i think is going on and seek some advice.

The code is available on GitHub if anybody is 
interested. https://github.com/vardhanv/cosbench_ng

Essentially, i am writing an S3 performance tester. Because it is a 
performance tester it needs to scale.. 

My architecture is
- An akka cluster, with a singleton consistent hashing router
- Many slaves running on other servers (default is 2 slave workers per host)
- An akka streams implementation that is routing jobs to the slaves 
(gets/PUTs) etc
- Slave workers that receive the commands and forward them to 
   - a stateless S3 scala object that does blocking S3 PUTs - but is 
protected by a custom thread pool executor with 40 threads

My default test setup is "one server that also has two slaves, and one 
agent that hosts two more slaves - for a total of 4 slaves".

All this in docker containers, so that deployment is simple.

I believe that the inconsistent behavior i am seeing with startup, message 
passing, unexpected shutdowns are to do with how these various pieces and 
their threads are getting scheduled.

I am wondering if my hypothesis is true, and what can i do to protect these 
components from each other (cluster message passing, from stream messages, 
from the blocking calls)

Best Regards
Vishnu



-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.