Re: [akka-user] Re: Is it possible to increase the number of serialization threads?

2016-07-02 Thread Eduardo Fernandes 
Hi Guido.

After implementing your suggestions I'm realizing that maybe my particular
case is a bit peculiar, I'm interested in a benchmark involving 2 hosts.
I'm getting around 500.000 transactions per second (echo of array of 48
bytes), which is not a so bad number. Increasing the number of worker
threads is not improving my numbers. I have two actors in a shard region
talking to other two actors in other shard region (in other physical node).
It looks like Akka/netty, when talking to the second shard region, never
asks for more than one thread when connecting to the second shard region. I
think that Akka could use as many threads as actors in the first shard
region, below a max number. In my case, with a 8 cores machine, we could
expect that, for example, two threads could be used to serialize from host1
to host2 but I'm afraid that is not the case.

Thanks again for your time on this.

Regards.


On Sat, Jul 2, 2016 at 1:35 PM, Guido Medina  wrote:

> The following might also be helpful:
>
>
> akka.remote {
>
>   #log-remote-lifecycle-events = off
>
>   netty.tcp {
>
> #port = 0
>
> server-socket-worker-pool {
>   pool-size-min = 4
>   pool-size-factor = 1
>   pool-size-max = 8
> }
>
> client-socket-worker-pool {
>   pool-size-min = 4
>   pool-size-factor = 1
>   pool-size-max = 8
> }
>   }
> }
>
>
>
>
> On Saturday, July 2, 2016 at 12:31:12 PM UTC+1, Guido Medina wrote:
>>
>> Didn't read the serialization part, must be that I need my coffee, Kryo
>> is very well optimized, you don't know if is going to help until you test
>> it.
>> Kryo also also has a pool of serializers that are ready to be used and
>> can be improved if you declared the classes you are serializing.
>>
>> On Saturday, July 2, 2016 at 12:27:28 PM UTC+1, Guido Medina wrote:
>>>
>>> Try the following, these values work great,
>>> you can try higher but I don't think it is going to help because the
>>> most of the bottleneck is because of design,
>>> a problem related with how akka-remote is designed around Netty, but
>>> that will change soon: https://github.com/akka/akka-meta/issues/22
>>>
>>> akka.remote.default-remote-dispatcher {
>>>   type = Dispatcher
>>>   executor = "fork-join-executor"
>>>
>>>   fork-join-executor {
>>> parallelism-min = 4
>>> parallelism-factor = 1
>>> parallelism-max = 8
>>>   }
>>> }
>>>
>>> Also, what serialization are you using? Hopefully is the not default
>>> Java serialization, most people use Kryo serialization.
>>>
>>> Non-related but could help:
>>>
>>>- Set Akka version to "2.3.15" (next week 2.3.16 is probably going
>>>to be released)
>>>- Set Netty version to "3.10.6.Final" which will be part of next
>>>release.
>>>
>>> See https://github.com/netty/netty/issues?q=milestone%3A3.10.6.Final
>>> and https://github.com/akka/akka/pull/20857
>>>
>>> HTH,
>>>
>>> Guido.
>>>
>>> On Saturday, July 2, 2016 at 12:14:24 AM UTC+1, Eduardo Fernandes wrote:

 Hi.

 I'm using Akka 2.3.13, Java edition.

 I'm making some performance tests and in the same machine with 8 cores
 I see that the serialization process is my bottleneck.  I know that because
 after an increment of actor cpu usage the throughput is exactly the same.

 My actor system talks to 2 other nodes so I see 2 cores dedicated to
 serialization. Is is possible to increase the number of threads for
 serialization?

 I'm using standard Java serialization but I have my own serialization
 implementation in my write/readObject methods so I think that switching to
 kryo or similar will not enhance too much the throughput.

 Many thanks for your help.

 /Eduardo

 --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Akka User List" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/akka-user/EVsIxMEDKeI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group,

Re: [akka-user] Re: Is it possible to increase the number of serialization threads?

2016-07-02 Thread Eduardo Fernandes 
Hi. 

Many thanks for your suggestions. 

I will try and let you know. 

Thanks again for your time. 

> El 2 jul 2016, a las 13:35, Guido Medina  escribió:
> 
> The following might also be helpful:
> 
> 
> akka.remote {
> 
>   #log-remote-lifecycle-events = off
> 
>   netty.tcp {
> 
> #port = 0
> 
> server-socket-worker-pool {
>   pool-size-min = 4
>   pool-size-factor = 1
>   pool-size-max = 8
> }
> 
> client-socket-worker-pool {
>   pool-size-min = 4
>   pool-size-factor = 1
>   pool-size-max = 8
> }
>   }
> }
> 
> 
> 
> 
>> On Saturday, July 2, 2016 at 12:31:12 PM UTC+1, Guido Medina wrote:
>> Didn't read the serialization part, must be that I need my coffee, Kryo is 
>> very well optimized, you don't know if is going to help until you test it.
>> Kryo also also has a pool of serializers that are ready to be used and can 
>> be improved if you declared the classes you are serializing.
>> 
>>> On Saturday, July 2, 2016 at 12:27:28 PM UTC+1, Guido Medina wrote:
>>> Try the following, these values work great,
>>> you can try higher but I don't think it is going to help because the most 
>>> of the bottleneck is because of design,
>>> a problem related with how akka-remote is designed around Netty, but that 
>>> will change soon: https://github.com/akka/akka-meta/issues/22
>>> 
>>> akka.remote.default-remote-dispatcher {
>>>   type = Dispatcher
>>>   executor = "fork-join-executor"
>>> 
>>>   fork-join-executor {
>>> parallelism-min = 4
>>> parallelism-factor = 1
>>> parallelism-max = 8
>>>   }
>>> }
>>> 
>>> Also, what serialization are you using? Hopefully is the not default Java 
>>> serialization, most people use Kryo serialization.
>>> 
>>> Non-related but could help:
>>> Set Akka version to "2.3.15" (next week 2.3.16 is probably going to be 
>>> released)
>>> Set Netty version to "3.10.6.Final" which will be part of next release.
>>> See https://github.com/netty/netty/issues?q=milestone%3A3.10.6.Final and 
>>> https://github.com/akka/akka/pull/20857
>>> 
>>> HTH,
>>> 
>>> Guido.
>>> 
 On Saturday, July 2, 2016 at 12:14:24 AM UTC+1, Eduardo Fernandes wrote:
 Hi.
 
 I'm using Akka 2.3.13, Java edition.
 
 I'm making some performance tests and in the same machine with 8 cores I 
 see that the serialization process is my bottleneck.  I know that because 
 after an increment of actor cpu usage the throughput is exactly the same. 
 
 My actor system talks to 2 other nodes so I see 2 cores dedicated to 
 serialization. Is is possible to increase the number of threads for 
 serialization?
 
 I'm using standard Java serialization but I have my own serialization 
 implementation in my write/readObject methods so I think that switching to 
 kryo or similar will not enhance too much the throughput. 
 
 Many thanks for your help.
 
 /Eduardo
> 
> -- 
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: 
> >> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> --- 
> You received this message because you are subscribed to a topic in the Google 
> Groups "Akka User List" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/akka-user/EVsIxMEDKeI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.