Re: [akka-user] Is it possible to increase the number of serialization threads?

Yes, I definitely will give a try to Kryo serialization. Right know, despite of using standard Java serialization, we're not sending class id's over the wire since we have a morphological serialization on each class. Nevertheless the Akka itself is doing that so Kryo could provide a good

Re: [akka-user] Is it possible to increase the number of serialization threads?

Thanks Patrik for clarifying the Artery part I had misunderstood, it is getting really close anyway which makes me *-and sure others-* happy. @Eduardo, the Kryo part will still apply, specially listing the classes as I did in my config example, I wouldn't worry much about the KryoQueueBuilder

Re: [akka-user] Is it possible to increase the number of serialization threads?

Ok. In our case the ack is per message... anyway I could expect a very good improvement in number of transactions upgrading to Artery. We don't need the factor 30x you have. If we have a factor 2 or 3 it would be very welcome. Thanks for the info. This numbers could help me to push it into a near

Re: [akka-user] Is it possible to increase the number of serialization threads?

On Mon, Jul 4, 2016 at 9:13 AM, Eduardo Fernandes wrote: > Wow!!! I have to try the new remoting! > > The measurement is quite basic: I have a pool of threads and I just call > the sends() and wait for the response and execute a new send. Nothing > particular. I made a simple

Re: [akka-user] Is it possible to increase the number of serialization threads?

Wow!!! I have to try the new remoting! The measurement is quite basic: I have a pool of threads and I just call the sends() and wait for the response and execute a new send. Nothing particular. I made a simple test with the old remote and my numbers were also around 25.000 msg/s, so we agree on

Re: [akka-user] Is it possible to increase the number of serialization threads?

Then I question how you measure this in your benchmark. With old (current) remoting I have observed max throughput of 25.000 msg/s (one way) using this test: https://github.com/akka/akka/blob/artery-dev/akka-remote-tests/src/multi-jvm/scala/akka/remote/artery/MaxThroughputSpec.scala With new

Re: [akka-user] Is it possible to increase the number of serialization threads?

Yes... 1.000.000 messages over the network (1.000.000 sent and 1.000.000 of ack's with the operation state, in this case a single echo). Sorry, I was not precise on that. We call it transaction becase we make a kind of commit in ram but in this case it is not relevant. On Mon, Jul 4, 2016 at 8:51

Re: [akka-user] Is it possible to increase the number of serialization threads?

Many thanks Patrik. I'll share it with our dev team. I've read it when we have indeed to change our code a bit. If we can distribute the serialization across the processors I'm pretty sure we could achieve around 1.000.000 transactions/s in an 8 cores machine with 4 or 5 actors on each node. Right

Re: [akka-user] Is it possible to increase the number of serialization threads?

On Mon, Jul 4, 2016 at 12:09 AM, Eduardo Fernandes wrote: > Ok. I'll manage to change from 2.10 to 2.11. Yes, my project is in Java. > We're using Java 8 but the transition to 2.4 is not direct since we've > overriden some behaviors in 2.3 which changed in 2.4 ( for ex. >

Re: [akka-user] Is it possible to increase the number of serialization threads?

Ok. I'll manage to change from 2.10 to 2.11. Yes, my project is in Java. We're using Java 8 but the transition to 2.4 is not direct since we've overriden some behaviors in 2.3 which changed in 2.4 ( for ex. AllocationStrategy). I've tried to change to 2.4 and I got many errors. Registering

Re: [akka-user] Is it possible to increase the number of serialization threads?

Questions: - Why not just replace 2.10 with 2.11? From the Java perspective it should be transparent and irrelevant to your project *-if I understood correctly, your project is in Java-* - Are you using Java 8? If just why not just go Akka 2.4.x? Next week Akka 2.4.8+ (if sprint

Re: [akka-user] Is it possible to increase the number of serialization threads?

Hi All. I'm using akka-kryo-serialization_2.10 with incremental strategy. I'm using 2.10 for everything so maybe I should update in a near future. I'm getting the error below. Anyway, after tunning kryo to work with my project, I never will get more cpu's working in my bench so I'm afraid that

Re: [akka-user] Is it possible to increase the number of serialization threads?

*Correction:* - Kryo offers a pool that you can use if you use Kryo directly, kind of the recommended way of using it. - Akka Kryo uses a similar model giving you the choice of using your own queue implementation for its internal pool of instances, each instance is an Akka

Re: [akka-user] Is it possible to increase the number of serialization threads?

Kryo and Akka Kryo don't use any threads, the remote-dispatcher is the one doing the work, and Netty threads sending/receiving the bytes from/to Akka remote dispatcher as far as I can tell. Both Kryo and Akka Kryo have a pool with Kryo instances which also happens to re-use the byte buffer

Re: [akka-user] Is it possible to increase the number of serialization threads?

Quite clean. Many thanks Guido! I'll try it out. I have to inventory my classes so it will take a while. I hope I'll have some numbers this night. Anyway, kryo will scale vertically, I suppose. Will kryo use more threads than I'm using right now? Thanks again for your help and knowledge. On

Re: [akka-user] Is it possible to increase the number of serialization threads?

I meant to say *"anything that implements Serializable"* The classes list is important as Kryo won't write class names on the messages but IDs of the classes: classes = [ "com.mypackage.Class1", "com.mypackage.Class2" ] Suffice to say every node of the cluster must have the same IDs so

Re: [akka-user] Is it possible to increase the number of serialization threads?

I have mimic-ed your configuration and corrected some errors, also added Kryo if you want to give it a chance with the configuration I believe will do best. I default the "java" serializer to Kryo, that way, everything that inherits "Serializable" will use Kryo, also, I list every class that I

Re: [akka-user] Is it possible to increase the number of serialization threads?

I'm using the configuration below, following Guido suggestions. Anyway, everything behaves like Akka/netty add more communication threads only if a add more endpoints. Adding more actores talking from host 1 to host 2 is not improving throughput, despite of the available idle cpu's. Does this

Re: [akka-user] Is it possible to increase the number of serialization threads?

Doesn't the classical remoting perform serialization within the single actor responsible for each connection? Sent from my iPhone > On 03 Jul 2016, at 12:57, Viktor Klang wrote: > > Eduardo, are you sure that there aren't any synchronized-blocks or locks > used? (i.e.

Re: [akka-user] Is it possible to increase the number of serialization threads?

The default remote dispatcher is configured with max 2 threads. Try Guido's advice and report back, thanks. /Patrik sön 3 juli 2016 kl. 12:57 skrev Viktor Klang : > Eduardo, are you sure that there aren't any synchronized-blocks or locks > used? (i.e. is this a contention

Re: [akka-user] Is it possible to increase the number of serialization threads?

Ups... sorry for misunderstanding your question. My principal problem is not the overhead itself. My problem is that I can't get more threads serializing objects to a node. Example: one client I have 30%, lets say. If I add other client talking to other actor instance in parallel I would expect

Re: [akka-user] Is it possible to increase the number of serialization threads?

Hi Eduardo, I meant the overhead of the Java Serialization envelope. On Sat, Jul 2, 2016 at 10:12 PM, Eduardo Fernandes wrote: > Hi Viktor. > > I'm using basic (binary) serialization of basic Java types (int, String > (UTF), long, arrays of basic types, etc...). > > The

Re: [akka-user] Is it possible to increase the number of serialization threads?

Hi Viktor. I'm using basic (binary) serialization of basic Java types (int, String (UTF), long, arrays of basic types, etc...). The overhead is that depending of internal values there is no send to serialize some members and other not. If you merge your functional logic with the serialization

Re: [akka-user] Is it possible to increase the number of serialization threads?

Hi Eduardo, Perhaps I misunderstood, what serialization format are you emitting in your readObject/writeObject? What overhead are you observing compared to using a custom Serializer? On Sat, Jul 2, 2016 at 10:02 PM, Eduardo Fernandes wrote: > Hi. > > If you have

Re: [akka-user] Is it possible to increase the number of serialization threads?

Hi. If you have writeObject/readObject defined in your class the Java plain serialization will invoke those methods. In my case all my internal members and class references are also serialized using the very same technique. So this is equivalent to technologies like kryo and similars since there

Re: [akka-user] Is it possible to increase the number of serialization threads?

I'm not sure I understand why write/readObject special methods would necessarily be faster? Most of the waste of Java Serialization is its envelopes and using class names etc. On Sat, Jul 2, 2016 at 1:14 AM, Eduardo Fernandes wrote: > Hi. > > I'm using Akka 2.3.13, Java