A small but significant correction in my earlier email. I misquoted Victor - He said Akka actors can handle millions of message per second per actor per core.
On Mon, Mar 3, 2014 at 10:56 AM, Shiva Ramagopal <[email protected]> wrote: > Hi Hiral, > > I agree with your comment on the bottleneck lying with the front-end HTTP > handling framework. That area became my focus after Victor said Akka actors > can handle 1M msg/s on a single core. I'll also check out how much one > remote agent (node.js-based) can handle. > > Really appreciate your help! > > Best regards, > Shiva > > > On Sun, Mar 2, 2014 at 9:48 AM, tazan007 <[email protected]> wrote: > >> Hi Shiva, just from reading this thread, I'd first suggest you validate >> your HTTP framework can even handle 1,000 TPS. Second, you should consider >> that the remote Agents are guaranteeing only 100ms response time, which >> means your main server is at the mercy of how many TPS the agents can >> handle. For example, if remote agent can handle 500 QPS, and there are 5 >> of them, then if the load test was evenly distributing load such that all >> agents were utilized evenly, then your main server would max out at 500 * >> 5, or 2500 QPS. >> >> -Hiral >> >> >> >> >> On Thu, Feb 27, 2014 at 10:43 PM, Shiva Ramagopal <[email protected]>wrote: >> >>> Hi Endre, >>> >>> I do have a router that translates the incoming HTTP request as a >>> message and routes it to 50 routees. Is that too much? >>> >>> Best regards, >>> Shiv >>> >>> >>> >>> On Thu, Feb 27, 2014 at 4:36 PM, Akka Team <[email protected]>wrote: >>> >>>> Hi Shiva, >>>> >>>> If the utilization does not go above 23% that indicates a potential for >>>> fourfold increase in throughput (well, at least in theory) if the app is >>>> structured differently. It might be that you have an IO bottleneck >>>> somewhere, or you don't exploit all the available parallelism. You should >>>> find the bottleneck and scale it out. You might get away by simply adding a >>>> router at a strategic point (but don't overuse routers or assign too many >>>> routees - both are typical mistakes). >>>> >>>> -Endre >>>> >>>> >>>> On Thu, Feb 27, 2014 at 10:36 AM, Roland Kuhn <[email protected]>wrote: >>>> >>>>> >>>>> 27 feb 2014 kl. 10:33 skrev Shiva Ramagopal <[email protected]>: >>>>> >>>>> Hi Roland, Victor, >>>>> >>>>> Thanks for the pointers! The link is quite informative. >>>>> >>>>> I will have to defer checking out Spray as it doesn't have a Java API >>>>> yet; I think it would have one when its integrated with Akka? >>>>> >>>>> >>>>> Yes. >>>>> >>>>> >>>>> Best regards, >>>>> Shiva >>>>> >>>>> >>>>> >>>>> On Thu, Feb 27, 2014 at 1:27 PM, Roland Kuhn <[email protected]>wrote: >>>>> >>>>>> Hi Shiva, >>>>>> >>>>>> looking at http://www.techempower.com/benchmarks/ you can see that >>>>>> e.g. using spray <http://spray.io/> you should be able to reach 6 >>>>>> digits tps numbers (Simple Framework is not in there and I don't know >>>>>> anything about it). I'm pointing out spray since we are in the process of >>>>>> integrating that as the upcoming Akka HTTP module for 2.4 (ETA this >>>>>> summer). >>>>>> >>>>>> Regards, >>>>>> >>>>>> Roland >>>>>> >>>>>> 27 feb 2014 kl. 08:42 skrev Shiva Ramagopal <[email protected]>: >>>>>> >>>>>> Hi Roland, >>>>>> >>>>>> Thanks for taking the time to read through my rather long question. >>>>>> I'm evaluating Akka (love it, btw) for my product and I'm looking to see >>>>>> if >>>>>> Akka can get me the throughput I want before recommending it to the >>>>>> higher >>>>>> ups. I'm afraid its too early for me to ask for commercial support. I >>>>>> need >>>>>> to first get to atleast 1000 tps before I can make a case for it. A >>>>>> 3-digit >>>>>> number is enough to grab some attention towards Akka in my org. >>>>>> >>>>>> I have used two load generators from different machines for my >>>>>> testing and am not able to push the CPU beyond 23%. What would be really >>>>>> helpful and motivating for me is to get a performance measure for a >>>>>> request-response system like I described above that's designed using >>>>>> Akka. >>>>>> >>>>>> Best regards, >>>>>> Shiva >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Feb 27, 2014 at 12:47 PM, Roland Kuhn <[email protected]>wrote: >>>>>> >>>>>>> Hi Shiva, >>>>>>> >>>>>>> this question is impossible to answer generically, the only thing >>>>>>> that strikes me is that your CPU utilization shows that the system is >>>>>>> not >>>>>>> yet maxed out (you might also want to consider testing the load >>>>>>> generator >>>>>>> whether it can go higher by itself). >>>>>>> >>>>>>> If you need specific help for this problem I recommend commercial >>>>>>> support, for example ping [email protected]. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Roland >>>>>>> >>>>>>> 26 feb 2014 kl. 15:15 skrev Shiva Ramagopal <[email protected]>: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I have designed the core of my application using Akka actors which >>>>>>> I'm describing below: >>>>>>> >>>>>>> The application is a request-response system. There are three >>>>>>> entities - the client, the main server and the agent. >>>>>>> >>>>>>> Communication between all the entities are over plain HTTP. For >>>>>>> every request the client sends to the main server, the main server >>>>>>> sends a >>>>>>> request each to multiple remote agents, usually between 5 to 10. Each >>>>>>> agent >>>>>>> responds to the request within 100 ms with a piece of information which >>>>>>> the >>>>>>> main server collates and responds back to the client. >>>>>>> >>>>>>> The main server is on a separate machine and each of the agents are >>>>>>> on different machines. >>>>>>> >>>>>>> In my design the main server is designed using Akka actors. HTTP >>>>>>> front-ending is done using the Simple Framework. Each received HTTP >>>>>>> request >>>>>>> is converted to a "DataRequest" message (containing the HTTP response >>>>>>> object) and round-robin dispatched by a RouterActor to 5 instances of >>>>>>> MasterActor. A MasterActor then creates multiple AgentActor instances. >>>>>>> Each >>>>>>> AgentActor makes an async HTTP request (using Ning AsyncHttpClient with >>>>>>> NettyAsyncHttpProvider) to the remote agent and responds to the >>>>>>> MasterActor with the received data as a "DataResponse" message. The >>>>>>> MasterActor finally responds back to client with the collated data. >>>>>>> >>>>>>> Peformance testing on a 4-core i7 machine with 16 GB ram gave the >>>>>>> following results: >>>>>>> >>>>>>> Throughput: ~640 tps, peak >>>>>>> CPU usage: ~23% overall (application only, across all cores). >>>>>>> There's no other load on the HW. >>>>>>> Response times, as seen by the client: 105 ms (median), which >>>>>>> includes the 100 ms agent response time. >>>>>>> >>>>>>> My objective now is to increase the throughput to about 4000 tps >>>>>>> while maintaining the same end-to-end response time. >>>>>>> >>>>>>> 1. What's the maximum ballpark throughput that I can expect with the >>>>>>> above design on the same HW? Does aiming for 4000 tps sound reasonable? >>>>>>> >>>>>>> 2. What design changes or tuning can be done to achieve the target >>>>>>> of 4000 tps? >>>>>>> >>>>>>> I would greatly appreciate any pointers or suggestions on how to do >>>>>>> this. >>>>>>> >>>>>>> Best regards, >>>>>>> Shiva >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>> >>>>>>>>>> Check the FAQ: http://akka.io/faq/ >>>>>>> >>>>>>>>>> Search the archives: >>>>>>> https://groups.google.com/group/akka-user >>>>>>> --- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Akka User List" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To post to this group, send email to [email protected]. >>>>>>> Visit this group at http://groups.google.com/group/akka-user. >>>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> *Dr. Roland Kuhn* >>>>>>> *Akka Tech Lead* >>>>>>> Typesafe <http://typesafe.com/> - Reactive apps on the JVM. >>>>>>> twitter: @rolandkuhn >>>>>>> <http://twitter.com/#!/rolandkuhn> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>> >>>>>>>>>> Check the FAQ: http://akka.io/faq/ >>>>>>> >>>>>>>>>> Search the archives: >>>>>>> https://groups.google.com/group/akka-user >>>>>>> --- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Akka User List" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To post to this group, send email to [email protected]. >>>>>>> Visit this group at http://groups.google.com/group/akka-user. >>>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>> >>>>>>>>>> Check the FAQ: http://akka.io/faq/ >>>>>> >>>>>>>>>> Search the archives: >>>>>> https://groups.google.com/group/akka-user >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Akka User List" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> Visit this group at http://groups.google.com/group/akka-user. >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *Dr. Roland Kuhn* >>>>>> *Akka Tech Lead* >>>>>> Typesafe <http://typesafe.com/> - Reactive apps on the JVM. >>>>>> twitter: @rolandkuhn >>>>>> <http://twitter.com/#!/rolandkuhn> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>> >>>>>>>>>> Check the FAQ: http://akka.io/faq/ >>>>>> >>>>>>>>>> Search the archives: >>>>>> https://groups.google.com/group/akka-user >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Akka User List" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> Visit this group at http://groups.google.com/group/akka-user. >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>> >>>>>>>>>> Check the FAQ: http://akka.io/faq/ >>>>> >>>>>>>>>> Search the archives: >>>>> https://groups.google.com/group/akka-user >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Akka User List" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at http://groups.google.com/group/akka-user. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>>> >>>>> >>>>> >>>>> *Dr. Roland Kuhn* >>>>> *Akka Tech Lead* >>>>> Typesafe <http://typesafe.com/> - Reactive apps on the JVM. >>>>> twitter: @rolandkuhn >>>>> <http://twitter.com/#!/rolandkuhn> >>>>> >>>>> -- >>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>> >>>>>>>>>> Check the FAQ: http://akka.io/faq/ >>>>> >>>>>>>>>> Search the archives: >>>>> https://groups.google.com/group/akka-user >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Akka User List" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at http://groups.google.com/group/akka-user. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>> >>>> >>>> >>>> -- >>>> Akka Team >>>> Typesafe - The software stack for applications that scale >>>> Blog: letitcrash.com >>>> Twitter: @akkateam >>>> >>>> -- >>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>> >>>>>>>>>> Check the FAQ: http://akka.io/faq/ >>>> >>>>>>>>>> Search the archives: >>>> https://groups.google.com/group/akka-user >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Akka User List" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at http://groups.google.com/group/akka-user. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://akka.io/faq/ >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: http://akka.io/faq/ >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: http://akka.io/faq/ >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/groups/opt_out.
