It could be a good idea to take stages out, measure the throughput while adding a stage at a time to see what actually is limiting your throughput.
If it in fact is Akka HTTP, then the default configuration for the connection pool only allows 4 connections per host, maybe increasing this (akka.http.host-connection-pool.max-connections) would help. It could actually be easier to reason about the flow by using singleRequest inside a mapAsync or a mapAsyncUnordered with a parallelism matching the max-connection per host setting. It could also make sense to insert async boundaries, to let the logic before the request execute in parallel with the request being executed. -- Johan Akka Team On Tue, Jan 9, 2018 at 4:23 AM, kor <[email protected]> wrote: > Hello all. I have a requirement where I need to stream records from a > couchbase database, do some json transformations and post the result to a > rest endpoint (implemented with Akka-Http). I've tested that endpoint in > isolation using gatling, and it can handle at least 200 requests/sec. The > problem I am having when I run it, I cannot get the speed to go past 10 > requests/sec. I've tested the speed that records are being pulled out of > couchbase and got a rate of about 2700 records/sec so something is causing > backpressure to be applied to the source. I've also tried posting to a > different public endpoint (probably not implemented with akka-http) and the > speed increases to about 80 requests/sec. I've posted the code below. How > can I make things go faster? Disclaimer: I am new to akka-streams. > > code: https://gist.github.com/khalidr/1a392680c0ed744b2d3e5cec1e1f193a > > Any help would be appreciated. > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ > current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
