Hi,

we've done recently a little research regarding the performance of current 
JVM based HTTP servers (and Nginx for comparison)  and as regards the 
results of Akka-Http we were rather unpleasantly surprised.

Here are the presumptions for the benchmark:

   - Single handler handling GET /benchmark returns HTTP 200 OK with 
   payload benchmark and HTTP headers Date, Server, Content-Type and 
   Content-Length.
   - Using mostly default settings, change them only when it's obvious that 
   the default settings do not work well in the benchmark.
   - We are interested in the maximum number of requests per second and 
   latencies.
   - Three rounds, first is warmup and the rest try to test the frameworks 
   under different loads (each test 60 seconds)
      - Warmup - ./wrk -t38 -c50 -d60s --latency 
      http://benchmark-server/benchmark
      - Round 1 - ./wrk -t38 -c500 -d60s --latency 
      http://benchmark-server/benchmark
      - Round 2 - ./wrk -t38 -c1000 -d60s --latency 
      http://benchmark-server/benchmark
      
Hardware : client and benchmark machines: 40 CPUs (Intel(R) Xeon(R) CPU 
E5-2630L v4 @ 1.80GHz), 64GB of RAM

Results:

Warmup

| metric          | nginx         | akka-http | colossus  | finagle   | 
http4s    | netty     | spring    | vertx     | 
|---              |---            |---        |---        |---        |--- 
       |---        |---        |---        |
| req/s**         | 377627.54     | 161957.96 | 341140.06 | 271592.28 | 
147829.99 | 337583.43 | 149290.22 | 356968.36 |
| latency avg     | 99.18μs       | 656.77μs  | 386.72μs  | 579.93μs  | 
8.28ms    | 336.11μs  | 21.98ms   | 345.04μs  |
| latency 75th    | 98.00μs       | 255.00μs  | 115.00μs  | 149.00μs  | 
248.00μs  | 131.00μs  | 250.00μs  | 101.00μs  | 
| latency 99th    | 149.00μs      | 815.00μs  | 271.00μs  | 820.00μs  | 
156.60ms  | 198.00μs  | 846.85ms  | 188.00μs  | 
| cpu idle %      | 90            | 36        | 76        | 61        | 13 
       | 75        | 57        | 76        | 


Round 1

| metric          | nginx         | akka-http | colossus   | finagle   | 
http4s    | netty      | spring    | vertx     |
|---              |---            |---        |---         |---        |--- 
       |---         |---        |---        |
| req/s           | 988019.69     | 245979.16 | 1007300.12 | 460936.09 | 
149288.46 | 1021265.73 | 258088.17 | 990153.39 |
| latency avg     | 642.29μs      | 3.31ms    | 831.16μs   | 4.49ms    | 
122.44ms  | 2.12ms     | 9.57ms    | 1.96ms    | 
| latency 75th    | 537.00μs      | 2.79ms    | 572.00μs   | 5.12ms    | 
157.41ms  | 0.88ms     | 2.20ms    | 1.17ms    | 
| latency 99th    | 1.06ms        | 13.84ms   | 3.77ms     | 33.31ms   | 
931.98ms  | 18.51ms    | 321.54ms  | 16.68ms   | 
| cpu idle %      | 59            | 63        | 5          | 1         | 16 
       | 2          | 4         | 2         |


Round 2

| metric           | nginx      | akka-http | colossus   | finagle   | 
http4s    | netty        | spring    | vertx     |
|---               |---         |---        |---         |---        |---   
     |---           |---        |---        |
| req/s            | 988149.09  | 242432.27 | 1028527.92 | 463750.89 | 
166627.09 | 1068348.53   | 257553.62 | 991568.36 | 
| latency avg      | 1.05ms     | 9.36ms    | 1.25ms     | 4.91ms    | 
89.26ms   | 3.82ms       | 11.00ms   | 3.31ms    | 
| latency 75th     | 1.10ms     | 10.66ms   | 1.14ms     | 6.40ms    | 
95.28ms   | 5.10ms       | 4.46ms    | 4.61ms    |
| latency 99th     | 1.41ms     | 43.96ms   | 3.50ms     | 33.56ms   | 
629.33ms  | 31.28ms      | 275.59ms  | 22.80ms   | 
| cpu idle %       | 57         | 62        | 4          | 1         | 14   
     | 2            | 5         | 1         | 


As you can see from the results Akka-Http (2.5.4 + Scala 2.12) didn't 
performed very well, from requests per seconds perspective as well as 
latency. We accounting it to the utilising of CPUs, which comparing to the 
other servers was idle for around 60%. We were trying to tweak the 
parallelism (set  parallelism-max to num of cores or higher), used 
different executers (fork-join-executer, affinity-pool-executer and 
thread-pool-executer,) and some others stream or http.server settings  but 
none of those helped much actually. 

Are we missing something fundamental or are there some other settings which 
might help to increase CPU utilisation or increase performance ?

Thank you

Jakub





-- 
>>>>>>>>>>      Read the docs: http://akka.io/docs/
>>>>>>>>>>      Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>      Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Reply via email to