Hi,
we've done recently a little research regarding the performance of current
JVM based HTTP servers (and Nginx for comparison) and as regards the
results of Akka-Http we were rather unpleasantly surprised.
Here are the presumptions for the benchmark:
- Single handler handling GET /benchmark returns HTTP 200 OK with
payload benchmark and HTTP headers Date, Server, Content-Type and
Content-Length.
- Using mostly default settings, change them only when it's obvious that
the default settings do not work well in the benchmark.
- We are interested in the maximum number of requests per second and
latencies.
- Three rounds, first is warmup and the rest try to test the frameworks
under different loads (each test 60 seconds)
- Warmup - ./wrk -t38 -c50 -d60s --latency
http://benchmark-server/benchmark
- Round 1 - ./wrk -t38 -c500 -d60s --latency
http://benchmark-server/benchmark
- Round 2 - ./wrk -t38 -c1000 -d60s --latency
http://benchmark-server/benchmark
Hardware : client and benchmark machines: 40 CPUs (Intel(R) Xeon(R) CPU
E5-2630L v4 @ 1.80GHz), 64GB of RAM
Results:
Warmup
| metric | nginx | akka-http | colossus | finagle |
http4s | netty | spring | vertx |
|--- |--- |--- |--- |--- |---
|--- |--- |--- |
| req/s** | 377627.54 | 161957.96 | 341140.06 | 271592.28 |
147829.99 | 337583.43 | 149290.22 | 356968.36 |
| latency avg | 99.18μs | 656.77μs | 386.72μs | 579.93μs |
8.28ms | 336.11μs | 21.98ms | 345.04μs |
| latency 75th | 98.00μs | 255.00μs | 115.00μs | 149.00μs |
248.00μs | 131.00μs | 250.00μs | 101.00μs |
| latency 99th | 149.00μs | 815.00μs | 271.00μs | 820.00μs |
156.60ms | 198.00μs | 846.85ms | 188.00μs |
| cpu idle % | 90 | 36 | 76 | 61 | 13
| 75 | 57 | 76 |
Round 1
| metric | nginx | akka-http | colossus | finagle |
http4s | netty | spring | vertx |
|--- |--- |--- |--- |--- |---
|--- |--- |--- |
| req/s | 988019.69 | 245979.16 | 1007300.12 | 460936.09 |
149288.46 | 1021265.73 | 258088.17 | 990153.39 |
| latency avg | 642.29μs | 3.31ms | 831.16μs | 4.49ms |
122.44ms | 2.12ms | 9.57ms | 1.96ms |
| latency 75th | 537.00μs | 2.79ms | 572.00μs | 5.12ms |
157.41ms | 0.88ms | 2.20ms | 1.17ms |
| latency 99th | 1.06ms | 13.84ms | 3.77ms | 33.31ms |
931.98ms | 18.51ms | 321.54ms | 16.68ms |
| cpu idle % | 59 | 63 | 5 | 1 | 16
| 2 | 4 | 2 |
Round 2
| metric | nginx | akka-http | colossus | finagle |
http4s | netty | spring | vertx |
|--- |--- |--- |--- |--- |---
|--- |--- |--- |
| req/s | 988149.09 | 242432.27 | 1028527.92 | 463750.89 |
166627.09 | 1068348.53 | 257553.62 | 991568.36 |
| latency avg | 1.05ms | 9.36ms | 1.25ms | 4.91ms |
89.26ms | 3.82ms | 11.00ms | 3.31ms |
| latency 75th | 1.10ms | 10.66ms | 1.14ms | 6.40ms |
95.28ms | 5.10ms | 4.46ms | 4.61ms |
| latency 99th | 1.41ms | 43.96ms | 3.50ms | 33.56ms |
629.33ms | 31.28ms | 275.59ms | 22.80ms |
| cpu idle % | 57 | 62 | 4 | 1 | 14
| 2 | 5 | 1 |
As you can see from the results Akka-Http (2.5.4 + Scala 2.12) didn't
performed very well, from requests per seconds perspective as well as
latency. We accounting it to the utilising of CPUs, which comparing to the
other servers was idle for around 60%. We were trying to tweak the
parallelism (set parallelism-max to num of cores or higher), used
different executers (fork-join-executer, affinity-pool-executer and
thread-pool-executer,) and some others stream or http.server settings but
none of those helped much actually.
Are we missing something fundamental or are there some other settings which
might help to increase CPU utilisation or increase performance ?
Thank you
Jakub
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
