Also, I'd recommend avoiding App for benches, I don't know how it works nowadays but I recall seeing issues with inlining (OSR) back in the day.
On Tue, Sep 6, 2016 at 4:38 PM, Konrad Malawski < [email protected]> wrote: > Without looking at the code, two things to fix right away in your > methodology: > > 1) You're running `ab` without persistent connections (also known as > "useless mode"). > Please run: `ab -k` to use keep-alive connections, which is what all http > clients and browsers do. > > 2) Please do not benchmark using the same host for running the app and > load generator... > You're stealing resources from the http server as you're load generator is > trying to hit it as hard as it can, and the other way around. > The benchmark will be completely skewed by that. > > ab is also not very good in general, use wrk instead - a more modern load > generator. > > -- > Konrad `ktoso` Malawski > Akka <http://akka.io> @ Lightbend <http://lightbend.com> > > On 6 September 2016 at 16:34:22, Adam ([email protected]) wrote: > > Hi, > > I've previously seen bad performance form Akka HTTP for a simple ping/pong > scenario (which I realize according to the release notes is still a > relatively bad scenario for Akka HTTP), but the results I'm getting are bad > enough that I'm wondering if my test is even valid. > > Here's my code: > > package com.example.rest > > import akka.NotUsed > import akka.actor.ActorSystem > import akka.http.scaladsl.Http > import akka.http.scaladsl.model._ > import akka.stream.scaladsl.{Flow, Keep, RunnableGraph, Sink, Source} > import akka.stream.{ActorMaterializer, ActorMaterializerSettings, Fusing} > > import scala.concurrent.Future > import scala.io.StdIn > > object WebServer extends App { > implicit val system = ActorSystem() > implicit val materializer = > ActorMaterializer(ActorMaterializerSettings(system).withAutoFusing(false)) > implicit val executionContext = system.dispatcher > > val response = HttpResponse(entity = > HttpEntity(ContentTypes.`text/plain(UTF-8)`,"Ok")) > val requestHandler: HttpRequest => HttpResponse = { > case _ => response > } > > val flow: Flow[HttpRequest, HttpResponse, NotUsed] = Flow[HttpRequest] map > requestHandler > val prefused = Fusing.aggressive(flow) > val httpHandler: Flow[HttpRequest, HttpResponse, NotUsed] = > Flow.fromGraph(prefused) > > if (args(0) == "http"){ > runHttpServer() > } else { > runPingPong(args(1).toInt) > } > > def runHttpServer() = { > val bindingFuture = Http().bindAndHandle(handler = httpHandler, interface > = "127.0.0.1 ", port = 3000) > > println("Type RETURN to exit") > StdIn.readLine() > > bindingFuture > .flatMap(_.unbind()) // trigger unbinding from the port > .onComplete(_ => system.terminate()) > } > > def runPingPong(times: Int) = { > val ping = HttpRequest().withUri(Uri("/")) > > val graph: RunnableGraph[Future[Int]] = Source(1 to times) > .map(_ => ping) > .via(httpHandler) > .map(_ => 1) > .toMat(Sink.fold[Int, Int](0)((a, b) => a + 1))(Keep.right) > > val startTime = System.nanoTime() > val count: Future[Int] = graph.run() > > count.onComplete(res => { > if (res.isFailure) { > Console.err.println(s"failed with: ${res.failed}") > } else { > println(s"ran for ${res.get} times") > val et = System.nanoTime() - startTime > val etMillis = et / 1000000.0 > println(s"et: ${etMillis}ms") > } > system.terminate() > }) > } > } > > The in memory scenario (runPingPong) yields the following results on my local > windows machine: > > ran for 100000 times > et: 322.150515ms > > > Running ab to test actual local HTTP calls get me this: > > ab -c 400 -n 100000 -m GET http://127.0.0.1:3000/ > > This is ApacheBench, Version 2.3 <$Revision: 1706008 $> > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > Licensed to The Apache Software Foundation, http://www.apache.org/ > > Benchmarking 127.0.0.1 (be patient) > Completed 10000 requests > Completed 20000 requests > Completed 30000 requests > Completed 40000 requests > Completed 50000 requests > Completed 60000 requests > Completed 70000 requests > Completed 80000 requests > Completed 90000 requests > Completed 100000 requests > Finished 100000 requests > > > Server Software: akka-http/2.4.9 > Server Hostname: 127.0.0.1 > Server Port: 3000 > > Document Path: / > Document Length: 2 bytes > > Concurrency Level: 400 > Time taken for tests: 139.083 seconds > Complete requests: 100000 > Failed requests: 0 > Total transferred: 16200000 bytes > HTML transferred: 200000 bytes > Requests per second: 719.00 [#/sec] (mean) > Time per request: 556.332 [ms] (mean) > Time per request: 1.391 [ms] (mean, across all concurrent requests) > Transfer rate: 113.75 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 1 24.7 0 593 > Processing: 13 552 394.6 562 2144 > Waiting: 2 388 347.1 526 1583 > Total: 13 554 395.2 562 2144 > > Percentage of the requests served within a certain time (ms) > 50% 562 > 66% 586 > 75% 639 > 80% 1047 > 90% 1065 > 95% 1096 > 98% 1560 > 99% 1574 > 100% 2144 (longest request) > > > I was able to use the same approach with other frameworks (e.g. Play & > Vertx) and got results which are much better. > I see in the 2.4.9 release notes that (on different hardware) 160K > reqs/sec ping/pong requests were achieved, so what I'm seeing on my machine > is really weird to me. > Is there something fundamentally wrong with my test above? > It would also be nice to see the code & configuration which was used to > test 2.4.9's performance on EC2, if that's available somewhere. > > Adam > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ > current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ > current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Cheers, √ -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
