On Friday, September 8, 2017 at 4:29:34 PM UTC+3, Neil Van Dyke wrote: > dbohdan wrote on 09/07/2017 04:52 PM: > > The #/sec for each implementation are suspiciously similar. I wonder > whether they're limited by something like an accounting limit imposed on > the VPS (such as network-bytes-per-second or TCP-connections-open), or > by some other host/network limit. >
While the VPS provider does impose a limit on throughput, at approximately 250 req/s * 5 KB/req = 1.25 MB/s I wasn't hitting it. The numbers were very similar for different applications because at 25 concurrent connections no application reached the maximum request rate it could sustain. I thought the memory constraints wouldn't allow for more than about 25 connections, but I was mistaken. With some tuning I was able to get the applications that ran with 25 concurrent connections to run with 50 and 100. I've rerun the benchmark with 1, 25, 50, 100, and 200 connections to show how the differences between the applications emerge. ====== CONNECTIONS=1 remote-results/caddy.txt:Requests per second: 12.57 [#/sec] (mean) remote-results/compojure.txt:Requests per second: 12.41 [#/sec] (mean) remote-results/custom-many-places.txt:Requests per second: 12.62 [#/sec] (mean) remote-results/custom-many.txt:Requests per second: 12.55 [#/sec] (mean) remote-results/custom-places.txt:Requests per second: 12.56 [#/sec] (mean) remote-results/custom-single.txt:Requests per second: 12.58 [#/sec] (mean) remote-results/flask.txt:Requests per second: 12.44 [#/sec] (mean) remote-results/guile.txt:Requests per second: 12.53 [#/sec] (mean) remote-results/plug.txt:Requests per second: 12.57 [#/sec] (mean) remote-results/scgi.txt:Requests per second: 12.46 [#/sec] (mean) remote-results/sinatra.txt:Requests per second: 12.08 [#/sec] (mean) remote-results/stateful.txt:Requests per second: 12.42 [#/sec] (mean) remote-results/stateless.txt:Requests per second: 12.41 [#/sec] (mean) ====== ====== CONNECTIONS=25 remote-results/caddy.txt:Requests per second: 311.19 [#/sec] (mean) remote-results/compojure.txt:Requests per second: 309.69 [#/sec] (mean) remote-results/custom-many-places.txt:(Killed) Total of 9153 requests completed remote-results/custom-many.txt:Requests per second: 309.63 [#/sec] (mean) remote-results/custom-places.txt:(Killed) Total of 13085 requests completed remote-results/custom-single.txt:Requests per second: 308.02 [#/sec] (mean) remote-results/flask.txt:Requests per second: 310.91 [#/sec] (mean) remote-results/guile.txt:Requests per second: 310.28 [#/sec] (mean) remote-results/plug.txt:Requests per second: 313.60 [#/sec] (mean) remote-results/sinatra.txt:Requests per second: 287.03 [#/sec] (mean) remote-results/stateful.txt:Requests per second: 298.05 [#/sec] (mean) remote-results/stateless.txt:Requests per second: 295.90 [#/sec] (mean) ====== ====== CONNECTIONS=50 remote-results/caddy.txt:Requests per second: 594.78 [#/sec] (mean) remote-results/compojure.txt:Requests per second: 604.64 [#/sec] (mean) remote-results/custom-many-places.txt:(Killed) Total of 9444 requests completed remote-results/custom-many.txt:Requests per second: 598.88 [#/sec] (mean) remote-results/custom-places.txt:(Killed) Total of 13088 requests completed remote-results/custom-single.txt:Requests per second: 591.44 [#/sec] (mean) remote-results/flask.txt:Requests per second: 605.75 [#/sec] (mean) remote-results/guile.txt:Requests per second: 612.28 [#/sec] (mean) remote-results/plug.txt:Requests per second: 617.95 [#/sec] (mean) remote-results/scgi.txt:(Killed) Total of 12020 requests completed remote-results/sinatra.txt:Requests per second: 367.58 [#/sec] (mean) remote-results/stateful.txt:Requests per second: 530.00 [#/sec] (mean) remote-results/stateless.txt:Requests per second: 546.76 [#/sec] (mean) ====== ====== CONNECTIONS=100 remote-results/caddy.txt:Requests per second: 1016.63 [#/sec] (mean) remote-results/compojure.txt:Requests per second: 1103.84 [#/sec] (mean) remote-results/custom-many-places.txt:(Killed) Total of 9908 requests completed remote-results/custom-many.txt:Requests per second: 1140.40 [#/sec] (mean) remote-results/custom-places.txt:(Killed) Total of 13081 requests completed remote-results/custom-single.txt:Requests per second: 1134.93 [#/sec] (mean) remote-results/flask.txt:Requests per second: 1024.25 [#/sec] (mean) remote-results/guile.txt:Requests per second: 1085.03 [#/sec] (mean) remote-results/plug.txt:Requests per second: 1140.43 [#/sec] (mean) remote-results/scgi.txt:(Killed) Total of 10969 requests completed remote-results/sinatra.txt:Requests per second: 384.41 [#/sec] (mean) remote-results/stateful.txt:Requests per second: 726.84 [#/sec] (mean) remote-results/stateless.txt:Requests per second: 682.58 [#/sec] (mean) ====== ====== CONNECTIONS=200 remote-results/caddy.txt:Requests per second: 1093.88 [#/sec] (mean) remote-results/compojure.txt:Requests per second: 1157.95 [#/sec] (mean) remote-results/custom-many-places.txt:(Killed) Total of 9728 requests completed remote-results/custom-many.txt:Requests per second: 1219.76 [#/sec] (mean) remote-results/custom-places.txt:(Killed) Total of 13154 requests completed remote-results/custom-single.txt:Requests per second: 1171.37 [#/sec] (mean) remote-results/flask.txt:Requests per second: 937.90 [#/sec] (mean) remote-results/guile.txt:Requests per second: 1182.95 [#/sec] (mean) remote-results/plug.txt:Requests per second: 1222.03 [#/sec] (mean) remote-results/scgi.txt:(Killed) Total of 7857 requests completed remote-results/sinatra.txt:Requests per second: 381.36 [#/sec] (mean) remote-results/stateful.txt:(Killed) Total of 97039 requests completed remote-results/stateless.txt:(Killed) Total of 24712 requests completed ====== For comparison, here is one concurrent connection over a local network. With greater computing resources and a lower round-trip time (the latter is probably far more important) you get a much higher request rate. ====== CONNECTIONS=1 remote-results/caddy.txt:Requests per second: 679.79 [#/sec] (mean) remote-results/compojure.txt:Requests per second: 680.88 [#/sec] (mean) remote-results/custom-many-places.txt:Requests per second: 842.54 [#/sec] (mean) remote-results/custom-many.txt:Requests per second: 899.55 [#/sec] (mean) remote-results/custom-places.txt:Requests per second: 841.69 [#/sec] (mean) remote-results/custom-single.txt:Requests per second: 775.15 [#/sec] (mean) remote-results/flask.txt:Requests per second: 513.99 [#/sec] (mean) remote-results/guile.txt:Requests per second: 661.27 [#/sec] (mean) remote-results/plug.txt:Requests per second: 678.93 [#/sec] (mean) remote-results/scgi.txt:Requests per second: 606.11 [#/sec] (mean) remote-results/sinatra.txt:Requests per second: 247.25 [#/sec] (mean) remote-results/stateful.txt:Requests per second: 406.68 [#/sec] (mean) remote-results/stateless.txt:Requests per second: 412.21 [#/sec] (mean) ====== > For purposes of this last humble-VPS benchmarking (if we can keep making > more benchmarking work for you), you might get those initial numbers > from places/many-places/racket-scgi by setting Racket's memory usage > limit. When I limit the memory usage in racket-custom to the total RAM on the VPS minus what the OS uses (through custodian-limit-memory) Racket quits with an out of memory error at the point when it would be killed by the OS. racket-scgi seems to behave the same, though I didn't look at the memory usage split between Racket and nginx when I tested it. > For the racket-scgi + nginx setup, if nginx can't quickly be tuned to > not be a problem itself, there are HTTP servers targeting smaller > devices, like what OpenWrt uses for its admin interface. But do they support SCGI? -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
