On Friday, September 8, 2017 at 4:29:34 PM UTC+3, Neil Van Dyke wrote:
> dbohdan wrote on 09/07/2017 04:52 PM:
> The #/sec for each implementation are suspiciously similar.  I wonder 
> whether they're limited by something like an accounting limit imposed on 
> the VPS (such as network-bytes-per-second or TCP-connections-open), or 
> by some other host/network limit.

While the VPS provider does impose a limit on throughput, at approximately 250 
req/s * 5 KB/req = 1.25 MB/s I wasn't hitting it. The numbers were very similar 
for different applications because at 25 concurrent connections no application 
reached the maximum request rate it could sustain. I thought the memory 
constraints wouldn't allow for more than about 25 connections, but I was 
mistaken. With some tuning I was able to get the applications that ran with 25 
concurrent connections to run with 50 and 100. I've rerun the benchmark with 1, 
25, 50, 100, and 200 connections to show how the differences between the 
applications emerge.

remote-results/caddy.txt:Requests per second:    12.57 [#/sec] (mean)
remote-results/compojure.txt:Requests per second:    12.41 [#/sec] (mean)
remote-results/custom-many-places.txt:Requests per second:    12.62 [#/sec] 
remote-results/custom-many.txt:Requests per second:    12.55 [#/sec] (mean)
remote-results/custom-places.txt:Requests per second:    12.56 [#/sec] (mean)
remote-results/custom-single.txt:Requests per second:    12.58 [#/sec] (mean)
remote-results/flask.txt:Requests per second:    12.44 [#/sec] (mean)
remote-results/guile.txt:Requests per second:    12.53 [#/sec] (mean)
remote-results/plug.txt:Requests per second:    12.57 [#/sec] (mean)
remote-results/scgi.txt:Requests per second:    12.46 [#/sec] (mean)
remote-results/sinatra.txt:Requests per second:    12.08 [#/sec] (mean)
remote-results/stateful.txt:Requests per second:    12.42 [#/sec] (mean)
remote-results/stateless.txt:Requests per second:    12.41 [#/sec] (mean)

remote-results/caddy.txt:Requests per second:    311.19 [#/sec] (mean)
remote-results/compojure.txt:Requests per second:    309.69 [#/sec] (mean)
remote-results/custom-many-places.txt:(Killed) Total of 9153 requests completed
remote-results/custom-many.txt:Requests per second:    309.63 [#/sec] (mean)
remote-results/custom-places.txt:(Killed) Total of 13085 requests completed
remote-results/custom-single.txt:Requests per second:    308.02 [#/sec] (mean)
remote-results/flask.txt:Requests per second:    310.91 [#/sec] (mean)
remote-results/guile.txt:Requests per second:    310.28 [#/sec] (mean)
remote-results/plug.txt:Requests per second:    313.60 [#/sec] (mean)
remote-results/sinatra.txt:Requests per second:    287.03 [#/sec] (mean)
remote-results/stateful.txt:Requests per second:    298.05 [#/sec] (mean)
remote-results/stateless.txt:Requests per second:    295.90 [#/sec] (mean)

remote-results/caddy.txt:Requests per second:    594.78 [#/sec] (mean)
remote-results/compojure.txt:Requests per second:    604.64 [#/sec] (mean)
remote-results/custom-many-places.txt:(Killed) Total of 9444 requests completed
remote-results/custom-many.txt:Requests per second:    598.88 [#/sec] (mean)
remote-results/custom-places.txt:(Killed) Total of 13088 requests completed
remote-results/custom-single.txt:Requests per second:    591.44 [#/sec] (mean)
remote-results/flask.txt:Requests per second:    605.75 [#/sec] (mean)
remote-results/guile.txt:Requests per second:    612.28 [#/sec] (mean)
remote-results/plug.txt:Requests per second:    617.95 [#/sec] (mean)
remote-results/scgi.txt:(Killed) Total of 12020 requests completed
remote-results/sinatra.txt:Requests per second:    367.58 [#/sec] (mean)
remote-results/stateful.txt:Requests per second:    530.00 [#/sec] (mean)
remote-results/stateless.txt:Requests per second:    546.76 [#/sec] (mean)

remote-results/caddy.txt:Requests per second:    1016.63 [#/sec] (mean)
remote-results/compojure.txt:Requests per second:    1103.84 [#/sec] (mean)
remote-results/custom-many-places.txt:(Killed) Total of 9908 requests completed
remote-results/custom-many.txt:Requests per second:    1140.40 [#/sec] (mean)
remote-results/custom-places.txt:(Killed) Total of 13081 requests completed
remote-results/custom-single.txt:Requests per second:    1134.93 [#/sec] (mean)
remote-results/flask.txt:Requests per second:    1024.25 [#/sec] (mean)
remote-results/guile.txt:Requests per second:    1085.03 [#/sec] (mean)
remote-results/plug.txt:Requests per second:    1140.43 [#/sec] (mean)
remote-results/scgi.txt:(Killed) Total of 10969 requests completed
remote-results/sinatra.txt:Requests per second:    384.41 [#/sec] (mean)
remote-results/stateful.txt:Requests per second:    726.84 [#/sec] (mean)
remote-results/stateless.txt:Requests per second:    682.58 [#/sec] (mean)

remote-results/caddy.txt:Requests per second:    1093.88 [#/sec] (mean)
remote-results/compojure.txt:Requests per second:    1157.95 [#/sec] (mean)
remote-results/custom-many-places.txt:(Killed) Total of 9728 requests completed
remote-results/custom-many.txt:Requests per second:    1219.76 [#/sec] (mean)
remote-results/custom-places.txt:(Killed) Total of 13154 requests completed
remote-results/custom-single.txt:Requests per second:    1171.37 [#/sec] (mean)
remote-results/flask.txt:Requests per second:    937.90 [#/sec] (mean)
remote-results/guile.txt:Requests per second:    1182.95 [#/sec] (mean)
remote-results/plug.txt:Requests per second:    1222.03 [#/sec] (mean)
remote-results/scgi.txt:(Killed) Total of 7857 requests completed
remote-results/sinatra.txt:Requests per second:    381.36 [#/sec] (mean)
remote-results/stateful.txt:(Killed) Total of 97039 requests completed
remote-results/stateless.txt:(Killed) Total of 24712 requests completed

For comparison, here is one concurrent connection over a local network. With 
greater computing resources and a lower round-trip time (the latter is probably 
far more important) you get a much higher request rate.

remote-results/caddy.txt:Requests per second:    679.79 [#/sec] (mean)
remote-results/compojure.txt:Requests per second:    680.88 [#/sec] (mean)
remote-results/custom-many-places.txt:Requests per second:    842.54 [#/sec] 
remote-results/custom-many.txt:Requests per second:    899.55 [#/sec] (mean)
remote-results/custom-places.txt:Requests per second:    841.69 [#/sec] (mean)
remote-results/custom-single.txt:Requests per second:    775.15 [#/sec] (mean)
remote-results/flask.txt:Requests per second:    513.99 [#/sec] (mean)
remote-results/guile.txt:Requests per second:    661.27 [#/sec] (mean)
remote-results/plug.txt:Requests per second:    678.93 [#/sec] (mean)
remote-results/scgi.txt:Requests per second:    606.11 [#/sec] (mean)
remote-results/sinatra.txt:Requests per second:    247.25 [#/sec] (mean)
remote-results/stateful.txt:Requests per second:    406.68 [#/sec] (mean)
remote-results/stateless.txt:Requests per second:    412.21 [#/sec] (mean)

> For purposes of this last humble-VPS benchmarking (if we can keep making
> more benchmarking work for you), you might get those initial numbers
> from places/many-places/racket-scgi by setting Racket's memory usage
> limit.

When I limit the memory usage in racket-custom to the total RAM on the VPS 
minus what the OS uses (through custodian-limit-memory) Racket quits with an 
out of memory error at the point when it would be killed by the OS. racket-scgi 
seems to behave the same, though I didn't look at the memory usage split 
between Racket and nginx when I tested it.

> For the racket-scgi + nginx setup, if nginx can't quickly be tuned to
> not be a problem itself, there are HTTP servers targeting smaller
> devices, like what OpenWrt uses for its admin interface.

But do they support SCGI?

You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to