With my lib, the -version=embedded_httpd_threads build should
give more consistent results in tests like this.
The process pool it uses by default in a dub build is more crash
resilient, but does have a habit of dropping excessive concurrent
connections. This forces them to retry which slaughters
benchmarks like this. It will have like 5 ms 99th percentile (2x
faster than the same test with the threads version btw), but then
that final 1% of responses can take several seconds complete
(indeed with 256 concurrent on my box it takes a whopping 30
seconds!). Even with only like 40 concurrent, there's a final 1%
spike there, but it is more like 10ms so it isn't so noticeable,
but with hundreds it grows fast.
That's probably what you're seeing here. The thread build accepts
more smoothly and thus evens it out giving a nicer benchmark
number... but it actually performs worse on average in real world
deployments in my experience and is not as resilient to buggy
code segfaulting (with processes, the individual handler respawns
and resets that individual connection with no other requests
affected. with threads, the whole server must respawn which also
often slips by unnoticed but is more likely to disrupt unrelated
users).
There is a potential "fix" for the process handler to complete
these benchmarks more smoothly too, but it comes at a cost: even
in the long retry cases, at least the client has some feedback.
It knows its connection is not accepted and can respond
appropriately. At a minimum, they won't be shoveling data at you
yet. The "fix" though breaks this - you accept ALL the
connections, even if you are too busy to actually process them.
This leads to more inbound data potentially worsening the
existing congestion and leaving users more likely to just hang.
At least the unaccepted connection is specified (by TCP) to retry
later automatically, but if it is accepted, acknowledged, yet
unprocessed, it is unclear what to do. Odds are the user will
just be left hanging until the browser decides to timeout and
display its error which can actually take longer than the TCP
retry window.
My threads version does it this way anyway though. So it'd
probably look better on the benchmark.
But BTW stuff like this is why I don't put too much stock in
benchmarks. Even if you aren't "cheating" like checking length
instead of path and other tricks like that (which btw I think are
totally legitimate in some cases, I said recently I see it as a
*strength* when you can do that), it still leaves some nuance on
the ground. Is it crash resilient? Debuggable when it crashes? Is
it compatible with third-party libraries or force you to choose
from ones that share your particular event loop at risk of
blocking the whole server when you disobey? Does it *actually*
provide the scalability it claims to under real world conditions,
or did it optimize to the controlled conditions of benchmarks at
the expense of dynamic adaptation to reality?
Harder to measure those.
