On 3/7/18 5:28 PM, David Osborne wrote:
Thank you very much for that info Gustaf.
We'll have a look at whether we can adapt the ideas in boomerang to our environment.

Yes we suspected the Accept timings might be outwith the control of the server. Am I right in thinking that Runtimes (if not using writer threads) are also affected by network conditions since they include the time taken to deliver content back to the client?

Here are a few images for visualizing what happens: a typical request has a receiving, processing and delivery phase:

if NaviServer processes a request without "read-ahead", and there are no writer threads involved, the full time is spent in a connection thread, all is accounted in the "runtime". The time of the "receive" and "delivery" phase is only under partial control of the server; a slow sending or receiving client can cause long blocking time of a connection thread, which is a limited resource. This can be as well the cause of potential "slow read/write" attacks to servers.

Fortunately, NaviServer offers the means to delegate the receive and delivery phases to separate threads using async i/o, where every of these threads is able to handle thousands of sockets concurrently. The "driver" and "spooling" threads of NaviServer handle the receiving part, and the "writer" threads the delivery part. The http/tls drivers are configured with "read-ahead", other drivers are not, so that the receiving part is just the accept operation. With "readahead", one can define multiple driver threads listening on one port, and 0..n spooling threads for huge (file) uploads.

With these i/o threads in place, the picture looks like the following for read-ahead threads:

where only the orange time spans are tunes spent in a connection thread. With these in i/o threads every connection thread is able to handle much more requests per second than without those.  For application development, the times in the connection threads are the one which are influenced primarily by the application developers, so these are reported as runtimes of the requests. In reality, the picture is more complicated, since the server sends data back as soon it is available (it does not wait, until the whole reply is computed), such that the delivery starts actually already in the the "orange" phase, 2 threads can work pipelined on one one request. When writer threads are enabled, every writer operation is essentially just an enqueuing operation. When there are no writer threads enabled, the runtime will be dependent on the client (and the connection quality and conditions on the way to the client). So, writer threads are highly recommended.

hope this helps, all the best


Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
naviserver-devel mailing list

Reply via email to