Dear all,
The forthcoming version of NaviServer will extend the current mapping
of requests to connection thread pools by making it introspectible and
dynamic (changeable at runtime). Here is the motivation for it:
Assume there is a web site expecting about 2000 requests per minute
taking <10ms, 300 requests <1sec, and 30 requests <5secs, where all
times are just the times spent in the connection thread (no input or
delivery spooling included). We assume for the time being, the request
processing times are independent from other requests, and that the
requests are taking actually this boundary times.
For such a website, a configuration with 8 connection threads would be
sufficient, even when all requests reach their boundary times. In the
table below, ERPM stands for expected requests per minute, and
MRPTM for maximum number of requests per thread and minute:
ms ERPM MRPTM Threads Needed
10 2000 6000 0.33
1000 300 60 5.00
5000 30 12 2.50
total 7.83
However, as we all know, good estimations are hard and furthermore,
sometimes, the requests are "bulky" (no uniform distribution). So let
us assume the server has to deal for a certain time period with 100
and not 30 of these slow requests. We see, that these requests can
bring the whole server to a still-stand, blocking the 8 available
connection threads alone (needing 8.33 threads).
ms ERPM MRPTM Threads Needed
10 2000 6000 0.33
1000 300 60 5.00
5000 100 12 8.33
total 13.67
When all available connection threads are busy, further incoming
requests are queued. When the queuing time is e.g. 1 second, a former
10ms request will take 1.01sec in the user's perception (~100 times
slower). When a "slow" request taking normally 1 sec is queued for
1sec, such a request would be just twice as slow for the user, so the
queuing penalty hurts much more for fast requests. When requests take
much longer than usual, users tend to press reload, and the situation
becomes even more worse, since there will be even more slow requests
to be handled. Once the queuing kicks in, and there is only one pool,
even an admin can't login and look what happens (except over the
control port, when enabled).
What's desired is a means to keep a server lively although certain
requests might become slow. When we can identify slow requests, and
let these be served by a separate pool, then the other 2.300 faster
requests can run mostly undisturbed by the 100 slow ones, and the harm
of underestimated slow requests will be much smaller. Therefore,
multiple queues can improve average QOS significantly.
With current NaviServer, the mapping of requests (method + urls) to
connection pools can happen only via config file. With the forthcoming
version, we can query and modify this mapping at runtime via the new
subcommands of ns_server:
ns_server ... map ...
ns_server ... mapped ...
ns_server ... unmap ...
When we measure the runtime of a request, we can use this information
to map future occurrences of the same request dynamically to a
different connection pool, e.g. a pool for slow requests. Note, that
for measuring the runtime, we must not include the queuing time,
otherwise it might happen, that all requests might be moved to the
slow pool due to queuing.
Therefore, the forthcoming version will as well provide a means to
obtain the "partialtimes" of a request (the same times, which can be
currently recorded to the access log)
ns_conn partialtimes
By using the partial times and the dynamic mapping commands, we can
define a request trace to perform the queue mapping like e.g.:
ns_register_trace GET /* {
set pt [ns_conn partialtimes]
set req [list [ns_conn method] [ns_conn url]]
set ctime [expr {[dict get $pt runtime] + [dict get $pt filtertime]}]
if {$ctime > 3.0 && [ns_server mapped $req] eq ""} {
ns_server -pool slow map -noinherit $req
}
}
The connection thread pools can bed defined as usual in the config
file:
ns_section "ns/server/${server}/pools"
ns_param fast "Fast lane pool"
ns_param slow "Slow lane pool"
ns_section "ns/server/${server}/pool/fast"
ns_param minthreads 4
ns_param maxthreads 4
ns_param map "GET /*.png"
ns_param map "GET /*.jpg"
ns_param map "GET /*.pdf"
ns_section "ns/server/${server}/pool/slow"
ns_param minthreads 5
ns_param maxthreads 15
ns_param maxconnections 200
By these definitions, the mappings to the "fast" pool are static, and
mapping to the "slow" pool are dynamic via the request trace. One can
certainly perform the mapping to the fast pool as well dynamically.
We had last week the most busy week of the year on our production
system, and the dynamic connection pool mapping helped as to improve
the user experience significantly. We are using there a more complex
setup, also using a "monitor" connection pool for admin requests; the
used rules for mapping are more differentiated, but these are
application specific and will differ from site to site.
These changes for naviserver are already committed to bitbucket.
I've also updated nsstats module to inspect the actual mappings.
all the best
-g
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
_______________________________________________
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
