On Thu, Dec 20, 2018 at 09:17:00AM +0100, Aleksandar Lazic wrote:
> Runtime API Improvements: It would be nice when you add a block that hanging
> or
> dead processes can also be debugged with this API now. Maybe I have overseen
> it.
It is already the case. It's not shown on the article, but the main benefit
that comes from this mechanism is that you have the list of current and old
processes, and that you can access them all. It is something I've been
missing for a long time, to have a CLI connection to an old process that
does not want to die, to see what's happening.
> Server Queue Priority Control: It would be nice to have a example for server
> decision based on Server Queue Priority.
It's not a server decision, it's a dequeuing decision. To give you a concrete
example of what I'm using on my build farm, I'm using haproxy to load-balance
distcc traffic to a bunch of build nodes. Some files are large and slow to
compile, others are small and build fast. The time it takes to build the
largest file can be almost as long as the total build time. If these files
start to build after the other ones, the total build time increases because
I have to wait for such a large file to be built on one node at the end. So
instead what I'm doing is that I sort the queue by file size. Each time a
connection slot is available on a server, instead of picking the oldest
entry, haproxy picks the one with the largest file. This way large files
start to build before small ones and the build completes much faster.
But there is a caveat to doing this : if you have a very large number of
large files, you can leave small files starving in the queue till the
timeout strikes. That's what happened to me when building my kernels. So
I changed from set-priority-class to set-priority-offset to address this.
Now large files are built up to 10 seconds earlier and small files are
built up to 10 seconds later. By doing this I can respect both the size
ordering and bound the distance between the extremes, and I don't have
a timeout anymore.
This is what my config looks like (with the old set-priority-class
still present but commented out), I'm copy-pasting it here because I
think it's self-explanatory :
# wait for the payload to arrive before connecting to the server
tcp-request inspect-delay 1m
tcp-request content reject unless { distcc_param(DOTI) -m found }
# convert kilobytes to classes (negated)
# test: tcp-request content set-var(sess.size) str(00002b95)
tcp-request content set-var(sess.size) distcc_param(DOTI)
tcp-request content set-var(sess.prio)
var(sess.size),div(-1024),add(2047)
tcp-request content set-var(sess.prio) int(0) if { var(sess.prio) -m
int lt 0 }
tcp-request content set-var(sess.prio) int(2047) if { var(sess.prio)
-m int gt 2047 }
#tcp-request content set-priority-class var(sess.prio)
# offset up to +10 seconds for small files
tcp-request content set-priority-offset var(sess.prio),mul(5)
balance leastconn
default-server on-marked-down shutdown-sessions
# mcbin: 4xA72: their 4 CPUs must be used before any of the miqi
server lg1 192.168.0.201:3632 check weight 5 maxconn 4
server lg2 192.168.0.202:3632 check weight 5 maxconn 4
server lg3 192.168.0.203:3632 check weight 5 maxconn 4
server lg4 192.168.0.204:3632 check weight 5 maxconn 4
server lg5 192.168.0.205:3632 check weight 5 maxconn 4
server lg6 192.168.0.206:3632 check weight 5 maxconn 4
server lg7 192.168.0.207:3632 check weight 5 maxconn 4
# miqi: 4xA17
server miqi-1 192.168.0.225:3632 check weight 1 maxconn 4
server miqi-2 192.168.0.226:3632 check weight 1 maxconn 4
server miqi-3 192.168.0.227:3632 check weight 1 maxconn 4
server miqi-4 192.168.0.228:3632 check weight 1 maxconn 4
server miqi-5 192.168.0.229:3632 check weight 1 maxconn 4
server miqi-6 192.168.0.230:3632 check weight 1 maxconn 4
server miqi-7 192.168.0.231:3632 check weight 1 maxconn 4
server miqi-8 192.168.0.232:3632 check weight 1 maxconn 4
server miqi-9 192.168.0.233:3632 check weight 1 maxconn 4
server miqi-a 192.168.0.234:3632 check weight 1 maxconn 4
Willy
