On 01/10/2013 01:08 AM, Willy Tarreau wrote:
On Wed, Jan 09, 2013 at 11:01:40PM +0000, Steven Acreman wrote:
We use chef and ohai which talks to AWS to calculate node counts for
servers based off tags and metadata. We then have a cookbook that generates
the haproxy.cfg every time chef runs (on a cron). If the file changes we
reload the config which seems to keep the sessions alive.
Yes it does :-)
I want to verify this point, since each time I reload haproxy config
I get a few dozen of different error at the app on failed mysql &
cassanra connections/reads/writes against haproxy frontends
This is on 1.4.13, maybe keeping the sessions alive works only in newer
versions?
I'm referring to the haproxy reload method that uses "-sf" option -
which seems to let in flight sessions to finish, but new conenctions are
dropped
during reload.
Also there are some long running processes which have connections
through haproxy with very high client/server timeout configs
afaiu idle connections are considered as live sessions, but obviously
haproxy reload does not wait until all idle sessions timeout,
so wdym by haproxy keeping sessions alive during reload?
and is there a way to do true zero downtime reloads without application
facing any failed connection/reads/writes attempts?
Also many protocol client driver implementations can not properly
handle/recover from haproxy closed connections due to client/server
timeout.
I always identify them by seeing "can't identify protocol" in lsof for
the connection socket, which brings the app process into unusable state
which requires process restart. So I have to set very high client/server
timeouts, which is bad in many cases. I was wondering if there is some
option to make haproxy close connection "differently" so that app driver
could notice it better, since I have never seen such issues with these
"bad" drivers then not using haproxy in the middle.
tnx
Alex
There are far simpler ways to achieve the above. For instance a really
simple bash script that echo's static content to haproxy.cfg and the amazon
command line tools feeding a for loop that prints out the server lines with
their options.
Having said all of this.. I think it would be cool to allow a file based
backend. What I mean by that is that I would prefer to generate backend
files and upon those files changing haproxy would automatically load them.
So for instance on my static backend I may update the static.servers file
which is referenced in haproxy.cfg. I believe you can do similar things
with mod_jdk and apache.
I've been thinking for some time about having a "servers" section
where we could declare a farm and have all backends use this/these
farms. One benefit would be shared checks and another one would be
shared maxconns. But that's not always trivial, it needs some more
thinking (eg: to arbitrate whom to serve next when maxconn is enforced).
However one point of using a dedicated section is that you could load
haproxy with 3 files then :
haproxy -f global.cfg -f servers.cfg -f services.cfg
And voila !
I'm sure we'd have some corner cases if we do this, reason why it still
needs some thinking. Anyway, it's still classified as non-urgent in my
head :-)
Willy
