Hi,
We use haproxy quite a but at soundhound for fronting our various
external and internal services. We are i the process of moving to a
container based deployment model. With kubernetes in particular, it's as
easy as editing 1 line to add a new server to a replication set. In one
of our applications, we have 50+ partitions, so when a new instance is
added, it is actually 50+ containers and 50+ proxy configs to edit.
Clearly we need to automate this more than we have so far.
I was looking around at how to do this and came up with an unusual idea.
Everybody uses DNS, but almost no one uses/knows about SRV records. SRV
records have the form of
_(service)._(protocol).name and return a target DNS name (like CNAME),
priority, weight and destination port. This is like MX records on
steroids.
This seems to have everything I need for the dynamic parts of a
configuration. The original name defines the port to listen on (this can
be a name that gets looked up in /etc/services or _31112 for a hard port
number). The set of records returned is the backend set. The ones that
have the lowest priority are active and the rest are backups. If you
want to do per backend commands, those can be put in as TXT records for
the name backend.system.name._(service)._(protocol).name. Similarly,
dynamic parts of the frontend and backend could be done with TXT records
for _frontend and _backend prefixes to the SRV name.
For now, we can assume that there is only one level of backup rather
than the arbitrary number that DNS allows.
There is an extra little bit that DNS can do for us. DNS has a notify
capability. When a zone updates, the dns server can notify it's slaves
with the new SOA record. It turns out that all you need to do is
generate a response that is almost an exact copy of the query you got,
in DNS wire form to make DNS happy and then do the lookups again.
My thought is that the typical thing you configure for a given proxy
would remain in the config file as now (what to check, connection
mode...). The front end becomes the SRV name and the backend is either
empty of have some command that says expand the SRV record and put it as
the backends.
The reason I like it is there are 75 different orchestration/automation
systems that could want to drive haproxy. They all talk to DNS already.
With this, there is a single interface that I need for the automation
from the view of both an orchestration system and haproxy. We have a
custom load balancer that we plan to drive the same way.
Here are my questions:
Do people think this is a interesting/good idea?
Is there anything that is missing and would make this a terrible idea?
If not fatally flawed, should this be run outside of haproxy as an
automation tool or should haproxy do this itself?
thanks,
jerry
