Re: VM Power Control/Elasticity
For what it's worth, I've implemented a basic script in PHP (please don't yell at me for using PHP) that sucks in the info from the haproxy socket and then executes commands to the XenServer hosts using the XenAPI (XAPI). It works decently well - I have the script running every 5 minutes, which seems to be okay so far for the demand on these systems. It seems to be an okay balance between boot time of the systems and people connecting to the systems. I will say it would still be better to have something trigger-based rather than polling. I even looked into trying to use iptables on Linux to trigger something upon connection to a certain port, but iptables doesn't quite have the right functionality to do it. In any case, the periodic script will suffice for now, and I'm sure I'll change and improve upon it a bit. For the time being it has been able to manage my overall usage of physical resources a little more efficiently as it can spin down VMs when they aren't being used, which frees those resources for other systems and cuts the overall heat generation and power consumption. -Nick From: Ben Timby bti...@gmail.com To: Nick E Couchman nick.couch...@seakr.com Cc: haproxy@formilux.org Sent: Monday, May 11, 2015 11:35:03 AM Subject: Re: VM Power Control/Elasticity Nick, Here is some information on using socat to interact with the stats socket. This might be useful for shell scripting. http://www.mgoff.in/2010/07/14/haproxy-gathering-stats-using-socat/
Re: VM Power Control/Elasticity
Thanks for the info, CJ. Has this feature been considered - the ability to add/remove on-the-fly? Anyway, I think in my situation I would just list all of the hosts in the config file initially and have some of them down, then I would already know which ones to start up to bump up the connection limit. I think the use of the socket will get me most of the way toward where I want to go - particularly since the output comes in easily-parsable CSV. Thanks, everyone! -Nick From: CJ Ess zxcvbn4...@gmail.com To: Nick E Couchman nick.couch...@seakr.com Cc: HAProxy haproxy@formilux.org Sent: Tuesday, May 12, 2015 9:46:00 AM Subject: Re: VM Power Control/Elasticity You can't add or remove hosts to a pool without doing a reload - you can change the weights, mark them up and down, but not add or remove. On Mon, May 11, 2015 at 1:00 PM, Nick Couchman nick.couch...@seakr.com wrote: I was wondering if it is possible or there's a recommended way to deal with dynamic capacity expansion for a given back-end. I searched through the documentation some and didn't see anything obvious, so figured I would ask here. Basically, I would like a way to trigger a script when the number of active connections gets within a certain range of the total available connections for the available backend servers. I would like this script to be able to do something like trigger a script or command that creates or powers on or off a virtual machine that is already added to or could be dynamically added to or removed from the back-end. The basic scenario is this: - Back-end starts with 10 configured systems or 10 connections each. - 5 of the 10 systems are powered on by default, with the other 5 down. - Users connect, and close to 40/50 available connections. - HAProxy detects the connection limit and triggers a script that starts up the 6th VM. - Once HAProxy detects that the 6th VM is running, the number of available connections moves to 60. - Users continue to connect and close to 50/60, triggering another power event. I'd also like the reverse of that to happen: - Users begin to disconnect and connections drop to 40/60. - HAProxy triggers another script to stop one of the the configured back-end systems that has zero connections. Is this possible? Or on the roadmap? Or something that, while not implemented directly in the HAProxy configuration could be done some other way - some periodic polling of HAProxy some other way? Thanks, Nick
VM Power Control/Elasticity
I was wondering if it is possible or there's a recommended way to deal with dynamic capacity expansion for a given back-end. I searched through the documentation some and didn't see anything obvious, so figured I would ask here. Basically, I would like a way to trigger a script when the number of active connections gets within a certain range of the total available connections for the available backend servers. I would like this script to be able to do something like trigger a script or command that creates or powers on or off a virtual machine that is already added to or could be dynamically added to or removed from the back-end. The basic scenario is this: - Back-end starts with 10 configured systems or 10 connections each. - 5 of the 10 systems are powered on by default, with the other 5 down. - Users connect, and close to 40/50 available connections. - HAProxy detects the connection limit and triggers a script that starts up the 6th VM. - Once HAProxy detects that the 6th VM is running, the number of available connections moves to 60. - Users continue to connect and close to 50/60, triggering another power event. I'd also like the reverse of that to happen: - Users begin to disconnect and connections drop to 40/60. - HAProxy triggers another script to stop one of the the configured back-end systems that has zero connections. Is this possible? Or on the roadmap? Or something that, while not implemented directly in the HAProxy configuration could be done some other way - some periodic polling of HAProxy some other way? Thanks, Nick
Re: VM Power Control/Elasticity
Thanks for the hints, Ben. I'll defer to those who are experts about whether or not something like that should be part of the core functionality; however, it seems that even though this case might not be a great one for adding that to the core, there are a multitude of reasons why you'd want some sort of trigger mechanism on events within HAProxy. I think people have asked here before about e-mail notifications for downed hosts, and I can think of a few other cases outside of mine corner case that would seem to warrant some generic trigger mechanism within HAProxy. Seems if a generic one is implemented the flexibility is there for everyone who needs a trigger of some sort or another to use it for whatever purposes suite their needs. I'm not a very good programmer (barely can be called a programmer at all), so I don't have code to contribute to make that a reality, and I don't want to start any sort of war over where that belongs, just my view as a HAProxy user. Thanks! -Nick From: Ben Timby bti...@gmail.com To: Nick E Couchman nick.couch...@seakr.com Cc: haproxy@formilux.org Sent: Monday, May 11, 2015 11:24:58 AM Subject: Re: VM Power Control/Elasticity Nick, HAProxy provides statistics via socket or HTTP interface. You can easily monitor these stats and run scripts. Some cron jobs and regex should suffice. Specific cases like this are usually not something I would imagine belongs in HAProxy core, since it is not directly related to load balancing but more of a specific requirement. You could simply configure all 10 servers, 5 of which would mostly be in the down state until your script brought them UP. HAProxy will happily balance traffic to all UP servers, and shift it away from DOWN servers. You can also use the stats socket to mark machines as up or down, so that traffic can be gracefully shifted before after VM power up/down. However, this may or may not be necessary depending on the services you are load balancing. https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#9.2
Re: VM Power Control/Elasticity
Thanks for the site - I will take a look. Sounds like the socket is the way to go - should be easier to parse than the web UI. -Nick From: Ben Timby bti...@gmail.com To: Nick E Couchman nick.couch...@seakr.com Cc: haproxy@formilux.org Sent: Monday, May 11, 2015 11:35:03 AM Subject: Re: VM Power Control/Elasticity Nick, Here is some information on using socat to interact with the stats socket. This might be useful for shell scripting. http://www.mgoff.in/2010/07/14/haproxy-gathering-stats-using-socat/
Re: haproxy and multiple ports
C - Original Message - From: PiBa-NL piba.nl@gmail.com To: Nick E Couchman nick.couch...@seakr.com, haproxy@formilux.org Sent: Friday, February 6, 2015 4:06:18 PM Subject: Re: haproxy and multiple ports Nick Couchman schreef op 6-2-2015 om 23:52: It's hard to figure out exactly how to phrase what I'm trying to do, but I essentially need a configuration for HAProxy where I can pin the load-balancing of one front-end port to another one, so that both go to the same back-end port. Here's what I'm trying to do...I'm using HAProxy to load-balance RDP connections. I also have a piece of software that goes between the RDP client and the RDP server that provides USB pass-through. So, the initial connection happens on port 3389, but then I need the client to also open a connection on another port - let's say 4000 - to the exact same back-end host. Is this possible in HAProxy? Thanks! -Nick This looks like a similar problem perhaps it will work for you to?: http://blog.haproxy.com/2011/07/14/send-users-to-the-same-server-for-imap-and-smtp/ Cool. Looks like the stick match line is what I need. I will try it out. Thanks!
haproxy and multiple ports
It's hard to figure out exactly how to phrase what I'm trying to do, but I essentially need a configuration for HAProxy where I can pin the load-balancing of one front-end port to another one, so that both go to the same back-end port. Here's what I'm trying to do...I'm using HAProxy to load-balance RDP connections. I also have a piece of software that goes between the RDP client and the RDP server that provides USB pass-through. So, the initial connection happens on port 3389, but then I need the client to also open a connection on another port - let's say 4000 - to the exact same back-end host. Is this possible in HAProxy? Thanks! -Nick
