Re: proxy health checks [was: Proxying OPTIONS *]
In our proxy, we launch an external helper app the does active health checking of the origin servers. This is a HEAD request on a configurable (per origin pool) uri (ie, http://host:port/url/blah). When an origin passes/fails a given number of checks it is marked up/down. For example, when an origin passes 2 health checks in a row, it is marked up; when it fails 3 in a row, it is marked down. We check each backend every x seconds (x usually equal to 5). The proxy module and the health checker are linked via a simple array in shared memory. The health checker marks the index of each origin as 1 (up) or 0 (down). I have been contemplating writing a balancer module that implements this, but haven't had the time. With this method, we don't have to check every time, as we assume if the origin is up in the health checker that it is up. This works several million times a day for us :) (Also, we do true connection pooling to the origins, but that's another story...) -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
proxy health checks [was: Proxying OPTIONS *]
Jim Jagielski wrote: On Oct 2, 2007, at 5:56 PM, Ruediger Pluem wrote: Slightly off topic, but this gives me the idea that we could use OPTIONS * as some kind of ping / health check for pooled connections in mod_proxy_http before sending a request (at least in the reverse proxy case before sending a request that is not idempotent or after the connection has not been used for a certain amount of time). The current is_connected check has a race condition if the keepalive timer of the backend server kills the connection just after our check and before it received our request. :) We already do similar with AJP where, right after ap_proxy_connect_backend() it does a quick check. I think we even discussed having HTTP do something similar awhile ago... It would be nice to have the health check running asynchronously from the normal request handling (error detection for idle connections, before requests fail). Even more important, the recovery test could be done independently from request handling. For this one would need some scheduling service (independent threads, API extension for callbacks). A simple start could be something like the monitor() hook. Although that one depends on the type of MPM and the module scheduling service should not be decoupled from the timing of important internal procedures. Regards, Rainer
Re: proxy health checks [was: Proxying OPTIONS *]
On Wed, 03 Oct 2007 15:04:52 +0200 Rainer Jung [EMAIL PROTECTED] wrote: It would be nice to have the health check running asynchronously from the normal request handling (error detection for idle connections, before requests fail). Even more important, the recovery test could be done independently from request handling. Indeed. That could apply to other backends (such as mod_dbd) too. For this one would need some scheduling service (independent threads, API extension for callbacks). A simple start could be something like the monitor() hook. Although that one depends on the type of MPM and the module scheduling service should not be decoupled from the timing of important internal procedures. Patches welcome! Maybe a scheduled event thread could be a standard feature for httpd-3, to make this kind of thing straightforward. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
