It shouldn't be too hard for a machine to keep up with logging. How are you logging? standard syslog? Make sure you have - in front of the filename? How connections per second are you logging?
Haven't done it with Haproxy, but have with other things that generate tons of logs. what you could do is dump the logs (don't forget the - as part of the file name) to /dev/shm/ (assuming linux), and then rotate the logs once a minute and process them. That way, you will not have any disk I/O from the logs, but would increase memory requirements. From: James Brady [mailto:[email protected]] Sent: Wednesday, February 11, 2009 12:14 PM To: [email protected] Subject: Reducing I/O load of logging The machine we run HAProxy on is approaching its limits in terms of disk I/O due to our debug-level logging. The CPU is barely used, memory is no problem at all - the bottleneck will soon be the rate at which we can log to disk. Because the machine is more than adequate in all other areas, we'd really like to reduce the I/O load of logging. We are using the debug log output to tell us the response time and status code for various URLs - this is an essential task we can't do without. Is there any way to get this information without logging and post-processing every request? Can HAProxy dump out averaged statistics like this? Can we somehow reduce the I/O load by just logging the exact fields we're interested in? Many thanks! James
