For reference, one of the sites I have is F12, 60-90 million hits/day 60-120+
Mbps logging via rsyslogd to 2 different logging servers (with separate error
logging turned on) and it runs fine on a Xen VM with 5 GB and 1 VCPU - always
below 80% CPU load.
Fedora 8 is hundreds of years old, but it along with rsyslog (logging locally
or remotely) and haproxy with a reasonable amount of RAM should be fine for
anything below facebook level loads. There are alot of network stack and NIC
driver fixes from F8 to F12.
You could even do the remote logging via a separate NIC.
You say:
"This is a bit of a problem for us to automate launching of a machine in our
environment since we need to edit syslog config files."
but if you used syslog-ng you have to create a customized config file for
that, too - so it doesn't seem like less work than adding a few lines to your
existing rsyslog config. I'm confused why that creates a problem...
Willie - From Fedora 8 on, they have been using rsyslog by default not syslog.
rsyslog is similar in goals and features to syslog-ng:
http://www.rsyslog.com/doc-rsyslog_ng_comparison.html For instance, it can
cache to a local disk if your remote logging server is not reachable.
On 3/11/10 10:11 AM, Praveen Patnala wrote:
Hi Willy,
Thanks for the clarification. On my fedora distribution, if I try to
install syslog-ng, I see the following errors and the install fails.
yum install syslog-ng
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* updates-newkey: kdeforge.unl.edu <http://kdeforge.unl.edu>
* updates: kdeforge.unl.edu <http://kdeforge.unl.edu>
* fedora: kdeforge.unl.edu <http://kdeforge.unl.edu>
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
--> Running transaction check
---> Package syslog-ng.i386 0:2.0.10-1.fc8 set to be updated
--> Processing Dependency: libevtlog.so.0 for package: syslog-ng
--> Running transaction check
---> Package eventlog.i386 0:0.2.7-1.fc8 set to be updated
--> Processing Conflict: rsyslog conflicts syslog-ng
--> Finished Dependency Resolution
rsyslog-2.0.2-3.fc8.i386 from installed has depsolving problems
--> rsyslog conflicts with syslog-ng
Error: rsyslog conflicts with syslog-ng
Do you have any other suggestion?
Thanks,
Praveen.
On Tue, Mar 9, 2010 at 9:49 PM, Willy Tarreau <[email protected]
<mailto:[email protected]>> wrote:
On Tue, Mar 09, 2010 at 06:16:07PM -0800, Praveen Patnala wrote:
> HI Willy,
>
> I had a question on the debug mode. I got the syslogd working
(using -r
> options) and we do get the logs on a different location than
> /var/log/messages by modifying the various syslog config files.
This is a
> bit of a problem for us to automate launching of a machine in our
> environment since we need to edit syslog config files. If we run
the haproxy
> in the debug mode and redirect the logs, do you recommend that?
Huh, not at all ! Debug mode dumps *all headers*, which is very verbose.
It will also not include important information such as IP addresses,
timers,
flags, etc... As its name implies, debug mode is for debugging.
> Can you
> share any information in terms of performance slowdown or
stability in this
> mode in a production environment?
It will be very expensive in terms of CPU and storage, and useless for
logging. Some people do that on development machines to get full
captures
of requests and responses, but that's on development machines only.
What I really recommend you to do is to install syslog-ng with just
one instance dedicated to local UDP logging. It is very fast, will
not interfere at all with existing syslog and will not require any
system config change.
I regularly recommend this basic configuration :
> options {
> sync (0);
> time_reopen (10);
> log_fifo_size (10000);
> long_hostnames (off);
> use_dns (no);
> use_fqdn (no);
> create_dirs (no);
> keep_hostname (yes);
> };
>
> source s_udp { udp(ip(127.0.0.1) port(514)); };
>
> destination d_haproxy { file("/var/log/haproxy"); };
> filter f_local0 { facility(local0); };
> log { source(s_udp); filter(f_local0); destination(d_haproxy); };
You can even start it on a non-privileged port, or have one instance
per haproxy instance, etc... It's the easiest solution to deploy and
probably the cleanest.
Regards,
Willy
