Hi Hank, Willy, Thanks for all the tips. We have our setup up and running with the syslog changes (rsyslog on fedora).
We did find an issue which could be related to http keepalive and we have since upgraded to 1.14. Thanks a lot for the prompt and useful feedback. - Praveen. On Thu, Mar 11, 2010 at 6:19 PM, Hank A. Paulson < [email protected]> wrote: > For reference, one of the sites I have is F12, 60-90 million hits/day > 60-120+ Mbps logging via rsyslogd to 2 different logging servers (with > separate error logging turned on) and it runs fine on a Xen VM with 5 GB and > 1 VCPU - always below 80% CPU load. > > Fedora 8 is hundreds of years old, but it along with rsyslog (logging > locally or remotely) and haproxy with a reasonable amount of RAM should be > fine for anything below facebook level loads. There are alot of network > stack and NIC driver fixes from F8 to F12. > > You could even do the remote logging via a separate NIC. > > You say: > > "This is a bit of a problem for us to automate launching of a machine in > our environment since we need to edit syslog config files." > but if you used syslog-ng you have to create a customized config file for > that, too - so it doesn't seem like less work than adding a few lines to > your existing rsyslog config. I'm confused why that creates a problem... > > Willie - From Fedora 8 on, they have been using rsyslog by default not > syslog. rsyslog is similar in goals and features to syslog-ng: > http://www.rsyslog.com/doc-rsyslog_ng_comparison.html For instance, it > can cache to a local disk if your remote logging server is not reachable. > > > On 3/11/10 10:11 AM, Praveen Patnala wrote: > >> Hi Willy, >> >> Thanks for the clarification. On my fedora distribution, if I try to >> install syslog-ng, I see the following errors and the install fails. >> >> yum install syslog-ng >> Loaded plugins: fastestmirror >> Loading mirror speeds from cached hostfile >> * updates-newkey: kdeforge.unl.edu <http://kdeforge.unl.edu> >> * updates: kdeforge.unl.edu <http://kdeforge.unl.edu> >> * fedora: kdeforge.unl.edu <http://kdeforge.unl.edu> >> >> Setting up Install Process >> Parsing package install arguments >> Resolving Dependencies >> --> Running transaction check >> ---> Package syslog-ng.i386 0:2.0.10-1.fc8 set to be updated >> --> Processing Dependency: libevtlog.so.0 for package: syslog-ng >> --> Running transaction check >> ---> Package eventlog.i386 0:0.2.7-1.fc8 set to be updated >> --> Processing Conflict: rsyslog conflicts syslog-ng >> --> Finished Dependency Resolution >> rsyslog-2.0.2-3.fc8.i386 from installed has depsolving problems >> --> rsyslog conflicts with syslog-ng >> Error: rsyslog conflicts with syslog-ng >> >> Do you have any other suggestion? >> >> Thanks, >> Praveen. >> >> On Tue, Mar 9, 2010 at 9:49 PM, Willy Tarreau <[email protected] >> <mailto:[email protected]>> wrote: >> >> On Tue, Mar 09, 2010 at 06:16:07PM -0800, Praveen Patnala wrote: >> > HI Willy, >> > >> > I had a question on the debug mode. I got the syslogd working >> (using -r >> > options) and we do get the logs on a different location than >> > /var/log/messages by modifying the various syslog config files. >> This is a >> > bit of a problem for us to automate launching of a machine in our >> > environment since we need to edit syslog config files. If we run >> the haproxy >> > in the debug mode and redirect the logs, do you recommend that? >> >> Huh, not at all ! Debug mode dumps *all headers*, which is very >> verbose. >> It will also not include important information such as IP addresses, >> timers, >> flags, etc... As its name implies, debug mode is for debugging. >> >> > Can you >> > share any information in terms of performance slowdown or >> stability in this >> > mode in a production environment? >> >> It will be very expensive in terms of CPU and storage, and useless for >> logging. Some people do that on development machines to get full >> captures >> of requests and responses, but that's on development machines only. >> >> What I really recommend you to do is to install syslog-ng with just >> one instance dedicated to local UDP logging. It is very fast, will >> not interfere at all with existing syslog and will not require any >> system config change. >> >> I regularly recommend this basic configuration : >> >> > options { >> > sync (0); >> > time_reopen (10); >> > log_fifo_size (10000); >> > long_hostnames (off); >> > use_dns (no); >> > use_fqdn (no); >> > create_dirs (no); >> > keep_hostname (yes); >> > }; >> > >> > source s_udp { udp(ip(127.0.0.1) port(514)); }; >> > >> > destination d_haproxy { file("/var/log/haproxy"); }; >> > filter f_local0 { facility(local0); }; >> > log { source(s_udp); filter(f_local0); destination(d_haproxy); }; >> >> You can even start it on a non-privileged port, or have one instance >> per haproxy instance, etc... It's the easiest solution to deploy and >> probably the cleanest. >> >> Regards, >> Willy >> > > > >
