Hi folks, I probably missed it, but after awhile searching the docs fruitlessly, I decided I'd ask the experts.
We have an rsyslog server that handles a couple million messages (from a single remote server) per minute. It logs these to several logfiles with a cpu load of about 40% of a single CPU core. logrotate currently takes about 12 hours to single-threadedly, consecutively, compress and rotate these logs. It's been suggested that we take logrotate out of the loop and just have rsyslog write compressed files. This seems like a great idea, but I'm curious about how it scales, since I don't have a good test environment (just some underpowered VM's which don't seem to generate comparable load no matter how I try) to work with. Suppose that rsyslog needs more than a core's worth of CPU to do the compression realtime. What happens then? Is rsyslogd multithreaded enough (or can it be setup to be multithreaded enough) to spin up more threads to handle the compressed writes? Will it ever drop messages? some more info: - The highest traffic logs are in 2 separate files, which have about 60% of the load together, the rest is going into a dozen other smaller files. - we'd be setting OMFileZipLevel to 1 - we're logging via tcp and splitting based on priority and sending IP (though 99.9% of everything comes from one IP) - we're willing to change some configuration, but here's the only special config we have now: - $MainMsgQueueType Direct Thanks for your insight! -- Chris McCraw | Operations New Relic - http://blog.newrelic.com - @NewRelic on Twitter _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards
