Hi Ben, 2012/12/7 Ben Bradley <[email protected]>
> Hi everyone > > I've just started investigating centralised logging and I'm gradually > building up a plan of action. > > I'd like to store the logs on a central server running > logstash/ElasticSearch so they can be searched and monitored using Kibana. > With rsyslog sending the logs over the network to a logstash server. I > don't want to run logstash as the log "sender" on each server, I'd prefer > to keep the servers (log "clients") as lean and simple possible. So that > means either using syslog, syslog-ng or the one I'm testing now, rsyslog. > > 1) Should I have rsyslog sending to logstash over the network? Or should I > be running another rsyslog on the collector server, which then sends to > logstash for processing? > Yes, I think it's better to send logs directly to logstash, so you won't have to maintain yet another rsyslog daemon. > > > For Apache, I would like to have separate vhost log files on the web > server, in addition to these logs being sent to a remote log collector. > > I've tested rsyslog using the imfile module to watch each Apache log > files, but this means I have to hard-code each vhost log file into my > rsyslog.conf. This is not ideal as people will invariably forget when they > add/remove sites on the server. > > 2) What's the best way to log to both vhost-specific log files on the web > server and to send these logs over the network, without using imfile and > manually watching tens of individual log files? > Just a quick note: I assume imfile can be changed to support wildcards and/or templates. > Get Apache to log to rsyslog, then have rsyslog split the log to both a > file and over the network to logstash? > Yeah, if you can do that it seems like the best solution. I don't know how you can do it, since I have little experience with Apache. > Are there big performance implications for logging both locally and over > the network? > I would assume rsyslog won't break a sweat, if that's what you're thinking about. I would only consider the storage as a possible issue (eg: you might want to make sure you logrotate&compress logs that you store locally) > > I could change my Apache config to log to a single access/error log for > all vhosts, then watch these main log files with imfile. So long as rsyslog > is then able to produce vhost-specific log files somewhere on the web > server machine. > Although I've never done it, I think you can use mmnormalize to make rsyslog parse your Apache logs: http://www.rsyslog.com/using-rsyslog-mmnormalize-module-effectively-with-adiscon-loganalyzer/ > > > Any comments/suggestions? > I am sure others have had a similar need. I just don't want to ditch local > log files until we fully know how well the centralised log server performs. > That sounds reasonable :) One suggestion: if you have lots of logs, you may want to skip using logstash, which is another moving piece which can also be a bottleneck. You can get your rsyslog to output directly to Elasticsearch, and still use Kibana on top of that. Of course, this is applicable only if you don't need a logstash-specific feature, such as grok. For rsyslog, you'll need omelasticsearch, here's a (little old) tutorial on using it: http://wiki.rsyslog.com/index.php/HOWTO:_rsyslog_%2B_elasticsearch and some more on queues (for performance and reliability): http://wiki.rsyslog.com/index.php/Queues_on_v6_with_omelasticsearch All you have to do in addition to that is to make sure your timestamp goes in a field called "@timestamp" in ES, because that's hardcoded in Kibana so it can sort your logs. Then, for a field to be shown in Kibana, it also needs to begin with an "@" as far as I know. Best regards, Radu _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
