On Fri, 7 Dec 2012 15:07:48 +0200 Radu Gheorghe <[email protected]> wrote:
> Hi Ben, > > 2012/12/7 Ben Bradley <[email protected]> > > > I've tested rsyslog using the imfile module to watch each Apache log > > files, but this means I have to hard-code each vhost log file into my > > rsyslog.conf. This is not ideal as people will invariably forget when they > > add/remove sites on the server. > > > > 2) What's the best way to log to both vhost-specific log files on the web > > server and to send these logs over the network, without using imfile and > > manually watching tens of individual log files? > > > > Just a quick note: I assume imfile can be changed to support wildcards > and/or templates. > Yeah wildcards would be a possible solution. But I wasn't aware that imfile supported wildcards out of the box? > One suggestion: if you have lots of logs, you may want to skip using > logstash, which is another moving piece which can also be a bottleneck. You > can get your rsyslog to output directly to Elasticsearch, and still use > Kibana on top of that. Of course, this is applicable only if you don't need > a logstash-specific feature, such as grok. > > For rsyslog, you'll need omelasticsearch, here's a (little old) tutorial on > using it: > http://wiki.rsyslog.com/index.php/HOWTO:_rsyslog_%2B_elasticsearch > > and some more on queues (for performance and reliability): > http://wiki.rsyslog.com/index.php/Queues_on_v6_with_omelasticsearch > > All you have to do in addition to that is to make sure your timestamp goes > in a field called "@timestamp" in ES, because that's hardcoded in Kibana so > it can sort your logs. Then, for a field to be shown in Kibana, it also > needs to begin with an "@" as far as I know. This is actually quite interesting. I agree that it's good to keep it as simple as possible so running rsyslog on the server as a collector and dumping straight into ElasticSearch could be a possible solution. The problem is there are so many different ways of doing this and all are slightly different. After a weekend of further research I've still not decided the best way to do this. Cheers for the info! _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
