At the risk of my first email to this list sounding heretical (hi everyone) logstash does have its place. If all you need to do is store logs in ElasticSearch, omelasticsearch is probably the better choice. However logstash has a lot of other capabilities including things like pulling stats from logs and sending them to statsd or graphite, a plugin for pager duty, outputs for a whole variety of monitoring platforms, and more. Having said that it does use substantially more memory and CPU than rsyslog. As always determine what you need to do first and then pick the best tool for the job. We're using rsyslog to forward logs to logstash, which stores a subset in ElasticSearch and sends stats to statsd/graphite.
As for Kibana, IIRC Kibana 3 is the ElasticSearch interface, it's been pulled into the ElasticSearch project. cheers mike -- Michael Hart Arctic Wolf Networks M: 226.388.4773 On 2013-10-29, at 6:30 AM, "Boylan, James" <[email protected]<mailto:[email protected]>> wrote: To answer some of the questions from my experience. Logstash - It is so highly documents merely because it has been around for a fair amount of time. The omelasticsearch module has been around for a while, but much of that time was as a community driven module that wasn't directly packaged with Rsyslog. This is no longer the case. Really I can't think of any other reason for that. Frankly, if you expect to be dealing with a high throughput of logs, I recommend not using logstash. My initial tests on it found that Logstash used almost 10 times more CPU to do the same processing that Rsyslog with omelasticsearch did. As to the second question I'm not sure what is being asked. Elasticsearch doesn't have a UI, it's a indexing engine with an API. You have to either build your own Front end or use one of the pre-created Front end (Like Kibana). -- James -- Sent from my mobile -- ----- Reply message ----- From: "David Lang" <[email protected]<mailto:[email protected]>> To: "rsyslog-users" <[email protected]<mailto:[email protected]>> Subject: [rsyslog] Fwd: RSyslog Logstash Kibana Elasticsearch Preference Date: Tue, Oct 29, 2013 2:34 am you would have to ask the people who are telling you to use logstash over rsyslog why they are doing so. The rsyslog people are not the ones telling you to use logstash David Lang On Tue, 29 Oct 2013, masoom alam wrote: Date: Tue, 29 Oct 2013 10:46:32 +0500 From: masoom alam <[email protected]<mailto:[email protected]>> Reply-To: rsyslog-users <[email protected]<mailto:[email protected]>> To: rsyslog-users <[email protected]<mailto:[email protected]>> Subject: [rsyslog] Fwd: RSyslog Logstash Kibana Elasticsearch Preference HI every one. My friend wants to ask the following questions, but he cannot post on the rsyslog list. The strange thing is that there is no yes/no response from the list. He has subscribed twice. Please help. Thanks. ---------- Forwarded message ---------- From: waqar afridi <[email protected]<mailto:[email protected]>> Date: Tue, Oct 29, 2013 at 10:42 AM Subject: Fwd: RSyslog Logstash Kibana Elasticsearch Preference To: Sir Masoom Alam <[email protected]<mailto:[email protected]>>, ---------- Forwarded message ---------- From: waqar afridi <[email protected]<mailto:[email protected]>> Date: Tue, Oct 29, 2013 at 10:04 AM Subject: RSyslog Logstash Kibana Elasticsearch Preference To: [email protected]<mailto:[email protected]> Hello All I hope everyone would be OK, First of All I would welcome my self to RSyslog family with this email. I have alot of questions that I will ask from time to time and I hope my rookie question would be tolerated :p I have few question *Log Collection through RSyslog or Logstash* I want to collect log and then store it in elasticsearch, I have already have did this btw, I can collect log through logstash, store it in elasticsearch and then display it through Kibana, my Question is why use Logstash? Can't RSyslog provide all these functionality of log collection? The only reason I see is that Logstash have filters, RSyslog can store logs in elasticsearch through OMELASTICSEARCH, why using logstash is so highly recommended? *Kibana vs Elasticsearch UI* This question does not concern this mailing list but I will ask it anyway may be in near future someone like me will come looking to answer the same questions :p which one should I prefer? Kibana or the UIs for elasticsearch? My main concern is to minimize rely my dependance on external tools as much as possible without weakening my Log-server. More Questions to come in Near Future Regards _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
