Hi Misc, I have been running syslog-ng centralized syslog server on OpenBSD for a about six months now. I have also started looking into more sophisticated ways to search, analyze, and visualize log data. Currently I use combination of regular expressions and sed/awk.
Browsing our ports collection I learnt about Logstash, Elasticsearch and I found out about Kibana too even that it is not in the ports. Those three usually go under the name of ELK. I am poking with the idea of adding ELK on the top of my centralized syslog server along the lines described in this article http://www.networkassassin.com/elk-for-network-operations/ I have several questions for people before I take a plunge. 1. I know that syslogd in the base have received lots of love recently. As a client it rocks and I don't want to replace stack syslogd on my OpenBSD machines with syslog-ng. However as mentioned before I use syslog-ng as a server. Is anybody using syslogd as a centralized syslog server in particular in the heterogeneous environment where it needs to receive log files possibly from rsyslog and FreeBSD's primitive syslogd? How does it compare to syslog-ng as a server? 2. How does Logstash (Java jar) compares to Fluentd written in Ruby. I saw sthen@openbsd expressing interest in porting Fluentd. I am inclined to implement simpler of the too and at least when it comes to deployment Logstash looks "easier". I am concern that Fluentd is not in ports but I am even more concern if Fluentd can process/receive logfiles from my centralized syslog-ng server. I have no intension of deploying Fluentd nor Logstash clients on my machines for that matter. 3. Any comments on EFK (Elasticsearch Fluentd Kibana) vs ELK (Elastricsearch Logstash Kibana) holly war. 4. Kibana is not in ports. My understanding is that it is written in JavaScript+HTML so deploying with Nginx or even a httpd from the base should not be a problem. Am I correct? I see that it has some binary blob but I am assuming that is built in webserver. 5. Finally I am open for simpler ideas. Any opinions on sysutils/logfmon Is it possible to visualize on the web output from logfmon? Best, Predrag Punosevac
