On Fri, 6 May 2016, Bhuvanesh Bhuvanachandran wrote:
Date: Fri, 6 May 2016 01:19:36 -0700 (PDT)
From: Bhuvanesh Bhuvanachandran <bhuvane...@gmail.com>
Reply-To: ossec-list@googlegroups.com
To: ossec-list <ossec-list@googlegroups.com>
Subject: [ossec-list] Syslog Server Help
Hi Guys,
I have a problem which I need some expert advise.
I have a number of systems with the following softwares.
1. Apache proxy server
2. Apache Tomcat
3. Oracle DB
I want to create a central syslog server, where all logs from the above and
other system logs get ported and is analyzed at the central server and a
dash board is required at the end.
I could see a few combination to achieve this possibly.
1. Ossec agents monitor log files and port all log to OSSEC server (/var/
ossec/logs/archives/archives.log) + logstash +elastic search + Kibana
2. Ossec agent port all log files + Ossec server syslog output + logstash
+elastic search + Kibana
3. rsyslog on client machines write logs to central syslog server + Ossec
monitor central syslog server output + logstash +elastic search + Kibana
What is expected on the dashboard is
1. PCI DSS compliance dash board. (This is possible with Ossec alerts
visualization I understand).
2. All access data in graphs, say from apache logs top hit hosts, top urls
, error counts etc.(This is possible only if archives log is active)
I want to happen both ossec alert log and archive log porting at the same
time. Is this possible with Ossec?
Or if this is a better way ? porting all logs with some syslog programs (I
am not sure what to use for this.) and ossec will process the central
server syslog and make alerts from that.
Also is it possible to pass multiple inputs to logstash (archive log input
and ossec syslog input) ?
How to parse the actual messages and categorize (since it can contain
messages from apache logs, messages , oracle logs etc) at logstash, is
there someone can provide a filter example?
Please advise how to go ahead with this requirements.
OSSEC is not a great syslog delivery system, so if you are wanting a central
logging system, you would be best off doing something like
rsyslog -> central logging system
| /
|- osssec
|
|- archive
|
|- alerting
|
|- ElasticSearch <- Kibana
https://www.usenix.org/conference/lisa12/technical-sessions/presentation/lang_david
https://www.usenix.org/publications/login/august-2013-volume-38-number-4/enterprise-logging
https://www.usenix.org/publications/login/october-2013-volume-38-number-5/log-filtering-rsyslog
https://www.usenix.org/publications/login/december-2013-volume-38-number-6/using-sec
https://www.usenix.org/publications/login/feb14/logging-reports-dashboards
https://www.usenix.org/publications/login/april14/lang
while I talk about Splunk in these articles, ElasticSearch + Kibana work very
similarly (including the tuning issues)
Rsyslog is great at transporting, parsing, filtering, and delivering logs.
ElasticSearch is great at giving you free-form search capabilities for your
logs.
Kibana is great as a search front-end to ElasticSearch, and creating graphs. It
will also create graphs from summary data, not just the raw logs. Use summary
data for your dashboards, not raw logs.
the latest version of rsyslog has gained some significant summarizing
capabilities, but you can also feed your logs to other tools (like sec) to do
the summary work for you
Adding logstash to the mix doesn't help much and can be quite 'interesting' to
manage at high volumes.
David Lang
