I actually like having rsyslog send the logs directly to ES, but I agree that ES
should not be the only place the logs go to.
As Orangepeel Beef notes, there are many other destinations that you want your
logs to go to.
I like to keep a feed of the logs as a plain, combined feed (so I can easily see
what's going on across multiple machines without having to go into all the
different directories)
Sending a copy of the logs to Simple Event correlator is extremely handy
and then you really want the logs from your core server to exist elsewhere so
that when you have problems with it, you can figure out what's going wrong
so I like to have a rule
if $fromhost-ip == "127.0.0.1" then {
<set some vars>
@log-relay;JSON_format
} else {
send to local combined file
send to file split by server
send to file split by programname
send to SEC (split by programname)
send to ES (or other search tool)
etc
}
When you get to the point of making everything redundant, and/or splitting the
workload across servers for different functions, having all your logs, including
from your log servers, relayed through your relay boxes makes things "just work"
David Lang
On Wed, 14 May 2014, Orangepeel Beef wrote:
There are a ton of headaches associated with directly logging to
elasticsearch as well.
How do you reindex if an index crashes if you are not storing your logs
somewhere else as an intermediary? ES crashes indexes if it runs out of
memory, or disk space, and they crash hard. I've rebuilt indexes many many
times already.
What happens when you have a large burst of traffic and elasticsearch can't
handle it? rsyslog can handle a very large amount of throughput, and
writing to files it won't lose anything, but writing to es, it can.
How do you pass data to Simple event correlator and then into
elasticsearch? pipe it out, and then back into rsyslog? no thanks.
How do you tag different file types if you are sending direct to ES? each
one of my different logtypes has patterns and filters setup to parse data
out of them that rely on the type being set appropriately.
How do you grok parse fields if you are going direct to ES? Logstash does
that bit, and you're bypassing it here.
I work in network security and can't lose pretty much *any* logs. Logging
to file bypasses all of these issues, and the logstash file input maintains
a sincedb state of file positioning and can index at its leisure, even if
logstash is stopped and restarted, it will pick up from where it left off.
Plus we have requirements to maintain the logs for 6+ months, but we do not
need to maintain 6 month elasticsearch searchable data.
I keep 3 days of uncompressed raw logs for easy indexing / reindexing, and
everything older than that is bzipped, backed up, and stored. Sure you
could use elasticsearch-knapsack to export/backup your ES data, but it's
far easier to just maintain the raw logs.
But hey, to each their own.
On Wed, May 7, 2014 at 12:43 PM, David Lang <[email protected]> wrote:
at my old job we had ossec configured to send to rsyslog
personally I really dislike the 'write to a file and then scrape it with
another program' approach to logs
Yes, it handles cases where your logserver is down, but you should have HA
so that's a very rare case.
But it causes a bunch of headaches
1. a lot more disk I/O
2. polling to check if the file has changed
3. headaches if the files roll too fast
4. problems deciding when you can delete the files
It's just so much easier to pass the data directly to rsyslog and let it
deal with everything :-)
David Lang
On Wed, 7 May 2014, Josh Bitto wrote:
Date: Wed, 7 May 2014 09:44:43 -0700
From: Josh Bitto <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana server
Hello Everyone and Good Morning!
I have a new question for you all. Does anyone have this current setup
with an OSSEC server as well? I'm wondering which would be the better
option to do. Just create an imfile for Rsyslog to monitor the logs from
OSSEC or forward them to rsyslog. I'm curious to find out if anyone else
has this implemented too!
Josh
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
