If it helps anyone, we've gotten our NetOps users happy (mostly) with a
fairly simple config (building on default logstash recipes). Sorry these
are still logstash format, but maybe a good place to start. This works
for a random assortment of IOS, ASA, Nexus, Catalyst and addresses
god-aweful variations in timestamp/format and multi-line messages.
Patterns:
# The format of timestamps on Cisco syslog messages are configurable, some
# permutations just don't make any sense, some are more useful than others.
#
# The ideal goal is for the timezone of the device to be set to UTC and the
# timestamps should contain all date/time fields from year right down to
# milliseconds and if possible, don't display the timezone.
#
# On IOS:
#
# service timestamps log datetime [msec] [year]
#
# Some devices both old and new don't support one or the other option. This
# should yield a timestamp of the form "Jan 1 1970 00:00:00.000".
#
# On NX-OS:
#
# logging timestamp milliseconds
#
# NX-OS does things slightly barse ackwards compared to IOS and will yield
a
# timestamp of the form "1970 Jan 1 00:00:00.000 UTC".
CISCOTIMESTAMP (?x:(?:(?:%{YEAR} \s)? %{MONTH} \s+ %{MONTHDAY}|%{MONTH}
\s+ %{MONTHDAY} \s %{YEAR}) \s %{TIME})
# All Cisco syslog messages usually contain a header of the format:
#
# %FACILITY-(SUBFACILITY-)SEVERITY-MNEMONIC
#
# The severity can be ignored as it is identical to the syslog pri field
CISCOFACILITY \w+
CISCOSUBFACILITY \w+
CISCOSEVERITY \d
CISCOMNEMONIC \w+
# Some Cisco devices have the notion of a context within a single device
CISCOCONTEXT \w+
# Some Cisco devices have a node ID like "RP/0/RSP0/CPU0" referring to a
# specific module within a chassis
CISCONODEID [A-Z\d/]+
Groks:
# Match only messages from Cisco ACS.
if [type] == "syslog" and [message] =~ /\sCSCOacs_/ {
# Capture the header & message ignoring the sequence numbers
grok {
match => {
"message" => "(?x) (?<acs_header><%{POSINT}> %{CISCOTIMESTAMP} \s
%{DATA} \s CSCOacs_%{DATA} ) \s %{NONNEGINT} \s %{POSINT} \s %{NONNEGINT}
\s %{GREEDYDATA:acs_message}"
}
patterns_dir => "/etc/logstash/patterns"
}
if "_grokparsefailure" not in [tags] {
# Multiline filter only acts on message so copy the captured message
in
mutate {
replace => { "message" => "%{acs_message}" }
}
# Any message not beginning with a datestamp should be joined to the
# previous line
multiline {
pattern => "^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{3}"
negate => true
what => "previous"
}
# The captured message fields are now an array so squash it
mutate {
join => [ "acs_message", "" ]
}
# Multiline filter joins lines together with \n's so once again
replace
# @message with the header and the joined message which doesn't have
\n's
mutate {
replace => [ "message", "%{acs_header} %{acs_message}" ]
}
# Remove any fields or tags we added so the message should look
normal
# again
mutate {
remove_field => [ "acs_header", "acs_message" ]
remove_tag => [ "multiline" ]
}
}
}
-----Original Message-----
From: masoom alam <[email protected]>
Reply-To: rsyslog-users <[email protected]>
Date: Thursday, April 10, 2014 at 11:16 PM
To: "rick.brown" <[email protected]>, rsyslog-users
<[email protected]>
Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana server
>Cisco Pix firewall is a great in the sense that it contains a lot of
>if-then-else and multiple regex. We have done not-complete conversion as
>logtash plugin. Can we combine our synergies regarding logtash Plugin-->
>Rsyslog JASON?
>
>Thanks.
>
>
>On Fri, Apr 11, 2014 at 6:34 AM, Brown, Richard A
><[email protected]
>> wrote:
>
>> I like where this is going. I'd rather not have to involve logstash as
>> well.. But their contrib package of plugins/filters is compelling.
>>If we
>> can build our own similar book of recipes, then let's do it! I'll
>>start on
>> a few of my common use cases and report back.
>>
>>
>> Sent via mobile
>>
>> -------- Original message --------
>> From: masoom alam <[email protected]>
>> Date: 04/10/2014 7:24 PM (GMT-05:00)
>> To: rsyslog-users <[email protected]>
>> Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana server
>>
>> Very detailed answer. Thanks!!!
>>
>> Since it is related with both Rsyslog and Logtash, thats why I am asking
>> here. After your kind guidance, Its now clear that we should use JASON
>> template in Rsyslog, and then use JASON in logtash. I did not find any
>> significant difference at the logtash end regarding Grok and JASON,
>>except
>> the word Jason in the filter instead of Grok, am I right? -- I mean as
>>for
>> as the syntax is concerned. For the execution it will definitely have
>> performance gains, as you suggested.
>>
>> Another thing which I think I did not explain well in my email is that
>>we
>> are thinking to place some regex at the Rsyslog end too. Suppose we have
>> >200 filters defined in Logtash, so will happen that when a log entry
>>will
>> arrive at the Logtash, it will have to match it against all the 200
>>filters
>> -- worst case, and/or some thing matches earlier and we compose the
>> configuration file of Logtash in a way that it escapes. Any ideas how to
>> optimize the log deep/fancy parsing at this end?
>>
>> Once Again thanks Radu. You are very helpful.
>>
>>
>>
>>
>>
>>
>> On Fri, Apr 11, 2014 at 12:36 AM, Radu Gheorghe
>> <[email protected]>wrote:
>>
>> > I've never actually tried this, but I think the best way for
>>performance
>> is
>> > to send over TCP, but make the template a JSON with everything rsyslog
>> can
>> > parse (by default, stuff like severity, date, etc). On the Logstash
>>side,
>> > you'll use the JSON that should parse much faster than grok can parse
>> > syslog. After that, you'd set the rest of the Logstash filters you
>>want
>> to
>> > use for fancy processing.
>> >
>> > Also, sending over TCP allows you to use rsyslog for buffering, and if
>> > you're using in-memory queues (or disk-assisted, assuming those rarely
>> > spill out to disk), this means you'll avoid the I/O penalty of
>>writing to
>> > disks and having Logstash poll from disk periodically.
>> >
>> > If you need help with any of those, please write here (or on the
>>Logstash
>> > ML for the Logstash part, people are really helpful there).
>> >
>> > Best regards,
>> > Radu
>> >
>> >
>> > On Thu, Apr 10, 2014 at 6:13 PM, masoom alam <[email protected]>
>> > wrote:
>> >
>> > > Is it necessary to fill the templates inside rsyslog so that rsyslog
>> > should
>> > > write each log source to a separate file for logtash - will be easy
>>for
>> > it
>> > > for parsing? - also due to the reason logrtash has to catch
>>rsyslog? .
>> > What
>> > > is the alternative if we are doing extensive parsing in logtash? -
>> simply
>> > > directing log on to a port and ask logtash to pick it up - match it
>> > against
>> > > 200 plugins?
>> > >
>> > > from phone thus brief.
>> > > On Apr 10, 2014 5:06 PM, "Radu Gheorghe"
>><[email protected]>
>> > > wrote:
>> > >
>> > > > Here's an article that explains how to configure squeeze
>>performance
>> > > from a
>> > > > rsyslog>ES>Kibana setup, and the numbers I got (20-30K EPS on my
>> > good-old
>> > > > laptop): http://www.rsyslog.com/performance-tuning-elasticsearch/
>> > > >
>> > > > You also have links there about other articles in this are (that
>>also
>> > > have
>> > > > config snippets and explanations).
>> > > >
>> > > > On Tue, Apr 8, 2014 at 11:34 PM, Josh Bitto
>><[email protected]>
>> > > > wrote:
>> > > >
>> > > > > If I'm reading this right your saying that you did
>> > > > > Rsyslog->Elasticsearch->gui?
>> > > > >
>> > > > > I've tried installing the rpm on centos and it installs but
>> > apparently
>> > > it
>> > > > > doesn't come with a config file and so the daemon starts it
>>errors
>> > out
>> > > in
>> > > > > the logs and just shuts down after that.
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > -----Original Message-----
>> > > > > From: [email protected] [mailto:
>> > > > > [email protected]] On Behalf Of Rick Brown
>> > > > > Sent: Tuesday, April 08, 2014 11:31 AM
>> > > > > To: rsyslog-users
>> > > > > Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana
>> > server
>> > > > >
>> > > > > Today I've setup my central rsyslog server to replay the logs
>>via
>> > > > > omudpspoof to a logstash server -> ES. It's already indexing
>>about
>> > > twice
>> > > > > as much as just rsyslog -> ES was using the recipe in the first
>> link
>> > > > below,
>> > > > > and I haven't even begun to dig into the scads of plugins
>>available
>> > for
>> > > > > logstash.
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>http://blog.sematext.com/2013/07/01/recipe-rsyslog-elasticsearch-kibana/i
>>sagoodplaceto start, although you can replace the omelasticsearch OM
>> > > > > with omudpspoof if you want to do logstash.
>> > > > >
>> > > > > http://cookbook.logstash.net/recipes/rsyslog-agent/ is a good
>> place
>> > to
>> > > > > start with rsyslog -> logstash, although I did UDP instead of
>>TCP,
>> > and
>> > > > used
>> > > > > the elasticsearch output module instead of stdout, which is
>> > documented
>> > > > > here: http://cookbook.logstash.net/recipes/central-syslog/
>> > > > >
>> > > > > Good luck to you! Those three links is basically all I needed,
>>and
>> > > > should
>> > > > > set you down the right path, regardless of how your path differs
>> from
>> > > > mine
>> > > > > ;)
>> > > > >
>> > > > > ----- Original Message -----
>> > > > > > From: "Orangepeel Beef" <[email protected]>
>> > > > > > To: "rsyslog-users" <[email protected]>
>> > > > > > Sent: Tuesday, April 8, 2014 2:17:42 PM
>> > > > > > Subject: Re: [rsyslog] Rsyslog w/
>>logstash-elasticsearch-kibana
>> > > server
>> > > > > >
>> > > > > > it works, but I find it overly complex for my environment.
>>read:
>> > I
>> > > > > > don't need it ;) On Apr 8, 2014 11:13 AM, "Josh Bitto"
>> > > > > > <[email protected]> wrote:
>> > > > > >
>> > > > > > > I have read about Redis as being the "broker" thoughts?
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > -----Original Message-----
>> > > > > > > From: [email protected] [mailto:
>> > > > > > > [email protected]] On Behalf Of Orangepeel
>> Beef
>> > > > > > > Sent: Tuesday, April 08, 2014 11:11 AM
>> > > > > > > To: rsyslog-users
>> > > > > > > Subject: Re: [rsyslog] Rsyslog w/
>>logstash-elasticsearch-kibana
>> > > > > > > server
>> > > > > > >
>> > > > > > > I use rsyslog to pipe into sec, and then use logstash file
>> input
>> > to
>> > > > > > > index.
>> > > > > > > could be done without SEC as well. I don't like delivering
>> > syslog
>> > > > > > > right into logstash.
>> > > > > > > On Apr 8, 2014 11:09 AM, "Sphonic" <[email protected]>
>> > wrote:
>> > > > > > >
>> > > > > > > > I use rsyslog to send all items to logstash which has a
>> syslog
>> > > > > > > > listener enabled.
>> > > > > > > >
>> > > > > > > > Sent from my iPhone
>> > > > > > > >
>> > > > > > > > > On 8 Apr 2014, at 18:05, Josh Bitto <
>> [email protected]>
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > Hello Everyone,
>> > > > > > > > >
>> > > > > > > > > I'm wanting to setup a syslog server that combines the
>> three
>> > > > > > > > > programs
>> > > > > > > > listed above with rsyslog. Has anyone had any success
>>using
>> > this?
>> > > > > > > > I'm
>> > > > > > > > running on a CentOS 6.5 and finding adequate instructions
>>on
>> > how
>> > > > > > > > to not only setup all three PLUS rsyslog has been somewhat
>> of a
>> > > > > > > > challenge.
>> > > > > > > > >
>> > > > > > > > > This issue that I run into is on how to get
>> > > > > > > > > logstash/elasticsearch and
>> > > > > > > > kibana to talk with rsyslog. Halp meh! Please!
>> > > > > > > > >
>> > > > >
>> > > > > > rsyslog mailing list
>> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > > > > > http://www.rsyslog.com/professional-services/
>> > > > > > What's up with rsyslog? Follow
>>https://twitter.com/rgerhardsNOTE
>> > > > > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad
>> > > of
>> > > > > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
>>if
>> you
>> > > > > > DON'T LIKE THAT.
>> > > > > >
>> > > > >
>> > > > > --
>> > > > > Rick Brown
>> > > > > Office of Information Technology
>> > > > > Georgia Institute of Technology
>> > > > > _______________________________________________
>> > > > > rsyslog mailing list
>> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > > > > http://www.rsyslog.com/professional-services/
>> > > > > What's up with rsyslog? Follow https://twitter.com/rgerhards
>>NOTE
>> > > WELL:
>> > > > > This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>> > sites
>> > > > > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>DON'T
>> > > LIKE
>> > > > > THAT.
>> > > > > _______________________________________________
>> > > > > rsyslog mailing list
>> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > > > > http://www.rsyslog.com/professional-services/
>> > > > > What's up with rsyslog? Follow https://twitter.com/rgerhards
>> > > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by
>>a
>> > > myriad
>> > > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
>>if
>> > you
>> > > > > DON'T LIKE THAT.
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Performance Monitoring * Log Analytics * Search Analytics
>> > > > Solr & Elasticsearch Support * http://sematext.com/
>> > > > _______________________________________________
>> > > > rsyslog mailing list
>> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > > > http://www.rsyslog.com/professional-services/
>> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards
>> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> > myriad
>> > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
>> you
>> > > > DON'T LIKE THAT.
>> > > >
>> > > _______________________________________________
>> > > rsyslog mailing list
>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > > http://www.rsyslog.com/professional-services/
>> > > What's up with rsyslog? Follow https://twitter.com/rgerhards
>> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad
>> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
>>you
>> > > DON'T LIKE THAT.
>> > >
>> >
>> >
>> >
>> > --
>> > Performance Monitoring * Log Analytics * Search Analytics
>> > Solr & Elasticsearch Support * http://sematext.com/
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com/professional-services/
>> > What's up with rsyslog? Follow https://twitter.com/rgerhards
>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>>myriad
>> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> > DON'T LIKE THAT.
>> >
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>_______________________________________________
>rsyslog mailing list
>http://lists.adiscon.net/mailman/listinfo/rsyslog
>http://www.rsyslog.com/professional-services/
>What's up with rsyslog? Follow https://twitter.com/rgerhards
>NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
