> Is anyone out there taking in raw windows, proxy, or other complex logs
and
> breaking them apart for either simplification or volume reduction?
We are using CEE modification on syslog gates like:
template(name="cee" type="list") {
constant(value="<") property(name="pri") constant(value=">")
property(name="timereported" dateFormat="rfc3339")
constant(value=" ") property(name="$myhostname")
constant(value=" ") property(name="programname")
constant(value=" ")
constant(value="@cee: {")
#SYSLOG
constant(value="\"using_cee_relp\":\"yes\", ")
....
On rsyslog-elasticsearch side ->
##STORE RULESETS
ruleset(name="store") {
if $parsesuccess == "OK" then {
#ES - views
if ( strlen( $!msg_class) >= 1 and strlen(
$!msg_view) >= 1 and $!msg_class != "msg") then {
set $.dynafile = "1.parsed";
if $.relp_server == '127.0.0.1' then {
call store_es
call file_json
} else {
call file_raw
}
stop
}
....
Do you have a more detailed question? Rsyslog is an awesome message
queuing engine =)
/---/
*/Best regards,/*
/Eugene Istomin/
> this is an interesting discussion, I'd be curious to see what people are
> doing to parse/normalize messages as they are coming in.
>
> Several projects that I'm associated with tend to revolve around
> extrapolating properties of a message and then assigning them
(typically
> post-receipt and generating more load.)
>
> Is anyone out there taking in raw windows, proxy, or other complex logs
and
> breaking them apart for either simplification or volume reduction? (for
> example eliminating the description field of windows events while
enhancing
> the event properties before sending on down the wire?)
>
> On Tue Dec 30 2014 at 3:20:12 PM David Lang <[email protected]> wrote:
> > On Tue, 30 Dec 2014, Kendall Green wrote:
> > > Hello, I would like to share experience with normalization of windows
> >
> > event
> >
> > > logs with rsyslog and have critique of configuration for the latest
> >
> > syntax
> >
> > > directives and supported functions. In response to a previous
message
> > > regarding the reparse() feature enhancement, there appears to be
> > > imminent
> > > refactoring of parser modules.
> >
> > parser modules are not the same as the mmnormalize rulebase,
parser
> > modules are
> > applied to messages as they arrive at the rsyslog server and populate
the
> > standard properties, mmnormalize is intended to populate other
variables..
> >
> > > Is it possible to output mmnormalize rulebase to json path and
output on
> > > template which does not include the msg/userawmsg field?
> >
> > if you have JSON, you should use mmjsonparse to extract the
variables, but
> > once
> > you have the variables parsed out, you can use them in any template.
> >
> > To give you more information, I would need a better idea of what you
are
> > trying
> > to do.
> >
> > > Thank you for any recommendations or examples related to new
> >
> > normalization
> >
> > > modules.
> >
> > While there may be enhancements to the normalization, that is
completely
> > separate from the parser modules (I know, it's a bit confusing)
> >
> > David Lang
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you
> > DON'T LIKE THAT.
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
