Hi, On Fri, Feb 6, 2015 at 3:06 PM, David Lang <[email protected]> wrote:
> On Fri, 6 Feb 2015, Joe Blow wrote: > > Hey folks, >> >> I'm trying to parse some cisco ASA logs using rulebase rules. Here is the >> example rule: >> >> rule=: %begin:char-to:"%"%\x25ASA-%ddd:char-to:-%-%eee:number%: Teardown >> TCP connection %cnumber:number% for outside:%ohost:ipv4%/%oport:number% >> to >> inside:%ihost:ipv4%/%iport:number% duration %duration:word% bytes >> %msgbytes:number% TCP %flags:word% (%uname:word%) >> >> Which i'm hoping matches this (broken) rsyslog line: >> >> 06 2015 13:12:55: %ASA-6-302014: Teardown TCP connection 2127154348 for >> outside:1.1.1.1/64053 to inside:2.2.2.2/443 duration 0:00:00 bytes 3267 >> TCP >> FINs (myusername) >> > > > <snip> > >> Does anyone have any ideas for why none of my rulebase objects seem to >> work? The only data I receive in ES is this, which I believe confirms the >> leading space and the broken date at the beginning (which i'm happy to >> just >> strip off). Has anyone seen anything like this before? Throwing rsyslog >> into debug mode doesn't give me great debug logging related to the >> mmnormalize module (that I can tell). >> > > rsyslog has no way to debug mmnormalize items, but liblognorm has the > tool: /usr/lib/lognorm/lognormalizer that you can use > > echo "log line" | /usr/lib/lognorm/lognormalizer -r ruleset -T > > will show you some output that will probably be originalmessage and > notparsed, the notparsed is the part of the log message left when the > parser gave up. If that doesn't help you figure out your problem, you can > add -v which will give you a character-by-character breakdown of the > parsing, look for the last thing before it starts backtracking. > > In your case, you have a problem that the flags is not always one word, > it's one or more words. I posted what I had to do for this in an e-mail a > day or so ago (mmnormalize thoughts). > > Also, to strip off the date and time at the beginning, you need to have a > prefix that will match them or have it as part of your rule, otherwise it > won't match. You can use a template to set a variable and have mmnormalize > parse that variable instead of parsing $msg or $rawmsg. > > I've been doing a bunch of work on this in the last few days, I'll see > about posting my config later today. > > We really do need to put together parse rulesets for the common log types, > Cisco being probably the most common one that people need to parse. > liblognorm has a spot on it's website for contributed rules, but nobody has > contributed any :-( > > I'll see about posting what I have later today and let's see about sharing > the effort for a bit. sound reasonable? Are you saying you want to use llnorm to create grok-like, llnorm-based patterns/rules for parsing common log formats and publishing them? If so, +1 for putting that somewhere VERY visible and easily contributable-to. :) Thanks, Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
