Ups, sorry this was a mistake. Please ignore my message this is the wrong topic.
Tim 2013/7/19 Tim Eifler <[email protected]> > Thank you for the the advice! I re-upload it and it should work now. > > > Tim > > > 2013/7/18 Todd Mortensen <[email protected]> > >> An another side note, I wonder if my liblognorm may be too old. I was >> not >> able to find what version rsyslog 7.4.2 requires. >> >> Here is the version I am using, the one supplied in the ubuntu repo's. >> >> ii liblognorm-dev >> 0.3.2-1 Log normalizing >> library >> ii liblognorm0 >> 0.3.2-1 Log normalizing >> library >> >> >> On Thu, Jul 18, 2013 at 7:48 AM, Todd Mortensen <[email protected] >> >wrote: >> >> > This is the sample line. >> > >> > [1084540.211910] Denied: IN=eth0 OUT= MAC=00:30:48:90:cc:a6:00:30: >> > 48:da:48:e8:08:00 SRC=10.10.10.10 DST=10.10.10.11 LEN=60 TOS=0x10 >> > PREC=0x00 TTL=64 ID=28843 DF PROTO=TCP SPT=44075 DPT=444 WINDOW=14600 >> > RES=0x00 SYN URGP=0 >> > >> > I will try to get a run going with valgrind and send the output. >> > >> > Todd >> > >> > >> > >> > On Thu, Jul 18, 2013 at 7:24 AM, Rainer Gerhards < >> [email protected] >> > > wrote: >> > >> >> I am not sure if I manage to look at it before vacation, but could you >> >> send >> >> me a log sample that generates the garbagge? Could you run it under >> >> valgrind control? I guess there is some mis-access taking place... >> >> >> >> Rainer >> >> >> >> >> >> On Wed, Jul 17, 2013 at 4:03 AM, Todd Mortensen <[email protected] >> >> >wrote: >> >> >> >> > I have made some progress in getting it to work, I removed the >> leading >> >> > space in the rule and it now matches. I totally missed that on log >> >> output. >> >> > >> >> > rule=:%kerntime:word% Denied: IN=%in:word% OUT=%out:word% >> MAC=%mac:word% >> >> > SRC=%src-ip:ipv4% DST=%dst-ip:ipv4% LEN=%len:number% TOS=%tos:word% >> >> > PREC=%prec:word% TTL=%ttl:number% ID=%id:number% %DF:word% >> >> > PROTO=%proto:word% SPT=%src-port:number% DPT=%dst-port:number% >> >> > WINDOW=%window:number% RES=%res:word% %pkt-type:word% >> URGP=%urgp:number% >> >> > >> >> > I did find how to use the iptables type but my results where not very >> >> good. >> >> > >> >> > using this rule my output was has a lot of garbage output in the log, >> >> looks >> >> > like it wrote out binary data: >> >> > >> >> > rule=:%kerntime:word% Denied: %iptables:iptables% >> >> > >> >> > { "IN": "eth0", "OUT": "", "MAC": >> >> > "00:30:48:90:cc:a6:00:30:48:da:48:e8:08:00", "SRC": "10.10.10.10", >> >> "DST": >> >> > "10.10.10.11", "LEN": "60", "TOS": "0x10", "PREC": "0x00", "TTL": >> "64", >> >> > "ID": "10181", "DF": "[*PRESENT*]", "PROTO": "TCP", "SPT": "51765", >> >> "DPT": >> >> > "444", "WINDOW": "14600", "RES": "0x00", "SYN": "[*PRESENT*]", >> "URGP": >> >> "0", >> >> > >> >> > >> >> >> "U)@00:30:48:90:cc:a6:00:30:48:da:48:e8:08:00ü«^?xü«^?ÿÿÿÿÿÿÿÿ%\u0003\u0010LENü«^?<80>¸ü«^?%\u0002\u001060ü«^?¼ü«^?%LEN«^?xü«^?@µü«^?%60ü«^?xü«^?¹ü«^?%\býý\u0001p¾ü«^?<80>ºü«^?%\u0003\bLENü«^?<80>»ü«^?%`´ü«^?ð¿ü«^?ðbü«^?%\u0003\u0010TOSü«^?ÿÿÿÿÿÿÿÿ%\u0004\u00100x10«^?ÿÿÿÿÿÿÿÿ%TOS«^?xü«^?¸ü«^?%0x10^?xü«^?ÿÿÿÿÿÿÿÿ%\býý\u0001P¿ü«^?ÿÿÿÿÿÿÿÿ%\u0003\bTOSü«^?»ü«^?%": >> >> > "[*PRESENT*]", "kerntime": "[1151929.993763]" } >> >> > >> >> > Using the full rule I get results like this, nice and clean. >> >> > >> >> > { "urgp": "0", "pkt-type": "SYN", "res": "0x00", "window": "14600", >> >> > "dst-port": "444", "src-port": "51954", "proto": "TCP", "DF": "DF", >> >> "id": >> >> > "31890", "ttl": "64", "prec": "0x00", "tos": "0x10", "len": "60", >> >> "dst-ip": >> >> > "10.10.10.10", "src-ip": "10.10.10.11", "mac": >> >> > "00:30:48:90:cc:a6:00:30:48:da:48:e8:08:00", "in": "eth0", >> "kerntime": >> >> > "[1152127.460873]" } >> >> > >> >> > I will continue my journey using the other method for now. >> >> > >> >> > >> >> > On Tue, Jul 16, 2013 at 3:19 PM, Todd Mortensen < >> [email protected] >> >> > >wrote: >> >> > >> >> > > I am not sure, I honestly stole the examples from the sagan >> project. >> >> > > >> >> > > I just tried using the following to test. >> >> > > >> >> > > rule=: %kerntime:word% Denied: %iptables% >> >> > > rule=: [%kerntime:word%] Denied: %iptables% >> >> > > >> >> > > I also tried adding [] to the above full line as well. Still >> doesn't >> >> > > return any parsed values. >> >> > > >> >> > > I forgot to mention this is version 7.4.2 on ubuntu 12.04 from my >> ppa >> >> > > >> >> > > ii rsyslog-mmnormalize 7.4.2-0ubuntu1ppa2 >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > On Tue, Jul 16, 2013 at 3:12 PM, David Lang <[email protected]> wrote: >> >> > > >> >> > >> it looks to me like your ruleset is not matching the rule >> >> > >> >> >> > >> are you sure that %kerntime:word% includes the square brackets >> and it >> >> > >> shouldn't be [%kerntime:word%] instead? >> >> > >> >> >> > >> there is a special type 'iptables' that was created explicitly for >> >> > >> matching the name=value format of iptables rules. >> >> > >> >> >> > >> I think your rule can be simplified to something like: >> >> > >> >> >> > >> rule=: [%kerntime:word%] Denied: %iptables% >> >> > >> >> >> > >> David Lang >> >> > >> >> >> > >> On Tue, 16 Jul 2013, Todd Mortensen wrote: >> >> > >> >> >> > >> Date: Tue, 16 Jul 2013 14:53:36 -0700 >> >> > >>> From: Todd Mortensen <[email protected]> >> >> > >>> Reply-To: rsyslog-users <[email protected]> >> >> > >>> To: rsyslog-users <[email protected]> >> >> > >>> Subject: [rsyslog] mmnormalize and iptable logs >> >> > >>> >> >> > >>> >> >> > >>> I am trying my first attempts at using mmnormalize. >> >> > >>> >> >> > >>> My goal is to parse out my iptable firewall logs into a cee >> format >> >> so I >> >> > >>> can >> >> > >>> then send them to elasticsearch. >> >> > >>> >> >> > >>> But so far my logs just show unparsed items. Can anyone shed any >> >> light >> >> > >>> to >> >> > >>> where I am going wrong here. >> >> > >>> >> >> > >>> Relevant sections from rsyslog.conf >> >> > >>> >> >> > >>> module(load="mmnormalize") >> >> > >>> action(type="mmnormalize" UseRawMsg="off" >> >> > >>> ruleBase="/etc/rsyslog.d/**normalize.rb") >> >> > >>> >> >> > >>> module(load="mmjsonparse") >> >> > >>> >> >> > >>> >> >> > >>> #template(name="cee" type="string" string="%$!all-json%\n") >> >> > >>> template(name="cee" type="string" string="%$!%\n") >> >> > >>> >> >> > >>> *.* { action (name="cee" >> type="omfile" >> >> > >>> Template="cee" file="/var/log/cee.log") } >> >> > >>> >> >> > >>> I have also tried a couple variations of this but they still >> don't >> >> > match >> >> > >>> >> >> > >>> normalize.rb >> >> > >>> rule=: %kerntime:word% Denied: IN=%in:word% OUT= MAC=%mac:word% >> >> > >>> SRC=%src-ip:ipv4% DST=%dst-ip:ipv4% LEN=%len:number% >> TOS=%tos:word% >> >> > >>> PREC=%prec:word% TTL=%ttl:number% ID=%id:number% %DF:word% >> >> > >>> PROTO=%proto:word% SPT=%src-port:number% DPT=%dst-port:number% >> >> > >>> WINDOW=%window:number% RES=%res:word% %pkt-type:word% >> >> > URGP=%urgp:number% >> >> > >>> >> >> > >>> I also tried to use the iptales type, but I am not able to find >> many >> >> > docs >> >> > >>> on its usage. >> >> > >>> >> >> > >>> rule=: %kerntime:word% Denied: %iptables:iptables% >> >> > >>> >> >> > >>> >> >> > >>> Sample log line: >> >> > >>> >> >> > >>> [1084540.211910] Denied: IN=eth0 OUT= >> >> > >>> MAC=00:30:48:90:cc:a6:00:30:**48:da:48:e8:08:00 SRC=10.10.10.10 >> >> > >>> DST=10.10.10.11 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=28843 DF >> >> PROTO=TCP >> >> > >>> SPT=44075 DPT=444 WINDOW=14600 RES=0x00 SYN URGP=0 >> >> > >>> >> >> > >>> >> >> > >>> This is what I see in my cee logfile and it shows unpared-data. >> >> > >>> >> >> > >>> # { "originalmsg": "[1084540.211910] Denied: IN=eth0 OUT= >> >> > >>> MAC=00:30:48:90:cc:a6:00:30:**48:da:48:e8:08:00 SRC=10.10.10.10 >> >> > >>> DST=10.10.10.11 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=28843 DF >> >> PROTO=TCP >> >> > >>> SPT=44075 DPT=444 WINDOW=14600 RES=0x00 SYN URGP=0 ", >> >> "unparsed-data": >> >> > >>> "[1084540.211910] Denied: IN=eth0 OUT= >> >> > >>> MAC=00:30:48:90:cc:a6:00:30:**48:da:48:e8:08:00 SRC=10.10.10.10 >> >> > >>> DST=10.10.10.11 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=28843 DF >> >> PROTO=TCP >> >> > >>> SPT=44075 DPT=444 WINDOW=14600 RES=0x00 SYN URGP=0 " } >> >> > >>> ______________________________**_________________ >> >> > >>> rsyslog mailing list >> >> > >>> http://lists.adiscon.net/**mailman/listinfo/rsyslog< >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog> >> >> > >>> http://www.rsyslog.com/**professional-services/< >> >> > http://www.rsyslog.com/professional-services/> >> >> > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> >> > myriad >> >> > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> if >> >> you >> >> > >>> DON'T LIKE THAT. >> >> > >>> >> >> > >>> ______________________________**_________________ >> >> > >> rsyslog mailing list >> >> > >> http://lists.adiscon.net/**mailman/listinfo/rsyslog< >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog> >> >> > >> http://www.rsyslog.com/**professional-services/< >> >> > http://www.rsyslog.com/professional-services/> >> >> > >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> >> myriad >> >> > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> >> you >> >> > >> DON'T LIKE THAT. >> >> > >> >> >> > > >> >> > > >> >> > _______________________________________________ >> >> > rsyslog mailing list >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > http://www.rsyslog.com/professional-services/ >> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> >> > DON'T LIKE THAT. >> >> > >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com/professional-services/ >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> DON'T LIKE THAT. >> >> >> > >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
