Hello Dan! I was wrong, when the log has 2 digits in the day field, there's only one space, the way you said it, sorry. But I still have a problem, as the date is as metadata, how do I decode it as timestamp?
See in the entry below from the kibana, that the date field is not recognized as timestamp, I sent an entry from another date, to test and that was the output: How do solve this? Em sábado, 11 de novembro de 2017 13:17:57 UTC-3, dan (ddpbsd) escreveu: > > On Thu, Nov 9, 2017 at 5:12 PM, <[email protected] <javascript:>> wrote: > > So, there are 2 spaces between the "MMM" and the day, but it's the > pfSense > > log, it's like this. And the problem is in have 1 digit, when it has 2 > > digits the problem does not occur. > > > > > >> 2 spaces confuses the pre-decoder > > > > And so anyway is it really a bug? > > Pretty much every log message I've seen follows the same rule (2 > spaces for single digit days, 1 space for double digit days). I'd say > it's a bug in pfsense logs. > > > > > I also tested as follows(whitout sucess): > > > > <prematch>^\w\w\w\s\s\d+ \d\d:\d\d:\d\d pfSense</prematch> > > > > The problem is the inconsistency in the pfsense logs. You could try > modifying the predecoding parts of the code to handle their strange > logs, but I'd be worried about fallout from such changes. > > > > > Em quinta-feira, 9 de novembro de 2017 16:25:11 UTC-3, dan (ddpbsd) > > escreveu: > >> > >> > >> > >> On Nov 9, 2017 14:01, <[email protected]> wrote: > >> > >> Hi Dan! > >> > >> I'm ashamed after your explanation, thank you very much for the answer. > >> But I will still expose another problem I had trying to make this > decoder. I > >> opened a call on github > >> > >> I think you have a problem with the pre-decoder. He is ignoring p "\d+" > in > >> "^\w\w\w \d+". That's why the "log:" entry is starting with > >> "5,,,1000102433,at0...". > >> > >> If a 2-digit log entry is placed on the day, the "log:" entry will > start > >> correctly. The "\d+" should serve with one or more numerical inputs. > >> > >> > >> > >> It's hard to tell feom a picture, but I think there are too many spaces > >> between Nov and 18. There should only be 1 space in double digit dates, > not > >> 2. 2 spaces confuses the pre-decoder, so everything gets decoded > instead of > >> just the log. > >> > >> > >> > >> Here is a print of the problem and the github call: > >> > >> > >> https://github.com/ossec/ossec-hids/issues/1315 > >> > >> > >> > >> Em quinta-feira, 9 de novembro de 2017 09:56:32 UTC-3, dan (ddpbsd) > >> escreveu: > >>> > >>> On Wed, Nov 8, 2017 at 11:52 AM, <[email protected]> wrote: > >>> > Hey guys! > >>> > I made a decoder for pfSense, but it is not being recognized by > ossec. > >>> > > >>> > Follow the decoder with a log sample: > >>> > > >>> > <!-- Nov 7 12:37:34 pfSense filterlog: > >>> > > >>> > > 5,,,1000102433,em0,match,block,in,4,0x0,,128,24677,0,none,17,udp,186,10.9.0.119,10.9.0.255,17500,17500,166 > > > >>> > --> > >>> > <decoder name="pfsense"> > >>> > <program_name>pfsense</program_name> > >>> > </decoder> > >>> > > >>> > <decoder name="pfsense"> > >>> > <prematch>^\w+ \d+ \d+:\d+:\d+ pfSense |\w+ \d+ \d+:\d+:\d+ > >>> > pfSense > >>> > </prematch> > >>> > </decoder> > >>> > > >>> > <decoder name="pfsense_filter"> > >>> > <parent>pfsense</parent> > >>> > <regex offset="after_parent">^filterlog: > >>> > > >>> > > \d+,,,\d+,\S+,\w+,\w+,\w+,\d+,\S+,,\d+,\d+,\d+,none,\d+,udp,\d+,(\d+.\d+.\d+.\d+),(\d+.\d+.\d+.\d+),\d+,\d+,\d+$</regex> > > > >>> > <order>srcip, dstip</order> > >>> > </decoder> > >>> > > >>> > I put it in the folder of the decoders and tested with the > >>> > ossec-logtest, > >>> > follow the output: > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > I'd like to know what's wrong. > >>> > > >>> > >>> Here is the output of the log sample before adding a decoder: > >>> **Phase 1: Completed pre-decoding. > >>> full event: 'Nov 7 12:37:34 pfSense filterlog: > >>> > >>> > 5,,,1000102433,em0,match,block,in,4,0x0,,128,24677,0,none,17,udp,186,10.9.0.119,10.9.0.255,17500,17500,166' > > > >>> hostname: 'pfSense' > >>> program_name: 'filterlog' > >>> log: > >>> > '5,,,1000102433,em0,match,block,in,4,0x0,,128,24677,0,none,17,udp,186,10.9.0.119,10.9.0.255,17500,17500,166' > > > >>> > >>> **Phase 2: Completed decoding. > >>> No decoder matched. > >>> > >>> > >>> Pay close attention to the "log:" line. Everything before that in the > >>> log message won't be parsed by decoders. It's metadata, and it's taken > >>> care of in pre-decoding. You can see the hostname and program there in > >>> Phase 1. > >>> > >>> Let's try your first decoder: > >>> <decoder name="pfsense"> > >>> <program_name>^filterlog</program_name> > >>> </decoder> > >>> > >>> Adding this simple decoder to `/var/ossec/etc/local_decoder.xml` gives > >>> us the following output: > >>> **Phase 1: Completed pre-decoding. > >>> full event: 'Nov 7 12:37:34 pfSense filterlog: > >>> > >>> > 5,,,1000102433,em0,match,block,in,4,0x0,,128,24677,0,none,17,udp,186,10.9.0.119,10.9.0.255,17500,17500,166' > > > >>> hostname: 'pfSense' > >>> program_name: 'filterlog' > >>> log: > >>> > '5,,,1000102433,em0,match,block,in,4,0x0,,128,24677,0,none,17,udp,186,10.9.0.119,10.9.0.255,17500,17500,166' > > > >>> > >>> **Phase 2: Completed decoding. > >>> decoder: 'pfsense' > >>> > >>> So now Phase 2 matches the pfsense decoder, but we want more. We can > >>> take one of your other decoders, remove some bits the decoder doesn't > >>> see, and get some useful information. Here's what I ended up with: > >>> <decoder name="pfsense_filter"> > >>> <parent>pfsense</parent> > >>> <regex > >>> > offset="after_parent">^\d+,,,\d+,\S+,\w+,\w+,\w+,\d+,\S+,,\d+,\d+,\d+,none,\d+,udp,\d+,(\d+.\d+.\d+.\d+),(\d+.\d+.\d+.\d+),\d+,\d+,\d+$</regex> > > > >>> <order>srcip, dstip</order> > >>> </decoder> > >>> > >>> In your decoder the <regex> started with '^filterlog', but as we see > >>> from the 'log:' entry in the logtest output, the decoders do not see > >>> that information. It's covered in the first decoder which is looking > >>> for the program_name which is handled by the pre-decoder (easy, > >>> right?). > >>> Here's the output I get after adding this decoder to > local_decoder.xml: > >>> **Phase 1: Completed pre-decoding. > >>> full event: 'Nov 7 12:37:34 pfSense filterlog: > >>> > >>> > 5,,,1000102433,em0,match,block,in,4,0x0,,128,24677,0,none,17,udp,186,10.9.0.119,10.9.0.255,17500,17500,166' > > > >>> hostname: 'pfSense' > >>> program_name: 'filterlog' > >>> log: > >>> > '5,,,1000102433,em0,match,block,in,4,0x0,,128,24677,0,none,17,udp,186,10.9.0.119,10.9.0.255,17500,17500,166' > > > >>> > >>> **Phase 2: Completed decoding. > >>> decoder: 'pfsense' > >>> srcip: '10.9.0.119' > >>> dstip: '10.9.0.255' > >>> > >>> We now have src and dst IPs. > >>> > >>> > >>> > -- > >>> > > >>> > --- > >>> > You received this message because you are subscribed to the Google > >>> > Groups > >>> > "ossec-list" group. > >>> > To unsubscribe from this group and stop receiving emails from it, > send > >>> > an > >>> > email to [email protected]. > >>> > For more options, visit https://groups.google.com/d/optout. > >> > >> ... > >> > >> > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups > > "ossec-list" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to [email protected] <javascript:>. > > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
