This email went through a few different iterations, so if something seems wonky let me know. It just means I didn't proofread the final very well and I don't mind fixing it.
On Sun, Sep 19, 2010 at 2:43 PM, Jeremy Lee <[email protected]> wrote: > Hey guys, > > I have another one I can't figure out. > > A couple sample logs: > > Sep 18 08:01:03 linksux ser2sock[2515]: > [000003011000--------],004,[f70000071004030028020000000000],"FAULT > 04 "#012 > > Sep 19 11:37:11 linksux ser2sock[2515]: > [100000011000--------],008,[f70000071008001c28020000000000]," DISARMED > CHIME Ready to Arm "#012 > > > My decoder rule - I checked this on a regex builder site, but it may have > been different than what OSSEC uses: > > <decoder name="ad2usb"> > <!-- <prematch>\S\d+\S+</prematch> --> > <prematch>^\w+\s\d*\d*\s\d+\S\d+\S\d+\s\w+\s\w+\d\w+\S\d+\S+\s</prematch> > <regex > offset="after_prematch">\S\d+\S+\d+\S+\w\d+\w\d+\S+\s*\w+\s*\S*\S*\S*\w+\S*\S*\S*\s*\w+\s*\w+\s*\w+\s*\S+</regex> > <order>id</order> > </decoder> > You have the <order> tag, but nothing is in parenthesis in the regex line. The things in parenthesis are what will populate the entries in <order>. You've also complicated this A LOT. Start simple. VERY basic decoder for alerts with ser2sock: <decoder name="forj"> <program_name>^ser2sock</program_name> </decoder> I'll try to break this down into pieces with the following log you provided: Sep 18 08:01:03 linksux ser2sock[2515]: [000003011000--------],004,[f70000071004030028020000000000],"FAULT 04 "#012 'Sep 18 08:01:03 ' This is the timestamp. This is a fairly standard format so ossec recognizes it nicely. Ignore it. 'linksux ' This is the hostname of the server. This is fairly standard. Ignore it unless you need it in a rule. 'ser2sock' This is the application name. This will populate <program_name> in a decoder. This is the easiest way to make a basic decoder for an application. '[2515]: ' This is the pid of the program name. Ignore it. '[000003011000--------],004,[f70000071004030028020000000000],"FAULT 04 "#012' This is your log message. The decoder I pasted above will recognize all log messages from ser2sock. Example (using the second log message you posted and ossec-logtest): **Phase 1: Completed pre-decoding. full event: 'Sep 19 11:37:11 linksux ser2sock[2515]: [100000011000--------],008,[f70000071008001c28020000000000]," DISARMED CHIME Ready to Arm "#012' hostname: 'linksux' program_name: 'ser2sock' log: '[100000011000--------],008,[f70000071008001c28020000000000]," DISARMED CHIME Ready to Arm "#012' **Phase 2: Completed decoding. decoder: 'forj' ------------------------------------------------------ Now that we have the basics out of the way, we want to pull out an 'id' from the message. Which part of the log is the "id?" I'll assume it is the 000003011000 and 100000011000 in the messages for this example. If I guessed wrong you can hopefully use the information I'm providing to fix it. If you try and don't succeed, reply and I'll help. Assuming all ser2sock messages are VERY similar we'll just add to the decoder above. If they aren't a child decoder can be created instead so multiple message formats can be supported. Decoder from above: <decoder name="forj"> <program_name>^ser2sock</program_name> </decoder> Adding the 'id': <decoder name="forj"> <program_name>^ser2sock</program_name> <regex>^\p(\d+)\p\.+#\d+$</regex> <order>id</order> </decoder> The id field will be populated with the string of numbers near the beginning the log message thanks to the parenthesis. Using this new decoder we get the following information via ossec-logtest: Sep 18 08:01:03 linksux ser2sock[2515]: [000003011000--------],004,[f70000071004030028020000000000],"FAULT 04 "#012 **Phase 1: Completed pre-decoding. full event: 'Sep 18 08:01:03 linksux ser2sock[2515]: [000003011000--------],004,[f70000071004030028020000000000],"FAULT 04 "#012' hostname: 'linksux' program_name: 'ser2sock' log: '[000003011000--------],004,[f70000071004030028020000000000],"FAULT 04 "#012' **Phase 2: Completed decoding. decoder: 'forj' id: '000003011000' Sep 19 11:37:11 linksux ser2sock[2515]: [100000011000--------],008,[f70000071008001c28020000000000]," DISARMED CHIME Ready to Arm "#012 **Phase 1: Completed pre-decoding. full event: 'Sep 19 11:37:11 linksux ser2sock[2515]: [100000011000--------],008,[f70000071008001c28020000000000]," DISARMED CHIME Ready to Arm "#012' hostname: 'linksux' program_name: 'ser2sock' log: '[100000011000--------],008,[f70000071008001c28020000000000]," DISARMED CHIME Ready to Arm "#012' **Phase 2: Completed decoding. decoder: 'forj' id: '100000011000' ---------------------------------------------------------------- We now have an id. Breakdown of the regex: <regex>^\p(\d+)\p\.+#\d+$</regex> '^' This is the beginning of the log message. '\p' This is a stand in for the '[' character. It actually matches a number of characters, not just this one. '(' This is the beginning of a section we would like to pull out of the log message and store in a variable (id). '\d+' This represents 1 or more numeric characters. This is the data that will populate the id field. If this information could be alpha-numeric, this will have to change. ')' This is the end of the section we want to populate a variable. Nothing after this will be put into the first variable. '\p' There is a dash at the end of the string of numbers, and the \p represents it. I don't know if some of your messages may include longer numbers without the dashes. In those cases the \p will represent the ending ']'. '\.+' This means 1 or more characters. Any characters. '"#\d+' These are the characters immediately following what we wanted in the id variable ("#), and the numbers at the end. Using '\d+" means that there will be 1 or more numbers (012 in this case). '$' This is the end of the message. Now it's time to add rules to these decoders. (see below) > > And lastly, the rules, I setup: > > <group name="local,syslog,errors,openvpnals,windows,ad2usb"> > > <rule id="100040" level="0"> // I also tried setting the level to 5 > <decoded_as>ad2usb</decoded_as> > <description>AD2USB Alarm Status.</description> > </rule> > > <rule id="100041" level="12"> > <if_sid>100040</if_sid> > <match>100000011000</match> > <description>Alarm armed</description> > </rule> > > </group> > Any log message that matches the "forj" decoder will trigger the following rule (becuase of the <decoded_as> option): <rule id="490019" level="0" noalert="1"> <decoded_as>forj</decoded_as> <description>Collected ser2sock alerts.</description> </rule> However, the 'noalert="1"' setting will make this not show up as an alert. We will use it to collect all log messages for ser2sock in one place. So a followup rule will be necessary to complete this. So here is a second rule to alert on id '100000011000'. <rule id="490020" level="12"> <if_sid>490019</if_sid> <id>100000011000</id> <description>blah blah blah</description> </rule> We have 2 fields that look for information in the log message in this rule. First the log message has to match rule id 490019 (the first rule above), as will any log message triggered by ser2sock. Second, we look for the id field being populated by '100000011000' as the second log message does. Now we feed the log messages through ossec-logtest again and get the following: Sep 18 08:01:03 linksux ser2sock[2515]: [000003011000--------],004,[f70000071004030028020000000000],"FAULT 04 "#012 **Phase 1: Completed pre-decoding. full event: 'Sep 18 08:01:03 linksux ser2sock[2515]: [000003011000--------],004,[f70000071004030028020000000000],"FAULT 04 "#012' hostname: 'linksux' program_name: 'ser2sock' log: '[000003011000--------],004,[f70000071004030028020000000000],"FAULT 04 "#012' **Phase 2: Completed decoding. decoder: 'forj' id: '000003011000' Sep 19 11:37:11 linksux ser2sock[2515]: [100000011000--------],008,[f70000071008001c28020000000000]," DISARMED CHIME Ready to Arm "#012 **Phase 1: Completed pre-decoding. full event: 'Sep 19 11:37:11 linksux ser2sock[2515]: [100000011000--------],008,[f70000071008001c28020000000000]," DISARMED CHIME Ready to Arm "#012' hostname: 'linksux' program_name: 'ser2sock' log: '[100000011000--------],008,[f70000071008001c28020000000000]," DISARMED CHIME Ready to Arm "#012' **Phase 2: Completed decoding. decoder: 'forj' id: '100000011000' **Phase 3: Completed filtering (rules). Rule id: '490020' Level: '12' Description: 'blah blah blah' **Alert to be generated. -------------------------------------------- Notice how the first message only makes it to Phase 2? The id field for that message is populated with '000003011000' and there is no rule associated with that content. A third rule would need to be setup for that message (leaving this as an exercise for the reader, PLEASE post if you get it/don't get it). The second message makes it to Phase 3 since there is a rule associated with that id. It is triggered and ossec-logtest tells you that an alert would be sent out. > > I have debug turned on for analyzer, logcollector, and agent (on the server > and agent) and I see the logcolelctor reading hte file on the agent but the > server doesn't display any log messages (I'm guessing it would?). > Interestingly, I echoed the same log message into /var/log/syslog and > /var/log/messages on the server, which are supposed to be monitored by OSSEC > as well, but I didn't see anything show up from the debug output like it > wason the OSSEC agent. > > > > > Using ossec-logtest makes this easier/cleaner. Using ossec-logtest you don't have to populate your real log files with junk. You could also use logger to do this if you can't use ossec-logtest for some reason. Example: echo '[100000011000--------],008,[f70000071008001c28020000000000]," DISARMED CHIME Ready to Arm "#012' | logger I usually write decoders and rules in pieces. I put in a basic decoder (like the first one I posted above), and run ossec-logtest. If it works I can then modify it do more (like adding the regex/order lines). I often have to do the regex lines themselves in a similar way, starting off very small and building it up as I get the syntax correct. Run ossec-logtest between steps to make sure everything is still working. If anyone has comments or questions about the information above please (please please please) let me know (on list if it's topical to this thread, offlist or IRC if it isn't). I'm thinking about doing some kind of post along these lines for the Week of OSSEC, and would love feedback so I can make it better/clearer for everyone.
