Yes but all from log monitoring. But I checked just now, and it has been running for some time now and I still can't see an alert. One thing I noticed is that without the custom decoder and having the rule set to match the output and alert when it sees any number by using the regex "/d" alerts are generated (although not for over 80% as I need it) which could mean that the regex is not matching the actual log. As I posted earlier, in archives.log I find the following log:
2013 Dec 30 15:26:28 (m-s-comm1) 10.152.1.227->mem-usage ossec: output: 'mem-usage': 71% The percentage is always in a line beneath the log. I don't know if that makes a difference. In ossec-logtest I input the following as a single line to test: ossec: output: 'mem-usage': 71% On 30 Dec 2013 19:43, "dan (ddp)" <[email protected]> wrote: > On Mon, Dec 30, 2013 at 1:35 PM, Robert Micallef <[email protected]> > wrote: > > Yes I restarted the process on both the server and the agent. The agent > is > > set to send the output every 10 seconds (to test). The server is > receiving > > the output of the command as I could see when tailing the archive.log. > Could > > the server not generate alerts in real time? > > > > Anything is possible. Are there other alerts in alerts.log while this > activity is going on? > > > On 30 Dec 2013 18:00, "dan (ddp)" <[email protected]> wrote: > >> > >> On Mon, Dec 30, 2013 at 9:34 AM, Robert Micallef <[email protected]> > >> wrote: > >> > Hi Dan, > >> > > >> > Ok fixed finally. > >> > > >> > I modified the rule to have ossec as decoder not ossec-mem. > >> > > >> > <group name="memory-usage"> > >> > <rule id="100080" level="0"> > >> > <decoded_as>ossec</decoded_as> > >> > > >> > <description>Custom Mem Usage Alerts</description> > >> > </rule> > >> > > >> > <rule id="100081" level="7"> > >> > <if_group>memory-usage</if_group> > >> > <extra_data>^7|^8|^9|^100</extra_data> > >> > <description>Test_Mem_Usage</description> > >> > </rule> > >> > > >> > This way the alerts are triggered if over 80%. They are being > triggered > >> > in > >> > ossec-logtest but I can't see them in alerts.log or the WebUI. > >> > > >> > I modified the scripts on the agents to return only a single number > and > >> > then > >> > modified the local_decoder which now looks like this: > >> > > >> > > >> > <decoder name="ossec-mem"> > >> > <parent>ossec</parent> > >> > <prematch offset="after_parent">'mem-usage': </prematch> > >> > <regex offset="after_prematch>^(\d+)%</regex> > >> > <order>extra_data</order> > >> > </decoder> > >> > > >> > In the archives.log I see the following output: > >> > 2013 Dec 30 15:26:28 (m-s-comm1) 10.152.1.227->mem-usage ossec: > output: > >> > 'mem-usage': > >> > 71% > >> > > >> > In ossec-logtest I see the following output: > >> > > >> > **Phase 1: Completed pre-decoding. > >> > full event: 'ossec: output: 'mem-usage': 71%' > >> > hostname: 'm-p-log1' > >> > program_name: '(null)' > >> > log: 'ossec: output: 'mem-usage': 71%' > >> > > >> > **Phase 2: Completed decoding. > >> > decoder: 'ossec' > >> > extra_data: '71' > >> > > >> > **Phase 3: Completed filtering (rules). > >> > Rule id: '100081' > >> > Level: '7' > >> > Description: 'Test_Mem_Usage' > >> > **Alert to be generated. > >> > > >> > I don't get it. The alert should be triggered. Any ideas? > >> > > >> > >> Are you receiving logs that should trigger this alert? Did you restart > >> the OSSEC processes on the server after putting this rule/decoder in > >> place? > >> > >> > Thanks. > >> > > >> > > >> > > >> > On 30 December 2013 14:50, dan (ddp) <[email protected]> wrote: > >> >> > >> >> On Mon, Dec 30, 2013 at 8:13 AM, Robert Micallef < > [email protected]> > >> >> wrote: > >> >> > Hi Dan, > >> >> > > >> >> > Thanks for your help so far. I have tried searching before asking > >> >> > again > >> >> > and > >> >> > as far as I can see this should work. > >> >> > > >> >> > The decoder works. I used ossec-logtest and up to phase 2, the > >> >> > percentage is > >> >> > taken in extra_data > >> >> > > >> >> > <decoder name="ossec-mem"> > >> >> > <parent>ossec</parent> > >> >> > <prematch offset="after_parent">'mem-usage': </prematch> > >> >> > <regex offset="after_prematch>^(\d+.\d+)%</regex> > >> >> > <order>extra_data</order> > >> >> > </decoder> > >> >> > > >> >> > However I cannot get the rule to trigger. Below is the rule I > >> >> > defined. I > >> >> > used /d to test. > >> >> > > >> >> > <group name="memory-usage"> > >> >> > <rule id="100080" level="0"> > >> >> > <decoded_as>ossec-mem</decoded_as> > >> >> > <description>Custom Mem Usage Alerts</description> > >> >> > </rule> > >> >> > > >> >> > <rule id="100081" level="7"> > >> >> > <if_group>memory-usage</if_group> > >> >> > <extra_data>\d</extra_data> > >> >> > >> >> I believe extra_data should be a number (and I don't think the field > >> >> is regex capable). > >> >> > >> >> > <description>Test_Mem_Usage</description> > >> >> > </rule> > >> >> > > >> >> > I also tried this instead of the one above: > >> >> > > >> >> > <rule id="100081" level="7"> > >> >> > <if_sid>100080</if_sid> > >> >> > <extra_data>\d</extra_data> > >> >> > <description>Test_Mem_Usage</description> > >> >> > </rule> > >> >> > > >> >> > I can't figure out why it's not working. > >> >> > > >> >> > Thanks again. > >> >> > > >> >> > > >> >> > On 27 December 2013 16:13, dan (ddp) <[email protected]> wrote: > >> >> >> > >> >> >> On Fri, Dec 27, 2013 at 10:00 AM, Robert Micallef > >> >> >> <[email protected]> > >> >> >> wrote: > >> >> >> > Hi Dan, > >> >> >> > > >> >> >> > Thanks for the feedback. I cannot figure out how to get the > >> >> >> > decoder > >> >> >> > to > >> >> >> > work. > >> >> >> > > >> >> >> > >> >> >> <decoder name="ossec-mem"> > >> >> >> <parent>ossec</parent> > >> >> >> <prematch offset="after_parent">'mem-usage': </prematch> > >> >> >> <regex offset="after_prematch>^(\d+.\d+)%</regex> > >> >> >> <order>extra_data</order> > >> >> >> </decoder> > >> >> >> > >> >> >> With that you should be able to include somethinglike: > >> >> >> <extra_data>^7</extra_data> > >> >> >> in your rule (untested though, so test first). > >> >> >> > >> >> >> > However are you sure that the actual log is being decoded as: > >> >> >> > 'ossec: > >> >> >> > output: 'mem-usage': 79,whatever%' > >> >> >> > > >> >> >> > >> >> >> Yes, I'm sure. You can verify for yourself. > >> >> >> > >> >> >> > I tried modifying the rule as follows: > >> >> >> > > >> >> >> > > >> >> >> > <rule id="100074" level="7" ignore="7200"> > >> >> >> > <if_sid>530</if_sid> > >> >> >> > <match>ossec: output: 'mem-usage':7</match> > >> >> >> > > >> >> >> > >> >> >> Double check your spacing. > >> >> >> > >> >> >> > <description>High Memory Usage</description> > >> >> >> > </rule> > >> >> >> > > >> >> >> > According to ossec-logtest the rule should be triggered, and yet > >> >> >> > it > >> >> >> > isn't. > >> >> >> > > >> >> >> > >> >> >> Did you restart the ossec processes on the server after changing > >> >> >> your > >> >> >> rule? > >> >> >> > >> >> >> > > >> >> >> > On 27 December 2013 14:57, dan (ddp) <[email protected]> wrote: > >> >> >> >> > >> >> >> >> On Fri, Dec 27, 2013 at 8:41 AM, Robert Micallef > >> >> >> >> <[email protected]> > >> >> >> >> wrote: > >> >> >> >> > Hi Dan, > >> >> >> >> > > >> >> >> >> > From archives.log: > >> >> >> >> > > >> >> >> >> > 2013 Dec 27 11:31:01 (m-s-comm1) 10.152.1.227->mem-usage > ossec: > >> >> >> >> > output: > >> >> >> >> > 'mem-usage': > >> >> >> >> > 70.85% > >> >> >> >> > > >> >> >> >> > From alerts.log I see nothing at those timestamps. > >> >> >> >> > > >> >> >> >> > Am I looking at the correct logs? > >> >> >> >> > > >> >> >> >> > >> >> >> >> Yes, archives.log gives you a sample of the log message you are > >> >> >> >> trying > >> >> >> >> to match against. > >> >> >> >> From reading the documentation or looking at the mailing list > >> >> >> >> archives, you can see that there is a header on this log > message. > >> >> >> >> So > >> >> >> >> the log we want to test against is: > >> >> >> >> ossec: output: 'mem-usage':70.85% > >> >> >> >> > >> >> >> >> I don't have ossec available at the moment to copy/paste the > >> >> >> >> whole > >> >> >> >> ossec-logtest output for you, but it's easy enough for you to > >> >> >> >> recreate > >> >> >> >> on your own. The important part I want to look at first is what > >> >> >> >> is > >> >> >> >> predecoded as the "log" field. This is what <match> and <regex> > >> >> >> >> entries will be looking at: > >> >> >> >> > >> >> >> >> log: 'ossec: output: 'mem-usage': 79,whatever%' > >> >> >> >> > >> >> >> >> From that one line we can tell that your regex is not correct, > >> >> >> >> the > >> >> >> >> first character is not a number. > >> >> >> >> > >> >> >> >> You can either adjust your rule to account for this, or create > a > >> >> >> >> decoder to put the % in a field and check against it in your > >> >> >> >> rule. I > >> >> >> >> personally think the decoder option would be easier, but I've > >> >> >> >> written > >> >> >> >> a few in the past. > >> >> >> >> > >> >> >> >> > Thanks. > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > On 27 December 2013 11:13, dan (ddp) <[email protected]> > wrote: > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> On Dec 27, 2013 5:11 AM, "Robert Micallef" > >> >> >> >> >> <[email protected]> > >> >> >> >> >> wrote: > >> >> >> >> >> > > >> >> >> >> >> > Thanks a lot Dan. That worked like a charm. It didn't > cross > >> >> >> >> >> > my > >> >> >> >> >> > mind > >> >> >> >> >> > to > >> >> >> >> >> > grep only the PID. > >> >> >> >> >> > > >> >> >> >> >> > I used the <check_diff /> option and: > >> >> >> >> >> > ps -ef | grep process-name | awk '{ print $2 }' > >> >> >> >> >> > > >> >> >> >> >> > It is working well now. Can you also please tell me what I > >> >> >> >> >> > did > >> >> >> >> >> > wrong > >> >> >> >> >> > with this rule? > >> >> >> >> >> > > >> >> >> >> >> > I created a script to output the Memory Usage. The output > >> >> >> >> >> > will > >> >> >> >> >> > be > >> >> >> >> >> > the > >> >> >> >> >> > percentage used. Ex: 67.5%. I want an alert when it is > over > >> >> >> >> >> > 80%. > >> >> >> >> >> > > >> >> >> >> >> > I have OSSEC running the script with the following: > >> >> >> >> >> > > >> >> >> >> >> > <localfile> > >> >> >> >> >> > <log_format>full_command</log_format> > >> >> >> >> >> > <command>sh /var/ossec/scripts/memusage.sh</command> > >> >> >> >> >> > <alias>mem-usage</alias> > >> >> >> >> >> > </localfile> > >> >> >> >> >> > > >> >> >> >> >> > On the server I created the following rule: > >> >> >> >> >> > > >> >> >> >> >> > <rule id="100074" level="7" ignore="7200"> > >> >> >> >> >> > <if_sid>530</if_sid> > >> >> >> >> >> > <match>ossec: output: 'mem-usage':</match> > >> >> >> >> >> > <regex>^8|^9|^10</regex> > >> >> >> >> >> > <description>High Memory Usage</description> > >> >> >> >> >> > </rule> > >> >> >> >> >> > > >> >> >> >> >> > To test that this is working I then created this rule: > >> >> >> >> >> > > >> >> >> >> >> > <rule id="100075" level="7" ignore="7200"> > >> >> >> >> >> > <if_sid>530</if_sid> > >> >> >> >> >> > <match>ossec: output: 'mem-usage':</match> > >> >> >> >> >> > <regex>^1|^2|^3|^4|^5|^6|^7</regex> > >> >> >> >> >> > <description>Test Memory Usage</description> > >> >> >> >> >> > </rule> > >> >> >> >> >> > > >> >> >> >> >> > I left it running for a few days and I see no alerts. Any > >> >> >> >> >> > idea > >> >> >> >> >> > how > >> >> >> >> >> > to > >> >> >> >> >> > fix this please? > >> >> >> >> >> > > >> >> >> >> >> > >> >> >> >> >> Turn on the log all option on the server and provide us > with a > >> >> >> >> >> sample > >> >> >> >> >> log > >> >> >> >> >> message. > >> >> >> >> >> > >> >> >> >> >> > Thanks. > >> >> >> >> >> > > >> >> >> >> >> > -- > >> >> >> >> >> > > >> >> >> >> >> > --- > >> >> >> >> >> > You received this message because you are subscribed to > the > >> >> >> >> >> > Google > >> >> >> >> >> > Groups "ossec-list" group. > >> >> >> >> >> > To unsubscribe from this group and stop receiving emails > >> >> >> >> >> > from > >> >> >> >> >> > it, > >> >> >> >> >> > send > >> >> >> >> >> > an email to [email protected]. > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > For more options, visit > >> >> >> >> >> > https://groups.google.com/groups/opt_out. > >> >> >> >> >> > >> >> >> >> >> -- > >> >> >> >> >> > >> >> >> >> >> --- > >> >> >> >> >> You received this message because you are subscribed to a > >> >> >> >> >> topic > >> >> >> >> >> in > >> >> >> >> >> the > >> >> >> >> >> Google Groups "ossec-list" group. > >> >> >> >> >> To unsubscribe from this topic, visit > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > https://groups.google.com/d/topic/ossec-list/QeNptAfzGQQ/unsubscribe. > >> >> >> >> >> To unsubscribe from this group and all its topics, send an > >> >> >> >> >> email > >> >> >> >> >> to > >> >> >> >> >> [email protected]. > >> >> >> >> >> > >> >> >> >> >> For more options, visit > >> >> >> >> >> https://groups.google.com/groups/opt_out. > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > -- > >> >> >> >> > > >> >> >> >> > --- > >> >> >> >> > You received this message because you are subscribed to the > >> >> >> >> > Google > >> >> >> >> > Groups > >> >> >> >> > "ossec-list" group. > >> >> >> >> > To unsubscribe from this group and stop receiving emails from > >> >> >> >> > it, > >> >> >> >> > send > >> >> >> >> > an > >> >> >> >> > email to [email protected]. > >> >> >> >> > For more options, visit > >> >> >> >> > https://groups.google.com/groups/opt_out. > >> >> >> >> > >> >> >> >> -- > >> >> >> >> > >> >> >> >> --- > >> >> >> >> You received this message because you are subscribed to a topic > >> >> >> >> in > >> >> >> >> the > >> >> >> >> Google Groups "ossec-list" group. > >> >> >> >> To unsubscribe from this topic, visit > >> >> >> >> > >> >> >> >> > >> >> >> >> > https://groups.google.com/d/topic/ossec-list/QeNptAfzGQQ/unsubscribe. > >> >> >> >> To unsubscribe from this group and all its topics, send an > email > >> >> >> >> to > >> >> >> >> [email protected]. > >> >> >> >> For more options, visit > https://groups.google.com/groups/opt_out. > >> >> >> > > >> >> >> > > >> >> >> > -- > >> >> >> > > >> >> >> > --- > >> >> >> > You received this message because you are subscribed to the > Google > >> >> >> > Groups > >> >> >> > "ossec-list" group. > >> >> >> > To unsubscribe from this group and stop receiving emails from > it, > >> >> >> > send > >> >> >> > an > >> >> >> > email to [email protected]. > >> >> >> > For more options, visit > https://groups.google.com/groups/opt_out. > >> >> >> > >> >> >> -- > >> >> >> > >> >> >> --- > >> >> >> You received this message because you are subscribed to a topic in > >> >> >> the > >> >> >> Google Groups "ossec-list" group. > >> >> >> To unsubscribe from this topic, visit > >> >> >> > >> >> >> > https://groups.google.com/d/topic/ossec-list/QeNptAfzGQQ/unsubscribe. > >> >> >> To unsubscribe from this group and all its topics, send an email > to > >> >> >> [email protected]. > >> >> >> For more options, visit https://groups.google.com/groups/opt_out. > >> >> > > >> >> > > >> >> > -- > >> >> > > >> >> > --- > >> >> > You received this message because you are subscribed to the Google > >> >> > Groups > >> >> > "ossec-list" group. > >> >> > To unsubscribe from this group and stop receiving emails from it, > >> >> > send > >> >> > an > >> >> > email to [email protected]. > >> >> > For more options, visit https://groups.google.com/groups/opt_out. > >> >> > >> >> -- > >> >> > >> >> --- > >> >> You received this message because you are subscribed to a topic in > the > >> >> Google Groups "ossec-list" group. > >> >> To unsubscribe from this topic, visit > >> >> https://groups.google.com/d/topic/ossec-list/QeNptAfzGQQ/unsubscribe > . > >> >> To unsubscribe from this group and all its topics, send an email to > >> >> [email protected]. > >> >> For more options, visit https://groups.google.com/groups/opt_out. > >> > > >> > > >> > -- > >> > > >> > --- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "ossec-list" group. > >> > To unsubscribe from this group and stop receiving emails from it, send > >> > an > >> > email to [email protected]. > >> > For more options, visit https://groups.google.com/groups/opt_out. > >> > >> -- > >> > >> --- > >> You received this message because you are subscribed to a topic in the > >> Google Groups "ossec-list" group. > >> To unsubscribe from this topic, visit > >> https://groups.google.com/d/topic/ossec-list/QeNptAfzGQQ/unsubscribe. > >> To unsubscribe from this group and all its topics, send an email to > >> [email protected]. > >> For more options, visit https://groups.google.com/groups/opt_out. > > > > -- > > > > --- > > You received this message because you are subscribed to the Google Groups > > "ossec-list" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > For more options, visit https://groups.google.com/groups/opt_out. > > -- > > --- > You received this message because you are subscribed to a topic in the > Google Groups "ossec-list" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/ossec-list/QeNptAfzGQQ/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
