On Fri, Dec 12, 2014 at 11:33 AM, <[email protected]> wrote: > I'm so sorry. This IS 2.8.1 code. My post says v1.7 and 1.8, but I meant v > 2.7 and 2.8. >
I'm not able to reproduce this, could you provide your entire local_rules.xml? > On Friday, December 12, 2014 9:59:34 AM UTC-6, dan (ddpbsd) wrote: >> >> On Fri, Dec 12, 2014 at 10:43 AM, <[email protected]> wrote: >> > I recently migrated our ossec server to new hardware, and at the same >> > time >> > upgraded from v 1.7 to 1.8. Unfortunately ossec-analysisd kept >> > segfaulting >> > in the new environment. It's taken me a couple of weeks to narrow it >> > down, >> > and it seems to be consistently caused by the following rule from my >> > local_rules.xml (downgrade of a base rule from level 7 to level 5): >> > >> > <rule id="533" level="5" overwrite="yes"> >> > <if_sid>530</if_sid> >> > <match>ossec: output: 'netstat -tan</match> >> > <check_diff /> >> > <description>Listened ports status (netstat) changed (new port >> > opened or >> > closed).</description> >> > </rule> >> > >> > This rule has been running under v1.7 in our old environment for over a >> > year. For now, I've just commented out this rule and all is good, but I >> > thought it might be helpful to the community if we figure out what's >> > going >> > on - is it a bug, or some boneheaded misconfiguration on my part. I'm >> > happy >> > to provide more detail if needed. >> > >> > Current environment is Amazon AWS VPC. All servers are running Amazon >> > Linux >> > 64-bit AMI's (based on RHEL/CentOS). >> > [root@smithers ossec]# uname -a >> > Linux smithers 3.14.23-22.44.amzn1.x86_64 #1 SMP Tue Nov 11 23:07:48 UTC >> > 2014 x86_64 x86_64 x86_64 GNU/Linux >> > >> > Initially I tried to migrate everything that was relevant (client.keys, >> > ossec.conf, local_rules.xml, rids files, etc). But with constant >> > crashes, I >> > backed everything out and used the following basic process to narrow >> > down >> > what was happening: >> > >> > 1) I installed v1.8 server, and ran the default install with no agents >> > for >> > about 24 hours. All good. >> > 2) Added one agent (fresh install, new key, no custom rules), ran for 24 >> > hours. All good. >> > 3) Added remaining 7 agents (fresh install, new key, no custom rules), >> > ran >> > for 24 hours. All good. >> > 4) Added my local_rules.xml, analysisd segfaulted within a few minutes. >> > 5) Removed local_rules.xml, ran for 24 hours with no problems. >> > 6) Added local_rules.xml back, analysisd segfaulted after a few minutes. >> > 7) Removed local_rules.xml. Started adding one rule at a time, and >> > running >> > for 6 - 12 hours with each new rule. >> > 8) The above rule seems to consistently be the problem. HOWEVER, since >> > it >> > can take anywhere from a few minutes to a few hours (up to 5) for the >> > segfault to occur, it is possible the above rule is not the problem. >> > >> > I did try running analysisd with gdb to get some more information, but >> > I'm >> > not confident that I got all the ossec processes started correctly when >> > I >> > did it. I can post all the output from that run if someone wants it. >> > >> > Here is an example of the output in /var/log/messages: >> > >> > Dec 9 12:02:03 smithers kernel: [513311.628392] ossec-analysisd[12455]: >> > segfault at 0 ip (null) sp 00007fff152c88f8 error 14 in >> > ossec-analysisd[400000+65000] >> > >> > Following the segfault, these are the ossec processes running (although >> > sometimes syscheckd and/or monitord is not running): >> > >> > [root@smithers ossec]# ps -ef | grep ossec >> > ossecm 12447 1 0 09:58 ? 00:00:00 >> > /var/ossec/bin/ossec-maild >> > root 12451 1 0 09:58 ? 00:00:00 >> > /var/ossec/bin/ossec-execd >> > root 12470 1 0 09:58 ? 00:00:03 >> > /var/ossec/bin/ossec-syscheckd >> > ossec 12473 1 0 09:58 ? 00:00:00 >> > /var/ossec/bin/ossec-monitord >> > root 13327 13302 0 13:12 pts/0 00:00:00 grep ossec >> > >> > Here is the ossec.log from start to finish (this time it took a couple >> > of >> > hours for the segfault to occur): >> > >> > [root@smithers ossec]# tail -f logs/ossec.log >> > 2014/12/09 09:58:28 ossec-testrule: INFO: Reading local decoder file. >> [snip] >> > 2014/12/09 12:28:39 ossec-monitord: socketerr (not available). >> > 2014/12/09 12:28:39 ossec-monitord(1224): ERROR: Error sending message >> > to >> > queue. >> > >> > Let me know if anything else would be helpful. >> > >> >> I'm not seeing this issue on post 2.8.1 code, any chance you can >> upgrade to something remotely recent? >> >> > -- >> > >> > --- >> > You received this message because you are subscribed to the Google >> > Groups >> > "ossec-list" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to [email protected]. >> > For more options, visit https://groups.google.com/d/optout. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "ossec-list" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
