Re: [ossec-list] Re: ossec-analysisd and ossec-logtest take about 3 minutes to start
If anybody else happens to experience this issue, Daniel and I were able to determine that it was due to a rather large fts-queue file (4.4M in my case). Removing the fts-queue file and letting OSSEC create a new file allowed ossec-analysisd and ossec-logtest to start instantly without excessive CPU usage. Daniel is going to work on improving the code that reads the fts-queue file. Regards, Doug Burks http://securityonion.blogspot.com/ On Tue, Mar 9, 2010 at 2:41 PM, Doug Burks mub...@gmail.com wrote: Hi Daniel, Thanks for your response. We're running OSSEC 2.3 on CentOS 5.4. Nothing unusual in ossec.conf or local_rules.xml (I sent you a direct email with a copy of my local rules). We have 33 agents total (24 Windows, 9 Linux). All agents are running 2.3 as well. ossec-logtest is exhibiting the same behavior; would it be affected by agents? Is there any additional logging that I can enable to determine what is taking so much time and CPU? Thanks, Doug Burks On Mar 9, 7:41 am, Daniel Cid daniel@gmail.com wrote: Hi Doug, I have no clue to what might be going on... syscheckd taking long doesn't matter, because it sleeps in the middle to save some CPU. All normal.. For analysisd and log-test to take that long, there must be something in your rules or environment that's causing all that delay. I never had this problem before... What version are you using? Which OS? How many agents pointing to that box? Thanks, -- Daniel B. Cid dcid ( at ) ossec.net On Fri, Mar 5, 2010 at 10:53 AM, Doug Burks mub...@gmail.com wrote: Yes, I saw that the log file showed a 3-minute gap between syscheckd starting and finishing pre-scan. However, ossec-syscheckd is not the process that is taking up 100% CPU. ossec-analysisd takes 100% CPU for 3 minutes. ossec-logtest does the same thing, and I wouldn't expect it to do anything with syscheckd. I've looked at 2 other OSSEC installs and neither of them exhibit this behavior. When starting OSSEC, they do show the standard 3-minute syscheckd gap in the log file, but there is NO process taking 100% CPU for any amount of time. Also, starting ossec-logtest on these other OSSEC installs is instantaneous with no excessive CPU usage. What would cause ossec-analysisd and ossec-logtest to hit 100% CPU usage for 3 minutes? Any ideas, Daniel Cid? Thanks, Doug Burks On Mar 4, 4:02 pm, Joshua Gimer jgi...@gmail.com wrote: On Thu, Mar 4, 2010 at 12:11 PM, Doug Burks mub...@gmail.com wrote: As I mentioned in my previous message, ossec-logtest takes about 3 minutes before it will accept input. During this time, it is stuck at 100% CPU usage. ossec-analysisd does the same thing when starting OSSEC. After the 3 minutes is up, ossec-analysisd settles down to about 30% CPU usage. 2010/03/04 13:59:55 ossec-syscheckd: INFO: Starting syscheck database (pre-scan). 2010/03/04 14:02:41 ossec-syscheckd: INFO: Finished creating syscheck database (pre-scan completed). Is this normal? Thanks, Doug Burks The majority of the time is being spent starting the syscheck database. Google seems to have a few results of OSSEC start logs that show around a 3 minute start as well. -- Thx Joshua Gimer -- Doug Burks, GCIA, GSEC, CISSP http://securityonion.blogspot.com
Re: [ossec-list] Re: ossec-analysisd and ossec-logtest take about 3 minutes to start
I just fixed the code and it is available on the latest snapshot: http://www.ossec.net/files/snapshots/ossec-hids-100311.tar.gz If anyone is having the same problems, please try this version to see if it goes away. thanks, -- Daniel B. Cid dcid ( at ) ossec.net On Thu, Mar 11, 2010 at 7:35 AM, Doug Burks mub...@gmail.com wrote: If anybody else happens to experience this issue, Daniel and I were able to determine that it was due to a rather large fts-queue file (4.4M in my case). Removing the fts-queue file and letting OSSEC create a new file allowed ossec-analysisd and ossec-logtest to start instantly without excessive CPU usage. Daniel is going to work on improving the code that reads the fts-queue file. Regards, Doug Burks http://securityonion.blogspot.com/ On Tue, Mar 9, 2010 at 2:41 PM, Doug Burks mub...@gmail.com wrote: Hi Daniel, Thanks for your response. We're running OSSEC 2.3 on CentOS 5.4. Nothing unusual in ossec.conf or local_rules.xml (I sent you a direct email with a copy of my local rules). We have 33 agents total (24 Windows, 9 Linux). All agents are running 2.3 as well. ossec-logtest is exhibiting the same behavior; would it be affected by agents? Is there any additional logging that I can enable to determine what is taking so much time and CPU? Thanks, Doug Burks On Mar 9, 7:41 am, Daniel Cid daniel@gmail.com wrote: Hi Doug, I have no clue to what might be going on... syscheckd taking long doesn't matter, because it sleeps in the middle to save some CPU. All normal.. For analysisd and log-test to take that long, there must be something in your rules or environment that's causing all that delay. I never had this problem before... What version are you using? Which OS? How many agents pointing to that box? Thanks, -- Daniel B. Cid dcid ( at ) ossec.net On Fri, Mar 5, 2010 at 10:53 AM, Doug Burks mub...@gmail.com wrote: Yes, I saw that the log file showed a 3-minute gap between syscheckd starting and finishing pre-scan. However, ossec-syscheckd is not the process that is taking up 100% CPU. ossec-analysisd takes 100% CPU for 3 minutes. ossec-logtest does the same thing, and I wouldn't expect it to do anything with syscheckd. I've looked at 2 other OSSEC installs and neither of them exhibit this behavior. When starting OSSEC, they do show the standard 3-minute syscheckd gap in the log file, but there is NO process taking 100% CPU for any amount of time. Also, starting ossec-logtest on these other OSSEC installs is instantaneous with no excessive CPU usage. What would cause ossec-analysisd and ossec-logtest to hit 100% CPU usage for 3 minutes? Any ideas, Daniel Cid? Thanks, Doug Burks On Mar 4, 4:02 pm, Joshua Gimer jgi...@gmail.com wrote: On Thu, Mar 4, 2010 at 12:11 PM, Doug Burks mub...@gmail.com wrote: As I mentioned in my previous message, ossec-logtest takes about 3 minutes before it will accept input. During this time, it is stuck at 100% CPU usage. ossec-analysisd does the same thing when starting OSSEC. After the 3 minutes is up, ossec-analysisd settles down to about 30% CPU usage. 2010/03/04 13:59:55 ossec-syscheckd: INFO: Starting syscheck database (pre-scan). 2010/03/04 14:02:41 ossec-syscheckd: INFO: Finished creating syscheck database (pre-scan completed). Is this normal? Thanks, Doug Burks The majority of the time is being spent starting the syscheck database. Google seems to have a few results of OSSEC start logs that show around a 3 minute start as well. -- Thx Joshua Gimer -- Doug Burks, GCIA, GSEC, CISSP http://securityonion.blogspot.com
Re: [ossec-list] Re: ossec-analysisd and ossec-logtest take about 3 minutes to start
Hi Doug, I have no clue to what might be going on... syscheckd taking long doesn't matter, because it sleeps in the middle to save some CPU. All normal.. For analysisd and log-test to take that long, there must be something in your rules or environment that's causing all that delay. I never had this problem before... What version are you using? Which OS? How many agents pointing to that box? Thanks, -- Daniel B. Cid dcid ( at ) ossec.net On Fri, Mar 5, 2010 at 10:53 AM, Doug Burks mub...@gmail.com wrote: Yes, I saw that the log file showed a 3-minute gap between syscheckd starting and finishing pre-scan. However, ossec-syscheckd is not the process that is taking up 100% CPU. ossec-analysisd takes 100% CPU for 3 minutes. ossec-logtest does the same thing, and I wouldn't expect it to do anything with syscheckd. I've looked at 2 other OSSEC installs and neither of them exhibit this behavior. When starting OSSEC, they do show the standard 3-minute syscheckd gap in the log file, but there is NO process taking 100% CPU for any amount of time. Also, starting ossec-logtest on these other OSSEC installs is instantaneous with no excessive CPU usage. What would cause ossec-analysisd and ossec-logtest to hit 100% CPU usage for 3 minutes? Any ideas, Daniel Cid? Thanks, Doug Burks On Mar 4, 4:02 pm, Joshua Gimer jgi...@gmail.com wrote: On Thu, Mar 4, 2010 at 12:11 PM, Doug Burks mub...@gmail.com wrote: As I mentioned in my previous message, ossec-logtest takes about 3 minutes before it will accept input. During this time, it is stuck at 100% CPU usage. ossec-analysisd does the same thing when starting OSSEC. After the 3 minutes is up, ossec-analysisd settles down to about 30% CPU usage. 2010/03/04 13:59:55 ossec-syscheckd: INFO: Starting syscheck database (pre-scan). 2010/03/04 14:02:41 ossec-syscheckd: INFO: Finished creating syscheck database (pre-scan completed). Is this normal? Thanks, Doug Burks The majority of the time is being spent starting the syscheck database. Google seems to have a few results of OSSEC start logs that show around a 3 minute start as well. -- Thx Joshua Gimer
