Hello and thanks for the quick replies. On Fri, Apr 20, 2012 at 6:01 PM, Christopher Moraes <[email protected]>wrote:
> Yes, that reminds me - > > Do you have any "Large" files being scanned with syscheck? I just ran a > test of syscheck monitoring a 10GB file and it took about 30 mins at 50-75% > CPU just to complete. > The problem is not the syscheck process (although when it runs it usually is busy for 12 hours or more - yes, there are alot of big files, about 1.2 TB of data that needs to be checked by syscheck (realtime + scheduled scan) that has not changed). The problem is the analysisd process and ONLY on ossec services restart (i did not try to restart only the analysisd process and see if it does the same - probably it does). > On Fri, Apr 20, 2012 at 10:53 AM, dan (ddp) <[email protected]> wrote: > >> Do you have a lot of custom rules? How active are your servers (events >> per second)? >> Do you have anything that's getting checked with syscheck that should >> be ignored? Something that changes VERY often? My older systems don't >> have syscheck dbs that large. >> > We only use 4 custom rules which just ignore "noise" rsyslog messages that used to generate alot of false alerts. However, we do have on servers with agents some files that keep changing and are monitored, which for some reason fail to be ignored by the auto_ignore setting which is activated on all ossec agents. I guess they change just a bit slower than ossec expects a "fast-changing-logfile" to change. Due to this and some other "normal" alerts, we get many events/day. And no, we can't just ignore them because when they are created random names are being used, so we can't predict that to ignore and what not to. About events/second, ossec-wui reports like this for yesterday: Ossec Stats for: 2012/Apr/19 Total: 67,460 Alerts: 8,296 Syscheck: 24,883 Firewall: 0 Average: 2810.8 events per hour. and for March 2012: Ossec Stats for: 2012/Mar Total: 46,619,212 Alerts: 258,755 Syscheck: 1,004,868 Firewall: 0 As such, yes, there are alot of events and alerts. Hopefully i'll manage to reduce the number, but i don't think i can cut it too much. About analysisd, from what i understand: - why the /var/ossec/queue/syscheck/syscheck file never gets smaller - about 2 weeks ago on a ossec services restart i noticed it had the same size (230-240 MB); since it's in a folder called "queue" i would expect events to be removed from a file once they are processed. - why does the analysisd re-read the whole file - including events/files/folders from months ago or even one year ago - i noticed that on the strace. - why is it random-seeking and reading only in 4K blocks (can the read buffer be tuned?) Thank you for your time. V.
