On Fri, Apr 20, 2012 at 11:36 AM, Valentin Avram <[email protected]> wrote: > Hello and thanks for the quick replies. > > On Fri, Apr 20, 2012 at 6:01 PM, Christopher Moraes <[email protected]> > wrote: >> >> Yes, that reminds me - >> >> Do you have any "Large" files being scanned with syscheck? I just ran a >> test of syscheck monitoring a 10GB file and it took about 30 mins at 50-75% >> CPU just to complete. > > > The problem is not the syscheck process (although when it runs it usually is > busy for 12 hours or more - yes, there are alot of big files, about 1.2 TB > of data that needs to be checked by syscheck (realtime + scheduled scan) > that has not changed). > > The problem is the analysisd process and ONLY on ossec services restart (i > did not try to restart only the analysisd process and see if it does the > same - probably it does). > > >> >> On Fri, Apr 20, 2012 at 10:53 AM, dan (ddp) <[email protected]> wrote: >>> >>> Do you have a lot of custom rules? How active are your servers (events >>> per second)? >>> Do you have anything that's getting checked with syscheck that should >>> be ignored? Something that changes VERY often? My older systems don't >>> have syscheck dbs that large. > > > We only use 4 custom rules which just ignore "noise" rsyslog messages that > used to generate alot of false alerts. > > However, we do have on servers with agents some files that keep changing and > are monitored, which for some reason fail to be ignored by the auto_ignore > setting which is activated on all ossec agents. I guess they change just a
I thought auto_ignore was a server side setting. After 3 change notifications you shouldn't get any more for that file. How quickly it changes doesn't matter. > bit slower than ossec expects a "fast-changing-logfile" to change. Due to > this and some other "normal" alerts, we get many events/day. > And no, we can't just ignore them because when they are created random names > are being used, so we can't predict that to ignore and what not to. > > About events/second, ossec-wui reports like this for yesterday: > > Ossec Stats for: 2012/Apr/19 > Total: 67,460 > Alerts: 8,296 > Syscheck: 24,883 > Firewall: 0 > Average: 2810.8 events per hour. > > and for March 2012: > > Ossec Stats for: 2012/Mar > Total: 46,619,212 > Alerts: 258,755 > Syscheck: 1,004,868 > Firewall: 0 > > As such, yes, there are alot of events and alerts. Hopefully i'll manage to > reduce the number, but i don't think i can cut it too much. > > About analysisd, from what i understand: > - why the /var/ossec/queue/syscheck/syscheck file never gets smaller - about > 2 weeks ago on a ossec services restart i noticed it had the same size > (230-240 MB); since it's in a folder called "queue" i would expect events to > be removed from a file once they are processed. The format isn't exactly optimal. The files don't get smaller until you make them smaller (clearing the DB). > - why does the analysisd re-read the whole file - including > events/files/folders from months ago or even one year ago - i noticed that > on the strace. The process isn't exactly optimal. It's something that's being looked into, but I don't think much has been done yet. > - why is it random-seeking and reading only in 4K blocks (can the read > buffer be tuned?) > No idea. > > Thank you for your time. > > V.
