Hi all, First of all i want to introduce myself as this is my first mail on this list. My name is Gonzalo Rodrigo and i'm a security engineer on s21sec, a security company based on Spain, and a sec user since december 2008.
My current issue is related with one sec that checks about 65 files, this sec simply sends an email and launch a connection with a remote jira. One month ago it was controlling about 40 files and it was working properly, but when we grow to 65 files i noticed some failures, for wxample, one file exists but the email was not send and the conn with jira was not stablished. I had checked everything that was involved on the process, the script that sends the mail, the connection with the remote server, the mail server itself, and the sec without success, i didn't find anything that could be the reason of this problem. And finally, last thursday, at 12:00 pm, when i was in bed, i had a "divine revelation", ¿Could be that the sec has too much files to check and, for some reason that i cannot understand, it simply discard some of them?" Next day i simply divided on two instances of the same sec, one with 32 files an the order one with 33, one minute later on file was created and the mail and conn were sent. And during this weekend everything was working great, so looks like it was the root of all my problems. The strange thing is that the machine was not heavily loaded after the change, we have an Intel Xeon at 3 Ghz with 12GB of ram and a cabinet connected via fiberchannel to the machine, the total cpu load was about 45% and ram usage was 10 GB (i have other processes running on that machine), and regarding the sec itself, it was at 12% more or less, and 60% of one single core. So, do you think that my problem is solved?, do sec have any limits regarding file handling or something related?, is any flag available to check a big amount of files? Do you think is a better solution available? Thans in advance and regards from Zaragoza (Spain) PD: Here you have the code of the sec, due to security issues i changed the email address and the patterns, mainly i changed every word for a single letter, but how the sec works weren't changed. It checks those files, if one of them is created it send an email and open a conn with the first line, and also creates a temporaly context of one and 12 hours. If we received the same event again, it was stored in one file and after that hour is passed, the file is sent as an attachement, it also do the same with 12 hours. type=Single ptype=RegExp desc=1h_storing_logs_$2 continue=takenext pattern=(.+?) bita.+?: a=(.+)\|\|b=(.+)\|\|c=(.+)\|\|d=(.+)\|\|e=(.+)\|\|f=(.+)\|\|g=(.+)\|\|h=(.+)\|\|i=(.+)\|\|j=(.+)\|\|k=(.+)\|\|(l=/|l=)(.+)\|\|m=(.+)\|\|n=(.+) context= context_1h_$2 && ! context_12h_$2 action= shellcmd (/usr/local/xxxx/sec/bin/sec2jira.pl --a '$3' --b '$4' --c '$5' --d '$9' --e '$10' --f '$11: $12' --g '$6' --h '$8' --i '$7' --j '$2' --k '$14' --l '$15' --m '$16'); write /usr/local/xxxxx/sec/tmp/1h-$2.txt A las $1, $11: $12 type=Single ptype=RegExp desc=1h_storing_logs_$2 continue=takenext pattern=(.+?) bita.+?: a=(.+)\|\|b=(.+)\|\|c=(.+)\|\|d=(.+)\|\|e=(.+)\|\|f=(.+)\|\|g=(.+)\|\|h=(.+)\|\|i=(.+)\|\|j=(.+)\|\|k=(.+)\|\|(l=/|l=)(.+)\|\|m=(.+)\|\|n=(.+) context= ! context_1h_$2 && context_12h_$2 action= shellcmd (/usr/local/xxxx/sec/bin/sec2jira.pl --a '$3' --b '$4' --c '$5' --d '$9' --e '$10' --f '$11: $12' --g '$6' --h '$8' --i '$7' --j '$2' --k '$14' --l '$15' --m '$16'); write /usr/local/xxxxx/sec/tmp/1h-$2.txt A las $1, $11: $12 type=Single ptype=RegExp desc=mail_notification_$2 pattern=(.+?) bita.+?: a=(.+)\|\|b=(.+)\|\|c=(.+)\|\|d=(.+)\|\|e=(.+)\|\|f=(.+)\|\|g=(.+)\|\|h=(.+)\|\|i=(.+)\|\|j=(.+)\|\|k=(.+)\|\|(l=/|l=)(.+)\|\|m=(.+)\|\|n=(.+) context= ! context_1h_$2 && ! context_12h_$2 action = shellcmd (/usr/bin/perl /usr/local/xxxxx/sec/bin/SendHTMLMail.pl 'x...@xxxxx.es' 'alertas-test.x...@xxxx.es,xxxx....@xxxx.es' "bitALERT: $7 - $8 $10" "$11" "$12" "blablabla %t"); shellcmd (/usr/local/xxxx/sec/bin/sec2jira.pl --a '$3' --b '$4' --c '$5' --d '$9' --e '$10' --f '$11: $12' --g '$6' --h '$8' --i '$7' --j '$2' --k '$14' --l '$15' --m '$16'); create context_1h_$2 3600 (spawn (/usr/local/xxxxx/sec/bin/alertaCorreo.sh 'x...@xxxxx.es' 'alertas-test.x...@xxxx.es,xxxx....@xxxx.es' "One hour notification: $7 - $8 $10" "$11" "$12" "blablabla %t" '/usr/local/xxxxx/sec/tmp/1h-$2.txt' 'echoing_12h_$2||$7||$8||$10')) type=Single ptype=RegExp desc=creation_12h_$1 pattern= echoing_12h_(.+?)\|\|(.+?)\|\|(.+?)\|\|(.+) context= ! context_1h_$1 && ! context_12h_$1 action= create context_12h_$1 43200 shellcmd (/usr/local/xxxxx/sec/bin/alertaCorreo.sh 'xx...@yyyyy.es' 'alertas-test.x...@yyyyy.es,xxxxx.zz...@wwwww.es' '12 hours notification' 'alert is still active' 'New alerts detected:' 'Time: %t' '/usr/local/xxxxx/sec/tmp/12h-$1.txt') ------------------------------------------------------------------------------ _______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
