hi Santhosh, Kontakt Santhosh Kumar (<santhoshkmrre...@gmail.com>) kirjutas kuupäeval T, 27. august 2019 kell 04:55:
> Hello Risto > > > I’ve been running tests on SEC for a while and stuck with below points. > I’m not familiar with Perl though I tried to find a solution from sec mail > bucket but no luck, please suggest if this can be achieved with high > performance, > > > > 1. I could see a log drops when I tested with the event rate of 15000 > logs/sec. A simple SEC rule to receive and forward all the logs to a > destination. The output shows relatively less number of logs. This also > increases the cpu usage from 0.3% to 45% > > ************************ > > Type=single > > Ptype=regexp > > Pattern=([.\d]+) > > Desc=$1 > > Action=pipe $0 nc syslog101 514 > > ************************ > The above rule is very inefficient, since 'pipe' action pipes the value of $0 to "nc syslog101 514" command and then closes the pipe, forcing the nc command to terminate. In other words, each time the above rule matches an event, new command is forked, and if you have 15000 events per second, the rule attempts to fork 15000 processes each second. This imposes considerable load on the system and if you send in events at a high rate, the rule might easily exhaust your system resources. Instead of forking nc on each matching event, I would recommend to utilize 'tcpsock' action which transmits events over a single TCP socket (since you haven't used -u flag with nc tool, I assume that your syslog server listens on port 514/tcp). For example, consider the following rule (the rule terminates each transmitted line with the newline): type=single ptype=regexp pattern=([.\d]+) desc=test event $1 action=tcpsock syslog101:514 $0%.nl If your syslog server speaks BSD syslog protocol ( https://tools.ietf.org/html/rfc3164), and incoming events are not in that format, you could use sec builtin action list variables for formatting the event and providing fields that syslog server expects (such as timestamp). For example, the following rule transmits each event line over TCP in BSD syslog format with priority 14 (facility of "user" and severity of "info"), with hostname "myhost", with program name "myprog", and using newline as a separator between messages: type=single ptype=regexp pattern=([.\d]+) desc=test event $1 action=tcpsock syslog101:514 <14>%.monstr %.mdaystr %.hmsstr myhost myprog: $0%.nl Finally, as David suggested, you can also pass messages to local syslog server via /dev/log socket, and let the local syslog server handle the messages (note that unlike for 'tcpsock' in previous example, there is no need for hostname and terminating newline for 'udgram' action): type=single ptype=regexp pattern=([.\d]+) desc=test event $1 action=udgram /dev/log <14>%.monstr %.mdaystr %.hmsstr myprog: $0 If your local syslog server is rsyslog, you could have the following rsyslog rule for forwarding messages: if $programname == "myprog" then @@syslog101:514 As you can see, there are several ways for achieving your goal, and hopefully above examples are helpful for selecting the most convenient solution. > > 1. On a different scenario, I was interested to match the logs with > list of IOC’s. Here i was trying to mail the detected log along with IOC > name. I could achieve it to certain level as mentioned in example but no > luck with this cases, "Split IP's from the IOC file and use it on the > “pattern” to match IP from logs" > > ************************ > > IOC_data_proposal.txt > > 187.163.222.244:465 - emotet > > 187.189.195.208:8443 - emotet > > 188.166.253.46:8080 - emotet > > 189.209.217.49:80 - heartbleed > > ************************ > > Please check and share some insights. > I am not sure I fully understood what exactly you want to achieve here. Can you provide some examples of input events and what output you would like to generate on each match? kind regards, risto > > > > Eg: I currently tested below case and its working fine as this is a > straight forward IOC matches. > > ************************ > > #Current Rule for matching IOC: > > type=Single > > ptype=RegExp > > pattern=(?:SEC_STARTUP|SEC_RESTART|SEC_SOFTRESTART) > > desc=load IOC data > > action=logonly; delete IP; create IP; \ > > lcall %iocevents -> (sub{scalar `cat > /usr/local/bin/sec-rules/ioc_data.txt`}); > \ > > cevent IOC_IP 0 %iocevents; > > > > type=Single > > ptype=RegExp > > pattern=. > > context=IOC_IP > > desc=create an entry > > action=logonly; alias IOC IOC_$0 > > > > type=Single > > ptype=regexp > > context=IOC_$2 > > pattern= syslog.*hostname=([\w\-\d]+).*IP=([\d\.]+) > > desc=Matched host & ip: $2 && $3 > > action=pipe '$0' mail -s ‘%s’ ‘test123.gmail.com’ > > > > IOC_data.txt > > 187.163.222.244 > > 187.189.195.208 > > 188.166.253.46 > > 189.209.217.49 > > 187.163.222.244 > > 187.189.195.208 > > 188.166.253.46 > > 189.209.217.49 > > ************************ > > > > Regards, > > san >
_______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
