hi Santhosh,
since your task involves not only matching IP addresses against a blacklist
but also includes reporting IoC information for a bad IP address, I would
recommend loading IoC data from file into a Perl hash which allows for
quick lookups. The example ruleset below uses a global hash %ioc which is a
global variable and can thus be accessed from all rules:
type=Single
ptype=RegExp
pattern=^(SEC_STARTUP|SEC_RESTART|SEC_SOFTRESTART)$
context=SEC_INTERNAL_EVENT
desc=load IoC information on SEC startup and HUP or ABRT signals
action=lcall %count -> ( sub { %ioc = (); \
if (!open(FILE, "/home/risto/IOC_data_proposal.txt")) { return -1; }
\
while (<FILE>) { if (/^\s*(([\d.]+):\d+\s*-.*)/) { \
$ioc{$2} = $1; } \
} close(FILE); return scalar keys %ioc; } ); \
logonly Loaded %count entries from IoC data file
type=Single
ptype=PerlFunc
pattern=sub { if ($_[0] !~ /ASA-\S+: Teardown \S+ connection \d+ for
outside: ([\d.]+)/) { return 0; } \
if (!exists($ioc{$1})) { return 0; } return ($1, $ioc{$1}); }
desc=Connection to IP address $1 with IoC information $2
action=pipe 'Log matched IoC:%{.nl}IoC: $2%{.nl}Log: $0' /bin/mail
some...@example.com
The first rule loads IoC information from file into %ioc hash table
whenever SEC is started or HUP or ABRT signal is received by SEC. IP
addresses serve as keys of the hash table, while each value is an entire
line from the IoC file. For example, if the file contains the following two
lines
187.163.222.244:465 - emotet
187.189.195.208:8443 - emotet
the %ioc hash table will contain the following mappings (keys and values
are separated by ->):
187.163.222.244 -> 187.163.222.244:465 - emotet
187.189.195.208 -> 187.189.195.208:8443 - emotet
Currently, the first rule assumes that IoC file is in the same format as
you described in your e-mail, and the rule uses regular expression
^\s*(([\d.]+):\d+\s*-.*) for parsing the file and extracting relevant
information. Should the format of the file change, this regular expression
needs to be adjusted accordingly. Also, the rule finds the number of
entries loaded from IoC file, stores it in %count action list variable and
logs a debug message with this value into SEC log file. If the rule was
unable to open the file, the value -1 is logged which is useful for
troubleshooting purposes.
The second rule uses PerlFunc pattern for matching incoming ASA firewall
events and first verifies that incoming event matches the regular expression
ASA-\S+: Teardown \S+ connection \d+ for outside: ([\d.]+)
If there is a match, IP address of remote host is extracted and assigned to
$1 variable, and %ioc hash table is looked up for IoC information for that
IP address. If lookup is successful, PerlFunc pattern returns a list with
two elements (IP address, IoC info) which are mapped to match variables $1
and $2 by SEC ($0 variable will hold the entire matching event log line).
The match variables are then used by 'pipe' action for sending an e-mail to
relevant mailbox.
Hope this helps,
risto
Kontakt Santhosh Kumar (<santhoshkmrre...@gmail.com>) kirjutas kuupäeval R,
30. august 2019 kell 06:37:
> Hello
>
> Thanks for the brief..!!
>
> In my case, udgram creates a event loop however tcpsock functionality
> achived the goal.
>
> Regarding second scenario, the goal is to match the IP address and notify.
>
> "ASA-6-302016: Teardown UDP connection 806353 for outside: 187.189.195.208
> <http://187.189.195.208:8443/>/24057 to
> identity: 172.18.124.136/161 duration 0:02:01 bytes 313"
>
> List of IOC's stored in a file(IOC_data_proposal.txt) as below. the rule
> suppose to match the IP address from the file(IOC_data_proposal.txt) though
> the file contains other informations like port number and name of the
> malware, etc. which is not part of the log.
>
> The reason for not removing extra data(port, ioc name) from the file(
> IOC_data_proposal.txt) is that its required for a notification.
>
> ************************
>
> IOC_data_proposal.txt
>
> 187.163.222.244:465 - emotet
>
> 187.189.195.208:8443 - emotet
>
> 188.166.253.46:8080 - emotet
>
> 189.209.217.49:80 <http://189.209.217.49/> - heartbleed
>
> ************************
>
>
> Expected mail Output:
>
>
> Log Matched IOC,
>
> IOC: 187.189.195.208:8443 - emotet
>
> Log: "ASA-6-302016: Teardown UDP connection 806353 for outside:
> 187.189.195.208 <http://187.189.195.208:8443/>/24057 to
>
> identity: 172.18.124.136/161 duration 0:02:01 bytes 313"
>
>
> regards,
> Santhosh
>
> On Wed, Aug 28, 2019 at 4:18 AM Risto Vaarandi <risto.vaara...@gmail.com>
> wrote:
>
>> hi Santhosh,
>>
>> Kontakt Santhosh Kumar (<santhoshkmrre...@gmail.com>) kirjutas kuupäeval
>> T, 27. august 2019 kell 04:55:
>>
>>> Hello Risto
>>>
>>>
>>> I’ve been running tests on SEC for a while and stuck with below points.
>>> I’m not familiar with Perl though I tried to find a solution from sec mail
>>> bucket but no luck, please suggest if this can be achieved with high
>>> performance,
>>>
>>>
>>>
>>> 1. I could see a log drops when I tested with the event rate of
>>> 15000 logs/sec. A simple SEC rule to receive and forward all the logs to
>>> a
>>> destination. The output shows relatively less number of logs. This also
>>> increases the cpu usage from 0.3% to 45%
>>>
>>> ************************
>>>
>>> Type=single
>>>
>>> Ptype=regexp
>>>
>>> Pattern=([.\d]+)
>>>
>>> Desc=$1
>>>
>>> Action=pipe $0 nc syslog101 514
>>>
>>> ************************
>>>
>>
>> The above rule is very inefficient, since 'pipe' action pipes the value
>> of $0 to "nc syslog101 514" command and then closes the pipe, forcing the
>> nc command to terminate. In other words, each time the above rule matches
>> an event, new command is forked, and if you have 15000 events per second,
>> the rule attempts to fork 15000 processes each second. This imposes
>> considerable load on the system and if you send in events at a high rate,
>> the rule might easily exhaust your system resources.
>>
>> Instead of forking nc on each matching event, I would recommend to
>> utilize 'tcpsock' action which transmits events over a single TCP socket
>> (since you haven't used -u flag with nc tool, I assume that your syslog
>> server listens on port 514/tcp). For example, consider the following rule
>> (the rule terminates each transmitted line with the newline):
>>
>> type=single
>> ptype=regexp
>> pattern=([.\d]+)
>> desc=test event $1
>> action=tcpsock syslog101:514 $0%.nl
>>
>> If your syslog server speaks BSD syslog protocol (
>> https://tools.ietf.org/html/rfc3164), and incoming events are not in
>> that format, you could use sec builtin action list variables for formatting
>> the event and providing fields that syslog server expects (such as
>> timestamp). For example, the following rule transmits each event line over
>> TCP in BSD syslog format with priority 14 (facility of "user" and severity
>> of "info"), with hostname "myhost", with program name "myprog", and using
>> newline as a separator between messages:
>>
>> type=single
>> ptype=regexp
>> pattern=([.\d]+)
>> desc=test event $1
>> action=tcpsock syslog101:514 <14>%.monstr %.mdaystr %.hmsstr myhost
>> myprog: $0%.nl
>>
>> Finally, as David suggested, you can also pass messages to local syslog
>> server via /dev/log socket, and let the local syslog server handle the
>> messages (note that unlike for 'tcpsock' in previous example, there is no
>> need for hostname and terminating newline for 'udgram' action):
>>
>> type=single
>> ptype=regexp
>> pattern=([.\d]+)
>> desc=test event $1
>> action=udgram /dev/log <14>%.monstr %.mdaystr %.hmsstr myprog: $0
>>
>> If your local syslog server is rsyslog, you could have the following
>> rsyslog rule for forwarding messages:
>>
>> if $programname == "myprog" then @@syslog101:514
>>
>> As you can see, there are several ways for achieving your goal, and
>> hopefully above examples are helpful for selecting the most convenient
>> solution.
>>
>>
>>>
>>> 1. On a different scenario, I was interested to match the logs with
>>> list of IOC’s. Here i was trying to mail the detected log along with IOC
>>> name. I could achieve it to certain level as mentioned in example but no
>>> luck with this cases, "Split IP's from the IOC file and use it on
>>> the “pattern” to match IP from logs"
>>>
>>> ************************
>>>
>>> IOC_data_proposal.txt
>>>
>>> 187.163.222.244:465 - emotet
>>>
>>> 187.189.195.208:8443 - emotet
>>>
>>> 188.166.253.46:8080 - emotet
>>>
>>> 189.209.217.49:80 - heartbleed
>>>
>>> ************************
>>>
>>> Please check and share some insights.
>>>
>>
>> I am not sure I fully understood what exactly you want to achieve here.
>> Can you provide some examples of input events and what output you would
>> like to generate on each match?
>>
>> kind regards,
>> risto
>>
>>
>>
>>>
>>>
>>>
>>> Eg: I currently tested below case and its working fine as this is a
>>> straight forward IOC matches.
>>>
>>> ************************
>>>
>>> #Current Rule for matching IOC:
>>>
>>> type=Single
>>>
>>> ptype=RegExp
>>>
>>> pattern=(?:SEC_STARTUP|SEC_RESTART|SEC_SOFTRESTART)
>>>
>>> desc=load IOC data
>>>
>>> action=logonly; delete IP; create IP; \
>>>
>>> lcall %iocevents -> (sub{scalar `cat /usr/local/bin/sec-rules/
>>> ioc_data.txt`}); \
>>>
>>> cevent IOC_IP 0 %iocevents;
>>>
>>>
>>>
>>> type=Single
>>>
>>> ptype=RegExp
>>>
>>> pattern=.
>>>
>>> context=IOC_IP
>>>
>>> desc=create an entry
>>>
>>> action=logonly; alias IOC IOC_$0
>>>
>>>
>>>
>>> type=Single
>>>
>>> ptype=regexp
>>>
>>> context=IOC_$2
>>>
>>> pattern= syslog.*hostname=([\w\-\d]+).*IP=([\d\.]+)
>>>
>>> desc=Matched host & ip: $2 && $3
>>>
>>> action=pipe '$0' mail -s ‘%s’ ‘test123.gmail.com’
>>>
>>>
>>>
>>> IOC_data.txt
>>>
>>> 187.163.222.244
>>>
>>> 187.189.195.208
>>>
>>> 188.166.253.46
>>>
>>> 189.209.217.49
>>>
>>> 187.163.222.244
>>>
>>> 187.189.195.208
>>>
>>> 188.166.253.46
>>>
>>> 189.209.217.49
>>>
>>> ************************
>>>
>>>
>>>
>>> Regards,
>>>
>>> san
>>>
>>
>
> --
> Regards,
> SanthoshKumar S
>
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
