Thanks again Jesus! I will definitely share what I come up with and thanks for all your suggestions and bearing with me through this (long) thread :)
Fredrik On Thursday, August 18, 2016 at 12:17:20 PM UTC+2, Jesus Linares wrote: > > Hi Fredik, > > Long time no see!. It is a hot summer here and, as always, playing with > OSSEC ;). > > I don't have time to create all the decoders, but here a template to help > you: > <decoder name="checkpoint"> > <prematch>^redirect \p|^prevent \p|^allow \p|^block \p|^mail System > Alert|\S+ alert Protection Name:</prematch> > <type>firewall</type> > </decoder> > > > <!-- > BLOCK > Jun 2 13:24:13 st4600fw01n1 block <eth6 mail src: 192.168.71.151; dst: > 54.164.78.72; proto: tcp; bytes: 1845; sent_bytes: 749; received_bytes: > 1096; app_id: 1347922162; browse_time: ******; Suppressed logs: 7; > Referrer_self_uid: ******; Referrer_Parent_uid: ******; product: URL > Filtering; service: http; s_port: 51096; product_family: Network; > --> > <decoder name="checkpoint-block"> > <parent>checkpoint</parent> > <prematch>^block \p</prematch> > <regex offset="after_prematch">src: (\S+); dst: (\S+); proto: (\S+); > </regex> > <order>srcip,dstip,protocol</order> > </decoder> > > > <!-- > ALLOW > ... > --> > > > <!-- > ALERT > May 31 08:05:18 > st4600fw01n1 alert Protection Name: TCP Urgent Data > Enforcement; Severity: 0; Confidence Level: 0; protection_id: > tcp_block_urg_bit_enable; SmartDefense Profile: Recommended_Protection; > Performance Impact: 0; Protection Type: settings; Attack Info: TCP segment > with urgent pointer (no data). Urgent data indication was stripped. Please > refer to sk36869.; attack: Streaming Engine: TCP Urgent Data Enforcement; > Total logs: 8; Suppressed logs: 7; proto: tcp; dst: 52.22.193.162; src: > 192.168.10.204; product: SmartDefense; service: http; s_port: 50869; > FollowUp: Not Followed; product_family: Network; > --> > <decoder name="checkpoint-alert"> > <parent>checkpoint</parent> > <prematch>alert Protection Name: </prematch> > <regex offset="after_prematch">(\.+); Severity: (\d+);</regex> > <order>url,status</order> > </decoder> > > > <!-- > REDIRECT > ... > --> > > > <!-- > SYSTEM ALERT > ... > --> > > If you have questions with a specific decoder, just post it here. > > To make useful decoders for the community, you should: > > - Add the version of your checkpoint at the begging of the file. > - Use a standard log format: are you using the default log format of > checkpoint?. If not, you should use it or at least, use the syslog > standard. In this way, the decoders will work for other users. > - Provide log samples for each decoder: I usually paste the log as > comments in the decoders. > > If you don't mind, when you have the decoders working, you could send them > to our ruleset repositoy <https://github.com/wazuh/ossec-rules> and to > ossec-hids <https://github.com/ossec/ossec-hids>. > > Also, you could use plugin decoders > <https://github.com/wazuh/ossec-wazuh/tree/master/src/analysisd/decoders/plugins> > > instead of standard decoders. Plugin decoders ara coded in c and it is > useful with complex logs (like firewalls). > > I hope it helps. > Regards. > > On Wednesday, August 17, 2016 at 2:38:47 PM UTC+2, Fredrik wrote: >> >> Hi Jesus! >> >> >> Hope you have had a nice summer so far :) I'm revisiting this decoder >> with, what I hoped would be, a fresh (rested) pair of eyes ;) >> Unfortunately, I realize I still have trouble sorting this one out in an >> efficient manner. I was hoping I could ask you for a few additional >> pointers especially regarding how to best extract the relevant information >> from all five types of messages (outlined below). My current challenge is >> that the 'catch all' parent decoder and allow-block-child, result in events >> of types redirect/alert/system alert not be parsed completely i.e. >> information like action/src/dst/message not being extracted. I have >> attacked the problem from the few angles I can think of, but have come up >> short :( >> >> Note: All messages except "System Alert" now share a commonality in the >> hostname which is the active node in the cluster, possible values being >> st4600fw01n1 or st4600fw01n2 . Previously (in thread) you showed me how the >> actions could be used in parent decoder <prematch>^redirect \p|^prevent >> \p|^allow \p|^block \p|^mail System Alert|\S+ alert Protection >> Name:</prematch>. >> >> - Should I use the hostname as the matching criteria for my parent >> decoder and group these four events together whilst creating a separate >> decoder for System Alert? >> - How would you then suggest I go about extracting the information >> action/srcip/dstip/url/extra data for the for types of messages in this >> 'group' using child decoders? >> >> BLOCK >> Jun 2 13:24:13 st4600fw01n1 block <eth6 mail src: 192.168.71.151; dst: >> 54.164.78.72; proto: tcp; bytes: 1845; sent_bytes: 749; received_bytes: >> 1096; app_id: 1347922162; browse_time: ******; Suppressed logs: 7; >> Referrer_self_uid: ******; Referrer_Parent_uid: ******; product: URL >> Filtering; service: http; s_port: 51096; product_family: Network; >> >> Jun 2 13:24:54 st4600fw01n1 block <eth6 mail src: 192.168.71.151; dst: >> 54.164.78.72; proto: tcp; bytes: 11122; sent_bytes: 4494; received_bytes: >> 6628; app_id: 1347922162; browse_time: ******; Referrer_self_uid: ******; >> product: URL Filtering; service: http; s_port: 51096; product_family: >> Network; >> >> Jun 2 13:31:57 st4600fw01n1 block <eth6 mail src: 192.168.71.3; dst: >> 152.115.75.210; proto: tcp; appi_name: ******; app_desc: ******; app_id: >> 1875144601; app_category: ******; matched_category: ******; app_properties: >> ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; >> web_client_type: Other: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; >> Touch; rv:11.0) like Gecko; web_server_type: Other: nginx; resource: >> http://adx.adform.net/adx/?rp=3&pv=1&bWNyPWhiX2FkaWQ6MmIwZmQyMDc0OTllYTEmZHByPTQuMjA2NjQmbWt2PWhiX2JpZGRlcjpydWJpY29uJm1rdz12ZWN0dXJhJTJDZmFzdGlnaGV0ZXIlMkNhYiUyQzU1NjkwMzA1ODcmbWlkPTExODY4NQ&bWNyPWhiX2FkaWQ6MWEwNDNmY2MyNjk2MTk4JmRwcj04LjMwODk2OSZta3Y9aGJfYmlkZGVyOnJ1Ymljb24mbWt3PXZlY3R1cmElMkNmYXN0aWdoZXRlciUyQ2FiJTJDNTU2OTAzMDU4NyZtaWQ9MTE4Njg0&callback=_adform_cb_1464866991419_10159400672628005; >> >> proxy_src_ip: 192.168.71.3; product: URL Filtering; service: http; s_port: >> 51311; product_family: Network; >> >> >> ALLOW >> Jun 2 13:50:15 st4600fw01n1 allow <eth6 mail src: 192.168.99.11; dst: >> 107.170.204.55; proto: tcp; appi_name: ******; app_desc: ******; app_id: >> 60520086; app_category: ******; matched_category: ******; app_properties: >> ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; >> app_sig_id: 60520086:4; proxy_src_ip: 192.168.99.11; product: Application >> Control; service: https; s_port: 54159; product_family: Network; >> >> Jun 2 13:59:05 st4600fw01n1 allow <eth1 mail src: 192.168.99.11; dst: >> 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: >> 10063753; app_category: ******; matched_category: ******; app_properties: >> ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; >> web_client_type: Chrome; web_server_type: Microsoft-IIS; app_sig_id: >> 10063753:5; resource: http://www.aliveproxy.com/; proxy_src_ip: >> 192.168.99.11; product: Application Control; service: http; s_port: 54473; >> product_family: Network; >> >> >> ALERT >> May 31 08:05:18 > st4600fw01n1 alert Protection Name: TCP Urgent Data >> Enforcement; Severity: 0; Confidence Level: 0; protection_id: >> tcp_block_urg_bit_enable; SmartDefense Profile: Recommended_Protection; >> Performance Impact: 0; Protection Type: settings; Attack Info: TCP segment >> with urgent pointer (no data). Urgent data indication was stripped. Please >> refer to sk36869.; attack: Streaming Engine: TCP Urgent Data Enforcement; >> Total logs: 8; Suppressed logs: 7; proto: tcp; dst: 52.22.193.162; src: >> 192.168.10.204; product: SmartDefense; service: http; s_port: 50869; >> FollowUp: Not Followed; product_family: Network; >> >> May 31 17:51:04 > st4600fw01n1 alert Protection Name: TCP Urgent Data >> Enforcement; Severity: 0; Confidence Level: 0; protection_id: >> tcp_block_urg_bit_enable; SmartDefense Profile: Recommended_Protection; >> Performance Impact: 0; Protection Type: settings; rule: 21; rule_uid: >> {3F67BCCB-8087-4974-95FA-F4A4FF466D49}; rule_name: #name; Attack Info: TCP >> segment with urgent pointer (no data). Urgent data indication was stripped. >> Please refer to sk36869.; attack: Streaming Engine: TCP Urgent Data >> Enforcement; Total logs: 24; Suppressed logs: 23; proto: tcp; dst: >> 54.239.168.11; src: 192.168.10.204; product: SmartDefense; service: http; >> s_port: 60324; FollowUp: Not Followed; product_family: Network; >> >> Aug 17 12:37:14 > st4600fw01n1 alert Protection Name: TCP Urgent Data >> Enforcement; Severity: 0; Confidence Level: 0; protection_id: >> tcp_block_urg_bit_enable; SmartDefense Profile: Recommended_Protection; >> Performance Impact: 0; Protection Type: settings; rule: 23; rule_uid: >> {3F67BCCB-8087-4974-95FA-F4A4FF466D49}; rule_name: #name; Attack Info: TCP >> segment with urgent pointer (no data). Urgent data indication was stripped. >> Please refer to sk36869.; attack: Streaming Engine: TCP Urgent Data >> Enforcement; Total logs: 11; Suppressed logs: 10; proto: tcp; dst: >> 80.251.201.102; src: 172.18.46.230; product: SmartDefense; service: https; >> s_port: 57991; FollowUp: Not Followed; product_family: Network; >> >> Aug 17 04:33:21 > st4600fw01n1 alert Protection Name: Packet Sanity; >> Severity: 2; Confidence Level: 5; protection_id: PacketSanity; SmartDefense >> Profile: Recommended_Protection; Performance Impact: 1; Industry Reference: >> CAN-2002-1071; Protection Type: anomaly; Attack Info: Invalid TCP packet - >> source / destination port 0; attack: Malformed Packet; Total logs: 3; >> Suppressed logs: 2; proto: tcp; dst: 80.169.184.240; src: 185.65.132.121; >> product: SmartDefense; FollowUp: Not Followed; product_family: Network; >> >> >> REDIRECT >> Jun 2 13:54:09 st4600fw01n1 redirect <eth1 alert web_client_type: >> Chrome; resource: >> http://sc1.checkpoint.com/za/images/threatwiki/pages/TestAntiBotBlade.html; >> src: 192.168.99.11; dst: 172.226.217.148; proto: tcp; session_id: >> {0x57501e61,0x1001b,0xc50d2e0a,0xc0000000}; Protection name: Check Point - >> Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence >> Level: 5; severity: 2; malware_action: Communication with C&C site; >> rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL >> reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; >> protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.99.11; scope: >> 192.168.99.11; product: Anti Malware; service: http; s_port: 54402; >> >> SYSTEM ALERT >> May 31 21:31:58 sto-fwm03 mail System Alert message: A Firewall Policy >> has been successfully installed on st4600fw01n1; Object: st4600fw01n1; >> Event: Change; Parameter: policy_time; Condition: changes Tue May 31 >> 14:44:13 2016; Current value: Tue May 31 21:04:34 2016; product: System >> Monitor; product_family: Network; >> >> I realize I'm pushing my luck here! You have done more than enough to set >> me off in the right direction, and it might just be that I should work up >> my skills on simpler messages. However, hoping that the concepts you share >> on creating parent/child decoders is useful to the rest of the community. >> >> Best regards, >> Fredrik >> >> >> On Friday, April 15, 2016 at 6:28:22 PM UTC+2, Jesus Linares wrote: >>> >>> Hi Fredrik, >>> >>> It is good progress. You can capture all events with: >>> >>> <decoder name="Checkpoint-test"> >>> <prematch>^redirect \p|^prevent \p|^allow \p|^block \p|^mail System >>> Alert|\S+ alert Protection Name:</prematch> >>> <type>firewall</type> >>> </decoder> >>> >>> >>> I know... It is not very elegant, but it controls all your events. Also, >>> you can add a tag in the beginning of the log (by the firewall settings or >>> with *rsyslog*) and the decoder will be vey easy: >>> >>> Logs: >>> *FredikFirewall *Apr 15 12:23:16 > st4600fw01n1 alert Protection Name: >>> TCP Urgent Data Enforcement; Severity: 0; Confidence Level: 0; >>> protection_id: tcp_block_urg_bit_enable; SmartDefense Profile: >>> Recommended_Protection; Performance Impact: 0; Protection Type: settings >>> ; rule: 20; rule_uid: {3F67BCCB-8087-4974-95FA-F4A4FF466D49}; rule_name: >>> #sample; >>> Attack Info: TCP segment with urgent pointer (no data). Urgent data >>> indication was stripped. Please refer to sk36869.; attack: Streaming >>> Engine: TCP Urgent Data Enforcement; Total logs: 3; Suppressed logs: 2; >>> proto: tcp; dst: 104.16.65.50; src: 192.168.10.204; product: SmartDefense; >>> service: https; s_port: 56814; FollowUp: Not Followed; product_family: >>> Network; >>> >>> >>> >>> Decoder: >>> <decoder name="Checkpoint-test"> >>> <prematch>^FredikFirewall </prematch> >>> <type>firewall</type> >>> </decoder> >>> >>> >>> Regards, >>> Jesus Linares. >>> >>> >>> >>> On Friday, April 15, 2016 at 3:47:17 PM UTC+2, Fredrik wrote: >>>> >>>> Hello Jesus! >>>> >>>> >>>> Story continues. Just wanted to let you know that I have been able, >>>> with help, to unify ALL the messages for easier handling in OSSEC. Thing >>>> is >>>> now that the hostname is extracted automagically (by OSSEC) from the >>>> message and I guess can't be used for my prematch, or? Ossec-logtest will >>>> treat the hostname as part of the header and start the 'Log:' section with >>>> e.g. >>>> >>>> block <eth6 mail src: 10.46.7.196; dst: 37.157.4.16; protocol ... >>>> >>>> How would you tackle this? Right a prematch with all operative words >>>> (actions) that is used with the messages I'm interested in (e.g. >>>> ^allow|^block|^prevent|^redirect)? In my scenario this shouldn't conflict >>>> with other type of messages. I'm guessing that you Ossec-pros will have >>>> options and better alternative though ;) I would also like to match the >>>> decoder regardless of which node in the firewall cluster is the source of >>>> the event? I The two possibilities are st4600fw01n1 and st4600fw01n2 . >>>> >>>> Here are more message samples: >>>> >>>> pr 15 14:41:53 st4600fw01n1 allow <eth6 mail src: 192.168.5.133; dst: >>>> 216.131.91.92; proto: tcp; appi_name: ******; app_desc: ******; app_id: >>>> 60461422; app_category: ******; matched_category: ******; app_properties: >>>> ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; >>>> web_client_type: Chrome; web_server_type: Apache; app_sig_id: 60461422:1; >>>> resource: >>>> http://strongvpn.com/difference_between_proxy_and_vpn.html?utm_source=adwords&utm_medium=sem&gclid=Cj0KEQjwosK4BRCYhsngx4_SybcBEiQAowaCJTFp6qNVmL7E-BhfeTkQouJTwpHN5v1wslK79jD62k4aAqBB8P8HAQ; >>>> >>>> proxy_src_ip: 192.168.5.133; product: Application Control; service: http; >>>> s_port: 59319; product_family: Network; >>>> >>>> Apr 15 14:21:37 st4600fw01n1 redirect <eth1 alert web_client_type: >>>> Chrome; resource: >>>> http://sc1.checkpoint.com/za/images/threatwiki/pages/TestAntiBotBlade.html; >>>> >>>> src: 192.168.5.133; dst: 184.31.90.152; proto: tcp; session_id: >>>> {0x5710dcd1,0x10002,0xc50d2e0a,0xc0000000}; Protection name: Check Point - >>>> Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence >>>> Level: 5; severity: 2; malware_action: Communication with C&C site; >>>> rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL >>>> reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; >>>> protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.5.133; scope: >>>> 192.168.5.133; product: Anti Malware; service: http; s_port: 57878; >>>> >>>> Apr 15 05:35:51 st4600fw01n1 prevent <eth6 alert src: 82.221.102.34; >>>> dst: 192.168.99.4; proto: tcp; session_id: >>>> {0x57106197,0x10003,0xc50d2e0a,0xc0000001}; Protection name: >>>> Trojan.Win32.HackerDefender.C; malware_family: HackerDefender; Source OS: >>>> Solaris; Confidence Level: 5; severity: 4; malware_action: Malicious >>>> network activity; rule_uid: {25157EEE-C09C-4FE0-A872-E0A1486526B8}; >>>> rule_name: #extweb; Protection Type: protection; malware_rule_id: >>>> {000000CE-00A4-0046-9658-621EA5468654}; protection_id: 000043FBC; log_id: >>>> 2; scope: 192.168.99.4; product: Anti Malware; service: http; s_port: >>>> 49228; >>>> >>>> Apr 15 14:13:17 st4600fw01n1 block <eth6 mail src: 192.168.7.196; dst: >>>> 37.157.2.24; proto: tcp; appi_name: ******; app_desc: ******; app_id: >>>> 1875144601; app_category: ******; matched_category: ******; >>>> app_properties: >>>> ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; >>>> web_client_type: Chrome; web_server_type: Other: nginx; resource: >>>> http://adx.adform.net/adx/?rp=3&pv=1&bWlkPTk3ODMyJmN0dXJsPWh0dHA6Ly9mdXNpb24uYm9ubmllcnRpZHNrcmlmdGVyLnNlL2V2ZW50L2ZsaXlmL2J0LmVrb25vbWkucGFmLnJvcy8xNDc3OTkyNzcvY2xpY2s_dXJsPQ&bWlkPTk3ODMxJmN0dXJsPWh0dHA6Ly9mdXNpb24uYm9ubmllcnRpZHNrcmlmdGVyLnNlL2V2ZW50L2V0YnN3L2J0LmVrb25vbWkucGFmLnJvcy8xNDc3OTkyNzAvY2xpY2s_dXJsPQ&bWlkPTk3ODI5JmN0dXJsPWh0dHA6Ly9mdXNpb24uYm9ubmllcnRpZHNrcmlmdGVyLnNlL2V2ZW50L3FmdWh5L2J0LmVrb25vbWkucGFmLnJvcy8xNDc3OTkyNTYvY2xpY2s_dXJsPQ&callback=_adform_cb_1460722287088_3438587873323349; >>>> >>>> proxy_src_ip: 192.168.7.196; product: URL Filtering; service: http; >>>> s_port: >>>> 51190; product_family: Network; >>>> >>>> Apr 15 11:16:05 st4600fw01n1 block <eth6 mail src: 192.168.8.67; dst: >>>> 64.207.139.185; proto: tcp; appi_name: ******; app_desc: ******; app_id: >>>> 3723664659; app_category: ******; matched_category: ******; >>>> app_properties: >>>> ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; >>>> web_client_type: Chrome; web_server_type: Apache; resource: >>>> http://cdn.wibiya.com/Toolbars/dir_0650/Toolbar_650079/Loader_650079.js; >>>> proxy_src_ip: 192.168.8.67; product: URL Filtering; service: http; s_port: >>>> 61907; product_family: Network; >>>> >>>> The two outliers now are the messages below. Not quite sure how to >>>> handle them, but two additional decoders seem required, At least I'm down >>>> to two outliers and not a whole bunch of exceptions as previously :) :) >>>> What would be your take on how to treat these two? >>>> >>>> Mar 7 13:07:53 sto-fwm03 mail System Alert message: A Firewall Policy >>>> has been successfully installed on st4600fw01n1; Object: st4600fw01n1; >>>> Event: Change; Parameter: policy_time; Condition: changes Mon Mar 7 >>>> 13:03:42 2016; Current value: Mon Mar 7 13:08:48 2016; product: Test >>>> Monitor; product_family: Network; >>>> >>>> Apr 15 12:23:16 > st4600fw01n1 alert Protection Name: TCP Urgent Data >>>> Enforcement; Severity: 0; Confidence Level: 0; protection_id: >>>> tcp_block_urg_bit_enable; SmartDefense Profile: Recommended_Protection; >>>> Performance Impact: 0; Protection Type: settings; rule: 20; rule_uid: >>>> {3F67BCCB-8087-4974-95FA-F4A4FF466D49}; rule_name: #sample; Attack Info: >>>> TCP segment with urgent pointer (no data). Urgent data indication was >>>> stripped. Please refer to sk36869.; attack: Streaming Engine: TCP Urgent >>>> Data Enforcement; Total logs: 3; Suppressed logs: 2; proto: tcp; dst: >>>> 104.16.65.50; src: 192.168.10.204; product: SmartDefense; service: https; >>>> s_port: 56814; FollowUp: Not Followed; product_family: Network; >>>> >>>> Best regards, >>>> Fredrik >>>> >>>> >>>> On Friday, April 1, 2016 at 1:18:17 PM UTC+2, Jesus Linares wrote: >>>> >>>> Hi Fredrik, >>>> >>>> here an example of decoding allow/block events (with the option >>>> *after_regex*): >>>> >>>> >>>> <!-- >>>> pattern: >>>> Mar 29 09:04:21 127.0.0.1 Mar 29 8:52:29 st4600fw01n1 Text >>>> --> >>>> <decoder name="Checkpoint-test"> >>>> <prematch>^\w\w\w \d+ \d+:\d+:\d+ \S+ </prematch> >>>> <type>firewall</type> >>>> </decoder> >>>> >>>> >>>> <!-- >>>> Mar 30 09:03:56 127.0.0.1 Mar 30 8:52:05 st4600fw01n1 allow <eth6 mail >>>> src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; bytes: 2396; >>>> sent_bytes: >>>> 1180; received_bytes: 1216; app_id: 10063753; browse_time: ******; >>>> Suppressed logs: 19; Referrer_self_uid: ******; product: Application >>>> Control; service: http; s_port: 64136; product_family: Network; >>>> --> >>>> <decoder name="Checkpoint-block-allow"> >>>> <parent>Checkpoint-test</parent> >>>> <prematch offset="after_parent">^block|^allow</prematch> >>>> <regex offset="after_parent">(\w+) \S+ \S+ src: (\d+.\d+.\d+.\d+); >>>> dst: (\d+.\d+.\d+.\d+)</regex> >>>> <order>action,srcip,dstip</order> >>>> </decoder> >>>> >>>> >>>> <!-- >>>> Checkpoint-block-allow: extra fields: resource and product >>>> Mar 30 09:03:35 127.0.0.1 Mar 30 8:51:43 st4600fw01n1 allow <eth6 mail >>>> src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; appi_name: ******; >>>> app_desc: ******; app_id: 10063753; app_category: ******; >>>> matched_category: >>>> ******; app_properties: ******; app_risk: ******; app_rule_id: ******; >>>> app_rule_name: ******; web_client_type: Chrome; web_server_type: >>>> Microsoft-IIS; app_sig_id: 10063753:5; resource: >>>> http://www.aliveproxy.com/; proxy_src_ip: 192.168.5.133; product: >>>> Application Control; service: http; s_port: 64136; product_family: Network; >>>> --> >>>> <decoder name="Checkpoint-block-allow"> >>>> <parent>Checkpoint-test</parent> >>>> <regex offset="after_regex">resource: (\S+); proxy_src_ip: \S+; >>>> product: (\.+); </regex> >>>> <order>url, extra_data</order> >>>> </decoder> >>>> >>>> >>>> I recommend you configure all your checkpoint devices with the same log >>>> format. If you can't you could use *several parents*: >>>> >>>> <!-- >>>> pattern: >>>> Mar 29 09:04:21 127.0.0.1 Mar 29 8:52:29 st4600fw01n1 Text >>>> --> >>>> <decoder name="Checkpoint-test"> >>>> <prematch>^\w\w\w \d+ \d+:\d+:\d+ \S+ </prematch> >>>> <type>firewall</type> >>>> </decoder> >>>> >>>> >>>> <!-- >>>> pattern: >>>> Mar 30 10:04:39 127.0.0.1 redirect <eth1 alert web_client_type: Chrome; >>>> resource: >>>> http://sc1.checkpoint.com/za/images/threatwiki/pages/TestAntiBotBlade.html; >>>> >>>> src: 192.168.5.133; dst: 172.226.217.148; proto: tcp; session_id: >>>> {0x56fb8896,0x10009,0xc50d2e0a,0xc0000001}; Protection name: Check Point - >>>> Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence >>>> Level: 5; severity: 2; malware_action: Communication with C&C site; >>>> rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL >>>> reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; >>>> protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.5.133; scope: >>>> 192.168.5.133; product: Anti Malware; service: http; s_port: 49244; >>>> >>>> >>>> Mar 29 20:13:14 127.0.0.1 prevent <eth1 alert web_client_type: >>>> Microsoft IE; resource: >>>> http://www.bing.com/fd/ls/GLinkPing.aspx?IG=9A0044152B65437D93F87086B9E730D9&&ID=SERP,5118.1&url=http://sc1.checkpoint.com/z...; >>>> >>>> src: 192.168.5.133; dst: 204.79.197.200; proto: tcp; session_id: >>>> {0x56fac5ba,0x10004,0xc50d2e0a,0xc0000000}; Protection name: Check Point - >>>> Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence >>>> Level: 5; severity: 2; malware_action: Communication with C&C site; >>>> rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL >>>> reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; >>>> protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.5.133; scope: >>>> 192.168.5.133; product: Anti Malware; service: http; s_port: 63119; >>>> --> >>>> <decoder name="Checkpoint-test"> >>>> <prematch>^redirect \p|^prevent \p</prematch> >>>> <type>firewall</type> >>>> </decoder> >>>> >>>> >>>> <!-- >>>> Mar 30 09:03:56 127.0.0.1 Mar 30 8:52:05 st4600fw01n1 allow <eth6 mail >>>> src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; bytes: 2396; >>>> sent_bytes: >>>> 1180; received_bytes: 1216; app_id: 10063753; browse_time: ******; >>>> Suppressed logs: 19; Referrer_self_uid: ******; product: Application >>>> Control; service: http; s_port: 64136; product_family: Network; >>>> --> >>>> <decoder name="Checkpoint-block-allow"> >>>> <parent>Checkpoint-test</parent> >>>> <prematch offset="after_parent">^block|^allow</prematch> >>>> <regex offset="after_parent">(\w+) \S+ \S+ src: (\d+.\d+.\d+.\d+); >>>> dst: (\d+.\d+.\d+.\d+)</regex> >>>> <order>action,srcip,dstip</order> >>>> </decoder> >>>> >>>> >>>> <!-- >>>> Checkpoint-block-allow: extra fields: resource and product >>>> Mar 30 09:03:35 127.0.0.1 Mar 30 8:51:43 st4600fw01n1 allow <eth6 mail >>>> src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; appi_name: ******; >>>> app_desc: ******; app_id: 10063753; app_category: ******; >>>> matched_category: >>>> ******; app_properties: ******; app_risk: ******; app_rule_id: ******; >>>> app_rule_name: ******; web_client_type: Chrome; web_server_type: >>>> Microsoft-IIS; app_sig_id: 10063753:5; resource: >>>> http://www.aliveproxy.com/; proxy_src_ip: 192.168.5.133; product: >>>> Application Control; service: http; s_port: 64136; product_family: Network; >>>> --> >>>> <decoder name="Checkpoint-block-allow"> >>>> <parent>Checkpoint-test</parent> >>>> <regex offset="after_regex">resource: (\S+); proxy_src_ip: \S+; >>>> product: (\.+); </regex> >>>> <order>url, extra_data</order> >>>> </decoder> >>>> >>>> >>>> P.S. My name is Jesus, not Jose ;). >>>> >>>> Regards, >>>> Jesus Linares. >>>> >>>> >>>> >>>> On Wednesday, March 30, 2016 at 10:28:09 AM UTC+2, Fredrik wrote: >>>> >>>> Hi Jose, >>>> >>>> >>>> I got some help to sort out the different timestamps (format) and all >>>> log types now use "Jan 27 09:41:01". You asked about the firewall, >>>> this particular one is a Checkpoint currently running version R77.20. >>>> >>>> The remaining question, that might be of interest to others on the path >>>> to OSSEC mastery ;) ;) is how to handle messages with different "format" >>>> coming from the same host. I have collected a bunch of messages that I >>>> would like to be able to decode, but I'm not sure about the most efficient >>>> way to build the parent/child decoder tree for this. >>>> >>>> With the help received previously in this thread, I currently have the >>>> following in my local_decoder and I'm experimenting with different >>>> addition >>>> - none of which is working so far ;) >>>> >>>> <decoder name="Checkpoint"> >>>> <prematch>^\w+ \d+ \d+:\d+:\d+ st4600fw01n\d</prematch> >>>> <type>firewall</type> >>>> </decoder> >>>> >>>> <decoder name="Checkpoint-alert"> >>>> <parent>Checkpoint</parent> >>>> <regex offset="after_parent">(\w+) \p\w+ \w+ >>>> src:\s(\d+.\d+.\d+.\d+);\sdst:\s(\d+.\d+.\d+.\d+)</regex> >>>> <order>action,srcip,dstip</order> >>>> </decoder> >>>> >>>> <decoder name="Checkpoint-alert"> >>>> <parent>Checkpoint</parent> >>>> <regex offset="after_regex">\.*resource: (\.*);\.*product: >>>> (\.*);</regex> >>>> <order>url,extra_data</order> >>>> </decoder> >>>> >>>> >>>> Below is a collection of syslog messages recieved from the firewall >>>> where the first section is currently decoded using the local_decoder above: >>>> >>>> >>>> Mar 29 10:09:40 127.0.0.1 Mar 29 9:57:49 st4600fw01n1 block <eth6 mail >>>> src: 192.168.7.206; dst: 54.72.9.51; proto: tcp; bytes: 4962; sent_bytes: >>>> 530; received_bytes: 4432; app_id: 3404393449; browse_time: ******; >>>> Suppressed logs: 1; Referrer_self_uid: ******; product: URL Filtering; >>>> service: http; s_port: 54693; product_family: Network; >>>> >>>> Mar 29 20:57:00, st4600fw01n1, allow <eth1 mail src: 192.168.5.133; >>>> dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; >>>> app_id: >>>> 10063753; app_category: ******; matched_category: ******; app_properties: >>>> ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; >>>> web_client_type: Microsoft IE; web_server_type: Microsoft-IIS; app_sig_id: >>>> 10063753:5; resource: http://aliveproxy.com/; proxy_src_ip: >>>> 192.168.5.133; product: Application Control; service: http; s_port: 63867; >>>> product_family: Network; >>>> >>>> Mar 30 09:04:14 127.0.0.1 Mar 30 8:52:22 st4600fw01n1 allow <eth6 mail >>>> src: 192.168.5.133; dst: 23.67.132.180; proto: tcp; appi_name: ******; >>>> app_desc: ******; app_id: 10003219; app_category: ******; >>>> matched_category: >>>> ******; app_properties: ******; app_risk: ******; app_rule_id: ******; >>>> app_rule_name: ******; app_sig_id: 10003219:4; proxy_src_ip: >>>> 192.168.5.133; >>>> product: Application Control; service: https; s_port: 64166; >>>> product_family: Network; >>>> >>>> Mar 30 09:03:56 127.0.0.1 Mar 30 8:52:05 st4600fw01n1 allow <eth6 mail >>>> src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; bytes: 2396; >>>> sent_bytes: >>>> 1180; received_bytes: 1216; app_id: 10063753; browse_time: ******; >>>> Suppressed logs: 19; Referrer_self_uid: ******; product: Application >>>> Control; service: http; s_port: 64136; product_family: Network; >>>> >>>> Mar 30 09:03:35 127.0.0.1 Mar 30 8:51:43 st4600fw01n1 allow <eth6 mail >>>> src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; appi_name: ******; >>>> app_desc: ******; app_id: 10063753; app_category: ******; >>>> matched_category: >>>> ******; app_properties: ******; app_risk: ******; app_rule_id: ******; >>>> app_rule_name: ******; web_client_type: Chrome; web_server_type: >>>> Microsoft-IIS; app_sig_id: 10063753:5; resource: >>>> http://www.aliveproxy.com/; proxy_src_ip: 192.168.5.133; product: >>>> Application Control; service: http; s_port: 64136; product_family: Network; >>>> >>>> Mar 29 09:04:21 127.0.0.1 Mar 29 8:52:29 st4600fw01n1 block <eth6 mail >>>> src: 192.168.5.136; dst: 37.157.4.15; proto: tcp; appi_name: ******; >>>> app_desc: ******; app_id: 1875144601; app_category: ******; >>>> matched_category: ******; app_properties: ******; app_risk: ******; >>>> app_rule_id: ******; app_rule_name: ******; web_client_type: Other: >>>> Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like >>>> Gecko; web_server_type: Other: nginx; resource: >>>> http://adx.adform.net/adx/?rp=3&pv=1&bWlkPTE1NzU3NiZybmQ9ODU1M2JjNGMtMmY2MC00YzVjLWFhMzAtYmY5NzZlNDllZDNk&callback=_adform_cb_1459234242747_7491414591872548; >>>> >>>> proxy_src_ip: 192.168.5.136; product: URL Filtering; service: http; >>>> s_port: >>>> 54051; product_family: Network; >>>> >>>> Mar 29 09:06:12 127.0.0.1 Mar 29 8:54:21 st4600fw01n1 block <eth6 mail >>>> src: 192.168.6.157; dst: 37.157.4.15; proto: tcp; appi_name: ******; >>>> app_desc: ******; app_id: 1875144601; app_category: ******; >>>> matched_category: ******; app_properties: ******; app_risk: ******; >>>> app_rule_id: ******; app_rule_name: ******; web_client_type: Other: >>>> Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like >>>> Gecko; web_server_type: Other: nginx; resource: >>>> http://adx.adform.net/adx/?rp=3&pv=1&bWlkPTk1NzI4&callback=_adform_cb_1459234355177_008705563130792681; >>>> >>>> proxy_src_ip: 192.168.6.157; product: URL Filtering; service: http; >>>> s_port: >>>> 51746; product_family: Network; >>>> >>>> Mar 29 08:37:54 127.0.0.1 Mar 29 8:26:03 st4600fw01n1 block <eth6 mail >>>> src: 192.168.6.157; dst: 152.115.75.210; proto: tcp; appi_name: ******; >>>> app_desc: ******; app_id: 1875144601; app_category: ******; >>>> matched_category: ******; app_properties: ******; app_risk: ******; >>>> app_rule_id: ******; app_rule_name: ******; web_client_type: Other: >>>> Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like >>>> Gecko; web_server_type: Other: nginx; resource: >>>> http://adx.adform.net/adx/?rp=3&pv=1&bWlkPTk3ODMyJmN0dXJsPWh0dHA6Ly9mdXNpb24uYm9ubmllcnRpZHNrcmlmdGVyLnNlL2V2ZW50L3VlZXVqL2J0LmVrb25vbWkucGFmLnJvcy8xNDc3OTkyNzcvY2xpY2s_dXJsPQ&bWlkPTk3ODMxJmN0dXJsPWh0dHA6Ly9mdXNpb24uYm9ubmllcnRpZHNrcmlmdGVyLnNlL2V2ZW50L3lydmd2L2J0LmVrb25vbWkucGFmLnJvcy8xNDc3OTkyNzAvY2xpY2s_dXJsPQ&callback=_adform_cb_1459232656248_7681010355476278; >>>> >>>> proxy_src_ip: 192.168.6.157; product: URL Filtering; service: http; >>>> s_port: >>>> 51104; product_family: Network; >>>> >>>> Mar 29 08:35:17 127.0.0.1 Mar 29 8:23:24 st4600fw01n1 block <eth6 mail >>>> src: 192.168.6.157; dst: 152.115.75.199; proto: tcp; appi_name: ******; >>>> app_desc: ******; app_id: 1875144601; app_category: ******; >>>> matched_category: ******; app_properties: ******; app_risk: ******; >>>> app_rule_id: ******; app_rule_name: ******; web_client_type: Other: >>>> Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like >>>> Gecko; web_server_type: Other: nginx; resource: >>>> http://adx.adform.net/adx/?rp=3&pv=1&bWlkPTk1NzMw&callback=_adform_cb_1459232497393_6046677836175733; >>>> >>>> proxy_src_ip: 192.168.6.157; product: URL Filtering; service: http; >>>> s_port: >>>> 50904; product_family: Network; >>>> >>>> >>>> >>>> >>>> Mar 30 10:04:39 127.0.0.1 redirect <eth1 alert web_client_type: Chrome; >>>> resource: >>>> http://sc1.checkpoint.com/za/images/threatwiki/pages/TestAntiBotBlade.html; >>>> >>>> src: 192.168.5.133; dst: 172.226.217.148; proto: tcp; session_id: >>>> {0x56fb8896,0x10009,0xc50d2e0a,0xc0000001}; Protection name: Check Point - >>>> Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence >>>> Level: 5; severity: 2; malware_action: Communication with C&C site; >>>> rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL >>>> reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; >>>> protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.5.133; scope: >>>> 192.168.5.133; product: Anti Malware; service: http; s_port: 49244; >>>> >>>> Mar 29 20:13:14 127.0.0.1 prevent <eth1 alert web_client_type: >>>> Microsoft IE; resource: >>>> http://www.bing.com/fd/ls/GLinkPing.aspx?IG=9A0044152B65437D93F87086B9E730D9&&ID=SERP,5118.1&url=http://sc1.checkpoint.com/z...; >>>> >>>> src: 192.168.5.133; dst: 204.79.197.200; proto: tcp; session_id: >>>> {0x56fac5ba,0x10004,0xc50d2e0a,0xc0000000}; Protection name: Check Point - >>>> Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence >>>> Level: 5; severity: 2; malware_action: Communication with C&C site; >>>> rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL >>>> reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; >>>> protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.5.133; scope: >>>> 192.168.5.133; product: Anti Malware; service: http; s_port: 63119; >>>> >>>> Mar 30 9:04:59, st4600fw01n1, allow <eth6 mail src: 192.168.5.133; dst: >>>> 207.244.85.73; proto: tcp; appi_name: ******; app_desc: ******; app_id >>>> : 10064017; app_category: ******; matched_category: ******; >>>> app_properties: ******; app_risk: >>>> >>>> ... >>> >>> -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to ossec-list+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
