Hi Yana, Thank you for the reply.
It's not really a 'fresh' installation. I did install it using the system PCRE2, with `PCRE2_SYSTEM=yes ./install.sh` I don't think the issue is with PCRE as such but the fact that the nginx logs are arriving in the syslog, and therefore the decoder regex somehow needs tweaking since the format of the log is not this traditional Nginx log: 11.22.33.44 - - [29/Jun/2021:21:55:30 +0000] "GET / HTTP/1.1" 200 467 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/98 Safari/537.4" ... but instead because it's coming into rsyslog, it looks like this: Jun 29 21:55:30 example.com nginx: 11.22.33.44 - - [29/Jun/2021:21:55:30 +0000] "GET / HTTP/1.1" 200 467 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/98 Safari/537.4" The web-accesslog decoder looks like this, I wonder if it needs overriding somehow to account for the extra fields and maybe that's why the PCRE matches aren't able to 'detect' the status code in the usual way? <decoder name="web-accesslog"> <type>web-log</type> <prematch_pcre2>^\S+ \S+ \S+ \[\S+ \S\d+\] "\w+ \S+ HTTP\S+" </prematch_pcre2> <pcre2>^(\S+) \S+ (\S+) \[\S+ \S\d+\] </pcre2> <pcre2>"(\w+) (\S+) HTTP\S+" (\d+) </pcre2> <order>srcip, srcuser, action, url, id</order> </decoder> I'll see if hopefully anyone else has any tips. Thanks again On Tuesday, June 29, 2021 at 8:26:06 PM UTC+10 [email protected] wrote: > Hi, > > My apologies for the late response. Is your installation a fresh > installation? It seems that from version 3.4, you must have the pcre2-10.32 > sources installed in *src/external. *You can obtain them by running: > > *cd ossec-hids-* * > *wget https://ftp.pcre.org/pub/pcre/pcre2-10.32.tar.gz > <https://ftp.pcre.org/pub/pcre/pcre2-10.32.tar.gz> * > *tar xzf pcre2-10.32.tar.gz -C src/external* > > After this, if you decide to use the pcre2-10.32 sources, you must set the > PCRE"_SYSTEM variable to *no:* > > *cd ossec-hids-* * > *PCRE2_SYSTEM=no ./install.sh* > > If you decide to use the system's PCRE2, set the PCRE2_SYSTEM variable to > *yes:* > > *cd ossec-hids-** > *PCRE2_SYSTEM=yes ./install.sh* > > You can check more information about this here > <http://www.ossec.net/docs/docs/manual/installation/installation-requirements.html> > . > > Hope I was helpful. Let me know if you need anything else. > > Regards, > Yana. > > On Tuesday, June 22, 2021 at 12:00:49 AM UTC+2 [email protected] wrote: > >> Thanks Yana, >> >> With the original 'id_pcre2' in rules 31120 and 31122, and my custom >> decoder per the original post, I get this: >> >> ossec-testrule: Type one log per line. >> >> Jun 21 12:35:37 example.com nginx: 22.33.44.55 - - [21/Jun/2021:12:35:37 >> +0000] "GET /something?bad HTTP/1.1" 500 10372 "https://something.com"; >> "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) >> Chrome/63.0.3239.26 Safari/537.36 Core/1.63.5083.400 QQBrowser/10.0.972.400" >> >> >> **Phase 1: Completed pre-decoding. >> full event: 'Jun 21 12:35:37 example.com nginx: 22.33.44.55 - - >> [21/Jun/2021:12:35:37 +0000] "GET /something?bad HTTP/1.1" 500 10372 " >> https://something.com"; "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 >> (KHTML, like Gecko) Chrome/63.0.3239.26 Safari/537.36 Core/1.63.5083.400 >> QQBrowser/10.0.972.400"' >> hostname: 'example.com' >> program_name: 'nginx' >> log: '22.33.44.55 - - [21/Jun/2021:12:35:37 +0000] "GET >> /something?bad HTTP/1.1" 500 10372 "https://something.com"; "Mozilla/5.0 >> (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.26 >> Safari/537.36 Core/1.63.5083.400 QQBrowser/10.0.972.400"' >> >> **Phase 2: Completed decoding. >> decoder: 'web-accesslog' >> >> **Phase 3: Completed filtering (rules). >> Rule id: '31100' >> Level: '0' >> Description: 'Access log messages grouped.' >> >> >> >> If I change the <id_prce2> to <match> and remove the ^ in the 50/500 >> match string, for rules 31120 and 31122, I get this: >> >> ossec-testrule: Type one log per line. >> >> Jun 21 12:35:37 example.com nginx: 22.33.44.55 - - [21/Jun/2021:12:35:37 >> +0000] "GET /something?bad HTTP/1.1" 500 10372 "https://something.com"; >> "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) >> Chrome/63.0.3239.26 Safari/537.36 Core/1.63.5083.400 QQBrowser/10.0.972.400" >> >> >> **Phase 1: Completed pre-decoding. >> full event: 'Jun 21 12:35:37 example.com nginx: 22.33.44.55 - - >> [21/Jun/2021:12:35:37 +0000] "GET /something?bad HTTP/1.1" 500 10372 " >> https://something.com"; "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 >> (KHTML, like Gecko) Chrome/63.0.3239.26 Safari/537.36 Core/1.63.5083.400 >> QQBrowser/10.0.972.400"' >> hostname: 'example.com' >> program_name: 'nginx' >> log: '22.33.44.55 - - [21/Jun/2021:12:35:37 +0000] "GET >> /something?bad HTTP/1.1" 500 10372 "https://something.com"; "Mozilla/5.0 >> (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.26 >> Safari/537.36 Core/1.63.5083.400 QQBrowser/10.0.972.400"' >> >> **Phase 2: Completed decoding. >> decoder: 'web-accesslog' >> >> **Phase 3: Completed filtering (rules). >> Rule id: '31122' >> Level: '5' >> Description: 'Web server 500 error code (Internal Error).' >> **Alert to be generated. >> >> >> >> Thanks >> >> On Tuesday, June 22, 2021 at 12:55:41 AM UTC+10 [email protected] wrote: >> >>> Hi Miguel, >>> >>> Could you please paste the output coming from *ossec-logtest* after >>> pasting these logs? >>> >>> Waiting for your reply, >>> Yana. >>> >>> On Monday, June 21, 2021 at 12:29:56 PM UTC+2 [email protected] wrote: >>> >>>> Hi, >>>> >>>> I am running a system whereby Nginx traffic logs are being sent from a >>>> Docker container to a remote syslog server, where they arrive in that >>>> remote syslog server's /var/log/syslog. This remote server is also the one >>>> running OSSEC. >>>> >>>> As a result, the Nginx logs look like this in the syslog - note the ' >>>> example.com' is effectively the 'program_name' which is the identifier >>>> of the container itself. >>>> >>>> Jun 20 15:52:09 example.com nginx: 11.22.33.44 - - >>>> [20/Jun/2021:15:52:09 +0000] "GET /something/ HTTP/1.1" 500 7910 " >>>> https://example.com/"; "Mozilla/5.0 (Windows NT 5.1; rv:52.0) >>>> Gecko/20100101 Firefox/52.0" >>>> >>>> My problem is that the OSSEC rules are not recognising the Nginx logs, >>>> because they are in the syslog. >>>> >>>> To 'half' solve that, I added this custom decoder which I borrowed from >>>> https://github.com/wazuh/wazuh/issues/352: >>>> >>>> <decoder name="web-accesslog"> >>>> <type>web-log</type> >>>> <program_name>nginx|apache</program_name> >>>> </decoder> >>>> >>>> Now, this is good because the above example log message will now appear >>>> as rule 31101 'Access log messages grouped'. Progress! >>>> >>>> However, note that the log message was a 500 internal server error. It >>>> seems that despite landing in 31101 thanks to the custom decoder, the >>>> other >>>> 'child' rules in web_rules.xml are not applying, e.g 31122: >>>> >>>> <rule id="31122" level="5"> >>>> <if_sid>31120</if_sid> >>>> <id_pcre2>^500</id_pcre2> >>>> <options>alert_by_email</options> >>>> <description>Web server 500 error code (Internal >>>> Error).</description> >>>> <group>system_error,</group> >>>> </rule> >>>> >>>> It doesn't seem to hit this error, it just stays as 31101 according to >>>> ossec-logtest. >>>> >>>> I am assuming it's the id_pcre2 not picking up the '500' because of the >>>> extra fields when it's from syslog? As a guess? >>>> >>>> If I change both rule 31120 and rule 31122 to use <match>50</match> and >>>> <match>500</match> respectively, then it works, and rule 31122 fires for >>>> the above. But not if it uses id_pcre2 *or* if it uses ^ at the start of >>>> the match - both make it skip. >>>> >>>> I'm not so great at regexes - so I would really appreciate any help to >>>> get the standard web rules detecting the above Nginx log message when it's >>>> coming as a 'syslog' message. >>>> >>>> I am running OSSEC 3.6.0 on Ubuntu 18.04. >>>> >>>> Thanks! >>>> >>> -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/ossec-list/c496d8f7-9318-4ad9-aa41-90f8e42efbb8n%40googlegroups.com.
