Thanks a lot Dan : it works !
I just changed :
<regex>"\w+ (\S+) HTTP\S+ (\d+) |</regex>
in :
<regex>"\w+ (\.+) HTTP\S+ (\d+) |</regex>
to get the url with spaces inside and I built a rule to catch the url with
"epub" or "mobi" to get an alert when the Calibre Web Server serves contents
for people outside our lan.
Well, may I ask another question :
- These logs lines come from a log file : "server_access_log.txt" that I
monitor with a <location> directive in "ossec.conf"
- There is no "distinctive" part in these logs such as a progname, so it is
very difficult to identify logs lines coming from the calibre-content-server
- It looks like a classic "web-access" log format without the timezone part
- Is it possible to have a decoder specific to a single file ? for exemple :
<localfile>
<log_format>syslog</log_format>
<decoder>specific.decoder.xml</decoder>
<location>/home/calibre/.config/calibre/server_access_log.txt</location>
</localfile>
Thanks again,
Alain
2011/8/13 dan (ddp) <[email protected]>
> Sorry about that. Try modifying web-accesslog to look like the
> following instead (remove the calibre one):
> <decoder name="web-accesslog">
> <type>web-log</type>
> <prematch>^\d+.\d+.\d+.\d+ |^::ffff:\d+.\d+.\d+.\d+ </prematch>
> <regex>^(\d+.\d+.\d+.\d+) \S+ \S+ [\S+ \S\d+] </regex>
> <regex>"\w+ (\S+) HTTP\S+ (\d+) |</regex>
> <regex>^(\S+) \S+ \S+ [\d\d/\S+/\d\d\d\d:\d\d:\d\d:\d\d] "\S+ (\S+)
> HTTP\S+" (\d+) </regex>
> <order>srcip, url, id</order>
> </decoder>
>
>
>
> On Sat, Aug 13, 2011 at 3:55 AM, Alain SPAITE <[email protected]>
> wrote:
> > Thanks dan for your quick answer.
> > I did add your calibre-decoder to local_rules.xml
> >
> > <decoder name="calibre">
> > <parent>web-accesslog</parent>
> > <type>web-log</type>
> > <prematch>^\S+ \S+ \S+ </prematch>
> > <regex>^(\S+) \S+ \S+ [\d\d/\S+/\d\d\d\d:\d\d:\d\d:\d\d] "(\S+) (\S+)
> > HTTP\S+" (\d+) </regex>
> > <order>srcip,action,url,id</order>
> > </decoder>
> >
> > Here is what I got when I test a NCSA formatted log :
> >
> > 83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET /get/epub/331
> HTTP/1.1"
> > 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)"
> >
> > **Phase 1: Completed pre-decoding.
> > full event: '83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET
> > /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+
> > Reader(Android)"'
> > hostname: 'ns1'
> > program_name: '(null)'
> > log: '83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET
> > /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+
> > Reader(Android)"'
> >
> > **Phase 2: Completed decoding.
> > decoder: 'web-accesslog'
> >
> > **Rule debugging:
> > Trying rule: 4 - Generic template for all web rules.
> > *Rule 4 matched.
> > *Trying child rules.
> > Trying rule: 31100 - Access log messages grouped.
> > *Rule 31100 matched.
> > *Trying child rules.
> > Trying rule: 31108 - Ignored URLs (simple queries).
> > Trying rule: 31115 - URL too long. Higher than allowed on most
> browsers.
> > Possible attack.
> > Trying rule: 31103 - SQL injection attempt.
> > Trying rule: 31104 - Common web attack.
> > Trying rule: 31105 - XSS (Cross Site Scripting) attempt.
> > Trying rule: 31101 - Web server 400 error code.
> > Trying rule: 31120 - Web server 500 error code (server error).
> >
> > **Phase 3: Completed filtering (rules).
> > Rule id: '31100'
> > Level: '0'
> > Description: 'Access log messages grouped.'
> >
> > It looks like srcip,id and url are no more decoded on these "classical"
> > logs.
> >
> > Here is what I got when I test a "Calibre" formatted log :
> >
> > 83.233.145.196 - - [10/Jul/2011:22:57:31] "GET /get/epub/331 HTTP/1.1"
> 200
> > 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)"
> >
> >
> > **Phase 1: Completed pre-decoding.
> > full event: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET
> > /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+
> > Reader(Android)"'
> > hostname: 'ns1'
> > program_name: '(null)'
> > log: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET /get/epub/331
> > HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)"'
> >
> > **Phase 2: Completed decoding.
> > decoder: 'web-accesslog'
> > srcip: '83.233.145.196'
> > action: 'GET'
> > url: '/get/epub/331'
> > id: '200'
> >
> > **Rule debugging:
> > Trying rule: 4 - Generic template for all web rules.
> > *Rule 4 matched.
> > *Trying child rules.
> > Trying rule: 31100 - Access log messages grouped.
> > *Rule 31100 matched.
> > *Trying child rules.
> > Trying rule: 31108 - Ignored URLs (simple queries).
> > *Rule 31108 matched.
> >
> > **Phase 3: Completed filtering (rules).
> > Rule id: '31108'
> > Level: '0'
> > Description: 'Ignored URLs (simple queries).'
> >
> > The "calibre" decoder did not fire. But, the srcip, action, url and id
> are
> > extracted from the "calibre" log.
> > I would like :
> > - the NCSA logs decoded like before with the "web-accesslog" decoder with
> > all the relevant info
> > - the "calibre" logs decoded with the "calibre" decoder so I could write
> a
> > rule with <decoded_as>calibre</decoded_as>
> >
> > Thanks a lot for your help.
> >
> >
> > 2011/8/12 dan (ddp) <[email protected]>
> >>
> >> <decoder name="calibre">
> >> <parent>web-accesslog</parent>
> >> <type>web-log</type>
> >> <prematch>^\S+ \S+ \S+ </prematch>
> >> <regex>^(\S+) \S+ \S+ [\d\d/\S+/\d\d\d\d:\d\d:\d\d:\d\d] "(\S+)
> >> (\S+) HTTP\S+" (\d+) </regex>
> >> <order>srcip,action,url,id</order>
> >> </decoder>
> >>
> >>
> >> On Fri, Aug 12, 2011 at 4:21 PM, Alain SPAITE <[email protected]>
> >> wrote:
> >> > Hi everyone,
> >> >
> >> > I'm new to Ossec configuration and I try to check the logs for a
> Calibre
> >> > content server (http://calibre-ebook.com/).
> >> > This content server works on the CherryPy web server written in
> Python.
> >> > The log format does not include the timezone info :
> >> >
> >> >> 83.233.145.196 - - [10/Jul/2011:22:57:31] "GET /get/epub/331
> HTTP/1.1"
> >> >> 200
> >> >> 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)"
> >> >
> >> > instead of the NCSA common log format which would be :
> >> >
> >> >> 83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET /get/epub/331
> >> >> HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)"
> >> >
> >> > Here is what I got with this format with the classic "web-accesslog"
> >> > decoder
> >> > :
> >> >
> >> >> **Phase 1: Completed pre-decoding.
> >> >> full event: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET
> >> >> /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+
> >> >> Reader(Android)"'
> >> >> hostname: 'ns1'
> >> >> program_name: '(null)'
> >> >> log: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET
> >> >> /get/epub/331
> >> >> HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)"'
> >> >>
> >> >> **Phase 2: Completed decoding.
> >> >> decoder: 'web-accesslog'
> >> >>
> >> >> **Rule debugging:
> >> >> Trying rule: 4 - Generic template for all web rules.
> >> >> *Rule 4 matched.
> >> >> *Trying child rules.
> >> >> Trying rule: 31100 - Access log messages grouped.
> >> >> *Rule 31100 matched.
> >> >> *Trying child rules.
> >> >> Trying rule: 31108 - Ignored URLs (simple queries).
> >> >> Trying rule: 31115 - URL too long. Higher than allowed on most
> >> >> browsers. Possible attack.
> >> >> Trying rule: 31103 - SQL injection attempt.
> >> >> Trying rule: 31104 - Common web attack.
> >> >> Trying rule: 31105 - XSS (Cross Site Scripting) attempt.
> >> >> Trying rule: 31101 - Web server 400 error code.
> >> >> Trying rule: 31120 - Web server 500 error code (server error).
> >> >>
> >> >> **Phase 3: Completed filtering (rules).
> >> >> Rule id: '31100'
> >> >> Level: '0'
> >> >> Description: 'Access log messages grouped.'
> >> >>
> >> >
> >> > The souceip,url and id are no extracted.
> >> > Then, I tried to write a new decoder for the Calibre-CherryPy format :
> >> >
> >> >> <decoder name="calibre-accesslog">
> >> >> <type>web-log</type>
> >> >> <parent>web-accesslog</parent>
> >> >> <prematch>^\d+.\d+.\d+.\d+ |^::ffff:\d+.\d+.\d+.\d+ </prematch>
> >> >> <regex>^(\d+.\d+.\d+.\d+) \S+ \S+ [\S+] </regex>
> >> >> <regex>"\w+ (\S+) HTTP\S+ (\d+) </regex>
> >> >> <order>srcip, url, id</order>
> >> >> </decoder>
> >> >
> >> > But it doesn't work and whichever log format I sent (NCSA or CherryPy)
> >> > it is
> >> > always the "web-accesslog" that fired...
> >> > Any help would be welcome !
> >> > You could find more logs samples and results on this pastebin :
> >> > http://pastebin.archlinux.fr/433501
> >> >
> >> > Thanks,
> >> > Alain
> >> >
> >> >
> >
> >
>
