Today I've been battling against an issue with logs from IIS, where
username="DOMAIN\ross" and similar, where the "\r" part of that just
happens be an escape sequence.
I can get various different behaviours of this, some of them rather break
the receiving nxlog that this nxlog is sending to (as JSON).
I've tried versions 2.7 and 2.8
Here's some test input logs:
$ cat testcas.log
2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\ross 1.2.3.4
UserAgent 200 0 0 68
2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\neville 1.2.3.4
UserAgent 200 0 0 68
2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\baxter 1.2.3.4
UserAgent 200 0 0 68
2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\anthony 1.2.3.4
UserAgent 200 0 0 68
2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\ROGER 1.2.3.4
UserAgent 200 0 0 68
2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\SAM 1.2.3.5
UserAgent 200 0 0 122
Here is a fairly minimal configuration that demonstrates the issue:
NoCache TRUE
#LogLevel DEBUG
<Extension syslog>
Module xm_syslog
</Extension>
<Extension json>
Module xm_json
</Extension>
# Exchange 2010 CAS IIS Logs
<Extension ms_exchange_2010_cas_iis__csv>
Module xm_csv
#2010Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port
cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status
time-taken
#2013Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port
cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus
sc-win32-status time-taken
Fields $date, $time, $serverip, $HTTPMethod, $URIStem,
$URIQuery, $port, $username, $srcip, $UserAgent, $HTTPStatus,
$SubStatus, $win32Status, $responseTime
FieldTypes string, string, string, string, string,
string, string, string, string, string, string,
string, string, string
# If I set EscapeControl to TRUE, I see that some DOMAIN\ross gets
reported as DOMAIN\ross, but DOMAIN\ROGER get output as DOMAINROGER
(inconsistent, due to interpreted escape sequence)
# If I set EscapeControl to FALSE, Both DOMAIN\ross and DOMAIN\ROGER
get output as DOMAINross and DOMAINROGER
# This has negative interactions with to_json() and to_syslog_ietf()
but is fine with to_syslog_bsd()
EscapeControl FALSE
Delimiter ' '
UndefValue -
# Note: NOT "-", which doesn't match... not sure why.
# Oh, and don't put a comment at the end of the UndefValue line above,
or it won't match either.
</Extension>
<Input ms_exchange_2010_cas_iis__input>
Module im_file
File "/tmp/testcas.log"
ReadFromLast FALSE
SavePos FALSE
#Drop comments from the log file
Exec if $raw_event =~ /^#/ \
{ \
drop(); \
} \
else \
{ \
ms_exchange_2010_cas_iis__csv->parse_csv(); \
$EventTime = parsedate($date + " " + $time); \
to_syslog_ietf(); \
}
</Input>
<Output out>
Module om_file
File "/dev/stdout"
OutputType LineBased
</Output>
<Route 1>
Path ms_exchange_2010_cas_iis__input => out
</Route>
Here are some results testing different combinations of EscapeControl and
each of (to_syslog_ietf, to_json, and to_syslog_bsd) that I get::
EscapeControl = TRUE with to_syslog_ietf(): Actually get an unexpected
(and quite unwelcome) control-character in the output.
EscapeControl = TRUE with to_json(): \r \n and \b are two separate
characters (good), although the ROGER and SAM (and athony, although \a
should be an escape) have the \ missing.
EscapeControl = TRUE with to_syslog_bsd(): All test-cases are expected
(DOMAIN\user)
EscapeControl = FALSE with to_syslog_ietf(): Get DOMAIN\user at the syslog
message page, but username="DOMAINuser" in the [NXLOG@14506 ...] section.
EscapeControl = FALSE with to_json(): Get DOMAINuser (backslash removed)
consistently.
EscapeControl = FALSE with to_syslog_bsd(): Get DOMAIN\user consistently.
Ideally, I'd like to consistently get DOMAIN\user, but with JSON
Is this a bug, or are my expectations faulty (if the latter; I'd appreciate
an explanation, or at least a workaround).
Cheers,
Cameron
--
Cameron Kerr <cameron.kerr...@gmail.com>
See my blog at http://distracted-it.blogspot.co.nz/ (previously
http://humbledown.org/)
Skype me on cameron.kerr.nz
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
nxlog-ce-users mailing list
nxlog-ce-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
