Hi,
You can use "EscapeChar \n" as a work-around to turn off escaping.
EscapeChar defaults to the backslash if you do not specify it explicitly.
There are two enhancements are needed:
* Do not swallow the escape character in invalid escape sequences.
* Make it possible to disable escaping.
Regards,
Botond
On Tue, 29 Jul 2014 12:23:05 +1200
Cameron Kerr <cameron.kerr...@gmail.com> wrote:
> Today I've been battling against an issue with logs from IIS, where
> username="DOMAIN\ross" and similar, where the "\r" part of that just
> happens be an escape sequence.
>
> I can get various different behaviours of this, some of them rather break
> the receiving nxlog that this nxlog is sending to (as JSON).
>
> I've tried versions 2.7 and 2.8
>
> Here's some test input logs:
>
> $ cat testcas.log
> 2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\ross 1.2.3.4
> UserAgent 200 0 0 68
> 2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\neville 1.2.3.4
> UserAgent 200 0 0 68
> 2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\baxter 1.2.3.4
> UserAgent 200 0 0 68
> 2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\anthony 1.2.3.4
> UserAgent 200 0 0 68
> 2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\ROGER 1.2.3.4
> UserAgent 200 0 0 68
> 2014-07-27 00:00:02 10.4.15.172 POST /path - 443 DOMAIN\SAM 1.2.3.5
> UserAgent 200 0 0 122
>
> Here is a fairly minimal configuration that demonstrates the issue:
>
> NoCache TRUE
> #LogLevel DEBUG
>
> <Extension syslog>
> Module xm_syslog
> </Extension>
>
> <Extension json>
> Module xm_json
> </Extension>
>
> # Exchange 2010 CAS IIS Logs
> <Extension ms_exchange_2010_cas_iis__csv>
> Module xm_csv
> #2010Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port
> cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status
> time-taken
> #2013Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port
> cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus
> sc-win32-status time-taken
> Fields $date, $time, $serverip, $HTTPMethod, $URIStem,
> $URIQuery, $port, $username, $srcip, $UserAgent, $HTTPStatus,
> $SubStatus, $win32Status, $responseTime
> FieldTypes string, string, string, string, string,
> string, string, string, string, string, string,
> string, string, string
>
> # If I set EscapeControl to TRUE, I see that some DOMAIN\ross gets
> reported as DOMAIN\ross, but DOMAIN\ROGER get output as DOMAINROGER
> (inconsistent, due to interpreted escape sequence)
> # If I set EscapeControl to FALSE, Both DOMAIN\ross and DOMAIN\ROGER
> get output as DOMAINross and DOMAINROGER
> # This has negative interactions with to_json() and to_syslog_ietf()
> but is fine with to_syslog_bsd()
> EscapeControl FALSE
> Delimiter ' '
> UndefValue -
> # Note: NOT "-", which doesn't match... not sure why.
> # Oh, and don't put a comment at the end of the UndefValue line above,
> or it won't match either.
> </Extension>
>
> <Input ms_exchange_2010_cas_iis__input>
> Module im_file
> File "/tmp/testcas.log"
> ReadFromLast FALSE
> SavePos FALSE
>
> #Drop comments from the log file
> Exec if $raw_event =~ /^#/ \
> { \
> drop(); \
> } \
> else \
> { \
> ms_exchange_2010_cas_iis__csv->parse_csv(); \
> $EventTime = parsedate($date + " " + $time); \
> to_syslog_ietf(); \
> }
>
> </Input>
>
> <Output out>
> Module om_file
> File "/dev/stdout"
> OutputType LineBased
> </Output>
>
> <Route 1>
> Path ms_exchange_2010_cas_iis__input => out
> </Route>
>
>
>
> Here are some results testing different combinations of EscapeControl and
> each of (to_syslog_ietf, to_json, and to_syslog_bsd) that I get::
>
> EscapeControl = TRUE with to_syslog_ietf(): Actually get an unexpected
> (and quite unwelcome) control-character in the output.
> EscapeControl = TRUE with to_json(): \r \n and \b are two separate
> characters (good), although the ROGER and SAM (and athony, although \a
> should be an escape) have the \ missing.
> EscapeControl = TRUE with to_syslog_bsd(): All test-cases are expected
> (DOMAIN\user)
>
> EscapeControl = FALSE with to_syslog_ietf(): Get DOMAIN\user at the syslog
> message page, but username="DOMAINuser" in the [NXLOG@14506 ...] section.
> EscapeControl = FALSE with to_json(): Get DOMAINuser (backslash removed)
> consistently.
> EscapeControl = FALSE with to_syslog_bsd(): Get DOMAIN\user consistently.
>
>
> Ideally, I'd like to consistently get DOMAIN\user, but with JSON
>
> Is this a bug, or are my expectations faulty (if the latter; I'd appreciate
> an explanation, or at least a workaround).
>
> Cheers,
> Cameron
>
> --
> Cameron Kerr <cameron.kerr...@gmail.com>
> See my blog at http://distracted-it.blogspot.co.nz/ (previously
> http://humbledown.org/)
> Skype me on cameron.kerr.nz
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
nxlog-ce-users mailing list
nxlog-ce-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
