Hi Jason, The im_file module uses LineBased input by default meaning that it will only put a complete line in $raw_event.
In this particular case the error is valid as the input has less fields than expected: > Not enough fields in CSV input, expected 21, got 17 in input ' > MSFTPSVC1 SRV-FTP-01 192.168.34 21 [23463560] SER TestAc - 331 0 0 0 0 FTP > - - - -' You should check your logs to make sure that this is in the file indeed. There will be a StrictMode option in a future release (once that gets merged into CE) which can be used to allow parsing CSV input where the number of fields is less than indicated, i.e. it will treat the missing fields as undef instead of giving the above error. Other cause for such parse errors is improper escaping. For example IIS does not always escape space in the URI and from there on the fields get messed up which could lead to trying to parse an earlier field as integer. The "cannot parse integer FTP" error might be such. A similar question was posted here: https://nxlog.co/question/1507/iis7-w3c-log-parsing-fails HTH, Botond On Fri, 13 May 2016 11:18:43 +1200 Jason Haar <jason_h...@trimble.com> wrote: > Hi there > > I'm using nxlog to feed IIS/FTP data into Graylog via GELF. The input side > involves the parse_csv function. > > What's happening is every few seconds nxlog errors out saying things like > > 2016-05-12 18:29:45 ERROR if-else failed at line 136, character 656 in > C:\Program Files\nxlog\conf\nxlog.conf. statement execution has been > aborted; procedure parse_csv' failed at line 136, character 164 in > C:\Program Files\nxlog\conf\nxlg.conf. statement execution has been > aborted; Not enough fields in CSV input, expected 21, got 17 in input ' > MSFTPSVC1 SRV-FTP-01 192.168.34 21 [23463560] SER TestAc - 331 0 0 0 0 FTP > - - - -' > > or > > cannot parse integer FTP > > To be clear: the parse_csv format is correct, nxlog is working fine - but > at the high rate of activity we have, every few seconds a bad record shows > up (eg 200 successful followed by 1 failure) > > Is the actual problem a Windows "flushing" issue? ie is it that IIS is > sloppy-writing (forgive me - not a programmer) to the logfile and nxlog is > too keen - thinking that once data shows up it must be a complete line when > sometimes it isn't? Could nxlog instead attempt to parse, and if it fails, > sleep for a second - or even just wait for new data to show up - and then > attempt to parse it again? ie are these parse_csv errors a standard issue > facing IIS logs, and could there be a better way of handling them. Of > course, the data could actually be corrupt, but I can't tell from the nxlog > error what line in the IIS file it's occurring on, and at our volumes > certainly can't find it by hand > > This is with nxlog-ce-2.9.1504 > > Thanks! > > -- > Cheers > > Jason Haar > Information Security Manager, Trimble Navigation Ltd. > Phone: +1 408 481 8171 > PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 ------------------------------------------------------------------------------ Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j _______________________________________________ nxlog-ce-users mailing list nxlog-ce-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
