Hi, There is an internal guard built into nxlog so that it does not allow strings larger than 1Mb. This is to avoid potential a DOS in various cases such as this and also to protect agains possible bugs where a string might end up being concatenated in an endless loop causing the process to blow up. In addition, there is another fixed sized buffer of 64K which limits the size of messages on the output side, so anything larger than 64K would be truncated anyway. The latter limitation will be resolved in a future release, but this is also not a typical issue.
Regarding the input size limit, I think this error case can be handled better in the input parser so that it does not propagate the exception to cause the whole input to stop (whole file in this case). On the other hand I'm not sure what's the best way to fix this: * Drop the line and log an error? * Read a truncated line and log an error? The latter can be problematic with some formats where the truncation can lead to other errors, e.g. when the input needs to be parsed as json, xml ,etc. Regards, Botond On Fri, 31 Jan 2014 10:18:08 +0100 Andrei Chiriaev <andrei.chiri...@gmail.com> wrote: > Dear Nxlog community, Hi. Special thanks to the development team that > brought us this exceptional piece of software. > > > I have the following use-case: It might happen that application logging is > miss-implemented and can produce a large output in one single line. For > everything, one liner, greater than 1 mega byte, Nxlog will trigger the > following error message: "ERROR string limit (1048576 bytes) reached". The > problem is that all the files monitored under that input type, "nxlog.conf" > wise, will be drop from the processing queue. That happens silently. > > > My question is whether this is known or not and is that a feature or a bug? > > > Regards, Andrei ------------------------------------------------------------------------------ WatchGuard Dimension instantly turns raw network data into actionable security intelligence. It gives you real-time visual feedback on key security issues and trends. Skip the complicated setup - simply import a virtual appliance and go from zero to informed in seconds. http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk _______________________________________________ nxlog-ce-users mailing list nxlog-ce-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
