|
Hi, I've run several tests on server and Not sure how to understand results. 1. My input looks like this: <Input in10> Module im_file File "/xxx/xxx/xxx/xxx/log/xxx-access_log" InputType LineBased Exec create_stat("unix", "RATE", 1); add_stat("unix", 1); log_info("Unix was added to log"); </Input> 2. In log file I see following lines: 2013-06-14 14:55:37 INFO last message repeated 249 times 2013-06-14 14:55:37 INFO Unix was added to log 2013-06-14 14:55:43 INFO last message repeated 49 times 2013-06-14 14:55:43 INFO Unix was added to log 2013-06-14 14:55:52 INFO last message repeated 449 times 2013-06-14 14:55:52 INFO Unix was added to log 2013-06-14 14:56:08 INFO last message repeated 99 times 2013-06-14 14:56:08 INFO Unix was added to log 2013-06-14 14:56:36 INFO last message repeated 198 times 3. In file I see now log with delay for ~1 hour. 4. How nxlog save log lines before it send them to output? is possible that it read log till cache fills and them send them to output? Each second logs are about 120K - 150K size. On 06/14/2013 12:28 PM, Botond
Botyanszki wrote:
Hi, Regarding 6: If you have lines longer than 65k characters this is normal. It can be fixed by recompiling nxlog with a larger buffer. Otherwise make sure your file has lines properly terminated with LF or CRLF.Regarding the delay, I don't have an idea. If this is not an environment issue and is reproducible on a standard windows/linux box, then please file a bug in the sourceforge trac. I also recommend to debug the issue using the statistical counters with the im_file module to see how it reads messages (e.g. print the EPS every second). Here is a similar example: http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html#idp5586560 Regards, Botond On Thu, 13 Jun 2013 18:55:57 +0300 Maxim Nikolaev <[email protected]> wrote:Hi I'm using nxlog 2.4. I need to read big log file and forward them to logstash. I encountered following problem: 1. Nxlog is configured to read lines from log file with im_file module. 2. It send to logstash with om_ssl output. 3. I see that logs on logstash arrives with large delay - up to 3 hours. 4. i've checked nxlog with om_file module to write log locally and found that it really read log with delay. Log lines that I get in file are in delay for 1-3 hours. 5. Logs that i need to read are big - total about 6GB day. 6. Also I get following error in nxlog.log: ERROR line is over the limit of 65000, will be truncated. Thanks.------------------------------------------------------------------------------ This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev _______________________________________________ nxlog-ce-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users |
------------------------------------------------------------------------------ This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev
_______________________________________________ nxlog-ce-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
