Hello,
I use Nxlog in version 2.7.1191 and I want to send data using tcp between two
nxlog hosts. I set a simple configuration:
1st host nxlog.conf modules:
<Input in_file>
Module im_file
File "input.txt"
SavePos TRUE
</Input>
<Output out_tcp>
Module om_tcp
Host {dst_addr}
Port 514
</Output>
<Route route1>
Path in_file => out_tcp
</Route>
2nd host nxlog.conf modules:
<Input in_tcp>
Module im_tcp
Host 0.0.0.0
Port 514
</Input>
<Output out_file>
Module om_file
File "output.txt"
</Output>
<Route route2>
Path in_tcp => out_file
</Route>
Normally, everything works fine - I write messages to input.txt, they are sent
over tcp to remote host and saved into output.txt. The problem occurs when
while sending the data connection between hosts is lost.
While sending data I unplugged the network cable and plugged it again.
Comparing input and output files it looks like data which was about to be sent
right before disconnection was lost ( about 7kBs missing ).
In
https://www.mail-archive.com/nxlog-ce-users@lists.sourceforge.net/msg00106.html
you wrote:
"When om_tcp detects that it cannot send, the input module (im_msvistalog) will
stop reading logs. When the tcp connection is resumed, the input module will
resume reading from the same position and will properly collect all messages
that were stored in the windows eventlog during that time. This works the same
way with im_file."
However, it looks like in my case after reconnecting, input module resumed
sending data from the wrong position. The same situation occurs when ssl module
is used.
The problem appears mostly when big amount of data is sent. Maybe in this case
information about connection loss is passed to input module too late, the
position is already shifted, but data not sent. Therefore, after reconnection
it starts from already shifted position.
Here are logs from nxlog.log file ( from DEBUG mode ) that probably describe
what happens in dissconnection moment:
2014-07-10 12:09:33 DEBUG worker 0 got signal for new job
2014-07-10 12:09:33 DEBUG worker 0 processing event 0x863f38
2014-07-10 12:09:33 DEBUG worker 1 waiting for new event
2014-07-10 12:09:33 DEBUG PROCESS_EVENT: POLL (out_nxlog)
2014-07-10 12:09:33 DEBUG nx_module_pollset_poll: out_nxlog
2014-07-10 12:09:33 DEBUG module out_nxlog got 1 poll events
2014-07-10 12:09:33 DEBUG Module out_nxlog can read
2014-07-10 12:09:33 DEBUG Module out_nxlog can write
2014-07-10 12:09:33 DEBUG nx_event_to_jobqueue: READ (out_nxlog)
2014-07-10 12:09:33 DEBUG event added to jobqueue
2014-07-10 12:09:33 DEBUG nx_event_to_jobqueue: WRITE (out_nxlog)
2014-07-10 12:09:33 DEBUG event added to jobqueue
2014-07-10 12:09:33 DEBUG worker 1 got signal for new job
2014-07-10 12:09:33 DEBUG worker 1 processing event 0x3d8720
2014-07-10 12:09:33 DEBUG worker 0 waiting for new event
2014-07-10 12:09:33 DEBUG PROCESS_EVENT: READ (out_nxlog)
2014-07-10 12:09:33 DEBUG om_tcp read
2014-07-10 12:09:33 DEBUG STOP: out_nxlog
2014-07-10 12:09:33 DEBUG om_tcp_stop
2014-07-10 12:09:33 DEBUG om_tcp closing socket
2014-07-10 12:09:33 DEBUG remove socket [25]
2014-07-10 12:09:33 DEBUG om_tcp_stop
2014-07-10 12:09:33 INFO reconnecting in 1 seconds
2014-07-10 12:09:33 DEBUG new event in event_thread [in_file:MODULE_SPECIFIC]
2014-07-10 12:09:33 ERROR [om_tcp.c:358/io_err_handler()] -;
[om_tcp.c:206/om_tcp_read()] om_tcp detected a connection error; An existing
connection was forcibly closed by the remote host.
2014-07-10 12:09:33 DEBUG new event in event_thread [out_nxlog:RECONNECT]
2014-07-10 12:09:33 DEBUG future event, event thread sleeping 796875ms in
cond_timedwait
2014-07-10 12:09:33 DEBUG worker 1 processing event 0x85af28
2014-07-10 12:09:33 DEBUG PROCESS_EVENT: WRITE (out_nxlog)
2014-07-10 12:09:33 DEBUG om_tcp_write
2014-07-10 12:09:33 DEBUG module out_nxlog not running, not writing any more
data
2014-07-10 12:09:33 DEBUG worker 1 waiting for new event
2014-07-10 12:09:34 DEBUG new event in event_thread [in_file:MODULE_SPECIFIC]
2014-07-10 12:09:34 DEBUG nx_event_to_jobqueue: MODULE_SPECIFIC (in_file)
2014-07-10 12:09:34 DEBUG event added to jobqueue
2014-07-10 12:09:34 DEBUG new event in event_thread [out_nxlog:RECONNECT]
2014-07-10 12:09:34 DEBUG future event, event thread sleeping 203125ms in
cond_timedwait
2014-07-10 12:09:34 DEBUG worker 0 got signal for new job
2014-07-10 12:09:34 DEBUG worker 0 processing event 0x3d8758
2014-07-10 12:09:34 DEBUG PROCESS_EVENT: MODULE_SPECIFIC (in_file)
Of course, I can attach the whole nxlog.log file if needed.
This situation is repeatable. How can I solve the problem?
Thank you for help in advance,
Slawomir
------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
_______________________________________________
nxlog-ce-users mailing list
nxlog-ce-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
