While testing rsyslogd sending logs to a remote server I encountered
this scenario:
- remote server (which is behind amazon elastic load balancer) closes
connection (the host is up but nobody listens on the port)
- rsyslogd seems to be sending data to the closed connection
- connection is in CLOSED_WAIT state
strace of rsyslogd reveals (replaced IP and content of message by XXX):
[pid 14188] socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 2
[pid 14188] connect(2, {sa_family=AF_INET, sin_port=htons(5140),
sin_addr=inet_addr("XXX")}, 16) = 0
[pid 14188] recvfrom(2, 0x7f44d42e765f, 1, 66, 0, 0) = -1 EAGAIN
(Resource temporarily unavailable)
[pid 14188] sendto(2, "<135>2013-07-26T20:43:08+00:00 XXX", 679, 0,
NULL, 0) = 679
And everybody thinks the connection is in CLOSE_WAIT state:
$ sudo lsof | grep rsyslogd | grep TCP
rsyslogd 14184 syslog 2u IPv4 188760
0t0 TCP
ip-10-2-35-151.ec2.internal:48228->ec2-54-225-181-82.compute-1.amazonaws.com:5140
(CLOSE_WAIT)
$ netstat -a | grep 5140
tcp 1 0 XXX:48229 ELB:5140 CLOSE_WAIT
ELB in above is name/IP of Amazon elastic load balancer, it seems
that it behaves slightly suspiciously (why does connect succeed?, why
does sendto succeed?)
Any ideas why rsyslogd does not close the connection that is in
CLOSE_WAIT state? The connection remains in CLOSE_WAIT state even after
the remote server starts listening on 5140 port. This does not happen
everytime there is no listener on remote host but when it happens it
doesn't seem to be fixed until I restart rsyslogd.
erik
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
