On Wed, 2 Mar 2016, Peter Viskup wrote:
Hi all,
just would like to understand TCP keepaliving option in more details.
Seems to be effective/sufficient on server side only.
We have issue with increasing number of open filehandles of our
rsyslog server and dicovered the number of open TCP connections from
some hosts is increasing on server side. The root cause is the
firewall just mark the TCP session as invalid after some time of
inactivity and send RST to client once it is trying to use that TCP
session. Unfortunately server side is not notified about it and it
just accept another connection with previous still keeping open.
Seems like the KeepAlive on server side will solve the issue. With one
side effect - even not used connections are kept open.
Just discovered the omfwd (client side) has RebindInterval option
available which seems to cover some sort of connection recycling. From
my point of view it would be helpful to perform connection close on
server side in case no data were transmitted during configured time
interval.
Does that make sense?
The rebind interval on the server side won't help if the connection is cut by a
firewall, it just counts how many messages were sent and reconnects after X
messages. This is so that load balancers can have a chance to assign the
connection to a new destination.
Similarly, having a server side disconnect after X messages also won't help, if
there are no messages, it won't disconnect.
If keepalive is enabled, it should send packets back and forth when the
connection is otherwise idle. However if the time between the keepalives is
longer than the time the firewall allows connections to be idle, it does no
good.
Personally, my approach to this problem is to cause systems to generate one
message per minute. This is an application-level keepalive that you can watch
for and alert if it disappears, providing a "dead man's switch" type detection
of network/server problems.
with rsyslog, impstats on the sender does a good job of this, or
vmstat 60 |logger -t vmstat
The vmstat output is a very dense report of a lot of information that you
normally don't care about, but after something has gone wrong, it frequently
provides invaluable information to track down the root cause of the failure, not
just when the system finally died.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
