On Wed, 23 Oct 2013, Erik Steffl wrote:
On 10/23/2013 12:56 PM, Pavel Levshin wrote:
IMHO, for high-load TCP, rebindinterval is counter-productive, because
you need to establish a new connection each time. But yes, this
parameter exists.
why would it be counter productive? It should not be done too frequently,
that would be counter productive. But done at the right interval the cost of
re-establishing TCP connection is neglible and makes it possible to do load
balancing.
so you are saying the RebindInterval is just missing in docs?
I think that the RELp module just does not have rebindInterval, you can try it
and see if it works, but I suspect that it won't.
Is there anybody successfully using RebindInterval with omrelp? Which
rsyslog version? I am using 7.5.5-0adiscon2 (from adiscon repo
http://ubuntu.adiscon.com/v7-devel quantal/)
Load balancing is very dependent on nature of your load, on your
bottlenecks, and so on. Why do you want to balance TCP connections?
If nothing else it provides high availability, spefically we are using
Amazon elastic load balancer (Amazon makes sure it's up and running) with
autoscaling group behind it (again, Amazon makes sure that the bad machines
are terminated, that at least N machines are up and running (we specify N)
etc.
well, with RELP, you actually don't need rebindinterval for HA, when the ELB or
an instance dies, traffic will pause temporarily, timeout, and establish a new
connection.
rebindinterval will not change this. rebindinterval just makes it so that when
everything is working well, it will disconnect after N messages and reconnect.
Other than that we are doing load balancing for the obvious reason, there
are tons of machines that send tons of logs, the collector machine should not
become a bottleneck so we want to have more than one machine. Load balancer
is a way to do that. Other solution would be to have bunch of machines that
all those log senders would magically know about which is not exactly a
solution (more like nightmare).
what sort of traffic volume are you dealing with? a single system can handle a
pretty high traffic volume, so unless you are sending the traffic to a slow
destination of have a _very_ high traffic volume (several hundred thousands of
messages per second) a single box should be able to handle the full load.
David Lang
erik
--
Pavel Levshin
23.10.2013 23:26, Erik Steffl:
What are recommendations for use of rsyslog with loadbalancers? We
have setup where number of machines run rsyslog and forward syslog
messages to central location using RELP protocol (using omrelp module).
The central location is a load balancer with cluster of machines
behind it (that run rsyslog and use imrelp).
I assume this is not an uncommon scenario so hoping that there is a
way to do this in a way that does load balancing.
The problem is that TCP connection stays open for a long time so there
is effectively no load balancing done. Looking for a solution I found
http://www.rsyslog.com/load-balancing-for-rsyslog/ which mentions
RebindInterval which is documented in omfwd docs
http://www.rsyslog.com/doc/omfwd.html
However I don't see RebindInterval in omrelp docs
http://www.rsyslog.com/doc/omrelp.html and when I try to use it
rsyslog segfaults (in relpCltHintBurstEnd in librelp.so.0, also sent
email specifically about that).
Should RebindInterval work for omrelp? Or is there some other way to
re-open TCP connection so that load balancing works?
Thanks!
erik
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
