Hello David, Thank you for these explainations. Actually, in my current job we use keepalived for this.
I was wondering, this should lead to the same behavior if one of the remote syslog server dies, no ? The sender can't know the state of the message, as with my current failover configuration. Regards, Smana ----- Mail original ----- De: "David Lang" <[email protected]> À: "rsyslog-users" <[email protected]> Envoyé: Mercredi 11 Février 2015 15:55:21 Objet: Re: [rsyslog] Relp failover : duplicate logs On Wed, 11 Feb 2015, [email protected] wrote: > I'll think about a third part application for loadbalancing. pacemaker can operate in two modes. 1. the IP shifts from one to the other. In this case, your config would get simplified because you would not need to have two outputs and the failover between them. You would only have one output and if the server fails, you reconnect and hit the second box. 2. the IP is shared between them at the same time This is load balancing, but it only load balances connections, not individual logs. So you would need to have multiple systems sending or have the senders reconnect every 1000 or 10000 messages to spread the load across the multiple servers. This is what you use when the back-end processing/storage of the logs is significant and you need to distribute the load. In most cases this load is not the resources rsyslog consumes, but rather whatever rsyslog is feeding the logs into. In either case, the big advantage of doing it on the receiving side is that the sender doesn't need to know about it. This simplifies the config on the sender (you don't need the server2 portion of the config) This also means that if you change things on the receiver side you don't have to change the senders (other than that when you go to a load balanced config you may want to set the rebind interval on the senders to make it balance more evenly) If you change the number of servers receiving, build new ones and shift load to them without taking the old ones down first, etc. It's all transparent to the senders. David Lang > Regards, > Smana > > ----- Mail original ----- > De: "David Lang" <[email protected]> > À: "rsyslog-users" <[email protected]> > Envoyé: Mercredi 11 Février 2015 14:23:22 > Objet: Re: [rsyslog] Relp failover : duplicate logs > > On Wed, 11 Feb 2015, [email protected] wrote: > >> Is there a way to keep a failover architeture without duplicating logs >> please ? > > unfortunantly no. If you work through exactly what can go wrong here, you > will > see that there is no way to both ensure all logs are delivered and none are > duplicated without the sender tagging each message and the receiver then > weeding > out duplicates > > The problem is that the communications between the two systems is not > instantanious, you are using TCP, which will not loose data under normal > conditions, but will loose data under failure conditions. RELP adds > application > level acknowlegements to the process, so that if the sender doesn't get an > acknowledgement from the receiving application it will assume the message > didn't > get through and re-send it. But this means that if something goes wrong that > prevents these acknowlegements from getting through, the receiver will thing > they have the message, all properly acknowleged, but the sender will thing it > never got to the receiver and will re-send it, causing a duplicate. > > In cases like this where the software exits without the ability to save any > state (kill -9, reboot, server failure, etc) the receiver has no way of > saving > it's exact state, so when it comes up again it can't know what messages had > been > received previously, so it can't tell that the messages it's receiveing are > the > same as ones that it received before the outage. > > You should also look into things like pacemaker http://clusterlabs.org/ that > will let you have multiple systems share a single IP address and fail it over > if > one dies. This will allow you to reduce the outage from a system > failure/reboot > to seconds. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
