On Wed, 19 Jan 2011, Dra?en Ka?ar wrote:
Robert Jennings wrote:
With TCP it appears the clients remote logging stops indefinitely, in
fact even when a failback occurs it still isnt logging messages to the
primary server until I restart the client rsyslog process.
How does your failover procedure work exactly? If it's crashing the OS
(or turning off power), then there's nothing which can send TCP close
packet, so your clients won't know they should close the connection.
as far as the client sees, the IP address changes it's MAC address. any
packets sent to the same IP address, but as part of the old connection,
will be refused as the server will have no record of that connection.
this should happen on the first packet after the IP moves to the new
machine.
This is the same thing that would happen if the server were to reboot.
David Lang
If that's the case, there should be retransmitting TCP timeout on the
client, which is usually 13-15 minutes by default. The exact number
depends on the OS and the network MTU. Did you wait that long?
With UDP behaviour is as expected, with a ~20-30 second failover period
where messages are lost (settings not tweaked in anyway)
I am using 3.22 simply as it is shipped with RHEL5/CENTOS5, but would be
interest to try a more recent version.
Which leads me to think perhaps the best solution may be:
Rsyslog clients UDP -> prd-syslog-000
Syslog clients & devices UDP -> prd-syslog-000
Prd-syslog-002 TCP -> prd-syslog-001
So prd-syslog-002 is only ever hit when prd-syslog-001 is down, and can
reliably queue and push messages to prd-syslog-001 once it is back
online.
Thanks,
Rob
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of [email protected]
Sent: Wednesday, 19 January 2011 2:00 PM
To: rsyslog-users
Subject: Re: [rsyslog] rsyslog with heartbeat
how long did you wait for the failover to take place? the first thing
that
will have to happen is that the sender will have to notice that the TCP
connection doesn't work anymore, close it, and open another one.
Since the system doesn't know what messages have actually been sent to
the
server (as opposed to mearly sent to the TCP stack on the sending
machine), there will be some logs lost at failover time. the longer it
takes the client to notice the problem, the worse this will be.
version 3.22 is fairly old at this point, Rainer will have to weigh in
on
how long it should take to notice the failure.
to fully avoid the lost messages issue, you would need to use RELP for
your transport.
I have opted to use UDP and accept that while the boxes are failing
over,
some logs will be lost (you can tune heartbeat for pretty short failover
times, subsecond if you really push it)
David Lang
On Wed,
19 Jan 2011, Robert Jennings wrote:
I am running 2 rsyslog servers (prd-syslog-001, prd-syslog-002)
sharing
a virtual IP managed by heartbeat (prd-syslog-000)
clients are configured to send logs to the virtualIP
*.info;mail.none;authpriv.none;cron.none
@@prd-syslog-000
authpriv.*
@@prd-syslog-000
local7.*
@@prd-syslog-000
I have found when the primary host goes down and the secondary takes
over, logs are not arriving at the secondary host unless I restart
rsyslog on the client, this was not the behaviour I was expecting. Is
there any default configuration that may be stopping logs from being
sent when the server sitting behind the virtual ip changes?
The functionality I was trying to achive is as follows:
* clients log to prd-syslog-000
* heartbeat controls prd-syslog-000 address (prd-syslog-001
primary, prd-syslog-002 on failover)
* prd-syslog-002 is configured to forward logs to prd-syslog-001
* during downtime of prd-syslog-001 these logs will sit queued
* when prd-syslog-001 comes back up it will take over
prd-syslog-000 and start recieving client logs
* prd-syslog-002 will flush its queue back to prd-syslog-001
The reason I am using this setup as opposed to logging to two rsyslog
hosts is to support devices that can only log to one host, and provide
one location with a consistent set of logs instead of two machines
with
out of sync logs.
running rsyslog-3.22
Thanks,
Rob
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
