For the Apache httpd cosign filter, the DNS lookup of "weblogin" only
happens on Apache child init. I'm not sure how the Java or IIS filters do
this.
If we can anticipate an outage where that isn't possible (such as the
outage of an entire data-center - or at least the VLAN itself for whatever
reason) then we pre-populate the DNS with extra IPv4 service addresses
about a week in advance in order to accomodate any load issues.
H
ere at the University of Michigan, we have separate IPv4 service addresses
(i.e.: separate from the machine/administrative server addresses), so we
could move the service address to another machine while we did maintenance
(or dealt with a problem.) Of course, this works only if the two machines
are on the same LAN.
The Apache filter seems to work OK with this, and the Java filter does
too. But we sometimes have problems with the IIS filter requiring that the
webservers on the IIS machines be restarted in order to catch on to the
fact that the back-channel cosign TCP connection isn't valid anymore.
Recently, we had a problem with our service when 2 (of 6) cosign servers
were unavailable because a router switch failed - but the network routing
still treated the LAN as available (other parts of the LAN were available.)
This caused the whole system to bog down because each of the remaining
cosign server was trying to replicate cosign tickets to the two that were
down. My pet theory is that it was the timeout on the TCP connect (not
session) that caused problems, since the network wasn't sending back ICMP
network unreachable packets. (BTW: Detecting that part of your LAN is down
isn't easy - and we've separated those two servers onto separate switches,
but still the same LAN as an immediate fix.)
--- Richard Conto
Webmaster and WebHosting
Application & Web Infrastructure, Core Services and Infrastructure
Infrastructure Services, Information and Technology Services
Arbor Lakes Bldg 1-2340
734.764.6991 office
its.is.csi....@umich.edu
On Thu, Jul 4, 2013 at 9:08 AM, Toby Blake <t...@inf.ed.ac.uk> wrote:
> Hi all,
>
> We use a DNS RR to load-balance across our cosign servers - so 'weblogin'
> points to 2 (or more) servers.
>
> For server maintenance, we bring up a temporary cosign server and add
> it to 'weblogin', enabling us to take one of the original servers down,
> while keeping the same number of active servers.
>
> This isn't always a smooth process and I this seems to be because the
> cosign filter populates its list of servers when it starts and doesn't
> refresh it. This means that a change to 'weblogin' in DNS has to be
> followed by restarting apache across all cosign-protected sites.
>
> Perhaps the filter should re-populate its connection list from DNS? I'm
> not overly familiar with the code, so I'm not quite sure where this
> re-population should occur. And, of course, there's a danger slowing
> things down by doing it too often.
>
> I wonder if others also see this problem and if so, how they address it?
>
> Cheers
> Toby Blake
> School of Informatics
> University of Edinburgh
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Cosign-discuss mailing list
> Cosign-discuss@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/cosign-discuss
>
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:
Build for Windows Store.
http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Cosign-discuss mailing list
Cosign-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/cosign-discuss
