-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 One other thought on that, the timeouts don't seem to be happening exclusively right after _server_ startup. In fact, they seem to increase in frequency (although not significantly so) as the server's uptime increases.
It may have something to do with _session_ startup, we've only observed the timeouts when the initial login page is loading, never (that we have observed) when the login page/credentials are submitted, the authentication token is returned or when the other servers check the authentication token that the client got from this server. The only thing is that tomcat (or the apps tomcat is running) shouldn't have to do DNS lookups _every_ time a new session is started, that's the whole point of caching. Also, LDAP shouldn't be involved in serving up that initial page (whether it is or not is a question for the CAS mailing list). So I guess the whole question here is what else is happening on session startup that's taking so long (this whole thing is seeming more and more like a CAS problem)? David King Goshen College ITS [email protected] 574-535-7726 Jacob Anderson wrote: > Hi David, > > I've seen that before as well with our servers loading the application > container after a server restart. We have our timeout set to 60 and it > appears to work well. I imagine that your first-time load does some name > lookups and IP address caching, which is probably causing the slow startup. > > You may have to set the timeout to something higher, like 120, to get around > the first-load overhead. > > Is your single-sign-on server CAMS from CafeSoft? > > -- Jake > > > -----Original Message----- > From: David W King [mailto:[email protected]] > Sent: Tuesday, March 16, 2010 12:37 PM > To: [email protected] > Subject: Re: [Pound Mailing List] Connection Timeouts > > Thanks for the response, Jacob. I would like to avoid NATting for two > reasons 1) the problem only occurs 5% of the time or less so if I did > NAT and it worked, it wouldn't really prove anything since _most_ of the > time it works anyway and 2) the tomcat app is a single sign on thing for > a bunch of our web apps. The web apps (which I don't have access to > change) are directing the clients to an https connection but the NAT > would connect them directly to the http tomcat connection (bypassing > pound and, therefore, SSL). > > I checked my tomcat/catalina logs but, unfortunately, tomcat isn't > logging connections. I'll get that fixed but not today. > > As far as external services go, it relies on DNS and LDAP both of which > are functioning very well. One thing we have noticed that may be of > interest is that the timeout error only occurs when the initial login > page is loading, it never occurs (that we know of) when the login form > is submitted. That points to a tomcat issue but it may be pertinent > here as well. > > David King > Goshen College ITS > [email protected] > 574-535-7726 > > > > Jacob Anderson wrote: >> Hello David, > >> First step - try direct routing with a static NAT rule to jump your users >> directly to one back end. If they timeout, then it's a back end issue. If > it >> works great, then read on. > >> Next, check your routing. The back end may have a firewall running that is >> not allowing the traffic on its local interface? I see that you are > running >> against 127.0.0.1, so next check the web logs. > >> /path_to_tomcat/logs/catalina.out >> /path_to_tomcat/logs/tomcat_xx-yy-zzzz.txt > >> If you are seeing connections entering tomcat, but not returning (in the >> log), then you have a container problem. Check the catalina.out file to > see >> if there are any errors on the command line output for the tomcat process. > >> Lastly, if your application needs to hit a database, check that too. Make >> sure that you can route and resolve the database. If you are using any > NAMES >> for hosts, then check your DNS. DNS name resolution is the #1 culprit for >> poor application performance. Try to use IP addresses if you can, but that >> makes administration harder. > >> Good luck! We use tomcat behind pound and it works great. > >> -- jake > > >> -----Original Message----- >> From: David W King [mailto:[email protected]] >> Sent: Tuesday, March 16, 2010 8:35 AM >> To: [email protected] >> Subject: [Pound Mailing List] Connection Timeouts > >> I started a new job two weeks ago and with it I inherited a server >> running pound as a front end for tomcat. I am a _complete_ pound noob >> but it worked fine until about a week ago when my users started >> reporting that intermittently they will get timeout messages in their >> browsers when trying to access this server. About the same time I >> started getting a lot of pound errors in syslog: > >>> Mar 15 21:53:04 login pound: (7f419410f950) error read from > 199.8.239.245: >> Connection timed out >>> Mar 15 21:53:09 login pound: (7f4194150950) error read from > 199.8.239.245: >> Connection timed out >>> Mar 15 22:02:01 login pound: (7f419410f950) error read from > 199.8.237.104: >> Connection timed out >>> Mar 15 22:02:40 login pound: (7f4194191950) error read from > 199.8.237.104: >> Connection timed out >>> Mar 15 22:13:12 login pound: (7f4194191950) error read from > 199.8.237.189: >> Connection timed out >>> Mar 15 22:15:02 login pound: (7f419410f950) error read from > 199.8.239.245: >> Connection timed out >>> Mar 15 22:15:03 login pound: (7f4194191950) error read from > 199.8.237.189: >> Connection timed out >>> Mar 15 22:20:12 login pound: (7f419410f950) error read from > 199.8.239.245: >> Connection timed out >>> Mar 15 22:21:54 login pound: (7f41941d2950) error read from > 199.8.239.245: >> Connection timed out >>> Mar 15 22:22:48 login pound: (7f41941d2950) error read from > 199.8.239.245: >> Connection timed out >>> Mar 15 22:23:24 login pound: (7f4194150950) error read from > 199.8.239.245: >> Connection timed out >>> Mar 15 22:29:12 login pound: (7f4194150950) error read from > 199.8.239.245: >> Connection timed out >>> Mar 15 22:41:40 login pound: (7f4194150950) error read from > 98.206.89.134: >> Connection timed out >>> Mar 15 22:43:23 login pound: (7f4194150950) error read from > 199.8.239.245: >> Connection timed out >>> Mar 15 22:43:23 login pound: (7f4194191950) error read from > 199.8.239.245: >> Connection timed out >>> Mar 15 23:02:53 login pound: (7f41941d2950) error read from > 199.8.237.235: >> Connection timed out > >> Some details about my environment: > >> $ pound -V >> starting... >> Version 2.4.3 >> Configuration switches: >> --enable-cert1l >> Exiting... > >> $ lsb_release -a >> No LSB modules are available. >> Distributor ID: Ubuntu >> Description: Ubuntu 9.04 >> Release: 9.04 >> Codename: jaunty > >> $ cat /etc/pound/pound.cfg >> ###################################################################### >> ## global options: > >> User "www-data" >> Group "www-data" >> #RootJail "/chroot/pound" > >> ## Logging: (goes to syslog by default) >> ## 0 no logging >> ## 1 normal >> ## 2 extended >> ## 3 Apache-style (common log format) >> LogLevel 2 > >> ## check backend every X secs: >> Alive 30 > >> ## use hardware-accelleration card supported by openssl(1): >> #SSLEngine "<hw>" > >> # poundctl control socket >> Control "/var/run/pound/poundctl.socket" > >> TimeOut 30 > > >> ###################################################################### >> ## listen, redirect and ... to: > >> ## redirect all requests on port 8080 ("ListenHTTP") to the local >> webserver (see "Service" below): >> ListenHTTPS >> Address <hostname removed> >> Port 443 >> Cert "/etc/pound/<hostname removed>.pem" > >> ## allow PUT and DELETE also (by default only GET, POST and HEAD)?: >> xHTTP 0 > >> Service >> BackEnd >> Address 127.0.0.1 >> Port 8080 >> End >> End >> End > >> ## EOF > >> I hope I'm just missing something obvious here. Can anyone else spot it? > >> Thanks! > > - -- To unsubscribe send an email with subject unsubscribe to [email protected]. Please contact [email protected] for questions. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkug6WkACgkQH+/Vg7DylXYHJgCdFzg2GVJdsl7hlxU5yo35NLf/ TLUAn2cobnSDd74icwgCVWMqSW0e4Vh2 =ANyn -----END PGP SIGNATURE----- -- To unsubscribe send an email with subject unsubscribe to [email protected]. Please contact [email protected] for questions.
