I added udp_preference_limit = 1 in the [libdefaults] section of the /etc/krb5.conf, but it didn't seem to address the issue. We are running on Linux (RHEL) with Java 1.6.0_03.
We have seen an unusually large number of blocked threads after a few hundred requests, and after enough connections Tomcat stops responding. There could be several things wrong with our environment such as a broken connection to Active Directory or a broken connection to Domino (custom code we wrote to generate an LtpaToken for single sign on to Lotus Notes apps). We have not seen an OutOfMemoryError since changing Xmx from 64m (the default) to 256m, however, the memory is still growing and eventually Tomcat becomes unresponsive, presumably due to the number of blocked threads. I'll continue to narrow down the areas which could be a problem, and repost to this list as I find more information. Thanks for the help so far! Brad Cupit Louisiana State University - UIS -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Spencer Sent: Thursday, January 31, 2008 3:47 AM To: Yale CAS mailing list Subject: Re: trying to track down jaas memory leak Sorry - it was late at night and I got my TCP and UDP back-to-front. com.sun.security.auth.module.Krb5LoginModule will ordinarily use UDP sockets and it is these that we were seeing accumulating. A "udp_preference_limit" can be set in the kerberos configuration (krb5.conf) and if the size of the message is greater than this limit TCP is used instead. By setting the udp_preference_limit to 1, we forced all messages to be sent by TCP and our UDP socket leak went away. Sorry if I've confused anyone! Dave --On 30 January 2008 23:06 +0000 David Spencer <[EMAIL PROTECTED]> wrote: > Brad, > > Possibly an unrelated problem and I don't have all the details to hand but > will look them up tomorrow at work if it seems relevant to you. > > We ran into a problem with com.sun.security.auth.module.Krb5LoginModule that > caused our CAS server to gradually accumulate TCP sockets and eventually fall > over when it had used up all the socket resources on the box. This was Java 5 > on some flavour of Linux. We hadn't seen the problem running the same code on > Solaris. I think we would have been running with a larger heap than 256Mb so > we perhaps hit a socket resource problem before we hit the memory limit you > are seeing? > > A bit of digging showed that it was forgetting to close the TCP socket but it > also showed that the section that dealt with UDP sockets didn't have the same > problem. We asked the module to always use UDP sockets and the leak went > away. CAS service was running uninterrupted throughout 2007. > > I'll dig out the details in the morning. > Dave > > > --On 30 January 2008 16:22 -0600 Brad A Cupit <[EMAIL PROTECTED]> wrote: > >> >> >> Hello, >> >> We have a CAS server using JAAS + Kerberos to authenticate users against >> Active Directory. We started seeing OutOfMemoryErrors with the default Xmx >> (of 64m) which we have since bumped up to 256m. We haven't had >> OutOfMemoryErrors since then, but the memory usage keeps rising. >> >> >> >> I've hooked up JProfiler to try and see where the memory is going, and >> noticed that it goes up with each request, and running the garbage collector >> (via System.gc()) doesn't reclaim many of the objects. I'm sure we just have >> a configuration error of sorts, but I've spent a few days and can't seem to >> figure it out. >> >> >> >> JProfiler tells me that after a few requests (500 or so), we have an enormous >> number of LinkedHashMap$Entry objects, as well as >> java.security.Provider$ServiceKey, java.security.Provider$Service, and >> HashMap$Entry instances. >> >> >> >> I've also noticed that instances of com.sun.crypto.provider.SunJCE go up by 2 >> per request, and don't get reclaimed with garbage collection. >> >> >> >> JProfiler's cumulative allocations point to >> javax.security.auth.login.LoginContext.login() method, but I've checked out >> the code and stepped through it with a debugger, but can't see anything wrong >> (no creation of instances that would be uncollectable by the gc). >> >> >> >> If it helps, here's our jaas.conf file: >> >> >> >> CAS { >> >> com.sun.security.auth.module.Krb5LoginModule required client=TRUE >> debug=FALSE useTicketCache=FALSE; >> >> }; >> >> >> >> I'm going to try to setup CAS to use the LDAP authentication handler to see >> if the problem is strictly JAAS related. >> >> >> >> Has anyone seen issues like this before? >> >> >> >> Thanks in advance! >> >> >> >> Brad Cupit >> Louisiana State University - UIS >> e-mail: [EMAIL PROTECTED] >> office: 225.578.4774 >> >> > > > > ---------------------- > David Spencer > Information Systems and Computing > University of Bristol > _______________________________________________ > Yale CAS mailing list > [email protected] > http://tp.its.yale.edu/mailman/listinfo/cas ---------------------- David Spencer Information Systems and Computing University of Bristol _______________________________________________ Yale CAS mailing list [email protected] http://tp.its.yale.edu/mailman/listinfo/cas _______________________________________________ Yale CAS mailing list [email protected] http://tp.its.yale.edu/mailman/listinfo/cas
