We are using a commerical servlet that seems to keep getting overloaded. We wrote a little script that monitors a number of things about the process - the number of open file descriptors, sockets, java threads, mysql connections, misc. hit types/second and memory sizes. We turned on -verbose:gc so that we could see the GC. I then did a tail of the two log files so that I could see the relationship between the two.
Any time a full garbage collection takes place, a queue of connections builds up. At busy times, this is enough to make tomcat unable to ever catch up. If tomcat isn't 'too' busy, then a spike immediately follows. A sample can be seen below: ==> /var/log/tomcat/catalina.out <== [GC 168312K->159078K(170540K), 0.0139380 secs] [GC 169766K->160496K(171308K), 0.0231370 secs] [Full GC ***** the full garbage starts here ******** ==> tomcat_fd_and_threads.log <== 2004-10-05 13:48:29 - 186 fds, 40 sockets, 79 jthreads, 10 mysql con [28, 4, '16.2', '1.2'] hits/s 219868 186960 ******* 28 max hits/sec ******* 2004-10-05 13:48:39 - 186 fds, 40 sockets, 79 jthreads, 10 mysql con [5, 1, '1.6', '0.0'] hits/s 219868 202660 ******* 5 max hits/sec ******* ==> /var/log/tomcat/catalina.out <== ***** the full garbage collection finishes 14 secs later ******** 160496K->84976K(171308K), 14.4078430 secs] [GC 95651K->86462K(170540K), 0.0303780 secs] [GC 97150K->87630K(170540K), 1.1270000 secs] [GC 98309K->88721K(170540K), 0.3796250 secs] ==> tomcat_fd_and_threads.log <== 2004-10-05 13:48:50 - 92 fds, 40 sockets, 79 jthreads, 10 mysql con [60, 2, '16.3', '0.5'] hits/s 219220 201996 ******* 60 max hits/sec ******** So before java started garbage collecting, Java was sitting at about max 28 connections / second processed. Only about 5 hits/second happened while java was garbage collecting (or immediately thereafter), and then the system processed about 60 hits/second to catch back up. Is there anything that I can do improve the scenario? I tried -Xincgc, but that didn't seem to help (it actually seemed to make it worse, although I don't have an accurate count of the load to know if was really the load on the server that was the problem). Is this typical for most servlets, or could there be a problem with the way it is written? Thanks for the help. --Kaleb --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
