Problem summary: My nio polling threads are using too much CPU time. Application overview: My application has from 1,300 - 4,000 users connected at any given time. Each user sends about 200 bytes, then waits 30 seconds, then sends about 200 bytes, and this just loops for each user. Each user connects with SSL, and we use a long keepalive to ensure that the HTTP connection doesn't close, so that we don't have to renegotiate SSL.
Configuration: Ubuntu 12.0.4 with Tomcat 7.0.35, 1.75 gigs of RAM. We are using Apache with SSL and mod_proxy_ajp to forward requests to Tomcat. It has MPM module enabled, with 500 ThreadsPerChild, so we typically have from 3-9 Apache instances running. > <IfModule mpm_worker_module> > ServerLimit 12 > ThreadLimit 1000 > > StartServers 1 > MinSpareThreads 25 > MaxRequestsPerChild 0 > MaxSpareThreads 500 > ThreadsPerChild 500 > MaxClients 5000 > </IfModule> > > ProxyPass /WSMRegister ajp://localhost:8009/WSMRegister We are using the AJP NIO connector on port 8009 on Tomcat with 15 worker threads: > <!-- Define an AJP 1.3 Connector on port 8009 --> > <Connector port="8009" > protocol="org.apache.coyote.ajp.AjpNioProtocol" > redirectPort="8443" > minSpareThreads="1" > maxThreads="15" > scheme="https" > secure="true" > URIDecoding="UTF-8" > proxyName="secure2.360works.com" > proxyPort="443" /> Problem detail: lsof is currently showing 564 open sockets between Apache and Tomcat on port 8009, with 1,352 users connected to Apache. The two threads consuming the most CPU time in Tomcat are "NioBlockingSelector.BlockPoller-2 / 15" and "ajp-nio-8009-ClientPoller-0 / 25". Between them, they are taking 20% of all CPU time for the Tomcat process. I get a few times a day when our monitoring software reports slow response times, and I'd like to solve this. Some guesses at solutions: I'm guessing that the high CPU usage is because they are polling all 564 open sockets constantly? Would it make sense to reduce the number of open sockets? I didn't configure any maximum and I don't know how to reduce this number. I'm also concerned that reducing that might negate any benefits by increasing the number of sockets opening and closing between ajp_mod_proxy and the NIO AJP connector. Maybe it's already running at optimal performance and I just need to throw hardware at it, but it seems like a solvable problem, because the actual worker threads are not doing much at all. --Jesse Barnum, President, 360Works http://www.360works.com Product updates and news on http://facebook.com/360Works (770) 234-9293 == Don't lose your data! http://360works.com/safetynet/ for FileMaker Server == --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
