DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT <http://nagoya.apache.org/bugzilla/show_bug.cgi?id=5729>. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE.
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=5729 Tomcat standalone bogs down very badly under load Summary: Tomcat standalone bogs down very badly under load Product: Tomcat 4 Version: 4.0.1 Final Platform: Sun URL: could be set up, if necessary OS/Version: Solaris Status: NEW Severity: Normal Priority: Other Component: Catalina AssignedTo: [EMAIL PROTECTED] ReportedBy: [EMAIL PROTECTED] In the process of doing some benchmarking of JSP vs. ASP and other platforms, I have run into some anomalies in the way Tomcat (version 4.0.1 final, running standalone on port 8080, on a Sun Netra using Solaris) behaves under load. The following pretty long-winded description was partly for the benefit of my manager, so he'd understand what I ran into. I hope that, long as it is, it might be of benefit in possibly reproducing the problem and/or results. The JSP page involved was truly tiny. To wit: <HTML><BODY> The time is <%= (new java.util.Date()).toString() %> </BODY></HTML> Really. That's all. Without further ado, the rest of my report. It is immediately followed by a copy of the "annotatedLogs.txt" that is referred to in the report. ************** FILE: JspReport.html **************** <HTML> <HEAD> <TITLE>JSP Testing</TITLE> </HEAD> <BODY> <FONT SIZE=4><P>Testing procedures:</P></FONT> <FONT Size=2> <P>We use "WebCat," which is a tool that will repeatedly hit a given URL with any (reasonable) number of simultaneous users (accomplished by spawning a number of threads, each of which simply repeats the given HTTP request as soon as it receives all the data from the prior request).</P> <P>A typical "run" of one of our tests takes one minute. During the first ten seconds, no results are measured (this is done so that the server can "come up to speed"—presumably any compilations or caching will be accomplished during this time). For the next 50 seconds, the tool captures the results of each "hit" on the web server. The "performance" number we are most interested in—and the one we always report—is the total number of page hits during those 50 seconds and, consequently, the average number of "hits per second" that the Web server manages to deliver.</P> <P>A typical "set" of tests of a given URL consists of doing all the above with a varying number of "threads." Empirical tests have shown us that some pages will exhibit noticeable overall performance differences only between one and two or one and four threads whereas other pages will show measurable differences up to as many as 20 or 30 threads. So, as a matter of course, we test with:</P> <UL><LI>1 Client Thread <LI>2 Client Threads <LI><P>4 Client Threads <LI><P>7 Client Threads <LI><P>10 Client Threads <LI><P>20 Client Threads <LI><P>30 Client Threads <LI><P>40 Client Threads <LI>50 Client Threads</UL> <P>Not surprisingly, in the case of the Tomcat web server (running standalone, at least) this results in the web server starting a like number of threads.</P> <P>To give an example, here are the results we get from Tomcat as it serves up some JSP pages that do <I>not</I> involve any access to a database. As noted, the numbers in the top row indicate the number of clients that are "simultaneously" (within the limits of the multi-threaded driver program, the capabilities of the network, and the socket mechanisms on both ends) "hitting" the web server. Numbers in other cells indicate the number of "hits" (pages) per second that the Web server is responding with.</P> </FONT> <P ALIGN="LEFT"><TABLE BORDER CELLSPACING=1 BORDERCOLOR="#000000" CELLPADDING=9 WIDTH=607> <TR><TD WIDTH="17%" VALIGN="TOP"> <FONT SIZE=2><P> Clients:</P> <P>Test name:</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>1</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>2</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>4</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>7</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>10</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>20</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>30</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>40</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>50</FONT></TD> </TR> <TR><TD WIDTH="17%" VALIGN="TOP"> <FONT SIZE=2><P>SUJ1</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>189</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>284</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>371</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>63</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>37</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>35</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>35</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>34</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>33</FONT></TD> </TR> <TR><TD WIDTH="17%" VALIGN="TOP"> <FONT SIZE=2><P>SUJ2</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>26</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>31</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>34</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>34</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>35</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>35</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>34</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>33</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>33</FONT></TD> </TR> <TR><TD WIDTH="17%" VALIGN="TOP"> <FONT SIZE=2><P>SUJ3N</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>36</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>53</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>61</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>60</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>58</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>55</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>55</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>49</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>48</FONT></TD> </TR> <TR><TD WIDTH="17%" VALIGN="TOP"> <FONT SIZE=2><P>SUJ3W</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>27</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>42</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>50</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>45</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>49</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>52</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>49</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>48</FONT></TD> <TD WIDTH="9%" VALIGN="TOP"> <FONT SIZE=2><P>45</FONT></TD> </TR> </TABLE> </P> <FONT SIZE=2> <P>And that data already shows one of the anomalies I keep seeing. </P> <HR> <P><FONT Size=4>What the tests are doing.</Font></P> <P>SUJ1 [SUn hardware, Jsp platform, test 1] is a bare bones, one line of Java code, page. Specifically, it consists of <%= (new Date( ) ).toString( ) % > and nothing more.</P> <P>SUJ2 contains no actual Java code, at all. Instead, it is just roughly 5K bytes of HTML that has been given a ".jsp" file name in order to (presumably) force it through the JSP server. </P> <P>SUJ3N and SUJ3W simulate database access and usage. They use a 2D array (or, in the case of Java, and array of arrays) to emulate the rows and columns of data one would retrieve from a database query. These tests perform a variety of manipulations on the data, including formatting the data (e.g., numbers, currency, etc.) for HTML display. Both tests work from the same data and do the same work. The only difference is that SUJ3N ("N" for Nowrite) does <I>not</I> write its results back to the client and SUJ3W ("W"rite) does. [In other words, any observed difference in time between pairs of results here could be presumed to be the overhead of sending data from Java to the web server and thence on to the client.]</P> <P>Those tests are ordered roughly in order of expected complexity and server performance. Indeed, on other servers the ordering has been shown to be correct. </P> <HR> <P><FONT Size=4>The anomalies</FONT></P> <P>But Tomcat shows some strange results. With 10 or more simultaneous clients, Tomcat performs worse on the two simple tests than it does on either of the two more complex ones. In point of fact, when test SUJ1 is repeated, it will often perform worse than any other test, even when serving only one or two simultaneous clients. On the other hand, on a very few occasions I saw it start off "slow" and then "jump" up to much higher performance, just as it jumps down to much lower performance in the numbers shown above. I’m sorry I don’t have a "catalina" log to show what the server was doing at that time [I inadvertently erased it], but please believe me that it shows nothing obvious.</P> <P>I have attached a file named "AnnotatedLogs.txt". This file is actually a combination of three files, intermixed so that the lines in the final result appear in chronological order, as best as I can make them. In addition, my annotations are added (they occur between lines of dollar signs, to make it clear they are not part of the original files).</P> <P>Which lines come from which original files is easy to determine: All the lines from <BR> "catalina_log.2001-12-24.txt"<BR> begin with a date and time stamp (indeed, with "2001-12-24"). All other lines, except the annotations, come from the Webcat logs.</P> <P>The results shown in this annotated log are fairly reproducible. The sudden "hit" that drops the server performance way down almost always occurs by the time the stress program is hitting the server with 30 clients. Other things noted in that annotated log are not as consistently reproducible, but I've seen all the "numbers" occur more than 2 or 3 times. </FONT> </BODY> </HTML> **************** END OF FILE ******************** ********** file: AnnotatedLogs.txt ********************* $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Tomcat was started up as a stand-alone server at roughly 13:10, as reflected by these first lines from the "catalina" log: $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ 2001-12-24 13:10:02 HttpConnector Opening server socket on all host IP addresses 2001-12-24 13:10:12 HttpConnector Opening server socket on all host IP addresses 2001-12-24 13:10:18 HttpConnector[8080] Starting background thread 2001-12-24 13:10:18 HttpProcessor[8080][0] Starting background thread 2001-12-24 13:10:18 HttpProcessor[8080][1] Starting background thread 2001-12-24 13:10:18 HttpProcessor[8080][2] Starting background thread 2001-12-24 13:10:18 HttpProcessor[8080][3] Starting background thread 2001-12-24 13:10:18 HttpProcessor[8080][4] Starting background thread $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Then, at just before 13:13, the stress tool was started, "hitting" Tomcat with repeated requests for the SUJ1 JSP test (yes, yes, it's name "Minimal.jsp" here...bad naming). $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Run [/BM/Minimal.jsp] Test run with 1 users: [13:13] Pages Requested, 10240, 204.80, 10240, 1 Errored, 0, 0.00, 0, 1 Fetched, 10240, 204.80, 10240, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 2 users: [13:14] $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Note that the test with 1 client had no errors. With two clients, though, we get 30 HTTP errors. Granted, as a percentage of the pages requested that is "noise", but it does seem to indicate that the system is already having trouble. $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Pages Requested, 14040, 280.80, 14040, 1 Errored, 30, 0.60, 30, 1 Fetched, 14010, 280.20, 14010, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 4 users: [13:15] 2001-12-24 13:15:39 HttpProcessor[8080][5] Starting background thread 2001-12-24 13:15:39 HttpProcessor[8080][6] Starting background thread 2001-12-24 13:15:39 HttpProcessor[8080][7] Starting background thread 2001-12-24 13:15:39 HttpProcessor[8080][8] Starting background thread Pages Requested, 18285, 365.70, 18285, 1 Errored, 4, 0.08, 4, 1 Fetched, 18281, 365.62, 18281, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 7 users: [13:16] Pages Requested, 20398, 407.96, 20398, 1 Errored, 6, 0.12, 6, 1 Fetched, 20392, 407.84, 20392, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 10 users: [13:17] $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ This run probably actually started at about 13:16:55, so the SocketException noted here probably occurred after quite a few pages had been successfully server up. This would explain the moderate Pages Fetched value we see, in comparison to the much lower number in the next sub-run. $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ 2001-12-24 13:17:01 HttpConnector[8080] accept: java.net.SocketException: Software caused connection abort at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:344) at java.net.ServerSocket.implAccept(ServerSocket.java:441) at java.net.ServerSocket.accept(ServerSocket.java:412) at org.apache.catalina.connector.http.HttpConnector.run(HttpConnector.java:979) at java.lang.Thread.run(Thread.java:539) 2001-12-24 13:17:01 HttpConnector[8080] Opening server socket on all host IP addresses 2001-12-24 13:17:01 HttpProcessor[8080][9] Starting background thread 2001-12-24 13:17:05 HttpProcessor[8080][10] Starting background thread $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Why do we need 11 threads (0 to 10) to service 10 clients? Or 21 threads to service 20 clients? Is that because Tomcat has "forgotten" about the thread that caused the connection abort, just above? $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Pages Requested, 5580, 111.60, 5580, 1 Errored, 3, 0.06, 3, 1 Fetched, 5577, 111.54, 5577, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 20 users: [13:18] 2001-12-24 13:18:04 HttpProcessor[8080][11] Starting background thread 2001-12-24 13:18:04 HttpProcessor[8080][12] Starting background thread 2001-12-24 13:18:04 HttpProcessor[8080][13] Starting background thread 2001-12-24 13:18:04 HttpProcessor[8080][14] Starting background thread 2001-12-24 13:18:04 HttpProcessor[8080][15] Starting background thread 2001-12-24 13:18:04 HttpProcessor[8080][16] Starting background thread 2001-12-24 13:18:04 HttpProcessor[8080][17] Starting background thread 2001-12-24 13:18:04 HttpProcessor[8080][18] Starting background thread 2001-12-24 13:18:04 HttpProcessor[8080][19] Starting background thread 2001-12-24 13:18:04 HttpProcessor[8080][20] Starting background thread Pages Requested, 1893, 37.86, 1893, 1 Errored, 1, 0.02, 1, 1 Fetched, 1892, 37.84, 1892, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 30 users: [13:19] 2001-12-24 13:19:07 HttpProcessor[8080][21] Starting background thread 2001-12-24 13:19:09 HttpProcessor[8080][22] Starting background thread 2001-12-24 13:19:09 HttpProcessor[8080][23] Starting background thread 2001-12-24 13:19:09 HttpProcessor[8080][24] Starting background thread 2001-12-24 13:19:09 HttpProcessor[8080][25] Starting background thread 2001-12-24 13:19:09 HttpProcessor[8080][26] Starting background thread 2001-12-24 13:19:09 HttpProcessor[8080][27] Starting background thread 2001-12-24 13:19:11 HttpProcessor[8080][28] Starting background thread 2001-12-24 13:19:11 HttpProcessor[8080][29] Starting background thread 2001-12-24 13:19:11 HttpProcessor[8080][30] Starting background thread Pages Requested, 1883, 37.66, 1883, 1 Errored, 2, 0.04, 2, 1 Fetched, 1881, 37.62, 1881, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 40 users: [13:20] 2001-12-24 13:20:13 HttpProcessor[8080][31] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][32] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][33] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][34] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][35] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][36] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][37] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][38] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][39] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][40] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][41] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][42] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][43] Starting background thread 2001-12-24 13:20:13 HttpProcessor[8080][44] Starting background thread $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ And why do we have 45 threads, here, to service 40 clients? $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Pages Requested, 1741, 34.82, 1741, 1 Errored, 0, 0.00, 0, 1 Fetched, 1741, 34.82, 1741, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 50 users: [13:21] 2001-12-24 13:21:16 HttpProcessor[8080][45] Starting background thread 2001-12-24 13:21:16 HttpProcessor[8080][46] Starting background thread 2001-12-24 13:21:16 HttpProcessor[8080][47] Starting background thread 2001-12-24 13:21:16 HttpProcessor[8080][48] Starting background thread 2001-12-24 13:21:16 HttpProcessor[8080][49] Starting background thread 2001-12-24 13:21:22 HttpProcessor[8080][50] Starting background thread $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Especially since now we are back to needing 51 threads to service 50 clients? $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Pages Requested, 1861, 37.22, 1861, 1 Errored, 2, 0.04, 2, 1 Fetched, 1859, 37.18, 1859, Avg Bytes per Page, 303, 6.06, 303, "*****************************" $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ At this point, I paused a few minutes to examine the numbers and scratch my head. Then I started a second run, identical to the first one. I got these results: $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Run B: started 13:26 Run [/BM/Minimal.jsp] Test run with 1 users: [13:26] Pages Requested, 1263, 25.26, 1263, 1 Errored, 0, 0.00, 0, 1 Fetched, 1263, 25.26, 1263, Pages Read, 1263, 25.26, 1263, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 2 users: [13:27] Pages Requested, 1127, 22.54, 1127, 1 Errored, 1, 0.02, 1, 1 Fetched, 1126, 22.52, 1126, Pages Read, 1126, 22.52, 1126, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 4 users: [13:28] Pages Requested, 970, 19.40, 970, 1 Errored, 0, 0.00, 0, 1 Fetched, 970, 19.40, 970, Pages Read, 970, 19.40, 970, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 7 users: [13:29] Pages Requested, 868, 17.36, 868, 1 Errored, 0, 0.00, 0, 1 Fetched, 868, 17.36, 868, Pages Read, 868, 17.36, 868, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 10 users: [13:30] Pages Requested, 714, 14.28, 714, 1 Errored, 0, 0.00, 0, 1 Fetched, 714, 14.28, 714, Pages Read, 714, 14.28, 714, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 20 users: [13:31] Pages Requested, 584, 11.68, 584, 1 Errored, 0, 0.00, 0, 1 Fetched, 584, 11.68, 584, Pages Read, 584, 11.68, 584, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 30 users: [13:32] Pages Requested, 477, 9.54, 477, 1 Errored, 0, 0.00, 0, 1 Fetched, 477, 9.54, 477, Pages Read, 477, 9.54, 477, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 40 users: [13:33] Pages Requested, 379, 7.58, 379, 1 Errored, 0, 0.00, 0, 1 Fetched, 379, 7.58, 379, Pages Read, 379, 7.58, 379, Avg Bytes per Page, 303, 6.06, 303, "*****************************" Test run with 50 users: [13:34] $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Up to here, we have been running with the same 51 threads (0 to 50) that were created from the first "run" of set of tests. But then, all of a sudden roughly half way through this test with 50 clients, Tomcat seems to decide it needs more threads. $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ 2001-12-24 13:34:26 HttpProcessor[8080][51] Starting background thread 2001-12-24 13:34:26 HttpProcessor[8080][52] Starting background thread 2001-12-24 13:34:26 HttpProcessor[8080][53] Starting background thread 2001-12-24 13:34:26 HttpProcessor[8080][54] Starting background thread 2001-12-24 13:34:26 HttpProcessor[8080][55] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][56] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][57] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][58] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][59] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][60] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][61] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][62] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][63] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][64] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][65] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][66] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][67] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][68] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][69] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][70] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][71] Starting background thread 2001-12-24 13:34:28 HttpProcessor[8080][72] Starting background thread 2001-12-24 13:34:30 HttpProcessor[8080][73] Starting background thread 2001-12-24 13:34:30 HttpProcessor[8080][74] Starting background thread 2001-12-24 13:34:30 HttpConnector[8080] No processor available, rejecting this connection 2001-12-24 13:34:30 HttpConnector[8080] No processor available, rejecting this connection 2001-12-24 13:34:30 HttpConnector[8080] No processor available, rejecting this connection $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ It runs out of threads, presumably. It rejects connections. AND YET we see no HTTP errors in the log?? I guess a rejected connection doesn't register to the stress tool as an error...it just tries again. $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Pages Requested, 202, 4.04, 202, 1 Errored, 0, 0.00, 0, 1 Fetched, 202, 4.04, 202, Pages Read, 202, 4.04, 202, Avg Bytes per Page, 303, 6.06, 303, "*****************************" $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Nevertheless, look at the ABYSMAL number of pages per second being served up by Tomcat at this point! FOUR per second! $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ **************** END OF FILE ******************** -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>