Hey, I made some basic servlets to test throughputs for HttpClient, HttpCore and sun's sun.net.www.protocol.http.HttpURLConnection (or something along those lines, I don't have an API at hand now).
I basically just put up an apache session listening on port 8580, and then Tomcat on port 80. In tomcat I dumped 3 servlets, each using one of the 3 methods to retrieve a 51kb page from the apache session. The benchmarks was done with "ab" with 10 second timeouts. The results are as follows (in byte readings, not bit readings). Doing a direct request to apache I get a throughput of about 33mb/s. Using HttpURLConnection, 4.7MB/s. Using HttpCore 3.6MB/s Using HttpClient 6.03mb/s The request times were in general best with apache/HttpClient, maxing at 38ms for the longest request. This was done at a concurrency of 1 connection. Pushing this up to 5, the HttpCore's connection times stayed the same, but throughput dropped to 980kb/s. HttpClient's connection times now came to the same as HttpCore (230ms for longest request), but it's throughput increased to 7.1mb/s. This is all good and well, but my question is this. Are there any ways I can increase these throughputs? Before just jumping into optimizations suggestions and referals to documentation, could someone perhaps just enlighten me into why these speeds are so different. And especially so low? I am obviously using HttpCore wrong for it to be so slow (I mean, HttpClient is built on core, so it's impossible to be faster). It's just that the documentation for these are sparse, and I'm having a very hard time finding my way around. And beyond this, Tomcat itself gives me fantastic speeds reading static data. Where are the bottlenecks? The HttpCore setup I used was the "Basic GET Requests" example available on the site. I modified it a bit to allow URL input and target server specified through parameters passed to the servlet. Beyond that I didn't change anything. My assumption is that connections aren't being reused. In fact, I know this is the case, so it logically and obviously removes the validity of the above benchmarks for HttpCore. I'm not going to remove them though. I would really like to get throughputs similar to direct Apache connections (at least 33+ mbyte/s, which is round about the general limit of hard drive transfer rates). Full ethernet throughput is preferable and I'm sure possible. Downloading large files I get speeds of up to 54mbyte/s with Apache. I'm sure this is limited only to the hard drive I tested with, as it's a gigabit connection, and this is just less than half it's full potential. I haven't done tests using memory transfer. Will try it when I can diagnose the bottlenecks I currently have, and then use them as measurements to gain insights into the speed of the program itself by taking as much of the other bottlenecks in the system away as possible (in this case the limitation of data transfer rates from the hard drives). The three benchmark servlets I used are here: HttpClient: http://quintin.dev.junkmail.co.za/httpclient/Bench.java HttpCore: http://quintin.dev.junkmail.co.za/httpclient/BenchCore.java HttpURLConn: http://quintin.dev.junkmail.co.za/httpclient/BenchURL.java And a utility class I use in the above: http://quintin.dev.junkmail.co.za/httpclient/StreamBinder.java Any insights would be greatly appreciated. regards, -- Quintin Beukes --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
