Hey, That's an idea (to first get the code optimized before squeezing it into tomcat). I guess I just continued this way, as our current system is using HttpURLConnection. So I wanted to get an idea of how it will compare.
I'll take your advice on running it in separate java apps? I assume that is what you meant "direct connections", as in no proxied connections via tomcat. If someone had to take a wild guess, which of the examples of the site can I use as a base if I'm targeting performance (a good balance between throughput and high connection/second rates would be the best)? Q On Thu, Jun 12, 2008 at 11:02 PM, Oleg Kalnichevski <[EMAIL PROTECTED]> wrote: > Quintin Beukes wrote: >> >> Hey, >> >> I made some basic servlets to test throughputs for HttpClient, >> HttpCore and sun's sun.net.www.protocol.http.HttpURLConnection (or >> something along those lines, I don't have an API at hand now). >> >> I basically just put up an apache session listening on port 8580, and >> then Tomcat on port 80. In tomcat I dumped 3 servlets, each using one >> of the 3 methods to retrieve a 51kb page from the apache session. >> >> The benchmarks was done with "ab" with 10 second timeouts. The results >> are as follows (in byte readings, not bit readings). >> >> Doing a direct request to apache I get a throughput of about 33mb/s. >> Using HttpURLConnection, 4.7MB/s. >> Using HttpCore 3.6MB/s >> Using HttpClient 6.03mb/s >> >> The request times were in general best with apache/HttpClient, maxing >> at 38ms for the longest request. >> >> This was done at a concurrency of 1 connection. Pushing this up to 5, >> the HttpCore's connection times stayed the same, but throughput >> dropped to 980kb/s. HttpClient's connection times now came to the same >> as HttpCore (230ms for longest request), but it's throughput increased >> to 7.1mb/s. >> >> This is all good and well, but my question is this. Are there any ways >> I can increase these throughputs? Before just jumping into >> optimizations suggestions and referals to documentation, could someone >> perhaps just enlighten me into why these speeds are so different. And >> especially so low? >> > > I do not want to jump into conclusions here, but I seriously suspect > something is wrong with your test cases. In particular I do not understand > why you are using Tomcat to host the code your are testing. What is the > point of that? > > I made an extensive comparison of HttpCore vs HttpClient (both versions 3.1 > and 4.0). HttpCore can easily achieve a throughput of 10,000 (small) > requests per second on a fairly average hardware. > > http://wiki.apache.org/HttpComponents/HttpClient3vsHttpClient4vsHttpCore > > >> I am obviously using HttpCore wrong for it to be so slow (I mean, >> HttpClient is built on core, so it's impossible to be faster). It's >> just that the documentation for these are sparse, and I'm having a >> very hard time finding my way around. >> >> And beyond this, Tomcat itself gives me fantastic speeds reading >> static data. Where are the bottlenecks? >> >> The HttpCore setup I used was the "Basic GET Requests" example >> available on the site. I modified it a bit to allow URL input and >> target server specified through parameters passed to the servlet. >> Beyond that I didn't change anything. My assumption is that >> connections aren't being reused. In fact, I know this is the case, so >> it logically and obviously removes the validity of the above >> benchmarks for HttpCore. I'm not going to remove them though. >> >> I would really like to get throughputs similar to direct Apache >> connections (at least 33+ mbyte/s, which is round about the general >> limit of hard drive transfer rates). Full ethernet throughput is >> preferable and I'm sure possible. Downloading large files I get speeds >> of up to 54mbyte/s with Apache. I'm sure this is limited only to the >> hard drive I tested with, as it's a gigabit connection, and this is >> just less than half it's full potential. I haven't done tests using >> memory transfer. Will try it when I can diagnose the bottlenecks I >> currently have, and then use them as measurements to gain insights >> into the speed of the program itself by taking as much of the other >> bottlenecks in the system away as possible (in this case the >> limitation of data transfer rates from the hard drives). >> >> The three benchmark servlets I used are here: >> HttpClient: http://quintin.dev.junkmail.co.za/httpclient/Bench.java >> HttpCore: http://quintin.dev.junkmail.co.za/httpclient/BenchCore.java >> HttpURLConn: http://quintin.dev.junkmail.co.za/httpclient/BenchURL.java >> >> And a utility class I use in the above: >> http://quintin.dev.junkmail.co.za/httpclient/StreamBinder.java >> >> Any insights would be greatly appreciated. >> > > I'll try to look at the benchmark code over the weekend. But I would > _strongly_ suggest measuring the performance of each respective framework by > using direct connections to the target server. Once you have established the > baseline you can work on optimizing the code when running inside a servlet > container. > > Oleg > >> regards, > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Quintin Beukes --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
