Jason,
This is an impressive piece of work: very systematic and very
well-documented. I have several questions. I ran some similar tests
(without adjusting configuration parameters) and noticed variability as
large as 10% in throughput that surprised me, even when N = 10,000. To
reduce that, I felt compelled to use N = 100,000. Your data seems pretty
stable: did you run more tests and just truncate the reporting? Do you
think it would be worthwhile to rerun the test with N = 100,000 maybe two
or three times ( my tests took hours )?
Costin's ab data shows a ramp up: the shell script I posted a while back
was based on that approach. I noticed doing my measurements that Apache
throughput increased as C increased (up to C = 100), whereas TC
throughput did not (actually it declined). I also wonder what the
architectural explanation might be for the different scalability behavior
(maybe this is obvious, I haven't thought about it yet). I wonder if you
could (in your spare time :-)} repeat the test 3 times for C = 20, 40,
60, 80, 100 with N = 100,000. That's my plan, along with using sar and sa
with C = 1 to get CPU demand measures. Maybe others will join this little
band of performance freaks.
BTW, little credit should go to me re: ab use (or should I say "abuse").
Craig suggested using ab and Costin did it (and did it, ... :-)). Pier
"inspired" me to look at ab code and think about how it can be used as a
tool for getting information about both behavior and the resource usage
that drives it. Your stuff (along with Costin's) might be a key part of a
great guide to TCx performance and tuning ala Dean Gaudet's stuff on
(Apache?) performance. Now I need to study your stuff more closely and
plan for the next round of measurement (and, in my case, modeling).
Roy
--
Roy Wilson
E-mail: [EMAIL PROTECTED]
>>>>>>>>>>>>>>>>>> Original Message <<<<<<<<<<<<<<<<<<
On 12/5/00, 1:47:46 PM, Jason Brittain <[EMAIL PROTECTED]> wrote regarding
Benchmarks: Catalina M5 vs. Apache 1.3.12:
> I wrote up a text file about benchmarking and comparing Tomcat-4.0-M5
> (pre-release)
> and Apache 1.3.12. It's attached to this message. I wrote it for
> anyone who is interested
> (even non-Java-saavy people) to know how the raw content serving
> performance of
> Catalina and its built-in web server compares to that of Apache 1.3.12.
> It contains lots of
> information to help people understand some of the important differences
> between the two
> servers.
> Feedback, flames, reproduced test results, etc. are welcome! :)
> Cheers.
> --
> Jason Brittain
> Software Engineer, Olliance Inc. http://www.Olliance.com
> Current Maintainer, Locomotive Project http://www.Locomotive.org
