----- Original Message ----- From: "Christopher K. St. John" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Saturday, June 22, 2002 6:36 AM Subject: Re: Proposal draft for Tomcat 5.0 : benchmarks
> Bill Barker wrote: > > > > I agree with Remy > > that any single benchmark suite isn't going to tell you how your particular > > web-app will perform. > > > > Bill, > > Actually, I agree, but this isn't what that is :-) > > There's a difference between: > > a) Benchmarks for users that compare the performance of > servers against one another to help with buying > decisions. These are hard to write, hard to interpret, > and generally not very useful in predicting the > performance of a particular user's web apps. > > b) Benchmarks for developers that help answer the > (suprisingly difficult) question "did my so called > performance enhancement help or hurt?" and "have > we met our performance goals". They also help set > up a framework for talking about performance. For > example, using the Apache ab utility is a quick and > easy way to get a very skewed performance estimate. > If there's something written down, everyone has > some clue that there's a skew, and some idea about > what the skew is. > > (b) is a much easier than than (a), because the point > is to nail down the vocabulary and set up a common framework > for measuring progress, not to develop some ultimate > web-app performance benchmark for users. It's a purely > internal thing. > > This isn't something I just made up :-) It's sort of > software engineering conventional wisdom that internal > benchmarks are necessary if you're going to make performance > a development goal, and I've used them to good effect on > several commercial projects. > I stand by my +0 (:= It's a good idea, but I don't expect to personally devote much of my time to it). If you decided to make Vincent very happy by basing the suite on Cactus, you might even be able to draw some of the Cactus people into the project. > -- > Christopher St. John [EMAIL PROTECTED] > DistribuTopia http://www.distributopia.com > > -- > To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> > For additional commands, e-mail: <mailto:[EMAIL PROTECTED]> > -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
