I totally agree with you on this one. Vendors (ie, the developers) are
the
only ones who are capable of this sort of tuning. And in large, complex
benchmark tests you will often see several vendors (hardware, software,
and database) collaborate to achieve the best results.
I would propose that EJB server vendors start working on TPC-C benchmark
testing with their products. There has been considerable time and effort
spent in making the TPC-C benchmark fair and unbiased. And the test
auditing
process is rigorous.
TPC-C not only focuses on raw performance, but on the cost of that
performance. And it gives you a fuller picture of hardware, application
server, and database required to get the performance.
Sure, everyone can do their own benchmark. Hell, I've done them. But it
would be nice to have standard, objective, audited, and published TPC-C
results for today's crop of EJB servers. I would guess that this will
eventually help "cull the herd" that is filled with too many mediocre
and non-scalable products.
-eric
Dwight Rexin wrote:
>
> There's 2 things going on here. Tim is talking about "out of the box"
> performance achievable by reasonably knowledgeable individuals. And that's
> an interesting measure in and of itself.
>
> The other is a measure of absolute best performance potential. In other
> words "What's the absolute fastest this race car can go on this race track
> in these weather conditions?" which is a very different measure. You'd want
> the chassis designer, engine designer, factory tire guys, best mechanics,
> and best drivers on hand for such an enterprise. And then it'd take
> collaboration and time to get to the absolute fastest lap times possible.
>
> Certainly, an average crew and an average driver with average tires could
> get a certain level of performance out of the race car. And that would tell
> us something, but it would be more indicative of the quality of the test
> crew and driver than the performance potential of the race car. If the goal
> is absolute fastest possible lap times you go with the pros.
>
> I wouldn't try to optimally tune a Solaris or Linux OS, or an Apache or
> Netscape web server, or an Oracle or DB2 relational engine without
> soliciting expert help. Most of the time the folks that wrote the code are
> the absolute best possible experts to have available for extreme performance
> potential tests. Assuming they're available and you can afford them. If you
> settle for something less than that you're testing something other than
> maximum performance potential.
>
> That's a personal perspective. Not a vendor position.
>
> Dwight
===========================================================================
To unsubscribe, send email to [EMAIL PROTECTED] and include in the body
of the message "signoff EJB-INTEREST". For general help, send email to
[EMAIL PROTECTED] and include in the body of the message "help".
