My company is a subscriber of the Giga Information Group. One of my co-workers just forwarded me a new paper by David Mastrobattista with the above title. Distribution of the document is restricted to only subscribers, so I can't post it on linuxvm.org, but if you or your company has a subscription, it can be accessed at http://www.gigaweb.com/search/1,2197,results,00.html?iContentID=260852
The abstract of the paper on Giga's public section states "Proceed with extreme caution when evaluating performance benchmarks that span different computing platforms. We highly recommend clients monitor production workloads and use their own real-world data whenever possible to gain the most accurate results...." To quote part of the paper... "as platform technologies and architectures advance, the means by which these systems can be effectively measured - even within a given architecture, let alone across platform architectures - becomes more challenging." Sound familiar to anyone? :) He gives recommendations on how to do the best job you can of figuring out how workload on one architecture translates to another architecture, but emphasizes that it's never going to be an exact science, to say the least. And, near the end he offers this advice, which I agree with completely: "...hold the vendor to any presales capacity claims." Nice job, David (I know you're out there!). Mark Post
