On Wed, 26 Jan 2000, Tarsa, Greg wrote:
> Now you
> end up with a permanent entry in the database labeled "non-compliant
> results." (and there were complaints that this was too heavy handed!)
>
I agree, they're not as harsh as they should be, but SPEC (possibly
naively), gives vendors the benefit of the doubt. The assumption is
that people will benchmark honestly. It has only been recently that
SPEC has started to make cheating carry a stiffer penalty.
Meanwhile, SPEC hasn't ignored the possibility of cheating, and has
taken steps to prevent it. For example, the CPU2000 tools MD5
checksum the results of a benchmark run. This will guarantee that
benchmark results are not tampered with after they are run. Like
everything, it can be circumvented, but it will stop the less dedicated
cheating.
A while ago, someone gave me an interesting way of looking at the world.
Everyone is a raider or a trader.
The raiders are people that pillage and take from all those around them.
The traders are people that share and exchange ideas freely. They
have a very open and trusting atmosphere.
On the whole, traders are more productive and efficient than raiders,
however, when raiders meet the traders, the raiders always win.
I suspect, because SPEC grew up around engineers, it assumed everyone
was a trader. It has loose rules with the goal of information &
knowledge exchange.
I'm not sure of the history of TPC, but I believe it grew up from
marketing & sales. Basically, it believes that everyone is a raider.
Trust no one.
If everyone is honest, SPEC's mindset works best. However, if people are
dishonest, TPC's model works best.
> The biggest impediment to cheating in the consortia, in my
> opinion, is that everyone is watching and that most people care about being
> fair.
Agreed. Sun, for example, is probably still living down the java
benchmark special of a few years back. Cheating in benchmarks, like
in everything, only hurts the person doing it.
--Phil
Compaq: High Performance Server Division/Benchmark Performance Engineering
---------------- Alpha, The Fastest Processor on Earth --------------------
[EMAIL PROTECTED] |C|O|M|P|A|Q| [EMAIL PROTECTED]
------------------- See the results at www.spec.org -----------------------
**********************************************************
To unsubscribe from this list, send mail to
[EMAIL PROTECTED] with the following text in the
*body* (*not* the subject line) of the letter:
unsubscribe gnhlug
**********************************************************
