Isaac Gouy <[EMAIL PROTECTED]> writes: > Again, you seem happy to say they are overrated and dismiss them > without actually having looked - that is not "scientific"!
Yes, my critique is not up to the standards I measure the shootout with. But my main point is: Performance has so many (sometimes hard to recognize) parameters that's really easy to draw false conclusions From some benchmark data. If you are interested for the best performance for you task at hand, you have to do benchmarks within your execution environment and you have to construct benachmarks that really resemble your task. Different OS, different CPU, even different cache sizes, different alogrithms and so on. All these things really matter and if you change any of these parameters the whole picture may change (on OS A, CPU B compiler S for language X may be produce code that executes faster for a benchmark than the code that is produced by compiler T for language Y on the same system, but on CPU C the combination T/Y may be better). One extra problem: As far as I can see you only compare freely available language implementations (and this point is overssen by many people -- I'm a big open source fan, but more often than not there is one or more commercial language implementation that produce better results than any of the freely available implementations). These games are funny and sometimes it's even possible to come to helpful conclusions given the data. But many people I talked with understand these shootouts as hard facts, they assume language X is always better than language Y and don't take into account for example that they are developing for a Sparc cluster while using benchmarks for Intel/AMD. OK, so I would say I don't really have any problems with the shootout but with the perception of these shootouts by many people. And that's why I dislike these shootouts in general. -- Until the next mail..., Stefan.
pgp8NSSB7jKTb.pgp
Description: PGP signature
_______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
