On 5/18/12 7:45 AM, Roman Werpachowski wrote:
On Fri, 18 May 2012 15:30:09 +1200, "Richard O'Keefe"<o...@cs.otago.ac.nz>
wrote:
The claim was and remains solely that
THE TIME DIFFERENCE BETWEEN *ALGORITHMS*
can be bigger than
THE TIME DIFFERENCE BETWEEN *LANGUAGES*
and is in this particular case.
Yes, but aren't the differences in the same ballpark, and if we want
to compare *languages*, we should use identical algorithms to make the
comparison fair.
"Fair" in what sense? That is, what _exactly_ are you hoping to compare?
If the goal is to benchmark the implementation of the runtime, VM, or
built-in types, then requiring the same algorithm makes sense--- because
the algorithm is irrelevant other than to provide a bunch of calls to
the runtime/vm/etc. However, benchmarking a language's implementation in
this way is rarely that helpful. It's great for comparing CPython to
PyPy (or any other in-language cross-compiler comparison), but what
would it tell you about Haskell vs C++?
If the goal is to compare, say, production costs for a given level of
performance, then fixing the algorithm is not at all fair. The fact of
the matter is that different languages make different algorithms easier
to (a) implement, and (b) discover/identify/generalize. Thus, when it
comes to real-world software, the language that makes it easy to
implement good algorithms has a major advantage--- an advantage which is
being specifically ignored by fixing the algorithm aforehand.
In order for "fair" to have any meaning whatsoever, we must first
specify what is being compared, so that we know what it is that things
are supposed to be fair with regard to.
--
Live well,
~wren
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
