[Haskell-cafe] RE: proposal: HaBench, a Haskell Benchmark Suite
I'm delighted that you are interested in this benchmarking stuff. Much needed. Thank you! | So I still very much like the approach taken by the venerable nofib | suite where it includes not only the "real" programs, but also the | microbenchmarks and the small programs; you don't have to use these in | published results, but they're invaluable to us compiler developers, and | having a shared framework for all the benchmarks makes things a lot easier. Yes yes. It's *essential* to retain the micro-benchmarks. They often show up in high relief a performance regression that would be hidden or much less prominent in a big program. The three-way split imaginary/spectral/real has served us well. Let's keep it! Simon ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
On 25/06/2010 14:01, Andy Georges wrote: Right. I have the distinct feeling this is a major lack in the Haskell world. SPEC evolved over time to include larger benchmarks that still excercise the various parts of the hardware, such that the benchmarks does not achieve suddenly a large improvement on a new architecture/implementation due to e.g. a larger cache and the working sets remain in the cache for the entire execution. The Haskell community has nothing that remotely resembles a decent suite. You could do experiments and show that over 10K iterations, the average execution time per iteration goes from 500ms to 450ms, but what does this really mean? We have a need not just for plain Haskell benchmarks, but benchmarks that test - GHC extensions, so we can catch regressions - parallelism (see nofib/parallel) - concurrency (see nofib/smp) - the garbage collector (see nofib/gc) I tend to like quantity over quality: it's very common to get just one benchmark in the whole suite that shows a regression or exercises a particular corner of the compiler or runtime. We should only keep benchmarks that have a tunable input size, however. I would suggest that the first category might be made up of microbenchmarks, as I do not think it really is needed for performance per se. However, the other categories really need long-running benchmarks, that use (preferable) heaps of RAM, even when they're well tuned. The categories you mention aren't necessarily distinct: we have several microbenchmarks that run for a long time and use a lot of heap. For testing the GC, as with other parts of the system, we need both microbenchmarks and larger programs. Different people want different things from a benchmark suite: if you're demonstrating the efficacy of an optimisation or a language implementation, then you want just the "real" benchmarks, whereas if you're a compiler developer you probably want the microbenchmarks too, because investigating their performance tends to be more tractable, and the hope is that if you optimise all the microbenchmarks then the real programs will take care of themselves (it doesn't always work like that, but it's a good way to start). So I still very much like the approach taken by the venerable nofib suite where it includes not only the "real" programs, but also the microbenchmarks and the small programs; you don't have to use these in published results, but they're invaluable to us compiler developers, and having a shared framework for all the benchmarks makes things a lot easier. If we made it *really easy* for people to submit their own programs (e.g. using 'darcs send') then we might get a lot of contributions, from which we could cherry-pick for the "real" benchmark suite, while keeping most/all of the submissions for the "full" suite. Similarly, we should make it really easy for people to run the benchmark suite on their own machines and compilers - make the tools cabal-installable, with easy ways to generate results. I'm definitely interested. If I want to make a strong case for my current research, I really need benchmarks that can be used. Additionally, coming up with a good suite, characterising it, can easily result is a decent paper, that is certain to be cited numerous times. I think it would have to be a group/community effort though. I've looked through the apps on the Haskell wiki pages, but there's not much usable there, imho. I'd like to illustrate this by the dacapo benchmark suite [2,3] example. It took a while, but now everybody in the Java camp is (or should be) using these benchmarks. Saying that we just do not want to do this, is simply not plausible to maintain. Oh, don't get me wrong - we absolutely do want to do this, it's just difficult to get motivated to actually do it. It's great that you're interested, I'll help any way that I can, and I'll start by digging up some suggestions for benchmarks. Cheers, Simon -- Andy [1] Computer systems are dynamical systems, Todd Mytkowicz, Amer Diwan, and Elizabeth Bradley, Chaos 19, 033124 (2009); doi:10.1063/1.3187791 (14 pages). [2] The DaCapo benchmarks: java benchmarking development and analysis, Stephen Blackburn et al, OOPSLA 2006 [3] Wake up and smell the coffee: evaluation methodology for the 21st century, Stephen Blackburn et al, CACM 2008 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
Hi Simon et al, On Jun 25, 2010, at 14:39 PM, Simon Marlow wrote: > On 25/06/2010 00:24, Andy Georges wrote: > >> >> Are there any inputs available that allow the real part of the suite >> to run for a sufficiently long time? We're going to use criterion in >> any case given our own expertise with rigorous benchmarking [3,4], >> but since we've made a case in the past against short running apps on >> managed runtime systems [5], we'd love to have stuff that runs at >> least in the order of seconds, while doing useful things. All >> pointers are much appreciated. > > The short answer is no, although some of the benchmarks have tunable input > sizes (mainly the spectral ones) and you can 'make mode=slow' to run those > with larger inputs. > > More generally, the nofib suite really needs an overhaul or replacement. > Unfortunately it's a tiresome job and nobody really wants to do it. There > have been various abortive efforts, including nobench and HaBench. Meanwhile > we in the GHC camp continue to use nofib, mainly because we have some tool > infrastructure set up to digest the results (nofib-analyse). Unfortunately > nofib has steadily degraded in usefulness over time due to both faster > processors and improvements in GHC, such that most of the programs now run > for less than 0.1s and are ignored by the tools when calculating averages > over the suite. Right. I have the distinct feeling this is a major lack in the Haskell world. SPEC evolved over time to include larger benchmarks that still excercise the various parts of the hardware, such that the benchmarks does not achieve suddenly a large improvement on a new architecture/implementation due to e.g. a larger cache and the working sets remain in the cache for the entire execution. The Haskell community has nothing that remotely resembles a decent suite. You could do experiments and show that over 10K iterations, the average execution time per iteration goes from 500ms to 450ms, but what does this really mean? > We have a need not just for plain Haskell benchmarks, but benchmarks that test > > - GHC extensions, so we can catch regressions > - parallelism (see nofib/parallel) > - concurrency (see nofib/smp) > - the garbage collector (see nofib/gc) > > I tend to like quantity over quality: it's very common to get just one > benchmark in the whole suite that shows a regression or exercises a > particular corner of the compiler or runtime. We should only keep benchmarks > that have a tunable input size, however. I would suggest that the first category might be made up of microbenchmarks, as I do not think it really is needed for performance per se. However, the other categories really need long-running benchmarks, that use (preferable) heaps of RAM, even when they're well tuned. > Criterion works best on programs that run for short periods of time, because > it runs the benchmark at least 100 times, whereas for exercising the GC we > really need programs that run for several seconds. I'm not sure how best to > resolve this conflict. I'm not sure about this. Given the fact that there's quite some non-determinism in modern CPUs and that computer systems seem to behave chaotically [1], I definitely see the need to employ Criterion for longer running applications as well. It might not need 100 executions, or multiple iterations per execution (incidentally, those iterations, can they be said to be independent?), but somewhere around 20 - 30 seems to be a minimum. > > Meanwhile, I've been collecting pointers to interesting programs that cross > my radar, in anticipation of waking up with an unexpectedly free week in > which to pull together a benchmark suite... clearly overoptimistic! But I'll > happily pass these pointers on to anyone with the inclination to do it. I'm definitely interested. If I want to make a strong case for my current research, I really need benchmarks that can be used. Additionally, coming up with a good suite, characterising it, can easily result is a decent paper, that is certain to be cited numerous times. I think it would have to be a group/community effort though. I've looked through the apps on the Haskell wiki pages, but there's not much usable there, imho. I'd like to illustrate this by the dacapo benchmark suite [2,3] example. It took a while, but now everybody in the Java camp is (or should be) using these benchmarks. Saying that we just do not want to do this, is simply not plausible to maintain. -- Andy [1] Computer systems are dynamical systems, Todd Mytkowicz, Amer Diwan, and Elizabeth Bradley, Chaos 19, 033124 (2009); doi:10.1063/1.3187791 (14 pages). [2] The DaCapo benchmarks: java benchmarking development and analysis, Stephen Blackburn et al, OOPSLA 2006 [3] Wake up and smell the coffee: evaluation methodology for the 21st century, Stephen Blackburn et al, CACM 2008 ___ Haskell-Cafe mailing list Haskell
[Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
On 25/06/2010 00:24, Andy Georges wrote: I've picked up the HaBench/nofib/nobench issue again, needing a decent set of real applications to do some exploring of what people these days call split-compilation. We have a framework that was able to explore GCC optimisations [1] -- the downside there was the dependency of these optimisations on each other, requiring them to be done in certain order -- for a multi-objective search space, and extended this to exploring a JIT compiler [2] for Java in our case -- which posed its own problems. Going one step further, we'd like to explore the tradeoffs that can be made when compiling on different levels: source to bytecode (in some sense) and bytecode to native. Given that LLVM is quicly becoming a state-of-the-art framework and with the recent GHC support, we figured that Haskell would be an excellent vehicle to conduct our exploration and research (and the fact that some people at our lab have a soft spot for Haskell helps too). Which brings me back to benchmarks. Are there any inputs available that allow the real part of the suite to run for a sufficiently long time? We're going to use criterion in any case given our own expertise with rigorous benchmarking [3,4], but since we've made a case in the past against short running apps on managed runtime systems [5], we'd love to have stuff that runs at least in the order of seconds, while doing useful things. All pointers are much appreciated. The short answer is no, although some of the benchmarks have tunable input sizes (mainly the spectral ones) and you can 'make mode=slow' to run those with larger inputs. More generally, the nofib suite really needs an overhaul or replacement. Unfortunately it's a tiresome job and nobody really wants to do it. There have been various abortive efforts, including nobench and HaBench. Meanwhile we in the GHC camp continue to use nofib, mainly because we have some tool infrastructure set up to digest the results (nofib-analyse). Unfortunately nofib has steadily degraded in usefulness over time due to both faster processors and improvements in GHC, such that most of the programs now run for less than 0.1s and are ignored by the tools when calculating averages over the suite. We have a need not just for plain Haskell benchmarks, but benchmarks that test - GHC extensions, so we can catch regressions - parallelism (see nofib/parallel) - concurrency (see nofib/smp) - the garbage collector (see nofib/gc) I tend to like quantity over quality: it's very common to get just one benchmark in the whole suite that shows a regression or exercises a particular corner of the compiler or runtime. We should only keep benchmarks that have a tunable input size, however. Criterion works best on programs that run for short periods of time, because it runs the benchmark at least 100 times, whereas for exercising the GC we really need programs that run for several seconds. I'm not sure how best to resolve this conflict. Meanwhile, I've been collecting pointers to interesting programs that cross my radar, in anticipation of waking up with an unexpectedly free week in which to pull together a benchmark suite... clearly overoptimistic! But I'll happily pass these pointers on to anyone with the inclination to do it. Cheers, Simon Or if any of you out there have (recent) apps with inputs that are open source ... let us know. -- Andy [1] COLE: Compiler Optimization Level Exploration, Kenneth Hoste and Lieven Eeckhout, CGO 2008 [2] Automated Just-In-Time Compiler Tuning, Kenneth Hoste, Andy Georges and Lieven Eeckhout, CGO 2010 [3] Statistically Rigorous Java Performance Evaluation, Andy Georges, Dries Buytaert and Lieven Eeckhout, OOPSLA 2007 [4] Java Performance Evaluation through Rigorous Replay Compilation, Andy Georges, Lieven Eeckhout and Dries Buytaert, OOPSLA 2008 [5] How Java Programs Interact with Virtual Machines at the Microarchitectural Level, Lieven Eeckhout, Andy Georges, Koen De Bosschere, OOPSLA 2003 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
On 6/24/10 4:24 PM, Andy Georges wrote: > Or if any of you out there have (recent) apps with inputs that are open source ... let us know. Hi Andy.. you could run the hledger benchmarks, roughly like so: $ cabal install tabular $ darcs get --lazy http://joyful.com/repos/hledger $ cd hledger $ make quickbench and edit bench.tests for different/slower/faster tests. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
On Wed, Jan 31, 2007 at 02:58:20AM +0300, Bulat Ziganshin wrote: > Hello David, > > Saturday, January 27, 2007, 8:48:39 PM, you wrote: > > > I agree that numerics and Doubles are very important, but am of the > > opinion that we'll be better off if we (try to?) restrict ourselves to > > code that is really used by someone who really cares about performance > > enough to optimize it. > > he should write an imperative code and in last Feb we've discussed why > GHC don't generate good code for low-level imperative Haskell > programs. you can see this discussion in GHC maillist. the problem is > well known since GHC birth and GHC HQ plan to fix it by writing their > own low-level optimizer. of course, it's hard to approach gcc, but > results close to Ocaml may be accomplished; the only problem is lack of > spare developer's hands :) It all depends on what the someone is doing. If he's doing linear algebra, for example, he shouldn't be writing low-level code for anything but O(N) operations in any language, because that's stupid. And factors of two in the speed of O(N) operations don't much matter. But it's still nice to see that Haskell implementations can do a decent job at O(N) applications, and there are certainly cases a clean optimized Haskell library should be able to outperform a clean optimized C++ library (if you can agree with me that template expressions are dirty), due to the possibility of array fusion. -- David Roundy Department of Physics Oregon State University ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
On Wed, Jan 31, 2007 at 01:56:32AM +0300, Bulat Ziganshin wrote: > Hello David, > > Friday, January 26, 2007, 6:23:26 PM, you wrote: > > >> performance was not very good (the OCaml version I based it on was at > >> least 10x faster). > > > I would think that what we'd want to benchmark would be clean, optimized > > actually-used code. I.e. things like Data.Bytestring, so that we could see > > so you propose to benchmark only low-level imperative code? :) > > (correct me if i'm wrong, but everything fast i've seen, including FPS, > is just imperative code) I guess it depends what you mean by imperative, and I'll admit I haven't looked at Data.Bytestring's internals recently, but most of the code in FastPackedString (its predecessor) is just using a ForeignPtr as an array, which isn't imperative at all. Certainly it (almost?) never uses a mutable variable. So what do you mean by "imperative"? In any case, I meant code that *uses* Data.Bytestring, which is certainly purely functional. -- David Roundy Department of Physics Oregon State University ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
Hello David, Saturday, January 27, 2007, 8:48:39 PM, you wrote: > I agree that numerics and Doubles are very important, but am of the opinion > that we'll be better off if we (try to?) restrict ourselves to code that is > really used by someone who really cares about performance enough to > optimize it. he should write an imperative code and in last Feb we've discussed why GHC don't generate good code for low-level imperative Haskell programs. you can see this discussion in GHC maillist. the problem is well known since GHC birth and GHC HQ plan to fix it by writing their own low-level optimizer. of course, it's hard to approach gcc, but results close to Ocaml may be accomplished; the only problem is lack of spare developer's hands :) -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
Hello David, Friday, January 26, 2007, 6:23:26 PM, you wrote: >> performance was not very good (the OCaml version I based it on was at >> least 10x faster). > I would think that what we'd want to benchmark would be clean, optimized > actually-used code. I.e. things like Data.Bytestring, so that we could see so you propose to benchmark only low-level imperative code? :) (correct me if i'm wrong, but everything fast i've seen, including FPS, is just imperative code) -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
On Fri, Jan 26, 2007 at 05:25:13PM +, Chris Kuklewicz wrote: > > I agree that common libraries like ByteString need to be well > > represented, but the original request additionally included programs > > that are representative of applications. A ray-tracer (even with a fixed > > scene and only one type of scene primitive) is a fairly nice > > approximation of a real numerical batch-oriented application while still > > being small enough to understand and modify. I expect thats why Jo > > chose it as his benchmark application in the first place. > > Writing numeric code that processes Doubles is hard to optimize. See the > shootout example for the n-body problem: I agree that numerics and Doubles are very important, but am of the opinion that we'll be better off if we (try to?) restrict ourselves to code that is really used by someone who really cares about performance enough to optimize it. Mostly because if the benchmark suite is going to serve as a useful guide for compiler optimization writers, it needs to reflect the performance of code that at least *someone* cares about. Artificial benchmarks all too often can be improved by artificial optimizations that only benefit artificial benchmarks. e.g. the famous specmark log(sqrt(x)) optimization (which is equal to 0.5*log(x), but no decent programmer would ever write that code, and it's a special case the compiler shouldn't bother looking for). -- David Roundy http://www.darcs.net ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
At Fri, 26 Jan 2007 07:23:26 -0800, David Roundy wrote: > I would think that what we'd want to benchmark would be clean, optimized > actually-used code. Maybe there should be two versions of each benchmark: 1) an clean, simple, idiomatic version, aka the code we would like to write if performance was not an issue. 2) a super-hand optimized version This would hopefully show, which benchmarks have the biggest room for improvement. And, it would provide information to the people working on the compilers as to what optimizations are actually needed for that test case. j. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
Al Falloon wrote: > David Roundy wrote: >> On Fri, Jan 26, 2007 at 10:17:28AM -0500, Al Falloon wrote: >>> Kenneth Hoste wrote: The idea is to gather a bunch of programs written in Haskell, and which are representative for the Haskell community (i.e. apps, libraries, ...). >>> A While ago I tried to write a Haskell version of John Harrops >>> ray-tracer benchmark >>> (http://www.ffconsultancy.com/free/ray_tracer/languages.html) but the >>> performance was not very good (the OCaml version I based it on was at >>> least 10x faster). >>> >>> I would be happy to contribute my code to the benchmark suite if you >>> are interested. Perhaps someone can point out obvious speed-ups that >>> I missed while trying to improve the performance. >> >> I would think that what we'd want to benchmark would be clean, optimized >> actually-used code. I.e. things like Data.Bytestring, so that we >> could see >> how compilers differed on important code, or how the code generated on >> different architectures differed. e.g. if jhc beats ghc on amd64, the >> ghc >> developers would probably be very curious as to why, and how to fix itere >> >> Code that's not been properly optimized with respect to strictness, etc, >> would fail to focus the tests on important optimizations of the compiler. >> But of course, the benchmark code should also be clean, since we want to >> ensure that our compilers are good enough that we can write useful >> beautiful code that is also fast. > > I tried to optimize it, but I couldn't approach the speed of the OCaml > version. I followed the performance tuning advice from the Wiki, and had > even resorted to writing the inner loop calculations using all unboxed > doubles, but without significant improvements. This is exactly the kind > of code that I write most often, and I would love to see improvements in > the optimizations for this kind of numerically intensive code > (especially without having to resort to compiler-specific unboxed > representations). > > I agree that common libraries like ByteString need to be well > represented, but the original request additionally included programs > that are representative of applications. A ray-tracer (even with a fixed > scene and only one type of scene primitive) is a fairly nice > approximation of a real numerical batch-oriented application while still > being small enough to understand and modify. I expect thats why Jo > chose it as his benchmark application in the first place. Writing numeric code that processes Doubles is hard to optimize. See the shootout example for the n-body problem: http://haskell.org/haskellwiki/Shootout/Nbody http://shootout.alioth.debian.org/debian/benchmark.php?test=nbody&lang=all Some of the best work on making Haskell perform wonderfully at numerics is summarized at http://haskell.org/haskellwiki/GHC/Data_Parallel_Haskell and their status report http://www.cse.unsw.edu.au/~chak/papers/CLPKM06.html ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
David Roundy wrote: On Fri, Jan 26, 2007 at 10:17:28AM -0500, Al Falloon wrote: Kenneth Hoste wrote: The idea is to gather a bunch of programs written in Haskell, and which are representative for the Haskell community (i.e. apps, libraries, ...). A While ago I tried to write a Haskell version of John Harrops ray-tracer benchmark (http://www.ffconsultancy.com/free/ray_tracer/languages.html) but the performance was not very good (the OCaml version I based it on was at least 10x faster). I would be happy to contribute my code to the benchmark suite if you are interested. Perhaps someone can point out obvious speed-ups that I missed while trying to improve the performance. I would think that what we'd want to benchmark would be clean, optimized actually-used code. I.e. things like Data.Bytestring, so that we could see how compilers differed on important code, or how the code generated on different architectures differed. e.g. if jhc beats ghc on amd64, the ghc developers would probably be very curious as to why, and how to fix it. Code that's not been properly optimized with respect to strictness, etc, would fail to focus the tests on important optimizations of the compiler. But of course, the benchmark code should also be clean, since we want to ensure that our compilers are good enough that we can write useful beautiful code that is also fast. I tried to optimize it, but I couldn't approach the speed of the OCaml version. I followed the performance tuning advice from the Wiki, and had even resorted to writing the inner loop calculations using all unboxed doubles, but without significant improvements. This is exactly the kind of code that I write most often, and I would love to see improvements in the optimizations for this kind of numerically intensive code (especially without having to resort to compiler-specific unboxed representations). I agree that common libraries like ByteString need to be well represented, but the original request additionally included programs that are representative of applications. A ray-tracer (even with a fixed scene and only one type of scene primitive) is a fairly nice approximation of a real numerical batch-oriented application while still being small enough to understand and modify. I expect thats why John chose it as his benchmark application in the first place. I think it would also be a good idea to include SPJ's web server (I think its from an STM paper). A lot of the people outside the Haskell community will be able to relate better to metrics like pages/second. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
On Fri, Jan 26, 2007 at 10:17:28AM -0500, Al Falloon wrote: > Kenneth Hoste wrote: > >The idea is to gather a bunch of programs written in Haskell, and which > >are representative for the Haskell community (i.e. apps, libraries, > >...). > > A While ago I tried to write a Haskell version of John Harrops > ray-tracer benchmark > (http://www.ffconsultancy.com/free/ray_tracer/languages.html) but the > performance was not very good (the OCaml version I based it on was at > least 10x faster). > > I would be happy to contribute my code to the benchmark suite if you are > interested. Perhaps someone can point out obvious speed-ups that I > missed while trying to improve the performance. I would think that what we'd want to benchmark would be clean, optimized actually-used code. I.e. things like Data.Bytestring, so that we could see how compilers differed on important code, or how the code generated on different architectures differed. e.g. if jhc beats ghc on amd64, the ghc developers would probably be very curious as to why, and how to fix it. Code that's not been properly optimized with respect to strictness, etc, would fail to focus the tests on important optimizations of the compiler. But of course, the benchmark code should also be clean, since we want to ensure that our compilers are good enough that we can write useful beautiful code that is also fast. Just my $0.02. -- David Roundy http://www.darcs.net ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
Kenneth Hoste wrote: The idea is to gather a bunch of programs written in Haskell, and which are representative for the Haskell community (i.e. apps, libraries, ...). A While ago I tried to write a Haskell version of John Harrops ray-tracer benchmark (http://www.ffconsultancy.com/free/ray_tracer/languages.html) but the performance was not very good (the OCaml version I based it on was at least 10x faster). I would be happy to contribute my code to the benchmark suite if you are interested. Perhaps someone can point out obvious speed-ups that I missed while trying to improve the performance. -- Alan Falloon ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
