#5793: make nofib not suck
--------------------------------------+-------------------------------------
Reporter: dterei | Owner: dterei
Type: task | Status: new
Priority: normal | Milestone:
Component: NoFib benchmark suite | Version:
Keywords: | Os: Unknown/Multiple
Architecture: Unknown/Multiple | Failure: None/Unknown
Difficulty: Unknown | Testcase:
Blockedby: | Blocking: 5794
Related: |
--------------------------------------+-------------------------------------
Changes (by simonmar):
* difficulty: => Unknown
Comment:
Here are a few notes I've been collecting about what we should do with
nofib.
* Build system:
* get rid of `runstdtest`
* Make it independent of a GHC build
* Maybe use Shake instead of make (but retain the ability to compile
and run individual benchmarks, with custom options etc.)
* beef up nofib-analyse:
* generate gnuplot graphs directly
* measure std.dev. better (omit results based on std.dev. rather than <
0.2s?)
* benchmarks themselves:
* we need a suite of ~10 large real-world programs for publishing
results in papers.
* generally: add new benchmarks, retire old ones
If we modify the programs and inputs to run for longer, then do them all
at once: we can't do this incrementally, because each change invalidates
all the old logs and they have to be regenerated. Note that we need the
benchmarks to build with old compilers, so that we can run comparisons (or
at least make sure we can get results even if some of the programs fail to
compile).
--
Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/5793#comment:7>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler
_______________________________________________
Glasgow-haskell-bugs mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
