[Mail I send to Dylan tends to bounce so I'm using both email addresses I have for him and hoping that he's subscribed to ghc-bugs in case both of those addresses fail...]
Dylan Thurston <[EMAIL PROTECTED]> writes: > Ken Shan told me about QuickCheck, which seems like the right way to > test such things. Incorporating QuickCheck into ghc or hugs test > suites is more work than I want to do right now, but it would be a > nice project sometime. I seem to remember being told that the GHC folk had started using QC but now preferred not to. I never heard the reason but I presume it's because there's many ways for a buggy Haskell compiler to 'correctly' print True without executing every single line of code correctly but many fewer ways for a buggy Haskell compiler to correctly print [0.0,0.0998334,0.198669,0.29552,0.389418,0.479426,0.564642,0.644218,0.717356,0.783327,0.841471] without executing every line of code correctly. This isn't to say that QuickCheck is bad (in fact, I think it's a great tool) just that it is better at detecting errors in libraries than in the compiler used to compile the libraries. A useful thing to do would be to adapt the QuickCheck approach so that instead of generating a bunch of Bools you generate a bunch of pairs of values which must be equal. For example, instead of checks _which boil down to_ code like this: print $ and [ a + b == a - (-b) | (a,b) <- testinputs ] you should use checks which boil down to code like this: print [ (a + b, a - (-b)) | (a,b) <- testinputs ] (Of course, you want to achieve this whilst maintaining most of QuickCheck's general look and feel - an exercise left for the reader.) Even better would be to be able to run in three different modes of operation: 1) Given two methods m1 and m2 of generating each answer (in the example, these are 'a+b' and 'a-(-b)', check that m1 and m2 generate the same answers as each other. This is (a special case of) the standard QuickCheck behaviour. 2) Given a previously generated file of answers, check that m1 generates those answers and check that m2 generates those answers. 3) Generate a file of sample answers for use in mode 2. [Note that you want to run the checker in mode 1 when doing this to check that your sample answers make some sort of sense. Ideally, you'd generate the file on a separate Haskell compiler...] I wonder if someone out there who teaches Haskell and/or software engineering could assign this project to their students. It'd be incredibly useful to have such a tool and to have extensive compiler-independent test-suites for Haskell98 (and common extensions) and a test harness (like the current GHC test harness) which runs these tests at regular intervals. And I think it'd be incredibly good for students to know how to go about setting up such a test suite: how to test, how not to test, how to construct a good test framework, etc. -- Alastair Reid [EMAIL PROTECTED] http://www.cs.utah.edu/~reid/ _______________________________________________ Glasgow-haskell-bugs mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
