On Sat, May 21, 2011 at 4:20 PM, Duncan Coutts <duncan.cou...@googlemail.com> wrote: > Here's the equivalent bit of my design (the TestResult is the same): > > data TestInstance > = TestInstance { > run :: IO TestResult > name :: String, > > concurrentSafe :: Bool, > expectedFail :: Bool, > > options :: [OptionDescr] > setOption :: String -> String -> Either String TestInstance > }
I cannot think of a straightforward way to implement setOption in the above design. One would have to "store" options in the run closure. A type class approach would allow the test framework to use extra fields in the record that implements the type class to store the options e.g. class TestInstance a where setOption :: String -> String -> a -> Either String TestInstance run :: a -> IO TestResult data MyTestType where options :: [MyOptionType] -- Test framework specific property :: Property -- Test framework specific instance TestInstance MyTestType where setOption k v t = case parse k v of Left -> Left "Failed to parse option" Right opt -> Right $ t { options = opt : options t } run t = runProperty (property t) (options t) --| Convert specific test type in this test framework to the test type wrapper. test prop = MyTestType [] prop > Minor difference: extra flags for indicating whether the test is safe > for concurrent execution (e.g. if it's pure or has only local side > effects) and whether the test case is in fact expected to fail (so > numbers of expected vs unexpected failures can be reported). I prefer "exclusive" to "concurrentSafe", as there might be tests that are concurrency safe but should still be run in isolation. Not a big difference in practice though. Do we really need expectedFail, it feels quite GHC specific and there are other options, like commenting out the test or using a tags mechanism (see my reply to your other email). Here are some other attributes me might want to consider: * size - How long is this test expected to take? You might want to run all small and medium size tests on every commit but reserve large and huge tests for before a release. * timeout - The maximum amount of time the test is expected to run. The test agent should kill the test after this time has passed. Timeout could get a default value based on size. Test agents should probably apply some sort of timeout even if we don't let users specify it on a per test basis. Johan _______________________________________________ cabal-devel mailing list cabal-devel@haskell.org http://www.haskell.org/mailman/listinfo/cabal-devel