I noticed this functionality just now.. thanks a lot! On 08/08/2011 12:38 PM, Jay McCarthy wrote: > Your wish is my command. > > On Mon, Aug 8, 2011 at 10:00 AM, Robby Findler > <ro...@eecs.northwestern.edu> wrote: >> PS: I'm also happy if this class of tests only emails the responsible >> person, and not the pusher. >> >> Robby >> >> On Mon, Aug 8, 2011 at 10:59 AM, Robby Findler >> <ro...@eecs.northwestern.edu> wrote: >>> I like the two-times-in-a-row thought. >>> >>> FWIW, please try to avoid race conditions of the second kind. >>> >>> I think the drracket test suites are special because they fail >>> not-so-often and I don't actually know how to fix them. If either of >>> those weren't true then I'd say they should just not run in drdr. (So >>> the race-condition/using the same file thing fails this test.) >>> >>> Robby >>> >>> On Mon, Aug 8, 2011 at 10:56 AM, Vincent St-Amour <stamo...@ccs.neu.edu> >>> wrote: >>>> I love DrDr, but there's a small thing that annoys me about it. >>>> >>>> Some tests are prone to intermittent failures. For example, some >>>> benchmarks need to create a file, and several benchmarks share the >>>> same file, which leads to race conditions. Similarly, some DrRacket >>>> tests sometimes fail for focus reasons. >>>> >>>> So, whenever someone pushes, they may get failures from these tests, >>>> then have go look at the actual errors, and try to figure out if they >>>> actually broke something or not. >>>> >>>> (Or, they ignore these failures, which is bad.) >>>> >>>> Here are two potential solutions. Let's assume that I just pushed >>>> something, and a test started failing. >>>> >>>> - Have DrDr send me email for every push about the broken test for as >>>> long as it fails. If I get email more than once, it's likely that I >>>> actually broke something. If I only get email once, the problem went >>>> away on its own, and was likely an intermittent failure. >>>> >>>> - Have the possiblity to flag some tests as intermittent (something >>>> like `drdr:random'), and only report failures for these tests if >>>> they fail twice in a row. This would reduce the amount of noise, >>>> since I expect most of these tests to pass most of the time. Actual >>>> breakage would still be detected, since it's unlikely that such >>>> failures would go away on their own. Detection would happen one push >>>> late, but that shouldn't be too much of an issue. >>>> >>>> Or, maybe only notify the pusher after two failures in a row, but >>>> notify the responsible person right away. >>>> >>>> Any thoughts? >>>> >>>> Vincent >>>> _________________________________________________ >>>> For list-related administrative tasks: >>>> http://lists.racket-lang.org/listinfo/dev >>>> >> _________________________________________________ >> For list-related administrative tasks: >> http://lists.racket-lang.org/listinfo/dev > >
_________________________________________________ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
