On Sat, Oct 25, 2014 at 1:20 PM, K. Macy <km...@freebsd.org> wrote:
>>>> Alan also suggested against integrating the test suite as-is, because as 
>>>> he said, "Remember, don't run these tests on a production system.  They 
>>>> WILL cause panics and deadlocks, and they may cause data loss too.”
>>>> Cheers,
>>>> -Garrett
>>> Wait, we want to sweep those bugs under the rug?  What exactly is wrong 
>>> with making a test harness that can very easily reproduce a known problem?  
>>> The chances are that anyone will dive into it once the bug is easily 
>>> reproducible.
>>         Sweeping bugs under the rug is not what I plan on doing; I’m marking 
>> these as expected failures, as opposed to having them continually panic a 
>> machine. Once a ZFS dev takes a look at the issue and resolves them, then 
>> the ZFS dev can remove the “bail” calls I’m adding to the testcases.
>> Cheers,
>> -Garrett
> Yes, disabling tests that fail leads to an ineffectual test suite. A
> test suite that never has any failures is not very useful. However,
> there are two factors to take in to account in this context:
> a) frequent failures can lead users to stop running a test suite
> leading to further regressions
> b) long-term repeated failures can desensitize users leading them to
> ignore *new* failures facilitating further regressions
> Thus it's really a question of what context you're talking about
> running the test suite in. For purposes of Jenkins we want full
> visibility in to what is passing and what is failing and how long this
> has been going on for.

Agreed. We're talking about doing an OpenZFS bug-tracker and maybe we
could have tests post there automatically, once the bug is opened. I
am going to put this on the board at MeetBSD Ca. next week, so we can
have a discussion about it in real-time with some of the people

freebsd-virtualization@freebsd.org mailing list
To unsubscribe, send any mail to 

Reply via email to