On Wed, Oct 19, 2011 at 6:12 PM, Paul Davis <[email protected]> wrote: > On Wed, Oct 19, 2011 at 5:00 PM, Sam Bisbee <[email protected]> wrote: >> On Wed, Oct 19, 2011 at 3:52 PM, Jan Lehnardt <[email protected]> wrote: >>> >>> On Oct 19, 2011, at 21:25 , Sam Bisbee wrote: >>> >>>> Yeah, it looks like it's just an unstable test. Which makes me wonder >>>> why we even have it or include it in the release process. >>> >>> It is only unstable in certain environments and we are constantly improving >>> all tests to be more stable in all environments, but the browser world is >>> harsh :) >> >> I'm all for improvement. :) But is this improvement being tracked >> anywhere, JIRA or otherwise? And if not, then could it be? I'm sure >> that folks (self included) would be willing to pitch in if we knew >> what had to be done. >> > > There was a thread about splitting Futon tests out and making them > proper CLI tests that are run as part of `make check` and become > actual release blockers. I have often and loudly voiced my support of > this approach. I refer you to the number of varying reports for 1.1.1 > with various browsers as Exhibit A in favor of this switch.
Yeah, that sounds like the thread I was remembering. For what it's worth, a CLI based test system is what I was imagining as well. Take Futon out of the mix and test CouchDB. >>> I'd still consider having the test better than not having it at all :) >> >> I semi remember this being discussed before, but are the Futon tests >> meant to test CouchDB, Futon, or the integration of the two? >> > > The test suite is there to test CouchDB. There's some merit to testing > Futon/Browser access as well. Jan's recent approach to the "verify > installation" seems to me like a good way forward. We could use a > small number of tests that check various browser capabilities and what > not. But I think a large amount of the Futon test code should be > permanently removed from the browser based test runner. I think that there is merit in unit testing futon.js and jquery.couch.js, which just makes those things sound more like their own project. As a side, semi related note I have always wondered why jquery.couch.js was part of Apache CouchDB. For browser/Futon checks, it seems to me that these are more of environmental checks: is the right version of jquery being used, is it a supported browser, etc. Verifying the installation is running `make check`, turning on the server (which should be checking more permissions), and then running your software. Cheers, -- Sam Bisbee
