> For what it's worth, a CLI based test system is what I was imagining > as well. Take Futon out of the mix and test CouchDB.
IMO, If CouchDB is intended to be a server that can be accessed from the browser directly, then there should continue to be some kind of browser-based test suite that would serve to confirm this capability. I have been looking closely at the Futon tests in 1.1.0 for the last several days, with the idea that I might begin to clean them up a bit as time permits. I have found that, while some of these test failures are totally bogus, *some* of them actually do stem from real issues -- minor incompatibilities between CouchDB's http interface, and the internal mechanisms of modern browsers (XHR, caching, etc). These are problems that we're not going to catch with a stateless, cache-less http client running on the CLI. (I can provide examples) These issues have the potential to cause real problems for developers of real browser-based apps "in the wild". That means, there's valuable info to be gathered from the browser tests, Iff we can clean them up, and make them behave consistently; so that when they fail or succeed, we can actually trust the results. After digging around a good bit, I can see no reason why the existing tests couldn't be cleaned up and made to work correctly in all current versions of major browsers. I also see no reason why the same tests couldn't be used successfully from the CLI and `make check` as well. I do see significant benefits to using the same javascript test code in all environments we test. -Lee (irc: coltr)
