Historically, test suite failures on Futon were release blockers. I'd
like to get some consensus on if this is still the case. It's
imperative we have a formal rule here, to help me, and to help in-
between development. Once we've cleared that up, I suggest that we
make this part of the documented release procedure. That way, asking
for a release signifies that the test suites pass acceptably in the
environments we have agreed the must pass in. I think this will reduce
a significant amount of friction for future releases. If I get
blocked, or stalled, in any way - that should indicate a process
failure. Thoughts?
On 19 Mar 2010, at 23:25, Jan Lehnardt <[email protected]> wrote:
On 19 Mar 2010, at 18:07, J Chris Anderson wrote:
On Mar 19, 2010, at 11:43 AM, Paul Davis wrote:
On Fri, Mar 19, 2010 at 2:02 PM, Jan Lehnardt <[email protected]>
wrote:
On 19 Mar 2010, at 12:50, Noah Slater wrote:
On 19 Mar 2010, at 17:11, Jan Lehnardt wrote:
We want to test the CouchDB code, not the browser's HTTP
handling.
Sure, but as one of CouchDB's primary interfaces is the browser,
it seems to makes sense that we would want to test how this
works. Testing from the browser allows us to test for and catch
problems introduced by caching, etc - which is what our real
world users would be running into.
Unless I'm missing something?
I fully agree, but we should have a separate browser interaction
suite for that. The test suite is a very untypical browser client
and
doesn't really test real-world browser use-cases.
Cheers
Jan
--
+a bajillion.
I prefer the browser tests because I'm much happier with JavaScript.
I'm not saying we should get rid of the browser tests. But
intermittent errors
in the current test suite are not to be worried about to block a
release.
If we want proper browser client testing, we'd need an additional
test suite
that covers common and uncommon use-cases. I believe the current test
suite is as untypical as a browser client can be.
Cheers
Jan
--
But maybe I'm crazy
I think its important to maintain *some* tests in the browser to
test
its ability to use CouchDB as a client, but we should put more work
into separating API tests and core tests.
Also, Zed Shaw has a very informative (and colorful) description of
confounding factors [1]. Its about two thirds of the way down
under a
heading of "Confounding, Confounding, Confounding."
http://www.zedshaw.com/essays/programmer_stats.html
