Jan, should this block the release? From what I can tell, it should.
On 20 Mar 2010, at 11:32, Robert Dionne <[email protected]>
wrote:
On Mar 19, 2010, at 7:25 PM, Jan Lehnardt wrote:
On 19 Mar 2010, at 18:07, J Chris Anderson wrote:
On Mar 19, 2010, at 11:43 AM, Paul Davis wrote:
On Fri, Mar 19, 2010 at 2:02 PM, Jan Lehnardt <[email protected]>
wrote:
On 19 Mar 2010, at 12:50, Noah Slater wrote:
On 19 Mar 2010, at 17:11, Jan Lehnardt wrote:
We want to test the CouchDB code, not the browser's HTTP
handling.
Sure, but as one of CouchDB's primary interfaces is the
browser, it seems to makes sense that we would want to test how
this works. Testing from the browser allows us to test for and
catch problems introduced by caching, etc - which is what our
real world users would be running into.
Unless I'm missing something?
I fully agree, but we should have a separate browser interaction
suite for that. The test suite is a very untypical browser
client and
doesn't really test real-world browser use-cases.
Cheers
Jan
--
+a bajillion.
I prefer the browser tests because I'm much happier with JavaScript.
I'm not saying we should get rid of the browser tests. But
intermittent errors
in the current test suite are not to be worried about to block a
release.
I agree with all the comments about separation of tests and so
forth. This particular changes test is not intermittent, it
consistently fails (on my machine), enough that it's a pleasant
surprise when it succeeds. When running from the CLI I get the
following:
not ok 10 changes expected '3', got '1'
When running in FF I also get the message above and occasionally:
• Exception raised: {"message":"JSON.parse","fileName":"http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11
.0","lineNumber":154,"stack":"(false)@http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11.0:154\u000arun(-2)@http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11.0:83\u000a
"}
I haven't looked into it closely to find the root cause, it might
just be the test, but it's definitely not intermittent. From the CLI
it happens almost always
If we want proper browser client testing, we'd need an additional
test suite
that covers common and uncommon use-cases. I believe the current test
suite is as untypical as a browser client can be.
Cheers
Jan
--
But maybe I'm crazy
I think its important to maintain *some* tests in the browser to
test
its ability to use CouchDB as a client, but we should put more work
into separating API tests and core tests.
Also, Zed Shaw has a very informative (and colorful) description of
confounding factors [1]. Its about two thirds of the way down
under a
heading of "Confounding, Confounding, Confounding."
http://www.zedshaw.com/essays/programmer_stats.html
