On Aug 9, 2011, at 4:48 AM, Paul Davis wrote: > On Tue, Aug 9, 2011 at 2:40 AM, Robert Dionne > <[email protected]> wrote: >>> >>> >>> Also, I've been thinking more and more about beefing up the JavaScript >>> test suite runner and moving more of our browser tests over to >>> dedicated code in those tests. If anyone's interested in hacking on >>> some C and JavaScript against an HTTP API, let me know. >> >> >> Paul, >> >> Jan and I talked about this a few times and I started a branch[1] along >> that idea. So far all I did was make a copy of the then current Futon tests >> into >> test/javascript/test and started looking at the small handful that fail. >> >> The browser tests are great (any test is good) but they have too many >> browser dependent quirks, or at least I assume that because of the pleasant >> surprise >> one gets when they all run. So I think the goal of these runner tests would >> be some sort of official HTTP API suite that's part of "make check". Would >> you agree? >> If so I'm happy to take this on. >> >> Also I've found eunit to be helpful in BigCouch and wondering how hard it >> would be to support eunit in couchdb. Having tests in the modules is very >> good for not >> only testing but also to help with reading and understanding what the code >> does. >> >> Bob >> >> >> [1] https://github.com/bdionne/couchdb/tree/cli-tests >> >> > > Bob, > > Exactly what I was thinking. By having our test suite have as little > code between the socket and the the test as possible we can ensure > that the tests are testing CouchDB and not some random change in the > behavior of our favorite web browser. I would definitely expect these > tests to be part of make check and hence part of the release > procedure. > > My current half formed thoughts are to basically split our test suite > into two halves. Tests that are in Erlang and are testing internals, > and tests that go through the HTTP interface. I love me some Erlang, > but I've not been think of an elegant way to make it easy to run lots > of HTTP tests. > > As to eunit, I'm not sure. I'm really not a huge fan of it, especially > mixing implementation and test code. I know rebar can separate them, > so its at least possible to get around that. I'd like to have a > unified environment for Erlang tests though. And TAP at seems like > it'd be easier to interface with non-Erlang tooling if we ever get > around to build matrices and what not. But I'm not opposed to it on > religious grounds if that's what people want to contribute.
OTP ships with eunit_surefire[1] so interfacing with general test infrastructures isn't really an issue. I had to dig a little deeper to find a decent TAP consumer for Jenkins and ended up executing the tests from a Perl script using TAP::Harness::JUnit. I'll grant that the TAP output is easier for a human to digest than the eunit output. I imagine testing the private module functions is a topic for a good flamewar. I find it useful, but maybe that's a sign I've got too much logic in a particular module. Either way, a viable alternative to writing unit tests that execute in the browser is something this project really needs. Cheers, Adam [1]: http://www.erlang.org/doc/man/eunit_surefire.html
