+1s all around, this is much needed attention! Feel free to file JIRAs for individual issues/milestones and discuss details there.
Jan -- On Mar 28, 2013, at 18:22 , Wendall Cada <[email protected]> wrote: > On 03/27/2013 02:05 PM, Wendall Cada wrote: >> In 1.3.0, there is a new part of the test suite to run the javascript tests >> from the command line. I'm running into various issues on different >> hardware/OS configurations. Mostly, tests hanging or timing out and failing. >> These are really hard to troubleshoot, as they all pass just fine if run >> individually. >> >> What I'm experimenting with today is rewriting how the tests are implemented >> to be run one at a time from a loop in bash, versus a loop in javascript. I >> think the failures I'm running into are improper setup/teardown. There may >> be an issue with rapid delete and adding a db, or rapidly starting and >> stopping couchdb, but I think this is not what's happening in my failures. >> >> The nature of spidermonkey doesn't allow for spawning threads, or >> sandboxing, etc, so it's hard looking at the test suite to see how I can >> improve running all tests. I think it's far better to have the setup spawn a >> new interpreter for each test. Tear down will kill the interpreter. >> >> Wendall > An update here. I spent some time with this yesterday. > > There are two problems here I'd like to address. > > 1. When running `make check`, the entire javascript test suite runs in a > single interpreter thread. This means that if there are any broken tests, > then entire suite bails, as it can crash the interpreter. As I said, to > address this, I simply want to refactor the test suite to spawn a new > interpreter thread for every test. This is working well for me with my quick > work yesterday, and allows me to put a small delay in between tests to ensure > proper tear down from the previous test. > > 2. Some of the tests are written in a way that can cause a race condition in > the test itself. Quite a few places exist in the current javascript tests > where infinite loops can get generated. I simply want to re-factor several of > these tests to work as expected and avoid the race conditions entirely. > > One additional thought I had was to mimic the output of the etap tests a bit > more. Show the entire path to the test file being run, with a status at the > end of the line. Change the individual test run to include the full path to > the test to be run. Additionally, the run scripts would benefit from having > some help output. I'm in the process of gathering notes for the documentation > as well. > > My thinking is that if the test suites are similar, it will be helpful to new > developers in troubleshooting issues. > > If nobody has any objections, I'd like to get started on this work. I have a > local branch in progress for this already, but it's mostly just a scratch > space for testing so far. > > Wendall
