On Wed, 13 Sep 2006, Heikki Toivonen wrote:
We have a handful of tests that try real connections over the web, against services we have no control over. Occasionally these fail. We have modified some tests to simply ignore some network errors. This is of course suboptimal, because it makes it hard to find out if there are real network coding errors in our code. We run the tests in several different situations (tbox runs the full suite, individuals run suite and tests individually), so it seems whatever the solution it must be contained in each individual test. That leads me to the actual proposal. Wrap each network call in a block that tries the call a few times (try once, sleep for a minute, try again, sleep 2 minutes, and repeating until we still fail after 8 minutes of sleep). I think Twisted actually provides something like this for tests that are based on twisted. If none of the attempts succeeds, report a failure. I believe this would get rid of most false errors. There have still been cases when flickr etc. have been unavailable for several hours, but I don't think we can do anything about that. Of course, the best solution would be to not make any network calls. If you have a test that does go over the network, see if you could change the test so that it does not need to do that.
+1 as long as you provide yet another command line flag to disable all such tests that access the network. This is important when no network is available or really slow. For example, running the flickr tests over dialup takes about 10 minutes.
Andi.. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev
