Summary: * If you are writing code in a test to wait for something to happen in wall-clock time...
* make sure that you are only doing this in setup or teardown, to prevent a permanent hang in the test suite; and * make sure that you provide a really generous amount of time--say, 10 or 20 times what you seem to require locally--before you make the test give up and fail. Negative corollaries: * Don't write tests that use wall-clock time to actually verify the performance of anything. * Don't be stingy with the wall-clock time before your test gives up on some setup or teardown happening. Give it *plenty* of time. Rationale: * Parallel tests stress system resources a *lot* more than we are used to. Specifically, production buildbot slaves scheduled to be purchased will run *24* test systems in parallel on the same machine (with parallel postgres instances, memcaches, rabbitmqs, and so on). This introduces a whole lot of variability in terms of resource contention. We're going from barely being able to tax a two core machine to really pushing a 12 core hyperthreaded machine. * Using wall-clock time to verify something is fragile anyway. Test something else related instead (e.g., the number of queries performed). Thanks for your attention. Let me know of your concerns. Gary _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
