fyi -Some failures will happen on the Macs because test_shell sets/restores user settings on startup/shutdown, so having more then one running can cause things to fail as one exits changing state on another that's running. It's on my list to move into a helper app so it can be done around the whole setup instead, it just hasn't bubble up in the priority list yet.
TVL On Thu, Apr 9, 2009 at 12:06 PM, Pam Greene <[email protected]> wrote: > On Mon, Apr 6, 2009 at 6:57 PM, David Levin <[email protected]> wrote: > >> >> >> On Mon, Apr 6, 2009 at 6:24 PM, Ojan Vafai <[email protected]> wrote: >> >>> run_webkit_test.sh now runs cpus+1 test_shells for Release builds. Please >>> keep an eye out over the next couple days for test flakyness that may have >>> resulted from this. >>> >> >> Nice job! >> >> >>> >>> Release tests on a dual core now take about half the time they used to. >>> There's still a lot of room for improvement and I'm a bit burnt out on this >>> stuff, so if anyone is willing to help that would be much appreciated. Here >>> are the remaining obvious things we could do to make a significant >>> performance improvement: >>> >>> 1. Test and turn on parallelizing for Debug builds >>> 2. Get 4 or 8 core webkit buildbots >>> 3. Shard LayoutTests/fast and LayoutTests/http. Right now, in order to >>> reduce test flakiness, we bucket tests by directory and run all those tests >>> in the same process (thanks to dlevin for this idea!). The problem is that >>> we're left with two very large buckets that can be further broken down. The >>> work of breaking them down further is trivial (just add the directory names >>> to a list in run_webkit_tests.py), the bigger problem is that some flakiness >>> starts to appear in the fast/http tests when we break them down further. So, >>> we need people to figure out what the source of the flakiness is and deal >>> with it appropriately. >>> >> >> For #3, an alternative may be to sort "http" tests to be first and don't >> break it down further. ("http" is less than one quarter of the time on OSX >> at least, so you can still scale up to quad core.) Also, I think fast (and >> dom) can be broken down into the 1st sub dir level without increased >> flakiness. >> >> So this may be an easy gain without having to figure out lots of test >> depedencies (which can be a bit painful). >> > > > On that note, though, it would be amazing if someone wanted to figure out > the interdependencies. We have three run_webkit_tests otpions available to > help: --randomize-order, which runs the tests in a random > order;--run-singly, which launches a fresh test_shell for each test; and > --num-test-shells, which sets how many test_shell threads to run at once. > It'll be time-consuming (--run-singly is especially slow), and there will be > some work involved in comparing the results to figure out which test(s) are > causing problems, but it would be a valuable thing. And no programing > knowledge required. > > - Pam > > > >> If we did all of the above, I expect we would see at least another factor >>> of two performance improvement. >>> >>> Let me know if you want to help out with any of this. >>> >>> Ojan >>> >>> >>> >> >> >> > > > > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
