On Thu, Apr 9, 2009 at 9:11 AM, Thomas Van Lenten <[email protected]>wrote:
> fyi -Some failures will happen on the Macs because test_shell sets/restores > user settings on startup/shutdown, so having more then one running can cause > things to fail as one exits changing state on another that's running. It's > on my list to move into a helper app so it can be done around the whole > setup instead, it just hasn't bubble up in the priority list yet. > When/If you do this, if you did it in WebKitTools, it would <singing>awesome</singing>. > TVL > > > > On Thu, Apr 9, 2009 at 12:06 PM, Pam Greene <[email protected]> wrote: > >> On Mon, Apr 6, 2009 at 6:57 PM, David Levin <[email protected]> wrote: >> >>> >>> >>> On Mon, Apr 6, 2009 at 6:24 PM, Ojan Vafai <[email protected]> wrote: >>> >>>> run_webkit_test.sh now runs cpus+1 test_shells for Release builds. >>>> Please keep an eye out over the next couple days for test flakyness that >>>> may >>>> have resulted from this. >>>> >>> >>> Nice job! >>> >>> >>>> >>>> Release tests on a dual core now take about half the time they used to. >>>> There's still a lot of room for improvement and I'm a bit burnt out on this >>>> stuff, so if anyone is willing to help that would be much appreciated. Here >>>> are the remaining obvious things we could do to make a significant >>>> performance improvement: >>>> >>>> 1. Test and turn on parallelizing for Debug builds >>>> 2. Get 4 or 8 core webkit buildbots >>>> 3. Shard LayoutTests/fast and LayoutTests/http. Right now, in order to >>>> reduce test flakiness, we bucket tests by directory and run all those tests >>>> in the same process (thanks to dlevin for this idea!). The problem is that >>>> we're left with two very large buckets that can be further broken down. The >>>> work of breaking them down further is trivial (just add the directory names >>>> to a list in run_webkit_tests.py), the bigger problem is that some >>>> flakiness >>>> starts to appear in the fast/http tests when we break them down further. >>>> So, >>>> we need people to figure out what the source of the flakiness is and deal >>>> with it appropriately. >>>> >>> >>> For #3, an alternative may be to sort "http" tests to be first and don't >>> break it down further. ("http" is less than one quarter of the time on OSX >>> at least, so you can still scale up to quad core.) Also, I think fast (and >>> dom) can be broken down into the 1st sub dir level without increased >>> flakiness. >>> >>> So this may be an easy gain without having to figure out lots of test >>> depedencies (which can be a bit painful). >>> >> >> >> On that note, though, it would be amazing if someone wanted to figure out >> the interdependencies. We have three run_webkit_tests otpions available to >> help: --randomize-order, which runs the tests in a random >> order;--run-singly, which launches a fresh test_shell for each test; and >> --num-test-shells, which sets how many test_shell threads to run at once. >> It'll be time-consuming (--run-singly is especially slow), and there will be >> some work involved in comparing the results to figure out which test(s) are >> causing problems, but it would be a valuable thing. And no programing >> knowledge required. >> >> - Pam >> >> >> >>> If we did all of the above, I expect we would see at least another >>>> factor of two performance improvement. >>>> >>>> Let me know if you want to help out with any of this. >>>> >>>> Ojan >>>> >>>> >>>> >>> >>> >>> >> >> >> >> > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
