Some http tests make use of stateful php scritps with different tests utlizing the same scripts in some cases. Does each 'worker' get a dedicated http server instance or do they share the same http server?
On Mon, Dec 5, 2011 at 1:01 PM, Dirk Pranke <[email protected]> wrote: > We never implemented the "general way of marking subdirectories as > needing to run serially", but it would be easy to do if we needed to > [the 'http' dirs are still special-cased in the code]. > > There is code now (landed a few months ago) to control how many http > tests run in parallel separately from the main parallelism flag (so > you can run 16 workers but only 2 http tests at a time). I implemented > it but never flipped the switch because it was in the middle of Eric > flipping the bots over to NRWT in the first place. We should try > experimenting with this to see at some point once everything > stabilizes otherwise (this is the max_locked_shards() call in > manager.py; there's no command line flag). > > -- Dirk > > On Mon, Dec 5, 2011 at 11:17 AM, Ojan Vafai <[email protected]> wrote: > > Why is that? I don't know about other ports, but AFAIK, chromium writes > to a > > mock clipboard and the Apple mac port writes to a local OS clipboard > > instance instead of the global one, specifically to avoid copy/paste > tests > > interacting. Even without running tests in parallel, it's probably a good > > idea in order to avoid copy-pastes the developer is doing from affecting > the > > test run. > > > > That said, I believe we have a general way of marking subdirectories as > > needing to run serially (which is what we do for http) if there are other > > reasons we need to. > > > > On Mon, Dec 5, 2011 at 11:12 AM, David Levin <[email protected]> wrote: > >> > >> I believe there are some tests (copy/paste) that it would be very hard > to > >> fully shard due to how they work. > >> > >> dave > >> > >> > >> > >> On Mon, Dec 5, 2011 at 11:08 AM, Ojan Vafai <[email protected]> wrote: > >>> > >>> I looked at one example that didn't exit > >>> early: > http://build.webkit.org/builders/SnowLeopard%20Intel%20Release%20%28Tests%29/builds/35153/steps/layout-test/logs/stdio > >>> > >>> In that case, the http tests were the long tail and took 6 minutes > longer > >>> than all the other tests. We don't split the http tests up because > every > >>> time we've tried it's caused too much flakiness. It's unclear if the > >>> flakiness points to a bug in the test harness (e.g. in how we setup > apache) > >>> or to bugs in the tests themselves or both. If someone has time to > look into > >>> this, this is probably the biggest benefit to be found in NRWT runtime > when > >>> running tests in parallel. > >>> > >>> FYI, NRWT outputs a log of the runtime after each run: > >>> > >>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/9: > 4696 > >>> tests, 1746.63 secs > >>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/8: > 1177 > >>> tests, 1693.47 secs > >>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/3: > 1408 > >>> tests, 2033.51 secs > >>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/2: > 941 > >>> tests, 2119.65 secs > >>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/1: > 1121 > >>> tests, 2041.97 secs > >>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/0: > 1453 > >>> tests, 2515.75 secs > >>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/7: > 1189 > >>> tests, 1731.12 secs > >>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/6: > 3556 > >>> tests, 2114.37 secs > >>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/5: > 948 > >>> tests, 2097.13 secs > >>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/4: > 1411 > >>> tests, 1716.66 secs > >>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/15: > 795 > >>> tests, 2027.16 secs > >>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/14: > 1123 > >>> tests, 1732.72 secs > >>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/13: > 425 > >>> tests, 2021.25 secs > >>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/12: > 1175 > >>> tests, 1710.09 secs > >>> 2011-12-03 03:09:30,020 58036 printing.py:462 INFO worker/11: > 3462 > >>> tests, 2096.30 secs > >>> 2011-12-03 03:09:30,020 58036 printing.py:462 INFO worker/10: > 1449 > >>> tests, 1722.68 secs > >>> 2011-12-03 03:09:30,020 58036 printing.py:462 INFO 31120.45 > >>> cumulative, 1945.03 optimal > >>> > >>> That shows you that, if we fully sharded all the tests, they would in > >>> theory take 1945 seconds to run, but worker/0 (the worker that runs > the http > >>> tests) took 2515 seconds to run. > >>> > >>> Ojan > >>> > >>> On Mon, Dec 5, 2011 at 6:55 AM, Adam Roben <[email protected]> wrote: > >>>> > >>>> On Dec 2, 2011, at 6:55 PM, Eric Seidel wrote: > >>>> > >>>> > The SnowLeopard bot went from a 1 hr 4 min (!?!) cycle time, to 38 > min > >>>> > (still !?!). > >>>> > >>>> I suspect our Mac test bots could use a dose of RAM. Many of them only > >>>> have 3GB, since when you're running tests one by one you don't really > need > >>>> much more. > >>>> > >>>> -Adam > >>>> > >>>> _______________________________________________ > >>>> webkit-dev mailing list > >>>> [email protected] > >>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > >>> > >>> > >>> > >>> _______________________________________________ > >>> webkit-dev mailing list > >>> [email protected] > >>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > >>> > >> > > > > > > _______________________________________________ > > webkit-dev mailing list > > [email protected] > > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > > > _______________________________________________ > webkit-dev mailing list > [email protected] > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev >
_______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
