On Mon, Dec 5, 2011 at 1:01 PM, Dirk Pranke <dpra...@chromium.org> wrote: > We never implemented the "general way of marking subdirectories as > needing to run serially", but it would be easy to do if we needed to > [the 'http' dirs are still special-cased in the code].
Does it also special-case the storage tests that may collide? I'm thinking particularly of the filesystem/filewriter tests, but anything that deals with quota may also have issues. > There is code now (landed a few months ago) to control how many http > tests run in parallel separately from the main parallelism flag (so > you can run 16 workers but only 2 http tests at a time). I implemented > it but never flipped the switch because it was in the middle of Eric > flipping the bots over to NRWT in the first place. We should try > experimenting with this to see at some point once everything > stabilizes otherwise (this is the max_locked_shards() call in > manager.py; there's no command line flag). > > -- Dirk > > On Mon, Dec 5, 2011 at 11:17 AM, Ojan Vafai <o...@chromium.org> wrote: >> Why is that? I don't know about other ports, but AFAIK, chromium writes to a >> mock clipboard and the Apple mac port writes to a local OS clipboard >> instance instead of the global one, specifically to avoid copy/paste tests >> interacting. Even without running tests in parallel, it's probably a good >> idea in order to avoid copy-pastes the developer is doing from affecting the >> test run. >> >> That said, I believe we have a general way of marking subdirectories as >> needing to run serially (which is what we do for http) if there are other >> reasons we need to. >> >> On Mon, Dec 5, 2011 at 11:12 AM, David Levin <le...@google.com> wrote: >>> >>> I believe there are some tests (copy/paste) that it would be very hard to >>> fully shard due to how they work. >>> >>> dave >>> >>> >>> >>> On Mon, Dec 5, 2011 at 11:08 AM, Ojan Vafai <o...@chromium.org> wrote: >>>> >>>> I looked at one example that didn't exit >>>> early: http://build.webkit.org/builders/SnowLeopard%20Intel%20Release%20%28Tests%29/builds/35153/steps/layout-test/logs/stdio >>>> >>>> In that case, the http tests were the long tail and took 6 minutes longer >>>> than all the other tests. We don't split the http tests up because every >>>> time we've tried it's caused too much flakiness. It's unclear if the >>>> flakiness points to a bug in the test harness (e.g. in how we setup apache) >>>> or to bugs in the tests themselves or both. If someone has time to look >>>> into >>>> this, this is probably the biggest benefit to be found in NRWT runtime when >>>> running tests in parallel. >>>> >>>> FYI, NRWT outputs a log of the runtime after each run: >>>> >>>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/9: 4696 >>>> tests, 1746.63 secs >>>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/8: 1177 >>>> tests, 1693.47 secs >>>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/3: 1408 >>>> tests, 2033.51 secs >>>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/2: 941 >>>> tests, 2119.65 secs >>>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/1: 1121 >>>> tests, 2041.97 secs >>>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/0: 1453 >>>> tests, 2515.75 secs >>>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/7: 1189 >>>> tests, 1731.12 secs >>>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/6: 3556 >>>> tests, 2114.37 secs >>>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/5: 948 >>>> tests, 2097.13 secs >>>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/4: 1411 >>>> tests, 1716.66 secs >>>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/15: 795 >>>> tests, 2027.16 secs >>>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/14: 1123 >>>> tests, 1732.72 secs >>>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/13: 425 >>>> tests, 2021.25 secs >>>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/12: 1175 >>>> tests, 1710.09 secs >>>> 2011-12-03 03:09:30,020 58036 printing.py:462 INFO worker/11: 3462 >>>> tests, 2096.30 secs >>>> 2011-12-03 03:09:30,020 58036 printing.py:462 INFO worker/10: 1449 >>>> tests, 1722.68 secs >>>> 2011-12-03 03:09:30,020 58036 printing.py:462 INFO 31120.45 >>>> cumulative, 1945.03 optimal >>>> >>>> That shows you that, if we fully sharded all the tests, they would in >>>> theory take 1945 seconds to run, but worker/0 (the worker that runs the >>>> http >>>> tests) took 2515 seconds to run. >>>> >>>> Ojan >>>> >>>> On Mon, Dec 5, 2011 at 6:55 AM, Adam Roben <aro...@apple.com> wrote: >>>>> >>>>> On Dec 2, 2011, at 6:55 PM, Eric Seidel wrote: >>>>> >>>>> > The SnowLeopard bot went from a 1 hr 4 min (!?!) cycle time, to 38 min >>>>> > (still !?!). >>>>> >>>>> I suspect our Mac test bots could use a dose of RAM. Many of them only >>>>> have 3GB, since when you're running tests one by one you don't really need >>>>> much more. >>>>> >>>>> -Adam >>>>> >>>>> _______________________________________________ >>>>> webkit-dev mailing list >>>>> webkit-dev@lists.webkit.org >>>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev >>>> >>>> >>>> >>>> _______________________________________________ >>>> webkit-dev mailing list >>>> webkit-dev@lists.webkit.org >>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev >>>> >>> >> >> >> _______________________________________________ >> webkit-dev mailing list >> webkit-dev@lists.webkit.org >> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev >> > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev