I believe there are some tests (copy/paste) that it would be very hard to fully shard due to how they work.
dave On Mon, Dec 5, 2011 at 11:08 AM, Ojan Vafai <o...@chromium.org> wrote: > I looked at one example that didn't exit early: > http://build.webkit.org/builders/SnowLeopard%20Intel%20Release%20%28Tests%29/builds/35153/steps/layout-test/logs/stdio > > In that case, the http tests were the long tail and took 6 minutes longer > than all the other tests. We don't split the http tests up because every > time we've tried it's caused too much flakiness. It's unclear if the > flakiness points to a bug in the test harness (e.g. in how we setup apache) > or to bugs in the tests themselves or both. If someone has time to look > into this, this is probably the biggest benefit to be found in NRWT runtime > when running tests in parallel. > > FYI, NRWT outputs a log of the runtime after each run: > > 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/9: 4696 > tests, 1746.63 secs > 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/8: 1177 > tests, 1693.47 secs > 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/3: 1408 > tests, 2033.51 secs > 2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/2: 941 > tests, 2119.65 secs > 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/1: 1121 > tests, 2041.97 secs > 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/0: 1453 > tests, 2515.75 secs > 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/7: 1189 > tests, 1731.12 secs > 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/6: 3556 > tests, 2114.37 secs > 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/5: 948 > tests, 2097.13 secs > 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/4: 1411 > tests, 1716.66 secs > 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/15: 795 > tests, 2027.16 secs > 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/14: 1123 > tests, 1732.72 secs > 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/13: 425 > tests, 2021.25 secs > 2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/12: 1175 > tests, 1710.09 secs > 2011-12-03 03:09:30,020 58036 printing.py:462 INFO worker/11: 3462 > tests, 2096.30 secs > 2011-12-03 03:09:30,020 58036 printing.py:462 INFO worker/10: 1449 > tests, 1722.68 secs > 2011-12-03 03:09:30,020 58036 printing.py:462 INFO 31120.45 cumulative, > 1945.03 optimal > > That shows you that, if we fully sharded all the tests, they would in > theory take 1945 seconds to run, but worker/0 (the worker that runs the > http tests) took 2515 seconds to run. > > Ojan > > On Mon, Dec 5, 2011 at 6:55 AM, Adam Roben <aro...@apple.com> wrote: > >> On Dec 2, 2011, at 6:55 PM, Eric Seidel wrote: >> >> > The SnowLeopard bot went from a 1 hr 4 min (!?!) cycle time, to 38 min >> (still !?!). >> >> I suspect our Mac test bots could use a dose of RAM. Many of them only >> have 3GB, since when you're running tests one by one you don't really need >> much more. >> >> -Adam >> >> _______________________________________________ >> webkit-dev mailing list >> webkit-dev@lists.webkit.org >> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev >> > > > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > >
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
