Re: [webkit-dev] Anyone else having these problems with run-webkit-tests?
On Mar 26, 2014, at 16:40 , Bem Jones-Bey bjone...@adobe.commailto:bjone...@adobe.com wrote: On Mar 24, 2014, at 15:54 , Dirk Pranke dpra...@chromium.orgmailto:dpra...@chromium.org wrote: On Mon, Mar 24, 2014 at 3:32 PM, Bem Jones-Bey bjone...@adobe.commailto:bjone...@adobe.com wrote: On Mar 24, 2014, at 14:54 , Dirk Pranke dpra...@chromium.orgmailto:dpra...@chromium.org wrote: On Sun, Mar 23, 2014 at 12:52 PM, Benjamin Poulain benja...@webkit.orgmailto:benja...@webkit.org wrote: Same here :( On 3/23/14, 12:15 PM, Darin Adler wrote: When I use run-webkit-tests to run the entire test suite, on a debug build of TOT WebKit, on Mavericks, I’m having the following problems: - Towards the end of the run, the tests run slowly, over a second per test on my 3.5GHz i7 iMac with tons of memory, which is about 10X too slow I think. The sample shows the vast majority of the time is spent in JavaScript garbage collection. This is running mostly the svg/custom tests. Yep. The time from 31k to 33k is a long as from 0 to 31k :( - Towards the end of the run, instead of the 8 parallel copies of DumpRenderTree, only 1 copy of DumpRenderTree seems to run. This is running mostly the svg/custom tests. I think the problem is we can only run DumpRenderTree in parallel on different folders. Toward the end, everything left is one or two slow folders. The problem is partially that the sharding is folder-at-a-time, partially that the svg folder is big and slow, and partially that svg comes near the end of the alphabet (i.e., we don't start the big slow directory until close to the end of run, so there ends up being one big long pole). At some point we added code to run-webkit-tests to have a list of slow directories that got started towards the beginning. I don't remember if we did that before or after the Blink fork, but it would be easy to port it over from Blink if it was after. I'm pretty sure this was after the Blink fork. I did a quick look through the history, and I found this: http://src.chromium.org/viewvc/blink?revision=152697view=revision Is that the correct change? If it will make tests run faster in WebKit land as well, I'm more than happy to port it myself. :-) Yup, that's the one. Obviously the whole virtual test suite thing is less relevant, but you can use it for any directory you want. Well, I tried that patch, replacing virtual with svg, and it doesn't seem to have any effect on the speed of the run for me. But I'm also not 100% sure that I am seeing the slowness issue that others are reporting. (I also get 18 copies of DRT, not 8, so) In case it matters, I ran the tests with WTR, and I see the issue; however, it's not svg tests for me: it's http and ietestcenter tests, and a couple of others. So it doesn't look like it's related to a single batch of tests. - Bem ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Anyone else having these problems with run-webkit-tests?
On Mar 24, 2014, at 15:54 , Dirk Pranke dpra...@chromium.orgmailto:dpra...@chromium.org wrote: On Mon, Mar 24, 2014 at 3:32 PM, Bem Jones-Bey bjone...@adobe.commailto:bjone...@adobe.com wrote: On Mar 24, 2014, at 14:54 , Dirk Pranke dpra...@chromium.orgmailto:dpra...@chromium.org wrote: On Sun, Mar 23, 2014 at 12:52 PM, Benjamin Poulain benja...@webkit.orgmailto:benja...@webkit.org wrote: Same here :( On 3/23/14, 12:15 PM, Darin Adler wrote: When I use run-webkit-tests to run the entire test suite, on a debug build of TOT WebKit, on Mavericks, I’m having the following problems: - Towards the end of the run, the tests run slowly, over a second per test on my 3.5GHz i7 iMac with tons of memory, which is about 10X too slow I think. The sample shows the vast majority of the time is spent in JavaScript garbage collection. This is running mostly the svg/custom tests. Yep. The time from 31k to 33k is a long as from 0 to 31k :( - Towards the end of the run, instead of the 8 parallel copies of DumpRenderTree, only 1 copy of DumpRenderTree seems to run. This is running mostly the svg/custom tests. I think the problem is we can only run DumpRenderTree in parallel on different folders. Toward the end, everything left is one or two slow folders. The problem is partially that the sharding is folder-at-a-time, partially that the svg folder is big and slow, and partially that svg comes near the end of the alphabet (i.e., we don't start the big slow directory until close to the end of run, so there ends up being one big long pole). At some point we added code to run-webkit-tests to have a list of slow directories that got started towards the beginning. I don't remember if we did that before or after the Blink fork, but it would be easy to port it over from Blink if it was after. I'm pretty sure this was after the Blink fork. I did a quick look through the history, and I found this: http://src.chromium.org/viewvc/blink?revision=152697view=revision Is that the correct change? If it will make tests run faster in WebKit land as well, I'm more than happy to port it myself. :-) Yup, that's the one. Obviously the whole virtual test suite thing is less relevant, but you can use it for any directory you want. Well, I tried that patch, replacing virtual with svg, and it doesn't seem to have any effect on the speed of the run for me. But I'm also not 100% sure that I am seeing the slowness issue that others are reporting. (I also get 18 copies of DRT, not 8, so) - Bem ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Anyone else having these problems with run-webkit-tests?
Maybe we need to find out why we are getting this slow garbage collection, then. The “one folder at a time” explains part of this, but not why svg/custom ends up being so slow. Not super-urgent, I guess, but might be a real GC bug of some sort? Some of the svg/custom tests use a debugging API to force synchronous GC. So, this might be a GC bug — but it might also be an inefficiency in how some tests are written. Geoff ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Anyone else having these problems with run-webkit-tests?
On Sun, Mar 23, 2014 at 12:52 PM, Benjamin Poulain benja...@webkit.orgwrote: Same here :( On 3/23/14, 12:15 PM, Darin Adler wrote: When I use run-webkit-tests to run the entire test suite, on a debug build of TOT WebKit, on Mavericks, I’m having the following problems: - Towards the end of the run, the tests run slowly, over a second per test on my 3.5GHz i7 iMac with tons of memory, which is about 10X too slow I think. The sample shows the vast majority of the time is spent in JavaScript garbage collection. This is running mostly the svg/custom tests. Yep. The time from 31k to 33k is a long as from 0 to 31k :( - Towards the end of the run, instead of the 8 parallel copies of DumpRenderTree, only 1 copy of DumpRenderTree seems to run. This is running mostly the svg/custom tests. I think the problem is we can only run DumpRenderTree in parallel on different folders. Toward the end, everything left is one or two slow folders. The problem is partially that the sharding is folder-at-a-time, partially that the svg folder is big and slow, and partially that svg comes near the end of the alphabet (i.e., we don't start the big slow directory until close to the end of run, so there ends up being one big long pole). At some point we added code to run-webkit-tests to have a list of slow directories that got started towards the beginning. I don't remember if we did that before or after the Blink fork, but it would be easy to port it over from Blink if it was after. There is also the --experimental-fully-parallel flag that you can use, which shards a test at a time and get you the full parallelization until the last 8 tests. The downside to this is that you'll probably have a lot more flakiness. Hope this is useful, -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Anyone else having these problems with run-webkit-tests?
On Mar 24, 2014, at 14:54 , Dirk Pranke dpra...@chromium.orgmailto:dpra...@chromium.org wrote: On Sun, Mar 23, 2014 at 12:52 PM, Benjamin Poulain benja...@webkit.orgmailto:benja...@webkit.org wrote: Same here :( On 3/23/14, 12:15 PM, Darin Adler wrote: When I use run-webkit-tests to run the entire test suite, on a debug build of TOT WebKit, on Mavericks, I’m having the following problems: - Towards the end of the run, the tests run slowly, over a second per test on my 3.5GHz i7 iMac with tons of memory, which is about 10X too slow I think. The sample shows the vast majority of the time is spent in JavaScript garbage collection. This is running mostly the svg/custom tests. Yep. The time from 31k to 33k is a long as from 0 to 31k :( - Towards the end of the run, instead of the 8 parallel copies of DumpRenderTree, only 1 copy of DumpRenderTree seems to run. This is running mostly the svg/custom tests. I think the problem is we can only run DumpRenderTree in parallel on different folders. Toward the end, everything left is one or two slow folders. The problem is partially that the sharding is folder-at-a-time, partially that the svg folder is big and slow, and partially that svg comes near the end of the alphabet (i.e., we don't start the big slow directory until close to the end of run, so there ends up being one big long pole). At some point we added code to run-webkit-tests to have a list of slow directories that got started towards the beginning. I don't remember if we did that before or after the Blink fork, but it would be easy to port it over from Blink if it was after. I'm pretty sure this was after the Blink fork. I did a quick look through the history, and I found this: http://src.chromium.org/viewvc/blink?revision=152697view=revision Is that the correct change? If it will make tests run faster in WebKit land as well, I'm more than happy to port it myself. :-) - Bem ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Anyone else having these problems with run-webkit-tests?
On Mon, Mar 24, 2014 at 3:32 PM, Bem Jones-Bey bjone...@adobe.com wrote: On Mar 24, 2014, at 14:54 , Dirk Pranke dpra...@chromium.org wrote: On Sun, Mar 23, 2014 at 12:52 PM, Benjamin Poulain benja...@webkit.orgwrote: Same here :( On 3/23/14, 12:15 PM, Darin Adler wrote: When I use run-webkit-tests to run the entire test suite, on a debug build of TOT WebKit, on Mavericks, I’m having the following problems: - Towards the end of the run, the tests run slowly, over a second per test on my 3.5GHz i7 iMac with tons of memory, which is about 10X too slow I think. The sample shows the vast majority of the time is spent in JavaScript garbage collection. This is running mostly the svg/custom tests. Yep. The time from 31k to 33k is a long as from 0 to 31k :( - Towards the end of the run, instead of the 8 parallel copies of DumpRenderTree, only 1 copy of DumpRenderTree seems to run. This is running mostly the svg/custom tests. I think the problem is we can only run DumpRenderTree in parallel on different folders. Toward the end, everything left is one or two slow folders. The problem is partially that the sharding is folder-at-a-time, partially that the svg folder is big and slow, and partially that svg comes near the end of the alphabet (i.e., we don't start the big slow directory until close to the end of run, so there ends up being one big long pole). At some point we added code to run-webkit-tests to have a list of slow directories that got started towards the beginning. I don't remember if we did that before or after the Blink fork, but it would be easy to port it over from Blink if it was after. I'm pretty sure this was after the Blink fork. I did a quick look through the history, and I found this: http://src.chromium.org/viewvc/blink?revision=152697view=revision Is that the correct change? If it will make tests run faster in WebKit land as well, I'm more than happy to port it myself. :-) Yup, that's the one. Obviously the whole virtual test suite thing is less relevant, but you can use it for any directory you want. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
[webkit-dev] Anyone else having these problems with run-webkit-tests?
Hi folks. When I use run-webkit-tests to run the entire test suite, on a debug build of TOT WebKit, on Mavericks, I’m having the following problems: - Towards the end of the run, the tests run slowly, over a second per test on my 3.5GHz i7 iMac with tons of memory, which is about 10X too slow I think. The sample shows the vast majority of the time is spent in JavaScript garbage collection. This is running mostly the svg/custom tests. - Towards the end of the run, instead of the 8 parallel copies of DumpRenderTree, only 1 copy of DumpRenderTree seems to run. This is running mostly the svg/custom tests. - Six audio tests always fail, and as a side effect of their failure, they dump expected.png files into my LayoutTests/platform/mac/webaudio directory. — Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Anyone else having these problems with run-webkit-tests?
Same here :( On 3/23/14, 12:15 PM, Darin Adler wrote: When I use run-webkit-tests to run the entire test suite, on a debug build of TOT WebKit, on Mavericks, I’m having the following problems: - Towards the end of the run, the tests run slowly, over a second per test on my 3.5GHz i7 iMac with tons of memory, which is about 10X too slow I think. The sample shows the vast majority of the time is spent in JavaScript garbage collection. This is running mostly the svg/custom tests. Yep. The time from 31k to 33k is a long as from 0 to 31k :( - Towards the end of the run, instead of the 8 parallel copies of DumpRenderTree, only 1 copy of DumpRenderTree seems to run. This is running mostly the svg/custom tests. I think the problem is we can only run DumpRenderTree in parallel on different folders. Toward the end, everything left is one or two slow folders. - Six audio tests always fail, and as a side effect of their failure, they dump expected.png files into my LayoutTests/platform/mac/webaudio directory. I often have flaky tests in webaudio but not systematic failures. Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Anyone else having these problems with run-webkit-tests?
On Mar 23, 2014, at 12:52 PM, Benjamin Poulain benja...@webkit.org wrote: On 3/23/14, 12:15 PM, Darin Adler wrote: - Towards the end of the run, the tests run slowly, over a second per test on my 3.5GHz i7 iMac with tons of memory, which is about 10X too slow I think. The sample shows the vast majority of the time is spent in JavaScript garbage collection. This is running mostly the svg/custom tests. Yep. The time from 31k to 33k is a long as from 0 to 31k :( - Towards the end of the run, instead of the 8 parallel copies of DumpRenderTree, only 1 copy of DumpRenderTree seems to run. This is running mostly the svg/custom tests. I think the problem is we can only run DumpRenderTree in parallel on different folders. Toward the end, everything left is one or two slow folders. Maybe we need to find out why we are getting this slow garbage collection, then. The “one folder at a time” explains part of this, but not why svg/custom ends up being so slow. Not super-urgent, I guess, but might be a real GC bug of some sort? — Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
