Re: [webkit-dev] LayoutTests take too long to run
On Jul 31, 2009, at 10:25 PM, Eric Seidel wrote: 681.70s total testing time That's 11.5 minutes for every patch I want to land. (because I run the layout tests before landing anything, as part of bugzilla-tool). I'm very interested in any suggestions folks have to make that number smaller. I know Chromium runs the layout tests in parallel using their python run_webkit_tests.py harness. Try-bots would be another solution to the make sure it passes all the tests before landing. Again, very interested in suggestions as to how to improve this. We're working on try-bots for the ports of particular interest to Apple. But independent of that I think speeding up the layout tests is a worthwhile goal. Besides parallelism for better performance on multi-core systems, here are some other ideas: - Look at the slowest tests and see if they can be made to run faster (I'm running with --slowest as we speak). - Look at the subdirectories that take the most time - it's possible that in some cases we can consolidate multiple tests to make the whole test suite run faster, if sheer number of tests is the bottleneck. I think a good goal to shoot for would be for the full test suite to run in under 5 minutes on a reasonably modern laptop. That should allow for reasonable use during development with some room for growth. Regards, Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] Towards a commit-queue
I've been experimenting with a wrapper for bugzilla-tool that runs in a loop and tries to land patches [1]. Ideally, there would be enough information in Bugzilla so I could kick off this script and make tea while it does its thing. However, deciding whether a bug with an r+ patch is ready to be landed requires human intervention. I went through the queue tonight and manually marked the bugs as commit-ready by adding [commit+] to their titles. This worked well, except that it made the bug titles ugly. I think a better solution is to have flag analogous to r+ which is commit+. The flag means I certify that this patch is ready to be landed according to WebKit process. Unlike r+, this flag would be settable by any committer because it is the moral equivalent of typing svn commit. In rare cases, such as unreviewed build fixes, a patch could have commit+ without an r+. In future, future land when we have a working commit queue, the actual svn commit command might be run by a bot instead of a human. Adam [1] You can see my progress here: https://bugs.webkit.org/show_bug.cgi?id=27918 ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] parallel layout research
It reminds me a lot of the work Badros did circa 2000 [1]. That work added linear constraints to CSS and a variety of other languages. It did not show how to do flow-based layout with linear constraints: ultimately, the utility and applicability of their techniques to actual layout systems was never really understood (e.g., can we cleanly implement and specify CSS with linear constraints? quadratic ones?). I don't recall the end-to-end solving time, but, in my experience, it's not the constraint solving of stuff like finding what point of font (which is essentially what they support). The way I think about that work is that there is awkwardness with strictly flow-based layout and Badros showed a useful primitive -- linearly constrained variables -- to help with some of it. they've implemented an abstract language for describing the CSS spec for parameterizing layout. Most of my implementation work has actually been on tweaking out selectors as a way to get used to multicore performance concerns in this space :) I'm working on language subsets, not full ones, in order to experiment with actual implementations and iterate over many algorithm ideas. For layout, I started with formalizing a subset of CSS 2.1 -- essentially inlines, inline blocks, blocks, and left-floats with no margins/padding/explicit positioning. Given that subset, I found places for parallelism at the node-level (with speculation that floats don't have an impact at the local level, which seems founded). Next I'll be bouncing between expanding it and implementing and then tweaking it for various memory systems. They suggest that this model does layout of the CSS 2.1 spec in time similar to the hand-tuned sequential engine in Gecko [2] using Cilk++ to handle low-level paralleism, they've seen significant speedups beyond that for multi-core [2] That work is now moving into the implementation phase: the slides were for a model showing a minimum of what we'd expect if solving constraints was the bottleneck (so parallelizing a tree of slow constraints). Interesting, they're not -- it's stuff like dealing with text and images that come into play -- so I actually expect a much better speedup. Because of this, my first implementation step was to actually reexamine how to call into a font library like freetype2 for better caching and multicore behavior. I think parallel layout is an interesting topic to explore, in light of where CPUs are going... We have had a lot of success with improving layout performance given test cases that demonstrate slowness. Worth mentioning our interest is half in traditional browser settings (faster slashdot) and half in the mobile space. For the former, you may or may not care if opening a page is as fast as hitting the back button (though there are studies to show this has a big impact on usability in practice), but, for the latter, we want to do at least a magnitude better *everywhere*. Moh has had luck with firefox in terms of better memory usage, but I think that's telling: you need to take advantage of the hardware with the algorithms, whether it be CPU or L1. We probably can't use code in Cilk++ directly, TBB might be a more viable route in terms of licensing (and more amenable to performance tools). Currently, I benefit from Cilk most in the early prototyping phases -- I then rewrite in TBB because the compiler seems to do better and I can do better profiling. I'll be back in the bay area starting mid-September and would love to meet anybody interested in this stuff. There are approaches that might work (e.g., STM or rewriting in a pure language), but at least we're starting to get proof that multicore helps for at least one approach :) Finally, it's worth noting that I also get significant speedups by being careful about l1/l2 usage -- I doubt that style of optimization works well in a huge codebase with many contributors, so parallelization (with a sane framework like cilk/tbb) might better scale with the code over time. -- View this message in context: http://www.nabble.com/parallel-layout-research-tp24761520p24765531.html Sent from the Webkit mailing list archive at Nabble.com. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Towards a commit-queue
I went through the queue tonight and manually marked the bugs as commit-ready by adding [commit+] to their titles. This worked well, except that it made the bug titles ugly. I think a better solution is to have flag analogous to r+ which is commit+. Adam, as I suggedted previously, bugzilla supports KEYWORDs, so that would be a matter of adding a special support for bugs where patches are ready to go in. 'checkin-needed' keyword would work , i believe. What do you think ? these are the currently supported keywords: https://bugs.webkit.org/describekeywords.cgi -- --Antonio Gomes ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Towards a commit-queue
On Sat, Aug 1, 2009 at 11:43 AM, tonikitoo (Antonio Gomes) toniki...@gmail.com wrote: I went through the queue tonight and manually marked the bugs as commit-ready by adding [commit+] to their titles. This worked well, except that it made the bug titles ugly. I think a better solution is to have flag analogous to r+ which is commit+. Adam, as I suggedted previously, bugzilla supports KEYWORDs, so that would be a matter of adding a special support for bugs where patches are ready to go in. Can keywords be applied to individual patches? This needs to work for bugs with multiple patches on them. Ojan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] Grant Gayed is out of the office.
I will be out of the office starting 07/31/2009 and will not return until 08/17/2009. I will respond to your message when I return.___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] Reporting exceptions from worker context to users
Hi all, Currently, unhandled exceptions are sent from worker context over to the parent page where they are logged to the console. This works fine for dedicated workers, but not for shared workers which can have multiple active windows. The immediate solution that springs to mind is to broadcast the exception to every window associated with a shared worker, and have each window log the unhandled exception to its console. Is there another option that might be better (is there the concept of an inspector/console that is not associated with a specific window, for example)? -atw ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] LayoutTests take too long to run
Yeah, I stared a slowest run before going to bed, and got the following: The 10 slowest tests: 9.58 secs: editing/selection/move-left-right.html 9.37 secs: fast/js/array-filter.html 7.71 secs: editing/selection/extend-selection.html 6.82 secs: fast/js/sort-randomly.html 6.25 secs: http/tests/cache/subresource-expiration.html 5.77 secs: fast/js/array-enumerators-functions.html 5.41 secs: fast/events/click-count.html 5.39 secs: fast/js/toString-and-valueOf-override.html 5.26 secs: fast/js/try-catch-crash.html 5.05 secs: http/tests/navigation/slowmetaredirect-basic.html that's 10% of our time right there. :) I bet some of those have 5 second timers and should be easy to fix. I'll take a look, anyone else who would like to help would be most welcome. :) I remember Ojan finding that running tests in parallel was the biggest overall win, but there is definitely a lot of low-hanging fruit in the slowest tests too. I'm curious what the fastest tests run in. I would expect the harness cost to be near-zero, but I'm not sure. (i.e. what's the runtime of run-webkit-tests with 1 empty test. I would expect only a few seconds.) -eric On Fri, Jul 31, 2009 at 11:21 PM, Maciej Stachowiak m...@apple.com wrote: On Jul 31, 2009, at 10:25 PM, Eric Seidel wrote: 681.70s total testing time That's 11.5 minutes for every patch I want to land. (because I run the layout tests before landing anything, as part of bugzilla-tool). I'm very interested in any suggestions folks have to make that number smaller. I know Chromium runs the layout tests in parallel using their python run_webkit_tests.py harness. Try-bots would be another solution to the make sure it passes all the tests before landing. Again, very interested in suggestions as to how to improve this. We're working on try-bots for the ports of particular interest to Apple. But independent of that I think speeding up the layout tests is a worthwhile goal. Besides parallelism for better performance on multi-core systems, here are some other ideas: - Look at the slowest tests and see if they can be made to run faster (I'm running with --slowest as we speak). - Look at the subdirectories that take the most time - it's possible that in some cases we can consolidate multiple tests to make the whole test suite run faster, if sheer number of tests is the bottleneck. I think a good goal to shoot for would be for the full test suite to run in under 5 minutes on a reasonably modern laptop. That should allow for reasonable use during development with some room for growth. Regards, Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Towards a commit-queue
On Sat, Aug 1, 2009 at 8:43 AM, tonikitoo (Antonio Gomes)toniki...@gmail.com wrote: Adam, as I suggedted previously, bugzilla supports KEYWORDs, so that would be a matter of adding a special support for bugs where patches are ready to go in. 'checkin-needed' keyword would work , i believe. What do you think ? In addition to the point Ojan made about applying to individual patches, a flag also has the advantage that it implicates who approved the patch to be committed. Sure, the information is available for keywords if you dig deep enough, but the system has more accountability if the name is right there. Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Reporting exceptions from worker context to users
Shared workers can also communicate directly with one another, is that right? And its possible to have a shared worker whose only client is another shared worker, is that right? Feels like we should be working towards inspecting shared workers directly. Where a page inspector would show which shared workers it was connected to, and you could open a separate inspector to see what is going on within each shared worker (including which shared workers it was connected to). Broadcasting exceptions within a worker to its clients for inspection purposes may be useful even with separate inspectors per shared worker. The client's 'shared worker' tab (or console) could log unhandled exceptions occurring in each, as you described, encouraging the developer to look into it... open the shared worker inspector and poke around. About broadcasting unhandled shared worker exceptions in general... It makes sense to have unhandled exceptions in a dedicated process propagated to the parent page's context (which may presumably handle the exception in some way). But I'm not sure that model applies to unhandled exceptions in shared workers. The possibility of multiple connected pages, which should handle it? Seems good for logging/debugging purposes to have them show up in the client's inspector, but doesn't sound like a good fit for application execution purposes. On Sat, Aug 1, 2009 at 9:31 AM, Drew Wilsonatwil...@google.com wrote: Hi all, Currently, unhandled exceptions are sent from worker context over to the parent page where they are logged to the console. This works fine for dedicated workers, but not for shared workers which can have multiple active windows. The immediate solution that springs to mind is to broadcast the exception to every window associated with a shared worker, and have each window log the unhandled exception to its console. Is there another option that might be better (is there the concept of an inspector/console that is not associated with a specific window, for example)? -atw ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] LayoutTests take too long to run
SUMMARY I doubt there's much room for improvement in the test harness itself outside of running tests in parallel. However, there's clearly a lot to be gained by making individual tests faster. For Chromium runs, the median time to run a test is ~16ms. It's not apples to apples, but I would be surprised if there were a significant difference between this and WebKit's run-webkit-tests. With 16ms / test and 10,000 tests, it would take ~2.7 minutes to run the whole test suite. DETAILS For the Chromium run_webkit_test.py, here's the stats in seconds from a TestShell run on Windows Debug. FWIW, we spit these stats out for every run. The eventual goal was to track this in a set of graphs, but I never got around to implementing that. Median: 0.016324249 Mean: 0.163495878646 90th percentile: 0.077999830246 99th percentile: 3.108951 Standard deviation: 0.519017629449 Here's the times for the five slowest top-level directories: LayoutTests\editing took 43.5 seconds to run 976 tests. LayoutTests\svg took 53.7 seconds to run 891 tests. LayoutTests\fast took 378.7 seconds to run 3695 tests. LayoutTests\http took 396.9 seconds to run 632 tests. LayoutTests\media took 596.3 seconds to run 84 tests. This might be Chromium specific, but I'm seeing many of the the media layout tests taking 8-10 seconds. Ojan On Sat, Aug 1, 2009 at 12:49 PM, Eric Seidel e...@webkit.org wrote: Yeah, I stared a slowest run before going to bed, and got the following: The 10 slowest tests: 9.58 secs: editing/selection/move-left-right.html 9.37 secs: fast/js/array-filter.html 7.71 secs: editing/selection/extend-selection.html 6.82 secs: fast/js/sort-randomly.html 6.25 secs: http/tests/cache/subresource-expiration.html 5.77 secs: fast/js/array-enumerators-functions.html 5.41 secs: fast/events/click-count.html 5.39 secs: fast/js/toString-and-valueOf-override.html 5.26 secs: fast/js/try-catch-crash.html 5.05 secs: http/tests/navigation/slowmetaredirect-basic.html that's 10% of our time right there. :) I bet some of those have 5 second timers and should be easy to fix. I'll take a look, anyone else who would like to help would be most welcome. :) I remember Ojan finding that running tests in parallel was the biggest overall win, but there is definitely a lot of low-hanging fruit in the slowest tests too. I'm curious what the fastest tests run in. I would expect the harness cost to be near-zero, but I'm not sure. (i.e. what's the runtime of run-webkit-tests with 1 empty test. I would expect only a few seconds.) -eric On Fri, Jul 31, 2009 at 11:21 PM, Maciej Stachowiak m...@apple.com wrote: On Jul 31, 2009, at 10:25 PM, Eric Seidel wrote: 681.70s total testing time That's 11.5 minutes for every patch I want to land. (because I run the layout tests before landing anything, as part of bugzilla-tool). I'm very interested in any suggestions folks have to make that number smaller. I know Chromium runs the layout tests in parallel using their python run_webkit_tests.py harness. Try-bots would be another solution to the make sure it passes all the tests before landing. Again, very interested in suggestions as to how to improve this. We're working on try-bots for the ports of particular interest to Apple. But independent of that I think speeding up the layout tests is a worthwhile goal. Besides parallelism for better performance on multi-core systems, here are some other ideas: - Look at the slowest tests and see if they can be made to run faster (I'm running with --slowest as we speak). - Look at the subdirectories that take the most time - it's possible that in some cases we can consolidate multiple tests to make the whole test suite run faster, if sheer number of tests is the bottleneck. I think a good goal to shoot for would be for the full test suite to run in under 5 minutes on a reasonably modern laptop. That should allow for reasonable use during development with some room for growth. Regards, Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Towards a commit-queue
On Saturday, August 1, 2009 10:16:39 AM, Adam Barth wrote: On Sat, Aug 1, 2009 at 8:43 AM, tonikitoo (Antonio Gomes)wrote: Adam, as I suggedted previously, bugzilla supports KEYWORDs, so that would be a matter of adding a special support for bugs where patches are ready to go in. 'checkin-needed' keyword would work , i believe. What do you think ? In addition to the point Ojan made about applying to individual patches, a flag also has the advantage that it implicates who approved the patch to be committed. Sure, the information is available for keywords if you dig deep enough, but the system has more accountability if the name is right there. Bugzilla has the ability to create additional 4-state flags at both the attachment level and at the bug level. (Note that bugs.webkit.org does not have bug-level flags enabled.) For example, we could create a commit attachment flag which would have four states (just like the review flag): none, commit?, commit+, commit-. I'm not sure if this helps or hurts, though it would have made abarth's desired workflow much nicer. Mozilla also uses a superreview flag and various approval flags as seen on the attachments on this bug: https://bugzilla.mozilla.org/show_bug.cgi?id=342677 Dave ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Towards a commit-queue
On Sat, Aug 1, 2009 at 11:04 AM, David Kilzerddkil...@webkit.org wrote: Bugzilla has the ability to create additional 4-state flags at both the attachment level and at the bug level. (Note that bugs.webkit.org does not have bug-level flags enabled.) For example, we could create a commit attachment flag which would have four states (just like the review flag): none, commit?, commit+, commit-. I'm not sure if this helps or hurts, though it would have made abarth's desired workflow much nicer. Yeah, that sounds great. Should we try that out for a while and see if it's useful? Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Reporting exceptions from worker context to users
it sounds like we have one vote for just log them to the console for every connected document sgtm On Sat, Aug 1, 2009 at 12:39 PM, Drew Wilsonatwil...@google.com wrote: Yes, SharedWorkers will eventually be able to communicate with one another, as will DedicatedWorkers. So at some point we'll have a big connected graph of workers that potentially might be interesting for people to traverse and inspect their global contexts (I'm not sure - I don't think we know yet what debugging tools will be useful for developers). Further agreed, the HTML5 spec states that exceptions from shared workers are *not* propagated to parent pages for application execution - this would be solely for logging purposes. In the meantime while we work towards our glorious future, I'd still like to log exceptions somewhere :) It sounds like we have one vote for just log them to the console for every connected document. -atw On Sat, Aug 1, 2009 at 10:35 AM, Michael Nordman micha...@google.com wrote: Shared workers can also communicate directly with one another, is that right? And its possible to have a shared worker whose only client is another shared worker, is that right? Feels like we should be working towards inspecting shared workers directly. Where a page inspector would show which shared workers it was connected to, and you could open a separate inspector to see what is going on within each shared worker (including which shared workers it was connected to). Broadcasting exceptions within a worker to its clients for inspection purposes may be useful even with separate inspectors per shared worker. The client's 'shared worker' tab (or console) could log unhandled exceptions occurring in each, as you described, encouraging the developer to look into it... open the shared worker inspector and poke around. About broadcasting unhandled shared worker exceptions in general... It makes sense to have unhandled exceptions in a dedicated process propagated to the parent page's context (which may presumably handle the exception in some way). But I'm not sure that model applies to unhandled exceptions in shared workers. The possibility of multiple connected pages, which should handle it? Seems good for logging/debugging purposes to have them show up in the client's inspector, but doesn't sound like a good fit for application execution purposes. On Sat, Aug 1, 2009 at 9:31 AM, Drew Wilsonatwil...@google.com wrote: Hi all, Currently, unhandled exceptions are sent from worker context over to the parent page where they are logged to the console. This works fine for dedicated workers, but not for shared workers which can have multiple active windows. The immediate solution that springs to mind is to broadcast the exception to every window associated with a shared worker, and have each window log the unhandled exception to its console. Is there another option that might be better (is there the concept of an inspector/console that is not associated with a specific window, for example)? -atw ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Towards a commit-queue
On Sat, Aug 1, 2009 at 11:45 AM, Ojan Vafai o...@chromium.org wrote: On Sat, Aug 1, 2009 at 2:08 PM, Adam Barth aba...@webkit.org wrote: On Sat, Aug 1, 2009 at 11:04 AM, David Kilzerddkil...@webkit.org wrote: Bugzilla has the ability to create additional 4-state flags at both the attachment level and at the bug level. (Note that bugs.webkit.org does not have bug-level flags enabled.) For example, we could create a commit attachment flag which would have four states (just like the review flag): none, commit?, commit+, commit-. I'm not sure if this helps or hurts, though it would have made abarth's desired workflow much nicer. Yeah, that sounds great. Should we try that out for a while and see if it's useful? This seems fine to me, except it adds yet another layer of complexity to the review process. I think getting patches landed more quickly is probably worth it though. This is essentially a slightly more complicated (but easier to implement?) version of Maciej's proposal from a few weeks ago to get rid of r* and have the following four review states: REQUESTED DENIED APPROVED WITH MODIFICATIONS APPROVED It seems to me that the only difference is APPROVED becomes READY TO LAND (or LANDABLE or something like that). Btw, I see one downside to a commit queue: When you manually commit something, you're supposed to watch the build bots for breakage. If the submit queue is running all the tests on all the platforms then it doesn't really matter, but if not there'll need to be automated rollbacks when things break on the waterfall build bots. This could easily become complicated if the commit queue can submit multiple patches before the waterfall bots show green for the previous patch. I guess the point that I'm trying to make is that tests should either be run on every buildbot configuration, there needs to be some pretty robust rollback code, or the queue needs to let everything cycle green before committing the next patch. The last suggestion seems like a good start. Note also that, if the system rolls back a patch and the bots don't go back to their previous state, the submit bot needs to stop so someone can take a look manually (so things don't get further hosed). One benefit over the above is that, I don't think we need to restrict commit+ to people with svn commit bit, as long as a patch is r+'ed or r+'ed with modifications. Ojan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Reporting exceptions from worker context to users
I think logging to all connected pages' console is fine for now, but I think Michael's suggestion (or something similar) should be implemented in the not too distant future. Definitely before shared workers are allowed to communicate with each other. J On Sat, Aug 1, 2009 at 12:45 PM, Michael Nordman micha...@google.comwrote: it sounds like we have one vote for just log them to the console for every connected document sgtm On Sat, Aug 1, 2009 at 12:39 PM, Drew Wilsonatwil...@google.com wrote: Yes, SharedWorkers will eventually be able to communicate with one another, as will DedicatedWorkers. So at some point we'll have a big connected graph of workers that potentially might be interesting for people to traverse and inspect their global contexts (I'm not sure - I don't think we know yet what debugging tools will be useful for developers). Further agreed, the HTML5 spec states that exceptions from shared workers are *not* propagated to parent pages for application execution - this would be solely for logging purposes. In the meantime while we work towards our glorious future, I'd still like to log exceptions somewhere :) It sounds like we have one vote for just log them to the console for every connected document. -atw On Sat, Aug 1, 2009 at 10:35 AM, Michael Nordman micha...@google.com wrote: Shared workers can also communicate directly with one another, is that right? And its possible to have a shared worker whose only client is another shared worker, is that right? Feels like we should be working towards inspecting shared workers directly. Where a page inspector would show which shared workers it was connected to, and you could open a separate inspector to see what is going on within each shared worker (including which shared workers it was connected to). Broadcasting exceptions within a worker to its clients for inspection purposes may be useful even with separate inspectors per shared worker. The client's 'shared worker' tab (or console) could log unhandled exceptions occurring in each, as you described, encouraging the developer to look into it... open the shared worker inspector and poke around. About broadcasting unhandled shared worker exceptions in general... It makes sense to have unhandled exceptions in a dedicated process propagated to the parent page's context (which may presumably handle the exception in some way). But I'm not sure that model applies to unhandled exceptions in shared workers. The possibility of multiple connected pages, which should handle it? Seems good for logging/debugging purposes to have them show up in the client's inspector, but doesn't sound like a good fit for application execution purposes. On Sat, Aug 1, 2009 at 9:31 AM, Drew Wilsonatwil...@google.com wrote: Hi all, Currently, unhandled exceptions are sent from worker context over to the parent page where they are logged to the console. This works fine for dedicated workers, but not for shared workers which can have multiple active windows. The immediate solution that springs to mind is to broadcast the exception to every window associated with a shared worker, and have each window log the unhandled exception to its console. Is there another option that might be better (is there the concept of an inspector/console that is not associated with a specific window, for example)? -atw ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] LayoutTests take too long to run
For editing/selection/extend-selection.html,
It seems like the bottleneck is testExtendingSelection, in particular,
positionExtendingInDirection and checkSameOrder:
function positionsExtendingInDirection(sel, direction, granularity)
{
var positions = [];
while (true) {
positions.push({ node: sel.extentNode, begin: sel.baseOffset, end:
sel.extentOffset });
sel.modify(extend, direction, granularity);
if (positions[positions.length - 1].node == sel.extentNode
positions[positions.length - 1].end == sel.extentOffset)
break;
};
return positions;
}
We return positions and compare the results for LTR and RTL
using checkSameOrder
function checkSameOrder(inputPositions, inputSamePositions)
{
var positions = inputPositions.slice();
var samePositions = inputSamePositions.slice();
var mismatch = positions.length != samePositions.length;
if (mismatch)
log(WARNING: positions should be the same, but the length are not
the same + positions.length + vs. + samePositions.length + \n);
while (!mismatch) {
var pos = positions.pop();
if (!pos)
break;
var samePos = samePositions.pop();
if (pos.node != samePos.node) {
mismatch = true;
log(WARNING: positions should be the same, but node are not the
same\n);
}
if (pos.begin != samePos.begin) {
mismatch = true;
log(WARNING: positions should be the same, but begin are not
the same + pos.begin + vs. + samePos.begin + \n);
}
if (pos.end != samePos.end) {
mismatch = true;
log(WARNING: positions should be the same, but end are not the
same + pos.end + vs. + samePos.end + \n);
}
}
}
Can anyone think of any way to make this faster?
Ryosuke
On Sat, Aug 1, 2009 at 9:49 AM, Eric Seidel e...@webkit.org wrote:
Yeah, I stared a slowest run before going to bed, and got the following:
The 10 slowest tests:
9.58 secs: editing/selection/move-left-right.html
9.37 secs: fast/js/array-filter.html
7.71 secs: editing/selection/extend-selection.html
6.82 secs: fast/js/sort-randomly.html
6.25 secs: http/tests/cache/subresource-expiration.html
5.77 secs: fast/js/array-enumerators-functions.html
5.41 secs: fast/events/click-count.html
5.39 secs: fast/js/toString-and-valueOf-override.html
5.26 secs: fast/js/try-catch-crash.html
5.05 secs: http/tests/navigation/slowmetaredirect-basic.html
that's 10% of our time right there. :) I bet some of those have 5 second
timers and should be easy to fix. I'll take a look, anyone else who would
like to help would be most welcome. :)
I remember Ojan finding that running tests in parallel was the biggest
overall win, but there is definitely a lot of low-hanging fruit in the
slowest tests too. I'm curious what the fastest tests run in. I would
expect the harness cost to be near-zero, but I'm not sure. (i.e. what's the
runtime of run-webkit-tests with 1 empty test. I would expect only a
few seconds.)
-eric
On Fri, Jul 31, 2009 at 11:21 PM, Maciej Stachowiak m...@apple.com wrote:
On Jul 31, 2009, at 10:25 PM, Eric Seidel wrote:
681.70s total testing time
That's 11.5 minutes for every patch I want to land. (because I run the
layout tests before landing anything, as part of bugzilla-tool).
I'm very interested in any suggestions folks have to make that number
smaller. I know Chromium runs the layout tests in parallel using their
python run_webkit_tests.py harness. Try-bots would be another solution to
the make sure it passes all the tests before landing.
Again, very interested in suggestions as to how to improve this.
We're working on try-bots for the ports of particular interest to Apple.
But independent of that I think speeding up the layout tests is a worthwhile
goal.
Besides parallelism for better performance on multi-core systems, here are
some other ideas:
- Look at the slowest tests and see if they can be made to run faster (I'm
running with --slowest as we speak).
- Look at the subdirectories that take the most time - it's possible that
in some cases we can consolidate multiple tests to make the whole test suite
run faster, if sheer number of tests is the bottleneck.
I think a good goal to shoot for would be for the full test suite to run
in under 5 minutes on a reasonably modern laptop. That should allow for
reasonable use during development with some room for growth.
Regards,
Maciej
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] LayoutTests take too long to run
editing/selection/move-left-right.html has very similar script where we
return positions and compare entires in reverse order. Since there is no
setTimeout, we need to rewrite those scripts so that they run faster or
improve the performance of WebKit.
Ryosuke
On Sat, Aug 1, 2009 at 1:09 PM, Ryosuke Niwa rn...@google.com wrote:
For editing/selection/extend-selection.html,
It seems like the bottleneck is testExtendingSelection, in particular,
positionExtendingInDirection and checkSameOrder:
function positionsExtendingInDirection(sel, direction, granularity)
{
var positions = [];
while (true) {
positions.push({ node: sel.extentNode, begin: sel.baseOffset, end:
sel.extentOffset });
sel.modify(extend, direction, granularity);
if (positions[positions.length - 1].node == sel.extentNode
positions[positions.length - 1].end == sel.extentOffset)
break;
};
return positions;
}
We return positions and compare the results for LTR and RTL
using checkSameOrder
function checkSameOrder(inputPositions, inputSamePositions)
{
var positions = inputPositions.slice();
var samePositions = inputSamePositions.slice();
var mismatch = positions.length != samePositions.length;
if (mismatch)
log(WARNING: positions should be the same, but the length are not
the same + positions.length + vs. + samePositions.length + \n);
while (!mismatch) {
var pos = positions.pop();
if (!pos)
break;
var samePos = samePositions.pop();
if (pos.node != samePos.node) {
mismatch = true;
log(WARNING: positions should be the same, but node are not
the same\n);
}
if (pos.begin != samePos.begin) {
mismatch = true;
log(WARNING: positions should be the same, but begin are not
the same + pos.begin + vs. + samePos.begin + \n);
}
if (pos.end != samePos.end) {
mismatch = true;
log(WARNING: positions should be the same, but end are not the
same + pos.end + vs. + samePos.end + \n);
}
}
}
Can anyone think of any way to make this faster?
Ryosuke
On Sat, Aug 1, 2009 at 9:49 AM, Eric Seidel e...@webkit.org wrote:
Yeah, I stared a slowest run before going to bed, and got the following:
The 10 slowest tests:
9.58 secs: editing/selection/move-left-right.html
9.37 secs: fast/js/array-filter.html
7.71 secs: editing/selection/extend-selection.html
6.82 secs: fast/js/sort-randomly.html
6.25 secs: http/tests/cache/subresource-expiration.html
5.77 secs: fast/js/array-enumerators-functions.html
5.41 secs: fast/events/click-count.html
5.39 secs: fast/js/toString-and-valueOf-override.html
5.26 secs: fast/js/try-catch-crash.html
5.05 secs: http/tests/navigation/slowmetaredirect-basic.html
that's 10% of our time right there. :) I bet some of those have 5 second
timers and should be easy to fix. I'll take a look, anyone else who would
like to help would be most welcome. :)
I remember Ojan finding that running tests in parallel was the biggest
overall win, but there is definitely a lot of low-hanging fruit in the
slowest tests too. I'm curious what the fastest tests run in. I would
expect the harness cost to be near-zero, but I'm not sure. (i.e. what's the
runtime of run-webkit-tests with 1 empty test. I would expect only a
few seconds.)
-eric
On Fri, Jul 31, 2009 at 11:21 PM, Maciej Stachowiak m...@apple.comwrote:
On Jul 31, 2009, at 10:25 PM, Eric Seidel wrote:
681.70s total testing time
That's 11.5 minutes for every patch I want to land. (because I run the
layout tests before landing anything, as part of bugzilla-tool).
I'm very interested in any suggestions folks have to make that number
smaller. I know Chromium runs the layout tests in parallel using their
python run_webkit_tests.py harness. Try-bots would be another solution to
the make sure it passes all the tests before landing.
Again, very interested in suggestions as to how to improve this.
We're working on try-bots for the ports of particular interest to Apple.
But independent of that I think speeding up the layout tests is a worthwhile
goal.
Besides parallelism for better performance on multi-core systems, here
are some other ideas:
- Look at the slowest tests and see if they can be made to run faster
(I'm running with --slowest as we speak).
- Look at the subdirectories that take the most time - it's possible that
in some cases we can consolidate multiple tests to make the whole test suite
run faster, if sheer number of tests is the bottleneck.
I think a good goal to shoot for would be for the full test suite to run
in under 5 minutes on a reasonably modern laptop. That should allow for
reasonable use during development with some room for growth.
Regards,
Maciej
___
webkit-dev mailing list
Re: [webkit-dev] Towards a commit-queue
On Saturday, August 1, 2009 11:45:43 AM, Ojan Vafai wrote: On Sat, Aug 1, 2009 at 2:08 PM, Adam Barth aba...@webkit.org wrote: On Sat, Aug 1, 2009 at 11:04 AM, David Kilzerddkil...@webkit.org wrote: Bugzilla has the ability to create additional 4-state flags at both the attachment level and at the bug level. (Note that bugs.webkit.org does not have bug-level flags enabled.) For example, we could create a commit attachment flag which would have four states (just like the review flag): none, commit?, commit+, commit-. I'm not sure if this helps or hurts, though it would have made abarth's desired workflow much nicer. Yeah, that sounds great. Should we try that out for a while and see if it's useful? This seems fine to me, except it adds yet another layer of complexity to the review process. I think getting patches landed more quickly is probably worth it though. This is essentially a slightly more complicated (but easier to implement?) version of Maciej's proposal from a few weeks ago to get rid of r* and have the following four review states: REQUESTED DENIED APPROVED WITH MODIFICATIONS APPROVED One benefit over the above is that, I don't think we need to restrict commit+ to people with svn commit bit, as long as a patch is r+'ed or r+'ed with modifications. This would require a non-trivial amount of modification to Bugzilla, and it would make future upgrades much more difficult. Basically, Adam's asking for a flag to make it easier for bugzilla-tool to know when to land patches without human intervention. Currently, a committer must look at each patch's review comments to make this determination. Either we should change the review process to only set the review+ flag if the patch is ready to go with zero modifications, or we should use the commit+ flag to signify that. I could go either way on this. I don't like the idea of setting review- flags for trivial fixes that could be landed with minor modifications. On the other hand, being able to commit patches directly from bugs.webkit.org with bugzilla-tool is really handy. Dave ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] LayoutTests take too long to run
On Aug 1, 2009, at 10:52 AM, Ojan Vafai wrote: Here's the times for the five slowest top-level directories: LayoutTests\editing took 43.5 seconds to run 976 tests. LayoutTests\svg took 53.7 seconds to run 891 tests. LayoutTests\fast took 378.7 seconds to run 3695 tests. LayoutTests\http took 396.9 seconds to run 632 tests. LayoutTests\media took 596.3 seconds to run 84 tests. This might be Chromium specific, but I'm seeing many of the the media layout tests taking 8-10 seconds. That seems unusually high, on a MacBook with TOT debug build the slowest test (video-played.html) takes less than 4 seconds. Running all of the media tests takes ~40 seconds: eric% run-webkit-tests --debug --slowest LayoutTests/media/ 40.87s total testing time The 10 slowest tests: 3.23 secs: media/video-played.html 2.53 secs: media/audio-constructor-autobuffer.html 2.05 secs: media/video-played-collapse.html 1.93 secs: media/video-loop.html 1.53 secs: media/controls-after-reload.html 1.39 secs: media/event-attributes.html 1.35 secs: media/video-aspect-ratio.html 1.16 secs: media/video-timeupdate-during-playback.html 1.12 secs: media/video-played-reset.html 1.09 secs: media/audio-data-url.html eric ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Reporting exceptions from worker context to users
OK, I really wanted to gloss over this, but now Jeremy's going to make me elaborate :) There are a few things I think we will eventually need to support worker development: 1) Some way to print out the values of things in a worker's global scope. 2) Similar debugging support for workers that we have for page script (breakpoints, etc). 3) When the user is selecting a worker for #1/#2, it might be useful to indicate what the parent(s) of the worker are to help developers select the right worker (in case there are nested workers). Or maybe it's not very useful - perhaps just having the URL of the worker base script is sufficient. We don't know yet. None of these issues have anything to do specifically with SharedWorkers. Nor do I think that we should block development on arbitrary features like nested workers because we think we need better tools. A better solution would be to roll out a just barely good enough solution, get developer feedback on real use cases, and improve the tools over time to reflect that feedback, just as we do for every other feature in WebKit. -atw On Sat, Aug 1, 2009 at 1:06 PM, Jeremy Orlow jor...@chromium.org wrote: I think logging to all connected pages' console is fine for now, but I think Michael's suggestion (or something similar) should be implemented in the not too distant future. Definitely before shared workers are allowed to communicate with each other. J On Sat, Aug 1, 2009 at 12:45 PM, Michael Nordman micha...@google.comwrote: it sounds like we have one vote for just log them to the console for every connected document sgtm On Sat, Aug 1, 2009 at 12:39 PM, Drew Wilsonatwil...@google.com wrote: Yes, SharedWorkers will eventually be able to communicate with one another, as will DedicatedWorkers. So at some point we'll have a big connected graph of workers that potentially might be interesting for people to traverse and inspect their global contexts (I'm not sure - I don't think we know yet what debugging tools will be useful for developers). Further agreed, the HTML5 spec states that exceptions from shared workers are *not* propagated to parent pages for application execution - this would be solely for logging purposes. In the meantime while we work towards our glorious future, I'd still like to log exceptions somewhere :) It sounds like we have one vote for just log them to the console for every connected document. -atw On Sat, Aug 1, 2009 at 10:35 AM, Michael Nordman micha...@google.com wrote: Shared workers can also communicate directly with one another, is that right? And its possible to have a shared worker whose only client is another shared worker, is that right? Feels like we should be working towards inspecting shared workers directly. Where a page inspector would show which shared workers it was connected to, and you could open a separate inspector to see what is going on within each shared worker (including which shared workers it was connected to). Broadcasting exceptions within a worker to its clients for inspection purposes may be useful even with separate inspectors per shared worker. The client's 'shared worker' tab (or console) could log unhandled exceptions occurring in each, as you described, encouraging the developer to look into it... open the shared worker inspector and poke around. About broadcasting unhandled shared worker exceptions in general... It makes sense to have unhandled exceptions in a dedicated process propagated to the parent page's context (which may presumably handle the exception in some way). But I'm not sure that model applies to unhandled exceptions in shared workers. The possibility of multiple connected pages, which should handle it? Seems good for logging/debugging purposes to have them show up in the client's inspector, but doesn't sound like a good fit for application execution purposes. On Sat, Aug 1, 2009 at 9:31 AM, Drew Wilsonatwil...@google.com wrote: Hi all, Currently, unhandled exceptions are sent from worker context over to the parent page where they are logged to the console. This works fine for dedicated workers, but not for shared workers which can have multiple active windows. The immediate solution that springs to mind is to broadcast the exception to every window associated with a shared worker, and have each window log the unhandled exception to its console. Is there another option that might be better (is there the concept of an inspector/console that is not associated with a specific window, for example)? -atw ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org
Re: [webkit-dev] LayoutTests take too long to run
Seems chromium tests are even slower: 10k * mean = 1,634s 50% (5000) of the tests are 0.016s, assuming they all take that long, that's at most 80seconds. 40% (4000) of the tests are between 0.016 and 0.077, that's at most 311 seconds. 1,634 (total) - 80 - 311 leaves at least 1243 seconds accounted by the last 10% of tests! Assuming you and I both did our math correctly, the bulk of our delay is from a few slow tests, specifically less than 10% of the tests are accounting for 1243s (76%) of the time! -eric On Sat, Aug 1, 2009 at 12:52 PM, Ojan Vafai o...@chromium.org wrote: SUMMARY I doubt there's much room for improvement in the test harness itself outside of running tests in parallel. However, there's clearly a lot to be gained by making individual tests faster. For Chromium runs, the median time to run a test is ~16ms. It's not apples to apples, but I would be surprised if there were a significant difference between this and WebKit's run-webkit-tests. With 16ms / test and 10,000 tests, it would take ~2.7 minutes to run the whole test suite. DETAILS For the Chromium run_webkit_test.py, here's the stats in seconds from a TestShell run on Windows Debug. FWIW, we spit these stats out for every run. The eventual goal was to track this in a set of graphs, but I never got around to implementing that. Median: 0.016324249 Mean: 0.163495878646 90th percentile: 0.077999830246 99th percentile: 3.108951 Standard deviation: 0.519017629449 Here's the times for the five slowest top-level directories: LayoutTests\editing took 43.5 seconds to run 976 tests. LayoutTests\svg took 53.7 seconds to run 891 tests. LayoutTests\fast took 378.7 seconds to run 3695 tests. LayoutTests\http took 396.9 seconds to run 632 tests. LayoutTests\media took 596.3 seconds to run 84 tests. This might be Chromium specific, but I'm seeing many of the the media layout tests taking 8-10 seconds. Ojan On Sat, Aug 1, 2009 at 12:49 PM, Eric Seidel e...@webkit.org wrote: Yeah, I stared a slowest run before going to bed, and got the following: The 10 slowest tests: 9.58 secs: editing/selection/move-left-right.html 9.37 secs: fast/js/array-filter.html 7.71 secs: editing/selection/extend-selection.html 6.82 secs: fast/js/sort-randomly.html 6.25 secs: http/tests/cache/subresource-expiration.html 5.77 secs: fast/js/array-enumerators-functions.html 5.41 secs: fast/events/click-count.html 5.39 secs: fast/js/toString-and-valueOf-override.html 5.26 secs: fast/js/try-catch-crash.html 5.05 secs: http/tests/navigation/slowmetaredirect-basic.html that's 10% of our time right there. :) I bet some of those have 5 second timers and should be easy to fix. I'll take a look, anyone else who would like to help would be most welcome. :) I remember Ojan finding that running tests in parallel was the biggest overall win, but there is definitely a lot of low-hanging fruit in the slowest tests too. I'm curious what the fastest tests run in. I would expect the harness cost to be near-zero, but I'm not sure. (i.e. what's the runtime of run-webkit-tests with 1 empty test. I would expect only a few seconds.) -eric On Fri, Jul 31, 2009 at 11:21 PM, Maciej Stachowiak m...@apple.comwrote: On Jul 31, 2009, at 10:25 PM, Eric Seidel wrote: 681.70s total testing time That's 11.5 minutes for every patch I want to land. (because I run the layout tests before landing anything, as part of bugzilla-tool). I'm very interested in any suggestions folks have to make that number smaller. I know Chromium runs the layout tests in parallel using their python run_webkit_tests.py harness. Try-bots would be another solution to the make sure it passes all the tests before landing. Again, very interested in suggestions as to how to improve this. We're working on try-bots for the ports of particular interest to Apple. But independent of that I think speeding up the layout tests is a worthwhile goal. Besides parallelism for better performance on multi-core systems, here are some other ideas: - Look at the slowest tests and see if they can be made to run faster (I'm running with --slowest as we speak). - Look at the subdirectories that take the most time - it's possible that in some cases we can consolidate multiple tests to make the whole test suite run faster, if sheer number of tests is the bottleneck. I think a good goal to shoot for would be for the full test suite to run in under 5 minutes on a reasonably modern laptop. That should allow for reasonable use during development with some room for growth. Regards, Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] LayoutTests take too long to run
On Sat, Aug 1, 2009 at 4:23 PM, Eric Seidel e...@webkit.org wrote: Seems chromium tests are even slower: 10k * mean = 1,634s For completeness sake, the Chromium numbers include the pixel tests. 50% (5000) of the tests are 0.016s, assuming they all take that long, that's at most 80seconds. 40% (4000) of the tests are between 0.016 and 0.077, that's at most 311 seconds. 1,634 (total) - 80 - 311 leaves at least 1243 seconds accounted by the last 10% of tests! Assuming you and I both did our math correctly, the bulk of our delay is from a few slow tests, specifically less than 10% of the tests are accounting for 1243s (76%) of the time! -eric On Sat, Aug 1, 2009 at 12:52 PM, Ojan Vafai o...@chromium.org wrote: SUMMARY I doubt there's much room for improvement in the test harness itself outside of running tests in parallel. However, there's clearly a lot to be gained by making individual tests faster. For Chromium runs, the median time to run a test is ~16ms. It's not apples to apples, but I would be surprised if there were a significant difference between this and WebKit's run-webkit-tests. With 16ms / test and 10,000 tests, it would take ~2.7 minutes to run the whole test suite. DETAILS For the Chromium run_webkit_test.py, here's the stats in seconds from a TestShell run on Windows Debug. FWIW, we spit these stats out for every run. The eventual goal was to track this in a set of graphs, but I never got around to implementing that. Median: 0.016324249 Mean: 0.163495878646 90th percentile: 0.077999830246 99th percentile: 3.108951 Standard deviation: 0.519017629449 Here's the times for the five slowest top-level directories: LayoutTests\editing took 43.5 seconds to run 976 tests. LayoutTests\svg took 53.7 seconds to run 891 tests. LayoutTests\fast took 378.7 seconds to run 3695 tests. LayoutTests\http took 396.9 seconds to run 632 tests. LayoutTests\media took 596.3 seconds to run 84 tests. This might be Chromium specific, but I'm seeing many of the the media layout tests taking 8-10 seconds. Ojan On Sat, Aug 1, 2009 at 12:49 PM, Eric Seidel e...@webkit.org wrote: Yeah, I stared a slowest run before going to bed, and got the following: The 10 slowest tests: 9.58 secs: editing/selection/move-left-right.html 9.37 secs: fast/js/array-filter.html 7.71 secs: editing/selection/extend-selection.html 6.82 secs: fast/js/sort-randomly.html 6.25 secs: http/tests/cache/subresource-expiration.html 5.77 secs: fast/js/array-enumerators-functions.html 5.41 secs: fast/events/click-count.html 5.39 secs: fast/js/toString-and-valueOf-override.html 5.26 secs: fast/js/try-catch-crash.html 5.05 secs: http/tests/navigation/slowmetaredirect-basic.html that's 10% of our time right there. :) I bet some of those have 5 second timers and should be easy to fix. I'll take a look, anyone else who would like to help would be most welcome. :) I remember Ojan finding that running tests in parallel was the biggest overall win, but there is definitely a lot of low-hanging fruit in the slowest tests too. I'm curious what the fastest tests run in. I would expect the harness cost to be near-zero, but I'm not sure. (i.e. what's the runtime of run-webkit-tests with 1 empty test. I would expect only a few seconds.) -eric On Fri, Jul 31, 2009 at 11:21 PM, Maciej Stachowiak m...@apple.comwrote: On Jul 31, 2009, at 10:25 PM, Eric Seidel wrote: 681.70s total testing time That's 11.5 minutes for every patch I want to land. (because I run the layout tests before landing anything, as part of bugzilla-tool). I'm very interested in any suggestions folks have to make that number smaller. I know Chromium runs the layout tests in parallel using their python run_webkit_tests.py harness. Try-bots would be another solution to the make sure it passes all the tests before landing. Again, very interested in suggestions as to how to improve this. We're working on try-bots for the ports of particular interest to Apple. But independent of that I think speeding up the layout tests is a worthwhile goal. Besides parallelism for better performance on multi-core systems, here are some other ideas: - Look at the slowest tests and see if they can be made to run faster (I'm running with --slowest as we speak). - Look at the subdirectories that take the most time - it's possible that in some cases we can consolidate multiple tests to make the whole test suite run faster, if sheer number of tests is the bottleneck. I think a good goal to shoot for would be for the full test suite to run in under 5 minutes on a reasonably modern laptop. That should allow for reasonable use during development with some room for growth. Regards, Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Reporting exceptions from worker context to users
Nor do I think that we should block development on arbitrary features like nested workers because we think we need better tools. A better solution would be to roll out a just barely good enough solution, get developer feedback on real use cases, and improve the tools over time to reflect that feedback, just as we do for every other feature in WebKit. absolutely... features come first ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Towards a commit-queue
On Sat, Aug 1, 2009 at 12:53 PM, Jeremy Orlowjor...@chromium.org wrote: Btw, I see one downside to a commit queue: When you manually commit something, you're supposed to watch the build bots for breakage. If the submit queue is running all the tests on all the platforms then it doesn't really matter, but if not there'll need to be automated rollbacks when things break on the waterfall build bots. This could easily become complicated if the commit queue can submit multiple patches before the waterfall bots show green for the previous patch. We're a ways from having a fully automated commit queue. When I run the commit-queue script, I also watch the bots and clean up manually as needed. (So far this hasn't been necessary, but I'm sure it will be at some point.) Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Towards a commit-queue
On Sat, Aug 1, 2009 at 1:13 PM, David Kilzerddkil...@webkit.org wrote: Either we should change the review process to only set the review+ flag if the patch is ready to go with zero modifications, or we should use the commit+ flag to signify that. I could go either way on this. I don't like the idea of setting review- flags for trivial fixes that could be landed with minor modifications. On the other hand, being able to commit patches directly from bugs.webkit.org with bugzilla-tool is really handy. The other factor is that some committers might not want the commit-queue script to land their changes. In those cases, their patches would get an r+ but never get a commit+. (Personally, I don't mind if other people want to land may changes, but I can see how it might disrupt other's workflow.) Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
