Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed
I created a page with instructions how to use content_shell here: https://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tests/content-shell At the bottom, I summarized next steps, esp. how we can reuse chromium's test controller for the content shell best -jochen On Mon, Jun 11, 2012 at 1:53 AM, Darin Fisher da...@chromium.org wrote: On Fri, Jun 8, 2012 at 12:51 PM, Jochen Eisinger joc...@chromium.orgwrote: On Fri, Jun 8, 2012 at 9:41 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Jun 8, 2012 at 12:18 PM, Jochen Eisinger joc...@chromium.orgwrote: I've implemented initial support for running layout tests using chromium's content_shell instead of test_shell as the current DRT implementation does. It's still a very experimental, but might already be interesting for some of you to try. This is great! Thanks a lot on working this. 1. Make sure your WebKit is at least at r119852 (see http://trac.webkit.org/wiki/Chromium for prerequisites) 2. Apply the attachment from https://bugs.webkit.org/show_bug.cgi?id=87045 3. In Source/WebKit/chromium run gclient sync 4. build webkit as usual E.g. for a debug build on Linux, this should give you out/Debug/content_shell You can now run layout tests like this: new-run-webkit-tests --chromium --debug --driver-name=content_shell --additional-drt-flag=--dump-render-tree LayoutTests/storage/indexeddb/* You'll notice that not all tests are passing yet, mainly because not all (or actually, almost none of the) layoutTestController features are implemented yet. Where is layoutTestController implemented? We definitely need to move the implementation of layoutTestController, eventSender, etc... into WebKit repository because we often rename functions, etc... in WebKit. It's unacceptable to require having to modify Chromium code in order to do this refactoring in the future. It's currently here: http://code.google.com/searchframe#OAMlx_jo-ck/src/content/shell/layout_test_controller.jsexact_package=chromium Per my other thread about sharing IDLs between DumpRenderTree and WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner instead of adding yet another binding code. Another missing feature is producing pixel results. However, I'm currently concentrating on text results, as I think the biggest benefit is the ability to run storage tests on the real storage implementation. That sounds great to me but we definitely need to support pixel results eventually. I'm more than happy to help you on that but that requires the codebase to be moved into WebKit repository. Here's the basic problem: content_shell depends on content, so moving this on the webkit repository would mean that webkit depended on content. Another solution would be to formalize the test shell API our current layout test controller in webkit uses (Tools/DumpRenderTree/chromium), and add a method to chromium's webkit support library that returns a webview that supports all the hooks required. The webview could then either be implemented by test_shell or by content_shell ^^^ This second solution seems like it will work well. It will enable the guts of layoutTestController to remain in the WebKit repository. This is just a variation of exactly what we do today. You only need to move creation of WebView to Chromium so that we can eliminate WebViewHost.cpp (and other simple application shell bits). -Darin best -jochen ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed
On Fri, Jun 8, 2012 at 12:51 PM, Jochen Eisinger joc...@chromium.orgwrote: On Fri, Jun 8, 2012 at 9:41 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Jun 8, 2012 at 12:18 PM, Jochen Eisinger joc...@chromium.orgwrote: I've implemented initial support for running layout tests using chromium's content_shell instead of test_shell as the current DRT implementation does. It's still a very experimental, but might already be interesting for some of you to try. This is great! Thanks a lot on working this. 1. Make sure your WebKit is at least at r119852 (see http://trac.webkit.org/wiki/Chromium for prerequisites) 2. Apply the attachment from https://bugs.webkit.org/show_bug.cgi?id=87045 3. In Source/WebKit/chromium run gclient sync 4. build webkit as usual E.g. for a debug build on Linux, this should give you out/Debug/content_shell You can now run layout tests like this: new-run-webkit-tests --chromium --debug --driver-name=content_shell --additional-drt-flag=--dump-render-tree LayoutTests/storage/indexeddb/* You'll notice that not all tests are passing yet, mainly because not all (or actually, almost none of the) layoutTestController features are implemented yet. Where is layoutTestController implemented? We definitely need to move the implementation of layoutTestController, eventSender, etc... into WebKit repository because we often rename functions, etc... in WebKit. It's unacceptable to require having to modify Chromium code in order to do this refactoring in the future. It's currently here: http://code.google.com/searchframe#OAMlx_jo-ck/src/content/shell/layout_test_controller.jsexact_package=chromium Per my other thread about sharing IDLs between DumpRenderTree and WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner instead of adding yet another binding code. Another missing feature is producing pixel results. However, I'm currently concentrating on text results, as I think the biggest benefit is the ability to run storage tests on the real storage implementation. That sounds great to me but we definitely need to support pixel results eventually. I'm more than happy to help you on that but that requires the codebase to be moved into WebKit repository. Here's the basic problem: content_shell depends on content, so moving this on the webkit repository would mean that webkit depended on content. Another solution would be to formalize the test shell API our current layout test controller in webkit uses (Tools/DumpRenderTree/chromium), and add a method to chromium's webkit support library that returns a webview that supports all the hooks required. The webview could then either be implemented by test_shell or by content_shell ^^^ This second solution seems like it will work well. It will enable the guts of layoutTestController to remain in the WebKit repository. This is just a variation of exactly what we do today. You only need to move creation of WebView to Chromium so that we can eliminate WebViewHost.cpp (and other simple application shell bits). -Darin best -jochen ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed
Hey, I've implemented initial support for running layout tests using chromium's content_shell instead of test_shell as the current DRT implementation does. It's still a very experimental, but might already be interesting for some of you to try. The main advantage is that content_shell includes chromium's full storage implementation, so layout tests for indexed db are running against the multi-processed IDB implementation instead of the simple stub used for DRT. Here's how to build webkit with content_shell: 1. Make sure your WebKit is at least at r119852 (see http://trac.webkit.org/wiki/Chromium for prerequisites) 2. Apply the attachment from https://bugs.webkit.org/show_bug.cgi?id=87045 3. In Source/WebKit/chromium run gclient sync 4. build webkit as usual E.g. for a debug build on Linux, this should give you out/Debug/content_shell You can now run layout tests like this: new-run-webkit-tests --chromium --debug --driver-name=content_shell --additional-drt-flag=--dump-render-tree LayoutTests/storage/indexeddb/* You'll notice that not all tests are passing yet, mainly because not all (or actually, almost none of the) layoutTestController features are implemented yet. Another missing feature is producing pixel results. However, I'm currently concentrating on text results, as I think the biggest benefit is the ability to run storage tests on the real storage implementation. Let me know if you have any feedback, comments, or questions. If somebody is implemented in helping adding features (e.g. to get your favorite part of the layout tests passing), I'm happy to point you at the code locations! best -jochen ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed
On Fri, Jun 8, 2012 at 12:18 PM, Jochen Eisinger joc...@chromium.orgwrote: I've implemented initial support for running layout tests using chromium's content_shell instead of test_shell as the current DRT implementation does. It's still a very experimental, but might already be interesting for some of you to try. This is great! Thanks a lot on working this. 1. Make sure your WebKit is at least at r119852 (see http://trac.webkit.org/wiki/Chromium for prerequisites) 2. Apply the attachment from https://bugs.webkit.org/show_bug.cgi?id=87045 3. In Source/WebKit/chromium run gclient sync 4. build webkit as usual E.g. for a debug build on Linux, this should give you out/Debug/content_shell You can now run layout tests like this: new-run-webkit-tests --chromium --debug --driver-name=content_shell --additional-drt-flag=--dump-render-tree LayoutTests/storage/indexeddb/* You'll notice that not all tests are passing yet, mainly because not all (or actually, almost none of the) layoutTestController features are implemented yet. Where is layoutTestController implemented? We definitely need to move the implementation of layoutTestController, eventSender, etc... into WebKit repository because we often rename functions, etc... in WebKit. It's unacceptable to require having to modify Chromium code in order to do this refactoring in the future. Per my other thread about sharing IDLs between DumpRenderTree and WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner instead of adding yet another binding code. Another missing feature is producing pixel results. However, I'm currently concentrating on text results, as I think the biggest benefit is the ability to run storage tests on the real storage implementation. That sounds great to me but we definitely need to support pixel results eventually. I'm more than happy to help you on that but that requires the codebase to be moved into WebKit repository. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed
On Fri, Jun 8, 2012 at 9:41 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Jun 8, 2012 at 12:18 PM, Jochen Eisinger joc...@chromium.orgwrote: I've implemented initial support for running layout tests using chromium's content_shell instead of test_shell as the current DRT implementation does. It's still a very experimental, but might already be interesting for some of you to try. This is great! Thanks a lot on working this. 1. Make sure your WebKit is at least at r119852 (see http://trac.webkit.org/wiki/Chromium for prerequisites) 2. Apply the attachment from https://bugs.webkit.org/show_bug.cgi?id=87045 3. In Source/WebKit/chromium run gclient sync 4. build webkit as usual E.g. for a debug build on Linux, this should give you out/Debug/content_shell You can now run layout tests like this: new-run-webkit-tests --chromium --debug --driver-name=content_shell --additional-drt-flag=--dump-render-tree LayoutTests/storage/indexeddb/* You'll notice that not all tests are passing yet, mainly because not all (or actually, almost none of the) layoutTestController features are implemented yet. Where is layoutTestController implemented? We definitely need to move the implementation of layoutTestController, eventSender, etc... into WebKit repository because we often rename functions, etc... in WebKit. It's unacceptable to require having to modify Chromium code in order to do this refactoring in the future. It's currently here: http://code.google.com/searchframe#OAMlx_jo-ck/src/content/shell/layout_test_controller.jsexact_package=chromium Per my other thread about sharing IDLs between DumpRenderTree and WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner instead of adding yet another binding code. Another missing feature is producing pixel results. However, I'm currently concentrating on text results, as I think the biggest benefit is the ability to run storage tests on the real storage implementation. That sounds great to me but we definitely need to support pixel results eventually. I'm more than happy to help you on that but that requires the codebase to be moved into WebKit repository. Here's the basic problem: content_shell depends on content, so moving this on the webkit repository would mean that webkit depended on content. Another solution would be to formalize the test shell API our current layout test controller in webkit uses (Tools/DumpRenderTree/chromium), and add a method to chromium's webkit support library that returns a webview that supports all the hooks required. The webview could then either be implemented by test_shell or by content_shell best -jochen ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed
On Fri, Jun 8, 2012 at 12:51 PM, Jochen Eisinger joc...@chromium.orgwrote: On Fri, Jun 8, 2012 at 9:41 PM, Ryosuke Niwa rn...@webkit.org wrote: Per my other thread about sharing IDLs between DumpRenderTree and WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner instead of adding yet another binding code. Another missing feature is producing pixel results. However, I'm currently concentrating on text results, as I think the biggest benefit is the ability to run storage tests on the real storage implementation. That sounds great to me but we definitely need to support pixel results eventually. I'm more than happy to help you on that but that requires the codebase to be moved into WebKit repository. Here's the basic problem: content_shell depends on content, so moving this on the webkit repository would mean that webkit depended on content. Another solution would be to formalize the test shell API our current layout test controller in webkit uses (Tools/DumpRenderTree/chromium), and add a method to chromium's webkit support library that returns a webview that supports all the hooks required. The webview could then either be implemented by test_shell or by content_shell I don't know any architectural details of content_shell so it's hard for me to comment on this matter, but I don't see a problem in WebKitChromiumTestRunner (hypothetical name to be consistent with WebKitTestRunner) depending on content_shell given that WebKitTestRunner also depends on WebKit2 API. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed
On Fri, Jun 8, 2012 at 9:55 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Jun 8, 2012 at 12:51 PM, Jochen Eisinger joc...@chromium.orgwrote: On Fri, Jun 8, 2012 at 9:41 PM, Ryosuke Niwa rn...@webkit.org wrote: Per my other thread about sharing IDLs between DumpRenderTree and WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner instead of adding yet another binding code. Another missing feature is producing pixel results. However, I'm currently concentrating on text results, as I think the biggest benefit is the ability to run storage tests on the real storage implementation. That sounds great to me but we definitely need to support pixel results eventually. I'm more than happy to help you on that but that requires the codebase to be moved into WebKit repository. Here's the basic problem: content_shell depends on content, so moving this on the webkit repository would mean that webkit depended on content. Another solution would be to formalize the test shell API our current layout test controller in webkit uses (Tools/DumpRenderTree/chromium), and add a method to chromium's webkit support library that returns a webview that supports all the hooks required. The webview could then either be implemented by test_shell or by content_shell I don't know any architectural details of content_shell so it's hard for me to comment on this matter, but I don't see a problem in WebKitChromiumTestRunner (hypothetical name to be consistent with WebKitTestRunner) depending on content_shell given that WebKitTestRunner also depends on WebKit2 API. Today, when somebody adds e.g. a new callback to LayoutTestController, they can just patch DumpRenderTree. If our test runner lived (in large parts) in chromium's repository, such a change would require a patch to chromium. This would either put an additional burden on all WebKit developers, or our test runner constantly gets out of sync. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Running layout tests for the chromium port, using the multi-processing architecture and fully sandboxed
On Fri, Jun 8, 2012 at 1:09 PM, Jochen Eisinger joc...@chromium.org wrote: On Fri, Jun 8, 2012 at 9:55 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Jun 8, 2012 at 12:51 PM, Jochen Eisinger joc...@chromium.orgwrote: On Fri, Jun 8, 2012 at 9:41 PM, Ryosuke Niwa rn...@webkit.org wrote: Per my other thread about sharing IDLs between DumpRenderTree and WebKitTestRunner, I would like to see us sharing IDL with WebKitTestRunner instead of adding yet another binding code. Another missing feature is producing pixel results. However, I'm currently concentrating on text results, as I think the biggest benefit is the ability to run storage tests on the real storage implementation. That sounds great to me but we definitely need to support pixel results eventually. I'm more than happy to help you on that but that requires the codebase to be moved into WebKit repository. Here's the basic problem: content_shell depends on content, so moving this on the webkit repository would mean that webkit depended on content. Another solution would be to formalize the test shell API our current layout test controller in webkit uses (Tools/DumpRenderTree/chromium), and add a method to chromium's webkit support library that returns a webview that supports all the hooks required. The webview could then either be implemented by test_shell or by content_shell I don't know any architectural details of content_shell so it's hard for me to comment on this matter, but I don't see a problem in WebKitChromiumTestRunner (hypothetical name to be consistent with WebKitTestRunner) depending on content_shell given that WebKitTestRunner also depends on WebKit2 API. Today, when somebody adds e.g. a new callback to LayoutTestController, they can just patch DumpRenderTree. If our test runner lived (in large parts) in chromium's repository, such a change would require a patch to chromium. This would either put an additional burden on all WebKit developers, or our test runner constantly gets out of sync. Yes, that's why I'm suggesting that this new test runner needs to be in the WebKit repository. It's not acceptable to require WebKit contributors to modify code in the Chromium repository in order to refactor or add or remove features to the test runner. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
