Re: [webkit-dev] Switching to new-run-webkit-tests
One of the Leopard Debug slaves (apple-mac-pro-6) is still wedged after the transition: http://build.webkit.org/builders/Leopard%20Intel%20Debug%20%28Tests%29/builds/32142/steps/layout-test/logs/stdio If someone could run killall httpd, or just restart the machine, that'd be wonderful. :) In other news, Qt, Gtk, Leopard and Snow Leopard all seem to be using NRWT successfully. We've had very few reports of trouble over the last 24 hours. I'm working on the last few details for WebKit2 and plan to enable NRWT on the WebKit2 bots tomorrow. -eric On Wed, Jul 6, 2011 at 2:10 AM, Eric Seidel e...@webkit.org wrote: Update: Leopard Release, Gtk and Qt have been successfully transitioned. The Leopard Debug bot has a stray httpd process (without corresponding /tmp/WebKit/httpd.pid) and will require manual intervention: http://build.webkit.org/builders/Leopard%20Intel%20Debug%20%28Tests%29/builds/32086/steps/layout-test/logs/stdio If someone at Apple could give it a kick, I'd be most grateful. I'll work on the WebKit2 ports more tomorrow. I'll need some help with Windows eventually. -eric p.s. As previously noted, run-webkit-tests is calling new-run-webkit-tests with --child-processes=1. So the bots are running in very very slow mode (about as fast as ORWT was). We'll turn on parallel testing with new-run-webkit-tests once we've transitioned all ports. On Tue, Jul 5, 2011 at 10:53 PM, Eric Seidel e...@webkit.org wrote: Update: Snow Leopard - Successful transition. Leopard - Had to roll-back due to a bug in webkitdirs.pm which errors in both ORWT and NRWT, but causes NRWT to fail hard. https://bugs.webkit.org/show_bug.cgi?id=63973 Gtk - Bot seems hard-hung (unclear if NRWT related). Waiting for assistance from a maintainer. Qt - All tests were crashing before I moved to NRWT (http://build.webkit.org/builders/Qt%20Linux%20Release/builds/35024/steps/layout-test/logs/stdio) all tests still crash after moving to NRWT (http://build.webkit.org/builders/Qt%20Linux%20Release/builds/35032/steps/layout-test/logs/stdio). Waiting for the Qt guys to wake. Windows - Have not attempted transition, will likely need some help. https://bugs.webkit.org/show_bug.cgi?id=38756 WebKit2 - Working on remaining blockers, plan to switch over tonight or tomorrow. https://bugs.webkit.org/show_bug.cgi?id=56729 Thanks for your patience in this process. Please let me know if you see any issues you think we may have missed. We're still watching the bots and fixing issues as they appear. -eric On Tue, Jul 5, 2011 at 4:03 PM, Eric Seidel e...@webkit.org wrote: We've turned NRWT back on for the WebKit1 Snow Leopard bots. We believe we've solved the http-lock issue and will be monitoring the bots. -eric On Fri, Jul 1, 2011 at 5:54 PM, Adam Barth aba...@webkit.org wrote: Thanks for your patience with the disruptions on the tree today. The bots are having some trouble with the HTTP lock that don't manifest locally or on our test slave. I've turned NRWT back off while we try to sort that out. Thanks, Adam On Fri, Jul 1, 2011 at 12:49 AM, Adam Barth aba...@webkit.org wrote: Sorry, no switch over tonight. The WebKit2 build is too destroyed to have any confidence that we're doing things correctly and the Snow Leopard buildbot appears to have an errant HTTP server bound to port 8080. :( Adam On Thu, Jun 30, 2011 at 11:47 AM, Adam Barth aba...@webkit.org wrote: According to https://bugs.webkit.org/show_bug.cgi?id=34984, there are only two remaining blocking issues: 1) Teaching new-run-webkit-tests to understand the difference between WebProcess crashes and other sorts of crashes in WebKit2. 2) Fixing some issues with the Apple Windows port. Once we get (1) fixed (hopefully today), we're going to start moving (non-Windows) ports over to new-run-webkit-tests (hopefully tonight). I suspect more issues will crop up once we actually flip the switch. Please do not hesitate to contact Eric or myself (either via #webkit or via bugs.webkit.org). We'll try to respond to any issues that arise as quickly as possible. Thanks for your patience. Adam On Mon, Jun 27, 2011 at 4:26 PM, Xan Lopez x...@gnome.org wrote: On Tue, Jun 28, 2011 at 1:17 AM, Eric Seidel e...@webkit.org wrote: There appear to be 6 remaining blocking issues: https://bugs.webkit.org/showdependencytree.cgi?id=34984hide_resolved=1 We would like to hear from others who have tried new-run-webkit-tests, if they have issues which they believe should block migrating to NRWT. (If so, please file and block the master bug!) I can see the GTK+ port thing with Xvfb is there already, so not a lot to add. NWRT is more sensitive to slow tests than the old infrastructure, so we had to add a bunch of them to the expectations file; I don't think this is particularly bad. In any case with the Xvfb patch and my local expectations file things run beautifully and way faster than before, so looking
Re: [webkit-dev] Remote Debugging v8 - Android
On Thu, Jul 7, 2011 at 10:46 AM, JOSE MANUEL CANTERA FONSECA j...@tid.eswrote: Hi there, ** ** I have two technical questions concerning the Webkit Remote Debugging Protocol. ** ** a/ What is the difference / relationship between this protocol and the v8 remote debugging capabilities? ** Webkit Remote Debugging Protocol is the one Web Inspector uses. It allows debugging all flavors of the running page, not only JavaScript. V8 protocol only exposes JavaScript debugging and is browser-agnostic. You should stick with the WebKit's one in case you are interested in WebKit-based browsers debugging. Use V8's one when you intend to debug applications such as node.js. ** b/ How this protocol could be enabled for an Android device? ** You need to check with Android port owners on whether they intend to enable INSPECTOR in their builds and expose remote debugging capabilities. ** all the best ** ** thanks ** ** -- Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo. This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at. http://www.tid.es/ES/PAGINAS/disclaimer.aspx ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)
On 07/06/2011 07:24 PM, Eric Seidel wrote: On Wed, Jul 6, 2011 at 10:18 AM, Xan Lopezx...@gnome.org wrote: On Wed, Jul 6, 2011 at 6:29 PM, Eric Seidele...@webkit.org wrote: NRWT uses both! It will read in all the port's Skipped files, covert them to SKIP text_expectations, and add them to your test_expectations file. http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit.py#L309 For better or worse, NRWT will error out, if you have duplicates in your test_expectations file, including duplicates between your test_expectations file and your Skipped lists. Right, this is what I meant in another email when I said you are not supposed to use both. Cannot really see a sane use case for this to be honest. When I transitioned I basically converted Skipped locally to the new format, got tons of duplicated errors, figured out what was going on and deleted then deleted Skipped. Maybe this is done so that you can leave Skipped as it is and start gradually adding stuff to the new file? This was done to make it possible to bring up NRWT on Mac over a year ago. :) I'm happy to look at moving to a different configuration now that the project has (mostly) moved to NRWT. So long term the best is to move from Skipped to text_expectations. But I worry about the lack of the cascading logic. At some point we decided that we need it in the old system. Why do we think that we won't need it with NRWT? I think the cascading reduce the cost of maintaining the skipped lists. WebKit2 is the best example. We have a common skip list that lists all the tests that are failing due to a common WebKit2 specific reason. In that way, I can skip tests that appearing when I work and Apple folks are sleeping and they don't need to worry about that and the same is true in the reverse direction. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Switching to new-run-webkit-tests
On Jul 7, 2011, at 2:55 AM, Eric Seidel wrote: One of the Leopard Debug slaves (apple-mac-pro-6) is still wedged after the transition: http://build.webkit.org/builders/Leopard%20Intel%20Debug%20%28Tests%29/builds/32142/steps/layout-test/logs/stdio If someone could run killall httpd, or just restart the machine, that'd be wonderful. :) It looks like apple-xserve-3, apple-macpro-5, and apple-macpro-6 were all having trouble. I rebooted all of them. -Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Remote Debugging v8 - Android
You need to check with Android port owners on whether they intend to enable INSPECTOR in their builds and expose remote debugging capabilities. I'm afraid we have no plans right now to enable this feature. Steve -- Google UK Limited Registered Office: Belgrave House, 76 Buckingham Palace Road, London SW1W 9TQ Registered in England Number: 3977902 ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] JavaScriptCore on Android
Hi, This mailing list is for WebKit development not WebKit general questions, use webkit-help for your next enquiries. I am trying to use JavaScriptCore in my Android application. First, understand that the Android bindings in WebKit have been removed (that includes JSC), thus I would advise asking the question on an Android specific mailing list as people here obviously don't do Android development (apart from some people). I build JavaScriptCore as a static library and I developed a JNI layer to access JSC methods. All my classes compile fine, but at the end of the build process (i guess in the linker action) i got some errors like undefined reference to JSStringCreateWithUTF8CString. I was not able to correctly link my code with the JavaScriptCore static library. I would double check that the symbols are present in your static library before looking any further as it could be the root cause. If that does not work, look for some help on the Android.mk side. Hope it helps, Julien ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)
I do not know the history as to why Chromium removed support for test_expectations cascading. Ideally we would have fewer test expectations, not more in the future. :) On Thu, Jul 7, 2011 at 3:16 AM, Balazs Kelemen kbal...@webkit.org wrote: On 07/06/2011 07:24 PM, Eric Seidel wrote: On Wed, Jul 6, 2011 at 10:18 AM, Xan Lopezx...@gnome.org wrote: On Wed, Jul 6, 2011 at 6:29 PM, Eric Seidele...@webkit.org wrote: NRWT uses both! It will read in all the port's Skipped files, covert them to SKIP text_expectations, and add them to your test_expectations file. http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit.py#L309 For better or worse, NRWT will error out, if you have duplicates in your test_expectations file, including duplicates between your test_expectations file and your Skipped lists. Right, this is what I meant in another email when I said you are not supposed to use both. Cannot really see a sane use case for this to be honest. When I transitioned I basically converted Skipped locally to the new format, got tons of duplicated errors, figured out what was going on and deleted then deleted Skipped. Maybe this is done so that you can leave Skipped as it is and start gradually adding stuff to the new file? This was done to make it possible to bring up NRWT on Mac over a year ago. :) I'm happy to look at moving to a different configuration now that the project has (mostly) moved to NRWT. So long term the best is to move from Skipped to text_expectations. But I worry about the lack of the cascading logic. At some point we decided that we need it in the old system. Why do we think that we won't need it with NRWT? I think the cascading reduce the cost of maintaining the skipped lists. WebKit2 is the best example. We have a common skip list that lists all the tests that are failing due to a common WebKit2 specific reason. In that way, I can skip tests that appearing when I work and Apple folks are sleeping and they don't need to worry about that and the same is true in the reverse direction. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)
One difference with the chromium port is that we try to use a single test_expectations.txt that covers all platforms and OS versions (win xp, vista, 7, mac leopard, snow leopard, linux 32, 64, GPU vs CPU, Debug vs Release). The tokens to the left of the test name specify what configuration the expectation applies to. Because of that, there hasn't been much need for multiple test_expectations.txt files. There is some code already in NRWT for cascading test_expectations.txt. Currently, it's specific to the chromium port where we merge the test_expectations.txt in the webkit repo with a test_expectations.txt file in the chromium repo (it just concatenates them together). It would be pretty straight forward to make this code generic for all ports. It seems like we have a few options. We could have a separate test_expectations.txt per layout test platform directory and have cascade logic hard coded into NRWT or with an #include directive. At the other extreme, we could have a single monolithic test_expectations.txt file that knows about all platforms. Or something in the middle: have a test_expectations.txt for mac, mac-leopard, mac-snowleopard, one for qt*, one for all the WebKit2 ports, etc. I suspect we'll want to go with something in the middle. On Thu, Jul 7, 2011 at 10:06 AM, Maciej Stachowiak m...@apple.com wrote: On Jul 7, 2011, at 10:03 AM, Eric Seidel wrote: I do not know the history as to why Chromium removed support for test_expectations cascading. Ideally we would have fewer test expectations, not more in the future. :) The cascading is really really useful for supporting multiple different Mac OS X versions with different results, and WebKit2 as an orthogonal dimension. Perhaps one possibility is to have something like an include directive in the expectations file, so the cascading can be defined by the expectations files themselves, rather than hardcoded in scripts. Regards, Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)
On Thu, Jul 7, 2011 at 10:27 AM, Tony Chang t...@chromium.org wrote: One difference with the chromium port is that we try to use a single test_expectations.txt that covers all platforms and OS versions (win xp, vista, 7, mac leopard, snow leopard, linux 32, 64, GPU vs CPU, Debug vs Release). The tokens to the left of the test name specify what configuration the expectation applies to. Because of that, there hasn't been much need for multiple test_expectations.txt files. There is some code already in NRWT for cascading test_expectations.txt. Currently, it's specific to the chromium port where we merge the test_expectations.txt in the webkit repo with a test_expectations.txt file in the chromium repo (it just concatenates them together). It would be pretty straight forward to make this code generic for all ports. It seems like we have a few options. We could have a separate test_expectations.txt per layout test platform directory and have cascade logic hard coded into NRWT or with an #include directive. At the other extreme, we could have a single monolithic test_expectations.txt file that knows about all platforms. Or something in the middle: have a test_expectations.txt for mac, mac-leopard, mac-snowleopard, one for qt*, one for all the WebKit2 ports, etc. I suspect we'll want to go with something in the middle. Tony's description is spot-on. The only reason we don't support cascading expectations files is because it wasn't clear to me how we would want things to work (i.e., which of the choices above) and I wasn't able to get much input a few months ago. If there is a consensus, it will be easy to implement, so how do we actually want this to work? -- Dirk On Thu, Jul 7, 2011 at 10:06 AM, Maciej Stachowiak m...@apple.com wrote: On Jul 7, 2011, at 10:03 AM, Eric Seidel wrote: I do not know the history as to why Chromium removed support for test_expectations cascading. Ideally we would have fewer test expectations, not more in the future. :) The cascading is really really useful for supporting multiple different Mac OS X versions with different results, and WebKit2 as an orthogonal dimension. Perhaps one possibility is to have something like an include directive in the expectations file, so the cascading can be defined by the expectations files themselves, rather than hardcoded in scripts. Regards, Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)
On Jul 7, 2011, at 10:39 AM, Dirk Pranke wrote: On Thu, Jul 7, 2011 at 10:27 AM, Tony Chang t...@chromium.org wrote: One difference with the chromium port is that we try to use a single test_expectations.txt that covers all platforms and OS versions (win xp, vista, 7, mac leopard, snow leopard, linux 32, 64, GPU vs CPU, Debug vs Release). The tokens to the left of the test name specify what configuration the expectation applies to. Because of that, there hasn't been much need for multiple test_expectations.txt files. There is some code already in NRWT for cascading test_expectations.txt. Currently, it's specific to the chromium port where we merge the test_expectations.txt in the webkit repo with a test_expectations.txt file in the chromium repo (it just concatenates them together). It would be pretty straight forward to make this code generic for all ports. It seems like we have a few options. We could have a separate test_expectations.txt per layout test platform directory and have cascade logic hard coded into NRWT or with an #include directive. At the other extreme, we could have a single monolithic test_expectations.txt file that knows about all platforms. Or something in the middle: have a test_expectations.txt for mac, mac-leopard, mac-snowleopard, one for qt*, one for all the WebKit2 ports, etc. I suspect we'll want to go with something in the middle. Tony's description is spot-on. The only reason we don't support cascading expectations files is because it wasn't clear to me how we would want things to work (i.e., which of the choices above) and I wasn't able to get much input a few months ago. If there is a consensus, it will be easy to implement, so how do we actually want this to work? Out of the different options raised so far, I like the idea of having an include directive. Then ports can decide for themselves how much factoring is appropriate. I think one giant expectations file for all ports is probably too complicated to be manageable, but an include-based setup would let specific ports share one expectations file for many configurations if they wish. (Also and incidentally, I'd suggest renaming the file to TestExpectations or TestExpectations.txt to better match WebKit naming style, but this is a much more trivial issue.) Regards, Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] documentation on the test_expectations.txt format for NRWT, other info
Hi all, I've started adding a bunch of information on NRWT to the wiki. Some of the pages are stubs at the moment, but the test_expectation page is pretty complete. Other pages will be filled in over today and tomorrow. Those of you who are also familiar with the functionality, feel free to contribute: http://trac.webkit.org/wiki/NewRunWebKitTests http://trac.webkit.org/wiki/TestExpectations http://trac.webkit.org/wiki/HackingOnNewRunWebKitTests http://trac.webkit.org/wiki/LayoutTestsSearchPath I will also being going around and updating the existing pages to point to the new content where I can find it. Cheers, -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
