Re: [webkit-dev] Subpixel layout - requesting help for big rebaseline
Hi Dirk, On 08/22/2012 10:49 PM, Dirk Pranke wrote: The Chromium canaries now exit after 5000 failures or 1000 crashes/timeouts. Can the failure limit be increased to 1 for example? Levi/Emil were saying the rebaseline touches about 8000 cases. Otherwise we'd have to go through a more complicated process like Zan explains. Dominik ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Subpixel layout - requesting help for big rebaseline
On Fri, Aug 24, 2012 at 4:35 AM, Dominik Röttsches dominik.rottsc...@intel.com wrote: Hi Dirk, On 08/22/2012 10:49 PM, Dirk Pranke wrote: The Chromium canaries now exit after 5000 failures or 1000 crashes/timeouts. Can the failure limit be increased to 1 for example? Levi/Emil were saying the rebaseline touches about 8000 cases. Otherwise we'd have to go through a more complicated process like Zan explains. Despite the testing exiting after a certain number of failures, the newly-registered failures would still be possible to rebaseline. So technically you could do three or four rebaseline cycles and get the bots back into a normal state. Dominik __**_ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/**mailman/listinfo/webkit-devhttp://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Subpixel layout - requesting help for big rebaseline
You can mark the failing tests as failing tests or skip them in TestExpectations/Skipped with the change. And then you can rebase and unskip them without disturbing the bot. Dominik Röttsches írta: Hi Dirk, On 08/22/2012 10:49 PM, Dirk Pranke wrote: The Chromium canaries now exit after 5000 failures or 1000 crashes/timeouts. Can the failure limit be increased to 1 for example? Levi/Emil were saying the rebaseline touches about 8000 cases. Otherwise we'd have to go through a more complicated process like Zan explains. Dominik ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Subpixel layout - requesting help for big rebaseline
On Fri, Aug 24, 2012 at 4:42 AM, Žan Doberšek zandober...@gmail.com wrote: On Fri, Aug 24, 2012 at 4:35 AM, Dominik Röttsches dominik.rottsc...@intel.com wrote: Hi Dirk, On 08/22/2012 10:49 PM, Dirk Pranke wrote: The Chromium canaries now exit after 5000 failures or 1000 crashes/timeouts. Can the failure limit be increased to 1 for example? Levi/Emil were saying the rebaseline touches about 8000 cases. Otherwise we'd have to go through a more complicated process like Zan explains. Despite the testing exiting after a certain number of failures, the newly-registered failures would still be possible to rebaseline. So technically you could do three or four rebaseline cycles and get the bots back into a normal state. Both of these are fine solutions. I don't think we need to do the more complicated approach. As long as the gardener on duty is informed and ready. Again, it's a huge simplification that Emil and Levi have already audited the new results since the gardener can just bulk rebaseline them. Dominik __**_ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/**mailman/listinfo/webkit-devhttp://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Subpixel layout - requesting help for big rebaseline
Perhaps one option of doing such a huge rebaseline might be the following: 1. Get the bots for the affected port into a green state. 2. Add temporary expectations for every LayoutTests/ subdirectory and perhaps for subdirectories of LayoutTests/fast, LayoutTests/svg and LayoutTests/editing as well[1]. This way the test runs wouldn't be exiting early due to too many test failures (though this might occur when removing expectation for just one subdirectory, for instance LayoutTests/tables). 3. Land the patch at the start of the daily period in which commit frequency is low (probably before the PST morning). This will ensure no actual regressions originating from other commits would get rebaselined or cause problems (crashes, for instance). 4. Remove expectations for subdirectories one by one, letting the bots to perform one cycle and report any failures in that subdirectory, and use the tools to perform the rebaseline. But it's still a huge task, especially for chromium as there are at least three flavors of the port that would require special attention. These bots are also running pixel tests, so there'll be large patches committed that AFAIK used to cause problems (for instance the Git tree went out of sync). [1] In Chromium's case, it would help to get the JSON output of the test run with the rounding patch applied just to know where to expect failures and how many of them. This would also help to better specify directory-specific expectations. On Wed, Aug 22, 2012 at 1:13 PM, Dominik Röttsches dominik.rottsc...@intel.com wrote: Hi Chromium folks, Levi and Emil did a great job in implementing SubPixel Layout. I was looking into enabling it on WebKit EFL. Unfortunately, the initial rebaseline for EFL would be huge. That's why I hesitated to enable it right away. Instead I started looking into what the reasons are. I found one case where rounding should be added, here's my patch: https://bugs.webkit.org/show_bug.cgi?id=89572 Landing this makes the rebaselining efforts for ports enabling subpixel layout smaller, so IMHO it'd accelerate adoption. On the other hand, landing this patch requires a rebaselining for Chromium in the area of several thousand testcases. I do feel the pain there. So I'd appreciate your help for getting this one landed. According to their comments on bugzilla, Emil and Levi approve the change. Thanks, Dominik -- Dominik Röttsches dominik.rottsc...@intel.com dominik.rottsc...@intel.com ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Subpixel layout - requesting help for big rebaseline
Hi, As another alternative, could we gather all the Chromium bot admins and ask them to temporarily take the bot offline in non-peak hours, create rebaselines per bot with the patch applied, collect those as binary-compatible diffs in one place, and manually land those collected baselines, in short succession together with the patch? Dominik ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Subpixel layout - requesting help for big rebaseline
IIRC, the chromium bots no longer exit early due to a large number of failures (right Dirk?), only due to a large number of crashes/timeouts. So, it should be possible to commit this, wait for the bots to cycle, do all the rebaselines at once and commit that. The only delay is that we'd have to wait for the slowest of the bots to cycle. Given that Emil/Levi have already verified the correctness of the new results, the gardener can just hit the rebaseline all button in garden-o-matic, so it shouldn't be too arduous as long as there aren't a bunch of commits at the same time that also break a bunch of tests. In short, I don't think committing this is too big of a deal if you are willing to coordinate with the Chromium gardener to find a good time. The PST gardener today and tomorrow is Ken Russell. If you're in a non-PST time zone, the gardener though Monday is Dominic Cooney. My intuition is that early in the morning or late at night would be best since 1-6pm PST is when the majority of WebKit commits go in, causing other failures that might be confused by this patch, but that's really up to Ken/Dominic and you to figure out what's best. On Wed, Aug 22, 2012 at 6:02 AM, Dominik Röttsches dominik.rottsc...@intel.com wrote: Hi, As another alternative, could we gather all the Chromium bot admins and ask them to temporarily take the bot offline in non-peak hours, create rebaselines per bot with the patch applied, collect those as binary-compatible diffs in one place, and manually land those collected baselines, in short succession together with the patch? Dominik __**_ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/**mailman/listinfo/webkit-devhttp://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Subpixel layout - requesting help for big rebaseline
The Chromium canaries now exit after 5000 failures or 1000 crashes/timeouts. Ideally we'd hold off on this change until we can get some sort of a fix or workaround to https://bugs.webkit.org/show_bug.cgi?id=94665 though (and I'm working on this today), or life might be annoyingly painful for us. -- Dirk On Wed, Aug 22, 2012 at 10:09 AM, Ojan Vafai o...@chromium.org wrote: IIRC, the chromium bots no longer exit early due to a large number of failures (right Dirk?), only due to a large number of crashes/timeouts. So, it should be possible to commit this, wait for the bots to cycle, do all the rebaselines at once and commit that. The only delay is that we'd have to wait for the slowest of the bots to cycle. Given that Emil/Levi have already verified the correctness of the new results, the gardener can just hit the rebaseline all button in garden-o-matic, so it shouldn't be too arduous as long as there aren't a bunch of commits at the same time that also break a bunch of tests. In short, I don't think committing this is too big of a deal if you are willing to coordinate with the Chromium gardener to find a good time. The PST gardener today and tomorrow is Ken Russell. If you're in a non-PST time zone, the gardener though Monday is Dominic Cooney. My intuition is that early in the morning or late at night would be best since 1-6pm PST is when the majority of WebKit commits go in, causing other failures that might be confused by this patch, but that's really up to Ken/Dominic and you to figure out what's best. On Wed, Aug 22, 2012 at 6:02 AM, Dominik Röttsches dominik.rottsc...@intel.com wrote: Hi, As another alternative, could we gather all the Chromium bot admins and ask them to temporarily take the bot offline in non-peak hours, create rebaselines per bot with the patch applied, collect those as binary-compatible diffs in one place, and manually land those collected baselines, in short succession together with the patch? Dominik ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
