Thanks for this Philip. I have started raising bugs and blocking https://bugzilla.mozilla.org/show_bug.cgi?id=1498357.
David On Fri, 14 Dec 2018 at 08:41, Philip Jägenstedt <foo...@chromium.org> wrote: > On Fri, Oct 19, 2018 at 2:42 PM Philip Jägenstedt <foo...@chromium.org> > wrote: > > > > On Wed, Oct 17, 2018 at 11:53 PM Boris Zbarsky <bzbar...@mit.edu> wrote: > > > > > > On 10/13/18 3:27 AM, Philip Jägenstedt wrote: > > > > Fiddling with these rules can reveal lots > > > > more potential issues, and if you like I could provide reports on > that too. > > > > > > I would be pretty interested in that, yes. In particular, a report > > > where there is 1 "not PASS and not FAIL" and 3 "PASS" would be pretty > > > helpful, I suspect. > > > > Rerunning my script it's apparent that unreliable Edge results [1] > > leads to the same tests being considered lone failures or not for the > > other browsers. So, I've use the same set of runs for this report of > > what you suggested: > > https://gist.github.com/foolip/e6014c9bcc8ca405219bf18542eb5d69 > > > > It's not a long list, so I checked them all and they are timeouts. > > This is sometimes the failure mode for genuine problems, so looking > > over these might be valuable. > > Given the recent news [1] it won't be as relevant to consider the > status of EdgeHTML for prioritization in other engines. Given that and > the unreliable results, I've updated my script to consider only > Chrome, Firefox and Safari. I also the reports auto-updating on a > daily basis: > > https://foolip.github.io/ad-hoc-wpt-results-analysis/chrome-lone-failures.html > > https://foolip.github.io/ad-hoc-wpt-results-analysis/firefox-lone-failures.html > > https://foolip.github.io/ad-hoc-wpt-results-analysis/safari-lone-failures.html > > [1] https://github.com/MicrosoftEdge/MSEdge/blob/master/README.md > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
