It turns out that many (most?) of the CSS failures are because we no longer expose user-installed fonts, e.g. Ahem.
Options: - update lots of tests to load Ahem via @font-face (yuck) - allow Ahem to be used if installed (weird to special case one font, but probably ok) Dean > On 12 Oct 2018, at 03:26, Philip Jägenstedt <foo...@chromium.org> wrote: > > Alright, I've written a one-off script [1] to find the Safari-only > failures, and here's the output: > https://gist.github.com/foolip/4d410ce79416bcdce71feb212159a02e > > Barring bugs, each of linked tests or one of its subtests should be > failing in Safari Technology Preview and passing in stable versions of > Chrome, Edge and Firefox. > > Numerically, most of the failures are in css (622), encoding (135) and > html (60). With css, it's mostly css/CSS2. > > I hope looking through this may be of use to you! > > [1] https://github.com/foolip/ad-hoc-wpt-results-analysis > > On Mon, Oct 8, 2018 at 11:50 PM Philip Jägenstedt <foo...@chromium.org> wrote: >> >> That filtering capability unfortunately does not yet exist on wpt.fyi >> but it's a high priority and actively being worked on: >> https://github.com/web-platform-tests/wpt.fyi/issues/201 >> >> FWIW, I suspect that these purposes, comparing to the stable versions >> of all *other* browsers might be the most useful: >> https://wpt.fyi/results/?product=chrome%5Bstable%5D&product=edge%5Bstable%5D&product=firefox%5Bstable%5D&product=safari%5Bexperimental%5D&aligned >> >> Again, no way to filter on wpt.fyi, but I'll see if I can download the >> full results and write a quick script. >> >> On Thu, Oct 4, 2018 at 11:49 PM Ryosuke Niwa <rn...@webkit.org> wrote: >>> >>> Thanks for the intriguing data, Philip. >>> >>> Is there a way to get a list of tests where all other browsers pass but >>> Safari / WebKit fail? >>> >>> That would allow us to quickly identify the set of tests we can fix to >>> improve the interoperability across browsers right away. >>> >>> - R. Niwa >>> >>> On Tue, Oct 2, 2018 at 3:45 AM Philip Jägenstedt <foo...@chromium.org> >>> wrote: >>>> >>>> Hi WebKittens, >>>> >>>> Fresh off the bots, I'm excited to report more robust Safari results, >>>> and that Safari WPT pass rates are clearly improving! Thanks to the >>>> hard work of Mike Pennisi [1] we now have the first Safari 12 results: >>>> https://wpt.fyi/results/?sha=ee2e69bfb1&product=safari-12.0 >>>> >>>> This uses the same setup as for Safari Technology Preview, which has >>>> been running for a while [2] and are the results you see on the >>>> "experimental" view: >>>> https://wpt.fyi/results/?label=experimental >>>> >>>> This appears much more robust than the Safari 11 data we've collected >>>> from Sauce Labs, and we can see a massive improvement between Safari >>>> 11 and 12: >>>> https://wpt.fyi/results/?sha=ee2e69bfb1&product=safari-11.1&product=safari-12.0&diff >>>> >>>> This lumps together infrastructure improvements as well as Safari >>>> 11->12 improvements, but improvements in service-workers/ [3] stands >>>> out, as well as in webdriver/, referrer-policy/, css/css-align/, and >>>> others. (The effect of moving away from Sauce is mainly less >>>> timeouts.) >>>> >>>> Also very interesting is to compare Safari 12 stable to TP: >>>> https://wpt.fyi/results/?sha=ee2e69bfb1&product=safari-12.0&product=safari-12.1&diff >>>> >>>> One can tell that work is going in canvas-related things, >>>> web-animations/, css/css-logical/ and more! \o/ >>>> >>>> I hope you'll all find these results valuable, and please report bugs >>>> or feature requests here: >>>> https://github.com/web-platform-tests/wpt.fyi/issues >>>> >>>> P.S. We're also trying to use use these diff views to spot >>>> regressions. It's a bit hard to use, [4] but a fix in in progress [5] >>>> and I might check back here when that works. I'll append to the end of >>>> this email a non-exhaustive list of possible regressions already >>>> possible to spot. >>>> >>>> [1] https://github.com/web-platform-tests/results-collection/issues/604 >>>> [2] https://wpt.fyi/test-runs?labels=safari,experimental >>>> [3] >>>> https://wpt.fyi/results/service-workers?sha=ee2e69bfb1&product=safari-11.1&product=safari-12.0&diff=true >>>> [4] https://github.com/web-platform-tests/wpt.fyi/issues/411 >>>> [5] https://github.com/web-platform-tests/wpt.fyi/pull/609 >>>> >>>> P.P.S. Possible regressions in Safari TP: >>>> https://wpt.fyi/results/css/vendor-imports/mozilla/mozilla-central-reftests/shapes1?sha=ee2e69bfb1&product=safari-12.0&product=safari-12.1 >>>> https://wpt.fyi/results/service-workers/service-worker/extendable-event-async-waituntil.https.html?sha=ee2e69bfb1&product=safari-12.0&product=safari-12.1 >>>> https://wpt.fyi/results/service-workers/service-worker/skip-waiting-installed.https.html?sha=ee2e69bfb1&product=safari-12.0&product=safari-12.1 >>>> _______________________________________________ >>>> webkit-dev mailing list >>>> webkit-dev@lists.webkit.org >>>> https://lists.webkit.org/mailman/listinfo/webkit-dev > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
