Thanks for clarifying the test situation, Mustafa! :) Can you also answer Rick's question regarding the impact of this change on parsed URLs? (vs. typed or pasted URL, that you already described)
On Fri, Dec 2, 2022 at 12:40 AM Mustafa Emre Acer <mea...@chromium.org> wrote: > Hi Philip, > > Pretty sure the remaining failures with URLs with "ß" are due to > crbug.com/724018. In fact a quick hack reduced the failures down to 28: > https://chromium-review.googlesource.com/c/chromium/src/+/4072454 > > While related to IDNA, it's a different issue and isn't affected by this > change. > > On Thu, Dec 1, 2022 at 3:07 AM Philip Jägenstedt <foo...@chromium.org> > wrote: > >> I see, so if we compare the expectations of the default setup >> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/url/toascii.window-expected.txt> >> to virtual test suite >> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/virtual/idna-2008/external/wpt/url/toascii.window-expected.txt>, >> we see the improvement from 154 failures to 73. Yay! >> >> Are those remaining failures for reasons unrelated to IDNA processing? >> There are still tests with "ß" in the name that fail, but I'm not sure if >> it's expected or not. >> >> On Wed, Nov 30, 2022 at 6:39 PM Mustafa Emre Acer <mea...@chromium.org> >> wrote: >> >>> There are actually tests, but as a virtual test suite since the >>> implementation is currently behind a flag: >>> >>> https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/virtual/idna-2008/ >>> >>> Chrome Status form asked for a link to wpt.fyi and I couldn't figure out >>> how to link to a virtual test suite so I said no. Updated the CS entry. >>> >>> On Wed, Nov 30, 2022 at 9:00 AM Philip Jägenstedt <foo...@chromium.org> >>> wrote: >>> >>>> Hi Mustafa, >>>> >>>> Thanks for much for working on this. The initial email says this isn't >>>> tested by WPT, but I think this is the change that will make this test >>>> (part of Interop 2022) pass: >>>> >>>> https://wpt.fyi/results/url/toascii.window.html?label=experimental&label=master&product=chrome&product=firefox&product=safari&aligned&view=interop&q=label%3Ainterop-2022-webcompat >>>> >>>> Is that right? >>>> >>>> Best regards, >>>> Philip >>>> >>>> On Wed, Nov 30, 2022 at 4:48 PM Rick Byers <rby...@chromium.org> wrote: >>>> >>>>> Thanks for investing in this alignment! Having a URL that goes one >>>>> place in Chrome and somewhere different in Safari/Firefox seems like a >>>>> very >>>>> bad thing in principle to me :-) >>>>> >>>>> Your metrics and comments are around user-typed/pasted URLs. Does this >>>>> change somehow impact only that, not URLs parsed from HTML and CSS? If so >>>>> then I can understand why there's no WPTs for this. But if not then we'd >>>>> definitely need confidence in the WPT tests and probably some more compat >>>>> analysis. >>>>> >>>>> On Wed, Nov 30, 2022 at 8:35 AM 'Yifan Luo' via blink-dev < >>>>> blink-dev@chromium.org> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Wednesday, November 30, 2022 at 6:37:57 AM UTC+1 >>>>>> yoav...@chromium.org wrote: >>>>>> >>>>>>> Thanks for working on alignment here!! >>>>>>> >>>>>>> On Tue, Nov 29, 2022 at 7:30 AM 'Harald Alvestrand' via blink-dev < >>>>>>> blin...@chromium.org> wrote: >>>>>>> >>>>>>>> This IDNA 2008 author applauds your decision. >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Nov 28, 2022 at 10:16 PM Mustafa Emre Acer < >>>>>>>> mea...@chromium.org> wrote: >>>>>>>> >>>>>>>>> Contact emailsmea...@chromium.org >>>>>>>>> >>>>>>>>> Specificationhttps://unicode.org/reports/tr46 >>>>>>>>> >>>>>>>>> Summary >>>>>>>>> >>>>>>>>> Enable IDNA 2008 in Non-Transitional Mode for URL processing, >>>>>>>>> aligning Chrome's behavior with Firefox and Safari. Chrome currently >>>>>>>>> uses >>>>>>>>> IDNA 2008 in Transitional Mode in URL processing. The main difference >>>>>>>>> between Transitional and Non-Transitional Mode is the handling of four >>>>>>>>> characters known as deviation characters: ß (LATIN SMALL LETTER SHARP >>>>>>>>> S), ς >>>>>>>>> (GREEK SMALL LETTER FINAL SIGMA), ZWJ (Zero width joiner) and ZWNJ >>>>>>>>> (Zero >>>>>>>>> width non-joiner). In Transitional mode, deviation characters are >>>>>>>>> handled >>>>>>>>> the same as IDNA2003: ß is mapped to ss, ς is mapped to σ, and ZWJ >>>>>>>>> and ZWNJ >>>>>>>>> are deleted. In Non-Transitional mode, domains containing these >>>>>>>>> characters >>>>>>>>> are allowed in domain names without mapping, and thus can resolve to >>>>>>>>> different IP addresses. For example, typing "faß.de >>>>>>>>> <http://fass.de>" in Chrome and Firefox opens different sites >>>>>>>>> today. Enabling Non-Transitional IDNA in Chrome will allow deviation >>>>>>>>> characters in domain names. Firefox and Safari already made this >>>>>>>>> change in >>>>>>>>> 2016 and continue to use Non-Transitional URL processing. >>>>>>>>> >>>>>>>>> >>>>>>>>> Blink componentUI>Security>UrlFormatting >>>>>>>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:UI%3ESecurity%3EUrlFormatting> >>>>>>>>> >>>>>>>>> Search tagsidna <https://chromestatus.com/features#tags:idna> >>>>>>>>> >>>>>>>>> TAG reviewThis feature addresses conformance to an existing spec >>>>>>>>> and other browsers already do it. >>>>>>>>> >>>>>>>>> TAG review statusNot applicable >>>>>>>>> >>>>>>>>> Risks >>>>>>>>> >>>>>>>>> >>>>>>>>> Interoperability and Compatibility >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> *Gecko*: Shipped/Shipping ( >>>>>>>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1218179) >>>>>>>>> >>>>>>>>> *WebKit*: Shipped/Shipping ( >>>>>>>>> https://trac.webkit.org/changeset/208902/webkit) >>>>>>>>> >>>>>>>>> *Web developers*: No signals >>>>>>>>> >>>>>>>>> *Other signals*: >>>>>>>>> >>>>>>>>> Security >>>>>>>>> >>>>>>>>> This change introduces a potential security issue where a domain >>>>>>>>> pointing to one IP may start pointing to another IP. As an example, >>>>>>>>> IDNA2003 and Transitional IDNA-2008 maps faß.de <http://fass.de> >>>>>>>>> to fass.de (ß is a deviation character). Non-Transitional >>>>>>>>> IDNA2008 maps it to xn--fa-hia.de which is the punycode >>>>>>>>> representation of faß.de <http://fass.de>. Typing "faß.de >>>>>>>>> <http://fass.de>" in Chrome and Firefox currently opens different >>>>>>>>> sites. Main mitigations discussed were domain bundling / blocking >>>>>>>>> where >>>>>>>>> registrars bundle domain names (e.g. registering faß.de >>>>>>>>> <http://fass.de> along with fass.de) or block the alternative >>>>>>>>> domain name (e.g. disallow faß.de <http://fass.de> if fass.de is >>>>>>>>> registered). According to data from Chrome 106 and 107: - Less than >>>>>>>>> 0.001% >>>>>>>>> of user-typed or pasted main frame navigations had a deviation >>>>>>>>> character in >>>>>>>>> the hostname. This excludes link clicks and renderer initiated >>>>>>>>> navigations, >>>>>>>>> so the percentage of affected domains among all navigations is even >>>>>>>>> lower. >>>>>>>>> - Only one hostname had a deviation character and had more than 50 >>>>>>>>> impressions over a 28 day period (fußball.de <http://fussball.de>). >>>>>>>>> Both fußball.de <http://fussball.de> and fussball.de have the >>>>>>>>> same owner so this change doesn't affect them. Thus, typing domain >>>>>>>>> names >>>>>>>>> with deviation characters is very rare. Domain bundling / blocking >>>>>>>>> aren't >>>>>>>>> blockers as this change won't have a significant impact on >>>>>>>>> navigations. >>>>>>>>> Finally, Firefox and Safari have been using Non-Transitional IDNA 2008 >>>>>>>>> since 2016 without issues. >>>>>>>>> >>>>>>>>> >>>>>>>>> WebView application risks >>>>>>>>> >>>>>>>>> Does this intent deprecate or change behavior of existing APIs, >>>>>>>>> such that it has potentially high risk for Android WebView-based >>>>>>>>> applications? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Debuggability >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Will this feature be supported on all six Blink platforms >>>>>>>>> (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?Yes >>>>>>>>> >>>>>>>>> Is this feature fully tested by web-platform-tests >>>>>>>>> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> >>>>>>>>> ?No >>>>>>>>> >>>>>>>> >>>>>>> Why not? >>>>>>> >>>>>> There seems to be some tests written by apple >>>>>> https://github.com/web-platform-tests/wpt/pull/4794. However, same >>>>>> question here: Why not? >>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> DevTrial instructions >>>>>>>>> https://bugs.chromium.org/p/chromium/issues/detail?id=694157#c70 >>>>>>>>> >>>>>>>>> Flag nameuse-idna2008-non-transitional >>>>>>>>> >>>>>>>>> Requires code in //chrome?False >>>>>>>>> >>>>>>>>> Tracking bug >>>>>>>>> https://bugs.chromium.org/p/chromium/issues/detail?id=694157 >>>>>>>>> >>>>>>>>> Launch bughttps://launch.corp.google.com/launch/4224656 >>>>>>>>> >>>>>>>>> Estimated milestones >>>>>>>>> DevTrial on desktop 110 >>>>>>>>> DevTrial on Android 110 >>>>>>>>> >>>>>>>>> Anticipated spec changes >>>>>>>>> >>>>>>>>> Open questions about a feature may be a source of future web >>>>>>>>> compat or interop issues. Please list open issues (e.g. links to known >>>>>>>>> github issues in the project for the feature specification) whose >>>>>>>>> resolution may introduce web compat/interop risk (e.g., changing to >>>>>>>>> naming >>>>>>>>> or structure of the API in a non-backward-compatible way). >>>>>>>>> >>>>>>>>> >>>>>>>>> Link to entry on the Chrome Platform Status >>>>>>>>> https://chromestatus.com/feature/5105856067141632 >>>>>>>>> >>>>>>>>> This intent message was generated by Chrome Platform Status >>>>>>>>> <https://chromestatus.com/>. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "blink-dev" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to blink-dev+...@chromium.org. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHafXh3rh2Hh35Pv1wNg8vBzUMy13NY%2Bh1y8HmHQrH2aD1i_Lg%40mail.gmail.com >>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHafXh3rh2Hh35Pv1wNg8vBzUMy13NY%2Bh1y8HmHQrH2aD1i_Lg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "blink-dev" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to blink-dev+...@chromium.org. >>>>>>>> >>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOqqYVFsCyiMPA4eVWZy-a%2Bv6XCgcYkCDzhq7XVSP4O_rQFFyA%40mail.gmail.com >>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOqqYVFsCyiMPA4eVWZy-a%2Bv6XCgcYkCDzhq7XVSP4O_rQFFyA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "blink-dev" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to blink-dev+unsubscr...@chromium.org. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/e83440db-ff48-46c5-8ca3-25a444cc063an%40chromium.org >>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/e83440db-ff48-46c5-8ca3-25a444cc063an%40chromium.org?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "blink-dev" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to blink-dev+unsubscr...@chromium.org. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFUtAY8_-3_YWsRzmCk4mLQgTU6eaUHQ09%3Dku4dD4_gbks1VNQ%40mail.gmail.com >>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFUtAY8_-3_YWsRzmCk4mLQgTU6eaUHQ09%3Dku4dD4_gbks1VNQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "blink-dev" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/a/chromium.org/d/topic/blink-dev/8pxRArGQlS4/unsubscribe >> . >> To unsubscribe from this group and all its topics, send an email to >> blink-dev+unsubscr...@chromium.org. >> To view this discussion on the web visit >> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYfmsGWwqFiRr2OKiVh2aq2AC7yoagUHJrPrdiVv8vJ7-Q%40mail.gmail.com >> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYfmsGWwqFiRr2OKiVh2aq2AC7yoagUHJrPrdiVv8vJ7-Q%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfW%3DkiaMYZgt60U2dKZgdbgiDYQ3%3DTPpm0vTOAB7atKwhA%40mail.gmail.com.
