I'm planning to ship it behind a feature (enabled by default, after experimenting on pre-release channels) to allow emergency disabling.
On Fri, Aug 20, 2021 at 12:53 AM Yoav Weiss <yoavwe...@chromium.org> wrote: > OK, I'm convinced that these are not "real" hosts, and that breaking them > will not result in actual user-visible breakage. Would it be possible to > ship this with a server-side flag that'd enable us to quickly revert in > case we're wrong? > > On Thu, Aug 19, 2021 at 6:09 PM Matt Menke <mme...@google.com> wrote: > >> So given information on this thread, I think the ways these can succeed >> are possibly: >> 1) Entries in the HOSTS file. >> 2) Intermediary DNS servers or MitMs typo squatting, or actively >> attacking the user. >> 3) Local DNS servers providing additional DNS results. >> 4) Suffix search (which would trigger on, e.g., "1533.67.89" but not >> "67.89.1533") >> 5) mDNS >> 6) Local tools injecting DNS lookup results. >> >> Unfortunately, we don't have a good way to gather hard data on which are >> more common. Given that there are potential security implications here, >> I'm reluctant to wait for another round of data gathering, though we could >> probably distinguish cases 4) and 5) from the others, and 1) as well, at >> least on some platforms. Also not sure how useful just knowing about those >> cases would be. >> On Thursday, August 19, 2021 at 11:54:29 AM UTC-4 Harald Alvestrand wrote: >> >>> Thanks - I misremembered which end of the class B network got jammed >>> together. 67.89.31533 is indeed the one that is "legal" syntax, and so of >>> course 67.89.1 is too. >>> >>> >>> On Thu, Aug 19, 2021 at 3:20 PM Matt Menke <mme...@google.com> wrote: >>> >>>> Note that 127.0.1 is mapped to 127.0.0.1 by GURL, following the URL >>>> spec, so that would generally not make it to the DNS resolver (unless >>>> something tried to resolve it directly). Per the URL spec, "31533.67.89" >>>> would not be normalized by GURL, but "67.89.31533" would be converted to >>>> 67.89.123.45. My instrumentation was at the DNS layer, so "127.0.1" and >>>> "67.89.31533" would not show up as problematic hostnames in my metrics, >>>> though "31533.67.89" would. >>>> >>>> On Thu, Aug 19, 2021 at 9:10 AM Harald Alvestrand <h...@google.com> >>>> wrote: >>>> >>>>> An interesting property of all-numeric hostnames is that they *may* be >>>>> legitimate IPv4 addresses using highly archaic IP address formats - we're >>>>> so used to the 123.45.67.89 syntax that we forget that 31533.67.89 once >>>>> was >>>>> regarded as a legitimate way to encode the same address (class B >>>>> notation). >>>>> >>>>> There are also certain resolvers that will "helpfully" map an >>>>> all-numeric hostname presented in DNS to an IP address without asking >>>>> anyone. >>>>> So if those two bugs (or "archaic features") occur together, the >>>>> result may be a successful resolution. >>>>> >>>>> No idea why it would occur more often on Android than on Windows, >>>>> though. And my Linux boxes don't resolve 127.0.1 to anything. >>>>> >>>>> >>>>> >>>>> On Thu, Aug 19, 2021 at 2:42 PM Yoav Weiss <yoav...@chromium.org> >>>>> wrote: >>>>> >>>>>> Interesting! What happens then in the "successful resolution" case >>>>>> Matt mentioned? >>>>>> >>>>>> On Thu, Aug 19, 2021 at 2:39 PM Harald Alvestrand <h...@google.com> >>>>>> wrote: >>>>>> >>>>>>> Department of odd facts: >>>>>>> >>>>>>> - The ICANN rules for new TLDs forbid all top level domain names >>>>>>> that start with a digit >>>>>>> - The IDNA rules for bidirectional scripts forbid domain names that >>>>>>> start with a digit (Unicode bidi afficandoes will know why) >>>>>>> - The only real reason why leading digits aren't outlawed in domain >>>>>>> names at the second level is 3com. >>>>>>> >>>>>>> It seems safe to say that no legitimate fully qualified hostname >>>>>>> will ever have a last component consisting only of digits. >>>>>>> That means the only time we could get a legitimate hostname is for >>>>>>> something that has to be resolved via a search path. >>>>>>> >>>>>>> >>>>>>> On Thu, Aug 19, 2021 at 2:33 PM Yoav Weiss <yoav...@chromium.org> >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Aug 19, 2021 at 2:28 PM Matt Menke <mme...@google.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I created the title using Chrome Status's deprecation template, so >>>>>>>>> any confusion should be blamed on that. >>>>>>>>> >>>>>>>> >>>>>>>> +Jason Robbins - on the title issues. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> I used the "Draft Intent to Deprecate and Remove email" button, >>>>>>>>> and assume I'd need to do a "Draft Intent to Ship email" before >>>>>>>>> shipping to >>>>>>>>> stable, after a 50% trial on prerelease channels. >>>>>>>>> >>>>>>>> >>>>>>>> There's no need for 2 emails for removals. We can discuss the full >>>>>>>> deprecation, experimentation/trials and removal on stable here. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Aug 19, 2021 at 3:15 AM Yoav Weiss <yoav...@chromium.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Aug 18, 2021 at 11:36 PM Matt Menke <mme...@google.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Aug 18, 2021 at 5:23 PM Yoav Weiss <yoav...@chromium.org> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Aug 18, 2021 at 11:18 PM Matt Menke <mme...@google.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Aug 18, 2021 at 4:53 PM Yoav Weiss < >>>>>>>>>>>>> yoav...@chromium.org> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Aug 18, 2021 at 8:47 PM 'Matt Menke' via blink-dev < >>>>>>>>>>>>>> blin...@chromium.org> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Contact emailsmme...@google.com >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ExplainerNone >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Specificationhttps://url.spec.whatwg.org/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Summary >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Most hostnames that aren't valid IPv4 addresses, but end in >>>>>>>>>>>>>>> numbers are treated as valid, and looked up via DNS (e.g., >>>>>>>>>>>>>>> http://foo.127.1/). Per the Public Suffix List spec, the >>>>>>>>>>>>>>> eTLD+1 of the hostname in that URL should be "127.1". If that >>>>>>>>>>>>>>> is ever fed >>>>>>>>>>>>>>> back into a URLs, "http://127.1/ <http://127.0.0.1/>" is >>>>>>>>>>>>>>> mapped to "http://127.0.0.1/"; by the URL spec, which seems >>>>>>>>>>>>>>> potentially dangerous. "127.0.0.0.1" could also potentially be >>>>>>>>>>>>>>> used to >>>>>>>>>>>>>>> confuse users. We want to reject URLs with these hostnames. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Blink componentInternals>Network>DNS >>>>>>>>>>>>>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Internals%3ENetwork%3EDNS> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Motivation >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Most hostnames that aren't valid IPv4 addresses, but end in >>>>>>>>>>>>>>> numbers are treated as valid hostnames, and looked up via DNS. >>>>>>>>>>>>>>> Example >>>>>>>>>>>>>>> hostnames: 127.0.0.0.1, foo.0.1, 10.0.0.09, 08.1.2.3. These can >>>>>>>>>>>>>>> be >>>>>>>>>>>>>>> problematic for the following reason: * "http://foo.127.1/"; >>>>>>>>>>>>>>> has an eTLD+1 of "127.1", per the public suffix list spec. If >>>>>>>>>>>>>>> that's ever >>>>>>>>>>>>>>> used as the hostname in a new URL, however, as in " >>>>>>>>>>>>>>> http://127.1 <http://127.0.0.1/>", it will then get mapped >>>>>>>>>>>>>>> to "http://127.0.0.1/";, per the URL spec, which is a >>>>>>>>>>>>>>> different host, which is not safe. * "http://127.0.0.0.1"; >>>>>>>>>>>>>>> and "http://1.2.3.09";, both of which are looked up via DNS >>>>>>>>>>>>>>> rather than failing or being treated as IPv4 hostnames, also >>>>>>>>>>>>>>> seem >>>>>>>>>>>>>>> potentially confusing. While no exploit is currently known >>>>>>>>>>>>>>> here, we want to >>>>>>>>>>>>>>> remove support for these as a preventative security measure. >>>>>>>>>>>>>>> The URL spec >>>>>>>>>>>>>>> has been updated so that any URL with a hostname ending in a >>>>>>>>>>>>>>> number that's >>>>>>>>>>>>>>> not an IPv4 address (including, e.g., http://foo.1./, but >>>>>>>>>>>>>>> not http://foo.1../) is considered invalid. Since this is >>>>>>>>>>>>>>> part of the URL spec, not the DNS spec, we want to reject these >>>>>>>>>>>>>>> URLs are >>>>>>>>>>>>>>> the GURL layer, for URLs with appropriate protocols (http, >>>>>>>>>>>>>>> https, ws, wss, >>>>>>>>>>>>>>> file). For consistency, we should also fail DNS lookup attempts >>>>>>>>>>>>>>> of these >>>>>>>>>>>>>>> sorts of hostnames. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Initial public proposal >>>>>>>>>>>>>>> https://github.com/whatwg/url/pull/619 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> TAG reviewNot required for an Intent to Deprecate, I >>>>>>>>>>>>>>> believe. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> TAG review statusNot applicable >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Risks >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Interoperability and Compatibility >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Any URL with an affected hostname will fail to load, and >>>>>>>>>>>>>>> will need to be migrated to another hostname. URLs of this form >>>>>>>>>>>>>>> do appear >>>>>>>>>>>>>>> to be in use, though it's not clear under what circumstances. >>>>>>>>>>>>>>> No entry in >>>>>>>>>>>>>>> the public suffix list is affected. Affected URLs make up no >>>>>>>>>>>>>>> more than >>>>>>>>>>>>>>> 0.0003% of hostnames looked up via the host resolver on any >>>>>>>>>>>>>>> platform, and >>>>>>>>>>>>>>> are basically not used in any file URLs, according to our >>>>>>>>>>>>>>> metrics. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Do we have reason to believe these hostnames are not >>>>>>>>>>>>>> legitimate ones? >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Unfortunately, we have no insight into them - they could be >>>>>>>>>>>>> mistyped URLs sent to typo squatting ISPs that OSX lets through >>>>>>>>>>>>> but the >>>>>>>>>>>>> Windows host resolver blocks, and various flavors of Linux treat >>>>>>>>>>>>> differently. Or they could be mapped via a hosts file, or they >>>>>>>>>>>>> could be >>>>>>>>>>>>> hostnames that only resolve on public networks. Could be some >>>>>>>>>>>>> network tool >>>>>>>>>>>>> that uses them when installed locally, but is only available on >>>>>>>>>>>>> certain >>>>>>>>>>>>> platforms. No reason to think one possibility is more likely >>>>>>>>>>>>> than the >>>>>>>>>>>>> others. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Do we have UKM for them that would enable us to test a random >>>>>>>>>>>> sample? >>>>>>>>>>>> I'm concerned about blocking those hostnames if they are >>>>>>>>>>>> legitimate, as that's something that a web developer can't do >>>>>>>>>>>> anything >>>>>>>>>>>> about. >>>>>>>>>>>> So even if the number of hosts is small, I'd like to get more >>>>>>>>>>>> certainty that they are *not* legitimate hosts before blocking >>>>>>>>>>>> them. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> We have UKM on their number (0.0003% of DNS lookups on OSX, less >>>>>>>>>>> elsewhere - we can't meaningfully instrument percent of created >>>>>>>>>>> GURLs), but >>>>>>>>>>> we don't have their hostnames, what they resolve to, or know >>>>>>>>>>> anything else >>>>>>>>>>> about them, unfortunately. >>>>>>>>>>> >>>>>>>>>>> Navigation to a subset of these as frame URLs were broken at one >>>>>>>>>>> point - I'm pretty sure the breakage even made it to stable: >>>>>>>>>>> https://crbug.com/1173238. There were no reports of problems. >>>>>>>>>>> Only non-IPv4 URLs where the last two components were broken, >>>>>>>>>>> though, and >>>>>>>>>>> it didn't affect subresources. On OSX and Android, over 99% of >>>>>>>>>>> successfully resolved problematic hostnames fit into that bucket, >>>>>>>>>>> though on >>>>>>>>>>> Linux, only about 2% do. >>>>>>>>>>> >>>>>>>>>>> That doesn't give us any hard conclusions, except they're either >>>>>>>>>>> not deliberate navigations on OSX/Android, or they're not >>>>>>>>>>> navigations. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> :| >>>>>>>>>> >>>>>>>>>> One more question: Is this an intent to Prototype or an intent to >>>>>>>>>> deprecate? The title is a bit unclear.. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On OSX and Android, about 90% of host resolver lookups for >>>>>>>>>>>>>>> these hostnames succeed, 60% do on Linux, and 2% on Windows and >>>>>>>>>>>>>>> ChromeOS. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Do you know where those failures are coming from? >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Could be typos, could be the Windows and ChromeOS host >>>>>>>>>>>>> resolvers don't let them through. Since we've had no filed bugs >>>>>>>>>>>>> about >>>>>>>>>>>>> them, I suspect the failures are not deliberate navigations or >>>>>>>>>>>>> intended >>>>>>>>>>>>> network requests. I'm much more interested in where the >>>>>>>>>>>>> successes are >>>>>>>>>>>>> coming from, myself. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> To allow for emergency disabling in case of wider than >>>>>>>>>>>>>>> expected breakage, I intend to add a feature for it, and do a >>>>>>>>>>>>>>> 50% field >>>>>>>>>>>>>>> trial on pre-release channels, though plan to just enable the >>>>>>>>>>>>>>> feature, >>>>>>>>>>>>>>> rather than do a gradual rollout to stable, given the low usage. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Gecko: Positive ( >>>>>>>>>>>>>>> https://github.com/whatwg/url/pull/619#issuecomment-890826499 >>>>>>>>>>>>>>> <https://www.chromestatus.com/admin/features/launch/5679790780579840/1?intent=1> >>>>>>>>>>>>>>> ) >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you file an official position request? >>>>>>>>>>>>>> https://bit.ly/blink-signals >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Done for Mozilla: >>>>>>>>>>>>> https://github.com/mozilla/standards-positions/issues/568 >>>>>>>>>>>>> >>>>>>>>>>>>> Should I also do this for WebKit as well? They have in >>>>>>>>>>>>> process CLs, so not sure if it's still needed. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Agree that in-flight patches for WebKit are a sufficient >>>>>>>>>>>> positive signal. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> WebKit: In development ( >>>>>>>>>>>>>>> https://bugs.webkit.org/show_bug.cgi?id=228826) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Web developers: No signals >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Activation >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This breaks anything using one of these domains, and >>>>>>>>>>>>>>> requires migrating to other hostnames. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Security >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> None >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Debuggability >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> These will act like any other invalid URL. Behavior is >>>>>>>>>>>>>>> context dependent. Since this is logic deep within GURL, and >>>>>>>>>>>>>>> GURLs are >>>>>>>>>>>>>>> created in a great many places, console warnings specifically >>>>>>>>>>>>>>> for this seem >>>>>>>>>>>>>>> not practical. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is this feature fully tested by web-platform-tests >>>>>>>>>>>>>>> <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md> >>>>>>>>>>>>>>> ?No. Javascript URL construction is tested, but URLs are >>>>>>>>>>>>>>> used in a great many other places, which don't have test >>>>>>>>>>>>>>> coverage, since >>>>>>>>>>>>>>> DNS lookups for these domains must succeed in the first place >>>>>>>>>>>>>>> for the tests >>>>>>>>>>>>>>> to be meaningful. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Flag name >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Requires code in //chrome?False >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Tracking bughttps://crbug.com/1237032 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Estimated milestones >>>>>>>>>>>>>>> DevTrial on desktop 95 >>>>>>>>>>>>>>> DevTrial on Webview 95 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Link to entry on the Chrome Platform Status >>>>>>>>>>>>>>> https://www.chromestatus.com/feature/5679790780579840 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This intent message was generated by Chrome Platform Status >>>>>>>>>>>>>>> <https://www.chromestatus.com/>. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>>>> Google Groups "blink-dev" group. >>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails >>>>>>>>>>>>>>> from it, send an email to blink-dev+...@chromium.org. >>>>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEK7mvq%2Bfnau%3DE%2BONhe0kr9HOpN84eCpoub84%3DswKzPkrGzi5A%40mail.gmail.com >>>>>>>>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEK7mvq%2Bfnau%3DE%2BONhe0kr9HOpN84eCpoub84%3DswKzPkrGzi5A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>>>> . >>>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "blink-dev" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to blink-dev+...@chromium.org. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfWB4wVuGshgPaLVXp%3DYsWUiXgJhUABD3ZFJ9xbhg1J3ww%40mail.gmail.com >>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfWB4wVuGshgPaLVXp%3DYsWUiXgJhUABD3ZFJ9xbhg1J3ww%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEK7mvoZMrGiAo_TjaDSdjPt51084ZRbEA1Pn3NOKJ66d%2BcRqQ%40mail.gmail.com.
