Re: [whatwg] Fetch, MSE, and MIX
I removed some people from the cc. The WHATWG list seems to bite. On Thu, Apr 16, 2015 at 6:57 PM, Ryan Sleevi wrote: > I think as we look to > provide a compelling story for EME over wholly-proprietary (... rather than > partially-proprietary) solutions, or look to improve the user experience in > streaming video with MSE vs the tag, this is still very much needed. Okay, so accepting that for the moment, that leaves us with deciding on the appropriate API on the MSE side. I don't have a strong preference, but at this point MSE is the only case I can think of that warrants an explicit opaque readable stream class. All the other use cases I have (such as feeding or
Re: [whatwg] Fetch, MSE, and MIX
On Wed, Apr 15, 2015 at 6:45 PM, Martin Thomson wrote: > I believe that the easiest way to avoid this is to make an attempt to > read Response.body raise a SecurityError if the origin is different > (in Firefox terms, we would say "if the response principal is not > subsumed by the script principal"). The proposal is that .body returns an opaque stream object that you cannot read from, but privileged code can. But yes, same general idea as the SOP dances elsewhere. Having said all this, it has come to my attention that Netflix had a change of heart so maybe we do not want to put effort into this new Mixed Content API? It could still be useful for same-scheme-cross-origin-"no-cors" of course, but nobody has asked for that. -- https://annevankesteren.nl/
Re: [whatwg] Fetch, MSE, and MIX
On Tue, Apr 14, 2015 at 9:38 PM, Domenic Denicola wrote:
> I also imagine it won't be too hard to spec, as the point of an opaque stream
> type is that most of its methods consist of "magic happens here" (roughly
> speaking).
Well, we also need to ensure that nothing that takes a stream and then
exposes it either client or server-side (e.g. Request, Response,
fetch()) can handle them. (Which is a different primitive from what
Credential Management wants by the way, they just want to avoid
exposing the request body stream client-side, probably with the
exception of service workers.)
That is, the following should reject:
fetch(url, {mode:"no-cors"}).then(res => fetch(url2, {body:res.body}))
None of that should be particularly hard, though I do worry that the
further we get away from Response, the more we might lose sight of
what we are trying to protect and make mistakes.
(Perhaps the Credential Management case does not need to use streams
at all. We could extend FormData to have an opaque mode or some such.)
--
https://annevankesteren.nl/
Re: [whatwg] Fetch, MSE, and MIX
From: whatwg [mailto:whatwg-boun...@lists.whatwg.org] On Behalf Of Anne van Kesteren > Well, except now you make yourself depend on some definition of an > opaque stream object which nobody has defined yet. Perhaps we should, > but that will take longer and won't be less work (though maybe less work > long term). I think opaque stream is a good primitive to have. Given that we'd like the Streams API to be a general mechanism for composition of data streams, and given that our platform necessarily makes some data streams opaque, it'd be good to introduce a type for this. It would act like a readable stream without any ability to get a reader (so, you can pipe it to trusted places, and cancel it, but not read individual chunks). This would allow some subset of the general stream-consuming code that developers write to "just work" with cross-origin/mixed resources and any other opaque cases. I also imagine it won't be too hard to spec, as the point of an opaque stream type is that most of its methods consist of "magic happens here" (roughly speaking). If there is consensus on this direction I am happy to get working on a PR for the Streams Standard that people can review. Given that they only have a few methods, nailing down the exact semantics should be pretty quick.
Re: [whatwg] Fetch, MSE, and MIX
On Mon, Apr 13, 2015 at 10:34 PM, Matthew Wolenetz wrote: > Certainly. As I understand it, the reasons for reusing appendStream() rather > than adding appendResponse() to MSE are generally two-fold: > a) MSE already has appendStream(). In combination with the other changes to > Streams API, Fetch, and Mixed Content specs, as well as the known work to > update MSE spec to use ReadableByteStream > (https://www.w3.org/Bugs/Public/show_bug.cgi?id=27239), the scope of further > changes to existing MSE appendStream() spec are expected to be significantly > less than adding a distinct appendResponse() method to MSE. > b) We think these other specs (Streams API, Fetch, Mixed Content) are likely > to incorporate these changes anyway. Well, except now you make yourself depend on some definition of an opaque stream object which nobody has defined yet. Perhaps we should, but that will take longer and won't be less work (though maybe less work long term). -- https://annevankesteren.nl/
Re: [whatwg] Fetch, MSE, and MIX
On Fri, Apr 10, 2015 at 11:11 PM, Anne van Kesteren wrote: > On Sat, Apr 11, 2015 at 12:24 AM, Matthew Wolenetz > wrote: > > After further internal discussion, we believe we can and should reuse > > appendStream() rather than adding appendResponse(). > > For those not privy, could you share the reasoning? > Certainly. As I understand it, the reasons for reusing appendStream() rather than adding appendResponse() to MSE are generally two-fold: a) MSE already has appendStream(). In combination with the other changes to Streams API, Fetch, and Mixed Content specs, as well as the known work to update MSE spec to use ReadableByteStream ( https://www.w3.org/Bugs/Public/show_bug.cgi?id=27239), the scope of further changes to existing MSE appendStream() spec are expected to be significantly less than adding a distinct appendResponse() method to MSE. b) We think these other specs (Streams API, Fetch, Mixed Content) are likely to incorporate these changes anyway. Matt
Re: [whatwg] Fetch, MSE, and MIX
On Sat, Apr 11, 2015 at 12:24 AM, Matthew Wolenetz wrote: > After further internal discussion, we believe we can and should reuse > appendStream() rather than adding appendResponse(). For those not privy, could you share the reasoning? -- https://annevankesteren.nl/
Re: [whatwg] Fetch, MSE, and MIX
On Fri, Mar 6, 2015 at 3:00 PM, Ryan Sleevi wrote: > > > On Fri, Mar 6, 2015 at 2:43 PM, Mark Watson wrote: > >> Ryan, >> >> Proposals like this might allow video-intensive sites to migrate to HTTPS >> sooner than otherwise and are thus very welcome. This one was originally >> suggested by Anne Van Kesteren, I believe. Or at least something very >> similar. However, this particular proposal suffers (IIUC) from the >> disadvantage that users are likely to be presented with mixed content >> warnings. That's not an acceptable user experience for a professional >> commercial service. >> > > Well, it's an accurate presentation of the security state of said > commercial service. > > That is, it is actively disclosing your activities to anyone on the > network, for any number of purposes - profiling, tracking, analysis, etc. I > wish we could say these were purely the result of paranoia or theoretical > concerns, but we have ample evidence that they are real, practical, and > concerning. Given how many professional commercial services' offerings (at > least in the context of MSE) are often in business competition with the > transit providers, I do find it somewhat surprising that this is seen as a > desirable state - that is, where the commercial services disclose their > users' viewing habits, preferences, and profiles to those who would most > benefit from the competitive insight. But I digress... > I didn't say the mixed content warnings were not accurate and I didn't say that disclosing viewing habits was desirable. > > >> I understand the reasons that mixed content warnings are presented: the >> properties of the HTTP media requests do not align with the user >> expectation of privacy and security which is set by the presence of the >> "green padlock" or other UI indications of secure transport. A viable >> interim solution - without such warnings - either needs to avoid setting >> this expectation or to include additional measures such that the warnings >> were not necessary. If the latter, we'd need to evaluate whether such >> measures were worthwhile as an interim step or whether the investment would >> be better spent on the move to HTTPS proper. >> > > Well, it's between unwise to impossible to meaningfully address the > privacy concerns, even if you were to attempt to address the security > concerns. That is, you're still disclosing activity to the network > observer, and that fundamentally undermines the confidentiality goal of > HTTPS. > > As it stands, the current design of MSE (and, therefore, EME, as some EME > implementations require MSE) incentivizes media sites towards HTTP as the > path of least resistance / least UI. However, in doing so, it also leaves > users open to serious privacy and security risks, both in the local level > (EME) but also at the overall platform level (cookies, local storage, etc). > > I would dispute that EME per se necessarily opens users to serious privacy and security risks. If done wrong, sure, but this is true of anything. I'm hopeful that UA vendors will provide EME solutions that are relatively benign in this respect and I know a number who are trying very hard to do so. > > It's also worth considering how proposals like [1][2] will affect the > overall UI state, such that HTTP would appear, to the user, worse than > mixed content (as the active injection of arbitrary code is far worse than > the tracking disclosure). Or to consider how deprecation plans for powerful > features, such as those in [3][4], might see the usage of features (such as > EME) over HTTP be presented with access permissions or negative UI. > > In this world in which the platform reasonably and rightfully is moving > away from HTTP - both for powerful features (as per [3]/[4]), but also as > the "Everything is fine, nothing to worry about" state (as per [1]/[2]) - > whether this provides a meaningful interim step between the "Actively > insecure" and "Meaningfully secured from network attacks" status. > All very worthy of consideration, as you say. Again, if browsers do provide a viable interim step, that could get sites improved security sooner than they otherwise would. …Mark > > [1] > http://www.chromium.org/Home/chromium-security/marking-http-as-non-secure > [2] > https://lists.w3.org/Archives/Public/public-webappsec/2014Dec/0062.html > [3] > https://lists.w3.org/Archives/Public/public-webappsec/2015Feb/0431.html > [4] > https://groups.google.com/a/chromium.org/d/topic/blink-dev/2LXKVWYkOus/discussion > >
Re: [whatwg] Fetch, MSE, and MIX
Ryan, Proposals like this might allow video-intensive sites to migrate to HTTPS sooner than otherwise and are thus very welcome. This one was originally suggested by Anne Van Kesteren, I believe. Or at least something very similar. However, this particular proposal suffers (IIUC) from the disadvantage that users are likely to be presented with mixed content warnings. That's not an acceptable user experience for a professional commercial service. I understand the reasons that mixed content warnings are presented: the properties of the HTTP media requests do not align with the user expectation of privacy and security which is set by the presence of the "green padlock" or other UI indications of secure transport. A viable interim solution - without such warnings - either needs to avoid setting this expectation or to include additional measures such that the warnings were not necessary. If the latter, we'd need to evaluate whether such measures were worthwhile as an interim step or whether the investment would be better spent on the move to HTTPS proper. …Mark On Thu, Feb 19, 2015 at 9:06 PM, Ryan Sleevi wrote: > Cross-posting, as this touches on the Fetch [1] spec, Media Source > Extensions [2], and Mixed Content [3]. This does cross-post WHATWG and > W3C, apologies if this is a mortal sin. > > TL;DR Proposal first: > - Amend MIX in [4] to add "fetch" as an > optionally-blockable-request-context > * This means that fetch() can now return HTTP content from HTTPS > pages. The implications of this, however, are described below, if you > can handle reading it all. > - Amend MSE in [5] to introduce a new method, appendResponse(Response > response), which accepts a Response [6] class > - In MSE, define a Response Append Loop similar to the Stream Append > Loop [7], that calls the consume body algorithm [8] on the internal > response [9] of Response to yield an ArrayBuffer, then executes the > buffer append [10] algorithm on the SourceBuffer > > > MUCH longer justification why: > As it stands, // tags today are optionally > blockable content, as noted in [4]. Thus, an HTTPS page may set the > source to HTTP content and load the content (although typically with > user-agent indication). MSE poses itself as a spec to offer much > greater control to site authors than /, as noted in its > use cases, and as a result, has seen a rapid adoption among a number > of popular video streaming sites. Most notably, the ability to do > adaptive streaming with MSE helps provide a better quality, better > performing experience for users. Finally, in some user agents, MSE is > a pre-requisite for the use of Encrypted Media Extensions [11]. > > However, there are limitations to using MSE that don't exist with > /. The most notable of these is that in order to > implement the adaptive streaming capabilities, most sites make use of > XMLHttpRequest to request portions of media content, which can then be > supplied to the SourceBuffer. Based on the feedback that MSE provides > the script author, it can then adjust the XHRs they make to use a > lower bitrate media source, to drop segments, etc. When using XHR, the > site author loses the ability to mix HTTPS pages with HTTP media, as > XHR is (rightfully so) treated as blocked content. > > The justification for why XHR does this is that it returns the full > buffer to the page author. In practice, we saw many sites then taking > that buffer and making security decisions on it - whether it be > "clearly" bad things such as eval()ing the content to more subtle > things like adjusting UI or links. All of these undermine all of the > security guarantees that HTTPS tries to provide, and thus XHR is > blocked. > > The result is that if an HTTPS site wants to use MSE with XHR, all of > the content needs to be served via HTTPS. We've already seen some > providers complain that this is prohibitively expensive in their > current networks [12], although it may be solvable in time, as > demonstrated by other video sharing sites [13]. > > In a choice between using MSE - which offers a better user experience > over / by reducing bandwidth and improving quality - and > using HTTPS - which offers better privacy and security controls - > sites are likely to choose solutions that reduce their costs rather > than protect their users, a reasonable but unfortunate business > reality. > > I'm hoping to find a way to close that gap - to allow sites to use MSE > (and potentially EME) via HTTPS documents, while still sourcing their > media content via HTTP. This may seem counter-intuitive, and a step > back from the efforts of the Chrome security team, but I think it is > actually consistent with our goals and our past comments. In > particular, this solution tries to provide a means and incentive for > sites to adopt MSE (improving user experience) AND to begin migrating > to HTTPS; first with their main document, and then, in time, all of > their media content. > > This won't protect adversaries from
Re: [whatwg] Fetch, MSE, and MIX
Hi Ryan, Thanks for writing this up. I know you already know this, but I wanted to publically declare my support as one of the MSE editors. While I wish we didn't need this, I can understand the concerns of content providers and I think this is a reasonable compromise. Aaron On Thu Feb 19 2015 at 9:06:17 PM Ryan Sleevi wrote: > Cross-posting, as this touches on the Fetch [1] spec, Media Source > Extensions [2], and Mixed Content [3]. This does cross-post WHATWG and > W3C, apologies if this is a mortal sin. > > TL;DR Proposal first: > - Amend MIX in [4] to add "fetch" as an optionally-blockable-request- > context > * This means that fetch() can now return HTTP content from HTTPS > pages. The implications of this, however, are described below, if you > can handle reading it all. > - Amend MSE in [5] to introduce a new method, appendResponse(Response > response), which accepts a Response [6] class > - In MSE, define a Response Append Loop similar to the Stream Append > Loop [7], that calls the consume body algorithm [8] on the internal > response [9] of Response to yield an ArrayBuffer, then executes the > buffer append [10] algorithm on the SourceBuffer > > > MUCH longer justification why: > As it stands, // tags today are optionally > blockable content, as noted in [4]. Thus, an HTTPS page may set the > source to HTTP content and load the content (although typically with > user-agent indication). MSE poses itself as a spec to offer much > greater control to site authors than /, as noted in its > use cases, and as a result, has seen a rapid adoption among a number > of popular video streaming sites. Most notably, the ability to do > adaptive streaming with MSE helps provide a better quality, better > performing experience for users. Finally, in some user agents, MSE is > a pre-requisite for the use of Encrypted Media Extensions [11]. > > However, there are limitations to using MSE that don't exist with > /. The most notable of these is that in order to > implement the adaptive streaming capabilities, most sites make use of > XMLHttpRequest to request portions of media content, which can then be > supplied to the SourceBuffer. Based on the feedback that MSE provides > the script author, it can then adjust the XHRs they make to use a > lower bitrate media source, to drop segments, etc. When using XHR, the > site author loses the ability to mix HTTPS pages with HTTP media, as > XHR is (rightfully so) treated as blocked content. > > The justification for why XHR does this is that it returns the full > buffer to the page author. In practice, we saw many sites then taking > that buffer and making security decisions on it - whether it be > "clearly" bad things such as eval()ing the content to more subtle > things like adjusting UI or links. All of these undermine all of the > security guarantees that HTTPS tries to provide, and thus XHR is > blocked. > > The result is that if an HTTPS site wants to use MSE with XHR, all of > the content needs to be served via HTTPS. We've already seen some > providers complain that this is prohibitively expensive in their > current networks [12], although it may be solvable in time, as > demonstrated by other video sharing sites [13]. > > In a choice between using MSE - which offers a better user experience > over / by reducing bandwidth and improving quality - and > using HTTPS - which offers better privacy and security controls - > sites are likely to choose solutions that reduce their costs rather > than protect their users, a reasonable but unfortunate business > reality. > > I'm hoping to find a way to close that gap - to allow sites to use MSE > (and potentially EME) via HTTPS documents, while still sourcing their > media content via HTTP. This may seem counter-intuitive, and a step > back from the efforts of the Chrome security team, but I think it is > actually consistent with our goals and our past comments. In > particular, this solution tries to provide a means and incentive for > sites to adopt MSE (improving user experience) AND to begin migrating > to HTTPS; first with their main document, and then, in time, all of > their media content. > > This won't protect adversaries from knowing what content the user is > actively watching, for example, but will help protect other vital > assets - such as their cookies, session identifiers, user information, > friends list, past viewing history, etc. > > Allowing fetch() to return HTTP content sourced from HTTPS pages seems > like it would re-open the XHR hole, but this isn't the case. As > described in [14], all requests whose mode is CORS or > CORS-with-forced-preflight are force-failed. This only leaves the > request modes of "no-cors", "same-origin", "about"and "data". Because > the origins are different between the document (https) and the request > URL (http), the request mode will be "no-cors", and thus the returned > Response object will be set to "opaque". > > The "opaque" response prevents direct access to the R
