Today, I've requested extension of this experiment. Please see https://groups.google.com/u/1/a/chromium.org/g/blink-dev/c/WETrrUYLrTM.
Thanks! Matt On Thu, Sep 2, 2021 at 12:35 PM Matthew Wolenetz <wolen...@chromium.org> wrote: > Thank you for the response - I fully intend to much more strictly restrict > the specification on what kinds of allowances are there, probably just say > that it must be like postMessage(), with a P&S section reference > explicitly describing why (to mitigate potential for enabling Spectre-like > attacks). Cross-origin isolation may be too restrictive for interop unless > implementations use different mechanisms (optimized if isolated, > unoptimized if not, for example). > > On Thu, Sep 2, 2021 at 11:15 AM Mike West <mk...@chromium.org> wrote: > >> LGTM to experiment (assuming a normal length of ~M95 through ~M99 (and my >> sincere apologies that it took me a while to get back to this). I think the >> mitigation y'all are putting into place is reasonable, and I'm happy with >> the CL's direction. >> >> I would like to see this mitigation explained in the specification; the >> current PR suggests that "implementations MAY choose to communicate in >> potentially faster ways", and I'd like to see that tempered a bit with >> considerations that implementers should run through before implementing a >> faster-than-postMessage mechanism (up to and including cross-origin >> isolation). >> >> Thanks! >> >> -mike >> >> >> On Wed, Aug 25, 2021 at 11:32 PM Matthew Wolenetz <wolen...@chromium.org> >> wrote: >> >>> The mitigation code (using kPostedMessage task runner type for >>> cross-thread MSE API state that an app might otherwise use for a high-res >>> timer if optimized too much) is currently in code review ( >>> https://chromium-review.googlesource.com/c/chromium/src/+/3095089). >>> >>> On Thu, Aug 12, 2021 at 12:57 PM Matthew Wolenetz <wolen...@chromium.org> >>> wrote: >>> >>>> I'm lining up the code to do this currently, since the various hooks >>>> into the shared state are complex. Hope to have something for review by >>>> EOW, but then I'm out next week on holiday. >>>> >>>> On Thu, Aug 12, 2021 at 12:19 PM Alex Russell <slightly...@chromium.org> >>>> wrote: >>>> >>>>> I know lots of folks are out on holiday, but wanted to check to see if >>>>> the postMessage timing experiment worked out. >>>>> >>>>> On Wednesday, July 28, 2021 at 12:24:23 PM UTC-7 Matthew Wolenetz >>>>> wrote: >>>>> >>>>>> [CrossOriginIsolated]'s isolation seems to be too strict of a >>>>>> requirement, especially given we can instead require (variable) delays in >>>>>> communication to be induced by queuing tasks to transfer the >>>>>> information. I >>>>>> will investigate adding task queueing for the worker-to-window >>>>>> communication of live seekable range and duration, as used by the window >>>>>> thread for seekable, buffered and duration attribute values. Similarly, >>>>>> "recent error" status of the media element may need task queueing at >>>>>> least >>>>>> in the specification, though I think Chromium's implementation may >>>>>> already >>>>>> be somewhat protected from using it to create a high-res timer. >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Jul 22, 2021 at 11:05 AM Matthew Wolenetz < >>>>>> wolen...@chromium.org> wrote: >>>>>> >>>>>>> Thank you for the background information. I have not discussed >>>>>>> CrossOriginIsolated], though will look and see if it might be a viable >>>>>>> option, as well as discuss with the WG. >>>>>>> >>>>>>> Matt >>>>>>> >>>>>>> On Thu, Jul 22, 2021 at 10:02 AM Mike West <mk...@chromium.org> >>>>>>> wrote: >>>>>>> >>>>>>>> On Thu, Jul 15, 2021 at 8:54 PM Matthew Wolenetz < >>>>>>>> wolen...@chromium.org> wrote: >>>>>>>> >>>>>>>>> Thank you for your question. I'm uncertain what is meant by a >>>>>>>>> "cross-thread stream" in the context of MSE-in-Workers, though I >>>>>>>>> believe I >>>>>>>>> understand your overall concern. >>>>>>>>> >>>>>>>>> The specification makes most of the MSE/HTMLMediaEelement >>>>>>>>> Worker/Window state communication operations rely upon a >>>>>>>>> "cross-context >>>>>>>>> communication mechanism" which allows for implementations to be as >>>>>>>>> slow as >>>>>>>>> an app doing postMessages to communicate state changes across >>>>>>>>> contexts. >>>>>>>>> However, the spec also allows for implementations to perform such >>>>>>>>> communication in more optimized fashion. The current experimental >>>>>>>>> Chromium >>>>>>>>> implementation does some, though not all, operations in this more >>>>>>>>> performant fashion. I'll attempt to list the ops which are/are not >>>>>>>>> optimized here: >>>>>>>>> >>>>>>>>> - Most operations in the MSE impl that touch the media >>>>>>>>> pipeline/demuxer are asynchronous: >>>>>>>>> - SourceBuffer >>>>>>>>> appendBuffer(..)/appendEncodedChunk(..)/remove(start,end): >>>>>>>>> apart from >>>>>>>>> initial sanity checks internal to the MSE API, these queue >>>>>>>>> tasks to parse >>>>>>>>> and buffer input (or remove a range of buffered media) with the >>>>>>>>> demuxer >>>>>>>>> asynchronously, with completion or error causing a change to >>>>>>>>> the 'updating' >>>>>>>>> attribute in Worker, newly queued events in Worker, and if the >>>>>>>>> media >>>>>>>>> pipeline deems the newly buffered information merits media >>>>>>>>> element >>>>>>>>> notification (readyState transition, playback state >>>>>>>>> progress/stall/etc), >>>>>>>>> that is done too (though concurrently and not synchronous with >>>>>>>>> the dispatch >>>>>>>>> or 'updating' transition in Worker). Also, at most one of these >>>>>>>>> operations >>>>>>>>> on a specific SourceBuffer can be in flight at a time. I don't >>>>>>>>> think it's >>>>>>>>> possible to construct a high-res timer from these operations. >>>>>>>>> - Duration attribute updates in Worker are trampolined via >>>>>>>>> media thread asynchronously to Window. >>>>>>>>> - HTMLMediaElement.buffered and seekable queries in Window >>>>>>>>> are dependent upon the result of asynchronous operations in >>>>>>>>> Worker >>>>>>>>> (appends/removes, described above), with the exception of: >>>>>>>>> - The buffered and seekable synchronous calculations are >>>>>>>>> done under a micro-lock on the shared Worker state, and >>>>>>>>> though the buffered >>>>>>>>> ranges are updated asynchronously (per above) in a >>>>>>>>> SourceBuffer, there are >>>>>>>>> other operations on MSE API that are synchronous and could >>>>>>>>> modify the >>>>>>>>> result of the seekable query: >>>>>>>>> - [*] set/clearLiveSeekableRange is used by .seekable >>>>>>>>> if duration is +Infinity. If duration is not used and >>>>>>>>> nothing is buffered, >>>>>>>>> then seekable contains the duration value as the single >>>>>>>>> range's end time. >>>>>>>>> This is the only information of which I am aware that the >>>>>>>>> implementation >>>>>>>>> might allow a Worker to set timing information using the >>>>>>>>> seekable range and >>>>>>>>> duration. >>>>>>>>> >>>>>>>>> Considering a successful MSE-in-Worker attachment/connection is >>>>>>>>> allowed only for same-origin workers, if I understand correctly, then >>>>>>>>> would >>>>>>>>> [*] be a cause for concern? >>>>>>>>> >>>>>>>>> Possible mitigations for [*]: >>>>>>>>> >>>>>>>>> - Make the seekable and buffered queries done by the media >>>>>>>>> element depend on asynchronously updated live-seekable-range and >>>>>>>>> duration >>>>>>>>> information. >>>>>>>>> >>>>>>>>> If one side of the optimized communication channel can set state >>>>>>>> that's visible to the other side, it seems like it might be a cause for >>>>>>>> concern, as it sounds like it might allow developers to create a very >>>>>>>> high-resolution timer in the same way that SharedArrayBuffer does >>>>>>>> (e.g. by >>>>>>>> incrementing a value in a tight loop on one side, and reading the value >>>>>>>> from the other, see >>>>>>>> https://xsleaks.dev/docs/attacks/timing-attacks/clocks/#implicit-clocks). >>>>>>>> We've addressed that concern for SharedArrayBuffer by gating it on >>>>>>>> cross-origin isolation to ensure that attackers using timers for >>>>>>>> side-channel attacks have a harder time gaining access to sensitive >>>>>>>> data. >>>>>>>> Could [CrossOriginIsolated >>>>>>>> <https://heycam.github.io/webidl/#CrossOriginIsolated>] be a >>>>>>>> reasonable restriction for this mechanism as well? Is this something >>>>>>>> you've >>>>>>>> discussed in the working group? Or am I misunderstanding the >>>>>>>> capability you >>>>>>>> refer to above? >>>>>>>> >>>>>>>> >>>>>>>>> Regarding P&S in the MSE spec, there's an MSE spec bug previously >>>>>>>>> already, separate from MSE-in-Workers, to add the P&S sections per >>>>>>>>> Media >>>>>>>>> WG: https://github.com/w3c/media-source/issues/261. I could >>>>>>>>> update it and the draft MSE-in-Worker spec PR to ensure that the >>>>>>>>> seekable >>>>>>>>> and buffered information is not over-optimized, if either >>>>>>>>> cross-origin MSE >>>>>>>>> attachments (worker scripts) are somehow allowed, or if there is >>>>>>>>> still the >>>>>>>>> high-res timer concern even if same-origin. >>>>>>>>> >>>>>>>>> Thanks again for your question - please help me further understand >>>>>>>>> the concern. >>>>>>>>> >>>>>>>>> -Matt >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Jul 15, 2021 at 12:45 AM Mike West <mk...@chromium.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Friendly ping on the question below. >>>>>>>>>> >>>>>>>>>> -mike >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Jul 1, 2021 at 8:45 PM Mike West <mk...@chromium.org> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> It sounds like there may be some potential for creating a >>>>>>>>>>> high-resolution timer using a cross-thread stream. Is this >>>>>>>>>>> something y'all >>>>>>>>>>> have looked into? Is memory shared between the Dedicated Worker and >>>>>>>>>>> its >>>>>>>>>>> Window? https://w3c.github.io/media-source/ doesn't have a >>>>>>>>>>> security considerations section, so it's not clear if this has been >>>>>>>>>>> considered. >>>>>>>>>>> >>>>>>>>>>> -mike >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Jun 30, 2021 at 11:55 PM Matthew Wolenetz < >>>>>>>>>>> wolen...@chromium.org> wrote: >>>>>>>>>>> >>>>>>>>>>>> Changed title (fixed typo) in case filters missed the original >>>>>>>>>>>> transmission. >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Jun 30, 2021 at 2:53 PM Matthew Wolenetz < >>>>>>>>>>>> wolen...@chromium.org> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Contact emailswolen...@chromium.org >>>>>>>>>>>>> >>>>>>>>>>>>> Explainer >>>>>>>>>>>>> https://github.com/wicg/media-source/blob/mse-in-workers-using-handle/mse-in-workers-using-handle-explainer.md >>>>>>>>>>>>> >>>>>>>>>>>>> Specificationhttps://github.com/w3c/media-source/pull/282 >>>>>>>>>>>>> >>>>>>>>>>>>> Summary >>>>>>>>>>>>> >>>>>>>>>>>>> Enable Media Source Extensions (MSE) API usage from >>>>>>>>>>>>> DedicatedWorker contexts to enable improved performance of >>>>>>>>>>>>> buffering media >>>>>>>>>>>>> for playback by an HTMLMediaElement on the main Window context. >>>>>>>>>>>>> By creating >>>>>>>>>>>>> a MediaSource object on a DedicatedWorker context, an application >>>>>>>>>>>>> may then >>>>>>>>>>>>> create an ObjectURL for it and postMessage that URL to the main >>>>>>>>>>>>> thread for >>>>>>>>>>>>> use in attaching to an HTMLMediaElement. The context that created >>>>>>>>>>>>> the >>>>>>>>>>>>> MediaSource object may then use it to buffer media. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Blink componentBlink>Media >>>>>>>>>>>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EMedia> >>>>>>>>>>>>> >>>>>>>>>>>>> Search tagsMSE <https://chromestatus.com/features#tags:MSE>, >>>>>>>>>>>>> MediaSource >>>>>>>>>>>>> <https://chromestatus.com/features#tags:MediaSource>, >>>>>>>>>>>>> MediaSourceExtensions >>>>>>>>>>>>> <https://chromestatus.com/features#tags:MediaSourceExtensions> >>>>>>>>>>>>> >>>>>>>>>>>>> TAG reviewhttps://github.com/w3ctag/design-reviews/issues/656 >>>>>>>>>>>>> >>>>>>>>>>>>> TAG review statusPending >>>>>>>>>>>>> >>>>>>>>>>>>> Risks >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Interoperability and Compatibility >>>>>>>>>>>>> >>>>>>>>>>>>> Main interoperability risk is that other browsers may not >>>>>>>>>>>>> implement it. Compatibility risk is mitigated by unchanged >>>>>>>>>>>>> same-thread MSE >>>>>>>>>>>>> API support and proactive feature-detectability of MSE-in-Workers >>>>>>>>>>>>> with a >>>>>>>>>>>>> new canConstructInDedicatedWorker attribute on the MediaSource >>>>>>>>>>>>> interface. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Gecko: No signal ( >>>>>>>>>>>>> https://github.com/mozilla/standards-positions/issues/547) >>>>>>>>>>>>> >>>>>>>>>>>>> WebKit: No signal ( >>>>>>>>>>>>> https://lists.webkit.org/pipermail/webkit-dev/2021-June/031916.html >>>>>>>>>>>>> ) >>>>>>>>>>>>> >>>>>>>>>>>>> Web developers: No signals >>>>>>>>>>>>> >>>>>>>>>>>>> Ergonomics >>>>>>>>>>>>> >>>>>>>>>>>>> DedicatedWorker, WorkerGlobalScope and postMessage/onmessage >>>>>>>>>>>>> availability is integral to using this feature. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Activation >>>>>>>>>>>>> >>>>>>>>>>>>> The primary risk is the potential difficulty in refactoring >>>>>>>>>>>>> existing MSE web applications to (conditionally, depending on >>>>>>>>>>>>> browser >>>>>>>>>>>>> support) perform buffering in a different execution context from >>>>>>>>>>>>> the Window >>>>>>>>>>>>> context that hosts the DOM and the media element. The benefit of >>>>>>>>>>>>> potentially improved buffering performance by offloading the MSE >>>>>>>>>>>>> API usage >>>>>>>>>>>>> to a worker context provides motivation to overcome this >>>>>>>>>>>>> challenge. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Security >>>>>>>>>>>>> >>>>>>>>>>>>> Unpredictability of racing context destruction of either the >>>>>>>>>>>>> main thread hosting the media element (and owning the underlying >>>>>>>>>>>>> MSE media >>>>>>>>>>>>> demuxer and player) or the dedicated worker thread hosting the >>>>>>>>>>>>> interface to >>>>>>>>>>>>> the MSE demuxer when using MSE from a worker context motivated >>>>>>>>>>>>> careful >>>>>>>>>>>>> design of a refcounted, outside-of-Oilpan, attachment abstraction >>>>>>>>>>>>> to ensure >>>>>>>>>>>>> safety of operations using locks and having a lifetime that >>>>>>>>>>>>> exceeds those >>>>>>>>>>>>> execution contexts. Preventing use-after-free and similar was a >>>>>>>>>>>>> primary >>>>>>>>>>>>> complexity of implementation and a focus area of added tests. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Goals for experimentation >>>>>>>>>>>>> >>>>>>>>>>>>> * Obtain developer feedback on API ergonomics and >>>>>>>>>>>>> implementation stability. * Increase confidence in benefits of >>>>>>>>>>>>> feature >>>>>>>>>>>>> (exposing and enabling MSE usage from DedicatedWorker contexts) >>>>>>>>>>>>> to support >>>>>>>>>>>>> its standardization. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Reason this experiment is being extended >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Ongoing technical constraints >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Will this feature be supported on all six Blink platforms >>>>>>>>>>>>> (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)? >>>>>>>>>>>>> Yes >>>>>>>>>>>>> >>>>>>>>>>>>> MSE and DedicatedWorker already exist on all six Blink >>>>>>>>>>>>> platforms. This feature lets MSE work from DedicatedWorker >>>>>>>>>>>>> contexts. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Is this feature fully tested by web-platform-tests >>>>>>>>>>>>> <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md> >>>>>>>>>>>>> ?Yes >>>>>>>>>>>>> >>>>>>>>>>>>> DevTrial instructions >>>>>>>>>>>>> https://github.com/w3c/media-source/issues/175#issuecomment-721395481 >>>>>>>>>>>>> >>>>>>>>>>>>> Flag nameMediaSourceInWorkers >>>>>>>>>>>>> >>>>>>>>>>>>> Tracking bughttps://crbug.com/878133 >>>>>>>>>>>>> >>>>>>>>>>>>> Link to entry on the Chrome Platform Status >>>>>>>>>>>>> https://chromestatus.com/feature/5177263249162240 >>>>>>>>>>>>> >>>>>>>>>>>>> Links to previous Intent discussionsIntent to prototype: >>>>>>>>>>>>> https://groups.google.com/u/1/a/chromium.org/g/blink-dev/c/CNRywDqgKjY/m/F0nnA4tTAwAJ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> This intent message was generated by Chrome Platform Status >>>>>>>>>>>>> <https://www.chromestatus.com/>. >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>> Google Groups "blink-dev" group. >>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>> it, send an email to blink-dev+unsubscr...@chromium.org. >>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAADho6OYdKY_VBN94cDZGKwDrR_D7JSG4EZq%3DvMas%2B5Q%2BXR-xA%40mail.gmail.com >>>>>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAADho6OYdKY_VBN94cDZGKwDrR_D7JSG4EZq%3DvMas%2B5Q%2BXR-xA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAADho6NTFE1rGL3tiRJ%2BBJn50xnEEZ-ziMPstJ-u1T_b%3D3syRQ%40mail.gmail.com.
