LGTM to experiment (assuming a normal length of ~M95 through ~M99 (and my sincere apologies that it took me a while to get back to this). I think the mitigation y'all are putting into place is reasonable, and I'm happy with the CL's direction.
I would like to see this mitigation explained in the specification; the current PR suggests that "implementations MAY choose to communicate in potentially faster ways", and I'd like to see that tempered a bit with considerations that implementers should run through before implementing a faster-than-postMessage mechanism (up to and including cross-origin isolation). Thanks! -mike On Wed, Aug 25, 2021 at 11:32 PM Matthew Wolenetz <wolen...@chromium.org> wrote: > The mitigation code (using kPostedMessage task runner type for > cross-thread MSE API state that an app might otherwise use for a high-res > timer if optimized too much) is currently in code review ( > https://chromium-review.googlesource.com/c/chromium/src/+/3095089). > > On Thu, Aug 12, 2021 at 12:57 PM Matthew Wolenetz <wolen...@chromium.org> > wrote: > >> I'm lining up the code to do this currently, since the various hooks into >> the shared state are complex. Hope to have something for review by EOW, but >> then I'm out next week on holiday. >> >> On Thu, Aug 12, 2021 at 12:19 PM Alex Russell <slightly...@chromium.org> >> wrote: >> >>> I know lots of folks are out on holiday, but wanted to check to see if >>> the postMessage timing experiment worked out. >>> >>> On Wednesday, July 28, 2021 at 12:24:23 PM UTC-7 Matthew Wolenetz wrote: >>> >>>> [CrossOriginIsolated]'s isolation seems to be too strict of a >>>> requirement, especially given we can instead require (variable) delays in >>>> communication to be induced by queuing tasks to transfer the information. I >>>> will investigate adding task queueing for the worker-to-window >>>> communication of live seekable range and duration, as used by the window >>>> thread for seekable, buffered and duration attribute values. Similarly, >>>> "recent error" status of the media element may need task queueing at least >>>> in the specification, though I think Chromium's implementation may already >>>> be somewhat protected from using it to create a high-res timer. >>>> >>>> >>>> >>>> On Thu, Jul 22, 2021 at 11:05 AM Matthew Wolenetz < >>>> wolen...@chromium.org> wrote: >>>> >>>>> Thank you for the background information. I have not discussed >>>>> CrossOriginIsolated], though will look and see if it might be a viable >>>>> option, as well as discuss with the WG. >>>>> >>>>> Matt >>>>> >>>>> On Thu, Jul 22, 2021 at 10:02 AM Mike West <mk...@chromium.org> wrote: >>>>> >>>>>> On Thu, Jul 15, 2021 at 8:54 PM Matthew Wolenetz < >>>>>> wolen...@chromium.org> wrote: >>>>>> >>>>>>> Thank you for your question. I'm uncertain what is meant by a >>>>>>> "cross-thread stream" in the context of MSE-in-Workers, though I >>>>>>> believe I >>>>>>> understand your overall concern. >>>>>>> >>>>>>> The specification makes most of the MSE/HTMLMediaEelement >>>>>>> Worker/Window state communication operations rely upon a "cross-context >>>>>>> communication mechanism" which allows for implementations to be as slow >>>>>>> as >>>>>>> an app doing postMessages to communicate state changes across contexts. >>>>>>> However, the spec also allows for implementations to perform such >>>>>>> communication in more optimized fashion. The current experimental >>>>>>> Chromium >>>>>>> implementation does some, though not all, operations in this more >>>>>>> performant fashion. I'll attempt to list the ops which are/are not >>>>>>> optimized here: >>>>>>> >>>>>>> - Most operations in the MSE impl that touch the media >>>>>>> pipeline/demuxer are asynchronous: >>>>>>> - SourceBuffer >>>>>>> appendBuffer(..)/appendEncodedChunk(..)/remove(start,end): apart >>>>>>> from >>>>>>> initial sanity checks internal to the MSE API, these queue tasks >>>>>>> to parse >>>>>>> and buffer input (or remove a range of buffered media) with the >>>>>>> demuxer >>>>>>> asynchronously, with completion or error causing a change to the >>>>>>> 'updating' >>>>>>> attribute in Worker, newly queued events in Worker, and if the >>>>>>> media >>>>>>> pipeline deems the newly buffered information merits media element >>>>>>> notification (readyState transition, playback state >>>>>>> progress/stall/etc), >>>>>>> that is done too (though concurrently and not synchronous with >>>>>>> the dispatch >>>>>>> or 'updating' transition in Worker). Also, at most one of these >>>>>>> operations >>>>>>> on a specific SourceBuffer can be in flight at a time. I don't >>>>>>> think it's >>>>>>> possible to construct a high-res timer from these operations. >>>>>>> - Duration attribute updates in Worker are trampolined via >>>>>>> media thread asynchronously to Window. >>>>>>> - HTMLMediaElement.buffered and seekable queries in Window >>>>>>> are dependent upon the result of asynchronous operations in Worker >>>>>>> (appends/removes, described above), with the exception of: >>>>>>> - The buffered and seekable synchronous calculations are >>>>>>> done under a micro-lock on the shared Worker state, and though >>>>>>> the buffered >>>>>>> ranges are updated asynchronously (per above) in a >>>>>>> SourceBuffer, there are >>>>>>> other operations on MSE API that are synchronous and could >>>>>>> modify the >>>>>>> result of the seekable query: >>>>>>> - [*] set/clearLiveSeekableRange is used by .seekable >>>>>>> if duration is +Infinity. If duration is not used and >>>>>>> nothing is buffered, >>>>>>> then seekable contains the duration value as the single >>>>>>> range's end time. >>>>>>> This is the only information of which I am aware that the >>>>>>> implementation >>>>>>> might allow a Worker to set timing information using the >>>>>>> seekable range and >>>>>>> duration. >>>>>>> >>>>>>> Considering a successful MSE-in-Worker attachment/connection is >>>>>>> allowed only for same-origin workers, if I understand correctly, then >>>>>>> would >>>>>>> [*] be a cause for concern? >>>>>>> >>>>>>> Possible mitigations for [*]: >>>>>>> >>>>>>> - Make the seekable and buffered queries done by the media >>>>>>> element depend on asynchronously updated live-seekable-range and >>>>>>> duration >>>>>>> information. >>>>>>> >>>>>>> If one side of the optimized communication channel can set state >>>>>> that's visible to the other side, it seems like it might be a cause for >>>>>> concern, as it sounds like it might allow developers to create a very >>>>>> high-resolution timer in the same way that SharedArrayBuffer does (e.g. >>>>>> by >>>>>> incrementing a value in a tight loop on one side, and reading the value >>>>>> from the other, see >>>>>> https://xsleaks.dev/docs/attacks/timing-attacks/clocks/#implicit-clocks). >>>>>> We've addressed that concern for SharedArrayBuffer by gating it on >>>>>> cross-origin isolation to ensure that attackers using timers for >>>>>> side-channel attacks have a harder time gaining access to sensitive data. >>>>>> Could [CrossOriginIsolated >>>>>> <https://heycam.github.io/webidl/#CrossOriginIsolated>] be a >>>>>> reasonable restriction for this mechanism as well? Is this something >>>>>> you've >>>>>> discussed in the working group? Or am I misunderstanding the capability >>>>>> you >>>>>> refer to above? >>>>>> >>>>>> >>>>>>> Regarding P&S in the MSE spec, there's an MSE spec bug previously >>>>>>> already, separate from MSE-in-Workers, to add the P&S sections per Media >>>>>>> WG: https://github.com/w3c/media-source/issues/261. I could update >>>>>>> it and the draft MSE-in-Worker spec PR to ensure that the seekable and >>>>>>> buffered information is not over-optimized, if either cross-origin MSE >>>>>>> attachments (worker scripts) are somehow allowed, or if there is still >>>>>>> the >>>>>>> high-res timer concern even if same-origin. >>>>>>> >>>>>>> Thanks again for your question - please help me further understand >>>>>>> the concern. >>>>>>> >>>>>>> -Matt >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thu, Jul 15, 2021 at 12:45 AM Mike West <mk...@chromium.org> >>>>>>> wrote: >>>>>>> >>>>>>>> Friendly ping on the question below. >>>>>>>> >>>>>>>> -mike >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Jul 1, 2021 at 8:45 PM Mike West <mk...@chromium.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> It sounds like there may be some potential for creating a >>>>>>>>> high-resolution timer using a cross-thread stream. Is this something >>>>>>>>> y'all >>>>>>>>> have looked into? Is memory shared between the Dedicated Worker and >>>>>>>>> its >>>>>>>>> Window? https://w3c.github.io/media-source/ doesn't have a >>>>>>>>> security considerations section, so it's not clear if this has been >>>>>>>>> considered. >>>>>>>>> >>>>>>>>> -mike >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jun 30, 2021 at 11:55 PM Matthew Wolenetz < >>>>>>>>> wolen...@chromium.org> wrote: >>>>>>>>> >>>>>>>>>> Changed title (fixed typo) in case filters missed the original >>>>>>>>>> transmission. >>>>>>>>>> >>>>>>>>>> On Wed, Jun 30, 2021 at 2:53 PM Matthew Wolenetz < >>>>>>>>>> wolen...@chromium.org> wrote: >>>>>>>>>> >>>>>>>>>>> Contact emailswolen...@chromium.org >>>>>>>>>>> >>>>>>>>>>> Explainer >>>>>>>>>>> https://github.com/wicg/media-source/blob/mse-in-workers-using-handle/mse-in-workers-using-handle-explainer.md >>>>>>>>>>> >>>>>>>>>>> Specificationhttps://github.com/w3c/media-source/pull/282 >>>>>>>>>>> >>>>>>>>>>> Summary >>>>>>>>>>> >>>>>>>>>>> Enable Media Source Extensions (MSE) API usage from >>>>>>>>>>> DedicatedWorker contexts to enable improved performance of >>>>>>>>>>> buffering media >>>>>>>>>>> for playback by an HTMLMediaElement on the main Window context. By >>>>>>>>>>> creating >>>>>>>>>>> a MediaSource object on a DedicatedWorker context, an application >>>>>>>>>>> may then >>>>>>>>>>> create an ObjectURL for it and postMessage that URL to the main >>>>>>>>>>> thread for >>>>>>>>>>> use in attaching to an HTMLMediaElement. The context that created >>>>>>>>>>> the >>>>>>>>>>> MediaSource object may then use it to buffer media. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Blink componentBlink>Media >>>>>>>>>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EMedia> >>>>>>>>>>> >>>>>>>>>>> Search tagsMSE <https://chromestatus.com/features#tags:MSE>, >>>>>>>>>>> MediaSource <https://chromestatus.com/features#tags:MediaSource> >>>>>>>>>>> , MediaSourceExtensions >>>>>>>>>>> <https://chromestatus.com/features#tags:MediaSourceExtensions> >>>>>>>>>>> >>>>>>>>>>> TAG reviewhttps://github.com/w3ctag/design-reviews/issues/656 >>>>>>>>>>> >>>>>>>>>>> TAG review statusPending >>>>>>>>>>> >>>>>>>>>>> Risks >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Interoperability and Compatibility >>>>>>>>>>> >>>>>>>>>>> Main interoperability risk is that other browsers may not >>>>>>>>>>> implement it. Compatibility risk is mitigated by unchanged >>>>>>>>>>> same-thread MSE >>>>>>>>>>> API support and proactive feature-detectability of MSE-in-Workers >>>>>>>>>>> with a >>>>>>>>>>> new canConstructInDedicatedWorker attribute on the MediaSource >>>>>>>>>>> interface. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Gecko: No signal ( >>>>>>>>>>> https://github.com/mozilla/standards-positions/issues/547) >>>>>>>>>>> >>>>>>>>>>> WebKit: No signal ( >>>>>>>>>>> https://lists.webkit.org/pipermail/webkit-dev/2021-June/031916.html >>>>>>>>>>> ) >>>>>>>>>>> >>>>>>>>>>> Web developers: No signals >>>>>>>>>>> >>>>>>>>>>> Ergonomics >>>>>>>>>>> >>>>>>>>>>> DedicatedWorker, WorkerGlobalScope and postMessage/onmessage >>>>>>>>>>> availability is integral to using this feature. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Activation >>>>>>>>>>> >>>>>>>>>>> The primary risk is the potential difficulty in refactoring >>>>>>>>>>> existing MSE web applications to (conditionally, depending on >>>>>>>>>>> browser >>>>>>>>>>> support) perform buffering in a different execution context from >>>>>>>>>>> the Window >>>>>>>>>>> context that hosts the DOM and the media element. The benefit of >>>>>>>>>>> potentially improved buffering performance by offloading the MSE >>>>>>>>>>> API usage >>>>>>>>>>> to a worker context provides motivation to overcome this challenge. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Security >>>>>>>>>>> >>>>>>>>>>> Unpredictability of racing context destruction of either the >>>>>>>>>>> main thread hosting the media element (and owning the underlying >>>>>>>>>>> MSE media >>>>>>>>>>> demuxer and player) or the dedicated worker thread hosting the >>>>>>>>>>> interface to >>>>>>>>>>> the MSE demuxer when using MSE from a worker context motivated >>>>>>>>>>> careful >>>>>>>>>>> design of a refcounted, outside-of-Oilpan, attachment abstraction >>>>>>>>>>> to ensure >>>>>>>>>>> safety of operations using locks and having a lifetime that exceeds >>>>>>>>>>> those >>>>>>>>>>> execution contexts. Preventing use-after-free and similar was a >>>>>>>>>>> primary >>>>>>>>>>> complexity of implementation and a focus area of added tests. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Goals for experimentation >>>>>>>>>>> >>>>>>>>>>> * Obtain developer feedback on API ergonomics and implementation >>>>>>>>>>> stability. * Increase confidence in benefits of feature (exposing >>>>>>>>>>> and >>>>>>>>>>> enabling MSE usage from DedicatedWorker contexts) to support its >>>>>>>>>>> standardization. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Reason this experiment is being extended >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Ongoing technical constraints >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Will this feature be supported on all six Blink platforms >>>>>>>>>>> (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)? >>>>>>>>>>> Yes >>>>>>>>>>> >>>>>>>>>>> MSE and DedicatedWorker already exist on all six Blink >>>>>>>>>>> platforms. This feature lets MSE work from DedicatedWorker contexts. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Is this feature fully tested by web-platform-tests >>>>>>>>>>> <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md> >>>>>>>>>>> ?Yes >>>>>>>>>>> >>>>>>>>>>> DevTrial instructions >>>>>>>>>>> https://github.com/w3c/media-source/issues/175#issuecomment-721395481 >>>>>>>>>>> >>>>>>>>>>> Flag nameMediaSourceInWorkers >>>>>>>>>>> >>>>>>>>>>> Tracking bughttps://crbug.com/878133 >>>>>>>>>>> >>>>>>>>>>> Link to entry on the Chrome Platform Status >>>>>>>>>>> https://chromestatus.com/feature/5177263249162240 >>>>>>>>>>> >>>>>>>>>>> Links to previous Intent discussionsIntent to prototype: >>>>>>>>>>> https://groups.google.com/u/1/a/chromium.org/g/blink-dev/c/CNRywDqgKjY/m/F0nnA4tTAwAJ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This intent message was generated by Chrome Platform Status >>>>>>>>>>> <https://www.chromestatus.com/>. >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "blink-dev" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to blink-dev+unsubscr...@chromium.org. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAADho6OYdKY_VBN94cDZGKwDrR_D7JSG4EZq%3DvMas%2B5Q%2BXR-xA%40mail.gmail.com >>>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAADho6OYdKY_VBN94cDZGKwDrR_D7JSG4EZq%3DvMas%2B5Q%2BXR-xA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3DcN7qWik%3DWSWX6kKa-qD8F4Hga6jfFqO65Xd0dLEbndNA%40mail.gmail.com.
