Excellent, we have a plan! On Wednesday, December 20, 2023 at 7:13:36 PM UTC+1 Mike Taylor wrote:
> LGTM3, with same condition. > On 12/20/23 11:58 AM, Chris Harrelson wrote: > > LGTM2 to ship, with a commitment to move this part of the spec to WICG if > it gets removed from the mediacapture-extensions spec. > > On Thu, Dec 14, 2023 at 11:06 AM Alex Russell <slightly...@chromium.org> > wrote: > >> LGTM1 >> >> On Thursday, December 14, 2023 at 8:00:41 AM UTC-8 Rick Byers wrote: >> >>> Sounds good Henrik! Yes, from our brief discussion in the API owners >>> meeting I believe you have support from at least 3 API owners to proceed in >>> this direction. It's important to us that we engage constructively in the >>> standards process even in areas of differing opinion, but it's also crucial >>> to our process that we not let such differences in opinion and priority be >>> an effective veto power over what we choose to ship in Chromium. I'd >>> encourage you and the WebRTC group to formalize processes for amicably >>> agreeing to disagree on the importance of different use cases, while still >>> being open to technical feedback and doing what we reasonably can to >>> maximize the chance that the APIs we ship can eventually be interoperable >>> if priorities of the other engines someday change [*]. API owners would >>> also be interested to hear any other arguments for why Chromium shipping >>> these APIs would be bad for the web (on this list, or anywhere else). I >>> know there's a messy history with WebRTC in particular and services coming >>> to depend on Chromium-only APIs when suitable standards-track alternatives >>> are available in other engines. That's IMHO definitely not a pattern we >>> want to risk repeating. >>> >>> Of course you also need Chrome privacy and security reviews, since it's >>> important that features like this don't create a hole in our careful >>> balance of side channel attack mitigations. But I see you already have >>> privacy approval so hopefully security isn't too far off. You might want to >>> wait for a signal there before starting implementation. >>> >>> Personally I'm also less concerned about interoperability risks when it >>> comes to metrics API. It's already the case that our top >>> performance metrics (Core Web Vitals) have APIs exposed only in Chromium. >>> There's certainly some interop risk there, eg. of sites optimizing in >>> engine-specific ways. But in practice we've seen developers use these APIs >>> mostly to make their sites faster in ways that generally apply to all >>> engines. So in that case, Safari and Firefox are getting most of the >>> benefit of these APIs existing without having to incur most of the cost, >>> which seems like a fine outcome to me. Also, I'm confident that if we >>> eventually agree with other engines on some better way to expose the same >>> information, then we can deprecate and remove any API shape we ship today >>> and customers can migrate over without causing user-visible breakage >>> (worst-case we just return dummy values on the deprecated APIs). >>> >>> Rick >>> >>> [*] My favorite example of this is Pointer Events where Apple was >>> opposed to the use case, but also had good technical critiques of the API. >>> We eventually (after a lot of research, open debate, and some flip-flopping >>> by me) shipped a version of the API that addressed the legitimate technical >>> concerns without addressing Apple's objections around the use case. Years >>> later when a use case materialized for Apple (the Apple Pen), they just >>> shipped the API in a fully interoperable fashion. That (as well as cases of >>> the inverse where we realize we were wrong and unship) is what we mean by >>> the blink process being designed for "eventual interoperability". >>> >>> >>> On Thu, Dec 14, 2023 at 4:34 AM Henrik Boström <h...@chromium.org> >>> wrote: >>> >>>> Thanks, Rick. Responses inline. >>>> >>>> On Wednesday, December 13, 2023 at 6:39:48 PM UTC+1 Rick Byers wrote: >>>> >>>> I looked into the details of the standards debate on this issue. It >>>> sounds like it's still unclear whether the spec for this has WG support or >>>> not, right? I certainly wouldn't want to mislead anyone as to API maturity >>>> / likely interoperability by shipping based on a WebRTC WG specification >>>> if >>>> there is an unresolved process concern. >>>> >>>> >>>> I think it's safe to say we don't have consensus on the frame counters >>>> (exposing dropped/glitches), while capture latency hopefully be less >>>> contentious. >>>> >>>> >>>> That said, I think Chromium's position on the technical debate here is >>>> pretty clear - we do believe there's value in such stats APIs, even IF >>>> they >>>> can only represent browser bugs (it's why we ship a crash reporting API >>>> <https://wicg.github.io/crash-reporting/>, which has been similarly >>>> controversial with Mozilla and Apple). We disagree that "there's nothing >>>> web developers can do about it". Maybe that's true for Apple and Mozilla, >>>> but for Google we rely critically on developer feedback through both open ( >>>> crbug.com) and private (Google partnerships) channels and we want to >>>> make it as easy as possible for developers to report an issue they're >>>> seeing to us in a way that's actionable. Our privacy policy limits >>>> Chrome's >>>> visibility into what's going on in the wild. So usually we find this >>>> requires a mix of both site-acquired telemetry and browser-required >>>> telemetry, and find the two can often complement each other nicely. >>>> >>>> Henrik, my advice if the WG doesn't have consensus for this API is to >>>> move it to some incubation venue (like a WICG group). You clearly have a >>>> community of web developers who want it, so it's probably more productive >>>> to focus standards energy among allies who share an interest for the use >>>> case, right? If you're willing to promptly move this to a WICG spec in the >>>> event the WG asks to remove it from their spec, then I don't think this >>>> debate changes anything from a blink API owner's perspective so I'm OK >>>> treating it as non-blocking. A subset of API owners met today (Daniel, >>>> Mike >>>> Taylor, Philip and I), and agreed with this stance. WDYT? >>>> >>>> >>>> Me moving this to a WICG spec in the event that the WG asks to remove >>>> them from mediacapture-extensions sounds good to me. >>>> If me and/or the WebRTC Audio Team has access to a WICG spec for these >>>> things, that may also give us a venue for, in the future, exploring >>>> *playout* glitch metrics in a more enthusiastic setting, which is in >>>> the early stages of discussions internally. >>>> >>>> >>>> In terms of testing, normally we ask to see the tests land before >>>> approving an I2S. Any reason we shouldn't wait for that here? >>>> >>>> >>>> Given the "controversy" around the glitch metrics, the WebRTC Audio >>>> Team wanted to get some Blink owners signals before they spend the >>>> engineering efforts to implement this (including WPTs). >>>> But if Blink owners also see the value in these metrics and we have a >>>> plan (= move to WICG in the event that they are removed) I see no reason >>>> not to ask them to start implementation today. >>>> >>>> For reference, see the WPTs we added for the video track stats here >>>> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/mediacapture-extensions/MediaStreamTrack-video-stats.https.html>. >>>> >>>> The audio track stats WPTs should be similar and also complemented by C++ >>>> unit tests in lower layers. >>>> >>>> >>>> Thanks, >>>> Rick >>>> >>>> >>>> >>>> On Fri, Dec 8, 2023 at 11:53 AM Henrik Boström <h...@chromium.org> >>>> wrote: >>>> >>>> Contact emails >>>> h...@chromium.org, o...@chromium.org, h...@chromium.org >>>> >>>> Specification >>>> https://w3c.github.io/mediacapture-extensions/#the- >>>> mediastreamtrackaudiostats-interface >>>> >>>> Summary >>>> >>>> The MediaStreamTrack Statistics API, or `track.stats`, has already >>>> shipped for video tracks: see previous I2S here >>>> <https://groups.google.com/a/chromium.org/g/blink-dev/c/ttzYv-30gY4/m/2FvJpxqMGQAJ> >>>> . >>>> >>>> >>>> This is the same API but for audio tracks, also motivated by the app's >>>> desire to measure capture quality. This is very important for conferencing >>>> websites such as Google Meet, Microsoft Teams, Zoom, Goto Meetings, etc. >>>> All of which has expressed an interest in the audio portion of this API. >>>> >>>> >>>> The API is only available for getUserMedia() sourced audio tracks, i.e. >>>> microphone, so the API is behind a user prompt and only available during >>>> capture. >>>> >>>> >>>> The new interface we want to ship is MediaStreamTrackAudioStats >>>> <https://w3c.github.io/mediacapture-extensions/#the-mediastreamtrackaudiostats-interface> >>>> >>>> which allow measuring two things from the audio capture pipeline: >>>> >>>> 1. The number of audio frames, including if any audio frames are >>>> dropped by the device, OS or User Agent. This allows measuring glitches in >>>> captured audio. >>>> >>>> 2. The input latency, such as due to buffering or other delays in >>>> delivering the audio frames to the track's sinks. >>>> >>>> Blink component >>>> Blink>GetUserMedia >>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EGetUserMedia> >>>> >>>> Risks >>>> Interoperability and Compatibility >>>> >>>> Because the API provides *statistics* about capture quality, rather >>>> than providing capture *functionality, *the interop/compatibility risk >>>> is small. >>>> >>>> *Gecko*: Standards position issue >>>> <https://github.com/mozilla/standards-positions/issues/935> >>>> *WebKit*: Standards position issue >>>> <https://github.com/WebKit/standards-positions/issues/260> >>>> >>>> Standardization >>>> While the audio stats API is written by the W3C WebRTC Working Group >>>> and track statistics overall is not controversial, there is an ongoing >>>> disagreement with Mozilla about whether or not dropped frames (= >>>> totalFrames - deliveredFrames) should be exposed to the web in the audio >>>> case. The disagreement is tracked by this issue >>>> <https://github.com/w3c/mediacapture-extensions/issues/129>. Our need >>>> for this metric has been discussed at Virtual Interims, see recording >>>> with 10:39 timestamp >>>> <https://www.youtube.com/watch?v=xJMXnf3Qwh8&t=639s> but no consensus >>>> could be reached (rough consensus was reached between everyone except >>>> Mozilla). Youenn (Apple) has shown that frames can be dropped due to >>>> Bluetooth connection >>>> <https://github.com/w3c/mediacapture-extensions/issues/129#issuecomment-1822624904> >>>> and >>>> is not just "measuring browser bugs". >>>> >>>> *Web developers*: Positive >>>> - The I2P >>>> <https://groups.google.com/a/chromium.org/g/blink-dev/c/vUbD_psbPL8/m/wqq3kmZFBwAJ> >>>> shows >>>> support from Teams, Zoom and GoTo meetings. >>>> - In the spec issue >>>> <https://github.com/w3c/mediacapture-extensions/issues/129> regarding >>>> the disagreement, more developer support is expressed (e.g. alfredh from >>>> Nvidia and steely-glint). >>>> >>>> WebView application risks >>>> >>>> None >>>> >>>> Will this feature be supported on all six Blink platforms (Windows, >>>> Mac, Linux, ChromeOS, Android, and Android WebView)? >>>> Yes >>>> >>>> Is this feature fully tested by web-platform-tests >>>> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> >>>> ? >>>> Yes and WPTs will be written as part of implementing this, however unit >>>> tests will also be needed to verify accuracy of metrics on a lower level. >>>> WPTs will be more general like "frame counters increase over time during >>>> capture" and that run-to-completion semantics are preserved. >>>> >>>> Link to entry on the Chrome Platform Status >>>> https://chromestatus.com/feature/5141112910249984 >>>> >>>> Links to previous Intent discussions >>>> Intent to prototype: >>>> https://groups.google.com/a/chromium.org/d/msgid/blink- >>>> dev/bb6c1af3-9eb3-4c6f-a136-dee709b7f906n%40chromium.org >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "blink-dev" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to blink-dev+unsubscr...@chromium.org. >>>> To view this discussion on the web visit https://groups.google.com/a/ >>>> chromium.org/d/msgid/blink-dev/a594fc55-8030-4848-9fe6- >>>> 549eccfdd8a8n%40chromium.org >>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/a594fc55-8030-4848-9fe6-549eccfdd8a8n%40chromium.org?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> -- >> You received this message because you are subscribed to the Google Groups >> "blink-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to blink-dev+unsubscr...@chromium.org. >> To view this discussion on the web visit >> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/2e451af1-5e0a-4dd9-a12c-fc30c7dff7dbn%40chromium.org >> >> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/2e451af1-5e0a-4dd9-a12c-fc30c7dff7dbn%40chromium.org?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "blink-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to blink-dev+unsubscr...@chromium.org. > > To view this discussion on the web visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-HbWWNeZ4jyRr38WR75FtxOA2DUhkwrHH0-OZnjkq9cg%40mail.gmail.com > > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-HbWWNeZ4jyRr38WR75FtxOA2DUhkwrHH0-OZnjkq9cg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/4fb6c5be-4576-4798-91fd-d3fd81d50ea9n%40chromium.org.
