Hello all. I wonder if I can use the audioContext.outputLatency to detect whether the user is using a Bluetooth headset as the audio output device. For instance, if the value of outputLatency exceeds 150ms, it may indicate that the user is using a Bluetooth headset 在2022年2月17日星期四 UTC+8 05:58:17<Hongchan Choi> 写道：
> Hello all, > > We landed the metric collection CL (https://crrev.com/c/3413794) and have > gathered some data: > > # Notable latency range > - Android Chrome: Mean 40.96, Peak 3~5ms > - Chrome OS: Mean 46.37, Peak at 15ms > - Mac OS X: Mean 935.36, Peak at 17ms > - Linux (dev): Mean 45.32, Peak at 33ms > - Windows: Mean 31.77, Peak at 31ms > > More details can be found > here: go/webaudio-outputlatency-metric-collection (sorry, googler-only) > > # Thoughts > - Notable peaks roughly match the platform. > - Mac OS X has a diverse distribution and there is a small peak at 200ms > on Mac OSX: it may indicate that the client is using the “AirPods”. (~1%) > > We believe the value exposed by this property does not pose a significant > issue on the user privacy, but any feedback would be appreciated! > > Best, > Hongchan > > On Tuesday, December 21, 2021 at 2:11:25 PM UTC-8 Ajay Rahatekar wrote: > >> Will do. Ty Joe. >> >> On Tue, Dec 21, 2021 at 1:32 PM Joe Medley <jme...@google.com> wrote: >> >>> Please update the Chrome Status entry as soon as you know. (That will >>> inform me as well.) >>> Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | >>> 816-678-7195 <(816)%20678-7195> >>> *If an API's not documented it doesn't exist.* >>> >>> >>> On Tue, Dec 21, 2021 at 12:32 PM Ajay Rahatekar <ajayra...@google.com> >>> wrote: >>> >>>> The metrics are landing in Jan. So likely shipping in 99. >>>> >>>> -Ajay >>>> >>>> On Tue, Dec 21, 2021 at 11:54 AM Joe Medley <jme...@google.com> wrote: >>>> >>>>> Is this shipping in 98? >>>>> >>>>> On Wednesday, December 15, 2021 at 9:02:09 AM UTC-8 hongchan wrote: >>>>> >>>>>> Thank you all! >>>>>> >>>>>> 1. I'll land the metrics CL and follow up here. >>>>>> 2. The discussion in the WebKit mailing list is ongoing, so I'll >>>>>> report back when it's done. >>>>>> >>>>>> Cheers, >>>>>> Hongchan >>>>>> >>>>>> >>>>>> On Wed, Dec 15, 2021 at 8:58 AM Chris Harrelson <chri...@chromium.org> >>>>>> wrote: >>>>>> >>>>>>> LGTM3. >>>>>>> >>>>>>> On Wed, Dec 15, 2021 at 8:47 AM Mike Taylor <mike...@chromium.org> >>>>>>> wrote: >>>>>>> >>>>>>>> LGTM2 w/ same condition. >>>>>>>> >>>>>>>> On 12/15/21 11:47 AM, Yoav Weiss wrote: >>>>>>>> >>>>>>>> LGTM1 conditional on those metrics landing (but no need to wait for >>>>>>>> data from the metrics to come in). >>>>>>>> >>>>>>>> On Monday, December 6, 2021 at 3:22:26 PM UTC+1 Mike Taylor wrote: >>>>>>>> >>>>>>>>> To circle back - last week we (myself + some other >>>>>>>>> privacy/anti-covert tracking reviewers) met with some folks working >>>>>>>>> on this >>>>>>>>> feature. We ended up with an AI for Hongchan to land metrics on the >>>>>>>>> values >>>>>>>>> that getOutputTimestamp would produce (if called, when a new >>>>>>>>> AudioContext >>>>>>>>> is created, IIRC) as a way to get some data on the utility for using >>>>>>>>> either >>>>>>>>> of these APIs for device fingerprinting. >>>>>>>>> >>>>>>>>> On 12/1/21 11:27 AM, Hongchan Choi wrote: >>>>>>>>> >>>>>>>>> So far we have not discussed the deprecation plan, but I believe >>>>>>>>> that's reasonable as well. >>>>>>>>> >>>>>>>>> I registered myself to the webkit-dev list to post a question, but >>>>>>>>> somehow I am getting 403 forbidden. The email is sent to the list, I >>>>>>>>> am not >>>>>>>>> seeing the message is getting posted on the list. Not sure what to do >>>>>>>>> there. >>>>>>>>> >>>>>>>>> On Wed, Dec 1, 2021 at 3:18 AM Yoav Weiss <yoav...@chromium.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> So are y'all planning to deprecate and remove >>>>>>>>>> `getOutputTimestamp` once this ships? If so, that sounds reasonable. >>>>>>>>>> >>>>>>>>>> I note Chris asked y'all to file for a webkit signal upthread. >>>>>>>>>> Did that happen? >>>>>>>>>> >>>>>>>>>> On Tuesday, November 30, 2021 at 12:54:17 AM UTC+1 hongchan wrote: >>>>>>>>>> >>>>>>>>>>> I agree that outputLatency is better in terms of both usability >>>>>>>>>>> and interoperability. >>>>>>>>>>> >>>>>>>>>>> FYI, the counter shows getOutputTimestamp is about 0.001% and >>>>>>>>>>> there are no partners who are using this API: >>>>>>>>>>> >>>>>>>>>>> https://chromestatus.com/metrics/feature/popularity#AudioContextGetOutputTimestamp >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Mon, Nov 22, 2021 at 3:00 PM Chris Cunningham < >>>>>>>>>>> chcunn...@google.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Re: privacy discussion, Hongchan and I scheduled to meet with >>>>>>>>>>>> Paul Jensen next week. >>>>>>>>>>>> >>>>>>>>>>>> Also, I recently determined that this new WebAudio API >>>>>>>>>>>> (outputLatency) is largely redundant with an existing API >>>>>>>>>>>> (getOutputTimestamp()). >>>>>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/2461 >>>>>>>>>>>> >>>>>>>>>>>> Chrome has already shipped the latter API, so I believe this >>>>>>>>>>>> new API doesn't add potential for additional fingerprinting. >>>>>>>>>>>> Shipping both >>>>>>>>>>>> APIs is still a good idea for the sake of inter-op. >>>>>>>>>>>> >>>>>>>>>>>> On Friday, November 19, 2021 at 8:45:28 AM UTC-8 hongchan wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Thanks for the clarification, Yoav! >>>>>>>>>>>>> >>>>>>>>>>>>> I hope the mitigation approach in the previous comment makes >>>>>>>>>>>>> sense from the privacy perspective, but it will hurt the >>>>>>>>>>>>> usability/value of >>>>>>>>>>>>> this API significantly. >>>>>>>>>>>>> For the privacy-focused discussion, should I use other venues >>>>>>>>>>>>> rather than this I2S thread? >>>>>>>>>>>>> >>>>>>>>>>>>> Cheers, >>>>>>>>>>>>> Hongchan >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, Nov 18, 2021 at 10:35 PM Yoav Weiss < >>>>>>>>>>>>> yoav...@chromium.org> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Apologies, I misread the discussion. A spec issue is indeed >>>>>>>>>>>>>> not needed, as this is already covered. >>>>>>>>>>>>>> >>>>>>>>>>>>>> From my perspective, since this exposes active entropy, the >>>>>>>>>>>>>> fact that it exposes more data than the (passively exposed) >>>>>>>>>>>>>> platform is not >>>>>>>>>>>>>> prohibitive on its own. >>>>>>>>>>>>>> At the same time, it'd be good to work with privacy-focused >>>>>>>>>>>>>> folks to make sure we expose as little information as needed to >>>>>>>>>>>>>> maintain >>>>>>>>>>>>>> the API's functionality, but not more. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Nov 19, 2021 at 4:57 AM Chris Cunningham < >>>>>>>>>>>>>> chcunn...@chromium.org> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> > If the reported value is incorrect, the A/V >>>>>>>>>>>>>>> synchronization won't be possible. (Perhaps chcunningham@ can >>>>>>>>>>>>>>> say more >>>>>>>>>>>>>>> about this use case.) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The typical video player design drives the clock with audio. >>>>>>>>>>>>>>> In short, you send buffers of audio to the OS, use the latency >>>>>>>>>>>>>>> value to >>>>>>>>>>>>>>> know when those buffers actually reach the user's ears, and >>>>>>>>>>>>>>> choose a video >>>>>>>>>>>>>>> frame accordingly. Apps using WebCodecs will design apps this >>>>>>>>>>>>>>> way (example >>>>>>>>>>>>>>> here >>>>>>>>>>>>>>> <https://github.com/chcunningham/wc-talk#simple_video_playerhtml> >>>>>>>>>>>>>>> ). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> As Mike notes, the latency value can vary by quite a lot >>>>>>>>>>>>>>> depending on the device (huge differences for bluetooth vs >>>>>>>>>>>>>>> wired), so it's >>>>>>>>>>>>>>> critical to have the platform tell you when this changes. In >>>>>>>>>>>>>>> terms of >>>>>>>>>>>>>>> precision, we can _probably_ do some rounding like to the >>>>>>>>>>>>>>> nearest >>>>>>>>>>>>>>> millisecond. Higher values may lead to issues; user studies >>>>>>>>>>>>>>> <https://en.wikipedia.org/wiki/Audio-to-video_synchronization#Recommendations> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> have shown sensitivity to fairly low values of drift. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Aside: we may eventually desire a display latency value for >>>>>>>>>>>>>>> similar reasons. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Thu, Nov 18, 2021 at 12:45 PM Hongchan Choi < >>>>>>>>>>>>>>> hong...@chromium.org> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Yoav, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Could you clarify what needs to be discussed from the spec >>>>>>>>>>>>>>>> side? The feature (and its behavior) is properly specified, >>>>>>>>>>>>>>>> and its privacy >>>>>>>>>>>>>>>> aspect is already reviewed by W3C: >>>>>>>>>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1498 >>>>>>>>>>>>>>>> (outputLatency) >>>>>>>>>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1495 >>>>>>>>>>>>>>>> (general security/privacy consideration) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I thought the request from this thread is to devise a >>>>>>>>>>>>>>>> Chrome-specific mitigation, not reopening the spec discussion. >>>>>>>>>>>>>>>> Please >>>>>>>>>>>>>>>> correct me if I'm mistaken. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>> Hongchan >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Thu, Nov 18, 2021 at 12:37 PM Yoav Weiss < >>>>>>>>>>>>>>>> yoav...@chromium.org> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hey folks! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> It seems like there are some things to figure out here >>>>>>>>>>>>>>>>> before shipping. Could y'all file a spec issue to discuss >>>>>>>>>>>>>>>>> this with the >>>>>>>>>>>>>>>>> broader community? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Tuesday, November 16, 2021 at 6:16:14 PM UTC+1 hongchan >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks for the feedback, Mike. This is really helpful! >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Mitigation strategies include adding jitter (dithering) >>>>>>>>>>>>>>>>>>> and quantization so that the exact skew is incorrectly >>>>>>>>>>>>>>>>>>> reported >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This bit refers to the randomization mechanism deployed >>>>>>>>>>>>>>>>>> by other browsers, but this approach defeats the purpose of >>>>>>>>>>>>>>>>>> this specific >>>>>>>>>>>>>>>>>> API and it is also argued in the spec: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Note however that most audio systems aim for low latency, >>>>>>>>>>>>>>>>>>> to synchronise the audio generated by WebAudio to other >>>>>>>>>>>>>>>>>>> audio or video >>>>>>>>>>>>>>>>>>> sources or to visual cues (for example in a game, or an >>>>>>>>>>>>>>>>>>> audio recording or >>>>>>>>>>>>>>>>>>> music making environment). Excessive latency decreases >>>>>>>>>>>>>>>>>>> usability and may be >>>>>>>>>>>>>>>>>>> an accessibility issue. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> If the reported value is incorrect, the A/V >>>>>>>>>>>>>>>>>> synchronization won't be possible. (Perhaps chcunningham@ >>>>>>>>>>>>>>>>>> can say more >>>>>>>>>>>>>>>>>> about this use case.) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> One mitigation idea: we can do some research to collect >>>>>>>>>>>>>>>>>> values reported by major platforms/devices and devise a >>>>>>>>>>>>>>>>>> reasonable "bucket >>>>>>>>>>>>>>>>>> system" for them. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Tue, Nov 16, 2021 at 8:14 AM Mike Taylor < >>>>>>>>>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I suppose it would be nice to do some testing to verify >>>>>>>>>>>>>>>>>>> the idea that there's a small number of platform + audio >>>>>>>>>>>>>>>>>>> device pairs. I >>>>>>>>>>>>>>>>>>> was hoping Mozilla's hardware survey >>>>>>>>>>>>>>>>>>> <https://data.firefox.com/dashboard/hardware> might >>>>>>>>>>>>>>>>>>> have something on that, but it doesn't track speakers. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Just from some simple testing with the following script >>>>>>>>>>>>>>>>>>> (thanks dmcardle@ for the snippet) in Firefox on macOS by 3 >>>>>>>>>>>>>>>>>>> people, we have >>>>>>>>>>>>>>>>>>> very different values that change dramatically when you >>>>>>>>>>>>>>>>>>> unplug a headset: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> (function getNextSample() { >>>>>>>>>>>>>>>>>>> console.log((new AudioContext).outputLatency); >>>>>>>>>>>>>>>>>>> setTimeout(getNextSample, 10); >>>>>>>>>>>>>>>>>>> })(); >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> Here's the values on my macbook, using Firefox Nightly: >>>>>>>>>>>>>>>>>>> bose headset plugged in: 0.005895833 >>>>>>>>>>>>>>>>>>> take headset out: 0.013083333 >>>>>>>>>>>>>>>>>>> plug bose headset back in: 0.005875 (different, but >>>>>>>>>>>>>>>>>>> stable) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> https://www.w3.org/TR/webaudio/#priv-sec mentions some >>>>>>>>>>>>>>>>>>> mitigation strategies - do we have a plan to apply any of >>>>>>>>>>>>>>>>>>> them? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 11/15/21 11:53 AM, Ajay Rahatekar wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The small number of platform + audio output device >>>>>>>>>>>>>>>>>>> combination i.e. a large number of users with the same >>>>>>>>>>>>>>>>>>> platform and output >>>>>>>>>>>>>>>>>>> device (users on macOS using airpods) would likely be a >>>>>>>>>>>>>>>>>>> natural >>>>>>>>>>>>>>>>>>> obfuscator. Not sure if we have data about the distribution. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Mon, Nov 15, 2021 at 8:05 AM Mike Taylor < >>>>>>>>>>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 11/15/21 10:56 AM, Hongchan Choi wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Mon, Nov 15, 2021 at 6:49 AM Mike Taylor < >>>>>>>>>>>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 11/15/21 1:47 AM, Yoav Weiss wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Fri, Nov 12, 2021 at 5:33 PM 'Ajay Rahatekar' via >>>>>>>>>>>>>>>>>>>>> blink-dev <blin...@chromium.org> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> TAG review status >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Completed: Web Audio API specification is W3C >>>>>>>>>>>>>>>>>>>>>> Recommendation. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Risks >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> There is a risk of the feature being used for >>>>>>>>>>>>>>>>>>>>>> fingerprinting. However outputLatency is the buffer >>>>>>>>>>>>>>>>>>>>>> size of the platform-provided audio callback, so the >>>>>>>>>>>>>>>>>>>>>> value is inherently >>>>>>>>>>>>>>>>>>>>>> platform-specific. That said, the majority of the >>>>>>>>>>>>>>>>>>>>>> platform audio buffer >>>>>>>>>>>>>>>>>>>>>> size is widely known. (MacOS = 128 frames, Windows = >>>>>>>>>>>>>>>>>>>>>> 10ms, Android = 96 >>>>>>>>>>>>>>>>>>>>>> frames, etc) >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> This feature does not expose more than what you can >>>>>>>>>>>>>>>>>>>>>> query/infer from the UA string. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Would that exposure map cleanly to UA-Platform >>>>>>>>>>>>>>>>>>>>> <https://wicg.github.io/ua-client-hints/#sec-ch-ua-platform>, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> which is considered low-entropy and exposed by default? >>>>>>>>>>>>>>>>>>>>> Or would it add >>>>>>>>>>>>>>>>>>>>> more than that? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> /cc +Mike Taylor >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Looking at >>>>>>>>>>>>>>>>>>>>> https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> it states that it depends on the platform _and_ the >>>>>>>>>>>>>>>>>>>>> hardware output device. >>>>>>>>>>>>>>>>>>>>> If I use an app using outputLatency with speaker A, then >>>>>>>>>>>>>>>>>>>>> switch to speaker >>>>>>>>>>>>>>>>>>>>> B, will the outputLatency remain the same? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The specification says: If the audio output device is >>>>>>>>>>>>>>>>>>>> changed the outputLatency attribute value will be updated >>>>>>>>>>>>>>>>>>>> accordingly. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> So the answer is no. The value will change accordingly. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> If that's the case, then outputLatency can reveal more >>>>>>>>>>>>>>>>>>>> entropy than just platform alone, right? It would be >>>>>>>>>>>>>>>>>>>> useful to know what >>>>>>>>>>>>>>>>>>>> this looks like in practice, or what mitigations we might >>>>>>>>>>>>>>>>>>>> be able to apply >>>>>>>>>>>>>>>>>>>> depending on the size of these latency differences. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "blink-dev" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to blink-dev+...@chromium.org. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/936f3e2f-2001-0449-13fd-d3a0548b673c%40chromium.org >>>>>>>> >>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/936f3e2f-2001-0449-13fd-d3a0548b673c%40chromium.org?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/0542f591-0988-4a20-80bd-70a273f1143bn%40chromium.org.
