Is this shipping in 98? On Wednesday, December 15, 2021 at 9:02:09 AM UTC-8 hongchan wrote:
> Thank you all! > > 1. I'll land the metrics CL and follow up here. > 2. The discussion in the WebKit mailing list is ongoing, so I'll report > back when it's done. > > Cheers, > Hongchan > > > On Wed, Dec 15, 2021 at 8:58 AM Chris Harrelson <chri...@chromium.org> > wrote: > >> LGTM3. >> >> On Wed, Dec 15, 2021 at 8:47 AM Mike Taylor <mike...@chromium.org> wrote: >> >>> LGTM2 w/ same condition. >>> >>> On 12/15/21 11:47 AM, Yoav Weiss wrote: >>> >>> LGTM1 conditional on those metrics landing (but no need to wait for data >>> from the metrics to come in). >>> >>> On Monday, December 6, 2021 at 3:22:26 PM UTC+1 Mike Taylor wrote: >>> >>>> To circle back - last week we (myself + some other privacy/anti-covert >>>> tracking reviewers) met with some folks working on this feature. We ended >>>> up with an AI for Hongchan to land metrics on the values that >>>> getOutputTimestamp would produce (if called, when a new AudioContext is >>>> created, IIRC) as a way to get some data on the utility for using either >>>> of >>>> these APIs for device fingerprinting. >>>> >>>> On 12/1/21 11:27 AM, Hongchan Choi wrote: >>>> >>>> So far we have not discussed the deprecation plan, but I believe that's >>>> reasonable as well. >>>> >>>> I registered myself to the webkit-dev list to post a question, but >>>> somehow I am getting 403 forbidden. The email is sent to the list, I am >>>> not >>>> seeing the message is getting posted on the list. Not sure what to do >>>> there. >>>> >>>> On Wed, Dec 1, 2021 at 3:18 AM Yoav Weiss <yoav...@chromium.org> wrote: >>>> >>>>> So are y'all planning to deprecate and remove `getOutputTimestamp` >>>>> once this ships? If so, that sounds reasonable. >>>>> >>>>> I note Chris asked y'all to file for a webkit signal upthread. Did >>>>> that happen? >>>>> >>>>> On Tuesday, November 30, 2021 at 12:54:17 AM UTC+1 hongchan wrote: >>>>> >>>>>> I agree that outputLatency is better in terms of both usability and >>>>>> interoperability. >>>>>> >>>>>> FYI, the counter shows getOutputTimestamp is about 0.001% and there >>>>>> are no partners who are using this API: >>>>>> >>>>>> https://chromestatus.com/metrics/feature/popularity#AudioContextGetOutputTimestamp >>>>>> >>>>>> >>>>>> On Mon, Nov 22, 2021 at 3:00 PM Chris Cunningham < >>>>>> chcunn...@google.com> wrote: >>>>>> >>>>>>> Re: privacy discussion, Hongchan and I scheduled to meet with Paul >>>>>>> Jensen next week. >>>>>>> >>>>>>> Also, I recently determined that this new WebAudio API >>>>>>> (outputLatency) is largely redundant with an existing API >>>>>>> (getOutputTimestamp()). >>>>>>> https://github.com/WebAudio/web-audio-api/issues/2461 >>>>>>> >>>>>>> Chrome has already shipped the latter API, so I believe this new API >>>>>>> doesn't add potential for additional fingerprinting. Shipping both APIs >>>>>>> is >>>>>>> still a good idea for the sake of inter-op. >>>>>>> >>>>>>> On Friday, November 19, 2021 at 8:45:28 AM UTC-8 hongchan wrote: >>>>>>> >>>>>>>> Thanks for the clarification, Yoav! >>>>>>>> >>>>>>>> I hope the mitigation approach in the previous comment makes sense >>>>>>>> from the privacy perspective, but it will hurt the usability/value of >>>>>>>> this >>>>>>>> API significantly. >>>>>>>> For the privacy-focused discussion, should I use other venues >>>>>>>> rather than this I2S thread? >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Hongchan >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Nov 18, 2021 at 10:35 PM Yoav Weiss <yoav...@chromium.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Apologies, I misread the discussion. A spec issue is indeed not >>>>>>>>> needed, as this is already covered. >>>>>>>>> >>>>>>>>> From my perspective, since this exposes active entropy, the fact >>>>>>>>> that it exposes more data than the (passively exposed) platform is >>>>>>>>> not >>>>>>>>> prohibitive on its own. >>>>>>>>> At the same time, it'd be good to work with privacy-focused folks >>>>>>>>> to make sure we expose as little information as needed to maintain >>>>>>>>> the >>>>>>>>> API's functionality, but not more. >>>>>>>>> >>>>>>>>> On Fri, Nov 19, 2021 at 4:57 AM Chris Cunningham < >>>>>>>>> chcunn...@chromium.org> wrote: >>>>>>>>> >>>>>>>>>> > If the reported value is incorrect, the A/V synchronization >>>>>>>>>> won't be possible. (Perhaps chcunningham@ can say more about this >>>>>>>>>> use >>>>>>>>>> case.) >>>>>>>>>> >>>>>>>>>> The typical video player design drives the clock with audio. In >>>>>>>>>> short, you send buffers of audio to the OS, use the latency value to >>>>>>>>>> know >>>>>>>>>> when those buffers actually reach the user's ears, and choose a >>>>>>>>>> video frame >>>>>>>>>> accordingly. Apps using WebCodecs will design apps this way (example >>>>>>>>>> here >>>>>>>>>> <https://github.com/chcunningham/wc-talk#simple_video_playerhtml> >>>>>>>>>> ). >>>>>>>>>> >>>>>>>>>> As Mike notes, the latency value can vary by quite a lot >>>>>>>>>> depending on the device (huge differences for bluetooth vs wired), >>>>>>>>>> so it's >>>>>>>>>> critical to have the platform tell you when this changes. In terms >>>>>>>>>> of >>>>>>>>>> precision, we can _probably_ do some rounding like to the nearest >>>>>>>>>> millisecond. Higher values may lead to issues; user studies >>>>>>>>>> <https://en.wikipedia.org/wiki/Audio-to-video_synchronization#Recommendations> >>>>>>>>>> >>>>>>>>>> have shown sensitivity to fairly low values of drift. >>>>>>>>>> >>>>>>>>>> Aside: we may eventually desire a display latency value for >>>>>>>>>> similar reasons. >>>>>>>>>> >>>>>>>>>> On Thu, Nov 18, 2021 at 12:45 PM Hongchan Choi < >>>>>>>>>> hong...@chromium.org> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Yoav, >>>>>>>>>>> >>>>>>>>>>> Could you clarify what needs to be discussed from the spec side? >>>>>>>>>>> The feature (and its behavior) is properly specified, and its >>>>>>>>>>> privacy >>>>>>>>>>> aspect is already reviewed by W3C: >>>>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1498 >>>>>>>>>>> (outputLatency) >>>>>>>>>>> https://github.com/WebAudio/web-audio-api/issues/1495 (general >>>>>>>>>>> security/privacy consideration) >>>>>>>>>>> >>>>>>>>>>> I thought the request from this thread is to devise a >>>>>>>>>>> Chrome-specific mitigation, not reopening the spec discussion. >>>>>>>>>>> Please >>>>>>>>>>> correct me if I'm mistaken. >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> Hongchan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thu, Nov 18, 2021 at 12:37 PM Yoav Weiss < >>>>>>>>>>> yoav...@chromium.org> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hey folks! >>>>>>>>>>>> >>>>>>>>>>>> It seems like there are some things to figure out here before >>>>>>>>>>>> shipping. Could y'all file a spec issue to discuss this with the >>>>>>>>>>>> broader >>>>>>>>>>>> community? >>>>>>>>>>>> >>>>>>>>>>>> On Tuesday, November 16, 2021 at 6:16:14 PM UTC+1 hongchan >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Thanks for the feedback, Mike. This is really helpful! >>>>>>>>>>>>> >>>>>>>>>>>>> Mitigation strategies include adding jitter (dithering) and >>>>>>>>>>>>>> quantization so that the exact skew is incorrectly reported >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> This bit refers to the randomization mechanism deployed by >>>>>>>>>>>>> other browsers, but this approach defeats the purpose of this >>>>>>>>>>>>> specific API >>>>>>>>>>>>> and it is also argued in the spec: >>>>>>>>>>>>> >>>>>>>>>>>>> Note however that most audio systems aim for low latency, to >>>>>>>>>>>>>> synchronise the audio generated by WebAudio to other audio or >>>>>>>>>>>>>> video sources >>>>>>>>>>>>>> or to visual cues (for example in a game, or an audio recording >>>>>>>>>>>>>> or music >>>>>>>>>>>>>> making environment). Excessive latency decreases usability and >>>>>>>>>>>>>> may be an >>>>>>>>>>>>>> accessibility issue. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> If the reported value is incorrect, the A/V synchronization >>>>>>>>>>>>> won't be possible. (Perhaps chcunningham@ can say more about this >>>>>>>>>>>>> use case.) >>>>>>>>>>>>> >>>>>>>>>>>>> One mitigation idea: we can do some research to collect values >>>>>>>>>>>>> reported by major platforms/devices and devise a reasonable >>>>>>>>>>>>> "bucket system" >>>>>>>>>>>>> for them. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Nov 16, 2021 at 8:14 AM Mike Taylor < >>>>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> I suppose it would be nice to do some testing to verify the >>>>>>>>>>>>>> idea that there's a small number of platform + audio device >>>>>>>>>>>>>> pairs. I was >>>>>>>>>>>>>> hoping Mozilla's hardware survey >>>>>>>>>>>>>> <https://data.firefox.com/dashboard/hardware> might have >>>>>>>>>>>>>> something on that, but it doesn't track speakers. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Just from some simple testing with the following script >>>>>>>>>>>>>> (thanks dmcardle@ for the snippet) in Firefox on macOS by 3 >>>>>>>>>>>>>> people, we have >>>>>>>>>>>>>> very different values that change dramatically when you unplug a >>>>>>>>>>>>>> headset: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> (function getNextSample() { >>>>>>>>>>>>>> console.log((new AudioContext).outputLatency); >>>>>>>>>>>>>> setTimeout(getNextSample, 10); >>>>>>>>>>>>>> })(); >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> Here's the values on my macbook, using Firefox Nightly: >>>>>>>>>>>>>> bose headset plugged in: 0.005895833 >>>>>>>>>>>>>> take headset out: 0.013083333 >>>>>>>>>>>>>> plug bose headset back in: 0.005875 (different, but stable) >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://www.w3.org/TR/webaudio/#priv-sec mentions some >>>>>>>>>>>>>> mitigation strategies - do we have a plan to apply any of them? >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 11/15/21 11:53 AM, Ajay Rahatekar wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> The small number of platform + audio output device >>>>>>>>>>>>>> combination i.e. a large number of users with the same platform >>>>>>>>>>>>>> and output >>>>>>>>>>>>>> device (users on macOS using airpods) would likely be a natural >>>>>>>>>>>>>> obfuscator. Not sure if we have data about the distribution. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Nov 15, 2021 at 8:05 AM Mike Taylor < >>>>>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 11/15/21 10:56 AM, Hongchan Choi wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Mon, Nov 15, 2021 at 6:49 AM Mike Taylor < >>>>>>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 11/15/21 1:47 AM, Yoav Weiss wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Fri, Nov 12, 2021 at 5:33 PM 'Ajay Rahatekar' via >>>>>>>>>>>>>>>> blink-dev <blin...@chromium.org> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> TAG review status >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Completed: Web Audio API specification is W3C >>>>>>>>>>>>>>>>> Recommendation. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Risks >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> There is a risk of the feature being used for >>>>>>>>>>>>>>>>> fingerprinting. However outputLatency is the buffer size >>>>>>>>>>>>>>>>> of the platform-provided audio callback, so the value is >>>>>>>>>>>>>>>>> inherently >>>>>>>>>>>>>>>>> platform-specific. That said, the majority of the platform >>>>>>>>>>>>>>>>> audio buffer >>>>>>>>>>>>>>>>> size is widely known. (MacOS = 128 frames, Windows = 10ms, >>>>>>>>>>>>>>>>> Android = 96 >>>>>>>>>>>>>>>>> frames, etc) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This feature does not expose more than what you can >>>>>>>>>>>>>>>>> query/infer from the UA string. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Would that exposure map cleanly to UA-Platform >>>>>>>>>>>>>>>> <https://wicg.github.io/ua-client-hints/#sec-ch-ua-platform>, >>>>>>>>>>>>>>>> which is considered low-entropy and exposed by default? Or >>>>>>>>>>>>>>>> would it add >>>>>>>>>>>>>>>> more than that? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> /cc +Mike Taylor >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Looking at >>>>>>>>>>>>>>>> https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> it states that it depends on the platform _and_ the hardware >>>>>>>>>>>>>>>> output device. >>>>>>>>>>>>>>>> If I use an app using outputLatency with speaker A, then >>>>>>>>>>>>>>>> switch to speaker >>>>>>>>>>>>>>>> B, will the outputLatency remain the same? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The specification says: If the audio output device is >>>>>>>>>>>>>>> changed the outputLatency attribute value will be updated >>>>>>>>>>>>>>> accordingly. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So the answer is no. The value will change accordingly. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If that's the case, then outputLatency can reveal more >>>>>>>>>>>>>>> entropy than just platform alone, right? It would be useful to >>>>>>>>>>>>>>> know what >>>>>>>>>>>>>>> this looks like in practice, or what mitigations we might be >>>>>>>>>>>>>>> able to apply >>>>>>>>>>>>>>> depending on the size of these latency differences. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to blink-dev+...@chromium.org. >>> To view this discussion on the web visit >>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/936f3e2f-2001-0449-13fd-d3a0548b673c%40chromium.org >>> >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/936f3e2f-2001-0449-13fd-d3a0548b673c%40chromium.org?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/e596e35f-e183-4e14-8e57-ae8cc52d2b43n%40chromium.org.
