So far we have not discussed the deprecation plan, but I believe that's reasonable as well.
I registered myself to the webkit-dev list to post a question, but somehow I am getting 403 forbidden. The email is sent to the list, I am not seeing the message is getting posted on the list. Not sure what to do there. On Wed, Dec 1, 2021 at 3:18 AM Yoav Weiss <yoavwe...@chromium.org> wrote: > So are y'all planning to deprecate and remove `getOutputTimestamp` once > this ships? If so, that sounds reasonable. > > I note Chris asked y'all to file for a webkit signal upthread. Did that > happen? > > On Tuesday, November 30, 2021 at 12:54:17 AM UTC+1 hongchan wrote: > >> I agree that outputLatency is better in terms of both usability and >> interoperability. >> >> FYI, the counter shows getOutputTimestamp is about 0.001% and there are >> no partners who are using this API: >> >> https://chromestatus.com/metrics/feature/popularity#AudioContextGetOutputTimestamp >> >> >> On Mon, Nov 22, 2021 at 3:00 PM Chris Cunningham <chcunning...@google.com> >> wrote: >> >>> Re: privacy discussion, Hongchan and I scheduled to meet with Paul >>> Jensen next week. >>> >>> Also, I recently determined that this new WebAudio API (outputLatency) >>> is largely redundant with an existing API (getOutputTimestamp()). >>> https://github.com/WebAudio/web-audio-api/issues/2461 >>> >>> Chrome has already shipped the latter API, so I believe this new API >>> doesn't add potential for additional fingerprinting. Shipping both APIs is >>> still a good idea for the sake of inter-op. >>> >>> On Friday, November 19, 2021 at 8:45:28 AM UTC-8 hongchan wrote: >>> >>>> Thanks for the clarification, Yoav! >>>> >>>> I hope the mitigation approach in the previous comment makes sense from >>>> the privacy perspective, but it will hurt the usability/value of this API >>>> significantly. >>>> For the privacy-focused discussion, should I use other venues rather >>>> than this I2S thread? >>>> >>>> Cheers, >>>> Hongchan >>>> >>>> >>>> On Thu, Nov 18, 2021 at 10:35 PM Yoav Weiss <yoav...@chromium.org> >>>> wrote: >>>> >>>>> Apologies, I misread the discussion. A spec issue is indeed not >>>>> needed, as this is already covered. >>>>> >>>>> From my perspective, since this exposes active entropy, the fact that >>>>> it exposes more data than the (passively exposed) platform is not >>>>> prohibitive on its own. >>>>> At the same time, it'd be good to work with privacy-focused folks to >>>>> make sure we expose as little information as needed to maintain the API's >>>>> functionality, but not more. >>>>> >>>>> On Fri, Nov 19, 2021 at 4:57 AM Chris Cunningham < >>>>> chcunn...@chromium.org> wrote: >>>>> >>>>>> > If the reported value is incorrect, the A/V synchronization won't >>>>>> be possible. (Perhaps chcunningham@ can say more about this use >>>>>> case.) >>>>>> >>>>>> The typical video player design drives the clock with audio. In >>>>>> short, you send buffers of audio to the OS, use the latency value to know >>>>>> when those buffers actually reach the user's ears, and choose a video >>>>>> frame >>>>>> accordingly. Apps using WebCodecs will design apps this way (example >>>>>> here >>>>>> <https://github.com/chcunningham/wc-talk#simple_video_playerhtml>). >>>>>> >>>>>> As Mike notes, the latency value can vary by quite a lot depending on >>>>>> the device (huge differences for bluetooth vs wired), so it's critical to >>>>>> have the platform tell you when this changes. In terms of precision, we >>>>>> can >>>>>> _probably_ do some rounding like to the nearest millisecond. Higher >>>>>> values >>>>>> may lead to issues; user studies >>>>>> <https://en.wikipedia.org/wiki/Audio-to-video_synchronization#Recommendations> >>>>>> have shown sensitivity to fairly low values of drift. >>>>>> >>>>>> Aside: we may eventually desire a display latency value for similar >>>>>> reasons. >>>>>> >>>>>> On Thu, Nov 18, 2021 at 12:45 PM Hongchan Choi <hong...@chromium.org> >>>>>> wrote: >>>>>> >>>>>>> Hi Yoav, >>>>>>> >>>>>>> Could you clarify what needs to be discussed from the spec side? The >>>>>>> feature (and its behavior) is properly specified, and its privacy >>>>>>> aspect is >>>>>>> already reviewed by W3C: >>>>>>> https://github.com/WebAudio/web-audio-api/issues/1498 >>>>>>> (outputLatency) >>>>>>> https://github.com/WebAudio/web-audio-api/issues/1495 (general >>>>>>> security/privacy consideration) >>>>>>> >>>>>>> I thought the request from this thread is to devise a >>>>>>> Chrome-specific mitigation, not reopening the spec discussion. Please >>>>>>> correct me if I'm mistaken. >>>>>>> >>>>>>> Cheers, >>>>>>> Hongchan >>>>>>> >>>>>>> >>>>>>> On Thu, Nov 18, 2021 at 12:37 PM Yoav Weiss <yoav...@chromium.org> >>>>>>> wrote: >>>>>>> >>>>>>>> Hey folks! >>>>>>>> >>>>>>>> It seems like there are some things to figure out here before >>>>>>>> shipping. Could y'all file a spec issue to discuss this with the >>>>>>>> broader >>>>>>>> community? >>>>>>>> >>>>>>>> On Tuesday, November 16, 2021 at 6:16:14 PM UTC+1 hongchan wrote: >>>>>>>> >>>>>>>>> Thanks for the feedback, Mike. This is really helpful! >>>>>>>>> >>>>>>>>> Mitigation strategies include adding jitter (dithering) and >>>>>>>>>> quantization so that the exact skew is incorrectly reported >>>>>>>>>> >>>>>>>>> >>>>>>>>> This bit refers to the randomization mechanism deployed by other >>>>>>>>> browsers, but this approach defeats the purpose of this specific API >>>>>>>>> and it >>>>>>>>> is also argued in the spec: >>>>>>>>> >>>>>>>>> Note however that most audio systems aim for low latency, to >>>>>>>>>> synchronise the audio generated by WebAudio to other audio or video >>>>>>>>>> sources >>>>>>>>>> or to visual cues (for example in a game, or an audio recording or >>>>>>>>>> music >>>>>>>>>> making environment). Excessive latency decreases usability and may >>>>>>>>>> be an >>>>>>>>>> accessibility issue. >>>>>>>>> >>>>>>>>> >>>>>>>>> If the reported value is incorrect, the A/V synchronization won't >>>>>>>>> be possible. (Perhaps chcunningham@ can say more about this use >>>>>>>>> case.) >>>>>>>>> >>>>>>>>> One mitigation idea: we can do some research to collect values >>>>>>>>> reported by major platforms/devices and devise a reasonable "bucket >>>>>>>>> system" >>>>>>>>> for them. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Nov 16, 2021 at 8:14 AM Mike Taylor <mike...@chromium.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> I suppose it would be nice to do some testing to verify the idea >>>>>>>>>> that there's a small number of platform + audio device pairs. I was >>>>>>>>>> hoping Mozilla's >>>>>>>>>> hardware survey <https://data.firefox.com/dashboard/hardware> >>>>>>>>>> might have something on that, but it doesn't track speakers. >>>>>>>>>> >>>>>>>>>> Just from some simple testing with the following script (thanks >>>>>>>>>> dmcardle@ for the snippet) in Firefox on macOS by 3 people, we >>>>>>>>>> have very different values that change dramatically when you unplug a >>>>>>>>>> headset: >>>>>>>>>> >>>>>>>>>> ``` >>>>>>>>>> (function getNextSample() { >>>>>>>>>> console.log((new AudioContext).outputLatency); >>>>>>>>>> setTimeout(getNextSample, 10); >>>>>>>>>> })(); >>>>>>>>>> ``` >>>>>>>>>> Here's the values on my macbook, using Firefox Nightly: >>>>>>>>>> bose headset plugged in: 0.005895833 >>>>>>>>>> take headset out: 0.013083333 >>>>>>>>>> plug bose headset back in: 0.005875 (different, but stable) >>>>>>>>>> >>>>>>>>>> https://www.w3.org/TR/webaudio/#priv-sec mentions some >>>>>>>>>> mitigation strategies - do we have a plan to apply any of them? >>>>>>>>>> >>>>>>>>>> On 11/15/21 11:53 AM, Ajay Rahatekar wrote: >>>>>>>>>> >>>>>>>>>> The small number of platform + audio output device combination >>>>>>>>>> i.e. a large number of users with the same platform and output >>>>>>>>>> device (users on macOS using airpods) would likely be a natural >>>>>>>>>> obfuscator. Not sure if we have data about the distribution. >>>>>>>>>> >>>>>>>>>> On Mon, Nov 15, 2021 at 8:05 AM Mike Taylor <mike...@chromium.org> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> On 11/15/21 10:56 AM, Hongchan Choi wrote: >>>>>>>>>>> >>>>>>>>>>> On Mon, Nov 15, 2021 at 6:49 AM Mike Taylor < >>>>>>>>>>> mike...@chromium.org> wrote: >>>>>>>>>>> >>>>>>>>>>>> On 11/15/21 1:47 AM, Yoav Weiss wrote: >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Nov 12, 2021 at 5:33 PM 'Ajay Rahatekar' via blink-dev < >>>>>>>>>>>> blin...@chromium.org> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> TAG review status >>>>>>>>>>>>> >>>>>>>>>>>>> Completed: Web Audio API specification is W3C Recommendation. >>>>>>>>>>>>> >>>>>>>>>>>>> Risks >>>>>>>>>>>>> >>>>>>>>>>>>> There is a risk of the feature being used for fingerprinting. >>>>>>>>>>>>> However outputLatency is the buffer size of the >>>>>>>>>>>>> platform-provided audio callback, so the value is inherently >>>>>>>>>>>>> platform-specific. That said, the majority of the platform audio >>>>>>>>>>>>> buffer >>>>>>>>>>>>> size is widely known. (MacOS = 128 frames, Windows = 10ms, >>>>>>>>>>>>> Android = 96 >>>>>>>>>>>>> frames, etc) >>>>>>>>>>>>> >>>>>>>>>>>>> This feature does not expose more than what you can >>>>>>>>>>>>> query/infer from the UA string. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Would that exposure map cleanly to UA-Platform >>>>>>>>>>>> <https://wicg.github.io/ua-client-hints/#sec-ch-ua-platform>, >>>>>>>>>>>> which is considered low-entropy and exposed by default? Or would >>>>>>>>>>>> it add >>>>>>>>>>>> more than that? >>>>>>>>>>>> >>>>>>>>>>>> /cc +Mike Taylor >>>>>>>>>>>> >>>>>>>>>>>>> Looking at >>>>>>>>>>>> https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency, >>>>>>>>>>>> it states that it depends on the platform _and_ the hardware >>>>>>>>>>>> output device. >>>>>>>>>>>> If I use an app using outputLatency with speaker A, then switch to >>>>>>>>>>>> speaker >>>>>>>>>>>> B, will the outputLatency remain the same? >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The specification says: If the audio output device is changed >>>>>>>>>>> the outputLatency attribute value will be updated accordingly. >>>>>>>>>>> >>>>>>>>>>> So the answer is no. The value will change accordingly. >>>>>>>>>>> >>>>>>>>>>> If that's the case, then outputLatency can reveal more entropy >>>>>>>>>>> than just platform alone, right? It would be useful to know what >>>>>>>>>>> this looks >>>>>>>>>>> like in practice, or what mitigations we might be able to apply >>>>>>>>>>> depending >>>>>>>>>>> on the size of these latency differences. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGJqXNurUNB5qoNFQnm-xSrfzuPXy_HZTZ_AR-5d5SDGNwOt0Q%40mail.gmail.com.
