Re: [webkit-dev] Request for position on AudioContext.outputLatency

[Yoav Weiss via webkit-dev]

(Copying Hongchan's reply to the list)

On Wed, Dec 15, 2021 at 5:44 PM Hongchan Choi  wrote:

> Hi Youenn,
>
> A first good step would be to add the corresponding fingerprinting markup
>> to the web audio spec.
>>
>
> Do you believe the current language is not enough? Then please file an
> issue against the spec, so we can reopen the discussion in WG:
> https://github.com/WebAudio/web-audio-api/issues
>
> https://w3c.github.io/mediacapture-output/ allows listing devices that a
>> user granted.
>> In case devices are exposed to JS, exposing latency for those devices
>> seems fine.
>>
>
> Doesn't it require a full permission from the user? I am not sure whether
> exposing the latency value has the same severity as allowing the microphone
> access.
>
> I also wonder whether output latency might be useful outside of
>> AudioContext and whether it would make sense to expose this information on
>> MediaDevices as well. Could that be a device selection choice?
>>
>
> MediaTrackConstraints have "latency" field (
> https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints/latency),
> but I am not sure that is within the scope of the new API in question.
>
> The main issue is with the default system audio output. Passive filtering
>> should not be allowed in that case. For instance, if the AudioContext is
>> not producing audio, exposing latency does not make sense.
>>
>
> When AudioContext is suspended or not allowed to play, the latency should
> return zero value. This makes sense because the delay information from the
> audio infra changes every audio callback, but the suspended AudioContext
> won't have any calls from the device. Hence, the zero value.
>
> Can you elaborate what you meant by "passive filtering"?
>
> I am a bit unclear about when latency can change, for instance say audio
>> context is running and the default system speaker is changed. If it can
>> change, should there be some form of events to let the application know. Or
>> should it be left to things like MediaDevices.ondevicechange?
>>
>
> MediaDevices.ondevicechange is designed to serve the purpose you
> mentioned. Upon this event, the user can enumerate the device listing again
> to get the latest information - which is gated by the user permission.
>
> Cheers,
> Hongchan
>
> PS: Whom should I talk to about the WebKit mailing list subscription? When
> I log in at https://lists.webkit.org/mailman/listinfo/webkit-dev, I am
> still getting 403.
>
>
> On Wed, Dec 15, 2021 at 8:06 AM youenn fablet  wrote:
>
>> Thanks Hongchan,
>>
>> It is good to hear that these discussions are happening.
>> A first good step would be to add the corresponding fingerprinting markup
>> to the web audio spec.
>>
>> A few additional thoughts:
>>
>> https://w3c.github.io/mediacapture-output/ allows listing devices that a
>> user granted.
>> In case devices are exposed to JS, exposing latency for those devices
>> seems fine.
>> I also wonder whether output latency might be useful outside of
>> AudioContext and whether it would make sense to expose this information on
>> MediaDevices as well. Could that be a device selection choice?
>>
>> The main issue is with the default system audio output. Passive filtering
>> should not be allowed in that case. For instance, if the AudioContext is
>> not producing audio, exposing latency does not make sense.
>>
>> I am a bit unclear about when latency can change, for instance say audio
>> context is running and the default system speaker is changed. If it can
>> change, should there be some form of events to let the application know. Or
>> should it be left to things like MediaDevices.ondevicechange?
>>
>>
>> Le lun. 13 déc. 2021 à 20:00, Hongchan Choi  a
>> écrit :
>>
>>> Hi Youenn,
>>>
>>> Thanks for your response.
>>>
>>> Say I switch from builtin speakers to Bluetooth headset using MacOS
 system menu.

>>>
>>> If changing a device doesn't reflect the current status correctly, the
>>> feature is not useful. That said, the accuracy/precision of a value being
>>> served is up to the user agent's discretion.
>>>
>>> If so, the spec should identify this as potential fingerprinting and
 should provide mitigations.
 And we should evaluate fingerprinting-free alternatives.
>>>
>>>
>>> We are aware of the issue and it is being discussed in the blink-dev I2S
>>> thread.
>>>
>>> What does PING WG think about this?
>>>
>>>
>>> Please note that Web Audio API is currently a W3C Recommendation.
>>> - The WG went through several PING reviews in 2020, for example:
>>> https://github.com/WebAudio/web-audio-api/issues/2061
>>> - One of device-related PING threads:
>>> https://github.com/w3cping/tracking-issues/issues/50
>>>
>>> As a result of the privacy discussion, the spec has a dedicated section
>>> on security and privacy:
>>> https://webaudio.github.io/web-audio-api/#priv-sec
>>>
>>> The relevant excerpt:
>>> "Fingerprinting via latency is also possible; it might be possible to
>>> deduce this from baseLatency 

Re: [webkit-dev] Request for position on AudioContext.outputLatency

[youenn fablet via webkit-dev]

Thanks Hongchan,

It is good to hear that these discussions are happening.
A first good step would be to add the corresponding fingerprinting markup
to the web audio spec.

A few additional thoughts:

https://w3c.github.io/mediacapture-output/ allows listing devices that a
user granted.
In case devices are exposed to JS, exposing latency for those devices seems
fine.
I also wonder whether output latency might be useful outside of
AudioContext and whether it would make sense to expose this information on
MediaDevices as well. Could that be a device selection choice?

The main issue is with the default system audio output. Passive filtering
should not be allowed in that case. For instance, if the AudioContext is
not producing audio, exposing latency does not make sense.

I am a bit unclear about when latency can change, for instance say audio
context is running and the default system speaker is changed. If it can
change, should there be some form of events to let the application know. Or
should it be left to things like MediaDevices.ondevicechange?


Le lun. 13 déc. 2021 à 20:00, Hongchan Choi  a
écrit :

> Hi Youenn,
>
> Thanks for your response.
>
> Say I switch from builtin speakers to Bluetooth headset using MacOS system
>> menu.
>>
>
> If changing a device doesn't reflect the current status correctly, the
> feature is not useful. That said, the accuracy/precision of a value being
> served is up to the user agent's discretion.
>
> If so, the spec should identify this as potential fingerprinting and
>> should provide mitigations.
>> And we should evaluate fingerprinting-free alternatives.
>
>
> We are aware of the issue and it is being discussed in the blink-dev I2S
> thread.
>
> What does PING WG think about this?
>
>
> Please note that Web Audio API is currently a W3C Recommendation.
> - The WG went through several PING reviews in 2020, for example:
> https://github.com/WebAudio/web-audio-api/issues/2061
> - One of device-related PING threads:
> https://github.com/w3cping/tracking-issues/issues/50
>
> As a result of the privacy discussion, the spec has a dedicated section on
> security and privacy:
> https://webaudio.github.io/web-audio-api/#priv-sec
>
> The relevant excerpt:
> "Fingerprinting via latency is also possible; it might be possible to
> deduce this from baseLatency and outputLatency. Mitigation strategies
> include adding jitter (dithering) and quantization so that the exact skew
> is incorrectly reported. Note however that most audio systems aim for low
> latency, to synchronise the audio generated by WebAudio to other audio or
> video sources or to visual cues (for example in a game, or an audio
> recording or music making environment). Excessive latency decreases
> usability and may be an accessibility issue."
>
> Cheers,
> Hongchan
>
>
> On Mon, Dec 13, 2021 at 1:15 AM youenn fablet  wrote:
>
>> Looking at https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency,
>> it states that:
>> > If the audio output device is changed the outputLatency
>>  attribute
>> value will be updated accordingly.
>>
>> The use case seems ok, but I worry about fingerprinting if it means this
>> allows a web page to passively identify user speakers.
>> Say I switch from builtin speakers to Bluetooth headset using MacOS
>> system menu.
>>
>> If so, the spec should identify this as potential fingerprinting and
>> should provide mitigations.
>> And we should evaluate fingerprinting-free alternatives.
>> What does PING WG think about this?
>>
>> Le lun. 13 déc. 2021 à 09:39, Yoav Weiss via webkit-dev <
>> webkit-dev@lists.webkit.org> a écrit :
>>
>>> (Sent on behalf of Hongchan Choi, who failed to subscribe to this
>>> mailing list)
>>>
>>> Hey folks!
>>>
>>> AudioContext.outputLatency is to inform the time at which the first
>>> sample in the buffer is actually processed by the audio output device. This
>>> is useful when synchronizing the audio generated by Web Audio to other
>>> audio or video sources or to visual cues [1].
>>>
>>> This is already implemented in FireFox and we're looking to ship it in
>>> Chrome soon [2][3]. Would you all be interested in this feature?
>>>
>>> Thanks,
>>> Hongchan
>>>
>>> [1] https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency
>>> [2] https://bugzilla.mozilla.org/show_bug.cgi?id=1324552
>>> [3]
>>> https://groups.google.com/a/chromium.org/g/blink-dev/c/dTQniJNVVMY/m/hPFwY1fbBQAJ
>>>
>>> ___
>>> webkit-dev mailing list
>>> webkit-dev@lists.webkit.org
>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>>
>>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Request for position on AudioContext.outputLatency

[Yoav Weiss via webkit-dev]

Copying Hongchan's reply to the list..

On Mon, Dec 13, 2021 at 8:00 PM Hongchan Choi  wrote:

> Hi Youenn,
>
> Thanks for your response.
>
> Say I switch from builtin speakers to Bluetooth headset using MacOS system
>> menu.
>>
>
> If changing a device doesn't reflect the current status correctly, the
> feature is not useful. That said, the accuracy/precision of a value being
> served is up to the user agent's discretion.
>
> If so, the spec should identify this as potential fingerprinting and
>> should provide mitigations.
>> And we should evaluate fingerprinting-free alternatives.
>
>
> We are aware of the issue and it is being discussed in the blink-dev I2S
> thread.
>
> What does PING WG think about this?
>
>
> Please note that Web Audio API is currently a W3C Recommendation.
> - The WG went through several PING reviews in 2020, for example:
> https://github.com/WebAudio/web-audio-api/issues/2061
> - One of device-related PING threads:
> https://github.com/w3cping/tracking-issues/issues/50
>
> As a result of the privacy discussion, the spec has a dedicated section on
> security and privacy:
> https://webaudio.github.io/web-audio-api/#priv-sec
>
> The relevant excerpt:
> "Fingerprinting via latency is also possible; it might be possible to
> deduce this from baseLatency and outputLatency. Mitigation strategies
> include adding jitter (dithering) and quantization so that the exact skew
> is incorrectly reported. Note however that most audio systems aim for low
> latency, to synchronise the audio generated by WebAudio to other audio or
> video sources or to visual cues (for example in a game, or an audio
> recording or music making environment). Excessive latency decreases
> usability and may be an accessibility issue."
>
> Cheers,
> Hongchan
>
>
> On Mon, Dec 13, 2021 at 1:15 AM youenn fablet  wrote:
>
>> Looking at https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency,
>> it states that:
>> > If the audio output device is changed the outputLatency
>>  attribute
>> value will be updated accordingly.
>>
>> The use case seems ok, but I worry about fingerprinting if it means this
>> allows a web page to passively identify user speakers.
>> Say I switch from builtin speakers to Bluetooth headset using MacOS
>> system menu.
>>
>> If so, the spec should identify this as potential fingerprinting and
>> should provide mitigations.
>> And we should evaluate fingerprinting-free alternatives.
>> What does PING WG think about this?
>>
>> Le lun. 13 déc. 2021 à 09:39, Yoav Weiss via webkit-dev <
>> webkit-dev@lists.webkit.org> a écrit :
>>
>>> (Sent on behalf of Hongchan Choi, who failed to subscribe to this
>>> mailing list)
>>>
>>> Hey folks!
>>>
>>> AudioContext.outputLatency is to inform the time at which the first
>>> sample in the buffer is actually processed by the audio output device. This
>>> is useful when synchronizing the audio generated by Web Audio to other
>>> audio or video sources or to visual cues [1].
>>>
>>> This is already implemented in FireFox and we're looking to ship it in
>>> Chrome soon [2][3]. Would you all be interested in this feature?
>>>
>>> Thanks,
>>> Hongchan
>>>
>>> [1] https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency
>>> [2] https://bugzilla.mozilla.org/show_bug.cgi?id=1324552
>>> [3]
>>> https://groups.google.com/a/chromium.org/g/blink-dev/c/dTQniJNVVMY/m/hPFwY1fbBQAJ
>>>
>>> ___
>>> webkit-dev mailing list
>>> webkit-dev@lists.webkit.org
>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>>
>>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Request for position on AudioContext.outputLatency

[youenn fablet via webkit-dev]

Looking at https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency,
it states that:
> If the audio output device is changed the outputLatency
 attribute
value will be updated accordingly.

The use case seems ok, but I worry about fingerprinting if it means this
allows a web page to passively identify user speakers.
Say I switch from builtin speakers to Bluetooth headset using MacOS system
menu.

If so, the spec should identify this as potential fingerprinting and should
provide mitigations.
And we should evaluate fingerprinting-free alternatives.
What does PING WG think about this?

Le lun. 13 déc. 2021 à 09:39, Yoav Weiss via webkit-dev <
webkit-dev@lists.webkit.org> a écrit :

> (Sent on behalf of Hongchan Choi, who failed to subscribe to this mailing
> list)
>
> Hey folks!
>
> AudioContext.outputLatency is to inform the time at which the first sample
> in the buffer is actually processed by the audio output device. This is
> useful when synchronizing the audio generated by Web Audio to other audio
> or video sources or to visual cues [1].
>
> This is already implemented in FireFox and we're looking to ship it in
> Chrome soon [2][3]. Would you all be interested in this feature?
>
> Thanks,
> Hongchan
>
> [1] https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency
> [2] https://bugzilla.mozilla.org/show_bug.cgi?id=1324552
> [3]
> https://groups.google.com/a/chromium.org/g/blink-dev/c/dTQniJNVVMY/m/hPFwY1fbBQAJ
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] Request for position on AudioContext.outputLatency

[Yoav Weiss via webkit-dev]

(Sent on behalf of Hongchan Choi, who failed to subscribe to this mailing
list)

Hey folks!

AudioContext.outputLatency is to inform the time at which the first sample
in the buffer is actually processed by the audio output device. This is
useful when synchronizing the audio generated by Web Audio to other audio
or video sources or to visual cues [1].

This is already implemented in FireFox and we're looking to ship it in
Chrome soon [2][3]. Would you all be interested in this feature?

Thanks,
Hongchan

[1] https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1324552
[3]
https://groups.google.com/a/chromium.org/g/blink-dev/c/dTQniJNVVMY/m/hPFwY1fbBQAJ
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev