Re: Intent to prototype: Web Speech API
Right, I can see the threat model being similar, but technically we're marrying two separate things under the same UI and more importantly the same permission name. This will instantly cause trouble once one of the two features changes its requirements or behavior in a way that's incompatible with the other, which I think is not unimaginable here. Hence my recommendation to avoid using the same permission name right now and using a separate UI as soon as that can be prioritized. On Wed, Oct 16, 2019, 19:43 Daniel Veditz wrote: > On Wed, Oct 16, 2019 at 4:40 AM Johann Hofmann > wrote: > >> as far as I can see the presented origin does not in fact get access to >> the user's microphone > > > The site doesn't get raw audio, but does get text representing what the > browser thinks it heard. It's the same kind of privacy risk as raw audio > for most people (though less opportunity for creative abuses like trying to > track what TV show you're watching). > > -Dan Veditz > > > > >> and it's a bit >> unclear what "Remember this decision" actually does. It makes no sense to >> set the "microphone" permission on that site, in the same way that it >> makes >> no sense to derive from a permanent "microphone" permission for some site >> that the user intends to submit their voice data to a third party. I feel >> like this feature needs to store a separate permanent permission. >> >> A perfect permissions UX may not be achievable or intended for an MVP of >> this feature, so I would recommend at least hiding the checkbox (to avoid >> setting the "microphone" permission) and prompting every time until a >> better solution can be found. >> >> Let me know if you need any help with that :) >> >> Johann >> >> On Tue, Oct 15, 2019 at 12:27 PM Henri Sivonen >> wrote: >> >> > On Tue, Oct 15, 2019 at 2:56 AM Andre Natal wrote: >> > > Regarding the UI, yes, the experience will be exactly the same in our >> > case: the user will get a prompt asking for permission to open the >> > microphone (I've attached a screenshot below [3]) >> > ... >> > > [3] >> > >> https://www.dropbox.com/s/fkyymiyryjjbix5/Screenshot%202019-10-14%2016.13.49.png?dl=0 >> > >> > Since the UI is the same as for getUserMedia(), is the permission bit >> > that gets stored the same as for getUserMedia()? I.e. if a site >> > obtains the permission for one, can it also use the other without >> > another prompt? >> > >> > If a user understands how WebRTC works and what this piece of UI meant >> > for WebRTC, this UI now represents a different trust decision on the >> > level of principle. How intentional or incidental is it that this >> > looks like a getUserMedia() use (audio goes to where the site named in >> > the dialog decides to route it) instead of surfacing to the user that >> > this is different (audio goes to where the browser vendor decides to >> > route it)? >> > >> > -- >> > Henri Sivonen >> > hsivo...@mozilla.com >> > ___ >> > dev-platform mailing list >> > dev-platform@lists.mozilla.org >> > https://lists.mozilla.org/listinfo/dev-platform >> > >> ___ >> dev-platform mailing list >> dev-platform@lists.mozilla.org >> https://lists.mozilla.org/listinfo/dev-platform >> > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to prototype: Web Speech API
I changed the subject of this thread to fit properly the current intent. We also moved the FAQ from Gdocs to mozilla wiki [1] with more current and updated info. I've also added more content about offline recognition and how to use deep speech to the ones interested about it. Let's just use the wiki as the main source of info about this work. Thanks Andre [1] https://wiki.mozilla.org/Web_Speech_API_-_Speech_Recognition On Mon, Oct 14, 2019 at 4:20 PM Andre Natal wrote: > > Hi Henri, > > the API isn't available in Nightly yet since the code wasn't fully > reviewed neither merged yet. You can follow its progress here [1] and here > [2]. If you want to try it before it's merged, just apply the patch [2] to > an updated gecko-dev branch, switch on the *media.**webspeech.* > *recognition.**enable *and *media.**webspeech.**recognition.**force_enable > *flags, and browse to that page again. > > Regarding the UI, yes, the experience will be exactly the same in our > case: the user will get a prompt asking for permission to open the > microphone (I've attached a screenshot below [3]), but in our case the > audio will be sent to the endpoint set in the > *media.webspeech.service.endpoint > *pref, which the user will be allowed to change (differently than > Chrome). But if that's unset, it will send it to Mozilla's own server which > is defaulted to in the code. > > [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1248897#c18 > [2] https://phabricator.services.mozilla.com/D26047 > [3] > [image: Screenshot 2019-10-14 16.13.49.png] > > On Mon, Oct 14, 2019 at 1:39 AM Henri Sivonen > wrote: > >> On Sat, Oct 12, 2019 at 12:29 PM Andre Natal wrote: >> > We tried to capture everything here [1], so please if you don't see your >> > question addressed in this document, just give us a shout either here in >> > the thread or directly. >> ... >> > [1] >> > >> https://docs.google.com/document/d/1BE90kgbwE37fWoQ8vqnsQ3YMiJCKJSvqQwa463yCN1Y/edit?ts=5da0f63f# >> >> Thanks. It doesn't address the question of what the UI in Firefox is >> like. Following the links for experimenting with the UI on one's own >> leads to https://mdn.github.io/web-speech-api/speech-color-changer/ , >> which doesn't work in Nightly even with prefs flipped. >> >> (Trying that example in Chrome shows that Chrome presents the >> permission prompt as a matter of sharing the microphone with >> mdn.github.io as if this was WebRTC, which suggests that mdn.github.io >> decides where the audio goes. Chrome does not surface that, if I >> understand correctly how this API works in Chrome, the audio is >> instead sent to a destination of Chrome's choosing and not to a >> destination of mdn.github.io's choosing. The example didn't work for >> me in Safari.) >> >> -- >> Henri Sivonen >> hsivo...@mozilla.com >> > > > -- > > -- > Thanks, > > Andre > -- -- Thanks, Andre ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to prototype: Web Speech API
Putting on my hat as one of the people maintaining our permissions UI, I generally agree with Henri that it would be nice to have a slightly different UI for this use-case, i.e. as far as I can see the presented origin does not in fact get access to the user's microphone and it's a bit unclear what "Remember this decision" actually does. It makes no sense to set the "microphone" permission on that site, in the same way that it makes no sense to derive from a permanent "microphone" permission for some site that the user intends to submit their voice data to a third party. I feel like this feature needs to store a separate permanent permission. A perfect permissions UX may not be achievable or intended for an MVP of this feature, so I would recommend at least hiding the checkbox (to avoid setting the "microphone" permission) and prompting every time until a better solution can be found. Let me know if you need any help with that :) Johann On Tue, Oct 15, 2019 at 12:27 PM Henri Sivonen wrote: > On Tue, Oct 15, 2019 at 2:56 AM Andre Natal wrote: > > Regarding the UI, yes, the experience will be exactly the same in our > case: the user will get a prompt asking for permission to open the > microphone (I've attached a screenshot below [3]) > ... > > [3] > https://www.dropbox.com/s/fkyymiyryjjbix5/Screenshot%202019-10-14%2016.13.49.png?dl=0 > > Since the UI is the same as for getUserMedia(), is the permission bit > that gets stored the same as for getUserMedia()? I.e. if a site > obtains the permission for one, can it also use the other without > another prompt? > > If a user understands how WebRTC works and what this piece of UI meant > for WebRTC, this UI now represents a different trust decision on the > level of principle. How intentional or incidental is it that this > looks like a getUserMedia() use (audio goes to where the site named in > the dialog decides to route it) instead of surfacing to the user that > this is different (audio goes to where the browser vendor decides to > route it)? > > -- > Henri Sivonen > hsivo...@mozilla.com > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to prototype: Web Speech API
On Tue, Oct 15, 2019 at 2:56 AM Andre Natal wrote: > Regarding the UI, yes, the experience will be exactly the same in our case: > the user will get a prompt asking for permission to open the microphone (I've > attached a screenshot below [3]) ... > [3] > https://www.dropbox.com/s/fkyymiyryjjbix5/Screenshot%202019-10-14%2016.13.49.png?dl=0 Since the UI is the same as for getUserMedia(), is the permission bit that gets stored the same as for getUserMedia()? I.e. if a site obtains the permission for one, can it also use the other without another prompt? If a user understands how WebRTC works and what this piece of UI meant for WebRTC, this UI now represents a different trust decision on the level of principle. How intentional or incidental is it that this looks like a getUserMedia() use (audio goes to where the site named in the dialog decides to route it) instead of surfacing to the user that this is different (audio goes to where the browser vendor decides to route it)? -- Henri Sivonen hsivo...@mozilla.com ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to prototype: Web Speech API
I changed the subject of this thread to properly fits the current intent. We also moved the FAQ from gdocs to mozilla wiki [1] with more current and updated info. I've just added more content about offline recognition and how to use deep speech to the ones interested. Let's just use that wiki as the main source of info about this work. Thanks Andre [1] https://wiki.mozilla.org/Web_Speech_API_-_Speech_Recognition On Mon, Oct 14, 2019 at 4:55 PM Andre Natal wrote: > > Hi Henri, > > the API isn't available in Nightly yet since the code wasn't fully > reviewed neither merged yet. You can follow its progress here [1] and here > [2]. If you want to try it before it's merged, just apply the patch [2] to > an updated gecko-dev branch, switch on the *media.**webspeech.* > *recognition.**enable *and *media.**webspeech.**recognition.**force_enable > *flags, and browse to that page again. > > Regarding the UI, yes, the experience will be exactly the same in our > case: the user will get a prompt asking for permission to open the > microphone (I've attached a screenshot below [3]), but in our case the > audio will be sent to the endpoint set in the > *media.webspeech.service.endpoint > *pref, which the user will be allowed to change (differently than > Chrome). But if that's unset, it will send it to Mozilla's own server which > is defaulted to in the code. > > [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1248897#c18 > [2] https://phabricator.services.mozilla.com/D26047 > [3] > https://www.dropbox.com/s/fkyymiyryjjbix5/Screenshot%202019-10-14%2016.13.49.png?dl=0 > > > > On Mon, Oct 14, 2019 at 1:39 AM Henri Sivonen > wrote: > >> On Sat, Oct 12, 2019 at 12:29 PM Andre Natal wrote: >> > We tried to capture everything here [1], so please if you don't see your >> > question addressed in this document, just give us a shout either here in >> > the thread or directly. >> ... >> > [1] >> > >> https://docs.google.com/document/d/1BE90kgbwE37fWoQ8vqnsQ3YMiJCKJSvqQwa463yCN1Y/edit?ts=5da0f63f# >> >> Thanks. It doesn't address the question of what the UI in Firefox is >> like. Following the links for experimenting with the UI on one's own >> leads to https://mdn.github.io/web-speech-api/speech-color-changer/ , >> which doesn't work in Nightly even with prefs flipped. >> >> (Trying that example in Chrome shows that Chrome presents the >> permission prompt as a matter of sharing the microphone with >> mdn.github.io as if this was WebRTC, which suggests that mdn.github.io >> decides where the audio goes. Chrome does not surface that, if I >> understand correctly how this API works in Chrome, the audio is >> instead sent to a destination of Chrome's choosing and not to a >> destination of mdn.github.io's choosing. The example didn't work for >> me in Safari.) >> >> -- >> Henri Sivonen >> hsivo...@mozilla.com >> > > > -- > > -- > Thanks, > > Andre > -- -- Thanks, Andre ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to prototype: Web Speech API
Hi Henri, the API isn't available in Nightly yet since the code wasn't fully reviewed neither merged yet. You can follow its progress here [1] and here [2]. If you want to try it before it's merged, just apply the patch [2] to an updated gecko-dev branch, switch on the *media.**webspeech.**recognition.**enable *and *media.**webspeech.**recognition.**force_enable *flags, and browse to that page again. Regarding the UI, yes, the experience will be exactly the same in our case: the user will get a prompt asking for permission to open the microphone (I've attached a screenshot below [3]), but in our case the audio will be sent to the endpoint set in the *media.webspeech.service.endpoint *pref, which the user will be allowed to change (differently than Chrome). But if that's unset, it will send it to Mozilla's own server which is defaulted to in the code. [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1248897#c18 [2] https://phabricator.services.mozilla.com/D26047 [3] https://www.dropbox.com/s/fkyymiyryjjbix5/Screenshot%202019-10-14%2016.13.49.png?dl=0 On Mon, Oct 14, 2019 at 1:39 AM Henri Sivonen wrote: > On Sat, Oct 12, 2019 at 12:29 PM Andre Natal wrote: > > We tried to capture everything here [1], so please if you don't see your > > question addressed in this document, just give us a shout either here in > > the thread or directly. > ... > > [1] > > > https://docs.google.com/document/d/1BE90kgbwE37fWoQ8vqnsQ3YMiJCKJSvqQwa463yCN1Y/edit?ts=5da0f63f# > > Thanks. It doesn't address the question of what the UI in Firefox is > like. Following the links for experimenting with the UI on one's own > leads to https://mdn.github.io/web-speech-api/speech-color-changer/ , > which doesn't work in Nightly even with prefs flipped. > > (Trying that example in Chrome shows that Chrome presents the > permission prompt as a matter of sharing the microphone with > mdn.github.io as if this was WebRTC, which suggests that mdn.github.io > decides where the audio goes. Chrome does not surface that, if I > understand correctly how this API works in Chrome, the audio is > instead sent to a destination of Chrome's choosing and not to a > destination of mdn.github.io's choosing. The example didn't work for > me in Safari.) > > -- > Henri Sivonen > hsivo...@mozilla.com > -- -- Thanks, Andre ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
