> > Have you written web platform tests for it? Have a link? I've added a few so far--WPT coverage for the Web Speech API in general is pretty basic at the moment. I'm planning on adding more comprehensive coverage. Here are the ones relevant to this proposal: https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/speech-api/SpeechRecognition-installOnDeviceSpeechRecognitiRecognition.https.html <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/speech-api/SpeechRecognition-installOnDeviceSpeechRecognition.https.html> https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/speech-api/SpeechRecognition-onDeviceWebSpeechAvailable.https.html
Should we update the Privacy considerations in the spec to describe these > risks? That's a good idea! There's a section in the spec that requires user consent to download language packs to mitigate these risks, but I'll update this section to explicitly mention the fingerprinting risks. this needs an async API, likely with a streams design. The new methods added to the SpeechRecognition interface (onDeviceSpeechRecognitionAvailable and installOnDeviceSpeechRecognition) will return promises--I'll update the spec to reflect this. Regarding dropping the webkit prefix and replacing the non-prefixed version with a modern API design with promises, this probably wouldn't be feasible due to interoperability and backwards compatibility issues. While Firefox doesn't officially support the speech recognition section of the Web Speech API, it has a unprefixed implementation behind a flag and most of the guides on how to use the Web Speech API do something like window.SpeechRecognition || window.webkitSpeechRecognition; (Examples from developer.mozilla.org <https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API/Using_the_Web_Speech_API> , codeburst.io <https://codeburst.io/html5-speech-recognition-api-670846a50e92>, dev.to <https://dev.to/nixx/building-a-real-time-speech-to-text-web-app-with-web-speech-api-4mc6>) and there are 17.8K instances of this kind of usage on Github <https://github.com/search?q=%22window.webkitSpeechRecognition+%7C%7C+window.SpeechRecognition%22+OR+%22window.SpeechRecognition+%7C%7C+window.webkitSpeechRecognition%22&type=code> alone. However, I think it's worth considering whether to create a new, modernized version of the API under a different name. I've filed a Github issue to continue this discussion with the Audio Working Group: https://github.com/WebAudio/web-speech-api/issues/130 Privacy Design Doc > I don't think that's a link.. Oops sorry, here's the actual link: https://docs.google.com/document/d/12NHh8eVGmuqRuG9H4na4jGwCPD5UqIbC0bW_CJQi2Sc You'll probably have to request access to view the doc. I also wonder if this should have a TAG review, especially given the > privacy/fingerprinting implications of websites being able to query which > on-device models are available. As a TAG member, I think a TAG review would probably result in useful > feedback for this API. Please do send one. I've sent a TAG design review request for this here: https://github.com/w3ctag/design-reviews/issues/1038 Thanks for all of the input so far! Evan On Wed, Jan 8, 2025 at 8:34 AM Jeffrey Yasskin <jyass...@chromium.org> wrote: > On Tue, Jan 7, 2025 at 10:34 AM 'Daniel Clark' via blink-dev < > blink-dev@chromium.org> wrote: > >> Adding to Yoav’s feedback about the spec: >> >> - It’s implied that installOnDeviceSpeechRecognition() happens >> synchronously. Making this a blocking call seems problematic since it >> could >> involve a fetch and a download. I’d expect it to return a Promise ( >> https://www.w3.org/TR/design-principles/#promises). And >> onDeviceWebSpeechAvailable should probably also be async since it could >> involve reading data from disk. >> - The SpeechRecognitionMode "ondevice-only" value is only defined by >> a comment in the IDL stating that it “Returns an error if on-device speech >> recognition is not available”. What specifically returns an error? >> SpeechRecognition.start() doesn’t return any value, and in other error >> conditions the behavior is to fire SpeechRecognitionErrorEvent. Also, what >> should the behavior be if SpeechRecognitionMode is changed after start() >> has already been called? >> >> >> >> I also wonder if this should have a TAG review, especially given the >> privacy/fingerprinting implications of websites being able to query which >> on-device models are available. >> > > As a TAG member, I think a TAG review would probably result in useful > feedback for this API. Please do send one. > > Thanks, > Jeffrey > > >> -- Dan Clark >> >> >> >> *From:* Yoav Weiss (@Shopify) <yoavwe...@chromium.org> >> *Sent:* Tuesday, January 7, 2025 12:29 AM >> *To:* Chromestatus <ad...@cr-status.appspotmail.com> >> *Cc:* blink-dev@chromium.org; ev...@google.com >> *Subject:* [EXTERNAL] Re: [blink-dev] Intent to Ship: On-device Web >> Speech API >> >> >> >> >> >> >> >> On Tue, Jan 7, 2025 at 2:10 AM Chromestatus < >> ad...@cr-status.appspotmail.com> wrote: >> >> Contact emails >> >> ev...@google.com >> Explainer >> >> https://github.com/WebAudio/web-speech-api/pull/122 >> >> >> >> An actual explainer with usage examples would've been useful. >> >> Also, the spec is not very detailed: >> >> * It seems to be triggering resource downloads, but Fetch >> <https://fetch.spec.whatwg.org/> integration is not specified. >> >> * Are the resources downloaded partitioned per top-level site? What >> should typical download sizes be? >> >> >> >> >> Specification >> >> https://webaudio.github.io/web-speech-api >> Summary >> >> This feature adds on-device speech recognition support to the Web Speech >> API, allowing websites to ensure that neither audio nor transcribed speech >> are sent to a third-party service for processing. Websites can query the >> availability of on-device speech recognition for specific languages, prompt >> users to install the necessary resources for on-device speech recognition, >> and choose between on-device or cloud-based speech recognition as needed. >> >> >> Blink component >> >> Blink>Speech >> <https://issues.chromium.org/issues?q=customfield1222907:%22Blink%3ESpeech%22> >> Search tags >> >> speech <http://features#tags:speech>, recognition >> <http://features#tags:recognition>, local <http://features#tags:local>, >> offline <http://features#tags:offline>, on-device >> <http://features#tags:on-device> >> TAG review >> >> None >> TAG review status >> >> Pending >> Risks >> >> >> Interoperability and Compatibility >> >> None >> >> >> >> *Gecko*: Positive Discussed at TPAC 2024 with representatives from >> Mozilla including Paul Adenot >> >> *WebKit*: Positive Discussed at TPAC 2024 with representatives from >> Apple including Eric Carlson. >> >> >> >> Links to the minutes would be helpful. Filing official positions would be >> even better. >> >> >> >> >> >> *Web developers*: Positive Commonly requested feature. Examples: >> https://webwewant.fyi/wants/55/ >> https://github.com/WebAudio/web-speech-api/issues/108 >> https://stackoverflow.com/questions/49473369/offline-speech-recognition-in-browser >> https://www.reddit.com/r/html5/comments/8jtv3u/offline_voice_recognition_without_the_webspeech/ >> >> *Other signals*: >> WebView application risks >> >> *Does this intent deprecate or change behavior of existing APIs, such >> that it has potentially high risk for Android WebView-based applications?* >> >> None >> >> >> Debuggability >> >> None >> >> >> Will this feature be supported on all six Blink platforms (Windows, Mac, >> Linux, ChromeOS, Android, and Android WebView)? >> >> No >> >> Initially supported on Windows, Mac, and Linux with ChromeOS support to >> follow. >> >> >> Is this feature fully tested by web-platform-tests >> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> >> ? >> >> No >> >> >> >> Why not? Is it tested otherwise? >> >> >> Flag name on about://flags >> >> None >> Finch feature name >> >> InstallOnDeviceSpeechRecognition,OnDeviceWebSpeechAvailable,OnDeviceWebSpeech >> >> Requires code in //chrome? >> >> False >> Estimated milestones >> >> Shipping on desktop >> >> 135 >> >> >> Anticipated spec changes >> >> *Open questions about a feature may be a source of future web compat or >> interop issues. Please list open issues (e.g. links to known github issues >> in the project for the feature specification) whose resolution may >> introduce web compat/interop risk (e.g., changing to naming or structure of >> the API in a non-backward-compatible way).* >> >> https://github.com/WebAudio/web-speech-api/pull/122 >> Link to entry on the Chrome Platform Status >> >> https://chromestatus.com/feature/6090916291674112?gate=4683906480340992 >> >> This intent message was generated by Chrome Platform Status >> <https://chromestatus.com/>. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "blink-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to blink-dev+unsubscr...@chromium.org. >> To view this discussion visit >> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/677c7f0e.2b0a0220.2e82a8.01f6.GAE%40google.com >> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/677c7f0e.2b0a0220.2e82a8.01f6.GAE%40google.com?utm_medium=email&utm_source=footer> >> . >> >> -- >> You received this message because you are subscribed to the Google Groups >> "blink-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to blink-dev+unsubscr...@chromium.org. >> To view this discussion visit >> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOmohSJFcq7nCbx372u8Qas0%3DUWbCUY9b37ak6fAN8CwGfFVcA%40mail.gmail.com >> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOmohSJFcq7nCbx372u8Qas0%3DUWbCUY9b37ak6fAN8CwGfFVcA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> -- >> You received this message because you are subscribed to the Google Groups >> "blink-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to blink-dev+unsubscr...@chromium.org. >> To view this discussion visit >> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CH4PR00MB23299535B21C262B4841395FC5112%40CH4PR00MB2329.namprd00.prod.outlook.com >> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CH4PR00MB23299535B21C262B4841395FC5112%40CH4PR00MB2329.namprd00.prod.outlook.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOVsCZm2%3DNhZrkv6U4DkC9fzYP76%3DBOO6vy95_Jq6qPSzZh4bA%40mail.gmail.com.
