Thank you Evan. Given the samples and github hits you've shared, I agree that web compat will constrain us from making breaking changes to the APi when we unprefix. That's a shame, but is a known reason why we long ago gave up on prefixes as a safe way to do experimental API development. So are you OK with adding unprefixing to this intent (or if you prefer, a new one that this is blocked on)? IMHO the webkit-prefixed entrypoint can just be an alias, but we should UseCounter it independently - with all the examples you've shared I'm hoping maybe we can even delete the prefixed API name relatively easily? But we should keep that as separate future work.
The TAG or others may argue that we should block on-device support on modernizing the API elsewhere (eg. exposing under a brand new name, or adding duplicate methods for the promise versions), but personally I think that goes too far so I won't withhold my personal LGTM on such an argument. Other API owners may feel differently. It's great to see Mozilla officially positive <https://github.com/mozilla/standards-positions/issues/1157> on this. There's no TAG or WebKit feedback yet, but let's keep our ears open. From my perspective this is currently only blocked on unprefixing and landing the spec updates you mentioned. On Thu, Jan 9, 2025 at 5:55 PM 'Evan Liu' via blink-dev < blink-dev@chromium.org> wrote: > Have you written web platform tests for it? Have a link? > > I've added a few so far--WPT coverage for the Web Speech API in general is > pretty basic at the moment. I'm planning on adding more > comprehensive coverage. Here are the ones relevant to this proposal: > > https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/speech-api/SpeechRecognition-installOnDeviceSpeechRecognitiRecognition.https.html > <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/speech-api/SpeechRecognition-installOnDeviceSpeechRecognition.https.html> > > https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/speech-api/SpeechRecognition-onDeviceWebSpeechAvailable.https.html > > Should we update the Privacy considerations in the spec to describe these >> risks? > > That's a good idea! There's a section in the spec that requires user > consent to download language packs to mitigate these risks, but I'll update > this section to explicitly mention the fingerprinting risks. > > this needs an async API, likely with a streams design. > > The new methods added to the SpeechRecognition interface > (onDeviceSpeechRecognitionAvailable and installOnDeviceSpeechRecognition) > will return promises--I'll update the spec to reflect this. Regarding > dropping the webkit prefix and replacing the non-prefixed version with a > modern API design with promises, this probably wouldn't be feasible due to > interoperability and backwards compatibility issues. While Firefox doesn't > officially support the speech recognition section of the Web Speech API, it > has a unprefixed implementation behind a flag and most of the guides on how > to use the Web Speech API do something like window.SpeechRecognition || > window.webkitSpeechRecognition; (Examples from developer.mozilla.org > <https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API/Using_the_Web_Speech_API> > , codeburst.io > <https://codeburst.io/html5-speech-recognition-api-670846a50e92>, dev.to > <https://dev.to/nixx/building-a-real-time-speech-to-text-web-app-with-web-speech-api-4mc6>) > and there are 17.8K instances of this kind of usage on Github > <https://github.com/search?q=%22window.webkitSpeechRecognition+%7C%7C+window.SpeechRecognition%22+OR+%22window.SpeechRecognition+%7C%7C+window.webkitSpeechRecognition%22&type=code> > alone. However, I think it's worth considering whether to create a new, > modernized version of the API under a different name. I've filed a Github > issue to continue this discussion with the Audio Working Group: > https://github.com/WebAudio/web-speech-api/issues/130 > > Privacy Design Doc >> > I don't think that's a link.. > > Oops sorry, here's the actual link: > https://docs.google.com/document/d/12NHh8eVGmuqRuG9H4na4jGwCPD5UqIbC0bW_CJQi2Sc > You'll probably have to request access to view the doc. > > I also wonder if this should have a TAG review, especially given the >> privacy/fingerprinting implications of websites being able to query which >> on-device models are available. > > As a TAG member, I think a TAG review would probably result in useful >> feedback for this API. Please do send one. > > I've sent a TAG design review request for this here: > https://github.com/w3ctag/design-reviews/issues/1038 > > Thanks for all of the input so far! > Evan > > > > On Wed, Jan 8, 2025 at 8:34 AM Jeffrey Yasskin <jyass...@chromium.org> > wrote: > >> On Tue, Jan 7, 2025 at 10:34 AM 'Daniel Clark' via blink-dev < >> blink-dev@chromium.org> wrote: >> >>> Adding to Yoav’s feedback about the spec: >>> >>> - It’s implied that installOnDeviceSpeechRecognition() happens >>> synchronously. Making this a blocking call seems problematic since it >>> could >>> involve a fetch and a download. I’d expect it to return a Promise ( >>> https://www.w3.org/TR/design-principles/#promises). And >>> onDeviceWebSpeechAvailable should probably also be async since it could >>> involve reading data from disk. >>> - The SpeechRecognitionMode "ondevice-only" value is only defined by >>> a comment in the IDL stating that it “Returns an error if on-device >>> speech >>> recognition is not available”. What specifically returns an error? >>> SpeechRecognition.start() doesn’t return any value, and in other error >>> conditions the behavior is to fire SpeechRecognitionErrorEvent. Also, >>> what >>> should the behavior be if SpeechRecognitionMode is changed after start() >>> has already been called? >>> >>> >>> >>> I also wonder if this should have a TAG review, especially given the >>> privacy/fingerprinting implications of websites being able to query which >>> on-device models are available. >>> >> >> As a TAG member, I think a TAG review would probably result in useful >> feedback for this API. Please do send one. >> >> Thanks, >> Jeffrey >> >> >>> -- Dan Clark >>> >>> >>> >>> *From:* Yoav Weiss (@Shopify) <yoavwe...@chromium.org> >>> *Sent:* Tuesday, January 7, 2025 12:29 AM >>> *To:* Chromestatus <ad...@cr-status.appspotmail.com> >>> *Cc:* blink-dev@chromium.org; ev...@google.com >>> *Subject:* [EXTERNAL] Re: [blink-dev] Intent to Ship: On-device Web >>> Speech API >>> >>> >>> >>> >>> >>> >>> >>> On Tue, Jan 7, 2025 at 2:10 AM Chromestatus < >>> ad...@cr-status.appspotmail.com> wrote: >>> >>> Contact emails >>> >>> ev...@google.com >>> Explainer >>> >>> https://github.com/WebAudio/web-speech-api/pull/122 >>> >>> >>> >>> An actual explainer with usage examples would've been useful. >>> >>> Also, the spec is not very detailed: >>> >>> * It seems to be triggering resource downloads, but Fetch >>> <https://fetch.spec.whatwg.org/> integration is not specified. >>> >>> * Are the resources downloaded partitioned per top-level site? What >>> should typical download sizes be? >>> >>> >>> >>> >>> Specification >>> >>> https://webaudio.github.io/web-speech-api >>> Summary >>> >>> This feature adds on-device speech recognition support to the Web Speech >>> API, allowing websites to ensure that neither audio nor transcribed speech >>> are sent to a third-party service for processing. Websites can query the >>> availability of on-device speech recognition for specific languages, prompt >>> users to install the necessary resources for on-device speech recognition, >>> and choose between on-device or cloud-based speech recognition as needed. >>> >>> >>> Blink component >>> >>> Blink>Speech >>> <https://issues.chromium.org/issues?q=customfield1222907:%22Blink%3ESpeech%22> >>> Search tags >>> >>> speech <http://features#tags:speech>, recognition >>> <http://features#tags:recognition>, local <http://features#tags:local>, >>> offline <http://features#tags:offline>, on-device >>> <http://features#tags:on-device> >>> TAG review >>> >>> None >>> TAG review status >>> >>> Pending >>> Risks >>> >>> >>> Interoperability and Compatibility >>> >>> None >>> >>> >>> >>> *Gecko*: Positive Discussed at TPAC 2024 with representatives from >>> Mozilla including Paul Adenot >>> >>> *WebKit*: Positive Discussed at TPAC 2024 with representatives from >>> Apple including Eric Carlson. >>> >>> >>> >>> Links to the minutes would be helpful. Filing official positions would >>> be even better. >>> >>> >>> >>> >>> >>> *Web developers*: Positive Commonly requested feature. Examples: >>> https://webwewant.fyi/wants/55/ >>> https://github.com/WebAudio/web-speech-api/issues/108 >>> https://stackoverflow.com/questions/49473369/offline-speech-recognition-in-browser >>> https://www.reddit.com/r/html5/comments/8jtv3u/offline_voice_recognition_without_the_webspeech/ >>> >>> *Other signals*: >>> WebView application risks >>> >>> *Does this intent deprecate or change behavior of existing APIs, such >>> that it has potentially high risk for Android WebView-based applications?* >>> >>> None >>> >>> >>> Debuggability >>> >>> None >>> >>> >>> Will this feature be supported on all six Blink platforms (Windows, Mac, >>> Linux, ChromeOS, Android, and Android WebView)? >>> >>> No >>> >>> Initially supported on Windows, Mac, and Linux with ChromeOS support to >>> follow. >>> >>> >>> Is this feature fully tested by web-platform-tests >>> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> >>> ? >>> >>> No >>> >>> >>> >>> Why not? Is it tested otherwise? >>> >>> >>> Flag name on about://flags >>> >>> None >>> Finch feature name >>> >>> InstallOnDeviceSpeechRecognition,OnDeviceWebSpeechAvailable,OnDeviceWebSpeech >>> >>> Requires code in //chrome? >>> >>> False >>> Estimated milestones >>> >>> Shipping on desktop >>> >>> 135 >>> >>> >>> Anticipated spec changes >>> >>> *Open questions about a feature may be a source of future web compat or >>> interop issues. Please list open issues (e.g. links to known github issues >>> in the project for the feature specification) whose resolution may >>> introduce web compat/interop risk (e.g., changing to naming or structure of >>> the API in a non-backward-compatible way).* >>> >>> https://github.com/WebAudio/web-speech-api/pull/122 >>> Link to entry on the Chrome Platform Status >>> >>> https://chromestatus.com/feature/6090916291674112?gate=4683906480340992 >>> >>> This intent message was generated by Chrome Platform Status >>> <https://chromestatus.com/>. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to blink-dev+unsubscr...@chromium.org. >>> To view this discussion visit >>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/677c7f0e.2b0a0220.2e82a8.01f6.GAE%40google.com >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/677c7f0e.2b0a0220.2e82a8.01f6.GAE%40google.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to blink-dev+unsubscr...@chromium.org. >>> To view this discussion visit >>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOmohSJFcq7nCbx372u8Qas0%3DUWbCUY9b37ak6fAN8CwGfFVcA%40mail.gmail.com >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOmohSJFcq7nCbx372u8Qas0%3DUWbCUY9b37ak6fAN8CwGfFVcA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to blink-dev+unsubscr...@chromium.org. >>> To view this discussion visit >>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CH4PR00MB23299535B21C262B4841395FC5112%40CH4PR00MB2329.namprd00.prod.outlook.com >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CH4PR00MB23299535B21C262B4841395FC5112%40CH4PR00MB2329.namprd00.prod.outlook.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "blink-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to blink-dev+unsubscr...@chromium.org. > To view this discussion visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOVsCZm2%3DNhZrkv6U4DkC9fzYP76%3DBOO6vy95_Jq6qPSzZh4bA%40mail.gmail.com > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOVsCZm2%3DNhZrkv6U4DkC9fzYP76%3DBOO6vy95_Jq6qPSzZh4bA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFUtAY-r4HYT_bU_1eBaER5xyqcqk_vTz2rbL3zT5iBVEs3BFw%40mail.gmail.com.
