Thanks Kaustubha and Chrome team for looking into it. I am excited to work with you on the security and privacy review. We'll discuss your follow-up questions within the W3C WebML WG and get back to you.
Regards, -ningxin On Tuesday, January 11, 2022 at 10:38:48 PM UTC+8 kaust...@chromium.org wrote: > Hi Ningxin, > > I am part of a group of Security & Privacy reviewers within the Chrome > team who recently reviewed your explainer and design doc. We have some > follow-up questions for you, which I will send to your list of email > contacts off-thread. We will report back on any findings/recommendations on > this thread after our review is complete. > > Thank you, > Kaustubha Govind > > > On Tuesday, November 30, 2021 at 9:11:17 PM UTC-5 ningxin hu wrote: > >> Contact emails >> >> ningx...@intel.com, wc...@microsoft.com, rafael....@microsoft.com >> >> >> Explainer >> >> https://github.com/webmachinelearning/webnn/blob/main/explainer.md >> >> >> Specification >> >> https://www.w3.org/TR/webnn/ >> >> >> Design docs >> >> >> https://docs.google.com/document/d/1KDVuz38fx3SpLVdE8FzCCqASjFfOBXcJWj124jP7ZZ4/edit?usp=sharing >> >> >> Summary >> >> Construct and execute computational graphs that are at the heart of >> neural networks, taking advantage of on-device hardware acceleration. >> Currently, JavaScript frameworks must repurpose web APIs like WebGL, WebGPU >> and WebAssembly for the individual compute operations when executing >> computational graphs. The WebNN API allows JavaScript frameworks to access >> the native operating system machine learning APIs and the hardware >> accelerators underneath for the best performance and reliability of results. >> >> >> Blink component >> >> Blink>WebML >> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EWebML> >> >> >> Motivation >> >> Deep Learning, a subfield of Machine Learning (ML), with its various >> neural network architectures enables new user experiences for web >> applications that range from improved video conferencing to >> accessibility-improving features, with potential for improved privacy over >> cloud-based solutions. Key neural network-based use cases that apply to a >> wide range of web applications include e.g., face detection, semantic >> segmentation, style transfer, super resolution, image captioning, machine >> translation, and noise suppression. While some of these use cases can be >> implemented in a constrained manner with existing Web APIs (e.g. WebGL, >> WebGPU and WebAssembly), the lack of access to platform capabilities >> beneficial for ML such as dedicated ML hardware accelerators constraints >> the scope of experiences and leads to inefficient implementations on modern >> hardware. This disadvantages the web platform in comparison to native >> platforms. >> >> >> Initial public proposal >> >> >> https://discourse.wicg.io/t/api-set-for-machine-learning-on-the-web/2491/10 >> >> >> TAG review >> >> https://github.com/w3ctag/design-reviews/issues/570 >> >> >> TAG review status >> >> Issues addressed >> >> >> Risks >> >> >> Interoperability and Compatibility >> >> All major browser vendors participated in the Community Group [1] that >> successfully completed the incubation of this API and graduated it to the >> Working Group [2] for standardization. >> >> >> >> A well-attended W3C workshop [3] that brought together browser vendors >> and machine learning practitioners reviewed the API and recommended it for >> standardization. >> >> >> >> Public support from Edge and Microsoft [4] with key contributors involved >> in the work. >> >> >> >> Significant interest and positive feedback from machine learning JS >> frameworks in using WebNN when it is available, see e.g. >> TensorFlow.js/TensorFlow-Lite Web [5] and OpenCV.js [6]. >> >> >> >> An interoperability risk is to define the core operators that can be >> implemented cross-platforms and support the breadth of ML JS frameworks, >> however the members of the working group have experience defining and >> testing such interoperable machine learning operator set. >> >> >> >> [1]: https://www.w3.org/groups/cg/webmachinelearning >> >> [2]: https://www.w3.org/groups/wg/webmachinelearning >> >> [3]: https://www.w3.org/2020/06/machine-learning-workshop/ >> >> [4]: >> https://www.w3.org/blog/2021/04/w3c-launches-the-web-machine-learning-working-group/ >> >> [5]: https://github.com/tensorflow/sig-tfjs/pull/2 >> >> [6]: https://github.com/opencv/opencv/pull/20406 >> >> >> >> Gecko: No signal >> >> WebKit: No signal >> >> Web developers: Positive (https://github.com/tensorflow/sig-tfjs/pull/2) >> The RFC of WebNN Delegate for TensorFlow Lite Web got good feedback and >> approval. >> >> >> Ergonomics >> >> The WebNN graph building (MLGraphBuilder.build [1]) and execution >> (MLGraph.compute [2]) APIs are synchronous. This is required to implement a >> backend for Wasm-based ML frameworks, such as ONNX Runtime Execution >> Provider [3], TensorFlow Lite Delegate [4] and OpenCV.js DNN backend [5]. >> These frameworks are written in C++ and expect calling sync APIs in its >> backend implementation. To avoid blocking the main thread, the good >> practice is to use these async APIs in a worker context. WebML working >> group is collecting feedback about whether to restrict the sync APIs to >> only exist in workers [6] and support async APIs [7]. >> >> >> >> [1]: https://www.w3.org/TR/webnn/#dom-mlgraphbuilder-build >> >> [2]: https://www.w3.org/TR/webnn/#dom-mlgraph-compute >> >> [3]: >> https://onnxruntime.ai/docs/execution-providers/add-execution-provider.html >> >> >> [4]: https://www.tensorflow.org/lite/performance/implementing_delegate >> >> [5]: https://github.com/opencv/opencv/pull/20406 >> >> [6]: https://github.com/webmachinelearning/webnn/issues/229 >> >> [7]: https://github.com/webmachinelearning/webnn/issues/230 >> >> >> Activation >> >> Developers could use webnn-polyfill [1] immediately. The machine learning >> JS frameworks with WebNN backend would make it easier to use. >> >> >> >> [1]: https://github.com/webmachinelearning/webnn-polyfill >> >> >> >> *Ethical* >> >> The WebNN API spec incorporates ethical considerations. The group is >> working on a complementary document [1] that outlines ethical issues >> associated with using Machine Learning on the Web and discusses how >> implementers and web developers can mitigate associated risks. >> >> >> >> [1]: https://webmachinelearning.github.io/ethical-webmachinelearning/ >> >> >> Debuggability >> >> Warnings and errors are exposed via dev tools. Specialized tools for >> debugging are TBD. >> >> >> Is this feature fully tested by web-platform-tests >> <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md> >> ? >> >> Not yet. Working in progress, e.g., WebNN IDL test merged in wpt [1] >> >> >> >> [1]: https://github.com/web-platform-tests/wpt/pull/31201 >> >> >> Flag name >> >> --enable-webnn >> >> >> Requires code in //chrome? >> >> False >> >> >> Tracking bug >> >> https://bugs.chromium.org/p/chromium/issues/detail?id=1273291 >> >> >> Estimated milestones >> >> TBD >> >> >> Link to entry on the Chrome Platform Status >> >> https://www.chromestatus.com/feature/5738583487938560 >> >> >> >> This intent message was generated by Chrome Platform Status >> <https://www.chromestatus.com/>. >> >> >> >> >> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/4334dba2-c3df-4458-871e-5b9eb4ba6de0n%40chromium.org.
