Hi Ningxin, I am part of a group of Security & Privacy reviewers within the Chrome team who recently reviewed your explainer and design doc. We have some follow-up questions for you, which I will send to your list of email contacts off-thread. We will report back on any findings/recommendations on this thread after our review is complete.
Thank you, Kaustubha Govind On Tuesday, November 30, 2021 at 9:11:17 PM UTC-5 ningxin hu wrote: > Contact emails > > ningxin...@intel.com, wc...@microsoft.com, rafael.cint...@microsoft.com > > > Explainer > > https://github.com/webmachinelearning/webnn/blob/main/explainer.md > > > Specification > > https://www.w3.org/TR/webnn/ > > > Design docs > > > https://docs.google.com/document/d/1KDVuz38fx3SpLVdE8FzCCqASjFfOBXcJWj124jP7ZZ4/edit?usp=sharing > > > Summary > > Construct and execute computational graphs that are at the heart of neural > networks, taking advantage of on-device hardware acceleration. Currently, > JavaScript frameworks must repurpose web APIs like WebGL, WebGPU and > WebAssembly for the individual compute operations when executing > computational graphs. The WebNN API allows JavaScript frameworks to access > the native operating system machine learning APIs and the hardware > accelerators underneath for the best performance and reliability of results. > > > Blink component > > Blink>WebML > <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EWebML> > > > Motivation > > Deep Learning, a subfield of Machine Learning (ML), with its various > neural network architectures enables new user experiences for web > applications that range from improved video conferencing to > accessibility-improving features, with potential for improved privacy over > cloud-based solutions. Key neural network-based use cases that apply to a > wide range of web applications include e.g., face detection, semantic > segmentation, style transfer, super resolution, image captioning, machine > translation, and noise suppression. While some of these use cases can be > implemented in a constrained manner with existing Web APIs (e.g. WebGL, > WebGPU and WebAssembly), the lack of access to platform capabilities > beneficial for ML such as dedicated ML hardware accelerators constraints > the scope of experiences and leads to inefficient implementations on modern > hardware. This disadvantages the web platform in comparison to native > platforms. > > > Initial public proposal > > https://discourse.wicg.io/t/api-set-for-machine-learning-on-the-web/2491/10 > > > TAG review > > https://github.com/w3ctag/design-reviews/issues/570 > > > TAG review status > > Issues addressed > > > Risks > > > Interoperability and Compatibility > > All major browser vendors participated in the Community Group [1] that > successfully completed the incubation of this API and graduated it to the > Working Group [2] for standardization. > > > > A well-attended W3C workshop [3] that brought together browser vendors and > machine learning practitioners reviewed the API and recommended it for > standardization. > > > > Public support from Edge and Microsoft [4] with key contributors involved > in the work. > > > > Significant interest and positive feedback from machine learning JS > frameworks in using WebNN when it is available, see e.g. > TensorFlow.js/TensorFlow-Lite Web [5] and OpenCV.js [6]. > > > > An interoperability risk is to define the core operators that can be > implemented cross-platforms and support the breadth of ML JS frameworks, > however the members of the working group have experience defining and > testing such interoperable machine learning operator set. > > > > [1]: https://www.w3.org/groups/cg/webmachinelearning > > [2]: https://www.w3.org/groups/wg/webmachinelearning > > [3]: https://www.w3.org/2020/06/machine-learning-workshop/ > > [4]: > https://www.w3.org/blog/2021/04/w3c-launches-the-web-machine-learning-working-group/ > > [5]: https://github.com/tensorflow/sig-tfjs/pull/2 > > [6]: https://github.com/opencv/opencv/pull/20406 > > > > Gecko: No signal > > WebKit: No signal > > Web developers: Positive (https://github.com/tensorflow/sig-tfjs/pull/2) > The RFC of WebNN Delegate for TensorFlow Lite Web got good feedback and > approval. > > > Ergonomics > > The WebNN graph building (MLGraphBuilder.build [1]) and execution > (MLGraph.compute [2]) APIs are synchronous. This is required to implement a > backend for Wasm-based ML frameworks, such as ONNX Runtime Execution > Provider [3], TensorFlow Lite Delegate [4] and OpenCV.js DNN backend [5]. > These frameworks are written in C++ and expect calling sync APIs in its > backend implementation. To avoid blocking the main thread, the good > practice is to use these async APIs in a worker context. WebML working > group is collecting feedback about whether to restrict the sync APIs to > only exist in workers [6] and support async APIs [7]. > > > > [1]: https://www.w3.org/TR/webnn/#dom-mlgraphbuilder-build > > [2]: https://www.w3.org/TR/webnn/#dom-mlgraph-compute > > [3]: > https://onnxruntime.ai/docs/execution-providers/add-execution-provider.html > > > [4]: https://www.tensorflow.org/lite/performance/implementing_delegate > > [5]: https://github.com/opencv/opencv/pull/20406 > > [6]: https://github.com/webmachinelearning/webnn/issues/229 > > [7]: https://github.com/webmachinelearning/webnn/issues/230 > > > Activation > > Developers could use webnn-polyfill [1] immediately. The machine learning > JS frameworks with WebNN backend would make it easier to use. > > > > [1]: https://github.com/webmachinelearning/webnn-polyfill > > > > *Ethical* > > The WebNN API spec incorporates ethical considerations. The group is > working on a complementary document [1] that outlines ethical issues > associated with using Machine Learning on the Web and discusses how > implementers and web developers can mitigate associated risks. > > > > [1]: https://webmachinelearning.github.io/ethical-webmachinelearning/ > > > Debuggability > > Warnings and errors are exposed via dev tools. Specialized tools for > debugging are TBD. > > > Is this feature fully tested by web-platform-tests > <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md> > ? > > Not yet. Working in progress, e.g., WebNN IDL test merged in wpt [1] > > > > [1]: https://github.com/web-platform-tests/wpt/pull/31201 > > > Flag name > > --enable-webnn > > > Requires code in //chrome? > > False > > > Tracking bug > > https://bugs.chromium.org/p/chromium/issues/detail?id=1273291 > > > Estimated milestones > > TBD > > > Link to entry on the Chrome Platform Status > > https://www.chromestatus.com/feature/5738583487938560 > > > > This intent message was generated by Chrome Platform Status > <https://www.chromestatus.com/>. > > > > > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/57b1dfcb-f2e4-47f9-b4bf-779eccc87652n%40chromium.org.
