Contact emails [email protected]<mailto:[email protected]>
Explainer https://github.com/riju/backgroundBlur/blob/main/explainer.md#background-segmentation-mask-api Specification https://github.com/w3c/mediacapture-extensions/pull/142 Summary The Background Segmentation/Mask API allows web developers to use the native platform's API for camera, leveraging on-device AI models for background segmentation. Today, on Windows/Apple, this can leverage NPU (Neural Processing Units) and bring power and performance gains. This extends our previously proposed Background Blur API [1] with some implementation difference [2] – * BG Blur preprocesses video frames and replaces the original video frames with ones with background blurred thus the original video frames become unavailable from a web application point of view (until BG Blur is disabled) * BG MASK retains the original frames intact, does segmentation and provides mask frames in addition to the original video frames thus web applications receive both the original frames and mask frames in the same video frame stream. In many cases, it might be important to have access to the original camera feed [3]. Green Screen, Background Replacement (with an Image) and many other use cases can be enabled [4] using this feature via Canvas and WebGPU/GL pipeline. There is still an ongoing discussion about how Mask data is best provided [5]. [1] https://developer.chrome.com/blog/background-blur/ [2] https://github.com/riju/backgroundBlur/blob/main/explainer.md#blur-vs-mask [3] https://github.com/w3c/mediacapture-extensions/issues/118#issuecomment-1772204182 [4] https://github.com/riju/backgroundBlur/blob/main/explainer.md#demo [5] https://github.com/riju/backgroundBlur/blob/main/explainer.md#mask-data----videoframe--imagebitmap--imagedata Blink component Blink>ImageCapture<https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EImageCapture> Motivation A vast majority of communication these days happens on our client devices and features like Blur, Background Replacement, Green Screen have become table stakes. This Background Segmentation-Mask API gives developers a choice to use the native platform's API and leverage the on-device AI models already present in the platform. Initial public proposal https://github.com/w3c/mediacapture-extensions/pull/142 TAG review None TAG review status Pending Risks Interoperability and Compatibility None Gecko: No signal. https://github.com/mozilla/standards-positions/issues/1041 WebKit: No signal. https://github.com/WebKit/standards-positions/issues/364 Web developers: No signals Other signals: WebView application risks Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications? None Debuggability None Is this feature fully tested by web-platform-tests<https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md>? No Flag name on chrome://flags None Finch feature name None Non-finch justification None Requires code in //chrome? False Estimated milestones No milestones specified Link to entry on the Chrome Platform Status https://chromestatus.com/feature/5187283610501120 This intent message was generated by Chrome Platform Status<https://chromestatus.com/>. -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/PH0PR11MB5191EE3FA99AFE2BF5DE907AF0D62%40PH0PR11MB5191.namprd11.prod.outlook.com.
