Re: [blink-dev] Intent to Implement and Ship: Conversion to RGB in VideoFrame.copyTo()

['Eugene Zemtsov' via blink-dev]

By reading an MDN article that I'll update after the launch.

Even now allocationSize() is supposed to be called before copyTo() anyway,
to figure out the size of the buffer for the output. (see example
<https://developer.mozilla.org/en-US/docs/Web/API/VideoFrame/copyTo#examples>1,
example2
<https://gist.github.com/Djuffin/9e2f98025ead49998524510cfeed8d33#example>)
That's why I don't think that it's an ergonomic burden for developers.

On Tue, May 14, 2024 at 1:41 PM Mike Taylor  wrote:

> Ergonomics-wise, it does not seem intuitive to use a method called
> allocationSize() to feature detect copyTo support, even if the semantics
> are the same. Besides a very careful reading of the spec, how do we expect
> developers to know about it?
> On 5/14/24 4:32 PM, Eugene Zemtsov wrote:
>
> In that discussion Marcos Cáceres asked for a synchronous way to detect if
> format conversion is supported.
> We have a synchronous call allocationSize()
> <https://www.w3.org/TR/webcodecs/#dom-videoframe-allocationsize> that
> throws an unsupported error in cases where format conversion is not
> supported.
> I added a WPT test around it in a recent CL
> <https://chromium-review.googlesource.com/c/chromium/src/+/5530313>.
>
> canCopyTo() might be needed if we have more settings in
> VideoFrameCopyToOptions
> <https://www.w3.org/TR/webcodecs/#dictdef-videoframecopytooptions> and
> might want to know which part exactly is causing an error.
>
> On Tue, May 14, 2024 at 12:25 PM Mike Taylor 
> wrote:
>
>> I see that WebKit raised an API concern around detectability
>> <https://github.com/WebKit/standards-positions/issues/341#issuecomment-2099746809>.
>> In your reply, you said "we should consider...". Has that consideration
>> happened, perhaps as follow-up work? :)
>> On 5/8/24 4:05 PM, Eugene Zemtsov wrote:
>>
>> This feature has Privacy, Security, Enterprise, and Debuggability
>> approvals. Webkit gave a positive signal.
>>
>> Any objections or questions from the API owners?
>>
>>
>>
>> On Wed, May 1, 2024 at 4:26 PM Eugene Zemtsov 
>> wrote:
>>
>>> I've filed issues for TAG review and firefox and webkit positions.
>>>
>>> >  Have you looked into other platform APIs that could benefit from
>>> being able to explicitly specify intermediate format hinting and/or
>>> transformation?
>>> I don't think this kind of review can be done for all web APIs by one
>>> person. It's up to spec editors in each area to identify these small
>>> changes that improve dev experience and performance.
>>>
>>> For media related things:
>>> -  VideoFrame is a versatile tool that can be created from
>>> HTMLOrSVGImageElement, HTMLVideoElement, HTMLCanvasElement, ImageBitmap,
>>>  OffscreenCanvas without a copy. In a way this change extends
>>> variety of export formats for all of them.
>>> So lots of things can be exported to RGB bitmaps without a canvas
>>> now.
>>> -  AudioData has a way to be exported into different sampling formats.
>>>
>>>
>>>
>>> On Wed, May 1, 2024 at 9:41 AM Mike Taylor 
>>> wrote:
>>>
>>>> +1. Would you mind also filing gecko and webkit positions? I expect
>>>> them to be positive, given the informal signals you have in the spec PRs
>>>> already - but this also lets them know we're moving ahead with shipping.
>>>> Thanks - Mike
>>>> On 5/1/24 11:51 AM, Alex Russell wrote:
>>>>
>>>> hey Eugene,
>>>>
>>>> This is an exciting an useful addition! Have you looked into other
>>>> platform APIs that could benefit from being able to explicitly specify
>>>> intermediate format hinting and/or transformation? It's a place where (had
>>>> the TAG been consulted) I would have expected to see a larger chain of
>>>> additions to make this work in other areas, or at least a discussion of why
>>>> they aren't (or are?) being pursued. Have you looked into that?
>>>>
>>>> Best,
>>>>
>>>> Alex
>>>>
>>>> On Tuesday, April 30, 2024 at 3:51:01 PM UTC-7 Eugene Zemtsov wrote:
>>>>
>>>>> > Can you please request reviews for privacy, security, enterprise,
>>>>> etc in the chromestatus entry?
>>>>>
>>>>> Done
>>>>>
>>>>> On Mon, Apr 29, 2024 at 7:44 AM Mike Taylor 
>>>>> wrote:
>>>>>
>>>>>> Can y

Re: [blink-dev] Intent to Implement and Ship: Conversion to RGB in VideoFrame.copyTo()

['Eugene Zemtsov' via blink-dev]

In that discussion Marcos Cáceres asked for a synchronous way to detect if
format conversion is supported.
We have a synchronous call allocationSize()
<https://www.w3.org/TR/webcodecs/#dom-videoframe-allocationsize> that
throws an unsupported error in cases where format conversion is not
supported.
I added a WPT test around it in a recent CL
<https://chromium-review.googlesource.com/c/chromium/src/+/5530313>.

canCopyTo() might be needed if we have more settings in
VideoFrameCopyToOptions
<https://www.w3.org/TR/webcodecs/#dictdef-videoframecopytooptions> and
might want to know which part exactly is causing an error.

On Tue, May 14, 2024 at 12:25 PM Mike Taylor  wrote:

> I see that WebKit raised an API concern around detectability
> <https://github.com/WebKit/standards-positions/issues/341#issuecomment-2099746809>.
> In your reply, you said "we should consider...". Has that consideration
> happened, perhaps as follow-up work? :)
> On 5/8/24 4:05 PM, Eugene Zemtsov wrote:
>
> This feature has Privacy, Security, Enterprise, and Debuggability
> approvals. Webkit gave a positive signal.
>
> Any objections or questions from the API owners?
>
>
>
> On Wed, May 1, 2024 at 4:26 PM Eugene Zemtsov  wrote:
>
>> I've filed issues for TAG review and firefox and webkit positions.
>>
>> >  Have you looked into other platform APIs that could benefit from being
>> able to explicitly specify intermediate format hinting and/or
>> transformation?
>> I don't think this kind of review can be done for all web APIs by one
>> person. It's up to spec editors in each area to identify these small
>> changes that improve dev experience and performance.
>>
>> For media related things:
>> -  VideoFrame is a versatile tool that can be created from
>> HTMLOrSVGImageElement, HTMLVideoElement, HTMLCanvasElement, ImageBitmap,
>>  OffscreenCanvas without a copy. In a way this change extends
>> variety of export formats for all of them.
>> So lots of things can be exported to RGB bitmaps without a canvas
>> now.
>> -  AudioData has a way to be exported into different sampling formats.
>>
>>
>>
>> On Wed, May 1, 2024 at 9:41 AM Mike Taylor 
>> wrote:
>>
>>> +1. Would you mind also filing gecko and webkit positions? I expect them
>>> to be positive, given the informal signals you have in the spec PRs already
>>> - but this also lets them know we're moving ahead with shipping. Thanks -
>>> Mike
>>> On 5/1/24 11:51 AM, Alex Russell wrote:
>>>
>>> hey Eugene,
>>>
>>> This is an exciting an useful addition! Have you looked into other
>>> platform APIs that could benefit from being able to explicitly specify
>>> intermediate format hinting and/or transformation? It's a place where (had
>>> the TAG been consulted) I would have expected to see a larger chain of
>>> additions to make this work in other areas, or at least a discussion of why
>>> they aren't (or are?) being pursued. Have you looked into that?
>>>
>>> Best,
>>>
>>> Alex
>>>
>>> On Tuesday, April 30, 2024 at 3:51:01 PM UTC-7 Eugene Zemtsov wrote:
>>>
>>>> > Can you please request reviews for privacy, security, enterprise,
>>>> etc in the chromestatus entry?
>>>>
>>>> Done
>>>>
>>>> On Mon, Apr 29, 2024 at 7:44 AM Mike Taylor 
>>>> wrote:
>>>>
>>>>> Can you please request reviews for privacy, security, enterprise, etc
>>>>> in the chromestatus entry?
>>>>> On 4/25/24 6:19 PM, 'Eugene Zemtsov' via blink-dev wrote:
>>>>>
>>>>> Contact emails ezemt...@google.com
>>>>>
>>>>> Explainer
>>>>> https://gist.github.com/Djuffin/9e2f98025ead49998524510cfeed8d33
>>>>>
>>>>> Specification https://www.w3.org/TR/webcodecs/#dom-videoframe-copyto
>>>>>
>>>>> Summary
>>>>>
>>>>> VideoFrame.copyTo() can convert pixel data to RGB pixel format
>>>>> Converting YUV video frames to RGB is often required for processing them 
>>>>> in
>>>>> libraries like TensorFlow.js and OpenCV.js. Previously the only possible
>>>>> way to achieve this was rendering the frame on a canvas. Specifying
>>>>> VideoFrameCopyToOptions.format and VideoFrameCopyToOptions.colorSpace 
>>>>> makes
>>>>> it possible to convert frames to RGB pixel formats by calling
>>>

Re: [blink-dev] Intent to Implement and Ship: Conversion to RGB in VideoFrame.copyTo()

['Eugene Zemtsov' via blink-dev]

This feature has Privacy, Security, Enterprise, and Debuggability
approvals. Webkit gave a positive signal.

Any objections or questions from the API owners?



On Wed, May 1, 2024 at 4:26 PM Eugene Zemtsov  wrote:

> I've filed issues for TAG review and firefox and webkit positions.
>
> >  Have you looked into other platform APIs that could benefit from being
> able to explicitly specify intermediate format hinting and/or
> transformation?
> I don't think this kind of review can be done for all web APIs by one
> person. It's up to spec editors in each area to identify these small
> changes that improve dev experience and performance.
>
> For media related things:
> -  VideoFrame is a versatile tool that can be created from
> HTMLOrSVGImageElement, HTMLVideoElement, HTMLCanvasElement, ImageBitmap,
>  OffscreenCanvas without a copy. In a way this change extends
> variety of export formats for all of them.
> So lots of things can be exported to RGB bitmaps without a canvas now.
> -  AudioData has a way to be exported into different sampling formats.
>
>
>
> On Wed, May 1, 2024 at 9:41 AM Mike Taylor  wrote:
>
>> +1. Would you mind also filing gecko and webkit positions? I expect them
>> to be positive, given the informal signals you have in the spec PRs already
>> - but this also lets them know we're moving ahead with shipping. Thanks -
>> Mike
>> On 5/1/24 11:51 AM, Alex Russell wrote:
>>
>> hey Eugene,
>>
>> This is an exciting an useful addition! Have you looked into other
>> platform APIs that could benefit from being able to explicitly specify
>> intermediate format hinting and/or transformation? It's a place where (had
>> the TAG been consulted) I would have expected to see a larger chain of
>> additions to make this work in other areas, or at least a discussion of why
>> they aren't (or are?) being pursued. Have you looked into that?
>>
>> Best,
>>
>> Alex
>>
>> On Tuesday, April 30, 2024 at 3:51:01 PM UTC-7 Eugene Zemtsov wrote:
>>
>>> > Can you please request reviews for privacy, security, enterprise, etc
>>> in the chromestatus entry?
>>>
>>> Done
>>>
>>> On Mon, Apr 29, 2024 at 7:44 AM Mike Taylor 
>>> wrote:
>>>
>>>> Can you please request reviews for privacy, security, enterprise, etc
>>>> in the chromestatus entry?
>>>> On 4/25/24 6:19 PM, 'Eugene Zemtsov' via blink-dev wrote:
>>>>
>>>> Contact emails ezemt...@google.com
>>>>
>>>> Explainer
>>>> https://gist.github.com/Djuffin/9e2f98025ead49998524510cfeed8d33
>>>>
>>>> Specification https://www.w3.org/TR/webcodecs/#dom-videoframe-copyto
>>>>
>>>> Summary
>>>>
>>>> VideoFrame.copyTo() can convert pixel data to RGB pixel format
>>>> Converting YUV video frames to RGB is often required for processing them in
>>>> libraries like TensorFlow.js and OpenCV.js. Previously the only possible
>>>> way to achieve this was rendering the frame on a canvas. Specifying
>>>> VideoFrameCopyToOptions.format and VideoFrameCopyToOptions.colorSpace makes
>>>> it possible to convert frames to RGB pixel formats by calling
>>>> VideoFrame.copyTo() without having to use an extra canvas.
>>>>
>>>> Blink component Blink>Media>WebCodecs
>>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EMedia%3EWebCodecs>
>>>>
>>>> Initial public proposal https://github.com/w3c/webcodecs/issues/92
>>>>
>>>> TAG review N/A since the change is minor
>>>>
>>>> Risks
>>>>
>>>>
>>>> Interoperability and Compatibility
>>>>
>>>> None
>>>>
>>>>
>>>> *Gecko*: Positive (
>>>> https://github.com/w3c/webcodecs/pull/754#pullrequestreview-2008590591)
>>>>
>>>> *WebKit*: Positive TPAC 2023. Media working group session.
>>>>
>>>> *Web developers*:
>>>> https://github.com/w3c/webcodecs/issues/92#issuecomment-1594083978
>>>>
>>>> *Other signals*:
>>>>
>>>> WebView application risks
>>>>
>>>> Does this intent deprecate or change behavior of existing APIs, such
>>>> that it has potentially high risk for Android WebView-based applications?
>>>>
>>>> None
>>>>
>>>>
>>>> Debuggability
>>>>
>>>

Re: [blink-dev] Intent to Implement and Ship: Conversion to RGB in VideoFrame.copyTo()

['Eugene Zemtsov' via blink-dev]

I've filed issues for TAG review and firefox and webkit positions.

>  Have you looked into other platform APIs that could benefit from being
able to explicitly specify intermediate format hinting and/or
transformation?
I don't think this kind of review can be done for all web APIs by one
person. It's up to spec editors in each area to identify these small
changes that improve dev experience and performance.

For media related things:
-  VideoFrame is a versatile tool that can be created from
HTMLOrSVGImageElement, HTMLVideoElement, HTMLCanvasElement, ImageBitmap,
 OffscreenCanvas without a copy. In a way this change extends
variety of export formats for all of them.
So lots of things can be exported to RGB bitmaps without a canvas now.
-  AudioData has a way to be exported into different sampling formats.



On Wed, May 1, 2024 at 9:41 AM Mike Taylor  wrote:

> +1. Would you mind also filing gecko and webkit positions? I expect them
> to be positive, given the informal signals you have in the spec PRs already
> - but this also lets them know we're moving ahead with shipping. Thanks -
> Mike
> On 5/1/24 11:51 AM, Alex Russell wrote:
>
> hey Eugene,
>
> This is an exciting an useful addition! Have you looked into other
> platform APIs that could benefit from being able to explicitly specify
> intermediate format hinting and/or transformation? It's a place where (had
> the TAG been consulted) I would have expected to see a larger chain of
> additions to make this work in other areas, or at least a discussion of why
> they aren't (or are?) being pursued. Have you looked into that?
>
> Best,
>
> Alex
>
> On Tuesday, April 30, 2024 at 3:51:01 PM UTC-7 Eugene Zemtsov wrote:
>
>> > Can you please request reviews for privacy, security, enterprise, etc
>> in the chromestatus entry?
>>
>> Done
>>
>> On Mon, Apr 29, 2024 at 7:44 AM Mike Taylor 
>> wrote:
>>
>>> Can you please request reviews for privacy, security, enterprise, etc in
>>> the chromestatus entry?
>>> On 4/25/24 6:19 PM, 'Eugene Zemtsov' via blink-dev wrote:
>>>
>>> Contact emails ezemt...@google.com
>>>
>>> Explainer
>>> https://gist.github.com/Djuffin/9e2f98025ead49998524510cfeed8d33
>>>
>>> Specification https://www.w3.org/TR/webcodecs/#dom-videoframe-copyto
>>>
>>> Summary
>>>
>>> VideoFrame.copyTo() can convert pixel data to RGB pixel format
>>> Converting YUV video frames to RGB is often required for processing them in
>>> libraries like TensorFlow.js and OpenCV.js. Previously the only possible
>>> way to achieve this was rendering the frame on a canvas. Specifying
>>> VideoFrameCopyToOptions.format and VideoFrameCopyToOptions.colorSpace makes
>>> it possible to convert frames to RGB pixel formats by calling
>>> VideoFrame.copyTo() without having to use an extra canvas.
>>>
>>> Blink component Blink>Media>WebCodecs
>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EMedia%3EWebCodecs>
>>>
>>> Initial public proposal https://github.com/w3c/webcodecs/issues/92
>>>
>>> TAG review N/A since the change is minor
>>>
>>> Risks
>>>
>>>
>>> Interoperability and Compatibility
>>>
>>> None
>>>
>>>
>>> *Gecko*: Positive (
>>> https://github.com/w3c/webcodecs/pull/754#pullrequestreview-2008590591)
>>>
>>> *WebKit*: Positive TPAC 2023. Media working group session.
>>>
>>> *Web developers*:
>>> https://github.com/w3c/webcodecs/issues/92#issuecomment-1594083978
>>>
>>> *Other signals*:
>>>
>>> WebView application risks
>>>
>>> Does this intent deprecate or change behavior of existing APIs, such
>>> that it has potentially high risk for Android WebView-based applications?
>>>
>>> None
>>>
>>>
>>> Debuggability
>>>
>>> None
>>>
>>>
>>> Is this feature fully tested by web-platform-tests
>>> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md>
>>> ? Yes
>>>
>>> https://wpt.fyi/results/webcodecs videoFrame-copyTo-rgb.any.js
>>>
>>>
>>> Estimated milestones
>>> Shipping on desktop 126
>>> Shipping on Android 126
>>>
>>> Link to entry on the Chrome Platform Status
>>> https://chromestatus.com/feature/4668827056209920
>>>
>>> --
>>> You received this message becaus

Re: [blink-dev] Intent to Implement and Ship: Conversion to RGB in VideoFrame.copyTo()

['Eugene Zemtsov' via blink-dev]

> Can you please request reviews for privacy, security, enterprise, etc in
the chromestatus entry?

Done

On Mon, Apr 29, 2024 at 7:44 AM Mike Taylor  wrote:

> Can you please request reviews for privacy, security, enterprise, etc in
> the chromestatus entry?
> On 4/25/24 6:19 PM, 'Eugene Zemtsov' via blink-dev wrote:
>
> Contact emails ezemt...@google.com
>
> Explainer https://gist.github.com/Djuffin/9e2f98025ead49998524510cfeed8d33
>
> Specification https://www.w3.org/TR/webcodecs/#dom-videoframe-copyto
>
> Summary
>
> VideoFrame.copyTo() can convert pixel data to RGB pixel format Converting
> YUV video frames to RGB is often required for processing them in libraries
> like TensorFlow.js and OpenCV.js. Previously the only possible way to
> achieve this was rendering the frame on a canvas. Specifying
> VideoFrameCopyToOptions.format and VideoFrameCopyToOptions.colorSpace makes
> it possible to convert frames to RGB pixel formats by calling
> VideoFrame.copyTo() without having to use an extra canvas.
>
> Blink component Blink>Media>WebCodecs
> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EMedia%3EWebCodecs>
>
> Initial public proposal https://github.com/w3c/webcodecs/issues/92
>
> TAG review N/A since the change is minor
>
> Risks
>
>
> Interoperability and Compatibility
>
> None
>
>
> *Gecko*: Positive (
> https://github.com/w3c/webcodecs/pull/754#pullrequestreview-2008590591)
>
> *WebKit*: Positive TPAC 2023. Media working group session.
>
> *Web developers*:
> https://github.com/w3c/webcodecs/issues/92#issuecomment-1594083978
>
> *Other signals*:
>
> WebView application risks
>
> Does this intent deprecate or change behavior of existing APIs, such that
> it has potentially high risk for Android WebView-based applications?
>
> None
>
>
> Debuggability
>
> None
>
>
> Is this feature fully tested by web-platform-tests
> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md>
> ? Yes
>
> https://wpt.fyi/results/webcodecs videoFrame-copyTo-rgb.any.js
>
>
> Estimated milestones
> Shipping on desktop 126
> Shipping on Android 126
>
> Link to entry on the Chrome Platform Status
> https://chromestatus.com/feature/4668827056209920
>
> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+unsubscr...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrGyVMDipu_nqd%3DKw_9eE2UMtdbiWjbqac0NquwNmm4DMg%40mail.gmail.com
> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrGyVMDipu_nqd%3DKw_9eE2UMtdbiWjbqac0NquwNmm4DMg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
>

-- 
Thanks,
Eugene Zemtsov.

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrF_ycwyYKh%2BCOELTG6do4%2BRqZrdvWEC_t62YCptH471sQ%40mail.gmail.com.

[blink-dev] Intent to Implement and Ship: Conversion to RGB in VideoFrame.copyTo()

['Eugene Zemtsov' via blink-dev]

Contact emailsezemt...@google.com

Explainerhttps://gist.github.com/Djuffin/9e2f98025ead49998524510cfeed8d33

Specificationhttps://www.w3.org/TR/webcodecs/#dom-videoframe-copyto

Summary

VideoFrame.copyTo() can convert pixel data to RGB pixel format Converting
YUV video frames to RGB is often required for processing them in libraries
like TensorFlow.js and OpenCV.js. Previously the only possible way to
achieve this was rendering the frame on a canvas. Specifying
VideoFrameCopyToOptions.format and VideoFrameCopyToOptions.colorSpace makes
it possible to convert frames to RGB pixel formats by calling
VideoFrame.copyTo() without having to use an extra canvas.

Blink componentBlink>Media>WebCodecs


Initial public proposalhttps://github.com/w3c/webcodecs/issues/92

TAG reviewN/A since the change is minor

Risks


Interoperability and Compatibility

None


*Gecko*: Positive (
https://github.com/w3c/webcodecs/pull/754#pullrequestreview-2008590591)

*WebKit*: Positive TPAC 2023. Media working group session.

*Web developers*:
https://github.com/w3c/webcodecs/issues/92#issuecomment-1594083978

*Other signals*:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that
it has potentially high risk for Android WebView-based applications?

None


Debuggability

None


Is this feature fully tested by web-platform-tests

?Yes

https://wpt.fyi/results/webcodecs videoFrame-copyTo-rgb.any.js


Estimated milestones
Shipping on desktop 126
Shipping on Android 126

Link to entry on the Chrome Platform Status
https://chromestatus.com/feature/4668827056209920

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrGyVMDipu_nqd%3DKw_9eE2UMtdbiWjbqac0NquwNmm4DMg%40mail.gmail.com.

Re: [blink-dev] Intent to Implement and Ship: VideoEncoderConfig.contentHint

['Eugene Zemtsov' via blink-dev]

Done for Privacy, Security, Enterprise, Debuggability and Testing.


On Mon, Nov 6, 2023 at 7:19 AM Mike Taylor  wrote:

> Hi Eugene,
>
> Could you please request reviews for all the other review gates in your
> chromestatus entry?
>
> thanks,
> Mike
> On 11/1/23 6:32 PM, 'Eugene Zemtsov' via blink-dev wrote:
>
> Contact emails ezemt...@google.com
>
> Specification
> https://www.w3.org/TR/webcodecs/#dom-videoencoderconfig-contenthint
>
> Explainer https://gist.github.com/Djuffin/c3742404b7c53ada227849c8b2b76b4c
>
> Summary
>
> Adding a contentHint field to VideoEncoderConfig Content hint takes values
> that are already used for MediaStreamTrack: "motion", "text", "detail".
> This gives web developers a way to communicate to VideoEncoder the expected
> type video frames they intend to encode.
>
> Blink component Blink>Media>WebCodecs
> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EMedia%3EWebCodecs>
>
> TAG review N/A
>
>
>
> Interoperability and Compatibility
>
>
>
> *Gecko*: No signal (
> https://github.com/mozilla/standards-positions/issues/919)
>
> *WebKit*: Positive (
> https://github.com/WebKit/standards-positions/issues/274)
>
> *Web developers*: Positive RTC software vendors (including Google Meet)
> are interested in using it for screen sharing vs live cam scenarios.
>
>
> Will this feature be supported on all six Blink platforms (Windows, Mac,
> Linux, Chrome OS, Android, and Android WebView)? Yes
>
> Is this feature fully tested by web-platform-tests
> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md>
> ? Yes
>
>
> https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/webcodecs/video-encoder-content-hint.https.any.js
>
>
> Finch feature name WebCodecsContentHint
>
> Requires code in //chrome? False
>
> Tracking bug https://crbug.com/1493588
>
> Estimated milestones
> Shipping on desktop 121
> Shipping on Android 121
> Link to entry on the Chrome Platform Status
> https://chromestatus.com/feature/5103493249761280
>
> --
> Thanks,
> Eugene Zemtsov.
> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+unsubscr...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrGKnc0Jc_doyR9ZsGVfcnk8aNfNS_t%2BTNQ-KLL%3DW%3DiZdA%40mail.gmail.com
> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrGKnc0Jc_doyR9ZsGVfcnk8aNfNS_t%2BTNQ-KLL%3DW%3DiZdA%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
>

-- 
Thanks,
Eugene Zemtsov.

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrHnO-en4d_%2BKcwRBCRWYeVEi8hzcnGUsSkb%3D3dNgh0%2Bdw%40mail.gmail.com.

[blink-dev] Intent to Implement and Ship: VideoEncoderConfig.contentHint

['Eugene Zemtsov' via blink-dev]

Contact emailsezemt...@google.com

Specification
https://www.w3.org/TR/webcodecs/#dom-videoencoderconfig-contenthint

Explainerhttps://gist.github.com/Djuffin/c3742404b7c53ada227849c8b2b76b4c

Summary

Adding a contentHint field to VideoEncoderConfig Content hint takes values
that are already used for MediaStreamTrack: "motion", "text", "detail".
This gives web developers a way to communicate to VideoEncoder the expected
type video frames they intend to encode.

Blink componentBlink>Media>WebCodecs


TAG reviewN/A



Interoperability and Compatibility



*Gecko*: No signal (
https://github.com/mozilla/standards-positions/issues/919)

*WebKit*: Positive (https://github.com/WebKit/standards-positions/issues/274
)

*Web developers*: Positive RTC software vendors (including Google Meet) are
interested in using it for screen sharing vs live cam scenarios.


Will this feature be supported on all six Blink platforms (Windows, Mac,
Linux, Chrome OS, Android, and Android WebView)?Yes

Is this feature fully tested by web-platform-tests

?Yes

https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/webcodecs/video-encoder-content-hint.https.any.js


Finch feature nameWebCodecsContentHint

Requires code in //chrome?False

Tracking bughttps://crbug.com/1493588

Estimated milestones
Shipping on desktop 121
Shipping on Android 121
Link to entry on the Chrome Platform Status
https://chromestatus.com/feature/5103493249761280

--
Thanks,
Eugene Zemtsov.

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrGKnc0Jc_doyR9ZsGVfcnk8aNfNS_t%2BTNQ-KLL%3DW%3DiZdA%40mail.gmail.com.

Re: [blink-dev] Re: Intent to Prototype And Ship: Allow transferring ArrayBuffer into VideoFrame, AudioData, EncodedVideoChunk, EncodedAudioChunk, ImageDecoder constructors

['Eugene Zemtsov' via blink-dev]

TAG review: https://github.com/w3ctag/design-reviews/issues/889

On Wed, Aug 30, 2023 at 9:50 AM Dale Curtis  wrote:

> Alex, I assume you mean TAG's views on consistency regarding transfer
> ergonomics? Otherwise https://www.w3.org/TR/design-principles/#consistency
> is what we followed here. We have not asked, given that we felt this was a
> small performance improvement, with pre-existing ergonomics, and already
> has Media WG approval. We can certainly file a TAG request, but as you
> know, litigating minor features like this through TAG is unlikely to have a
> timely resolution.
>
> Regarding Yoav's proposal above of a single boolean, that might make sense
> today where we have a single transfer, but we expect more input
> ArrayBuffers over time for some of these APIs, which would mean it becomes
> all-or-nothing for developers. E.g., we are likely to accept arrays of
> metadata, HDR data, etc. The boolean would mean they must transfer
> everything, which may lead to them making temporary copies of smaller
> buffers to get transfer effects on the larger ones.
>
> Daniel, sorry, that's just an oversight in the chromestatus entry. There
> are tests added (here's the one for videoFrame):
>
> https://chromium-review.googlesource.com/c/chromium/src/+/4529012/17/third_party/blink/web_tests/external/wpt/webcodecs/videoFrame-construction.any.js
>
> - dale
>
> On Wed, Aug 30, 2023 at 9:19 AM Daniel Bratell 
> wrote:
>
>> In addition to Alex's question, I also noticed that you answered the
>> web-platform-tests with a "no", which is a bit unexpected to me. Is there a
>> reason this cannot or won't be tested in web-platform-tests?
>>
>> /Daniel
>> On 2023-08-30 18:03, Alex Russell wrote:
>>
>> Hey Eugene,
>>
>> I'm a little worried that we're debating API shape here when there hasn't
>> been any guidance from the TAG on design consistency. Have you either asked
>> the TAG to weigh in (didn't see a review link in the Intent) or asked
>> Chromium (ex)TAG members to give the API a once-over?
>>
>> Best,
>>
>> Alex
>>
>> On Thursday, August 24, 2023 at 9:45:42 AM UTC-7 Eugene Zemtsov wrote:
>>
>>> > Can you clarify if this was in response to my questions or to
>>> Jonathan's?
>>>
>>> Yours.
>>> Sorry, I missed Jonathan's question.
>>>
>>> >  Once an ArrayBuffer is transferred and detached, any further access
>>> will result in some sort of TypeError, right?
>>>
>>> Detached ArrayBuffers look like an empty ArrayBuffers.
>>> you can play with them using this code
>>>
>>> let ab = new ArrayBuffer(100);
>>> let ab2 = structuredClone(ab,  { transfer: [ab] })
>>> ab is empty now
>>>
>>>
>>> On Thu, Aug 24, 2023 at 12:51 AM Yoav Weiss 
>>> wrote:
>>>


 On Wed, Aug 23, 2023 at 12:26 PM Jonathan Hao 
 wrote:

> Thanks for the clarification!
>
> On Tue, Aug 22, 2023 at 9:20 PM Eugene Zemtsov 
> wrote:
>
>> A transfer list is consistent with the approach taken by
>> structuredClone
>> 
>>  and postMessage
>> 
>> .
>> And it's already a part of the WebCodecs spec.
>>
>
 Can you clarify if this was in response to my questions or to
 Jonathan's?


>
>>
>> On Tue, Aug 22, 2023 at 7:36 AM Yoav Weiss 
>> wrote:
>>
>>>
>>>
>>> On Tuesday, August 22, 2023 at 11:08:24 AM UTC+2 Jonathan Hao wrote:
>>>
>>> Hi Eugene,
>>>
>>> Just to double check.  Once an ArrayBuffer is transferred and
>>> detached, any further access will result in some sort of TypeError, 
>>> right?
>>>
>>> Thanks,
>>> Jonathan
>>>
>>> On Wednesday, August 16, 2023 at 10:22:00 PM UTC+1 Eugene Zemtsov
>>> wrote:
>>>
>>> Contact emailsezemt...@google.com
>>>
>>> Explainerhttps://gist.github.com/Djuffi
>>> n/1c8fac486ca9f402be85074166e89a16
>>>
>>> Specificationhttps://www.w3.org/TR/webcodecs/#dictdef-videoframeinit
>>>
>>> Summary
>>>
>>> This will allow detaching array buffers and using corresponding
>>> buffers inside VideoFrame, ImageDecoder, EncodedVideoChunk,
>>> EncodedAudioChunk, AudioData without a copy.
>>>
>>> Blink componentBlink>Media>WebCodecs
>>> 
>>>
>>> TAG reviewNone
>>>
>>> Risks
>>>
>>>
>>> Interoperability and Compatibility
>>>
>>> None
>>>
>>>
>>> *Gecko*: N/A (https://www.w3.org/2023/05/30-mediawg-minutes.html#t01)
>>> Change is too small to justify this effort. The change was discussed and
>>> approved by the w3c media working group.
>>>
>>> *WebKit*: N/A (https://www.w3.org/2023/05/30
>>> -mediawg-minutes.html#t01) Change is too small to justify this
>>> effort. The change was discussed and approved by the w3c media working
>

[blink-dev] Re: Intent to Prototype And Ship: Allow transferring ArrayBuffer into VideoFrame, AudioData, EncodedVideoChunk, EncodedAudioChunk, ImageDecoder constructors

['Eugene Zemtsov' via blink-dev]

> Can you clarify if this was in response to my questions or to Jonathan's?

Yours.
Sorry, I missed Jonathan's question.

>  Once an ArrayBuffer is transferred and detached, any further access will
result in some sort of TypeError, right?

Detached ArrayBuffers look like an empty ArrayBuffers.
you can play with them using this code

let ab = new ArrayBuffer(100);
let ab2 = structuredClone(ab,  { transfer: [ab] })
ab is empty now


On Thu, Aug 24, 2023 at 12:51 AM Yoav Weiss  wrote:

>
>
> On Wed, Aug 23, 2023 at 12:26 PM Jonathan Hao  wrote:
>
>> Thanks for the clarification!
>>
>> On Tue, Aug 22, 2023 at 9:20 PM Eugene Zemtsov 
>> wrote:
>>
>>> A transfer list is consistent with the approach taken by structuredClone
>>>  and
>>> postMessage
>>> .
>>> And it's already a part of the WebCodecs spec.
>>>
>>
> Can you clarify if this was in response to my questions or to Jonathan's?
>
>
>>
>>>
>>> On Tue, Aug 22, 2023 at 7:36 AM Yoav Weiss 
>>> wrote:
>>>


 On Tuesday, August 22, 2023 at 11:08:24 AM UTC+2 Jonathan Hao wrote:

 Hi Eugene,

 Just to double check.  Once an ArrayBuffer is transferred and detached,
 any further access will result in some sort of TypeError, right?

 Thanks,
 Jonathan

 On Wednesday, August 16, 2023 at 10:22:00 PM UTC+1 Eugene Zemtsov wrote:

 Contact emailsezemt...@google.com

 Explainerhttps://gist.github.com/Djuffin/1c8fac486ca9f402be85074166e8
 9a16

 Specificationhttps://www.w3.org/TR/webcodecs/#dictdef-videoframeinit

 Summary

 This will allow detaching array buffers and using corresponding buffers
 inside VideoFrame, ImageDecoder, EncodedVideoChunk, EncodedAudioChunk,
 AudioData without a copy.

 Blink componentBlink>Media>WebCodecs
 

 TAG reviewNone

 Risks


 Interoperability and Compatibility

 None


 *Gecko*: N/A (https://www.w3.org/2023/05/30-mediawg-minutes.html#t01)
 Change is too small to justify this effort. The change was discussed and
 approved by the w3c media working group.

 *WebKit*: N/A (https://www.w3.org/2023/05/30-mediawg-minutes.html#t01)
 Change is too small to justify this effort. The change was discussed and
 approved by the w3c media working group.


 I somewhat share Youenn's concerns about the API shape, but I'm not
 familiar with the examples this is supposed to be consistent with. Would it
 be possible to explore different API shapes in the explainer? (e.g. a
 boolean that detaches the input buffer after init would be my first choice)
 Typically we defer such questions to a TAG review. I'd hate to
 introduce significant delay at this point, but it might be possible to
 expedite this specific question and get it in front of them.



 *Web developers*: No signals

 *Other signals*:

 WebView application risks

 Does this intent deprecate or change behavior of existing APIs, such
 that it has potentially high risk for Android WebView-based applications?

 None


 Debuggability

 None


 Is this feature fully tested by web-platform-tests
 
 ?No

 Flag name on chrome://flagsNone

 Finch feature nameNone

 Non-finch justificationNone

 Requires code in //chrome?False

 Tracking bughttps://crbug.com/1446808

 Estimated milestonesShipping on desktop120Shipping on Android120

 Link to entry on the Chrome Platform Status
 https://chromestatus.com/feature/5075602045927424

 --
 Thanks,
 Eugene Zemtsov.


>>>
>>> --
>>> Thanks,
>>> Eugene Zemtsov.
>>>
>>

-- 
Thanks,
Eugene Zemtsov.

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrGsBMPJ0OZfr9GBDLoQ70B_EvFj%3DQOTqxJUmN7us%2BZxKw%40mail.gmail.com.

[blink-dev] Re: Intent to Prototype And Ship: Allow transferring ArrayBuffer into VideoFrame, AudioData, EncodedVideoChunk, EncodedAudioChunk, ImageDecoder constructors

['Eugene Zemtsov' via blink-dev]

A transfer list is consistent with the approach taken by structuredClone
 and
postMessage
.
And it's already a part of the WebCodecs spec.


On Tue, Aug 22, 2023 at 7:36 AM Yoav Weiss  wrote:

>
>
> On Tuesday, August 22, 2023 at 11:08:24 AM UTC+2 Jonathan Hao wrote:
>
> Hi Eugene,
>
> Just to double check.  Once an ArrayBuffer is transferred and detached,
> any further access will result in some sort of TypeError, right?
>
> Thanks,
> Jonathan
>
> On Wednesday, August 16, 2023 at 10:22:00 PM UTC+1 Eugene Zemtsov wrote:
>
> Contact emailsezemt...@google.com
>
> Explainerhttps://gist.github.com/Djuffin/1c8fac486ca9f402be85074166e89a16
>
> Specificationhttps://www.w3.org/TR/webcodecs/#dictdef-videoframeinit
>
> Summary
>
> This will allow detaching array buffers and using corresponding buffers
> inside VideoFrame, ImageDecoder, EncodedVideoChunk, EncodedAudioChunk,
> AudioData without a copy.
>
> Blink componentBlink>Media>WebCodecs
> 
>
> TAG reviewNone
>
> Risks
>
>
> Interoperability and Compatibility
>
> None
>
>
> *Gecko*: N/A (https://www.w3.org/2023/05/30-mediawg-minutes.html#t01)
> Change is too small to justify this effort. The change was discussed and
> approved by the w3c media working group.
>
> *WebKit*: N/A (https://www.w3.org/2023/05/30-mediawg-minutes.html#t01)
> Change is too small to justify this effort. The change was discussed and
> approved by the w3c media working group.
>
>
> I somewhat share Youenn's concerns about the API shape, but I'm not
> familiar with the examples this is supposed to be consistent with. Would it
> be possible to explore different API shapes in the explainer? (e.g. a
> boolean that detaches the input buffer after init would be my first choice)
> Typically we defer such questions to a TAG review. I'd hate to introduce
> significant delay at this point, but it might be possible to expedite this
> specific question and get it in front of them.
>
>
>
> *Web developers*: No signals
>
> *Other signals*:
>
> WebView application risks
>
> Does this intent deprecate or change behavior of existing APIs, such that
> it has potentially high risk for Android WebView-based applications?
>
> None
>
>
> Debuggability
>
> None
>
>
> Is this feature fully tested by web-platform-tests
> 
> ?No
>
> Flag name on chrome://flagsNone
>
> Finch feature nameNone
>
> Non-finch justificationNone
>
> Requires code in //chrome?False
>
> Tracking bughttps://crbug.com/1446808
>
> Estimated milestonesShipping on desktop120Shipping on Android120
>
> Link to entry on the Chrome Platform Statushttps://chromestatus.com/featu
> re/5075602045927424
>
> --
> Thanks,
> Eugene Zemtsov.
>
>

-- 
Thanks,
Eugene Zemtsov.

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrG3GBqcheKaZcnDov9vxnazzT23AWiCzcx%2BVUCPqi4phg%40mail.gmail.com.

[blink-dev] Intent to Prototype And Ship: Allow transferring ArrayBuffer into VideoFrame, AudioData, EncodedVideoChunk, EncodedAudioChunk, ImageDecoder constructors

['Eugene Zemtsov' via blink-dev]

Contact emailsezemt...@google.com

Explainerhttps://gist.github.com/Djuffin/1c8fac486ca9f402be85074166e89a16

Specificationhttps://www.w3.org/TR/webcodecs/#dictdef-videoframeinit

Summary

This will allow detaching array buffers and using corresponding buffers
inside VideoFrame, ImageDecoder, EncodedVideoChunk, EncodedAudioChunk,
AudioData without a copy.

Blink componentBlink>Media>WebCodecs


TAG reviewNone

Risks


Interoperability and Compatibility

None


*Gecko*: N/A (https://www.w3.org/2023/05/30-mediawg-minutes.html#t01)
Change is too small to justify this effort. The change was discussed and
approved by the w3c media working group.

*WebKit*: N/A (https://www.w3.org/2023/05/30-mediawg-minutes.html#t01)
Change is too small to justify this effort. The change was discussed and
approved by the w3c media working group.

*Web developers*: No signals

*Other signals*:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that
it has potentially high risk for Android WebView-based applications?

None


Debuggability

None


Is this feature fully tested by web-platform-tests

?No

Flag name on chrome://flagsNone

Finch feature nameNone

Non-finch justificationNone

Requires code in //chrome?False

Tracking bughttps://crbug.com/1446808

Estimated milestones
Shipping on desktop 120
Shipping on Android 120

Link to entry on the Chrome Platform Status
https://chromestatus.com/feature/5075602045927424

-- 
Thanks,
Eugene Zemtsov.

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrE6f1K-rBtAr%3D2fr3MZ8Oopm%2BbEmcxBPK11ytLS4n-m_Q%40mail.gmail.com.

Re: [blink-dev] Intent to Implement and Ship: Per-frame quantizer in VideoEncoder

['Eugene Zemtsov' via blink-dev]

Thanks for the feedback.
I put together an explainer and linked it on the ChromeStatus feature page:

https://gist.github.com/Djuffin/3722232679b977058be787be0dff4254

On Wed, Jul 19, 2023 at 8:57 AM Alex Russell 
wrote:

> I *think* I grok what this is for, and I'm still pretty frustrated that
> there isn't a crisp summary along the lines of "this parameter helps sites
> implement custom bitrate vs. quality vs. CPU use tradeoffs for different
> kinds of media and streams, which are important to customers like X, Y, and
> Z".
>
> Eugene, Philipp: it's important that the Blink process show that we are
> shipping important features that solve real problems, particularly when
> we're in the position of shipping first. We *want* to trust the
> WebRTC/media community to work with us to launch API changes quickly, and
> demonstrating need is part of that. Can you respond with an overview
> (perhaps in the form of an Explainer), and/or perhaps have potential users
> of this API chime in?
>
> Best,
>
> Alex
>
> On Wednesday, July 19, 2023 at 8:14:16 AM UTC-7 Yoav Weiss wrote:
>
>> On Wed, Jul 19, 2023 at 2:51 PM Philipp Hancke <
>> philipp.han...@googlemail.com> wrote:
>>
>>> Am Mi., 19. Juli 2023 um 14:25 Uhr schrieb Yoav Weiss <
>>> yoavwe...@chromium.org>:
>>>
>>>>
>>>>
>>>> On Thu, Jul 13, 2023 at 10:53 PM 'Eugene Zemtsov' via blink-dev <
>>>> blink-dev@chromium.org> wrote:
>>>>
>>>>> Any new feedback or resolution on this one?
>>>>>
>>>>> On Fri, Jul 7, 2023 at 5:53 AM Sangwhan Moon  wrote:
>>>>>
>>>>>> (resending from correct email)
>>>>>>
>>>>>> On 2023年07月07日 00時32分12秒 (+09:00), Mike Taylor wrote:
>>>>>>
>>>>>> On 7/5/23 8:57 PM, 'Eugene Zemtsov' via blink-dev wrote:
>>>>>>
>>>>>> Intent to Implement and Ship: Per-frame quantizer in VideoEncoder
>>>>>>
>>>>>> Contact emails
>>>>>>
>>>>>> ezemt...@google.com
>>>>>>
>>>>>> Explainer
>>>>>>
>>>>>> None
>>>>>>
>>>>>>
>>>> I think an explainer can be significantly helpful in helping us
>>>> understand how developers will be using this feature and what use cases
>>>> it'd cover.
>>>> Could you write one or add an inline explanation outlining that?
>>>>
>>>
>>> Explaining that is tough without going into the details "what is
>>> quantization for video codecs. See
>>>   https://www.vcodex.com/h264avc-4x4-transform-and-quantization/
>>> for a very detailed explanation for H264.
>>>
>>> the tl;dr is that folks who encode video like tuning all kinds of knobs
>>> to get the "best" result and qp is one of those.
>>>
>>
>> An explainer doesn't have to assume folks reading it don't know what
>> quantization means :)
>>
>> Basically, clicking through the specs, it's still not clear to me if the
>> quantization values are provided as a single int that quantization tables
>> are supposed to be divided by. a "quality" int that represents a certain
>> quantization table, or something else entirely. Clarity on that would be
>> great.
>>
>>
>>>
>>>>
>>>>>
>>>>>> Specification
>>>>>>
>>>>>> https://www.w3.org/TR/webcodecs/#video-encoder-bitrate-mode
>>>>>>
>>>>>> Summary
>>>>>>
>>>>>> Adds "quantizer" VideoEncoderBitrateMode for VideoEncoder. This
>>>>>> allows to specify a quantizer parameter for each frame for AV1, VP9, and
>>>>>> AVC video codecs. The quantizer parameter is set via codec specific
>>>>>> extensions for VideoEncoderEncodeOptions.
>>>>>>
>>>>>> Assuming I know very little about video codecs, what use cases does
>>>>>> this enable for developers?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Blink component
>>>>>>
>>>>>> Blink>Media>WebCodecs
>>>>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EMedia%3EWebCodecs>
>>>>>>
>>>>>> TAG review
>>>>>>
>&

Re: [blink-dev] Intent to Implement and Ship: Per-frame quantizer in VideoEncoder

['Eugene Zemtsov' via blink-dev]

Any new feedback or resolution on this one?

On Fri, Jul 7, 2023 at 5:53 AM Sangwhan Moon  wrote:

> (resending from correct email)
>
> On 2023年07月07日 00時32分12秒 (+09:00), Mike Taylor wrote:
>
> On 7/5/23 8:57 PM, 'Eugene Zemtsov' via blink-dev wrote:
>
> Intent to Implement and Ship: Per-frame quantizer in VideoEncoder
>
> Contact emails
>
> ezemt...@google.com
>
> Explainer
>
> None
>
> Specification
>
> https://www.w3.org/TR/webcodecs/#video-encoder-bitrate-mode
>
> Summary
>
> Adds "quantizer" VideoEncoderBitrateMode for VideoEncoder. This allows to
> specify a quantizer parameter for each frame for AV1, VP9, and AVC video
> codecs. The quantizer parameter is set via codec specific extensions for
> VideoEncoderEncodeOptions.
>
> Assuming I know very little about video codecs, what use cases does this
> enable for developers?
>
>
>
> Blink component
>
> Blink>Media>WebCodecs
> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EMedia%3EWebCodecs>
>
> TAG review
>
> None.
>
> Previously WebCodecs API had TAG review as a whole:
>
> https://github.com/w3ctag/design-reviews/issues/612
>
> This is a new addition since that review, but it's a trivial addition
> (IMHO) so a new review request isn't needed.
>
>
> I agree, I don't see anything particularly contentious in the proposal
> that would warrant a full review.
>
>
>
> TAG review status
>
> Not applicable
>
> Risks
>
> Interoperability and Compatibility
>
> Gecko: Neutral (
> https://github.com/mozilla/standards-positions/issues/837#issuecomment-1614666364)
> Paul Adenot (Mozilla) expressed that minor changes to WebCodecs spec don't
> need to go through the full "Request for Mozilla Position" process assuming
> they were approved by the Media Working Group.
>
> WebKit: Positive (https://www.w3.org/2023/03/07-mediawg-minutes.html#t02)
> The issue was discussed on 07 March 2023 by w3c Media working group. Jer
> Noble (Apple) was actively participating and provided input for spec
> details.
>
> Can we request a formal position from WebKit, at least to let them know
> we're intending to ship?
>
>
> Web developers: Positive (https://github.com/w3c/webcodecs/issues/56)
> People ask for this on GitHub
>
> Other signals:
>
> WebView application risks
>
> Does this intent deprecate or change behavior of existing APIs, such that
> it has potentially high risk for Android WebView-based applications?
>
>
>
> Will this feature be supported on all six Blink platforms (Windows, Mac,
> Linux, Chrome OS, Android, and Android WebView)?
>
> Yes
>
> Is this feature fully tested by web-platform-tests
> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md>
> ?
>
> Yes
>
> Tracking bug
>
> https://bugs.chromium.org/p/chromium/issues/detail?id=1424154
>
> Estimated milestones
>
> Shipping on desktop
>
> 117
>
> Shipping on Android
>
> 117
>
>
>
>
> Link to entry on the Chrome Platform Status
>
> https://chromestatus.com/feature/5783986600673280
>
>
>
>
>
> --
> Thanks,
> Eugene Zemtsov.
> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+unsubscr...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrF0PKfpVbaYWX-hR0wJ%2Bb9H4YtwFBUc6Y6JGSmFT7pVgQ%40mail.gmail.com
> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrF0PKfpVbaYWX-hR0wJ%2Bb9H4YtwFBUc6Y6JGSmFT7pVgQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+unsubscr...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/0a1d0f96-c5f0-fdf8-44b0-b49c3f180839%40chromium.org
> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/0a1d0f96-c5f0-fdf8-44b0-b49c3f180839%40chromium.org?utm_medium=email&utm_source=footer>
> .
>
>

-- 
Thanks,
Eugene Zemtsov.

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrFxVaQnqU4dGQesuw%2B_AaBV763a0-t%3Dp0273LPNbyespQ%40mail.gmail.com.

Re: [blink-dev] Intent to Implement and Ship: Per-frame quantizer in VideoEncoder

['Eugene Zemtsov' via blink-dev]

>  what use cases does this enable for developers?

Controlling the quantizer for every frame makes possible at least two new
use cases for VideoEncoder:

1. Constant quality encoding when complex frames and parts get more
bandwidth than less complex ones.

2. External rate control algorithms implemented by web developers. Such
algorithms can take into account a website's knowledge about contents of
each frame or quick changes in available network bandwidth.

> Can we request a formal position from WebKit, at least to let them know
we're intending to ship?

https://github.com/WebKit/standards-positions/issues/223


On Thu, Jul 6, 2023 at 8:32 AM Mike Taylor  wrote:

> On 7/5/23 8:57 PM, 'Eugene Zemtsov' via blink-dev wrote:
>
> Intent to Implement and Ship: Per-frame quantizer in VideoEncoder
>
> Contact emails
>
> ezemt...@google.com
>
> Explainer
>
> None
>
> Specification
>
> https://www.w3.org/TR/webcodecs/#video-encoder-bitrate-mode
>
> Summary
>
> Adds "quantizer" VideoEncoderBitrateMode for VideoEncoder. This allows to
> specify a quantizer parameter for each frame for AV1, VP9, and AVC video
> codecs. The quantizer parameter is set via codec specific extensions for
> VideoEncoderEncodeOptions.
>
> Assuming I know very little about video codecs, what use cases does this
> enable for developers?
>
>
>
> Blink component
>
> Blink>Media>WebCodecs
> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EMedia%3EWebCodecs>
>
> TAG review
>
> None.
>
> Previously WebCodecs API had TAG review as a whole:
>
> https://github.com/w3ctag/design-reviews/issues/612
>
> This is a new addition since that review, but it's a trivial addition
> (IMHO) so a new review request isn't needed.
>
>
> TAG review status
>
> Not applicable
>
> Risks
>
> Interoperability and Compatibility
>
> Gecko: Neutral (
> https://github.com/mozilla/standards-positions/issues/837#issuecomment-1614666364)
> Paul Adenot (Mozilla) expressed that minor changes to WebCodecs spec don't
> need to go through the full "Request for Mozilla Position" process assuming
> they were approved by the Media Working Group.
>
> WebKit: Positive (https://www.w3.org/2023/03/07-mediawg-minutes.html#t02)
> The issue was discussed on 07 March 2023 by w3c Media working group. Jer
> Noble (Apple) was actively participating and provided input for spec
> details.
>
> Can we request a formal position from WebKit, at least to let them know
> we're intending to ship?
>
>
> Web developers: Positive (https://github.com/w3c/webcodecs/issues/56)
> People ask for this on GitHub
>
> Other signals:
>
> WebView application risks
>
> Does this intent deprecate or change behavior of existing APIs, such that
> it has potentially high risk for Android WebView-based applications?
>
>
>
> Will this feature be supported on all six Blink platforms (Windows, Mac,
> Linux, Chrome OS, Android, and Android WebView)?
>
> Yes
>
> Is this feature fully tested by web-platform-tests
> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md>
> ?
>
> Yes
>
> Tracking bug
>
> https://bugs.chromium.org/p/chromium/issues/detail?id=1424154
>
> Estimated milestones
>
> Shipping on desktop
>
> 117
>
> Shipping on Android
>
> 117
>
>
>
>
> Link to entry on the Chrome Platform Status
>
> https://chromestatus.com/feature/5783986600673280
>
>
>
>
>
> --
> Thanks,
> Eugene Zemtsov.
> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+unsubscr...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrF0PKfpVbaYWX-hR0wJ%2Bb9H4YtwFBUc6Y6JGSmFT7pVgQ%40mail.gmail.com
> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrF0PKfpVbaYWX-hR0wJ%2Bb9H4YtwFBUc6Y6JGSmFT7pVgQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
>

-- 
Thanks,
Eugene Zemtsov.

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrHCKH%3DtrrSdxp50oVnPf9Jw-N%2BeD6fii_k24gmkemvMXw%40mail.gmail.com.

[blink-dev] Intent to Implement and Ship: Per-frame quantizer in VideoEncoder

['Eugene Zemtsov' via blink-dev]

Intent to Implement and Ship: Per-frame quantizer in VideoEncoder

Contact emails

ezemt...@google.com

Explainer

None

Specification

https://www.w3.org/TR/webcodecs/#video-encoder-bitrate-mode

Summary

Adds "quantizer" VideoEncoderBitrateMode for VideoEncoder. This allows to
specify a quantizer parameter for each frame for AV1, VP9, and AVC video
codecs. The quantizer parameter is set via codec specific extensions for
VideoEncoderEncodeOptions.


Blink component

Blink>Media>WebCodecs


TAG review

None.

Previously WebCodecs API had TAG review as a whole:

https://github.com/w3ctag/design-reviews/issues/612

TAG review status

Not applicable

Risks

Interoperability and Compatibility

Gecko: Neutral (
https://github.com/mozilla/standards-positions/issues/837#issuecomment-1614666364)
Paul Adenot (Mozilla) expressed that minor changes to WebCodecs spec don't
need to go through the full "Request for Mozilla Position" process assuming
they were approved by the Media Working Group.

WebKit: Positive (https://www.w3.org/2023/03/07-mediawg-minutes.html#t02)
The issue was discussed on 07 March 2023 by w3c Media working group. Jer
Noble (Apple) was actively participating and provided input for spec
details.

Web developers: Positive (https://github.com/w3c/webcodecs/issues/56)
People ask for this on GitHub

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that
it has potentially high risk for Android WebView-based applications?



Will this feature be supported on all six Blink platforms (Windows, Mac,
Linux, Chrome OS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests

?

Yes

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1424154

Estimated milestones

Shipping on desktop

117

Shipping on Android

117




Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5783986600673280





-- 
Thanks,
Eugene Zemtsov.

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK8JDrF0PKfpVbaYWX-hR0wJ%2Bb9H4YtwFBUc6Y6JGSmFT7pVgQ%40mail.gmail.com.