Re: [FFmpeg-devel] [PATCH 2/2] lavc/videotoolboxenc: enable low-latency mode for HEVC
> On Sep 1, 2023, at 12:07 AM, myp...@gmail.com wrote:
>
> On Fri, Sep 1, 2023 at 11:50 AM "zhilizhao(赵志立)"
> wrote:
>>
>>
>>
>>>> On Sep 1, 2023, at 09:13, myp...@gmail.com wrote:
>>>
>>> On Fri, Sep 1, 2023 at 2:13 AM Zhao Zhili wrote:
>>>>
>>>>
>>>>> From: ffmpeg-devel On Behalf Of Jun Zhao
>>>>> Sent: 2023年8月31日 22:50
>>>>> To: ffmpeg-devel@ffmpeg.org
>>>>> Cc: Jun Zhao ; Jun Zhao
>>>>> Subject: [FFmpeg-devel] [PATCH 2/2] lavc/videotoolboxenc: enable
>>>>> low-latency mode for HEVC
>>>>>
>>>>> Enable the HEVC encoder that supports low-latency mode.
>>>>>
>>>>> Signed-off-by: Jun Zhao
>>>>> ---
>>>>> libavcodec/videotoolboxenc.c | 3 ++-
>>>>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
>>>>> index 8e493c4f7a..bfc03787a0 100644
>>>>> --- a/libavcodec/videotoolboxenc.c
>>>>> +++ b/libavcodec/videotoolboxenc.c
>>>>> @@ -1568,7 +1568,8 @@ static int vtenc_configure_encoder(AVCodecContext
>>>>> *avctx)
>>>>> #endif
>>>>>
>>>>>// low-latency mode: eliminate frame reordering, follow a
>>>>> one-in-one-out encoding mode
>>>>> -if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) && avctx->codec_id ==
>>>>> AV_CODEC_ID_H264) {
>>>>> +if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) &&
>>>>> +(avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id ==
>>>>> AV_CODEC_ID_HEVC)) {
>>>>
>>>> How about just remove the codec_id check?
>>>> Is there any unwelcome side effects for encoders other than H264/HEVC?
>>>
>>> 1. One reason is that there are now other encoders available, such as
>>> prores_videotoolbox.
>>>
>>> 2. It may be more appropriate to use a strictly qualified check
>>> instead of an open condition.
>>
>> Both HEVC and ProRes failed with low_latency on my machine (with or without
>> -allow_sw 1)
>>
>> [hevc_videotoolbox @ 0x6192a380] Error: cannot create compression
>> session: -12902
>> [prores_videotoolbox @ 0x6192a380] Error: cannot create compression
>> session: -12902
>>
>>kVTParameterErr =
>> -12902,
>>
> I think you missed the other options, low delay in VT encoder need to
> co-work with some other options
>
>> 1. FFmpeg users should have an idea whether or not to set
>> AV_CODEC_FLAG_LOW_DELAY.
>> AV_CODEC_FLAG_LOW_DELAY doesn’t make sense for ProRes, but if user wants to
>> try, let he/she
>> do.
>>
>> 2. I can’t find API to query the capability. Only VideoToolBox framework
>> itself know how
>> to handle the argument. There isn’t much protect to provide for users.
>>
> Yes, now FFmpeg VT encoder didn't used the capability query, I think
> need to used the some API VTCopySupportedPropertyDictionaryForEncoder
> for capability query, this my next step
That’s a good idea. We don’t have to worry about codec checks or unexpected
failures if the HEVC encoder on some devices doesn’t support it. Ideally we can
print out a warn message and ignore the low-delay flag.
>> Even with H.264, the results is unexpected: the input framerate is 25 but
>> the output
>> framerate is 5, it dropped 80 percent frames silently.
>>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/2] lavc/videotoolboxenc: enable low-latency mode for HEVC
On Fri, Sep 1, 2023 at 11:50 AM "zhilizhao(赵志立)" wrote:
>
>
>
> > On Sep 1, 2023, at 09:13, myp...@gmail.com wrote:
> >
> > On Fri, Sep 1, 2023 at 2:13 AM Zhao Zhili wrote:
> >>
> >>
> >>> From: ffmpeg-devel On Behalf Of Jun Zhao
> >>> Sent: 2023年8月31日 22:50
> >>> To: ffmpeg-devel@ffmpeg.org
> >>> Cc: Jun Zhao ; Jun Zhao
> >>> Subject: [FFmpeg-devel] [PATCH 2/2] lavc/videotoolboxenc: enable
> >>> low-latency mode for HEVC
> >>>
> >>> Enable the HEVC encoder that supports low-latency mode.
> >>>
> >>> Signed-off-by: Jun Zhao
> >>> ---
> >>> libavcodec/videotoolboxenc.c | 3 ++-
> >>> 1 file changed, 2 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
> >>> index 8e493c4f7a..bfc03787a0 100644
> >>> --- a/libavcodec/videotoolboxenc.c
> >>> +++ b/libavcodec/videotoolboxenc.c
> >>> @@ -1568,7 +1568,8 @@ static int vtenc_configure_encoder(AVCodecContext
> >>> *avctx)
> >>> #endif
> >>>
> >>> // low-latency mode: eliminate frame reordering, follow a
> >>> one-in-one-out encoding mode
> >>> -if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) && avctx->codec_id ==
> >>> AV_CODEC_ID_H264) {
> >>> +if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) &&
> >>> +(avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id ==
> >>> AV_CODEC_ID_HEVC)) {
> >>
> >> How about just remove the codec_id check?
> >> Is there any unwelcome side effects for encoders other than H264/HEVC?
> >
> > 1. One reason is that there are now other encoders available, such as
> > prores_videotoolbox.
> >
> > 2. It may be more appropriate to use a strictly qualified check
> > instead of an open condition.
>
> Both HEVC and ProRes failed with low_latency on my machine (with or without
> -allow_sw 1)
>
> [hevc_videotoolbox @ 0x6192a380] Error: cannot create compression
> session: -12902
> [prores_videotoolbox @ 0x6192a380] Error: cannot create compression
> session: -12902
>
> kVTParameterErr =
> -12902,
>
I think you missed the other options, low delay in VT encoder need to
co-work with some other options
> 1. FFmpeg users should have an idea whether or not to set
> AV_CODEC_FLAG_LOW_DELAY.
> AV_CODEC_FLAG_LOW_DELAY doesn’t make sense for ProRes, but if user wants to
> try, let he/she
> do.
>
> 2. I can’t find API to query the capability. Only VideoToolBox framework
> itself know how
> to handle the argument. There isn’t much protect to provide for users.
>
Yes, now FFmpeg VT encoder didn't used the capability query, I think
need to used the some API VTCopySupportedPropertyDictionaryForEncoder
for capability query, this my next step
> Even with H.264, the results is unexpected: the input framerate is 25 but the
> output
> framerate is 5, it dropped 80 percent frames silently.
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/2] lavc/videotoolboxenc: enable low-latency mode for HEVC
> On Sep 1, 2023, at 09:13, myp...@gmail.com wrote:
>
> On Fri, Sep 1, 2023 at 2:13 AM Zhao Zhili wrote:
>>
>>
>>> From: ffmpeg-devel On Behalf Of Jun Zhao
>>> Sent: 2023年8月31日 22:50
>>> To: ffmpeg-devel@ffmpeg.org
>>> Cc: Jun Zhao ; Jun Zhao
>>> Subject: [FFmpeg-devel] [PATCH 2/2] lavc/videotoolboxenc: enable
>>> low-latency mode for HEVC
>>>
>>> Enable the HEVC encoder that supports low-latency mode.
>>>
>>> Signed-off-by: Jun Zhao
>>> ---
>>> libavcodec/videotoolboxenc.c | 3 ++-
>>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
>>> index 8e493c4f7a..bfc03787a0 100644
>>> --- a/libavcodec/videotoolboxenc.c
>>> +++ b/libavcodec/videotoolboxenc.c
>>> @@ -1568,7 +1568,8 @@ static int vtenc_configure_encoder(AVCodecContext
>>> *avctx)
>>> #endif
>>>
>>> // low-latency mode: eliminate frame reordering, follow a
>>> one-in-one-out encoding mode
>>> -if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) && avctx->codec_id ==
>>> AV_CODEC_ID_H264) {
>>> +if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) &&
>>> +(avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id ==
>>> AV_CODEC_ID_HEVC)) {
>>
>> How about just remove the codec_id check?
>> Is there any unwelcome side effects for encoders other than H264/HEVC?
>
> 1. One reason is that there are now other encoders available, such as
> prores_videotoolbox.
>
> 2. It may be more appropriate to use a strictly qualified check
> instead of an open condition.
Both HEVC and ProRes failed with low_latency on my machine (with or without
-allow_sw 1)
[hevc_videotoolbox @ 0x6192a380] Error: cannot create compression session:
-12902
[prores_videotoolbox @ 0x6192a380] Error: cannot create compression
session: -12902
kVTParameterErr =
-12902,
1. FFmpeg users should have an idea whether or not to set
AV_CODEC_FLAG_LOW_DELAY.
AV_CODEC_FLAG_LOW_DELAY doesn’t make sense for ProRes, but if user wants to
try, let he/she
do.
2. I can’t find API to query the capability. Only VideoToolBox framework itself
know how
to handle the argument. There isn’t much protect to provide for users.
Even with H.264, the results is unexpected: the input framerate is 25 but the
output
framerate is 5, it dropped 80 percent frames silently.
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/2] lavc/videotoolboxenc: enable low-latency mode for HEVC
On Fri, Sep 1, 2023 at 2:13 AM Zhao Zhili wrote:
>
>
> > From: ffmpeg-devel On Behalf Of Jun Zhao
> > Sent: 2023年8月31日 22:50
> > To: ffmpeg-devel@ffmpeg.org
> > Cc: Jun Zhao ; Jun Zhao
> > Subject: [FFmpeg-devel] [PATCH 2/2] lavc/videotoolboxenc: enable
> > low-latency mode for HEVC
> >
> > Enable the HEVC encoder that supports low-latency mode.
> >
> > Signed-off-by: Jun Zhao
> > ---
> > libavcodec/videotoolboxenc.c | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
> > index 8e493c4f7a..bfc03787a0 100644
> > --- a/libavcodec/videotoolboxenc.c
> > +++ b/libavcodec/videotoolboxenc.c
> > @@ -1568,7 +1568,8 @@ static int vtenc_configure_encoder(AVCodecContext
> > *avctx)
> > #endif
> >
> > // low-latency mode: eliminate frame reordering, follow a
> > one-in-one-out encoding mode
> > -if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) && avctx->codec_id ==
> > AV_CODEC_ID_H264) {
> > +if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) &&
> > +(avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id ==
> > AV_CODEC_ID_HEVC)) {
>
> How about just remove the codec_id check?
> Is there any unwelcome side effects for encoders other than H264/HEVC?
1. One reason is that there are now other encoders available, such as
prores_videotoolbox.
2. It may be more appropriate to use a strictly qualified check
instead of an open condition.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/2] lavc/videotoolboxenc: enable low-latency mode for HEVC
> From: ffmpeg-devel On Behalf Of Jun Zhao
> Sent: 2023年8月31日 22:50
> To: ffmpeg-devel@ffmpeg.org
> Cc: Jun Zhao ; Jun Zhao
> Subject: [FFmpeg-devel] [PATCH 2/2] lavc/videotoolboxenc: enable low-latency
> mode for HEVC
>
> Enable the HEVC encoder that supports low-latency mode.
>
> Signed-off-by: Jun Zhao
> ---
> libavcodec/videotoolboxenc.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
> index 8e493c4f7a..bfc03787a0 100644
> --- a/libavcodec/videotoolboxenc.c
> +++ b/libavcodec/videotoolboxenc.c
> @@ -1568,7 +1568,8 @@ static int vtenc_configure_encoder(AVCodecContext
> *avctx)
> #endif
>
> // low-latency mode: eliminate frame reordering, follow a one-in-one-out
> encoding mode
> -if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) && avctx->codec_id ==
> AV_CODEC_ID_H264) {
> +if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) &&
> +(avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id ==
> AV_CODEC_ID_HEVC)) {
How about just remove the codec_id check?
Is there any unwelcome side effects for encoders other than H264/HEVC?
> CFDictionarySetValue(enc_info,
>
> compat_keys.kVTVideoEncoderSpecification_EnableLowLatencyRateControl,
> kCFBooleanTrue);
> --
> 2.25.1
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/2] lavc/videotoolboxenc: enable low-latency mode for HEVC
Enable the HEVC encoder that supports low-latency mode.
Signed-off-by: Jun Zhao
---
libavcodec/videotoolboxenc.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
index 8e493c4f7a..bfc03787a0 100644
--- a/libavcodec/videotoolboxenc.c
+++ b/libavcodec/videotoolboxenc.c
@@ -1568,7 +1568,8 @@ static int vtenc_configure_encoder(AVCodecContext *avctx)
#endif
// low-latency mode: eliminate frame reordering, follow a one-in-one-out
encoding mode
-if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) && avctx->codec_id ==
AV_CODEC_ID_H264) {
+if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) &&
+(avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id ==
AV_CODEC_ID_HEVC)) {
CFDictionarySetValue(enc_info,
compat_keys.kVTVideoEncoderSpecification_EnableLowLatencyRateControl,
kCFBooleanTrue);
--
2.25.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
