Re: [FFmpeg-devel] [PATCH v2 1/1] avcodec/vaapi_encode: add frame-skip func

2019-02-25 Thread Mark Thompson 
On 22/02/2019 03:09, Sun, Jing A wrote:
> -Original Message-
> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf Of Mark 
> Thompson
> Sent: Thursday, February 21, 2019 5:35 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avcodec/vaapi_encode: add 
> frame-skip func
> 
> On 20/02/2019 10:33, Jing SUN wrote:
>> This implements app controlled frame skipping in vaapi encoding. To 
>> make a frame skipped, allocate its frame side data of the newly added 
>> AV_FRAME_DATA_SKIP_FRAME type and set its value to 1.
>>
>> Signed-off-by: Jing SUN 
>> ---
>>  libavcodec/vaapi_encode.c | 112 
>> --
>>  libavcodec/vaapi_encode.h |   5 +++
>>  libavutil/frame.c |   1 +
>>  libavutil/frame.h |   5 +++
>>  4 files changed, 119 insertions(+), 4 deletions(-)
> 
> Have a look at 
> <https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2019-February/239989.html>, 
> which tries to implement this feature in a more general way without adding 
> any ad-hoc API.
> 
> - Mark
> -Original Message-
> 
> Hi Mark,
> 
> I think the 239989 patch is quite a good one to implement VFR by adapting 
> VAAPI frame-skip feature, but it does not fulfill the feature requirement 
> that we have been asked to achieve, which is to skip the frames that are 
> selected by apps, and the purpose to skip any frames is not just to control 
> frame-rate, but also to skip duplicated ones, or to skip abandoned ones due 
> to some changes just made, etc. Could we please keep both and let users 
> decide which way they want to use?

I think these use-cases are all supported by the same method used for other 
encoders in libavcodec, by not sending the skipped frames to the encoder.  The 
linked patch is only wanted to fix the rate control in these cases - that's 
otherwise broken because VAAPI lacks any support for timestamps.

Can you explain a specific use-case which wants the extra message?

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH v2 1/1] avcodec/vaapi_encode: add frame-skip func

2019-02-21 Thread Sun, Jing A 
-Original Message-
From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf Of Mark 
Thompson
Sent: Thursday, February 21, 2019 5:35 AM
To: ffmpeg-devel@ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avcodec/vaapi_encode: add frame-skip 
func

On 20/02/2019 10:33, Jing SUN wrote:
> This implements app controlled frame skipping in vaapi encoding. To 
> make a frame skipped, allocate its frame side data of the newly added 
> AV_FRAME_DATA_SKIP_FRAME type and set its value to 1.
> 
> Signed-off-by: Jing SUN 
> ---
>  libavcodec/vaapi_encode.c | 112 
> --
>  libavcodec/vaapi_encode.h |   5 +++
>  libavutil/frame.c |   1 +
>  libavutil/frame.h |   5 +++
>  4 files changed, 119 insertions(+), 4 deletions(-)

Have a look at 
<https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2019-February/239989.html>, 
which tries to implement this feature in a more general way without adding any 
ad-hoc API.

- Mark
-Original Message-

Hi Mark,

I think the 239989 patch is quite a good one to implement VFR by adapting VAAPI 
frame-skip feature, but it does not fulfill the feature requirement that we 
have been asked to achieve, which is to skip the frames that are selected by 
apps, and the purpose to skip any frames is not just to control frame-rate, but 
also to skip duplicated ones, or to skip abandoned ones due to some changes 
just made, etc. Could we please keep both and let users decide which way they 
want to use?

Regards,
SUN, Jing
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH v2 1/1] avcodec/vaapi_encode: add frame-skip func

2019-02-20 Thread Mark Thompson 
On 20/02/2019 10:33, Jing SUN wrote:
> This implements app controlled frame skipping
> in vaapi encoding. To make a frame skipped,
> allocate its frame side data of the newly
> added AV_FRAME_DATA_SKIP_FRAME type and set
> its value to 1.
> 
> Signed-off-by: Jing SUN 
> ---
>  libavcodec/vaapi_encode.c | 112 
> --
>  libavcodec/vaapi_encode.h |   5 +++
>  libavutil/frame.c |   1 +
>  libavutil/frame.h |   5 +++
>  4 files changed, 119 insertions(+), 4 deletions(-)

Have a look at 
, 
which tries to implement this feature in a more general way without adding any 
ad-hoc API.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

[FFmpeg-devel] [PATCH v2 1/1] avcodec/vaapi_encode: add frame-skip func

2019-02-20 Thread Jing SUN 
This implements app controlled frame skipping
in vaapi encoding. To make a frame skipped,
allocate its frame side data of the newly
added AV_FRAME_DATA_SKIP_FRAME type and set
its value to 1.

Signed-off-by: Jing SUN 
---
 libavcodec/vaapi_encode.c | 112 --
 libavcodec/vaapi_encode.h |   5 +++
 libavutil/frame.c |   1 +
 libavutil/frame.h |   5 +++
 4 files changed, 119 insertions(+), 4 deletions(-)

diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index b4e9fad..debcfa6 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -23,6 +23,7 @@
 #include "libavutil/common.h"
 #include "libavutil/log.h"
 #include "libavutil/pixdesc.h"
+#include "libavutil/intreadwrite.h"
 
 #include "vaapi_encode.h"
 #include "avcodec.h"
@@ -103,6 +104,41 @@ static int vaapi_encode_make_param_buffer(AVCodecContext 
*avctx,
 return 0;
 }
 
+static int vaapi_encode_check_if_skip(AVCodecContext *avctx,
+  VAAPIEncodePicture *pic)
+{
+AVFrameSideData *fside = NULL;
+VAAPIEncodeContext *ctx = avctx->priv_data;
+VAAPIEncodePicture *cur = NULL;
+int i = 0;
+if (!pic || !pic->input_image)
+return AVERROR(EINVAL);
+fside = av_frame_get_side_data(pic->input_image, AV_FRAME_DATA_SKIP_FRAME);
+if (fside)
+pic->skipped_flag = AV_RL8(fside->data);
+else
+pic->skipped_flag = 0;
+if (0 == pic->skipped_flag)
+return 0;
+if ((pic->type == PICTURE_TYPE_IDR) || (pic->type == PICTURE_TYPE_I)) {
+av_log(avctx, AV_LOG_INFO, "Can't skip IDR/I pic 
%"PRId64"/%"PRId64".\n",
+   pic->display_order, pic->encode_order);
+pic->skipped_flag = 0;
+return 0;
+}
+for (cur = ctx->pic_start; cur; cur = cur->next) {
+for (i=0; i < cur->nb_refs; ++i) {
+if (cur->refs[i] == pic) {
+av_log(avctx, AV_LOG_INFO, "Can't skip ref pic 
%"PRId64"/%"PRId64".\n",
+   pic->display_order, pic->encode_order);
+pic->skipped_flag = 0;
+return 0;
+}
+}
+}
+return 0;
+}
+
 static int vaapi_encode_wait(AVCodecContext *avctx,
  VAAPIEncodePicture *pic)
 {
@@ -412,6 +448,50 @@ static int vaapi_encode_issue(AVCodecContext *avctx,
 }
 }
 
+err = vaapi_encode_check_if_skip(avctx, pic);
+if (err != 0)
+av_log(avctx, AV_LOG_ERROR, "Fail to check if skip.\n");
+
+#if VA_CHECK_VERSION(0,38,1)
+if (pic->skipped_flag) {
+av_log(avctx, AV_LOG_INFO, "Skip pic %"PRId64"/%"PRId64" as 
requested.\n",
+   pic->display_order, pic->encode_order);
+++ctx->skipped_pic_count;
+pic->encode_issued = 1;
+return 0;
+} else if (ctx->skipped_pic_count > 0) {
+VAEncMiscParameterBuffer *misc_param = NULL;
+VAEncMiscParameterSkipFrame *skip_param = NULL;
+
+misc_param = av_malloc(sizeof(VAEncMiscParameterBuffer) + 
sizeof(VAEncMiscParameterSkipFrame));
+misc_param->type = 
(VAEncMiscParameterType)VAEncMiscParameterTypeSkipFrame;
+skip_param = (VAEncMiscParameterSkipFrame *)misc_param->data;
+
+skip_param->skip_frame_flag = 1;
+skip_param->num_skip_frames = ctx->skipped_pic_count;
+skip_param->size_skip_frames = 0;
+
+err = vaapi_encode_make_param_buffer(avctx, pic,
+  VAEncMiscParameterBufferType, (void *)misc_param,
+  (sizeof(VAEncMiscParameterBuffer) +
+  sizeof(VAEncMiscParameterSkipFrame)));
+
+free(misc_param);
+
+if (err < 0)
+goto fail;
+
+ctx->skipped_pic_count = 0;
+}
+#else
+if (pic->skipped_flag) {
+av_log(avctx, AV_LOG_INFO, "Skip-frame isn't supported and pic 
%"PRId64"/%"PRId64" isn't skipped.\n",
+   pic->display_order, pic->encode_order);
+pic->skipped_flag = 0;
+ctx->skipped_pic_count = 0;
+}
+#endif
+
 vas = vaBeginPicture(ctx->hwctx->display, ctx->va_context,
  pic->input_surface);
 if (vas != VA_STATUS_SUCCESS) {
@@ -491,9 +571,23 @@ static int vaapi_encode_output(AVCodecContext *avctx,
 VAStatus vas;
 int err;
 
-err = vaapi_encode_wait(avctx, pic);
-if (err < 0)
-return err;
+if (!pic->skipped_flag) {
+err = vaapi_encode_wait(avctx, pic);
+if (err < 0)
+return err;
+} else {
+av_frame_free(>input_image);
+pic->encode_complete = 1;
+err = av_new_packet(pkt, 0);
+if (err < 0)
+goto fail;
+pkt->pts = pic->pts;
+av_buffer_unref(>output_buffer_ref);
+pic->output_buffer = VA_INVALID_ID;
+av_log(avctx, AV_LOG_DEBUG, "Output 0 byte for pic 
%"PRId64"/%"PRId64".\n",
+   pic->display_order, pic->encode_order);
+return 0;
+}
 
 buf_list = NULL;