Re: [FFmpeg-devel] [PATCH v12 15/15] avcodec/hw_base_encode: add avctx pointer for FFHWBaseEncodeContext

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of Lynne
>via ffmpeg-devel
>Sent: Wednesday, May 29, 2024 1:08 AM
>To: ffmpeg-devel@ffmpeg.org
>Cc: Lynne 
>Subject: Re: [FFmpeg-devel] [PATCH v12 15/15] avcodec/hw_base_encode: add
>avctx pointer for FFHWBaseEncodeContext
>
>On 28/05/2024 17:48, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> An avctx pointer is added to FFHWBaseEncodeContext. This is to make
>> FFHWBaseEncodeContext a standalone component for ff_hw_base_*
>functions.
>> This patch also removes some unnecessary AVCodecContext arguments.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>   libavcodec/d3d12va_encode.c |  6 +++---
>>   libavcodec/hw_base_encode.c | 31 +--
>>   libavcodec/hw_base_encode.h |  8 +---
>>   libavcodec/vaapi_encode.c   |  6 +++---
>>   4 files changed, 24 insertions(+), 27 deletions(-)
>>
>> diff --git a/libavcodec/d3d12va_encode.c b/libavcodec/d3d12va_encode.c
>> index 0fbf8eb07c..6d3a53c6ca 100644
>> --- a/libavcodec/d3d12va_encode.c
>> +++ b/libavcodec/d3d12va_encode.c
>> @@ -1351,7 +1351,7 @@ static int
>d3d12va_encode_create_recon_frames(AVCodecContext *avctx)
>>   enum AVPixelFormat recon_format;
>>   int err;
>>
>> -err = ff_hw_base_get_recon_format(avctx, NULL, _format);
>> +err = ff_hw_base_get_recon_format(base_ctx, NULL, _format);
>>   if (err < 0)
>>   return err;
>>
>> @@ -1398,7 +1398,7 @@ int ff_d3d12va_encode_init(AVCodecContext
>*avctx)
>>   int err;
>>   HRESULT hr;
>>
>> -err = ff_hw_base_encode_init(avctx);
>> +err = ff_hw_base_encode_init(avctx, base_ctx);
>>   if (err < 0)
>>   goto fail;
>>
>> @@ -1552,7 +1552,7 @@ int ff_d3d12va_encode_close(AVCodecContext
>*avctx)
>>   D3D12_OBJECT_RELEASE(ctx->video_device3);
>>   D3D12_OBJECT_RELEASE(ctx->device3);
>>
>> -ff_hw_base_encode_close(avctx);
>> +ff_hw_base_encode_close(base_ctx);
>>
>>   return 0;
>>   }
>> diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c
>> index 92f69bb78c..88efdf672c 100644
>> --- a/libavcodec/hw_base_encode.c
>> +++ b/libavcodec/hw_base_encode.c
>> @@ -94,14 +94,13 @@ static void
>hw_base_encode_remove_refs(FFHWBaseEncodePicture *pic, int level)
>>   pic->ref_removed[level] = 1;
>>   }
>>
>> -static void hw_base_encode_set_b_pictures(AVCodecContext *avctx,
>> +static void hw_base_encode_set_b_pictures(FFHWBaseEncodeContext *ctx,
>> FFHWBaseEncodePicture *start,
>> FFHWBaseEncodePicture *end,
>> FFHWBaseEncodePicture *prev,
>> int current_depth,
>> FFHWBaseEncodePicture **last)
>>   {
>> -FFHWBaseEncodeContext *ctx = avctx->priv_data;
>>   FFHWBaseEncodePicture *pic, *next, *ref;
>>   int i, len;
>>
>> @@ -148,20 +147,19 @@ static void
>hw_base_encode_set_b_pictures(AVCodecContext *avctx,
>>   hw_base_encode_add_ref(pic, ref, 0, 1, 0);
>>
>>   if (i > 1)
>> -hw_base_encode_set_b_pictures(avctx, start, pic, pic,
>> +hw_base_encode_set_b_pictures(ctx, start, pic, pic,
>> current_depth + 1, );
>>   else
>>   next = pic;
>>
>> -hw_base_encode_set_b_pictures(avctx, pic, end, next,
>> +hw_base_encode_set_b_pictures(ctx, pic, end, next,
>> current_depth + 1, last);
>>   }
>>   }
>>
>> -static void hw_base_encode_add_next_prev(AVCodecContext *avctx,
>> +static void hw_base_encode_add_next_prev(FFHWBaseEncodeContext
>*ctx,
>>FFHWBaseEncodePicture *pic)
>>   {
>> -FFHWBaseEncodeContext *ctx = avctx->priv_data;
>>   int i;
>>
>>   if (!pic)
>> @@ -333,12 +331,12 @@ static int
>hw_base_encode_pick_next(AVCodecContext *avctx,
>>   }
>>
>>   if (b_counter > 0) {
>> -hw_base_encode_set_b_pictures(avctx, start, pic, pic, 1,
>> +hw_base_encode_set_b_pictures(ctx, start, pic, pic, 1,
>> );
>>   } else {
>>   prev = pic;
>>   }
>> -hw_base_encode_add_next_prev(avctx, prev);
>> +hw_base_encode_add_next_prev(ctx, prev);
>>
>>   return 0;
>>   }
>> @@ -687,9 +685,9 @@ int ff_hw_base_init_gop_structure(AVCodecContext
>*avctx, uint32_t ref_l0, uint32
>>   return 0;
>>   }
>>
>> -int ff_hw_base_get_recon_format(AVCodecContext *avctx, const void
>*hwconfig, enum AVPixelFormat *fmt)
>> +int ff_hw_base_get_recon_format(FFHWBaseEncodeContext *ctx, const
>void *hwconfig,
>> +enum AVPixelFormat *fmt)
>>   {
>> -FFHWBaseEncodeContext *ctx = avctx->priv_data;
>>   AVHWFramesConstraints *constraints = NULL;
>>   enum AVPixelFormat recon_format;
>>   int err, i;
>> @@ -722,14 +720,14 @@ int

Re: [FFmpeg-devel] [PATCH v11 07/14] avcodec/vaapi_encode: extract the init and close function to base layer

[Wu, Tong1]

>From: Lynne 
>Sent: Monday, May 27, 2024 10:04 AM
>To: Wu, Tong1 ; FFmpeg development discussions and
>patches 
>Subject: Re: [FFmpeg-devel] [PATCH v11 07/14] avcodec/vaapi_encode: extract
>the init and close function to base layer
>
>On 27/05/2024 02:35, Wu, Tong1 wrote:
>>> From: ffmpeg-devel  On Behalf Of
>Lynne
>>> via ffmpeg-devel
>>> Sent: Saturday, May 25, 2024 10:07 PM
>>> To: ffmpeg-devel@ffmpeg.org
>>> Cc: Lynne 
>>> Subject: Re: [FFmpeg-devel] [PATCH v11 07/14] avcodec/vaapi_encode:
>extract
>>> the init and close function to base layer
>>>
>>> On 25/05/2024 12:30, tong1.wu-at-intel@ffmpeg.org wrote:
>>>> From: Tong Wu 
>>>>
>>>> Related parameters such as device context, frame context are also moved
>>>> to base layer.
>>>>
>>>> Signed-off-by: Tong Wu 
>>>> ---
>>>>libavcodec/hw_base_encode.c | 49 ++
>>>>libavcodec/hw_base_encode.h | 17 +++
>>>>libavcodec/vaapi_encode.c   | 90 +++--
>>>>libavcodec/vaapi_encode.h   | 10 
>>>>libavcodec/vaapi_encode_av1.c   |  2 +-
>>>>libavcodec/vaapi_encode_h264.c  |  2 +-
>>>>libavcodec/vaapi_encode_h265.c  |  2 +-
>>>>libavcodec/vaapi_encode_mjpeg.c |  6 ++-
>>>>8 files changed, 102 insertions(+), 76 deletions(-)
>>>>
>>>> diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c
>>>> index 16afaa37be..c4789380b6 100644
>>>> --- a/libavcodec/hw_base_encode.c
>>>> +++ b/libavcodec/hw_base_encode.c
>>>> @@ -592,3 +592,52 @@ end:
>>>>
>>>>return 0;
>>>>}
>>>> +
>>>> +int ff_hw_base_encode_init(AVCodecContext *avctx)
>>>> +{
>>>> +FFHWBaseEncodeContext *ctx = avctx->priv_data;
>>>
>>> This is the issue I was talking about, this requires that
>>> FFHWBaseEncodeContext is always the main context.
>>>
>>> Could you change it so everything takes FFHWBaseEncodeContext as an
>>> argument, rather than AVCodecContext (apart from where the function
>>> absolutely must read some data from it)?
>>
>> I'm trying to understand it more.
>>
>> In ff_hw_base_encode_init we also have the following code:
>>
>>  if (!avctx->hw_frames_ctx) {
>>  av_log(avctx, AV_LOG_ERROR, "A hardware frames reference is "
>> "required to associate the encoding device.\n");
>>  return AVERROR(EINVAL);
>>  }
>>
>> In this scenario we still need avctx right? So maybe this is the "must read 
>> data
>from it" situation and we keep AVCodecContext as the main context?
>
>Yup. My point is that FFHWBaseEncodeContext doesn't become the primary
>context that must be used, but a separate component other users can use
>standalone.
>
>> Plus I have this av_log concern which is there's indeed some function that
>the only use of avctx is its av_log's context. Do you think I should instead 
>use
>FFHWBaseEncodeContext as the main context and pass it to av_log for this
>situation while keeping AVCodecContext as the main context for other
>functions which actually read from AVCodecContext. That might lead to
>different av_log context in the same file.
>
>Just save a pointer to avctx on init in FFHWBaseEncodeContext. That's
>what most of our code does.
>
>> Do you think the callbacks in FFHWEncodePictureOperation should be
>changed too? Or only all the functions in hw_base_encode.c should be
>concerned. Thank you.
>
>No, the callbacks should stay as-is. Users need to retrieve their own
>context via avctx. That's also a reason why you should keep a pointer to
>avctx on init.
>

I've updated v12 and made this change with a separate(15th) patch. Please see 
if that is what you wanted. Thank you.

-Tong

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v11 07/14] avcodec/vaapi_encode: extract the init and close function to base layer

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of Lynne
>via ffmpeg-devel
>Sent: Saturday, May 25, 2024 10:07 PM
>To: ffmpeg-devel@ffmpeg.org
>Cc: Lynne 
>Subject: Re: [FFmpeg-devel] [PATCH v11 07/14] avcodec/vaapi_encode: extract
>the init and close function to base layer
>
>On 25/05/2024 12:30, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> Related parameters such as device context, frame context are also moved
>> to base layer.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>   libavcodec/hw_base_encode.c | 49 ++
>>   libavcodec/hw_base_encode.h | 17 +++
>>   libavcodec/vaapi_encode.c   | 90 +++--
>>   libavcodec/vaapi_encode.h   | 10 
>>   libavcodec/vaapi_encode_av1.c   |  2 +-
>>   libavcodec/vaapi_encode_h264.c  |  2 +-
>>   libavcodec/vaapi_encode_h265.c  |  2 +-
>>   libavcodec/vaapi_encode_mjpeg.c |  6 ++-
>>   8 files changed, 102 insertions(+), 76 deletions(-)
>>
>> diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c
>> index 16afaa37be..c4789380b6 100644
>> --- a/libavcodec/hw_base_encode.c
>> +++ b/libavcodec/hw_base_encode.c
>> @@ -592,3 +592,52 @@ end:
>>
>>   return 0;
>>   }
>> +
>> +int ff_hw_base_encode_init(AVCodecContext *avctx)
>> +{
>> +FFHWBaseEncodeContext *ctx = avctx->priv_data;
>
>This is the issue I was talking about, this requires that
>FFHWBaseEncodeContext is always the main context.
>
>Could you change it so everything takes FFHWBaseEncodeContext as an
>argument, rather than AVCodecContext (apart from where the function
>absolutely must read some data from it)?

I'm trying to understand it more.

In ff_hw_base_encode_init we also have the following code:

if (!avctx->hw_frames_ctx) {
av_log(avctx, AV_LOG_ERROR, "A hardware frames reference is "
   "required to associate the encoding device.\n");
return AVERROR(EINVAL);
}

In this scenario we still need avctx right? So maybe this is the "must read 
data from it" situation and we keep AVCodecContext as the main context? 

Plus I have this av_log concern which is there's indeed some function that the 
only use of avctx is its av_log's context. Do you think I should instead use 
FFHWBaseEncodeContext as the main context and pass it to av_log for this 
situation while keeping AVCodecContext as the main context for other functions 
which actually read from AVCodecContext. That might lead to different av_log 
context in the same file.

Do you think the callbacks in FFHWEncodePictureOperation should be changed too? 
Or only all the functions in hw_base_encode.c should be concerned. Thank you.

-Tong


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v9 01/13] avcodec/vaapi_encode: introduce a base layer for vaapi encode

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of Lynne
>via ffmpeg-devel
>Sent: Friday, May 24, 2024 12:11 AM
>To: ffmpeg-devel@ffmpeg.org
>Cc: Lynne 
>Subject: Re: [FFmpeg-devel] [PATCH v9 01/13] avcodec/vaapi_encode:
>introduce a base layer for vaapi encode
>
>On 20/05/2024 16:52, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> Since VAAPI and future D3D12VA implementation may share some common
>parameters,
>> a base layer encode context is introduced as vaapi context's base.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>   libavcodec/hw_base_encode.h | 56
>+
>>   libavcodec/vaapi_encode.h   | 39 +-
>>   2 files changed, 63 insertions(+), 32 deletions(-)
>>   create mode 100644 libavcodec/hw_base_encode.h
>>
>> diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h
>> new file mode 100644
>> index 00..1996179456
>> --- /dev/null
>> +++ b/libavcodec/hw_base_encode.h
>> @@ -0,0 +1,56 @@
>> +/*
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with FFmpeg; if not, write to the Free Software
>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
>USA
>> + */
>> +
>> +#ifndef AVCODEC_HW_BASE_ENCODE_H
>> +#define AVCODEC_HW_BASE_ENCODE_H
>> +
>> +#define MAX_DPB_SIZE 16
>> +#define MAX_PICTURE_REFERENCES 2
>> +#define MAX_REORDER_DELAY 16
>> +#define MAX_ASYNC_DEPTH 64
>> +#define MAX_REFERENCE_LIST_NUM 2
>> +
>> +enum {
>> +PICTURE_TYPE_IDR = 0,
>> +PICTURE_TYPE_I   = 1,
>> +PICTURE_TYPE_P   = 2,
>> +PICTURE_TYPE_B   = 3,
>> +};
>> +
>> +enum {
>> +// Codec supports controlling the subdivision of pictures into slices.
>> +FLAG_SLICE_CONTROL = 1 << 0,
>> +// Codec only supports constant quality (no rate control).
>> +FLAG_CONSTANT_QUALITY_ONLY = 1 << 1,
>> +// Codec is intra-only.
>> +FLAG_INTRA_ONLY= 1 << 2,
>> +// Codec supports B-pictures.
>> +FLAG_B_PICTURES= 1 << 3,
>> +// Codec supports referencing B-pictures.
>> +FLAG_B_PICTURE_REFERENCES  = 1 << 4,
>> +// Codec supports non-IDR key pictures (that is, key pictures do
>> +// not necessarily empty the DPB).
>> +FLAG_NON_IDR_KEY_PICTURES  = 1 << 5,
>> +};
>> +
>> +typedef struct HWBaseEncodeContext {
>> +const AVClass *class;
>> +} HWBaseEncodeContext;
>> +
>> +#endif /* AVCODEC_HW_BASE_ENCODE_H */
>> +
>> diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h
>> index 0eed9691ca..f5c9be8973 100644
>> --- a/libavcodec/vaapi_encode.h
>> +++ b/libavcodec/vaapi_encode.h
>> @@ -33,34 +33,27 @@
>>
>>   #include "avcodec.h"
>>   #include "hwconfig.h"
>> +#include "hw_base_encode.h"
>>
>>   struct VAAPIEncodeType;
>>   struct VAAPIEncodePicture;
>>
>> +// Codec output packet without timestamp delay, which means the
>> +// output packet has same PTS and DTS.
>> +#define FLAG_TIMESTAMP_NO_DELAY 1 << 6
>> +
>>   enum {
>>   MAX_CONFIG_ATTRIBUTES  = 4,
>>   MAX_GLOBAL_PARAMS  = 4,
>> -MAX_DPB_SIZE   = 16,
>> -MAX_PICTURE_REFERENCES = 2,
>> -MAX_REORDER_DELAY  = 16,
>>   MAX_PARAM_BUFFER_SIZE  = 1024,
>>   // A.4.1: table A.6 allows at most 22 tile rows for any level.
>>   MAX_TILE_ROWS  = 22,
>>   // A.4.1: table A.6 allows at most 20 tile columns for any level.
>>   MAX_TILE_COLS  = 20,
>> -MAX_ASYNC_DEPTH= 64,
>> -MAX_REFERENCE_LIST_NUM = 2,
>>   };
>>
>>   extern const AVCodecHWConfigInternal *const
>ff_vaapi_encode_hw_configs[];
>>
>> -enum {
>> -PICTURE_TYPE_IDR = 0,
>> -PICTURE_TYPE_I   = 1,
>> -PICTURE_TYPE_P   = 2,
>> -PICTURE_TYPE_B   = 3,
>> -};
>> -
>>   typedef struct VAAPIEncodeSlice {
>>   int index;
>>   int row_start;
>> @@ -193,7 +186,8 @@ typedef struct VAAPIEncodeRCMode {
>>   } VAAPIEncodeRCMode;
>>
>>   typedef struct VAAPIEncodeContext {
>> -const AVClass *class;
>> +// Base context.
>> +HWBaseEncodeContext base;
>>
>>   // Codec-specific hooks.
>>   const struct VAAPIEncodeType *codec;
>> @@ -397,25 +391,6 @@ typedef struct VAAPIEncodeContext {
>>   AVPacket*tail_pkt;
>>   } VAAPIEncodeContext;
>>
>> -enum {
>> -// Codec supports controlling the subdivision of pictures into slices.
>> -FLAG_SLICE_CONTROL   

Re: [FFmpeg-devel] [PATCH v9 11/13] avutil/hwcontext_d3d12va: add Flags for resource creation

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of
>Andrew Sayers
>Sent: Wednesday, May 22, 2024 12:32 AM
>To: FFmpeg development discussions and patches de...@ffmpeg.org>
>Subject: Re: [FFmpeg-devel] [PATCH v9 11/13] avutil/hwcontext_d3d12va: add
>Flags for resource creation
>
>(Only reviewing documentation, not code)
>
>On Mon, May 20, 2024 at 10:52:20PM +0800, tong1.wu-at-
>intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> Flags field is added to support diffferent resource creation.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>  doc/APIchanges| 3 +++
>>  libavutil/hwcontext_d3d12va.c | 2 +-
>>  libavutil/hwcontext_d3d12va.h | 8 
>>  libavutil/version.h   | 2 +-
>>  4 files changed, 13 insertions(+), 2 deletions(-)
>>
>> diff --git a/doc/APIchanges b/doc/APIchanges
>> index 269fd36559..808ba02f2d 100644
>> --- a/doc/APIchanges
>> +++ b/doc/APIchanges
>> @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-
>07
>>
>>  API changes, most recent first:
>>
>> +2024-01-xx - xx - lavu 59.20.100 - hwcontext_d3d12va.h
>> + Add AVD3D12VAFramesContext.flags
>> +
>>  2024-05-xx - xx - lavu 59.19.100 - hwcontext_qsv.h
>>Add AVQSVFramesContext.info
>>
>> diff --git a/libavutil/hwcontext_d3d12va.c b/libavutil/hwcontext_d3d12va.c
>> index cfc016315d..6507cf69c1 100644
>> --- a/libavutil/hwcontext_d3d12va.c
>> +++ b/libavutil/hwcontext_d3d12va.c
>> @@ -247,7 +247,7 @@ static AVBufferRef *d3d12va_pool_alloc(void
>*opaque, size_t size)
>>  .Format   = hwctx->format,
>>  .SampleDesc   = {.Count = 1, .Quality = 0 },
>>  .Layout   = D3D12_TEXTURE_LAYOUT_UNKNOWN,
>> -.Flags= D3D12_RESOURCE_FLAG_NONE,
>> +.Flags= hwctx->flags,
>>  };
>>
>>  frame = av_mallocz(sizeof(AVD3D12VAFrame));
>> diff --git a/libavutil/hwcontext_d3d12va.h b/libavutil/hwcontext_d3d12va.h
>> index ff06e6f2ef..608dbac97f 100644
>> --- a/libavutil/hwcontext_d3d12va.h
>> +++ b/libavutil/hwcontext_d3d12va.h
>> @@ -129,6 +129,14 @@ typedef struct AVD3D12VAFramesContext {
>>   * If unset, will be automatically set.
>>   */
>>  DXGI_FORMAT format;
>> +
>> +/**
>> + * This field is used to specify options for working with resources.
>> + * If unset, this will be D3D12_RESOURCE_FLAG_NONE.
>> + *
>> + * @see: https://learn.microsoft.com/en-
>us/windows/win32/api/d3d12/ne-d3d12-d3d12_resource_flags.
>> + */
>> +D3D12_RESOURCE_FLAGS flags;
>
>Some nitpicks:
>
>* "This field is used to specify" is redundant, you can save the reader
>  a few seconds by starting the sentence with just "Options..."
>* "@see" starts a paragraph, so the rendered documentation will look better
>  without the ":"
>* the full stop after the URL makes it harder to copy/paste the text -
>  remove the full stop or use a [markdown link](...)

Sounds good. I've updated it with a new version.

-Tong


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 12/15] avcodec/vaapi_encode: extract a free funtion to base layer

[Wu, Tong1]

>Subject: Re: [FFmpeg-devel] [PATCH v8 12/15] avcodec/vaapi_encode: extract a
>free funtion to base layer
>
>On 18/04/2024 09:59, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> Signed-off-by: Tong Wu 
>> ---
>>  libavcodec/hw_base_encode.c | 11 +++
>>  libavcodec/hw_base_encode.h |  2 ++
>>  libavcodec/vaapi_encode.c   |  6 +-
>>  3 files changed, 14 insertions(+), 5 deletions(-)
>
>"... free funtion to ..."
>
>While I do approve of fun, maybe this should be a function.
>

Hi Mark,

I've sent patch v9 which addressed multiple concerns of yours.
1. AVClass definition is put in the first patch.
2. Used "base_foo = >base_field"
3. Used vaapi_pic and pic instead of pic and base_pic in vaapi_encode_*.c to 
simplify code lines.
4. Squashed init and close into one patch.

However, I did not change the way for .alloc and .free since I feel like the 
malloc(sizeof(VAAPIEncodePicture)) and ctx->codec are both part of vaapi 
context and should not be moved to base. 
As for the .free function I have this patch to extract a base layer free 
function which avoid the mix of two layer parameters. 

Feel free to comment on V9. Thanks.

-Tong



___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avutil/hwcontext_qsv: fix GCC 14.1 warnings

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of Oleg
>Tolmatcev
>Sent: Saturday, May 18, 2024 3:39 AM
>To: ffmpeg-devel@ffmpeg.org
>Subject: [FFmpeg-devel] [PATCH] avutil/hwcontext_qsv: fix GCC 14.1 warnings
>
>This patch fixes warnings produced by GCC 14.1 in hwcontext_qsv.c. It
>fixes the issue https://trac.ffmpeg.org/ticket/11004.
>

Patch works for me. Thanks.

-Tong

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 11/15] avcodec/vaapi_encode: extract a get_recon_format function to base layer

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of Mark
>Thompson
>Sent: Wednesday, May 15, 2024 5:06 AM
>To: ffmpeg-devel@ffmpeg.org
>Subject: Re: [FFmpeg-devel] [PATCH v8 11/15] avcodec/vaapi_encode: extract a
>get_recon_format function to base layer
>
>On 18/04/2024 09:59, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> Surface size and block size parameters are also moved to base layer.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>  libavcodec/hw_base_encode.c | 58 +++
>>  libavcodec/hw_base_encode.h | 12 +
>>  libavcodec/vaapi_encode.c   | 81 -
>>  libavcodec/vaapi_encode.h   | 10 
>>  libavcodec/vaapi_encode_av1.c   | 10 ++--
>>  libavcodec/vaapi_encode_h264.c  | 11 +++--
>>  libavcodec/vaapi_encode_h265.c  | 25 +-
>>  libavcodec/vaapi_encode_mjpeg.c |  5 +-
>>  libavcodec/vaapi_encode_vp9.c   |  6 +--
>>  9 files changed, 118 insertions(+), 100 deletions(-)
>>
>> diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c
>> index a4223d90f0..af85bb99aa 100644
>> --- a/libavcodec/hw_base_encode.c
>> +++ b/libavcodec/hw_base_encode.c
>> @@ -693,6 +693,64 @@ int ff_hw_base_init_gop_structure(AVCodecContext
>*avctx, uint32_t ref_l0, uint32
>>  return 0;
>>  }
>>
>> +int ff_hw_base_get_recon_format(AVCodecContext *avctx, const void
>*hwconfig, enum AVPixelFormat *fmt)
>> +{
>> +HWBaseEncodeContext *ctx = avctx->priv_data;
>> +AVHWFramesConstraints *constraints = NULL;
>> +enum AVPixelFormat recon_format;
>> +int err, i;
>> +
>> +constraints = av_hwdevice_get_hwframe_constraints(ctx->device_ref,
>> +  hwconfig);
>
>Does this mechanism actually make sense for D3D12?
>
>VAAPI is the currently the only implementation of this function with non-null
>hwconfig, and this is really relying on it to get useful information (otherwise
>the formats are just everything the device can allocate as a surface and the
>sizes are 0/INT_MAX).
>
>If D3D12 has something which would fit into the hwconfig method then this
>could work very nicely as well, but if it doesn't then presumably it does have
>some other calls to check things like the maximum frame size supported by the
>encoder and we should be using those rather than making this code generic?
>
>(Also consider Vulkan if possible; if two thirds of the cases want it then 
>maybe
>we should do this even if it doesn't fit in one of them.)

For Vulkan the constrains also includes the maximum frame size stuff so I guess 
this mechanism also works for Vulkan. If so shall we keep this patch? I'm ok 
for keeping or dropping this patch. And for D3D12 we should probably add the 
max frame size check afterwards.

>
>> +if (!constraints) {
>> +err = AVERROR(ENOMEM);
>> +goto fail;
>> +}
>> +
>> +// Probably we can use the input surface format as the surface format
>> +// of the reconstructed frames.  If not, we just pick the first (only?)
>> +// format in the valid list and hope that it all works.
>> +recon_format = AV_PIX_FMT_NONE;
>> +if (constraints->valid_sw_formats) {
>> +for (i = 0; constraints->valid_sw_formats[i] != AV_PIX_FMT_NONE; 
>> i++) {
>> +if (ctx->input_frames->sw_format ==
>> +constraints->valid_sw_formats[i]) {
>> +recon_format = ctx->input_frames->sw_format;
>> +break;
>> +}
>> +}
>> +if (recon_format == AV_PIX_FMT_NONE) {
>> +// No match.  Just use the first in the supported list and
>> +// hope for the best.
>> +recon_format = constraints->valid_sw_formats[0];
>> +}
>> +} else {
>> +// No idea what to use; copy input format.
>> +recon_format = ctx->input_frames->sw_format;
>> +}
>> +av_log(avctx, AV_LOG_DEBUG, "Using %s as format of "
>> +   "reconstructed frames.\n", av_get_pix_fmt_name(recon_format));
>> +
>> +if (ctx->surface_width  < constraints->min_width  ||
>> +ctx->surface_height < constraints->min_height ||
>> +ctx->surface_width  > constraints->max_width ||
>> +ctx->surface_height > constraints->max_height) {
>> +av_log(avctx, AV_LOG_ERROR, "Hardware does not support encoding at
>"
>> +   "size %dx%d (constraints: width %d-%d height %d-%d).\n",
>> +   ctx->surface_width, ctx->surface_height,
>> +   constraints->min_width,  constraints->max_width,
>> +   constraints->min_height, constraints->max_height);
>> +err = AVERROR(EINVAL);
>> +goto fail;
>> +}
>> +
>> +*fmt = recon_format;
>> +err = 0;
>> +fail:
>> +av_hwframe_constraints_free();
>> +return err;
>> +}
>> +
>>  int ff_hw_base_encode_init(AVCodecContext *avctx)
>>  {
>>  HWBaseEncodeContext *ctx = avctx->priv_data;
>> diff --git a/libavcodec/hw_base_encode.h 

Re: [FFmpeg-devel] [PATCH v8 06/15] avcodec/vaapi_encode: extract the init function to base layer

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of Mark
>Thompson
>Sent: Wednesday, May 15, 2024 4:56 AM
>To: ffmpeg-devel@ffmpeg.org
>Subject: Re: [FFmpeg-devel] [PATCH v8 06/15] avcodec/vaapi_encode: extract
>the init function to base layer
>
>On 18/04/2024 09:59, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> Related parameters are also moved to base layer.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>  libavcodec/hw_base_encode.c | 33 
>>  libavcodec/hw_base_encode.h | 11 ++
>>  libavcodec/vaapi_encode.c   | 68 ++---
>>  libavcodec/vaapi_encode.h   |  6 ---
>>  libavcodec/vaapi_encode_av1.c   |  2 +-
>>  libavcodec/vaapi_encode_h264.c  |  2 +-
>>  libavcodec/vaapi_encode_h265.c  |  2 +-
>>  libavcodec/vaapi_encode_mjpeg.c |  6 ++-
>>  8 files changed, 72 insertions(+), 58 deletions(-)
>>
>> diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c
>> index 1d9a255f69..14f3ecfc94 100644
>> --- a/libavcodec/hw_base_encode.c
>> +++ b/libavcodec/hw_base_encode.c
>> @@ -598,3 +598,36 @@ end:
>>
>>  return 0;
>>  }
>> +
>> +int ff_hw_base_encode_init(AVCodecContext *avctx)
>> +{
>> +HWBaseEncodeContext *ctx = avctx->priv_data;
>> +
>> +ctx->frame = av_frame_alloc();
>> +if (!ctx->frame)
>> +return AVERROR(ENOMEM);
>> +
>> +if (!avctx->hw_frames_ctx) {
>> +av_log(avctx, AV_LOG_ERROR, "A hardware frames reference is "
>> +   "required to associate the encoding device.\n");
>> +return AVERROR(EINVAL);
>> +}
>> +
>> +ctx->input_frames_ref = av_buffer_ref(avctx->hw_frames_ctx);
>> +if (!ctx->input_frames_ref)
>> +return AVERROR(ENOMEM);
>> +
>> +ctx->input_frames = (AVHWFramesContext *)ctx->input_frames_ref-
>>data;
>> +
>> +ctx->device_ref = av_buffer_ref(ctx->input_frames->device_ref);
>> +if (!ctx->device_ref)
>> +return AVERROR(ENOMEM);
>> +
>> +ctx->device = (AVHWDeviceContext *)ctx->device_ref->data;
>> +
>> +ctx->tail_pkt = av_packet_alloc();
>> +if (!ctx->tail_pkt)
>> +return AVERROR(ENOMEM);
>> +
>> +return 0;
>> +}
>> diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h
>> index b5b676b9a8..f7e385e840 100644
>> --- a/libavcodec/hw_base_encode.h
>> +++ b/libavcodec/hw_base_encode.h
>> @@ -19,6 +19,7 @@
>>  #ifndef AVCODEC_HW_BASE_ENCODE_H
>>  #define AVCODEC_HW_BASE_ENCODE_H
>>
>> +#include "libavutil/hwcontext.h"
>>  #include "libavutil/fifo.h"
>>
>>  #define MAX_DPB_SIZE 16
>> @@ -117,6 +118,14 @@ typedef struct HWBaseEncodeContext {
>>  // Hardware-specific hooks.
>>  const struct HWEncodePictureOperation *op;
>>
>> +// The hardware device context.
>> +AVBufferRef*device_ref;
>> +AVHWDeviceContext *device;
>> +
>> +// The hardware frame context containing the input frames.
>> +AVBufferRef*input_frames_ref;
>> +AVHWFramesContext *input_frames;
>> +
>>  // Current encoding window, in display (input) order.
>>  HWBaseEncodePicture *pic_start, *pic_end;
>>  // The next picture to use as the previous reference picture in
>> @@ -183,6 +192,8 @@ typedef struct HWBaseEncodeContext {
>>
>>  int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket
>*pkt);
>>
>> +int ff_hw_base_encode_init(AVCodecContext *avctx);
>> +
>>  #define HW_BASE_ENCODE_COMMON_OPTIONS \
>>  { "async_depth", "Maximum processing parallelism. " \
>>"Increase this to improve single channel performance.", \
>
>Maybe this patch should be merged with 9/15 to keep the init/close symmetry?
>It's not clear that the intermediate makes sense, and it has some churn.
>

Will do it.

-Tong
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 05/15] avcodec/vaapi_encode: move the dpb logic from VAAPI to base layer

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of Mark
>Thompson
>Sent: Wednesday, May 15, 2024 4:47 AM
>To: ffmpeg-devel@ffmpeg.org
>Subject: Re: [FFmpeg-devel] [PATCH v8 05/15] avcodec/vaapi_encode: move
>the dpb logic from VAAPI to base layer
>
>On 18/04/2024 09:58, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> Move receive_packet function to base. This requires adding *alloc,
>> *issue, *output, *free as hardware callbacks. HWBaseEncodePicture is
>> introduced as the base layer structure. The related parameters in
>> VAAPIEncodeContext are also extracted to HWBaseEncodeContext. Then DPB
>> management logic can be fully extracted to base layer as-is.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>  libavcodec/Makefile |   2 +-
>>  libavcodec/hw_base_encode.c | 600 
>>  libavcodec/hw_base_encode.h | 123 +
>>  libavcodec/vaapi_encode.c   | 793 +---
>>  libavcodec/vaapi_encode.h   | 102 +---
>>  libavcodec/vaapi_encode_av1.c   |  51 +-
>>  libavcodec/vaapi_encode_h264.c  | 176 +++
>>  libavcodec/vaapi_encode_h265.c  | 121 ++---
>>  libavcodec/vaapi_encode_mjpeg.c |   7 +-
>>  libavcodec/vaapi_encode_mpeg2.c |  47 +-
>>  libavcodec/vaapi_encode_vp8.c   |  18 +-
>>  libavcodec/vaapi_encode_vp9.c   |  54 +--
>>  12 files changed, 1097 insertions(+), 997 deletions(-)
>>  create mode 100644 libavcodec/hw_base_encode.c
>>
>> diff --git a/libavcodec/Makefile b/libavcodec/Makefile
>> index 7f6de4470e..a2174dcb2f 100644
>> --- a/libavcodec/Makefile
>> +++ b/libavcodec/Makefile
>> @@ -162,7 +162,7 @@ OBJS-$(CONFIG_STARTCODE)   += startcode.o
>>  OBJS-$(CONFIG_TEXTUREDSP)  += texturedsp.o
>>  OBJS-$(CONFIG_TEXTUREDSPENC)   += texturedspenc.o
>>  OBJS-$(CONFIG_TPELDSP) += tpeldsp.o
>> -OBJS-$(CONFIG_VAAPI_ENCODE)+= vaapi_encode.o
>> +OBJS-$(CONFIG_VAAPI_ENCODE)+= vaapi_encode.o
>hw_base_encode.o
>>  OBJS-$(CONFIG_AV1_AMF_ENCODER) += amfenc_av1.o
>>  OBJS-$(CONFIG_VC1DSP)  += vc1dsp.o
>>  OBJS-$(CONFIG_VIDEODSP)+= videodsp.o
>> diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c
>> new file mode 100644
>> index 00..1d9a255f69
>> --- /dev/null
>> +++ b/libavcodec/hw_base_encode.c
>> @@ -0,0 +1,600 @@
>> +/*
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with FFmpeg; if not, write to the Free Software
>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
>USA
>> + */
>> +
>> +#include "libavutil/avassert.h"
>> +#include "libavutil/common.h"
>> +#include "libavutil/internal.h"
>> +#include "libavutil/log.h"
>> +#include "libavutil/mem.h"
>> +#include "libavutil/pixdesc.h"
>> +
>> +#include "encode.h"
>> +#include "avcodec.h"
>> +#include "hw_base_encode.h"
>> +
>> ...
>
>Everything above here looks like a copy of the VAAPI code with the names
>changed, good if so.  (If not then please highlight any differences.)
>
>> +
>> +static int hw_base_encode_send_frame(AVCodecContext *avctx, AVFrame
>*frame)
>> +{
>> +HWBaseEncodeContext *ctx = avctx->priv_data;
>> +HWBaseEncodePicture *pic;
>> +int err;
>> +
>> +if (frame) {
>> +av_log(avctx, AV_LOG_DEBUG, "Input frame: %ux%u (%"PRId64").\n",
>> +   frame->width, frame->height, frame->pts);
>> +
>> +err = hw_base_encode_check_frame(avctx, frame);
>> +if (err < 0)
>> +return err;
>> +
>> +pic = ctx->op->alloc(avctx, frame);
>> +if (!pic)
>> +return AVERROR(ENOMEM);
>
>Can you push the allocation of this picture out into the base layer?
>vaapi_encode_alloc() and d3d12va_encode_alloc() are identical except for the
>types and the input_surface setting.

You still need to know the API-specific picture type size for allocation right? 
I could add a hw_encode_alloc() which calls API-specific alloc() to alloc with 
the specific picture type size and move priv_data to the base. Or the picture 
type size has to be part of the callback structure.

>
>> +
>> +pic->input_image = av_frame_alloc();
>> +if (!pic->input_image) {
>> +err = AVERROR(ENOMEM);
>> +goto fail;
>> +}
>> +
>> +pic->recon_image = av_frame_alloc();
>> +if (!pic->recon_image) 

Re: [FFmpeg-devel] [PATCH v8 01/15] avcodec/vaapi_encode: introduce a base layer for vaapi encode

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of Mark
>Thompson
>Sent: Wednesday, May 15, 2024 3:56 AM
>To: ffmpeg-devel@ffmpeg.org
>Subject: Re: [FFmpeg-devel] [PATCH v8 01/15] avcodec/vaapi_encode:
>introduce a base layer for vaapi encode
>
>On 18/04/2024 09:58, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> Since VAAPI and future D3D12VA implementation may share some common
>parameters,
>> a base layer encode context is introduced as vaapi context's base.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>  libavcodec/hw_base_encode.h | 52
>+
>>  libavcodec/vaapi_encode.h   | 36 -
>>  2 files changed, 57 insertions(+), 31 deletions(-)
>>  create mode 100644 libavcodec/hw_base_encode.h
>>
>> diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h
>> new file mode 100644
>> index 00..3d1974bba3
>> --- /dev/null
>> +++ b/libavcodec/hw_base_encode.h
>> @@ -0,0 +1,52 @@
>> +/*
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with FFmpeg; if not, write to the Free Software
>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
>USA
>> + */
>> +
>> +#ifndef AVCODEC_HW_BASE_ENCODE_H
>> +#define AVCODEC_HW_BASE_ENCODE_H
>> +
>> +#define MAX_DPB_SIZE 16
>> +#define MAX_PICTURE_REFERENCES 2
>> +#define MAX_REORDER_DELAY 16
>> +#define MAX_ASYNC_DEPTH 64
>> +#define MAX_REFERENCE_LIST_NUM 2
>
>Is there a reason to change these from enum to defines?  I'm not seeing
>anywhere they should be visible to the preprocessor, and this means they are
>normally invisible to a debugger.

It was a enum structure before V7 until Lynne suggested "Define these, don't 
enum unnecessarily".

>
>> +
>> +enum {
>> +PICTURE_TYPE_IDR = 0,
>> +PICTURE_TYPE_I   = 1,
>> +PICTURE_TYPE_P   = 2,
>> +PICTURE_TYPE_B   = 3,
>> +};
>> +
>> +enum {
>> +// Codec supports controlling the subdivision of pictures into slices.
>> +FLAG_SLICE_CONTROL = 1 << 0,
>> +// Codec only supports constant quality (no rate control).
>> +FLAG_CONSTANT_QUALITY_ONLY = 1 << 1,
>> +// Codec is intra-only.
>> +FLAG_INTRA_ONLY= 1 << 2,
>> +// Codec supports B-pictures.
>> +FLAG_B_PICTURES= 1 << 3,
>> +// Codec supports referencing B-pictures.
>> +FLAG_B_PICTURE_REFERENCES  = 1 << 4,
>> +// Codec supports non-IDR key pictures (that is, key pictures do
>> +// not necessarily empty the DPB).
>> +FLAG_NON_IDR_KEY_PICTURES  = 1 << 5,
>> +};
>> +
>> +#endif /* AVCODEC_HW_BASE_ENCODE_H */
>> ...
>
>Would it make more sense to put the HWBaseEncodeContext in this patch as
>well?  (With just the AVClass member.)
>

Yes and will do it.

Thanks,
Tong
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v8 14/15] avcodec: add D3D12VA hardware HEVC encoder

[Wu, Tong1]

>From: Wu, Tong1 
>Sent: Thursday, April 18, 2024 4:59 PM
>To: ffmpeg-devel@ffmpeg.org
>Cc: Wu, Tong1 
>Subject: [FFmpeg-devel][PATCH v8 14/15] avcodec: add D3D12VA hardware
>HEVC encoder
>
>From: Tong Wu 
>
>This implementation is based on D3D12 Video Encoding Spec:
>https://microsoft.github.io/DirectX-Specs/d3d/D3D12VideoEncoding.html
>
>Sample command line for transcoding:
>ffmpeg.exe -hwaccel d3d12va -hwaccel_output_format d3d12 -i input.mp4
>-c:v hevc_d3d12va output.mp4
>
>Signed-off-by: Tong Wu 

Kindly ping again. Since this patch set has been in the mail list for quite a 
while, can we push it if there's no more objections?

Tong
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avutil/hwcontext_d3d12va: wait the texture is used before to free it.

[Wu, Tong1]

Hi,

>From: ffmpeg-devel  On Behalf Of Renan
>Lavarec via ffmpeg-devel
>Sent: Monday, April 8, 2024 11:27 PM
>To: ffmpeg-devel@ffmpeg.org
>Cc: Renan Lavarec 
>Subject: [FFmpeg-devel] [PATCH] avutil/hwcontext_d3d12va: wait the texture is
>used before to free it.
>
>From: Renan Lavarec 124602499+rlavarec-
>g...@users.noreply.github.comg...@users.noreply.github.com>
>Date: Mon, 8 Apr 2024 14:38:10 +0200
>Subject: [PATCH] avutil/hwcontext_d3d12va: wait the texture is used inside the
>GPU before to free it.
>
>fix: ID3D12Resource2::: CORRUPTION: An ID3D12Resource
>object (0x0222D58B5450:'Unnamed Object') is referenced by GPU
>operations in-flight on Command Queue (0x0222EEC87090:'Unnamed
>ID3D12CommandQueue Object').
> It is not safe to final-release objects that may have GPU operations 
> pending.
>This can result in application instability. [ EXECUTION ERROR #921:
>OBJECT_DELETED_WHILE_STILL_IN_USE]
>---
>libavutil/hwcontext_d3d12va.c | 3 +++
>1 file changed, 3 insertions(+)
>
>diff --git a/libavutil/hwcontext_d3d12va.c b/libavutil/hwcontext_d3d12va.c
>index cfc016315d..621a79d257 100644
>--- a/libavutil/hwcontext_d3d12va.c
>+++ b/libavutil/hwcontext_d3d12va.c
>@@ -220,6 +220,9 @@ static void free_texture(void *opaque, uint8_t *data)
>{
> AVD3D12VAFrame *frame = (AVD3D12VAFrame *)data;
>
>+// Wait texture to be available
>+d3d12va_fence_completion(>sync_ctx);
>+
> D3D12_OBJECT_RELEASE(frame->texture);
> D3D12_OBJECT_RELEASE(frame->sync_ctx.fence);
> if (frame->sync_ctx.event)
>--
>2.44.0.windows.1

At what scenario will this happen? I think decoder should be responsible for 
the completion of the CommandQueue before releasing the buffer pool. And that's 
what d3d12va_decode has already done in uninit call. Thanks.

-Tong 
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] avcodec/hevc_ps: fix the problem of memcmp losing effectiveness

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of James
>Almer
>Sent: Friday, March 29, 2024 11:18 PM
>To: ffmpeg-devel@ffmpeg.org
>Subject: Re: [FFmpeg-devel] [PATCH v2] avcodec/hevc_ps: fix the problem of
>memcmp losing effectiveness
>
>On 3/29/2024 12:13 PM, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> HEVCHdrParams* receives a pointer which points to a dynamically
>> allocated memory block. It causes the memcmp always returning 1.
>> Add a function to do the comparision. A condition is also added to
>> avoid malloc(0).
>>
>> Signed-off-by: Tong Wu 
>> ---
>>   libavcodec/hevc_ps.c | 22 ++
>>   libavcodec/hevc_ps.h |  4 +++-
>>   2 files changed, 21 insertions(+), 5 deletions(-)
>>
>> diff --git a/libavcodec/hevc_ps.c b/libavcodec/hevc_ps.c
>> index cbef3ef4cd..d3c589ec24 100644
>> --- a/libavcodec/hevc_ps.c
>> +++ b/libavcodec/hevc_ps.c
>> @@ -449,6 +449,18 @@ static void uninit_vps(FFRefStructOpaque opaque,
>void *obj)
>>   av_freep(>hdr);
>>   }
>>
>> +static int compare_vps(const HEVCVPS *vps1, const HEVCVPS *vps2)
>> +{
>> +if (!vps1->hdr && !vps2->hdr && !memcmp(vps1, vps2, offsetof(HEVCVPS,
>hdr)))
>> +return 1;
>> +
>> +if (vps1->hdr && vps2->hdr && !memcmp(vps1, vps2, offsetof(HEVCVPS,
>hdr)) &&
>> +!memcmp(vps1->hdr, vps2->hdr, vps1->vps_num_hrd_parameters *
>sizeof(*vps1->hdr)))
>> +return 1;
>> +
>> +return 0;
>> +}
>
>Something like this should be simpler
>
>> +static int compare_vps(const HEVCVPS *vps1, const HEVCVPS *vps2)
>> +{
>> +if (!memcmp(vps1, vps2, offsetof(HEVCVPS, hdr)))
>> +return !vps1->vps_num_hrd_parameters ||
>> +   !memcmp(vps1->hdr, vps2->hdr, vps1->vps_num_hrd_parameters *
>sizeof(*vps1->hdr));
>> +
>> +return 0;
>> +}
>
>If vps1 is equal to vps2 up to hdr, then vps_num_hrd_parameters will be
>the same in both and you can safely just go ahead with the memcmp for hdr.

That makes sense. Thank you.

>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/hevc_ps: fix the problem of memcmp losing effectiveness

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of James
>Almer
>Sent: Friday, March 29, 2024 8:46 PM
>To: ffmpeg-devel@ffmpeg.org
>Subject: Re: [FFmpeg-devel] [PATCH] avcodec/hevc_ps: fix the problem of
>memcmp losing effectiveness
>
>On 3/28/2024 10:15 AM, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> HEVCHdrParams* receives a pointer which points to a dynamically
>> allocated memory block. It causes the memcmp always returning 1.
>> Add a function to do the comparision. A condition is also added to
>> avoid malloc(0).
>>
>> Signed-off-by: Tong Wu 
>> ---
>>   libavcodec/hevc_ps.c | 20 
>>   libavcodec/hevc_ps.h |  4 +++-
>>   2 files changed, 19 insertions(+), 5 deletions(-)
>>
>> diff --git a/libavcodec/hevc_ps.c b/libavcodec/hevc_ps.c
>> index cbef3ef4cd..8b3a27a17c 100644
>> --- a/libavcodec/hevc_ps.c
>> +++ b/libavcodec/hevc_ps.c
>> @@ -449,6 +449,16 @@ static void uninit_vps(FFRefStructOpaque opaque,
>void *obj)
>>   av_freep(>hdr);
>>   }
>>
>> +static int compare_vps(const HEVCVPS *vps1, const HEVCVPS *vps2)
>> +{
>> +if ((!vps1->hdr && !vps2->hdr) ||
>> +(vps1->hdr && vps2->hdr && !memcmp(vps1->hdr, vps2->hdr,
>sizeof(*vps1->hdr {
>
>I think this should be vps1->vps_num_hrd_parameters *
>sizeof(*vps1->hdr), and done after the memcmp below to ensure
>vps_num_hrd_parameters is the same value in both structs.

Updated as suggested. Thanks.

Tong

>
>> +return !memcmp(vps1, vps2, offsetof(HEVCVPS, hdr));
>> +}
>> +
>> +return 0;
>> +}
>> +
>>   int ff_hevc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx,
>>  HEVCParamSets *ps)
>>   {
>> @@ -545,9 +555,11 @@ int ff_hevc_decode_nal_vps(GetBitContext *gb,
>AVCodecContext *avctx,
>>   goto err;
>>   }
>>
>> -vps->hdr = av_calloc(vps->vps_num_hrd_parameters, sizeof(*vps-
>>hdr));
>> -if (!vps->hdr)
>> -goto err;
>> +if (vps->vps_num_hrd_parameters) {
>> +vps->hdr = av_calloc(vps->vps_num_hrd_parameters, sizeof(*vps-
>>hdr));
>> +if (!vps->hdr)
>> +goto err;
>> +}
>>
>>   for (i = 0; i < vps->vps_num_hrd_parameters; i++) {
>>   int common_inf_present = 1;
>> @@ -569,7 +581,7 @@ int ff_hevc_decode_nal_vps(GetBitContext *gb,
>AVCodecContext *avctx,
>>   }
>>
>>   if (ps->vps_list[vps_id] &&
>> -!memcmp(ps->vps_list[vps_id], vps, sizeof(*vps))) {
>> +compare_vps(ps->vps_list[vps_id], vps)) {
>>   ff_refstruct_unref();
>>   } else {
>>   remove_vps(ps, vps_id);
>> diff --git a/libavcodec/hevc_ps.h b/libavcodec/hevc_ps.h
>> index cc75aeb8d3..0d8eaf2b3e 100644
>> --- a/libavcodec/hevc_ps.h
>> +++ b/libavcodec/hevc_ps.h
>> @@ -153,7 +153,6 @@ typedef struct PTL {
>>
>>   typedef struct HEVCVPS {
>>   unsigned int vps_id;
>> -HEVCHdrParams *hdr;
>>
>>   uint8_t vps_temporal_id_nesting_flag;
>>   int vps_max_layers;
>> @@ -175,6 +174,9 @@ typedef struct HEVCVPS {
>>
>>   uint8_t data[4096];
>>   int data_size;
>> +/* Put this at the end of the structure to make it easier to calculate 
>> the
>> + * size before this pointer, which is used for memcmp */
>> +HEVCHdrParams *hdr;
>>   } HEVCVPS;
>>
>>   typedef struct ScalingList {
>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/hevc_ps: fix the problem of memcmp losing effectiveness

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of
>Andreas Rheinhardt
>Sent: Friday, March 29, 2024 10:03 PM
>To: ffmpeg-devel@ffmpeg.org
>Subject: Re: [FFmpeg-devel] [PATCH] avcodec/hevc_ps: fix the problem of
>memcmp losing effectiveness
>
>Mark Thompson:
>> On 28/03/2024 13:15, tong1.wu-at-intel@ffmpeg.org wrote:
>>> From: Tong Wu 
>>>
>>> HEVCHdrParams* receives a pointer which points to a dynamically
>>> allocated memory block. It causes the memcmp always returning 1.
>>> Add a function to do the comparision. A condition is also added to
>>> avoid malloc(0).
>>>
>>> Signed-off-by: Tong Wu 
>>> ---
>>>   libavcodec/hevc_ps.c | 20 
>>>   libavcodec/hevc_ps.h |  4 +++-
>>>   2 files changed, 19 insertions(+), 5 deletions(-)
>>
>> It doesn't seem like this method works at all, even before the recent
>> change with the pointer.
>>
>> Structs can contain arbitrary padding, and any write to the struct makes
>> the padding unspecified.  memcmp() is therefore never valid as a method
>> of comparing after writing some fields, as done here.  (It could only be
>> valid if the structs compared were made by memcpy() with no fields
>> written directly.)
>>
>> The problem is mostly harmless because the nondeterministic replacement
>> of structs which we were expecting to be equivalent doesn't actually
>> change anything, so why don't we just remove the comparison and always
>> replace?
>>
>
>remove_vps() also removes any SPS referencing this VPS (and remove_sps()
>does the same with PPS). Therefore if you simply repeat a VPS without
>also repeating the other parameter sets directly after the new VPS and
>before the first video NALU after the VPS, your extradata will have been
>discarded.
>This is not what the spec says.
>
>

Yes and I observed for hevc decoder with hwaccel, get_format() is called 
multiple times which initializes the hwaccel context multiple times, as 
s->ps.sps is unexpectedly removed because of that.

Hendrik also observed some playback glitches(see previous email) so it's not 
really harmless.


>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/hevc_ps: fix the problem of memcmp losing effectiveness

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of
>Hendrik Leppkes
>Sent: Thursday, March 28, 2024 5:43 PM
>To: FFmpeg development discussions and patches de...@ffmpeg.org>
>Subject: Re: [FFmpeg-devel] [PATCH] avcodec/hevc_ps: fix the problem of
>memcmp losing effectiveness
>
>On Thu, Mar 28, 2024 at 10:12 AM 
>wrote:
>>
>> From: Tong Wu 
>>
>> HEVCHdrParams* receives a pointer which points to a dynamically
>> allocated memory block. It causes the memcmp always returning 1.
>> Add a function to do the comparision. A condition is also added to
>> avoid malloc(0).
>>
>> Signed-off-by: Tong Wu 
>
>I've been looking into some playback glitches after
>456c8ebe7c7dcd766d36cd0296815d89fd1166b5 and I can confirm that this
>patch fixes those as well.
>
>Using the fixed position of the *hdr member and its offset seems a bit
>icky though, and _at least_ could use a comment so future changes keep
>it last.

Comment is added to the header.

Thanks,
Tong
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v7 11/12] avcodec: add D3D12VA hardware HEVC encoder

[Wu, Tong1]

Kindly ping. Is there any more comment on v7?

>-Original Message-
>From: Wu, Tong1 
>Sent: Thursday, March 14, 2024 4:15 PM
>To: ffmpeg-devel@ffmpeg.org
>Cc: Wu, Tong1 
>Subject: [FFmpeg-devel][PATCH v7 11/12] avcodec: add D3D12VA hardware
>HEVC encoder
>
>From: Tong Wu 
>
>This implementation is based on D3D12 Video Encoding Spec:
>https://microsoft.github.io/DirectX-Specs/d3d/D3D12VideoEncoding.html
>
>Sample command line for transcoding:
>ffmpeg.exe -hwaccel d3d12va -hwaccel_output_format d3d12 -i input.mp4
>-c:v hevc_d3d12va output.mp4
>
>Signed-off-by: Tong Wu 
>---
> configure|6 +
> libavcodec/Makefile  |4 +-
> libavcodec/allcodecs.c   |1 +
> libavcodec/d3d12va_encode.c  | 1550 ++
> libavcodec/d3d12va_encode.h  |  321 +++
> libavcodec/d3d12va_encode_hevc.c |  957 ++
> 6 files changed, 2838 insertions(+), 1 deletion(-)
> create mode 100644 libavcodec/d3d12va_encode.c
> create mode 100644 libavcodec/d3d12va_encode.h
> create mode 100644 libavcodec/d3d12va_encode_hevc.c
>
>diff --git a/configure b/configure
>index c34bdd13f5..53076fbf22 100755
>--- a/configure
>+++ b/configure
>@@ -2570,6 +2570,7 @@ CONFIG_EXTRA="
> tpeldsp
> vaapi_1
> vaapi_encode
>+d3d12va_encode
> vc1dsp
> videodsp
> vp3dsp
>@@ -3214,6 +3215,7 @@ wmv3_vaapi_hwaccel_select="vc1_vaapi_hwaccel"
> wmv3_vdpau_hwaccel_select="vc1_vdpau_hwaccel"
>
> # hardware-accelerated codecs
>+d3d12va_encode_deps="d3d12va ID3D12VideoEncoder
>d3d12_encoder_feature"
> mediafoundation_deps="mftransform_h MFCreateAlignedMemoryBuffer"
> omx_deps="libdl pthreads"
> omx_rpi_select="omx"
>@@ -3280,6 +3282,7 @@ h264_v4l2m2m_encoder_deps="v4l2_m2m
>h264_v4l2_m2m"
> hevc_amf_encoder_deps="amf"
> hevc_cuvid_decoder_deps="cuvid"
> hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf"
>+hevc_d3d12va_encoder_select="cbs_h265 d3d12va_encode"
> hevc_mediacodec_decoder_deps="mediacodec"
> hevc_mediacodec_decoder_select="hevc_mp4toannexb_bsf hevc_parser"
> hevc_mediacodec_encoder_deps="mediacodec"
>@@ -6620,6 +6623,9 @@ check_type "windows.h d3d11.h"
>"ID3D11VideoDecoder"
> check_type "windows.h d3d11.h" "ID3D11VideoContext"
> check_type "windows.h d3d12.h" "ID3D12Device"
> check_type "windows.h d3d12video.h" "ID3D12VideoDecoder"
>+check_type "windows.h d3d12video.h" "ID3D12VideoEncoder"
>+test_code cc "windows.h d3d12video.h" "D3D12_FEATURE_VIDEO feature =
>D3D12_FEATURE_VIDEO_ENCODER_CODEC" && \
>+test_code cc "windows.h d3d12video.h"
>"D3D12_FEATURE_DATA_VIDEO_ENCODER_RESOURCE_REQUIREMENTS req"
>&& enable d3d12_encoder_feature
> check_type "windows.h" "DPI_AWARENESS_CONTEXT" -
>D_WIN32_WINNT=0x0A00
> check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -
>D_WIN32_WINNT=0x0602
> check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat
>diff --git a/libavcodec/Makefile b/libavcodec/Makefile
>index cbfae5f182..cdda3f0d0a 100644
>--- a/libavcodec/Makefile
>+++ b/libavcodec/Makefile
>@@ -84,6 +84,7 @@ OBJS-$(CONFIG_CBS_JPEG)+= cbs_jpeg.o
> OBJS-$(CONFIG_CBS_MPEG2)   += cbs_mpeg2.o
> OBJS-$(CONFIG_CBS_VP8) += cbs_vp8.o vp8data.o
> OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o
>+OBJS-$(CONFIG_D3D12VA_ENCODE)  += d3d12va_encode.o
>hw_base_encode.o
> OBJS-$(CONFIG_DEFLATE_WRAPPER) += zlib_wrapper.o
> OBJS-$(CONFIG_DOVI_RPU)+= dovi_rpu.o
> OBJS-$(CONFIG_ERROR_RESILIENCE)+= error_resilience.o
>@@ -435,6 +436,7 @@ OBJS-$(CONFIG_HEVC_DECODER)+= hevcdec.o
>hevc_mvs.o \
>   h274.o
> OBJS-$(CONFIG_HEVC_AMF_ENCODER)+= amfenc_hevc.o
> OBJS-$(CONFIG_HEVC_CUVID_DECODER)  += cuviddec.o
>+OBJS-$(CONFIG_HEVC_D3D12VA_ENCODER)+= d3d12va_encode_hevc.o
> OBJS-$(CONFIG_HEVC_MEDIACODEC_DECODER) += mediacodecdec.o
> OBJS-$(CONFIG_HEVC_MEDIACODEC_ENCODER) += mediacodecenc.o
> OBJS-$(CONFIG_HEVC_MF_ENCODER) += mfenc.o mf_utils.o
>@@ -1263,7 +1265,7 @@ SKIPHEADERS+= %_tablegen.h
>\
>
> SKIPHEADERS-$(CONFIG_AMF)  += amfenc.h
> SKIPHEADERS-$(CONFIG_D3D11VA)  += d3d11va.h dxva2_internal.h
>-SKIPHEADERS-$(CONFIG_D3D12VA)  += d3d12va_decode.h
>+SKIPHEADERS-$(CONFIG_D3D12VA)  += d

Re: [FFmpeg-devel] [PATCH] avcodec/hevc_ps: allocate only the required HEVCHdrParams within a VPS

[Wu, Tong1]

Hi James,

>From: ffmpeg-devel  On Behalf Of James
>Almer
>Sent: Thursday, March 21, 2024 8:29 PM
>To: ffmpeg-devel@ffmpeg.org
>Subject: [FFmpeg-devel] [PATCH] avcodec/hevc_ps: allocate only the required
>HEVCHdrParams within a VPS
>
>Signed-off-by: James Almer 
>---
> libavcodec/hevc_ps.c | 14 +-
> libavcodec/hevc_ps.h |  2 +-
> 2 files changed, 14 insertions(+), 2 deletions(-)
>
>diff --git a/libavcodec/hevc_ps.c b/libavcodec/hevc_ps.c
>index fb997066d9..d29cf9f372 100644
>--- a/libavcodec/hevc_ps.c
>+++ b/libavcodec/hevc_ps.c
>@@ -438,13 +438,20 @@ static int decode_hrd(GetBitContext *gb, int
>common_inf_present,
> return 0;
> }
>
>+static void uninit_vps(FFRefStructOpaque opaque, void *obj)
>+{
>+HEVCVPS *vps = obj;
>+
>+av_freep(>hdr);
>+}
>+
> int ff_hevc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx,
>HEVCParamSets *ps)
> {
> int i,j;
> int vps_id = 0;
> ptrdiff_t nal_size;
>-HEVCVPS *vps = ff_refstruct_allocz(sizeof(*vps));
>+HEVCVPS *vps = ff_refstruct_alloc_ext(sizeof(*vps), 0, NULL, uninit_vps);
>
> if (!vps)
> return AVERROR(ENOMEM);
>@@ -533,6 +540,11 @@ int ff_hevc_decode_nal_vps(GetBitContext *gb,
>AVCodecContext *avctx,
>"vps_num_hrd_parameters %d is invalid\n", vps-
>>vps_num_hrd_parameters);
> goto err;
> }
>+
>+vps->hdr = av_calloc(vps->vps_num_hrd_parameters, sizeof(*vps->hdr));
>+if (!vps->hdr)
>+goto err;
>+

It looks like this will cause the following !memcmp(ps->vps_list[vps_id], vps, 
sizeof(*vps)) becomes 0 and furtherly remove_vps is called. Is that expected?

Thanks,
Tong


> for (i = 0; i < vps->vps_num_hrd_parameters; i++) {
> int common_inf_present = 1;
>
>diff --git a/libavcodec/hevc_ps.h b/libavcodec/hevc_ps.h
>index 786c896709..87cea479e9 100644
>--- a/libavcodec/hevc_ps.h
>+++ b/libavcodec/hevc_ps.h
>@@ -152,7 +152,7 @@ typedef struct PTL {
>
> typedef struct HEVCVPS {
> unsigned int vps_id;
>-HEVCHdrParams hdr[HEVC_MAX_LAYER_SETS];
>+HEVCHdrParams *hdr;
>
> uint8_t vps_temporal_id_nesting_flag;
> int vps_max_layers;
>--
>2.44.0
>
>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v6 2/9] avcodec/vaapi_encode: introduce a base layer for vaapi encode

[Wu, Tong1]

>
>Feb 29, 2024, 06:35 by tong1.wu-at-intel@ffmpeg.org:
>
>> From: Tong Wu 
>>
>> Since VAAPI and future D3D12VA implementation may share the same dpb
>> logic and some other functions. A base layer encode context is introduced
>> as vaapi context's base and extract the related funtions to a common
>> file such as receive_packet, init, close, etc.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>  libavcodec/Makefile |   2 +-
>>  libavcodec/hw_base_encode.c | 644 ++
>>  libavcodec/hw_base_encode.h | 252 +
>>  libavcodec/vaapi_encode.c   | 927 ++--
>>  libavcodec/vaapi_encode.h   | 202 +--
>>  libavcodec/vaapi_encode_av1.c   |  71 +--
>>  libavcodec/vaapi_encode_h264.c  | 199 +++
>>  libavcodec/vaapi_encode_h265.c  | 161 +++---
>>  libavcodec/vaapi_encode_mjpeg.c |  22 +-
>>  libavcodec/vaapi_encode_mpeg2.c |  51 +-
>>  libavcodec/vaapi_encode_vp8.c   |  26 +-
>>  libavcodec/vaapi_encode_vp9.c   |  68 +--
>>  12 files changed, 1400 insertions(+), 1225 deletions(-)
>>  create mode 100644 libavcodec/hw_base_encode.c
>>  create mode 100644 libavcodec/hw_base_encode.h
>>
>> +#define AVCODEC_HW_BASE_ENCODE_H
>> +
>> +#include "libavutil/hwcontext.h"
>> +#include "libavutil/fifo.h"
>> +
>> +#include "avcodec.h"
>> +
>> +static inline const char *ff_hw_base_encode_get_pictype_name(const int
>type) {
>> +const char * const picture_type_name[] = { "IDR", "I", "P", "B" };
>> +return picture_type_name[type];
>> +}
>> +
>> +enum {
>> +MAX_DPB_SIZE   = 16,
>> +MAX_PICTURE_REFERENCES = 2,
>>
>
>Only two max refs per picture?

For current VAAPI implementation yes.

>
>
>> +MAX_REORDER_DELAY  = 16,
>> +MAX_ASYNC_DEPTH= 64,
>> +MAX_REFERENCE_LIST_NUM = 2,
>> +};
>>
>
>Define these, don't enum them unnecessarily.

Sure.

>
>
>> +enum {
>> +PICTURE_TYPE_IDR = 0,
>> +PICTURE_TYPE_I   = 1,
>> +PICTURE_TYPE_P   = 2,
>> +PICTURE_TYPE_B   = 3,
>> +};
>> +
>> +enum {
>> +// Codec supports controlling the subdivision of pictures into slices.
>> +FLAG_SLICE_CONTROL = 1 << 0,
>> +// Codec only supports constant quality (no rate control).
>> +FLAG_CONSTANT_QUALITY_ONLY = 1 << 1,
>> +// Codec is intra-only.
>> +FLAG_INTRA_ONLY= 1 << 2,
>> +// Codec supports B-pictures.
>> +FLAG_B_PICTURES= 1 << 3,
>> +// Codec supports referencing B-pictures.
>> +FLAG_B_PICTURE_REFERENCES  = 1 << 4,
>> +// Codec supports non-IDR key pictures (that is, key pictures do
>> +// not necessarily empty the DPB).
>> +FLAG_NON_IDR_KEY_PICTURES  = 1 << 5,
>> +// Codec output packet without timestamp delay, which means the
>> +// output packet has same PTS and DTS.
>> +FLAG_TIMESTAMP_NO_DELAY= 1 << 6,
>>
>
>This is a low-level API. Concepts such as delay don't exist
>if you're the one setting up references.

I'll remove it from here and define this flag in vaapi_encode.h then.

>
>
>> +};
>> +
>> +typedef struct HWBaseEncodePicture {
>> +struct HWBaseEncodePicture *next;
>> +
>> +int64_t display_order;
>> +int64_t encode_order;
>> +int64_t pts;
>> +int64_t duration;
>> +int force_idr;
>> +
>> +void   *opaque;
>> +AVBufferRef*opaque_ref;
>> +
>> +int type;
>> +int b_depth;
>> +int encode_issued;
>> +int encode_complete;
>> +
>> +AVFrame*input_image;
>> +AVFrame*recon_image;
>> +
>> +void   *priv_data;
>> +
>> +// Whether this picture is a reference picture.
>> +int is_reference;
>> +
>> +// The contents of the DPB after this picture has been decoded.
>> +// This will contain the picture itself if it is a reference picture,
>> +// but not if it isn't.
>> +int nb_dpb_pics;
>> +struct HWBaseEncodePicture *dpb[MAX_DPB_SIZE];
>> +// The reference pictures used in decoding this picture. If they are
>> +// used by later pictures they will also appear in the DPB. ref[0][] for
>> +// previous reference frames. ref[1][] for future reference frames.
>> +int nb_refs[MAX_REFERENCE_LIST_NUM];
>> +struct HWBaseEncodePicture
>*refs[MAX_REFERENCE_LIST_NUM][MAX_PICTURE_REFERENCES];
>> +// The previous reference picture in encode order.  Must be in at least
>> +// one of the reference list and DPB list.
>> +struct HWBaseEncodePicture *prev;
>> +// Reference count for other pictures referring to this one through
>> +// the above pointers, directly from incomplete pictures and indirectly
>> +// through completed pictures.
>> +int ref_count[2];
>> +int ref_removed[2];
>> +} HWBaseEncodePicture;
>> +
>> +typedef struct HWEncodePictureOperation {
>> +// Alloc memory for the picture structure.
>> +

Re: [FFmpeg-devel] [PATCH v6 8/9] avcodec: add D3D12VA hardware HEVC encoder

[Wu, Tong1]

Kindly ping. 

Hi Mark, do you have more comments on this patch set?

>Subject: [FFmpeg-devel][PATCH v6 8/9] avcodec: add D3D12VA hardware HEVC
>encoder
>
>From: Tong Wu 
>
>This implementation is based on D3D12 Video Encoding Spec:
>https://microsoft.github.io/DirectX-Specs/d3d/D3D12VideoEncoding.html
>
>Sample command line for transcoding:
>ffmpeg.exe -hwaccel d3d12va -hwaccel_output_format d3d12 -i input.mp4
>-c:v hevc_d3d12va output.mp4
>
>Signed-off-by: Tong Wu 
>---
> configure|6 +
> libavcodec/Makefile  |4 +-
> libavcodec/allcodecs.c   |1 +
> libavcodec/d3d12va_encode.c  | 1554 ++
> libavcodec/d3d12va_encode.h  |  321 ++
> libavcodec/d3d12va_encode_hevc.c |  957 ++
> 6 files changed, 2842 insertions(+), 1 deletion(-)
> create mode 100644 libavcodec/d3d12va_encode.c
> create mode 100644 libavcodec/d3d12va_encode.h
> create mode 100644 libavcodec/d3d12va_encode_hevc.c
>
>diff --git a/configure b/configure
>index bb5e630bad..e1e302a005 100755
>--- a/configure
>+++ b/configure
>@@ -2566,6 +2566,7 @@ CONFIG_EXTRA="
> tpeldsp
> vaapi_1
> vaapi_encode
>+d3d12va_encode
> vc1dsp
> videodsp
> vp3dsp
>@@ -3210,6 +3211,7 @@ wmv3_vaapi_hwaccel_select="vc1_vaapi_hwaccel"
> wmv3_vdpau_hwaccel_select="vc1_vdpau_hwaccel"
>
> # hardware-accelerated codecs
>+d3d12va_encode_deps="d3d12va ID3D12VideoEncoder
>d3d12_encoder_feature"
> mediafoundation_deps="mftransform_h MFCreateAlignedMemoryBuffer"
> omx_deps="libdl pthreads"
> omx_rpi_select="omx"
>@@ -3277,6 +3279,7 @@ h264_v4l2m2m_encoder_deps="v4l2_m2m
>h264_v4l2_m2m"
> hevc_amf_encoder_deps="amf"
> hevc_cuvid_decoder_deps="cuvid"
> hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf"
>+hevc_d3d12va_encoder_select="cbs_h265 d3d12va_encode"
> hevc_mediacodec_decoder_deps="mediacodec"
> hevc_mediacodec_decoder_select="hevc_mp4toannexb_bsf hevc_parser"
> hevc_mediacodec_encoder_deps="mediacodec"
>@@ -6620,6 +6623,9 @@ check_type "windows.h d3d11.h"
>"ID3D11VideoDecoder"
> check_type "windows.h d3d11.h" "ID3D11VideoContext"
> check_type "windows.h d3d12.h" "ID3D12Device"
> check_type "windows.h d3d12video.h" "ID3D12VideoDecoder"
>+check_type "windows.h d3d12video.h" "ID3D12VideoEncoder"
>+test_code cc "windows.h d3d12video.h" "D3D12_FEATURE_VIDEO feature =
>D3D12_FEATURE_VIDEO_ENCODER_CODEC" && \
>+test_code cc "windows.h d3d12video.h"
>"D3D12_FEATURE_DATA_VIDEO_ENCODER_RESOURCE_REQUIREMENTS req"
>&& enable d3d12_encoder_feature
> check_type "windows.h" "DPI_AWARENESS_CONTEXT" -
>D_WIN32_WINNT=0x0A00
> check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -
>D_WIN32_WINNT=0x0602
> check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat
>diff --git a/libavcodec/Makefile b/libavcodec/Makefile
>index c1b3dff055..965aaba80e 100644
>--- a/libavcodec/Makefile
>+++ b/libavcodec/Makefile
>@@ -86,6 +86,7 @@ OBJS-$(CONFIG_CBS_MPEG2)   += cbs_mpeg2.o
> OBJS-$(CONFIG_CBS_VP8) += cbs_vp8.o vp8data.o
> OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o
> OBJS-$(CONFIG_CRYSTALHD)   += crystalhd.o
>+OBJS-$(CONFIG_D3D12VA_ENCODE)  += d3d12va_encode.o
>hw_base_encode.o
> OBJS-$(CONFIG_DEFLATE_WRAPPER) += zlib_wrapper.o
> OBJS-$(CONFIG_DOVI_RPU)+= dovi_rpu.o
> OBJS-$(CONFIG_ERROR_RESILIENCE)+= error_resilience.o
>@@ -437,6 +438,7 @@ OBJS-$(CONFIG_HEVC_DECODER)+= hevcdec.o
>hevc_mvs.o \
>   h274.o
> OBJS-$(CONFIG_HEVC_AMF_ENCODER)+= amfenc_hevc.o
> OBJS-$(CONFIG_HEVC_CUVID_DECODER)  += cuviddec.o
>+OBJS-$(CONFIG_HEVC_D3D12VA_ENCODER)+= d3d12va_encode_hevc.o
> OBJS-$(CONFIG_HEVC_MEDIACODEC_DECODER) += mediacodecdec.o
> OBJS-$(CONFIG_HEVC_MEDIACODEC_ENCODER) += mediacodecenc.o
> OBJS-$(CONFIG_HEVC_MF_ENCODER) += mfenc.o mf_utils.o
>@@ -1267,7 +1269,7 @@ SKIPHEADERS+= %_tablegen.h
>\
>
> SKIPHEADERS-$(CONFIG_AMF)  += amfenc.h
> SKIPHEADERS-$(CONFIG_D3D11VA)  += d3d11va.h dxva2_internal.h
>-SKIPHEADERS-$(CONFIG_D3D12VA)  += d3d12va_decode.h
>+SKIPHEADERS-$(CONFIG_D3D12VA)  += d3d12va_decode.h
>d3d12va_encode.h
> SKIPHEADERS-$(CONFIG_DXVA2)+= dxva2.h dxva2_internal.h
> SKIPHEADERS-$(CONFIG_JNI)  += ffjni.h
> SKIPHEADERS-$(CONFIG_LCMS2)+= fflcms2.h
>diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
>index ef8c3a6d7d..9a34974141 100644
>--- a/libavcodec/allcodecs.c
>+++ b/libavcodec/allcodecs.c
>@@ -865,6 +865,7 @@ extern const FFCodec ff_h264_vaapi_encoder;
> extern const FFCodec ff_h264_videotoolbox_encoder;
> extern const FFCodec ff_hevc_amf_encoder;
> extern const FFCodec ff_hevc_cuvid_decoder;
>+extern const FFCodec ff_hevc_d3d12va_encoder;
> extern const FFCodec ff_hevc_mediacodec_decoder;
> extern const FFCodec ff_hevc_mediacodec_encoder;
> extern const FFCodec ff_hevc_mf_encoder;

Re: [FFmpeg-devel] [PATCH 2/2] avcodec/vaapi_encode_h265: use is_reference to fill reference_pic_flag

[Wu, Tong1]

>Subject: Re: [FFmpeg-devel] [PATCH 2/2] avcodec/vaapi_encode_h265: use
>is_reference to fill reference_pic_flag
>
>On Di, 2024-02-27 at 11:48 +0800, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> This codec supports FLAG_B_PICTURE_REFERENCES. We need to fill
>> reference_pic_flag with pic->is_reference.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>  libavcodec/vaapi_encode_h265.c | 8 
>>  1 file changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/libavcodec/vaapi_encode_h265.c
>b/libavcodec/vaapi_encode_h265.c
>> index c4aabbf5ed..b5d3468152 100644
>> --- a/libavcodec/vaapi_encode_h265.c
>> +++ b/libavcodec/vaapi_encode_h265.c
>> @@ -949,22 +949,22 @@ static int
>> vaapi_encode_h265_init_picture_params(AVCodecContext *avctx,
>>  case PICTURE_TYPE_IDR:
>>  vpic->pic_fields.bits.idr_pic_flag   = 1;
>>  vpic->pic_fields.bits.coding_type    = 1;
>> -    vpic->pic_fields.bits.reference_pic_flag = 1;
>> +    vpic->pic_fields.bits.reference_pic_flag = pic->is_reference;
>>  break;
>>  case PICTURE_TYPE_I:
>>  vpic->pic_fields.bits.idr_pic_flag   = 0;
>>  vpic->pic_fields.bits.coding_type    = 1;
>> -    vpic->pic_fields.bits.reference_pic_flag = 1;
>> +    vpic->pic_fields.bits.reference_pic_flag = pic->is_reference;
>>  break;
>>  case PICTURE_TYPE_P:
>>  vpic->pic_fields.bits.idr_pic_flag   = 0;
>>  vpic->pic_fields.bits.coding_type    = 2;
>> -    vpic->pic_fields.bits.reference_pic_flag = 1;
>> +    vpic->pic_fields.bits.reference_pic_flag = pic->is_reference;
>>  break;
>>  case PICTURE_TYPE_B:
>>  vpic->pic_fields.bits.idr_pic_flag   = 0;
>>  vpic->pic_fields.bits.coding_type    = 3;
>> -    vpic->pic_fields.bits.reference_pic_flag = 0;
>> +    vpic->pic_fields.bits.reference_pic_flag = pic->is_reference;
>
>Better to move the assignment out of the switch statement.

Sorry I misunderstood. Please see patch V3.

>
>Thanks
>Haihao
>
>>  break;
>>  default:
>>  av_assert0(0 && "invalid picture type");

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v5 4/9] avcodec/vaapi_encode: extract rc parameter configuration to base layer

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of Mark
>Thompson
>Sent: Friday, February 23, 2024 3:22 AM
>To: ffmpeg-devel@ffmpeg.org
>Subject: Re: [FFmpeg-devel] [PATCH v5 4/9] avcodec/vaapi_encode: extract rc
>parameter configuration to base layer
>
>On 22/02/2024 10:10, Wu, Tong1 wrote:
>>> On 18/02/2024 08:45, tong1.wu-at-intel@ffmpeg.org wrote:
>>>> From: Tong Wu 
>>>>
>>>> VAAPI and D3D12VA can share rate control configuration codes. Hence, it
>>>> can be moved to base layer for simplification.
>>>>
>>>> Signed-off-by: Tong Wu 
>>>> ---
>>>>libavcodec/hw_base_encode.c| 151 
>>>>libavcodec/hw_base_encode.h|  34 ++
>>>>libavcodec/vaapi_encode.c  | 210 ++---
>>>>libavcodec/vaapi_encode.h  |  14 +--
>>>>libavcodec/vaapi_encode_av1.c  |   2 +-
>>>>libavcodec/vaapi_encode_h264.c |   2 +-
>>>>libavcodec/vaapi_encode_vp9.c  |   2 +-
>>>>7 files changed, 227 insertions(+), 188 deletions(-)
>>>>
>>>> ...
>>>> diff --git a/libavcodec/hw_base_encode.h
>b/libavcodec/hw_base_encode.h
>>>> index b836b22e6b..4072b514d3 100644
>>>> --- a/libavcodec/hw_base_encode.h
>>>> +++ b/libavcodec/hw_base_encode.h
>>>> @@ -72,6 +72,37 @@ enum {
>>>>RC_MODE_MAX = RC_MODE_AVBR,
>>>>};
>>>>
>>>> +typedef struct HWBaseEncodeRCMode {
>>>> +// Mode from above enum (RC_MODE_*).
>>>> +int mode;
>>>> +// Name.
>>>> +const char *name;
>>>> +// Uses bitrate parameters.
>>>> +int bitrate;
>>>> +// Supports maxrate distinct from bitrate.
>>>> +int maxrate;
>>>> +// Uses quality value.
>>>> +int quality;
>>>> +// Supports HRD/VBV parameters.
>>>> +int hrd;
>>>> +} HWBaseEncodeRCMode;
>>>> +
>>>> +typedef struct HWBaseEncodeRCConfigure
>>>> +{
>>>> +int64_t rc_bits_per_second;
>>>> +int rc_target_percentage;
>>>> +int rc_window_size;
>>>> +
>>>> +int rc_quality;
>>>> +
>>>> +int64_t hrd_buffer_size;
>>>> +int64_t hrd_initial_buffer_fullness;
>>>> +
>>>> +int fr_num;
>>>> +int fr_den;
>>>> +} HWBaseEncodeRCConfigure;
>>>
>>> The set of fields here needs more thought to match up the common parts
>of
>>> the APIs.
>>>
>>> Just have target rate and maxrate and let VAAPI deal with the percentage
>stuff
>>> maybe?  Not sure what to do with window_size which isn't present at all in
>>> D3D12.  The convergence/accuracy stuff for VAAPI AVBR is also unclear.
>>>
>>
>> Could you please explain more about your thoughts on why the percentage
>stuff should not be in the common part? I think D3D12 can share the same
>code in terms of percentage stuff and hrd stuff. I can let VAAPI do the AVBR
>stuff and remove window_size from common part and keep everything else
>as-is.
>
>The libavcodec API is a better match for D3D12 than VAAPI is: e.g. for D3D12
>VBR you have TargetAvgBitRate = AVCodecContext.bit_rate, PeakBitRate =
>AVCodecContext.rc_max_rate etc. with all fields nicely matching.
>
>If you use the VAAPI fields here then you are translating the rate into a
>percentage (with rounding error because it's an integer) and then back again
>for no reason, hence I think you should prefer the libavcodec/D3D12 fields and
>do the conversion for VAAPI in its own code.

OK, that makes sense. I think if that's the case, probably it's not necessary 
to have this patch to extract the code to a common function. Just let VAAPI and 
D3D12 do their own jobs separately. I'm going to drop this patch from this 
patchset.


>
>>> (Do you know why QVBR is missing the VBV parameters in D3D12?  On the
>face
>>> of it that doesn't make any sense, but maybe I'm missing something.)
>>>
>>
>> It is presented in QVBR1 structure(defined in DirectX-header but not yet in
>Windows SDK).
>> I can add this support afterwards maybe along with AV1 implementation.
>
>Ah, so it was a bug and has been fixed.  That's good!
>
>>>> +
>>>> +
>>>>typedef struct HWBaseEncodePicture {
>>>>struct HWBaseEncodePicture *next;
>>>>
>>>> @@ -242,6 +273

Re: [FFmpeg-devel] [PATCH v5 4/9] avcodec/vaapi_encode: extract rc parameter configuration to base layer

[Wu, Tong1]

>On 18/02/2024 08:45, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> VAAPI and D3D12VA can share rate control configuration codes. Hence, it
>> can be moved to base layer for simplification.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>   libavcodec/hw_base_encode.c| 151 
>>   libavcodec/hw_base_encode.h|  34 ++
>>   libavcodec/vaapi_encode.c  | 210 ++---
>>   libavcodec/vaapi_encode.h  |  14 +--
>>   libavcodec/vaapi_encode_av1.c  |   2 +-
>>   libavcodec/vaapi_encode_h264.c |   2 +-
>>   libavcodec/vaapi_encode_vp9.c  |   2 +-
>>   7 files changed, 227 insertions(+), 188 deletions(-)
>>
>> diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c
>> index f0e4ef9655..c20c47bf55 100644
>> --- a/libavcodec/hw_base_encode.c
>> +++ b/libavcodec/hw_base_encode.c
>> @@ -631,6 +631,157 @@ end:
>>   return 0;
>>   }
>>
>> +int ff_hw_base_rc_mode_configure(AVCodecContext *avctx, const
>HWBaseEncodeRCMode *rc_mode,
>> + int default_quality, 
>> HWBaseEncodeRCConfigure *rc_conf)
>> +{
>> +HWBaseEncodeContext *ctx = avctx->priv_data;
>> +
>> +if (!rc_mode || !rc_conf)
>> +return -1;
>> +
>> +if (rc_mode->bitrate) {
>> +if (avctx->bit_rate <= 0) {
>> +av_log(avctx, AV_LOG_ERROR, "Bitrate must be set for %s "
>> +   "RC mode.\n", rc_mode->name);
>> +return AVERROR(EINVAL);
>> +}
>> +
>> +if (rc_mode->mode == RC_MODE_AVBR) {
>> +// For maximum confusion AVBR is hacked into the existing API
>> +// by overloading some of the fields with completely different
>> +// meanings.
>
>You definitely don't want this absurd wart from libva to leak into the common
>API parts.  Make new fields which have the desired meaning in the common
>parts and let the VAAPI code deal with this stupidity.

Agreed.

>
>> +
>> +// Target percentage does not apply in AVBR mode.
>> +rc_conf->rc_bits_per_second = avctx->bit_rate;
>> +
>> +// Accuracy tolerance range for meeting the specified target
>> +// bitrate.  It's very unclear how this is actually intended
>> +// to work - since we do want to get the specified bitrate,
>> +// set the accuracy to 100% for now.
>> +rc_conf->rc_target_percentage = 100;
>> +
>> +// Convergence period in frames.  The GOP size reflects the
>> +// user's intended block size for cutting, so reusing that
>> +// as the convergence period seems a reasonable default.
>> +rc_conf->rc_window_size = avctx->gop_size > 0 ? avctx->gop_size 
>> :
>60;
>> +
>> +} else if (rc_mode->maxrate) {
>> +if (avctx->rc_max_rate > 0) {
>> +if (avctx->rc_max_rate < avctx->bit_rate) {
>> +av_log(avctx, AV_LOG_ERROR, "Invalid bitrate settings: "
>> +   "bitrate (%"PRId64") must not be greater than "
>> +   "maxrate (%"PRId64").\n", avctx->bit_rate,
>> +   avctx->rc_max_rate);
>> +return AVERROR(EINVAL);
>> +}
>> +rc_conf->rc_bits_per_second   = avctx->rc_max_rate;
>> +rc_conf->rc_target_percentage = (avctx->bit_rate * 100) /
>> + avctx->rc_max_rate;
>> +} else {
>> +// We only have a target bitrate, but this mode requires
>> +// that a maximum rate be supplied as well.  Since the
>> +// user does not want this to be a constraint, arbitrarily
>> +// pick a maximum rate of double the target rate.
>> +rc_conf->rc_bits_per_second   = 2 * avctx->bit_rate;
>> +rc_conf->rc_target_percentage = 50;
>> +}
>> +} else {
>> +if (avctx->rc_max_rate > avctx->bit_rate) {
>> +av_log(avctx, AV_LOG_WARNING, "Max bitrate is ignored "
>> +   "in %s RC mode.\n", rc_mode->name);
>> +}
>> +rc_conf->rc_bits_per_second   = avctx->bit_rate;
>> +rc_conf->rc_target_percentage = 100;
>> +}
>> +} else {
>> +rc_conf->rc_bits_per_second   = 0;
>> +rc_conf->rc_target_percentage = 100;
>> +}
>> +
>> +if (rc_mode->quality) {
>> +if (ctx->explicit_qp) {
>> +rc_conf->rc_quality = ctx->explicit_qp;
>> +} else if (avctx->global_quality > 0) {
>> +rc_conf->rc_quality = avctx->global_quality;
>> +} else {
>> +rc_conf->rc_quality = default_quality;
>> +av_log(avctx, AV_LOG_WARNING, "No quality level set; "
>> +   "using default (%d).\n", rc_conf->rc_quality);
>> +}
>> +} else {
>> +rc_conf->rc_quality = 0;
>> +}
>> +

Re: [FFmpeg-devel] [PATCH v5 2/9] avcodec/vaapi_encode: introduce a base layer for vaapi encode

[Wu, Tong1]

Hi Mark. Thanks for your careful review.

>On 18/02/2024 08:45, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> Since VAAPI and future D3D12VA implementation may share the same dpb
>> logic and some other functions. A base layer encode context is introduced
>> as vaapi context's base and extract the related funtions to a common
>> file such as receive_packet, init, close, etc.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>   libavcodec/Makefile |   2 +-
>>   libavcodec/hw_base_encode.c | 637 ++
>>   libavcodec/hw_base_encode.h | 277 ++
>>   libavcodec/vaapi_encode.c   | 924 +++-
>>   libavcodec/vaapi_encode.h   | 229 +---
>>   libavcodec/vaapi_encode_av1.c   |  73 +--
>>   libavcodec/vaapi_encode_h264.c  | 201 +++
>>   libavcodec/vaapi_encode_h265.c  | 163 +++---
>>   libavcodec/vaapi_encode_mjpeg.c |  22 +-
>>   libavcodec/vaapi_encode_mpeg2.c |  53 +-
>>   libavcodec/vaapi_encode_vp8.c   |  28 +-
>>   libavcodec/vaapi_encode_vp9.c   |  70 +--
>>   12 files changed, 1427 insertions(+), 1252 deletions(-)
>>   create mode 100644 libavcodec/hw_base_encode.c
>>   create mode 100644 libavcodec/hw_base_encode.h
>>
>> diff --git a/libavcodec/Makefile b/libavcodec/Makefile
>> index 470d7cb9b1..23946f6ea3 100644
>> --- a/libavcodec/Makefile
>> +++ b/libavcodec/Makefile
>> @@ -164,7 +164,7 @@ OBJS-$(CONFIG_STARTCODE)   += startcode.o
>>   OBJS-$(CONFIG_TEXTUREDSP)  += texturedsp.o
>>   OBJS-$(CONFIG_TEXTUREDSPENC)   += texturedspenc.o
>>   OBJS-$(CONFIG_TPELDSP) += tpeldsp.o
>> -OBJS-$(CONFIG_VAAPI_ENCODE)+= vaapi_encode.o
>> +OBJS-$(CONFIG_VAAPI_ENCODE)+= vaapi_encode.o
>hw_base_encode.o
>>   OBJS-$(CONFIG_AV1_AMF_ENCODER) += amfenc_av1.o
>>   OBJS-$(CONFIG_VC1DSP)  += vc1dsp.o
>>   OBJS-$(CONFIG_VIDEODSP)+= videodsp.o
>> diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c
>> ...
>
>I'm going to trust that the contents of this file are only moved around with no
>functional changes.  If that isn't the case please do say.

You are absolutely right.

>
>> diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h
>> new file mode 100644
>> index 00..70982c97b2
>> --- /dev/null
>> +++ b/libavcodec/hw_base_encode.h
>> @@ -0,0 +1,277 @@
>> +/*
>> + * Copyright (c) 2024 Intel Corporation
>
>I would avoid adding this.  Most of these files have content mostly copied from
>other files which are not exclusively copyright by Intel Corporation.

Will delete.

>
>> + *
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with FFmpeg; if not, write to the Free Software
>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
>USA
>> + */
>> +
>> +#ifndef AVCODEC_HW_BASE_ENCODE_H
>> +#define AVCODEC_HW_BASE_ENCODE_H
>> +
>> +#include "libavutil/hwcontext.h"
>> +#include "libavutil/fifo.h"
>> +
>> +#include "avcodec.h"
>> +#include "hwconfig.h"
>
>This file doesn't do anything with hwconfig stuff?
>

Will delete.

>> +
>> +enum {
>> +MAX_DPB_SIZE   = 16,
>> +MAX_PICTURE_REFERENCES = 2,
>> +MAX_REORDER_DELAY  = 16,
>> +MAX_ASYNC_DEPTH= 64,
>> +MAX_REFERENCE_LIST_NUM = 2,
>> +};
>> +
>> +enum {
>> +PICTURE_TYPE_IDR = 0,
>> +PICTURE_TYPE_I   = 1,
>> +PICTURE_TYPE_P   = 2,
>> +PICTURE_TYPE_B   = 3,
>> +};
>> +
>> +enum {
>> +// Codec supports controlling the subdivision of pictures into slices.
>> +FLAG_SLICE_CONTROL = 1 << 0,
>> +// Codec only supports constant quality (no rate control).
>> +FLAG_CONSTANT_QUALITY_ONLY = 1 << 1,
>> +// Codec is intra-only.
>> +FLAG_INTRA_ONLY= 1 << 2,
>> +// Codec supports B-pictures.
>> +FLAG_B_PICTURES= 1 << 3,
>> +// Codec supports referencing B-pictures.
>> +FLAG_B_PICTURE_REFERENCES  = 1 << 4,
>> +// Codec supports non-IDR key pictures (that is, key pictures do
>> +// not necessarily empty the DPB).
>> +FLAG_NON_IDR_KEY_PICTURES  = 1 << 5,
>> +// Codec output packet without timestamp delay, which means the
>> +// output packet has same PTS and DTS.
>> +FLAG_TIMESTAMP_NO_DELAY= 1 << 6,
>> +};
>> +
>> +enum {
>> +RC_MODE_AUTO,
>> +RC_MODE_CQP,
>> 

Re: [FFmpeg-devel] [PATCH v3 8/9] avcodec: add D3D12VA hardware HEVC encoder

[Wu, Tong1]

>> From: Tong Wu 
>>
>> This implementation is based on D3D12 Video Encoding Spec:
>> https://microsoft.github.io/DirectX-Specs/d3d/D3D12VideoEncoding.html
>>
>> Sample command line for transcoding:
>> ffmpeg.exe -hwaccel d3d12va -hwaccel_output_format d3d12 -i input.mp4
>> -c:v hevc_d3d12va output.mp4
>>
>> Signed-off-by: Tong Wu 
>> ---
>> configure|6 +
>> libavcodec/Makefile  |4 +-
>> libavcodec/allcodecs.c   |1 +
>> libavcodec/d3d12va_encode.c  | 1441 ++
>> libavcodec/d3d12va_encode.h  |  275 ++
>> libavcodec/d3d12va_encode_hevc.c | 1011 +
>> libavcodec/hw_base_encode.h  |2 +-
>> 7 files changed, 2738 insertions(+), 2 deletions(-)
>> create mode 100644 libavcodec/d3d12va_encode.c
>> create mode 100644 libavcodec/d3d12va_encode.h
>> create mode 100644 libavcodec/d3d12va_encode_hevc.c
>>
>>
>>+min_cu_size = d3d12va_encode_hevc_map_cusize(ctx-
>>codec_conf.pHEVCConfig->MinLumaCodingUnitSize);
>>+max_cu_size = d3d12va_encode_hevc_map_cusize(ctx-
>>codec_conf.pHEVCConfig->MaxLumaCodingUnitSize);
>>+min_tu_size = d3d12va_encode_hevc_map_tusize(ctx-
>>codec_conf.pHEVCConfig->MinLumaTransformUnitSize);
>>+max_tu_size = d3d12va_encode_hevc_map_tusize(ctx-
>>codec_conf.pHEVCConfig->MaxLumaTransformUnitSize);
>>+
>>+// VPS
>>+
>>+vps->nal_unit_header = (H265RawNALUnitHeader) {
>
>Should this blank line be removed, because the comment is for the codes
>below?
>
>> +vps->vps_timing_info_present_flag = 0;
>> +
>> +// SPS
>> +
>> +sps->nal_unit_header = (H265RawNALUnitHeader) {
>> +.nal_unit_type = HEVC_NAL_SPS,
>> +.nuh_layer_id  = 0,
>> +.nuh_temporal_id_plus1 = 1,
>> +};
>The same as above.
>
>> +static uint8_t
>d3d12va_encode_hevc_map_cusize(D3D12_VIDEO_ENCODER_CODEC_CONFI
>GURATION_HEVC_CUSIZE cusize)
>> +{
>> +switch (cusize) {
>> +case
>D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8:
>return 8;
>> +case
>D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_16x16:
>return 16;
>> +case
>D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_32x32:
>return 32;
>> +case
>D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_64x64:
>return 64;
>> +}
>> +return 0;
>> +}
>> +
>> +static uint8_t
>d3d12va_encode_hevc_map_tusize(D3D12_VIDEO_ENCODER_CODEC_CONFI
>GURATION_HEVC_TUSIZE tusize)
>> +{
>> +switch (tusize) {
>> +case
>D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4:
>return 4;
>> +case
>D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_8x8:
>return 8;
>> +case
>D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_16x16:
>return 16;
>> +case
>D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32:
>return 32;
>> +}
>> +return 0;
>> +}
>
>A default branch is needed or we can use 8 << cusize and 4 << tusize for
>simplification.
>
>> +hr = ID3D12Device3_QueryInterface(ctx->device3,
>_ID3D12VideoDevice3, (void **)>video_device3);
>> +if (FAILED(hr)) {
>> +err = AVERROR_UNKNOWN;
>> +goto fail;
>> +}
>> +
>> +if (FAILED(ID3D12VideoDevice3_CheckFeatureSupport(ctx-
>>video_device3, D3D12_FEATURE_VIDEO_FEATURE_AREA_SUPPORT,
>> +  , 
>> sizeof(support)))
>&& !support.VideoEncodeSupport) {
>> +av_log(avctx, AV_LOG_ERROR, "D3D12 video device has no video
>encoder support");
>> +err = AVERROR(EINVAL);
>> +goto fail;
>> +}
>
>We need to output the log for the ID3D12Device3_QueryInterface call, or the
>user will not know the error is resulting from that,
>the OS and the driver don't support the ID3D12VideoDevice3 interface.

Have updated in v4. Thanks for the review.

BRs,
Tong

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 8/9] avcodec: add D3D12VA hardware HEVC encoder

[Wu, Tong1]

>>
>> From: Tong Wu 
>>
>> This implementation is based on D3D12 Video Encoding Spec:
>> https://microsoft.github.io/DirectX-Specs/d3d/D3D12VideoEncoding.html
>>
>> Sample command line for transcoding:
>> ffmpeg.exe -hwaccel d3d12va -hwaccel_output_format d3d12 -i input.mp4
>> -c:v hevc_d3d12va output.mp4
>>
>> Signed-off-by: Tong Wu 
>> ---
> > configure|6 +
> > libavcodec/Makefile  |4 +-
> > libavcodec/allcodecs.c   |1 +
> > libavcodec/d3d12va_encode.c  | 1441
>++
> > libavcodec/d3d12va_encode.h  |  200 +
> > libavcodec/d3d12va_encode_hevc.c | 1016 +
> > libavcodec/hw_base_encode.h  |2 +-
> > 7 files changed, 2668 insertions(+), 2 deletions(-)
> > create mode 100644 libavcodec/d3d12va_encode.c
> > create mode 100644 libavcodec/d3d12va_encode.h
> > create mode 100644 libavcodec/d3d12va_encode_hevc.c
>
>> +D3D12_OBJECT_RELEASE(ctx->sync_ctx.fence);
>> +if (ctx->sync_ctx.event)
>> +CloseHandle(ctx->sync_ctx.event);
>> +
>> +D3D12_OBJECT_RELEASE(ctx->video_device3);
>> +D3D12_OBJECT_RELEASE(ctx->device);
>> +D3D12_OBJECT_RELEASE(ctx->encoder_heap);
>> +D3D12_OBJECT_RELEASE(ctx->encoder);
>
>We need to release all of the objects, including the encoder and
>encoder_heap, created by the device before releasing the device.
>
>> +
>> +typedef struct D3D12VAEncodeProfile {
>> +//lavc profile value (AV_PROFILE_*).
>> +int   av_profile;
>> +//Supported bit depth.
>> +int   depth;
>> +//Number of components.
>> +int   nb_components;
>> +//Chroma subsampling in width dimension.
>> +int   log2_chroma_w;
>> +//Chroma subsampling in height dimension.
>> +int   log2_chroma_h;
>> +//D3D12 profile value.
>> +D3D12_VIDEO_ENCODER_PROFILE_DESC d3d12_profile;
>> +} D3D12VAEncodeProfile;
>> +
>> +typedef struct D3D12VAEncodeRCMode {
>> +// Base.
>> +HWBaseEncodeRCMode base;
>> +// Supported by D3D12 HW.
>> +int supported;
>> +// D3D12 mode value.
>> +D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE d3d12_mode;
>> +} D3D12VAEncodeRCMode;
>> +
>> +typedef struct D3D12VAEncodeContext {
>> +HWBaseEncodeContext base;
>> +
>> +//Codec-specific hooks.
>> +const struct D3D12VAEncodeType *codec;
>> +
>> +//Chosen encoding profile details.
>> +const D3D12VAEncodeProfile *profile;
>> +
>> +//Chosen rate control mode details.
>> +const D3D12VAEncodeRCMode *rc_mode;
>> +
>> +AVD3D12VADeviceContext *hwctx;
>> +
>> +//Device3 interface.
>> +ID3D12Device3 *device3;
>> +
>> +ID3D12VideoDevice3 *video_device3;
>> +
>> +//Pool of (reusable) bitstream output buffers.
>> +AVBufferPool   *output_buffer_pool;
>> +
>> +//D3D12 video encoder.
>> +AVBufferRef *encoder_ref;
>> +
>> +ID3D12VideoEncoder *encoder;
>> +
>> +//D3D12 video encoder heap.
>> +ID3D12VideoEncoderHeap *encoder_heap;
>> +
>> +//A cached queue for reusing the D3D12 command allocators.
>> +//@see https://learn.microsoft.com/en-
>us/windows/win32/direct3d12/recording-command-lists-and-
>bundles#id3d12commandallocator
>> +AVFifo *allocator_queue;
>> +
>> +//D3D12 command queue.
>> +ID3D12CommandQueue *command_queue;
>> +
>> +//D3D12 video encode command list.
>> +ID3D12VideoEncodeCommandList2 *command_list;
>> +
>> +//The sync context used to sync command queue.
>> +AVD3D12VASyncContext sync_ctx;
>> +
>> +//bi_not_empty feature.
>> +int bi_not_empty;
>> +
>> +//D3D12 hardware structures.
>> +D3D12_VIDEO_ENCODER_PICTURE_RESOLUTION_DESC resolution;
>> +
>> +D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION codec_conf;
>> +
>> +D3D12_VIDEO_ENCODER_RATE_CONTROL rc;
>> +
>> +D3D12_FEATURE_DATA_VIDEO_ENCODER_RESOURCE_REQUIREMENTS
>req;
>> +
>> +D3D12_VIDEO_ENCODER_SEQUENCE_GOP_STRUCTURE GOP;
>> +
>> +
>D3D12_FEATURE_DATA_VIDEO_ENCODER_RESOLUTION_SUPPORT_LIMITS
>res_limits;
>> +
>> +D3D12_VIDEO_ENCODER_LEVEL_SETTING level;
>> +} D3D12VAEncodeContext;
>> +
>Can we use the comment style the same as D3D12VADecodeContext?
>

Will update in V2 thanks for the review.

Thanks,
Tong

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 7/9] avutil/hwcontext_d3d12va: add Flags for resource creation

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of Mark
>Thompson
>Sent: Tuesday, January 23, 2024 5:52 AM
>To: ffmpeg-devel@ffmpeg.org
>Subject: Re: [FFmpeg-devel] [PATCH 7/9] avutil/hwcontext_d3d12va: add Flags
>for resource creation
>
>On 22/01/2024 05:57, tong1.wu-at-intel@ffmpeg.org wrote:
>> From: Tong Wu 
>>
>> Flags field is added to support diffferent resource creation.
>>
>> Signed-off-by: Tong Wu 
>> ---
>>   doc/APIchanges| 3 +++
>>   libavutil/hwcontext_d3d12va.c | 2 +-
>>   libavutil/hwcontext_d3d12va.h | 5 +
>>   libavutil/version.h   | 2 +-
>>   4 files changed, 10 insertions(+), 2 deletions(-)
>>
>> diff --git a/doc/APIchanges b/doc/APIchanges
>> index e477ed78e0..a33e54dd3b 100644
>> --- a/doc/APIchanges
>> +++ b/doc/APIchanges
>> @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2023-02-
>09
>>
>>   API changes, most recent first:
>>
>> +2024-01-xx - xx - lavu 58.37.100 - hwcontext_d3d12va.h
>> + Add AVD3D12VAFramesContext.Flags
>> +
>>   2023-11-xx - xx - lavfi 9.16.100 - buffersink.h buffersrc.h
>> Add av_buffersink_get_colorspace and av_buffersink_get_color_range.
>> Add AVBufferSrcParameters.color_space and
>AVBufferSrcParameters.color_range.
>> diff --git a/libavutil/hwcontext_d3d12va.c b/libavutil/hwcontext_d3d12va.c
>> index 414dd44290..0d94f48543 100644
>> --- a/libavutil/hwcontext_d3d12va.c
>> +++ b/libavutil/hwcontext_d3d12va.c
>> @@ -237,7 +237,7 @@ static AVBufferRef *d3d12va_pool_alloc(void
>*opaque, size_t size)
>>   .Format   = hwctx->format,
>>   .SampleDesc   = {.Count = 1, .Quality = 0 },
>>   .Layout   = D3D12_TEXTURE_LAYOUT_UNKNOWN,
>> -.Flags= D3D12_RESOURCE_FLAG_NONE,
>> +.Flags= hwctx->Flags,
>
>This seems like a hole in the existing decoder implementation?  How does it
>work without making
>D3D12_RESOURCE_FLAG_VIDEO_DECODE_REFERENCE_ONLY textures when
>required by the device?

For current decoder implementation it applies to most devices and it's indeed 
not supported when driver requests this REFERENCE_ONLY flag. It seems that I 
haven't met this request in my working environment. We may add it later if 
needed.

>
>>   };
>>
>>   frame = av_mallocz(sizeof(AVD3D12VAFrame));
>> diff --git a/libavutil/hwcontext_d3d12va.h b/libavutil/hwcontext_d3d12va.h
>> index ff06e6f2ef..dc1c17d3f9 100644
>> --- a/libavutil/hwcontext_d3d12va.h
>> +++ b/libavutil/hwcontext_d3d12va.h
>> @@ -129,6 +129,11 @@ typedef struct AVD3D12VAFramesContext {
>>* If unset, will be automatically set.
>>*/
>>   DXGI_FORMAT format;
>> +
>> +/**
>> + * This field is used for resource creation.
>
>This documentation could be better.  Used to do what?  Can it not be set, and
>what is the behaviour if it isn't?

Will update in V2.

>
>> + */
>> +D3D12_RESOURCE_FLAGS Flags;
>
>Use lowercase for structure elements.
>
>>   } AVD3D12VAFramesContext;
>>
>>   #endif /* AVUTIL_HWCONTEXT_D3D12VA_H */
>> diff --git a/libavutil/version.h b/libavutil/version.h
>> index 772c4e209c..3ad1a9446c 100644
>> --- a/libavutil/version.h
>> +++ b/libavutil/version.h
>> @@ -79,7 +79,7 @@
>>*/
>>
>>   #define LIBAVUTIL_VERSION_MAJOR  58
>> -#define LIBAVUTIL_VERSION_MINOR  36
>> +#define LIBAVUTIL_VERSION_MINOR  37
>>   #define LIBAVUTIL_VERSION_MICRO 101
>>
>>   #define LIBAVUTIL_VERSION_INT
>AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \
>
>It's not good to be changing the user-facing API like this.  Are there any more
>properties here which really should be user-visible?  Adding them one at a
>time later would be very unfortunate and make compatibility harder.
>
>Thanks,
>
>- Mark

AFAIK, I don't think there're more properties that need to be added one by one 
in a short time. It's going to be just this one that is really needed since we 
have to give specific flags for recon frames creation during encoding.

Thanks for the opinions and I'll update those in next version.

Best Regards,
Tong


>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] avcodec/d3d12va_decode: don't change the resource state if the referenced frame is the same as the current frame

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of Wu
>Jianhua
>Sent: Wednesday, December 27, 2023 9:44 PM
>To: FFmpeg development discussions and patches de...@ffmpeg.org>
>Subject: [FFmpeg-devel] [PATCH v2] avcodec/d3d12va_decode: don't change
>the resource state if the referenced frame is the same as the current frame
>
>This commit removes the follow warning and error:
>
>D3D12 WARNING: ID3D12CommandList::ResourceBarrier: Called on the
>same subresource(s) of
>Resource(0x02236E0E00D0:'Unnamed ID3D12Resource Object') in
>separate Barrier Descs
>which is inefficient and likely unintentional. Desc[0] and Desc[1] on
>(subresource :
>4294967295). [RESOURCE_MANIPULATION WARNING #1008:
>RESOURCE_BARRIER_DUPLICATE_SUBRESOURCE_TRANSITIONS]
>
>D3D12 ERROR: ID3D12CommandList::ResourceBarrier: Before state (0x0:
>D3D12_RESOURCE_STATE_[COMMON|PRESENT])
>of resource (0x02236E0E00D0:'Unnamed ID3D12Resource Object')
>(subresource: 0) specified
>by transition barrier does not match with the state (0x2:
>D3D12_RESOURCE_STATE_VIDEO_DECODE_WRITE)
>specified in the previous call to ResourceBarrier [RESOURCE_MANIPULATION
>ERROR #527:
>RESOURCE_BARRIER_BEFORE_AFTER_MISMATCH]
>
>Patch attached

Tested. It worked for me. Thx.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/d3d12va_decode: don't change the resource state if the referenced frame is the same as the current frame

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of Wu
>Jianhua
>Sent: Tuesday, December 26, 2023 9:21 PM
>To: FFmpeg development discussions and patches de...@ffmpeg.org>
>Subject: [FFmpeg-devel] [PATCH] avcodec/d3d12va_decode: don't change the
>resource state if the referenced frame is the same as the current frame
>
>avcodec/d3d12va_decode: don't change the resource state if the referenced
>frame is the same as the current frame
>
> This commit removes the follow warning and error:
>
>D3D12 WARNING: ID3D12CommandList::ResourceBarrier: Called on the
>same subresource(s) of
>Resource(0x02236E0E00D0:'Unnamed ID3D12Resource Object') in
>separate Barrier Descs
>which is inefficient and likely unintentional. Desc[0] and Desc[1] on
>(subresource :
>4294967295). [RESOURCE_MANIPULATION WARNING #1008:
>RESOURCE_BARRIER_DUPLICATE_SUBRESOURCE_TRANSITIONS]
>
>D3D12 ERROR: ID3D12CommandList::ResourceBarrier: Before state (0x0:
>D3D12_RESOURCE_STATE_[COMMON|PRESENT])
>of resource (0x02236E0E00D0:'Unnamed ID3D12Resource Object')
>(subresource: 0) specified
>by transition barrier does not match with the state (0x2:
>D3D12_RESOURCE_STATE_VIDEO_DECODE_WRITE)
>specified in the previous call to ResourceBarrier [RESOURCE_MANIPULATION
>ERROR #527:
>RESOURCE_BARRIER_BEFORE_AFTER_MISMATCH]
>
>Patch attached

Could you please split the function declaration(header) into 2 lines since it's 
a little bit long?

Thx,
Tong

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v12 2/9] avcodec: add D3D12VA hardware accelerated H264 decoding

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of
>Michael Niedermayer
>Sent: Monday, December 25, 2023 7:07 AM
>To: FFmpeg development discussions and patches de...@ffmpeg.org>
>Subject: Re: [FFmpeg-devel] [PATCH v12 2/9] avcodec: add D3D12VA hardware
>accelerated H264 decoding
>
>On Tue, Dec 05, 2023 at 02:46:44PM +0800, Tong Wu wrote:
>> From: Wu Jianhua 
>>
>> The implementation is based on:
>> https://learn.microsoft.com/en-us/windows/win32/medfound/direct3d-12-
>video-overview
>>
>> With the Direct3D 12 video decoding support, we can render or process
>> the decoded images by the pixel shaders or compute shaders directly
>> without the extra copy overhead, which is beneficial especially if you
>> are trying to render or post-process a 4K or 8K video.
>>
>> The command below is how to enable d3d12va:
>> ffmpeg -hwaccel d3d12va -i input.mp4 output.mp4
>>
>> Signed-off-by: Wu Jianhua 
>> Signed-off-by: Tong Wu 
>> ---
>[...]
>> +unsigned ff_d3d12va_get_surface_index(const AVCodecContext *avctx,
>> +  D3D12VADecodeContext *ctx, const 
>> AVFrame *frame,
>> +  int curr)
>> +{
>> +AVD3D12VAFrame *f;
>> +ID3D12Resource *res;
>> +unsigned i;
>> +
>> +f = (AVD3D12VAFrame *)frame->data[0];
>> +if (!f)
>> +goto fail;
>> +
>> +res = f->texture;
>> +if (!res)
>> +goto fail;
>> +
>> +if (!curr) {
>> +for (i = 0; i < ctx->max_num_ref; i++) {
>> +if (ctx->ref_resources[i] && res == ctx->ref_resources[i]) {
>> +ctx->used_mask |= 1 << i;
>> +return i;
>> +}
>> +}
>> +} else {
>> +for (i = 0; i < ctx->max_num_ref; i++) {
>> +if (!((ctx->used_mask >> i) & 0x1)) {
>> +ctx->ref_resources[i] = res;
>> +return i;
>> +}
>> +}
>> +}
>> +
>> +fail:
>> +assert(0);
>
>this should probably be some av_assert*
>
>thx

Thanks, will send a patch to fix later.

>
>[...]
>--
>Michael GnuPG fingerprint:
>9FF2128B147EF6730BADF133611EC787040B0FAB
>
>If the United States is serious about tackling the national security threats
>related to an insecure 5G network, it needs to rethink the extent to which it
>values corporate profits and government espionage over security.-Bruce
>Schneier
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] configure: Remove a redundant check for UWP mode

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of
>Martin Storsjö
>Sent: Friday, December 22, 2023 4:41 PM
>To: FFmpeg development discussions and patches de...@ffmpeg.org>
>Subject: Re: [FFmpeg-devel] [PATCH 1/2] configure: Remove a redundant check
>for UWP mode
>
>On Fri, 22 Dec 2023, Wu, Tong1 wrote:
>
>>
>>> Subject: [FFmpeg-devel] [PATCH 1/2] configure: Remove a redundant check
>for
>>> UWP mode
>>>
>>> The check for UWP mode was duplicated from right above, in
>>> d54127c41a81cf2078a3504f78e0e4232cfe11b7.
>>>
>>> Also, instead of several lines with "enabled uwp && ...", make one
>>> "if enabled uwp; then" block.
>>> ---
>>> configure | 30 +++---
>>> 1 file changed, 11 insertions(+), 19 deletions(-)
>>>
>>> diff --git a/configure b/configure
>>> index af0bebc1ac..69b755f274 100755
>>> --- a/configure
>>> +++ b/configure
>>> @@ -7102,9 +7102,8 @@ fi
>>>
>>> check_func_headers "windows.h" CreateDIBSection
>>> "$gdigrab_indev_extralibs"
>>>
>>> -# d3d11va requires linking directly to dxgi and d3d11 if not building for
>>> -# the desktop api partition
>>> -test_cpp <>> +# check if building for desktop or uwp
>>> +test_cpp <>> #ifdef WINAPI_FAMILY
>>> #include 
>>> #if WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)
>>> @@ -7117,23 +7116,16 @@ test_cpp <>> d3d11va_extralibs="-ldxgi -ld3d11"
>>> #endif
>>> EOF
>>>
>>> -# vaapi_win32 requires linking directly to dxgi if not building for
>>> -# the desktop api partition
>>> -test_cpp <>> -#ifdef WINAPI_FAMILY
>>> -#include 
>>> -#if WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)
>>> -#error desktop, not uwp
>>> -#else
>>> -// WINAPI_FAMILY_APP, WINAPI_FAMILY_PHONE_APP => UWP
>>> -#endif
>>> -#else
>>> -#error no family set
>>> -#endif
>>> -EOF
>>> +mediafoundation_extralibs="-lmfuuid -lole32 -lstrmiids"
>>>
>>> -# mediafoundation requires linking directly to mfplat if building for uwp
>target
>>> -enabled uwp && mediafoundation_extralibs="-lmfplat -lmfuuid -lole32 -
>>> lstrmiids" || mediafoundation_extralibs="-lmfuuid -lole32 -lstrmiids"
>>> +if enabled uwp; then
>>> +# In UWP mode, we can't use LoadLibrary+GetProcAddress to
>conditionally
>>> +# try to load these APIs at runtime, like we do in regular desktop 
>>> mode -
>>> +# therefore, we need to link directly against these APIs.
>>> +d3d11va_extralibs="-ldxgi -ld3d11"
>>> +vaapi_win32_extralibs="-ldxgi"
>>> +mediafoundation_extralibs="-lmfplat $mediafoundation_extralibs"
>>> +fi
>>>
>>> enabled libdrm &&
>>> check_pkg_config libdrm_getfb2 libdrm "xf86drmMode.h"
>>> drmModeGetFB2
>>> --
>>> 2.34.1
>>
>> LGTM, thx.
>
>Does that cover patch 2/2, which fixes linking errors in the UWP
>configurations, as well?
>
>// Martin

>From the code it looks good to me but I don't have the UWP test environment to 
>try out.

>
>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] configure: Remove a redundant check for UWP mode

[Wu, Tong1]

>Subject: [FFmpeg-devel] [PATCH 1/2] configure: Remove a redundant check for
>UWP mode
>
>The check for UWP mode was duplicated from right above, in
>d54127c41a81cf2078a3504f78e0e4232cfe11b7.
>
>Also, instead of several lines with "enabled uwp && ...", make one
>"if enabled uwp; then" block.
>---
> configure | 30 +++---
> 1 file changed, 11 insertions(+), 19 deletions(-)
>
>diff --git a/configure b/configure
>index af0bebc1ac..69b755f274 100755
>--- a/configure
>+++ b/configure
>@@ -7102,9 +7102,8 @@ fi
>
> check_func_headers "windows.h" CreateDIBSection
>"$gdigrab_indev_extralibs"
>
>-# d3d11va requires linking directly to dxgi and d3d11 if not building for
>-# the desktop api partition
>-test_cpp <+# check if building for desktop or uwp
>+test_cpp < #ifdef WINAPI_FAMILY
> #include 
> #if WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)
>@@ -7117,23 +7116,16 @@ test_cpp  #endif
> EOF
>
>-# vaapi_win32 requires linking directly to dxgi if not building for
>-# the desktop api partition
>-test_cpp <-#ifdef WINAPI_FAMILY
>-#include 
>-#if WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)
>-#error desktop, not uwp
>-#else
>-// WINAPI_FAMILY_APP, WINAPI_FAMILY_PHONE_APP => UWP
>-#endif
>-#else
>-#error no family set
>-#endif
>-EOF
>+mediafoundation_extralibs="-lmfuuid -lole32 -lstrmiids"
>
>-# mediafoundation requires linking directly to mfplat if building for uwp 
>target
>-enabled uwp && mediafoundation_extralibs="-lmfplat -lmfuuid -lole32 -
>lstrmiids" || mediafoundation_extralibs="-lmfuuid -lole32 -lstrmiids"
>+if enabled uwp; then
>+# In UWP mode, we can't use LoadLibrary+GetProcAddress to conditionally
>+# try to load these APIs at runtime, like we do in regular desktop mode -
>+# therefore, we need to link directly against these APIs.
>+d3d11va_extralibs="-ldxgi -ld3d11"
>+vaapi_win32_extralibs="-ldxgi"
>+mediafoundation_extralibs="-lmfplat $mediafoundation_extralibs"
>+fi
>
> enabled libdrm &&
> check_pkg_config libdrm_getfb2 libdrm "xf86drmMode.h"
>drmModeGetFB2
>--
>2.34.1

LGTM, thx.

>
>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/3] avutil/hwcontext_d3d12va: remove unused variables

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of James
>Almer
>Sent: Thursday, December 21, 2023 8:47 PM
>To: ffmpeg-devel@ffmpeg.org
>Subject: Re: [FFmpeg-devel] [PATCH 3/3] avutil/hwcontext_d3d12va: remove
>unused variables
>
>On 12/21/2023 9:42 AM, Martin Storsjö wrote:
>> On Thu, 21 Dec 2023, James Almer wrote:
>>
>>> Removes -Wunused-variable warnings.
>>>
>>> Signed-off-by: James Almer 
>>> ---
>>> Were this meant to be used? Or are just a remnant of a previous
>>> interation of
>>> the set?

They are just leftovers of the previous version. Patch set LGTM, thanks.

>>>
>>> libavutil/hwcontext_d3d12va.c | 4 
>>> 1 file changed, 4 deletions(-)
>>>
>>> diff --git a/libavutil/hwcontext_d3d12va.c
>>> b/libavutil/hwcontext_d3d12va.c
>>> index 4ce29f250c..46832fd19b 100644
>>> --- a/libavutil/hwcontext_d3d12va.c
>>> +++ b/libavutil/hwcontext_d3d12va.c
>>> @@ -173,7 +173,6 @@ fail:
>>>
>>> static void d3d12va_frames_uninit(AVHWFramesContext *ctx)
>>> {
>>> -    AVD3D12VAFramesContext *frames_hwctx = ctx->hwctx;
>>>     D3D12VAFramesContext   *s    = ctx->internal->priv;
>>
>> Here, I'd prefer to get rid of the extra spaces once the other line,
>> which it was aligned to, is gone.
>>
>>> static int d3d12va_frames_init(AVHWFramesContext *ctx)
>>> {
>>>     AVD3D12VAFramesContext *hwctx    = ctx->hwctx;
>>> -    AVD3D12VADeviceContext *device_hwctx = ctx->device_ctx->hwctx;
>>> -    D3D12VAFramesContext   *s    = ctx->internal->priv;
>>
>> Ditto
>
>Amended locally. Thanks.
>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avutil/hwcontext_d3d12va: remove an unused function

[Wu, Tong1]

>
>On Thu, 21 Dec 2023, Tong Wu wrote:
>
>> Signed-off-by: Tong Wu 
>> ---
>> libavutil/hwcontext_d3d12va.c | 9 -
>> 1 file changed, 9 deletions(-)
>>
>> diff --git a/libavutil/hwcontext_d3d12va.c b/libavutil/hwcontext_d3d12va.c
>> index 1600d94cb0..4995518dbd 100644
>> --- a/libavutil/hwcontext_d3d12va.c
>> +++ b/libavutil/hwcontext_d3d12va.c
>> @@ -71,15 +71,6 @@ static void d3d12va_default_unlock(void *ctx)
>> ReleaseMutex(ctx);
>> }
>>
>> -DXGI_FORMAT av_d3d12va_map_sw_to_hw_format(enum AVPixelFormat
>pix_fmt)
>> -{
>> -switch (pix_fmt) {
>> -case AV_PIX_FMT_NV12:return DXGI_FORMAT_NV12;
>> -case AV_PIX_FMT_P010:return DXGI_FORMAT_P010;
>> -default: return DXGI_FORMAT_UNKNOWN;
>> -}
>> -}
>> -
>> static int d3d12va_fence_completion(AVD3D12VASyncContext *psync_ctx)
>> {
>> uint64_t completion = ID3D12Fence_GetCompletedValue(psync_ctx-
>>fence);
>> --
>> 2.41.0.windows.1
>
>LGTM, thanks.
>
>It might be good to mention that it was lacking a public declaration,
>which caused builds with -Werror=missing-prototypes to fail, and the
>reason why it was left behind and not needed.
>
>// Martin

Thanks, updated it.

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v12 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

>There are still no public external API users to actually test the API,
>which is concerning. It took months for the Vulkan API to be portable
>enough to work with the major Vulkan users out there. Granted, Vulkan
>is special.
>
>Is there any public test code out there? If not, I'd prefer the API to be
>declared as experimental for now, so we can make changes if needed.
>

Actually there have already been a number of discussions on this before for VLC 
and MPV.
https://code.videolan.org/videolan/vlc/-/issues/28360
https://github.com/mpv-player/mpv/issues/12071

Some have tried by themselves and it's working well. Since it cannot be 
official and public before it lands in FFmpeg, I believe it's probably very 
soon to have d3d12 hwdec support for them.

Thanks,
Tong


>Dec 21, 2023, 09:31 by haihao.xiang-at-intel@ffmpeg.org:
>
>> On Ma, 2023-12-18 at 06:28 +, Xiang, Haihao wrote:
>>
>>> On Di, 2023-12-05 at 14:46 +0800, Tong Wu wrote:
>>> > From: Wu Jianhua 
>>> >
>>> > Signed-off-by: Wu Jianhua 
>>> > Signed-off-by: Tong Wu 
>>> > ---
>>> >
>>> > Compared to v11, v12 set the initial value to 0 for the fence value,
>>> > fixing a potential dynamic pool sync issue. Hence removed the non zero
>>> > initial_pool_size for
>>> > d3d12va decoder.
>>> >
>>> > Some other minor error message changes.
>>> >
>>> >
>>> >
>>> >  configure  |   5 +
>>> >  doc/APIchanges |   5 +
>>> >  libavutil/Makefile |   3 +
>>> >  libavutil/hwcontext.c  |   4 +
>>> >  libavutil/hwcontext.h  |   1 +
>>> >  libavutil/hwcontext_d3d12va.c  | 703 +
>>> >  libavutil/hwcontext_d3d12va.h  | 134 +
>>> >  libavutil/hwcontext_d3d12va_internal.h |  59 +++
>>> >  libavutil/hwcontext_internal.h |   1 +
>>> >  libavutil/pixdesc.c    |   4 +
>>> >  libavutil/pixfmt.h |   7 +
>>> >  libavutil/tests/hwdevice.c |   2 +
>>> >  libavutil/version.h    |   2 +-
>>> >  13 files changed, 929 insertions(+), 1 deletion(-)
>>> >  create mode 100644 libavutil/hwcontext_d3d12va.c
>>> >  create mode 100644 libavutil/hwcontext_d3d12va.h
>>> >  create mode 100644 libavutil/hwcontext_d3d12va_internal.h
>>> >
>>> > diff --git a/configure b/configure
>>> > index d77c053226..1a343ac6b9 100755
>>> > --- a/configure
>>> > +++ b/configure
>>> > @@ -336,6 +336,7 @@ External library support:
>>> >    --disable-cuda-llvm  disable CUDA compilation using clang
>>> > [autodetect]
>>> >    --disable-cuvid  disable Nvidia CUVID support [autodetect]
>>> >    --disable-d3d11va    disable Microsoft Direct3D 11 video 
>>> > acceleration
>>> > code [autodetect]
>>> > +  --disable-d3d12va    disable Microsoft Direct3D 12 video 
>>> > acceleration
>>> > code [autodetect]
>>> >    --disable-dxva2  disable Microsoft DirectX 9 video acceleration
>>> > code [autodetect]
>>> >    --disable-ffnvcodec  disable dynamically linked Nvidia code
>>> > [autodetect]
>>> >    --enable-libdrm  enable DRM code (Linux) [no]
>>> > @@ -1926,6 +1927,7 @@ HWACCEL_AUTODETECT_LIBRARY_LIST="
>>> >  cuda_llvm
>>> >  cuvid
>>> >  d3d11va
>>> > +    d3d12va
>>> >  dxva2
>>> >  ffnvcodec
>>> >  nvdec
>>> > @@ -3048,6 +3050,7 @@ crystalhd_deps="libcrystalhd_libcrystalhd_if_h"
>>> >  cuda_deps="ffnvcodec"
>>> >  cuvid_deps="ffnvcodec"
>>> >  d3d11va_deps="dxva_h ID3D11VideoDecoder ID3D11VideoContext"
>>> > +d3d12va_deps="dxva_h ID3D12Device ID3D12VideoDecoder"
>>> >  dxva2_deps="dxva2api_h DXVA2_ConfigPictureDecode ole32 user32"
>>> >  ffnvcodec_deps_any="libdl LoadLibrary"
>>> >  mediacodec_deps="android"
>>> > @@ -6575,6 +6578,8 @@ check_type "windows.h dxgi1_2.h"
>"IDXGIOutput1"
>>> >  check_type "windows.h dxgi1_5.h" "IDXGIOutput5"
>>> >  check_type "windows.h d3d11.h" "ID3D11VideoDecoder"
>>> >  check_type "windows.h d3d11.h" "ID3D11VideoContext"
>>> > +check_type "windows.h d3d12.h" "ID3D12Device"
>>> > +check_type "windows.h d3d12video.h" "ID3D12VideoDecoder"
>>> >  check_type "windows.h" "DPI_AWARENESS_CONTEXT" -
>D_WIN32_WINNT=0x0A00
>>> >  check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -
>>> > D_WIN32_WINNT=0x0602
>>> >  check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat
>>> > diff --git a/doc/APIchanges b/doc/APIchanges
>>> > index 4a2dc1c44f..0435926d80 100644
>>> > --- a/doc/APIchanges
>>> > +++ b/doc/APIchanges
>>> > @@ -2,6 +2,11 @@ The last version increases of all libraries were on
>2023-
>>> > 02-
>>> > 09
>>> >
>>> >  API changes, most recent first:
>>> >
>>> > +2023-11-07 - xx - lavu 58.33.100 - pixfmt.h hwcontext.h
>>> > hwcontext_d3d12va.h
>>> > +  Add AV_HWDEVICE_TYPE_D3D12VA and AV_PIX_FMT_D3D12.
>>> > +  Add AVD3D12VADeviceContext, AVD3D12VASyncContext,
>AVD3D12VAFrame and
>>> > +  AVD3D12VAFramesContext.
>>> > +
>>> >  2023-11-08 - b82957a66a7 - lavu 58.32.100 - 

Re: [FFmpeg-devel] [PATCH] d3d12va: Add a missing include for the declaration of ff_d3d12va_get_surface_index

[Wu, Tong1]

>From: ffmpeg-devel  On Behalf Of
>Martin Storsjö
>Sent: Thursday, December 21, 2023 5:48 PM
>To: ffmpeg-devel@ffmpeg.org
>Cc: Xiang, Haihao 
>Subject: Re: [FFmpeg-devel] [PATCH] d3d12va: Add a missing include for the
>declaration of ff_d3d12va_get_surface_index
>
>On Thu, 21 Dec 2023, Martin Storsjö wrote:
>
>> This fixes the following build error:
>>
>> src/libavcodec/d3d12va_decode.c:49:10: error: no previous prototype for
>function
>> 'ff_d3d12va_get_surface_index' [-Werror,-Wmissing-prototypes]
>>   49 | unsigned ff_d3d12va_get_surface_index(const AVCodecContext *avctx,
>>  |  ^
>> ---
>> libavcodec/d3d12va_decode.c | 1 +
>> 1 file changed, 1 insertion(+)
>
>Even after this change, the build still fails on a later file:
>
>src/libavutil/hwcontext_d3d12va.c:74:13: error: no previous prototype for
>function 'av_d3d12va_map_sw_to_hw_format' [-Werror,-Wmissing-
>prototypes]
>74 | DXGI_FORMAT av_d3d12va_map_sw_to_hw_format(enum
>AVPixelFormat pix_fmt)
>   | ^
>
>There's no declaration of this in any header - so please either make it
>static or ff_ prefixed, or add it to a header with the declaration visible
>at the function definition.
>
>// Martin

Thanks for pointing this out. I've sent a patch to remove this function. It's 
been useless after the d3d format became public at some previous version.

>
>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] d3d12va: Add a missing include for the declaration of ff_d3d12va_get_surface_index

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of
>Martin Storsjö
>Sent: Thursday, December 21, 2023 5:45 PM
>To: ffmpeg-devel@ffmpeg.org
>Cc: Xiang, Haihao 
>Subject: [FFmpeg-devel] [PATCH] d3d12va: Add a missing include for the
>declaration of ff_d3d12va_get_surface_index
>
>This fixes the following build error:
>
>src/libavcodec/d3d12va_decode.c:49:10: error: no previous prototype for
>function
> 'ff_d3d12va_get_surface_index' [-Werror,-Wmissing-prototypes]
>   49 | unsigned ff_d3d12va_get_surface_index(const AVCodecContext *avctx,
>  |  ^
>---
> libavcodec/d3d12va_decode.c | 1 +
> 1 file changed, 1 insertion(+)
>
>diff --git a/libavcodec/d3d12va_decode.c b/libavcodec/d3d12va_decode.c
>index 03e565066c..e0b67bf964 100644
>--- a/libavcodec/d3d12va_decode.c
>+++ b/libavcodec/d3d12va_decode.c
>@@ -33,6 +33,7 @@
> #include "avcodec.h"
> #include "decode.h"
> #include "d3d12va_decode.h"
>+#include "dxva2_internal.h"
>
> typedef struct HelperObjects {
> ID3D12CommandAllocator *command_allocator;
>--
>2.34.1
>

LGTM, thx.

>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v10 2/9] avcodec: add D3D12VA hardware accelerated H264 decoding

[Wu, Tong1]

>> The implementation is based on:
>> https://learn.microsoft.com/en-us/windows/win32/medfound/direct3d-12-
>video-overview
>>
>> With the Direct3D 12 video decoding support, we can render or process
>> the decoded images by the pixel shaders or compute shaders directly
>> without the extra copy overhead, which is beneficial especially if you
>> are trying to render or post-process a 4K or 8K video.
>>
>> The command below is how to enable d3d12va:
>> ffmpeg -hwaccel d3d12va -i input.mp4 output.mp4
>>
>> Signed-off-by: Wu Jianhua 
>> Signed-off-by: Tong Wu 
>> ---
>>  configure   |   2 +
>>  libavcodec/Makefile |   3 +
>>  libavcodec/d3d11va.h|   3 -
>>  libavcodec/d3d12va_decode.c | 564
>
>>  libavcodec/d3d12va_decode.h | 179 
>>  libavcodec/d3d12va_h264.c   | 207 +
>>  libavcodec/dxva2.c  |  29 +-
>>  libavcodec/dxva2.h  |   3 -
>>  libavcodec/dxva2_av1.c  |   4 +-
>>  libavcodec/dxva2_h264.c |  26 +-
>>  libavcodec/dxva2_hevc.c |   6 +-
>>  libavcodec/dxva2_internal.h |  74 ++---
>>  libavcodec/dxva2_mpeg2.c|   6 +-
>>  libavcodec/dxva2_vc1.c  |   6 +-
>>  libavcodec/dxva2_vp9.c  |   6 +-
>>  libavcodec/h264_slice.c |   4 +
>>  libavcodec/h264dec.c|   3 +
>>  libavcodec/hwaccels.h   |   1 +
>>  libavcodec/hwconfig.h   |   2 +
>>  19 files changed, 1061 insertions(+), 67 deletions(-)
>>  create mode 100644 libavcodec/d3d12va_decode.c
>>  create mode 100644 libavcodec/d3d12va_decode.h
>>  create mode 100644 libavcodec/d3d12va_h264.c
>
>breaks build with mingw64 here
>
>make
>CC libavcodec/dxva2.o
>In file included from src/libavcodec/dxva2.c:33:0:
>src/libavcodec/dxva2_internal.h:141:39: error: unknown type name
>‘D3D12VADecodeContext’; did you mean ‘AVD3D11VADeviceContext’?
>   D3D12VADecodeContext *ctx, const 
> AVFrame *frame,
>   ^~~~
>   AVD3D11VADeviceContext
>src/ffbuild/common.mak:81: recipe for target 'libavcodec/dxva2.o' failed
>make: *** [libavcodec/dxva2.o] Error 1
>

Thanks for pointing this out. I'll add #if CONFIG_D3D12VA for this function int 
next patch set. BTW it seems CONFIG_D3D12VA was not automatically enabled on 
your side.

Thanks,
Tong

>
>[...]
>--
>Michael GnuPG fingerprint:
>9FF2128B147EF6730BADF133611EC787040B0FAB
>
>Old school: Use the lowest level language in which you can solve the problem
>conveniently.
>New school: Use the highest level language in which the latest supercomputer
>can solve the problem without the user falling asleep waiting.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v10 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

Patch set V10 major changes:

1. Put more explanations for event object (jb)
2. Put DXGI_FORMAT to public structure (Lynne)
3. Implemented uploading method (Lynne)
4. Supported dynamic pool for hwcontext_d3d12va. Removed useless index from 
every texture. Unbind texture from frames context, giving users more freedom to 
reuse the textures. (Lynne)
5. Avoided the thread-safe problem in transfer_data(Lynne)
6. Removed av_d3d12va_sync_context_alloc and av_d3d12va_sync_context_free. 
Avoided to allocated sync_ctx dynamically. (Lynne)

Thanks,
Tong



>
>From: Wu Jianhua 
>
>Signed-off-by: Wu Jianhua 
>Signed-off-by: Tong Wu 
>---
> configure  |   5 +
> doc/APIchanges |   5 +
> libavutil/Makefile |   3 +
> libavutil/hwcontext.c  |   4 +
> libavutil/hwcontext.h  |   1 +
> libavutil/hwcontext_d3d12va.c  | 704 +
> libavutil/hwcontext_d3d12va.h  | 134 +
> libavutil/hwcontext_d3d12va_internal.h |  59 +++
> libavutil/hwcontext_internal.h |   1 +
> libavutil/pixdesc.c|   4 +
> libavutil/pixfmt.h |   7 +
> libavutil/tests/hwdevice.c |   2 +
> libavutil/version.h|   2 +-
> 13 files changed, 930 insertions(+), 1 deletion(-)
> create mode 100644 libavutil/hwcontext_d3d12va.c
> create mode 100644 libavutil/hwcontext_d3d12va.h
> create mode 100644 libavutil/hwcontext_d3d12va_internal.h
>
>diff --git a/configure b/configure
>index 838e627084..cdeed9bab1 100755
>--- a/configure
>+++ b/configure
>@@ -336,6 +336,7 @@ External library support:
>   --disable-cuda-llvm  disable CUDA compilation using clang [autodetect]
>   --disable-cuvid  disable Nvidia CUVID support [autodetect]
>   --disable-d3d11vadisable Microsoft Direct3D 11 video acceleration 
> code
>[autodetect]
>+  --disable-d3d12vadisable Microsoft Direct3D 12 video acceleration 
>code
>[autodetect]
>   --disable-dxva2  disable Microsoft DirectX 9 video acceleration code
>[autodetect]
>   --disable-ffnvcodec  disable dynamically linked Nvidia code [autodetect]
>   --enable-libdrm  enable DRM code (Linux) [no]
>@@ -1926,6 +1927,7 @@ HWACCEL_AUTODETECT_LIBRARY_LIST="
> cuda_llvm
> cuvid
> d3d11va
>+d3d12va
> dxva2
> ffnvcodec
> nvdec
>@@ -3048,6 +3050,7 @@ crystalhd_deps="libcrystalhd_libcrystalhd_if_h"
> cuda_deps="ffnvcodec"
> cuvid_deps="ffnvcodec"
> d3d11va_deps="dxva_h ID3D11VideoDecoder ID3D11VideoContext"
>+d3d12va_deps="dxva_h ID3D12Device ID3D12VideoDecoder"
> dxva2_deps="dxva2api_h DXVA2_ConfigPictureDecode ole32 user32"
> ffnvcodec_deps_any="libdl LoadLibrary"
> mediacodec_deps="android"
>@@ -6575,6 +6578,8 @@ check_type "windows.h dxgi1_2.h" "IDXGIOutput1"
> check_type "windows.h dxgi1_5.h" "IDXGIOutput5"
> check_type "windows.h d3d11.h" "ID3D11VideoDecoder"
> check_type "windows.h d3d11.h" "ID3D11VideoContext"
>+check_type "windows.h d3d12.h" "ID3D12Device"
>+check_type "windows.h d3d12video.h" "ID3D12VideoDecoder"
> check_type "windows.h" "DPI_AWARENESS_CONTEXT" -
>D_WIN32_WINNT=0x0A00
> check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -
>D_WIN32_WINNT=0x0602
> check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat
>diff --git a/doc/APIchanges b/doc/APIchanges
>index 4a2dc1c44f..0435926d80 100644
>--- a/doc/APIchanges
>+++ b/doc/APIchanges
>@@ -2,6 +2,11 @@ The last version increases of all libraries were on 2023-02-
>09
>
> API changes, most recent first:
>
>+2023-11-07 - xx - lavu 58.33.100 - pixfmt.h hwcontext.h
>hwcontext_d3d12va.h
>+  Add AV_HWDEVICE_TYPE_D3D12VA and AV_PIX_FMT_D3D12.
>+  Add AVD3D12VADeviceContext, AVD3D12VASyncContext, AVD3D12VAFrame
>and
>+  AVD3D12VAFramesContext.
>+
> 2023-11-08 - b82957a66a7 - lavu 58.32.100 - channel_layout.h
>   Add AV_CH_LAYOUT_7POINT2POINT3 and
>AV_CHANNEL_LAYOUT_7POINT2POINT3.
>   Add AV_CH_LAYOUT_9POINT1POINT4_BACK and
>AV_CHANNEL_LAYOUT_9POINT1POINT4_BACK.
>diff --git a/libavutil/Makefile b/libavutil/Makefile
>index 4711f8cde8..6a8566f1d9 100644
>--- a/libavutil/Makefile
>+++ b/libavutil/Makefile
>@@ -42,6 +42,7 @@ HEADERS = adler32.h  
>   \
>   hwcontext.h   \
>   hwcontext_cuda.h  \
>   hwcontext_d3d11va.h   \
>+  hwcontext_d3d12va.h   \
>   hwcontext_drm.h   \
>   hwcontext_dxva2.h \
>   hwcontext_qsv.h   \
>@@ -190,6 +191,7 @@ OBJS = adler32.o   
> \
>
> OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o
> 

Re: [FFmpeg-devel] [PATCH v9 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

>Nov 8, 2023, 02:06 by tong1.wu-at-intel@ffmpeg.org:
>
>> From: Wu Jianhua <> toq...@outlook.com> >
>>
>> Signed-off-by: Wu Jianhua <> toq...@outlook.com> >
>> Signed-off-by: Tong Wu <> tong1...@intel.com> >
>> ---
>> configure  |   5 +
>> doc/APIchanges |   7 +
>> libavutil/Makefile |   3 +
>> libavutil/hwcontext.c  |   4 +
>> libavutil/hwcontext.h  |   1 +
>> libavutil/hwcontext_d3d12va.c  | 695 +
>> libavutil/hwcontext_d3d12va.h  | 142 +
>> libavutil/hwcontext_d3d12va_internal.h |  59 +++
>> libavutil/hwcontext_internal.h |   1 +
>> libavutil/pixdesc.c|   4 +
>> libavutil/pixfmt.h |   7 +
>> libavutil/tests/hwdevice.c |   2 +
>> libavutil/version.h|   2 +-
>> 13 files changed, 931 insertions(+), 1 deletion(-)
>> create mode 100644 libavutil/hwcontext_d3d12va.c
>> create mode 100644 libavutil/hwcontext_d3d12va.h
>> create mode 100644 libavutil/hwcontext_d3d12va_internal.h
>>
>> diff --git a/configure b/configure
>> index 8ab658f730..69d7a48280 100755
>> --- a/configure
>> +++ b/configure
>> @@ -334,6 +334,7 @@ External library support:
>> --disable-cuda-llvm  disable CUDA compilation using clang [autodetect]
>> --disable-cuvid  disable Nvidia CUVID support [autodetect]
>> --disable-d3d11vadisable Microsoft Direct3D 11 video acceleration 
>> code
>[autodetect]
>> +  --disable-d3d12vadisable Microsoft Direct3D 12 video acceleration
>code [autodetect]
>> --disable-dxva2  disable Microsoft DirectX 9 video acceleration code
>[autodetect]
>> --disable-ffnvcodec  disable dynamically linked Nvidia code [autodetect]
>> --enable-libdrm  enable DRM code (Linux) [no]
>> @@ -1922,6 +1923,7 @@ HWACCEL_AUTODETECT_LIBRARY_LIST="
>> cuda_llvm
>> cuvid
>> d3d11va
>> +d3d12va
>> dxva2
>> ffnvcodec
>> nvdec
>> @@ -3041,6 +3043,7 @@ crystalhd_deps="libcrystalhd_libcrystalhd_if_h"
>> cuda_deps="ffnvcodec"
>> cuvid_deps="ffnvcodec"
>> d3d11va_deps="dxva_h ID3D11VideoDecoder ID3D11VideoContext"
>> +d3d12va_deps="dxva_h ID3D12Device ID3D12VideoDecoder"
>> dxva2_deps="dxva2api_h DXVA2_ConfigPictureDecode ole32 user32"
>> ffnvcodec_deps_any="libdl LoadLibrary"
>> mediacodec_deps="android"
>> @@ -6563,6 +6566,8 @@ check_type "windows.h dxgi1_2.h"
>"IDXGIOutput1"
>> check_type "windows.h dxgi1_5.h" "IDXGIOutput5"
>> check_type "windows.h d3d11.h" "ID3D11VideoDecoder"
>> check_type "windows.h d3d11.h" "ID3D11VideoContext"
>> +check_type "windows.h d3d12.h" "ID3D12Device"
>> +check_type "windows.h d3d12video.h" "ID3D12VideoDecoder"
>> check_type "windows.h" "DPI_AWARENESS_CONTEXT" -
>D_WIN32_WINNT=0x0A00
>> check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -
>D_WIN32_WINNT=0x0602
>> check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat
>> diff --git a/doc/APIchanges b/doc/APIchanges
>> index d4511ce2dd..2e71943b34 100644
>> --- a/doc/APIchanges
>> +++ b/doc/APIchanges
>> @@ -2,6 +2,13 @@ The last version increases of all libraries were on 2023-
>02-09
>>
>> API changes, most recent first:
>>
>> +2023-11-07 - xx - lavu 58.32.100 - pixfmt.h hwcontext.h
>hwcontext_d3d12va.h
>> +  Add AV_HWDEVICE_TYPE_D3D12VA and AV_PIX_FMT_D3D12.
>> +  Add AVD3D12VADeviceContext, AVD3D12VASyncContext,
>AVD3D12VAFrame and
>> +  AVD3D12VAFramesContext.
>> +  Add av_d3d12va_map_sw_to_hw_format,
>av_d3d12va_sync_context_alloc,
>> +  av_d3d12va_sync_context_free.
>> +
>> 2023-10-31 - xx - lavu 58.31.100 - pixdesc.h
>> Add AV_PIX_FMT_FLAG_XYZ.
>>
>> diff --git a/libavutil/Makefile b/libavutil/Makefile
>> index 4711f8cde8..6a8566f1d9 100644
>> --- a/libavutil/Makefile
>> +++ b/libavutil/Makefile
>> @@ -42,6 +42,7 @@ HEADERS = adler32.h
>>  \
>> hwcontext.h   \
>> hwcontext_cuda.h  \
>> hwcontext_d3d11va.h   \
>> +  hwcontext_d3d12va.h  

Re: [FFmpeg-devel] [PATCH v9 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of Jean-
>Baptiste Kempf
>Sent: Wednesday, November 8, 2023 7:04 PM
>To: fmpeg-devel 
>Subject: Re: [FFmpeg-devel] [PATCH v9 1/9] libavutil: add hwcontext_d3d12va
>and AV_PIX_FMT_D3D12
>
>Hello,
>
>On Wed, 8 Nov 2023, at 02:05, Tong Wu wrote:
>> +/**
>> + * @brief This struct is used to sync d3d12 execution
>> + *
>> + */
>> +typedef struct AVD3D12VASyncContext {
>> +/**
>> + * D3D12 fence object
>> + */
>> +ID3D12Fence *fence;
>> +
>> +/**
>> + * A handle to the event object
>> + */
>> +HANDLE event;
>
>Sorry, but which event object? I would guess it's the ID3D12Fence::Signal, but
>I'm not even sure.
>
>I think this needs a bit more doc and explanations.
>

Hi there. Yes it's the event that's raised when the fence reaches a certain 
value.
I will put more details in the doc. Thanks.

>> +
>> +/**
>> + * The fence value used for sync
>> + */
>> +uint64_t fence_value;
>> +} AVD3D12VASyncContext;
>> +
>> +/**
>> + * @brief D3D12VA frame descriptor for pool allocation.
>> + *
>> + */
>> +typedef struct AVD3D12VAFrame {
>> +/**
>> + * The texture in which the frame is located. The reference count
>> is
>> + * managed by the AVBufferRef, and destroying the reference will
>> release
>> + * the interface.
>> + */
>> +ID3D12Resource *texture;
>> +
>> +/**
>> + * The index into the array texture element representing the frame
>> + */
>> +intptr_t index;
>> +
>> +/**
>> + * The sync context for the texture
>> + *
>> + * Use av_d3d12va_wait_idle(sync_ctx) to ensure the decoding or
>> encoding have been finised
>> + * @see:
>> https://learn.microsoft.com/en-us/windows/win32/medfound/direct3d-12-
>video-overview#directx-12-fences
>> + */
>> +AVD3D12VASyncContext *sync_ctx;
>> +} AVD3D12VAFrame;
>> +
>> +/**
>> + * @brief This struct is allocated as AVHWFramesContext.hwctx
>> + *
>> + */
>> +typedef struct AVD3D12VAFramesContext {
>> +/**
>> + * This field is not able to be user-allocated at the present.
>> + */
>> +AVD3D12VAFrame *texture_infos;
>> +} AVD3D12VAFramesContext;
>> +
>> +/**
>> + * @brief Map sw pixel format to d3d12 format
>> + *
>> + * @return d3d12 specified format
>> + */
>> +DXGI_FORMAT av_d3d12va_map_sw_to_hw_format(enum AVPixelFormat
>pix_fmt);
>> +
>> +/**
>> + * @brief Allocate an AVD3D12VASyncContext
>> + *
>> + * @return Error code (ret < 0 if failed)
>> + */
>> +int av_d3d12va_sync_context_alloc(AVD3D12VADeviceContext *ctx,
>> AVD3D12VASyncContext **sync_ctx);
>> +
>> +/**
>> + * @brief Free an AVD3D12VASyncContext
>> + */
>> +void av_d3d12va_sync_context_free(AVD3D12VASyncContext **sync_ctx);
>> +
>> +#endif /* AVUTIL_HWCONTEXT_D3D12VA_H */
>> diff --git a/libavutil/hwcontext_d3d12va_internal.h
>> b/libavutil/hwcontext_d3d12va_internal.h
>> new file mode 100644
>> index 00..bfd89b3545
>> --- /dev/null
>> +++ b/libavutil/hwcontext_d3d12va_internal.h
>> @@ -0,0 +1,59 @@
>> +/*
>> + * Direct3D 12 HW acceleration.
>> + *
>> + * copyright (c) 2022-2023 Wu Jianhua 
>> + *
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with FFmpeg; if not, write to the Free Software
>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
>> 02110-1301 USA
>> + */
>> +
>> +#ifndef AVUTIL_HWCONTEXT_D3D12VA_INTERNAL_H
>> +#define AVUTIL_HWCONTEXT_D3D12VA_INTERNAL_H
>> +
>> +/**
>> + * @def COBJMACROS
>> + *
>> + * @brief Enable C style interface for D3D12
>> + */
>> +#ifndef COBJMACROS
>> +#define COBJMACROS
>> +#endif
>> +
>> +/**
>> + * @def DX_CHECK
>> + *
>> + * @brief A check macro used by D3D12 functions highly frequently
>> + */
>> +#define DX_CHECK(hr)  \
>> +do {  \
>> +if (FAILED(hr))   \
>> +goto fail;\
>> +} while (0)
>> +
>> +/**
>> + * @def D3D12_OBJECT_RELEASE
>> + *
>> + * @brief A release macro used by D3D12 objects highly frequently
>> + */
>> +#define D3D12_OBJECT_RELEASE(pInterface)  \
>> +do {  \
>> +if (pInterface) { \
>> +IUnknown_Release((IUnknown *)pInterface); \
>> +pInterface = NULL;   

Re: [FFmpeg-devel] [PATCH v9 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

Since 6.1 release has branched out. Resend this patch set and add this feature 
to the next release version.

> configure  |   5 +
> doc/APIchanges |   7 +
> libavutil/Makefile |   3 +
> libavutil/hwcontext.c  |   4 +
> libavutil/hwcontext.h  |   1 +
> libavutil/hwcontext_d3d12va.c  | 695 +
> libavutil/hwcontext_d3d12va.h  | 142 +
> libavutil/hwcontext_d3d12va_internal.h |  59 +++
> libavutil/hwcontext_internal.h |   1 +
> libavutil/pixdesc.c|   4 +
> libavutil/pixfmt.h |   7 +
> libavutil/tests/hwdevice.c |   2 +
> libavutil/version.h|   2 +-
> 13 files changed, 931 insertions(+), 1 deletion(-)
> create mode 100644 libavutil/hwcontext_d3d12va.c
> create mode 100644 libavutil/hwcontext_d3d12va.h
> create mode 100644 libavutil/hwcontext_d3d12va_internal.h
>
>diff --git a/configure b/configure
>index 8ab658f730..69d7a48280 100755
>--- a/configure
>+++ b/configure
>@@ -334,6 +334,7 @@ External library support:
>   --disable-cuda-llvm  disable CUDA compilation using clang [autodetect]
>   --disable-cuvid  disable Nvidia CUVID support [autodetect]
>   --disable-d3d11vadisable Microsoft Direct3D 11 video acceleration 
> code
>[autodetect]
>+  --disable-d3d12vadisable Microsoft Direct3D 12 video acceleration 
>code
>[autodetect]
>   --disable-dxva2  disable Microsoft DirectX 9 video acceleration code
>[autodetect]
>   --disable-ffnvcodec  disable dynamically linked Nvidia code [autodetect]
>   --enable-libdrm  enable DRM code (Linux) [no]
>@@ -1922,6 +1923,7 @@ HWACCEL_AUTODETECT_LIBRARY_LIST="
> cuda_llvm
> cuvid
> d3d11va
>+d3d12va
> dxva2
> ffnvcodec
> nvdec
>@@ -3041,6 +3043,7 @@ crystalhd_deps="libcrystalhd_libcrystalhd_if_h"
> cuda_deps="ffnvcodec"
> cuvid_deps="ffnvcodec"
> d3d11va_deps="dxva_h ID3D11VideoDecoder ID3D11VideoContext"
>+d3d12va_deps="dxva_h ID3D12Device ID3D12VideoDecoder"
> dxva2_deps="dxva2api_h DXVA2_ConfigPictureDecode ole32 user32"
> ffnvcodec_deps_any="libdl LoadLibrary"
> mediacodec_deps="android"
>@@ -6563,6 +6566,8 @@ check_type "windows.h dxgi1_2.h" "IDXGIOutput1"
> check_type "windows.h dxgi1_5.h" "IDXGIOutput5"
> check_type "windows.h d3d11.h" "ID3D11VideoDecoder"
> check_type "windows.h d3d11.h" "ID3D11VideoContext"
>+check_type "windows.h d3d12.h" "ID3D12Device"
>+check_type "windows.h d3d12video.h" "ID3D12VideoDecoder"
> check_type "windows.h" "DPI_AWARENESS_CONTEXT" -
>D_WIN32_WINNT=0x0A00
> check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -
>D_WIN32_WINNT=0x0602
> check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat
>diff --git a/doc/APIchanges b/doc/APIchanges
>index d4511ce2dd..2e71943b34 100644
>--- a/doc/APIchanges
>+++ b/doc/APIchanges
>@@ -2,6 +2,13 @@ The last version increases of all libraries were on 2023-02-
>09
>
> API changes, most recent first:
>
>+2023-11-07 - xx - lavu 58.32.100 - pixfmt.h hwcontext.h
>hwcontext_d3d12va.h
>+  Add AV_HWDEVICE_TYPE_D3D12VA and AV_PIX_FMT_D3D12.
>+  Add AVD3D12VADeviceContext, AVD3D12VASyncContext, AVD3D12VAFrame
>and
>+  AVD3D12VAFramesContext.
>+  Add av_d3d12va_map_sw_to_hw_format, av_d3d12va_sync_context_alloc,
>+  av_d3d12va_sync_context_free.
>+
> 2023-10-31 - xx - lavu 58.31.100 - pixdesc.h
>   Add AV_PIX_FMT_FLAG_XYZ.
>
>diff --git a/libavutil/Makefile b/libavutil/Makefile
>index 4711f8cde8..6a8566f1d9 100644
>--- a/libavutil/Makefile
>+++ b/libavutil/Makefile
>@@ -42,6 +42,7 @@ HEADERS = adler32.h  
>   \
>   hwcontext.h   \
>   hwcontext_cuda.h  \
>   hwcontext_d3d11va.h   \
>+  hwcontext_d3d12va.h   \
>   hwcontext_drm.h   \
>   hwcontext_dxva2.h \
>   hwcontext_qsv.h   \
>@@ -190,6 +191,7 @@ OBJS = adler32.o   
> \
>
> OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o
> OBJS-$(CONFIG_D3D11VA)  += hwcontext_d3d11va.o
>+OBJS-$(CONFIG_D3D12VA)  += hwcontext_d3d12va.o
> OBJS-$(CONFIG_DXVA2)+= hwcontext_dxva2.o
> OBJS-$(CONFIG_LIBDRM)   += hwcontext_drm.o
> OBJS-$(CONFIG_MACOS_KPERF)  += macos_kperf.o
>@@ -213,6 +215,7 @@ SKIPHEADERS-$(HAVE_CUDA_H) +=
>hwcontext_cuda.h
> SKIPHEADERS-$(CONFIG_CUDA) += hwcontext_cuda_internal.h \
>   cuda_check.h
> SKIPHEADERS-$(CONFIG_D3D11VA)   

Re: [FFmpeg-devel] [PATCH v7 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

>Aug 30, 2023, 11:05 by tong1.wu-at-intel@ffmpeg.org:
>
>>
>>
>> >-Original Message-
>> >From: ffmpeg-devel  On Behalf Of
>Lynne
>> >Sent: Wednesday, August 30, 2023 4:39 PM
>> >To: FFmpeg development discussions and patches > >de...@ffmpeg.org>
>> >Subject: Re: [FFmpeg-devel] [PATCH v7 1/9] libavutil: add
>hwcontext_d3d12va
>> >and AV_PIX_FMT_D3D12
>>
>>>
>>>
>> >Aug 30, 2023, 04:27 by tong1.wu-at-intel@ffmpeg.org:
>>
 >Aug 29, 2023, 06:15 by tong1.wu-at-intel@ffmpeg.org:

>> +
>> +/**
>> + * Specifed by sync=1 when init d3d12va
>> + *
>> + * Execute commands as sync mode
>> + */
>> +int sync;
>>
 >This is not needed, particularly in the public API.

>> +    if (download) {
>> +    ID3D12GraphicsCommandList_ResourceBarrier(s->command_list,
>1,
>>
 >);

>> +
>> +    ID3D12GraphicsCommandList_CopyTextureRegion(s-
>>command_list,
>> +    _y_location, 0, 0, 0, _y_location, NULL);
>> +
>> +    ID3D12GraphicsCommandList_CopyTextureRegion(s-
>>command_list,
>> +    _uv_location, 0, 0, 0, _uv_location, NULL);
>> +
>> +    barrier.Transition.StateBefore = barrier.Transition.StateAfter;
>> +    barrier.Transition.StateAfter =
>D3D12_RESOURCE_STATE_COMMON;
>> +    ID3D12GraphicsCommandList_ResourceBarrier(s->command_list,
>1,
>>
 >);

>> +
>> +    DX_CHECK(ID3D12GraphicsCommandList_Close(s-
>>command_list));
>> +
>> +    if (!hwctx->sync)
>> +    DX_CHECK(ID3D12CommandQueue_Wait(s->command_queue,
>>
 >sync_ctx->fence, sync_ctx->fence_value));

>
>
 >This is wrong. When downloading to RAM, the frames must
 >always be available and fully downloaded by the time the function
 >returns. Therefore, a wait must always occur.
 >Since this is the only place where sync is used, better remove the
 >option altogether.

 There's another place where it's used. It's designed for API users to
>blocking

>> >wait every decoded frame.
>>
 In end_frame() call in d3d12va_decode.c.

  if (ctx->device_ctx->sync) {
  ret = d3d12va_wait_idle(ctx->sync_ctx);
  if (ret < 0)
  return ret;
  }

 That's why when downloading, you don't have to wait again if hwctx->sync
>is

>> >specified. The frame has already been blocking synced in decoding process.
>>

 But yes I agree it's not that needed. I'll remove the whole design in next

>> >version.
>>


>> >That's not correct either, waiting should happen at the last possible
>moment,
>> >rather than immediately after decoding, and waiting should happen
>> >asynchronously
>> >on the GPU, rather than the CPU.
>>
>>>
>>>
>> >Would you mind holding off on posting the redesign until v6.1 is tagged?
>> >Thanks.
>>
>> I've already sent the v8 with the removal of the design.
>>
>
>The code is still present in v8 though, the d3d12va_wait_idle() call
>in ff_d3d12va_common_end_frame. Is there no way to wait for
>the operation on the GPU by recording it in the command buffer?

Actually d3d12va_wait_idle is just a query of timeline fence. At most of time, 
the block wait is not triggered. It only gets the fence value to know if the 
GPU command is completed. It's a known d3d12 design for high performance and 
for users to sync with GPU completion. 

The function name might be a bit confusing. I may change the name from "wait" 
to "sync" in the next version.

Thanks,
Tong

>
>
>>  But yes it's ok for me to wait for 6.1 tagged and rebase again after that. 
>> BTW
>is there any schedule for 6.1? Haven't seen a lot of discussions on that 
>recently.
>Thanks.
>>
>
>Thanks.
>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v7 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of Lynne
>Sent: Wednesday, August 30, 2023 4:39 PM
>To: FFmpeg development discussions and patches de...@ffmpeg.org>
>Subject: Re: [FFmpeg-devel] [PATCH v7 1/9] libavutil: add hwcontext_d3d12va
>and AV_PIX_FMT_D3D12
>
>Aug 30, 2023, 04:27 by tong1.wu-at-intel@ffmpeg.org:
>
>> >Aug 29, 2023, 06:15 by tong1.wu-at-intel@ffmpeg.org:
>>
 +
 +/**
 + * Specifed by sync=1 when init d3d12va
 + *
 + * Execute commands as sync mode
 + */
 +int sync;

>> >This is not needed, particularly in the public API.
>>
 +    if (download) {
 +    ID3D12GraphicsCommandList_ResourceBarrier(s->command_list, 1,

>> >);
>>
 +
 +    ID3D12GraphicsCommandList_CopyTextureRegion(s->command_list,
 +    _y_location, 0, 0, 0, _y_location, NULL);
 +
 +    ID3D12GraphicsCommandList_CopyTextureRegion(s->command_list,
 +    _uv_location, 0, 0, 0, _uv_location, NULL);
 +
 +    barrier.Transition.StateBefore = barrier.Transition.StateAfter;
 +    barrier.Transition.StateAfter = D3D12_RESOURCE_STATE_COMMON;
 +    ID3D12GraphicsCommandList_ResourceBarrier(s->command_list, 1,

>> >);
>>
 +
 +    DX_CHECK(ID3D12GraphicsCommandList_Close(s->command_list));
 +
 +    if (!hwctx->sync)
 +    DX_CHECK(ID3D12CommandQueue_Wait(s->command_queue,

>> >sync_ctx->fence, sync_ctx->fence_value));
>>
>>>
>>>
>> >This is wrong. When downloading to RAM, the frames must
>> >always be available and fully downloaded by the time the function
>> >returns. Therefore, a wait must always occur.
>> >Since this is the only place where sync is used, better remove the
>> >option altogether.
>>
>> There's another place where it's used. It's designed for API users to 
>> blocking
>wait every decoded frame.
>> In end_frame() call in d3d12va_decode.c.
>>
>>  if (ctx->device_ctx->sync) {
>>  ret = d3d12va_wait_idle(ctx->sync_ctx);
>>  if (ret < 0)
>>  return ret;
>>  }
>>
>> That's why when downloading, you don't have to wait again if hwctx->sync is
>specified. The frame has already been blocking synced in decoding process.
>>
>> But yes I agree it's not that needed. I'll remove the whole design in next
>version.
>>
>
>That's not correct either, waiting should happen at the last possible moment,
>rather than immediately after decoding, and waiting should happen
>asynchronously
>on the GPU, rather than the CPU.
>
>Would you mind holding off on posting the redesign until v6.1 is tagged?
>Thanks.

I've already sent the v8 with the removal of the design. But yes it's ok for me 
to wait for 6.1 tagged and rebase again after that. BTW is there any schedule 
for 6.1? Haven't seen a lot of discussions on that recently. Thanks.

>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v7 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

>Aug 29, 2023, 06:15 by tong1.wu-at-intel@ffmpeg.org:
>
>> +
>> +/**
>> + * Specifed by sync=1 when init d3d12va
>> + *
>> + * Execute commands as sync mode
>> + */
>> +int sync;
>>
>
>This is not needed, particularly in the public API.
>
>> +    if (download) {
>> +    ID3D12GraphicsCommandList_ResourceBarrier(s->command_list, 1,
>);
>> +
>> +    ID3D12GraphicsCommandList_CopyTextureRegion(s->command_list,
>> +    _y_location, 0, 0, 0, _y_location, NULL);
>> +
>> +    ID3D12GraphicsCommandList_CopyTextureRegion(s->command_list,
>> +    _uv_location, 0, 0, 0, _uv_location, NULL);
>> +
>> +    barrier.Transition.StateBefore = barrier.Transition.StateAfter;
>> +    barrier.Transition.StateAfter = D3D12_RESOURCE_STATE_COMMON;
>> +    ID3D12GraphicsCommandList_ResourceBarrier(s->command_list, 1,
>);
>> +
>> +    DX_CHECK(ID3D12GraphicsCommandList_Close(s->command_list));
>> +
>> +    if (!hwctx->sync)
>> +    DX_CHECK(ID3D12CommandQueue_Wait(s->command_queue,
>sync_ctx->fence, sync_ctx->fence_value));
>
>This is wrong. When downloading to RAM, the frames must
>always be available and fully downloaded by the time the function
>returns. Therefore, a wait must always occur.
>Since this is the only place where sync is used, better remove the
>option altogether.

There's another place where it's used. It's designed for API users to blocking 
wait every decoded frame.
In end_frame() call in d3d12va_decode.c. 

if (ctx->device_ctx->sync) {
ret = d3d12va_wait_idle(ctx->sync_ctx);
if (ret < 0)
return ret;
}

That's why when downloading, you don't have to wait again if hwctx->sync is 
specified. The frame has already been blocking synced in decoding process.

But yes I agree it's not that needed. I'll remove the whole design in next 
version.

Thanks,
Tong


>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v7 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

V7 removed av_d3d12_va_wait_idle from public API

Thanks,
Tong

>-Original Message-
>From: Wu, Tong1 
>Sent: Tuesday, August 29, 2023 12:10 PM
>To: ffmpeg-devel@ffmpeg.org
>Cc: Wu Jianhua ; Wu, Tong1 
>Subject: [PATCH v7 1/9] libavutil: add hwcontext_d3d12va and
>AV_PIX_FMT_D3D12
>
>From: Wu Jianhua 
>
>Signed-off-by: Wu Jianhua 
>Signed-off-by: Tong Wu 
>---
> configure  |   5 +
> doc/APIchanges |   7 +
> libavutil/Makefile |   3 +
> libavutil/hwcontext.c  |   4 +
> libavutil/hwcontext.h  |   1 +
> libavutil/hwcontext_d3d12va.c  | 693 +
> libavutil/hwcontext_d3d12va.h  | 148 ++
> libavutil/hwcontext_d3d12va_internal.h |  59 +++
> libavutil/hwcontext_internal.h |   1 +
> libavutil/pixdesc.c|   4 +
> libavutil/pixfmt.h |   7 +
> libavutil/tests/hwdevice.c |   2 +
> libavutil/version.h|   2 +-
> 13 files changed, 935 insertions(+), 1 deletion(-)
> create mode 100644 libavutil/hwcontext_d3d12va.c
> create mode 100644 libavutil/hwcontext_d3d12va.h
> create mode 100644 libavutil/hwcontext_d3d12va_internal.h
>
>diff --git a/configure b/configure
>index bd7f7697c8..c604302017 100755
>--- a/configure
>+++ b/configure
>@@ -338,6 +338,7 @@ External library support:
>   --disable-cuda-llvm  disable CUDA compilation using clang [autodetect]
>   --disable-cuvid  disable Nvidia CUVID support [autodetect]
>   --disable-d3d11vadisable Microsoft Direct3D 11 video acceleration 
> code
>[autodetect]
>+  --disable-d3d12vadisable Microsoft Direct3D 12 video acceleration 
>code
>[autodetect]
>   --disable-dxva2  disable Microsoft DirectX 9 video acceleration code
>[autodetect]
>   --disable-ffnvcodec  disable dynamically linked Nvidia code [autodetect]
>   --enable-libdrm  enable DRM code (Linux) [no]
>@@ -1926,6 +1927,7 @@ HWACCEL_AUTODETECT_LIBRARY_LIST="
> cuda_llvm
> cuvid
> d3d11va
>+d3d12va
> dxva2
> ffnvcodec
> nvdec
>@@ -3054,6 +3056,7 @@ crystalhd_deps="libcrystalhd_libcrystalhd_if_h"
> cuda_deps="ffnvcodec"
> cuvid_deps="ffnvcodec"
> d3d11va_deps="dxva_h ID3D11VideoDecoder ID3D11VideoContext"
>+d3d12va_deps="dxva_h ID3D12Device ID3D12VideoDecoder"
> dxva2_deps="dxva2api_h DXVA2_ConfigPictureDecode ole32 user32"
> ffnvcodec_deps_any="libdl LoadLibrary"
> mediacodec_deps="android"
>@@ -6547,6 +6550,8 @@ check_type "windows.h dxgi1_2.h" "IDXGIOutput1"
> check_type "windows.h dxgi1_5.h" "IDXGIOutput5"
> check_type "windows.h d3d11.h" "ID3D11VideoDecoder"
> check_type "windows.h d3d11.h" "ID3D11VideoContext"
>+check_type "windows.h d3d12.h" "ID3D12Device"
>+check_type "windows.h d3d12video.h" "ID3D12VideoDecoder"
> check_type "windows.h" "DPI_AWARENESS_CONTEXT" -
>D_WIN32_WINNT=0x0A00
> check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -
>D_WIN32_WINNT=0x0602
> check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat
>diff --git a/doc/APIchanges b/doc/APIchanges
>index ad1efe708d..5004a11267 100644
>--- a/doc/APIchanges
>+++ b/doc/APIchanges
>@@ -2,6 +2,13 @@ The last version increases of all libraries were on 2023-02-
>09
>
> API changes, most recent first:
>
>+2023-07-xx - xx - lavu 58.18.100 - pixfmt.h hwcontext.h
>hwcontext_d3d12va.h
>+  Add AV_HWDEVICE_TYPE_D3D12VA and AV_PIX_FMT_D3D12.
>+  Add AVD3D12VADeviceContext, AVD3D12VASyncContext, AVD3D12VAFrame
>and
>+  AVD3D12VAFramesContext.
>+  Add av_d3d12va_map_sw_to_hw_format, av_d3d12va_sync_context_alloc,
>+  av_d3d12va_sync_context_free.
>+
> 2023-08-18 - xx - lavu 58.17.100 - channel_layout.h
>   All AV_CHANNEL_LAYOUT_* macros are now compatible with C++ 17 and
>older.
>
>diff --git a/libavutil/Makefile b/libavutil/Makefile
>index 7828c94dc5..db318534eb 100644
>--- a/libavutil/Makefile
>+++ b/libavutil/Makefile
>@@ -41,6 +41,7 @@ HEADERS = adler32.h  
>   \
>   hwcontext.h   \
>   hwcontext_cuda.h  \
>   hwcontext_d3d11va.h   \
>+  hwcontext_d3d12va.h   \
>   hwcontext_drm.h  

Re: [FFmpeg-devel] [PATCH v6 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

>-Original Message-
>From: ffmpeg-devel  On Behalf Of Lynne
>Sent: Saturday, August 26, 2023 3:17 AM
>To: FFmpeg development discussions and patches de...@ffmpeg.org>
>Subject: Re: [FFmpeg-devel] [PATCH v6 1/9] libavutil: add hwcontext_d3d12va
>and AV_PIX_FMT_D3D12
>
>Aug 25, 2023, 10:49 by tong1.wu-at-intel@ffmpeg.org:
>
>> V6 major changes
>> 1. the way d3dlibs created, adding a load function. Added dlclose in free
>function.
>> 2. Simplified the public sync API by only keeping av_d3d12va_wait_idle.
>> 3. Wrapping data->[0], data[1], data[2] into one single structure.
>>
>
>This hasn't fixed my issues with the previous version.
>av_d3d12va_wait_idle is in particular looks like a convenience function
>rather than something that must always be called for valid API usage.

Ok it makes sense. I'll remove it from API in v7.
>Also, we have dlopen/dlclose wrappers that you should be using
>instead.

dlopen has already been a wrapper for LoadLibrary on Windows. Please see 
compat/w32dlfcn.h

#define dlopen(name, flags) win32_dlopen(name)

Thanks,
Tong

>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v6 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

V6 major changes
1. the way d3dlibs created, adding a load function. Added dlclose in free 
function.
2. Simplified the public sync API by only keeping av_d3d12va_wait_idle.
3. Wrapping data->[0], data[1], data[2] into one single structure.


>Subject: [PATCH v6 1/9] libavutil: add hwcontext_d3d12va and
>AV_PIX_FMT_D3D12
>
>From: Wu Jianhua 
>
>Signed-off-by: Wu Jianhua 
>Signed-off-by: Tong Wu 
>---
> configure  |   5 +
> doc/APIchanges |   7 +
> libavutil/Makefile |   3 +
> libavutil/hwcontext.c  |   4 +
> libavutil/hwcontext.h  |   1 +
> libavutil/hwcontext_d3d12va.c  | 693 +
> libavutil/hwcontext_d3d12va.h  | 155 ++
> libavutil/hwcontext_d3d12va_internal.h |  59 +++
> libavutil/hwcontext_internal.h |   1 +
> libavutil/pixdesc.c|   4 +
> libavutil/pixfmt.h |   7 +
> libavutil/tests/hwdevice.c |   2 +
> libavutil/version.h|   2 +-
> 13 files changed, 942 insertions(+), 1 deletion(-)
> create mode 100644 libavutil/hwcontext_d3d12va.c
> create mode 100644 libavutil/hwcontext_d3d12va.h
> create mode 100644 libavutil/hwcontext_d3d12va_internal.h
>
>diff --git a/configure b/configure
>index 04bb9fe9dd..b74a668f87 100755
>--- a/configure
>+++ b/configure
>@@ -338,6 +338,7 @@ External library support:
>   --disable-cuda-llvm  disable CUDA compilation using clang [autodetect]
>   --disable-cuvid  disable Nvidia CUVID support [autodetect]
>   --disable-d3d11vadisable Microsoft Direct3D 11 video acceleration 
> code
>[autodetect]
>+  --disable-d3d12vadisable Microsoft Direct3D 12 video acceleration 
>code
>[autodetect]
>   --disable-dxva2  disable Microsoft DirectX 9 video acceleration code
>[autodetect]
>   --disable-ffnvcodec  disable dynamically linked Nvidia code [autodetect]
>   --enable-libdrm  enable DRM code (Linux) [no]
>@@ -1926,6 +1927,7 @@ HWACCEL_AUTODETECT_LIBRARY_LIST="
> cuda_llvm
> cuvid
> d3d11va
>+d3d12va
> dxva2
> ffnvcodec
> nvdec
>@@ -3053,6 +3055,7 @@ crystalhd_deps="libcrystalhd_libcrystalhd_if_h"
> cuda_deps="ffnvcodec"
> cuvid_deps="ffnvcodec"
> d3d11va_deps="dxva_h ID3D11VideoDecoder ID3D11VideoContext"
>+d3d12va_deps="dxva_h ID3D12Device ID3D12VideoDecoder"
> dxva2_deps="dxva2api_h DXVA2_ConfigPictureDecode ole32 user32"
> ffnvcodec_deps_any="libdl LoadLibrary"
> mediacodec_deps="android"
>@@ -6546,6 +6549,8 @@ check_type "windows.h dxgi1_2.h" "IDXGIOutput1"
> check_type "windows.h dxgi1_5.h" "IDXGIOutput5"
> check_type "windows.h d3d11.h" "ID3D11VideoDecoder"
> check_type "windows.h d3d11.h" "ID3D11VideoContext"
>+check_type "windows.h d3d12.h" "ID3D12Device"
>+check_type "windows.h d3d12video.h" "ID3D12VideoDecoder"
> check_type "windows.h" "DPI_AWARENESS_CONTEXT" -
>D_WIN32_WINNT=0x0A00
> check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -
>D_WIN32_WINNT=0x0602
> check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat
>diff --git a/doc/APIchanges b/doc/APIchanges
>index ad1efe708d..37ce29323d 100644
>--- a/doc/APIchanges
>+++ b/doc/APIchanges
>@@ -2,6 +2,13 @@ The last version increases of all libraries were on 2023-02-
>09
>
> API changes, most recent first:
>
>+2023-07-xx - xx - lavu 58.18.100 - pixfmt.h hwcontext.h
>hwcontext_d3d12va.h
>+  Add AV_HWDEVICE_TYPE_D3D12VA and AV_PIX_FMT_D3D12.
>+  Add AVD3D12VADeviceContext, AVD3D12VASyncContext, AVD3D12VAFrame
>and
>+  AVD3D12VAFramesContext.
>+  Add av_d3d12va_map_sw_to_hw_format, av_d3d12va_sync_context_alloc,
>+  av_d3d12va_sync_context_free, av_d3d12va_wait_idle.
>+
> 2023-08-18 - xx - lavu 58.17.100 - channel_layout.h
>   All AV_CHANNEL_LAYOUT_* macros are now compatible with C++ 17 and
>older.
>
>diff --git a/libavutil/Makefile b/libavutil/Makefile
>index 7828c94dc5..db318534eb 100644
>--- a/libavutil/Makefile
>+++ b/libavutil/Makefile
>@@ -41,6 +41,7 @@ HEADERS = adler32.h  
>   \
>   hwcontext.h   \
>   hwcontext_cuda.h  \
>   hwcontext_d3d11va.h   \
>+  hwcontext_d3d12va.h   \
>   hwcontext_drm.h   \
>   hwcontext_dxva2.h \
>   hwcontext_qsv.h   \
>@@ -188,6 +189,7 @@ OBJS = adler32.o   
> \
>
> OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o
> OBJS-$(CONFIG_D3D11VA)  += hwcontext_d3d11va.o
>+OBJS-$(CONFIG_D3D12VA)  += hwcontext_d3d12va.o
> OBJS-$(CONFIG_DXVA2)+= hwcontext_dxva2.o

Re: [FFmpeg-devel] [PATCH v5 1/9] libavutil: add hwcontext_d3d12va and AV_PIX_FMT_D3D12

[Wu, Tong1]

>Aug 4, 2023, 15:50 by tong1.wu-at-intel@ffmpeg.org:
>
>> From: Wu Jianhua 
>>
>> Signed-off-by: Wu Jianhua 
>> Signed-off-by: Tong Wu 
>> ---
>>  configure  |   5 +
>>  doc/APIchanges |   7 +
>>  libavutil/Makefile |   3 +
>>  libavutil/hwcontext.c  |   4 +
>>  libavutil/hwcontext.h  |   1 +
>>  libavutil/hwcontext_d3d12va.c  | 703 +
>>  libavutil/hwcontext_d3d12va.h  | 169 ++
>>  libavutil/hwcontext_d3d12va_internal.h |  59 +++
>>  libavutil/hwcontext_internal.h |   1 +
>>  libavutil/pixdesc.c|   4 +
>>  libavutil/pixfmt.h |   9 +
>>  libavutil/tests/hwdevice.c |   2 +
>>  libavutil/version.h|   2 +-
>>  13 files changed, 968 insertions(+), 1 deletion(-)
>>  create mode 100644 libavutil/hwcontext_d3d12va.c
>>  create mode 100644 libavutil/hwcontext_d3d12va.h
>>  create mode 100644 libavutil/hwcontext_d3d12va_internal.h
>>
>> diff --git a/configure b/configure
>> index 99388e7664..9919551ad0 100755
>> --- a/configure
>> +++ b/configure
>> @@ -338,6 +338,7 @@ External library support:
>>  --disable-cuda-llvm  disable CUDA compilation using clang [autodetect]
>>  --disable-cuvid  disable Nvidia CUVID support [autodetect]
>>  --disable-d3d11vadisable Microsoft Direct3D 11 video acceleration 
>> code
>[autodetect]
>> +  --disable-d3d12vadisable Microsoft Direct3D 12 video acceleration
>code [autodetect]
>>  --disable-dxva2  disable Microsoft DirectX 9 video acceleration code
>[autodetect]
>>  --disable-ffnvcodec  disable dynamically linked Nvidia code [autodetect]
>>  --enable-libdrm  enable DRM code (Linux) [no]
>> @@ -1926,6 +1927,7 @@ HWACCEL_AUTODETECT_LIBRARY_LIST="
>>  cuda_llvm
>>  cuvid
>>  d3d11va
>> +d3d12va
>>  dxva2
>>  ffnvcodec
>>  nvdec
>> @@ -3053,6 +3055,7 @@ crystalhd_deps="libcrystalhd_libcrystalhd_if_h"
>>  cuda_deps="ffnvcodec"
>>  cuvid_deps="ffnvcodec"
>>  d3d11va_deps="dxva_h ID3D11VideoDecoder ID3D11VideoContext"
>> +d3d12va_deps="dxva_h ID3D12Device ID3D12VideoDecoder"
>>  dxva2_deps="dxva2api_h DXVA2_ConfigPictureDecode ole32 user32"
>>  ffnvcodec_deps_any="libdl LoadLibrary"
>>  mediacodec_deps="android"
>> @@ -6543,6 +6546,8 @@ check_type "windows.h dxgi1_2.h"
>"IDXGIOutput1"
>>  check_type "windows.h dxgi1_5.h" "IDXGIOutput5"
>>  check_type "windows.h d3d11.h" "ID3D11VideoDecoder"
>>  check_type "windows.h d3d11.h" "ID3D11VideoContext"
>> +check_type "windows.h d3d12.h" "ID3D12Device"
>> +check_type "windows.h d3d12video.h" "ID3D12VideoDecoder"
>>  check_type "windows.h" "DPI_AWARENESS_CONTEXT" -
>D_WIN32_WINNT=0x0A00
>>  check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -
>D_WIN32_WINNT=0x0602
>>  check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat
>> diff --git a/doc/APIchanges b/doc/APIchanges
>> index 5afe8bcb75..c14ab0b394 100644
>> --- a/doc/APIchanges
>> +++ b/doc/APIchanges
>> @@ -2,6 +2,13 @@ The last version increases of all libraries were on 2023-
>02-09
>>
>>  API changes, most recent first:
>>
>> +2023-07-xx - xx - lavu 58.15.100 - pixfmt.h hwcontext.h
>hwcontext_d3d12va.h
>> +  Add AV_HWDEVICE_TYPE_D3D12VA and AV_PIX_FMT_D3D12.
>> +  Add AVD3D12VADeviceContext, AVD3D12VASyncContext,
>AVD3D12FrameDescriptor,
>> +  and AVD3D12VAFramesContext.
>> +  Add av_d3d12va_map_sw_to_hw_format,
>av_d3d12va_create_sync_context,
>> +  av_d3d12va_release_sync_context, av_d3d12va_wait_idle, and
>av_d3d12va_wait_queue_idle.
>> +
>>  2023-07-xx - xx - lavc 60 - avcodec.h
>>  Deprecate AV_CODEC_FLAG_DROPCHANGED without replacement.
>>
>> diff --git a/libavutil/Makefile b/libavutil/Makefile
>> index bd9c6f9e32..40d49d76dd 100644
>> --- a/libavutil/Makefile
>> +++ b/libavutil/Makefile
>> @@ -41,6 +41,7 @@ HEADERS = adler32.h
>>  \
>>  hwcontext.h   \
>>  hwcontext_cuda.h  \
>>  hwcontext_d3d11va.h   \
>> +  hwcontext_d3d12va.h   \
>>  hwcontext_drm.h   \
>>  hwcontext_dxva2.h \
>>  hwcontext_qsv.h   \
>> @@ -186,6 +187,7 @@ OBJS = adler32.o 
>>\
>>
>>  OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o
>>  OBJS-$(CONFIG_D3D11VA)  += hwcontext_d3d11va.o
>> +OBJS-$(CONFIG_D3D12VA)  += hwcontext_d3d12va.o
>>  OBJS-$(CONFIG_DXVA2)+= hwcontext_dxva2.o
>>  OBJS-$(CONFIG_LIBDRM)   += hwcontext_drm.o
>>  OBJS-$(CONFIG_MACOS_KPERF)  += macos_kperf.o
>> @@ -209,6 +211,7 @@ 

Re: [FFmpeg-devel] [PATCH v3 2/9] avcodec: add D3D12VA hardware accelerated H264 decoding

[Wu, Tong1]

>On Vr, 2023-06-02 at 16:06 +0800, Tong Wu wrote:
>> From: Wu Jianhua 
>>
>> The implementation is based on:
>> https://learn.microsoft.com/en-us/windows/win32/medfound/direct3d-12-
>video-overview
>>
>> With the Direct3D 12 video decoding support, we can render or process
>> the decoded images by the pixel shaders or compute shaders directly
>> without the extra copy overhead, which is beneficial especially if you
>> are trying to render or post-process a 4K or 8K video.
>>
>> The command below is how to enable d3d12va:
>> ffmpeg -hwaccel d3d12va -i input.mp4 output.mp4
>>
>> Signed-off-by: Wu Jianhua 
>> Signed-off-by: Tong Wu 
>> ---
>>  configure   |   2 +
>>  libavcodec/Makefile |   3 +
>>  libavcodec/d3d11va.h    |   3 -
>>  libavcodec/d3d12va.c    | 552 
>>  libavcodec/d3d12va.h    | 184 
>>  libavcodec/d3d12va_h264.c   | 210 ++
>>  libavcodec/dxva2.c  |  24 ++
>>  libavcodec/dxva2.h  |   3 -
>>  libavcodec/dxva2_h264.c |  12 +-
>>  libavcodec/dxva2_internal.h |  67 +++--
>>  libavcodec/h264_slice.c |   4 +
>>  libavcodec/h264dec.c    |   3 +
>>  libavcodec/hwaccels.h   |   1 +
>>  libavcodec/hwconfig.h   |   2 +
>>  14 files changed, 1028 insertions(+), 42 deletions(-)
>>  create mode 100644 libavcodec/d3d12va.c
>>  create mode 100644 libavcodec/d3d12va.h
>>  create mode 100644 libavcodec/d3d12va_h264.c
>>
>> diff --git a/configure b/configure
>> index b86064e36f..f5dad4653f 100755
>> --- a/configure
>> +++ b/configure
>> @@ -3033,6 +3033,8 @@ h264_d3d11va_hwaccel_deps="d3d11va"
>>  h264_d3d11va_hwaccel_select="h264_decoder"
>>  h264_d3d11va2_hwaccel_deps="d3d11va"
>>  h264_d3d11va2_hwaccel_select="h264_decoder"
>> +h264_d3d12va_hwaccel_deps="d3d12va"
>> +h264_d3d12va_hwaccel_select="h264_decoder"
>>  h264_dxva2_hwaccel_deps="dxva2"
>>  h264_dxva2_hwaccel_select="h264_decoder"
>>  h264_nvdec_hwaccel_deps="nvdec"
>> diff --git a/libavcodec/Makefile b/libavcodec/Makefile
>> index 9aacc1d477..ae143d8821 100644
>> --- a/libavcodec/Makefile
>> +++ b/libavcodec/Makefile
>> @@ -977,6 +977,7 @@ OBJS-$(CONFIG_ADPCM_ZORK_DECODER) +=
>adpcm.o
>> adpcm_data.o
>>
>>  # hardware accelerators
>>  OBJS-$(CONFIG_D3D11VA)    += dxva2.o
>> +OBJS-$(CONFIG_D3D12VA)    += dxva2.o d3d12va.o
>>  OBJS-$(CONFIG_DXVA2)  += dxva2.o
>>  OBJS-$(CONFIG_NVDEC)  += nvdec.o
>>  OBJS-$(CONFIG_VAAPI)  += vaapi_decode.o
>> @@ -994,6 +995,7 @@ OBJS-$(CONFIG_H263_VAAPI_HWACCEL) +=
>vaapi_mpeg4.o
>>  OBJS-$(CONFIG_H263_VIDEOTOOLBOX_HWACCEL)  += videotoolbox.o
>>  OBJS-$(CONFIG_H264_D3D11VA_HWACCEL)   += dxva2_h264.o
>>  OBJS-$(CONFIG_H264_DXVA2_HWACCEL) += dxva2_h264.o
>> +OBJS-$(CONFIG_H264_D3D12VA_HWACCEL)   += dxva2_h264.o
>d3d12va_h264.o
>>  OBJS-$(CONFIG_H264_NVDEC_HWACCEL) += nvdec_h264.o
>>  OBJS-$(CONFIG_H264_QSV_HWACCEL)   += qsvdec.o
>>  OBJS-$(CONFIG_H264_VAAPI_HWACCEL) += vaapi_h264.o
>> @@ -1277,6 +1279,7 @@ SKIPHEADERS    +=
>> %_tablegen.h  \
>>
>>  SKIPHEADERS-$(CONFIG_AMF)  += amfenc.h
>>  SKIPHEADERS-$(CONFIG_D3D11VA)  += d3d11va.h dxva2_internal.h
>> +SKIPHEADERS-$(CONFIG_D3D12VA)  += d3d12va.h
>>  SKIPHEADERS-$(CONFIG_DXVA2)    += dxva2.h dxva2_internal.h
>>  SKIPHEADERS-$(CONFIG_JNI)  += ffjni.h
>>  SKIPHEADERS-$(CONFIG_LCMS2)    += fflcms2.h
>> diff --git a/libavcodec/d3d11va.h b/libavcodec/d3d11va.h
>> index 6816b6c1e6..27f40e5519 100644
>> --- a/libavcodec/d3d11va.h
>> +++ b/libavcodec/d3d11va.h
>> @@ -45,9 +45,6 @@
>>   * @{
>>   */
>>
>> -#define FF_DXVA2_WORKAROUND_SCALING_LIST_ZIGZAG 1 ///< Work
>around for
>> Direct3D11 and old UVD/UVD+ ATI video cards
>> -#define FF_DXVA2_WORKAROUND_INTEL_CLEARVIDEO    2 ///< Work
>around for
>> Direct3D11 and old Intel GPUs with ClearVideo interface
>> -
>>  /**
>>   * This structure is used to provides the necessary configurations and data
>>   * to the Direct3D11 FFmpeg HWAccel implementation.
>> diff --git a/libavcodec/d3d12va.c b/libavcodec/d3d12va.c
>> new file mode 100644
>> index 00..7f1fab7251
>> --- /dev/null
>> +++ b/libavcodec/d3d12va.c
>> @@ -0,0 +1,552 @@
>> +/*
>> + * Direct3D 12 HW acceleration video decoder
>> + *
>> + * copyright (c) 2022-2023 Wu Jianhua 
>> + *
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

Re: [FFmpeg-devel] [PATCH v5 5/5] lavfi/format: add a hwmap auto conversion filter

[Wu, Tong1]

>On 25 Apr 2023, at 9:26, Tong Wu wrote:
>
>> When two formats lists cannot be merged, a scale filter is
>> auto-inserted. However, when it comes to hardware map, we have to
>> manually add a hwmap filter to do the conversion. This patch introduces
>> an auto hwmap filter to do the hwmap conversion automatically.
>>
>
>Thanks for trying to improve this!
>
>I've recently done quite a bit of experimentation with hardware
>filters and at least for the Cuda - Vulkan - Cuda case, hwmap
>was useless, and I was told I need to use hwupload instead, so I wonder
>what cases this would help with?

Thanks for the reply!

Actually we are using hwmap filter to do vaapi<->qsv, d3d11<->qsv conversions.

For the use case I have replied in another mail. I'll just copy here again.
For example,
ffmpeg -init_hw_device d3d11va=d3d11 -init_hw_device qsv=qsv@d3d11 -
hwaccel d3d11va -hwaccel_output_format d3d11 -i input.mp4 -vf
"hwmap=derive_device=qsv:extra_hw_frames=16,format=qsv" -c:v h264_qsv
output.mp4

Now we don't have to explicitly specify hwmap. The auto filter mechanism will
try to insert a hwmap filter automatically. The command line will be much
simpler.

ffmpeg -init_hw_device d3d11va=d3d11 -init_hw_device qsv=qsv@d3d11 -
hwaccel d3d11va -hwaccel_output_format d3d11 -i input.mp4 -c:v h264_qsv
output.mp4



>
>I just fear that, especially given the bad error messages hwmap gives,
>this will just implicitly insert it because it seemingly works but then
>just fail to actually do the job and give an absolutely indescriptive
>error to the user for a filter they did not even insert themselves.

Yes I agree with that. However, we already have an auto scale filter in the 
framework.
It will always be inserted when two filters cannot be linked.

In my opinion, I'm just kinda appending another filter. It should be no harmful 
to the original framework. The only way you can get to this auto hwmap filter 
is that you already fail to insert scale filter and the program will return 
error. That's the current situation. With this patch set, those failed cmdlines 
could still fail but some cmdlines like above could pass. It just give it 
another chance to run and simplify the cmdlines.

Regards,
Tong


>
>> Signed-off-by: Tong Wu 
>> ---
>>  libavfilter/avfiltergraph.c | 3 ++-
>>  libavfilter/formats.c   | 4 
>>  2 files changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
>> index 8af0467bc5..b2b627ad6a 100644
>> --- a/libavfilter/avfiltergraph.c
>> +++ b/libavfilter/avfiltergraph.c
>> @@ -402,7 +402,8 @@ static int insert_auto_filter(AVFilterContext
>**convert, AVFilterGraph *graph,
>>  AVFilterContext *ctx;
>>  AVFilterLink *inlink, *outlink;
>>  char inst_name[30];
>> -const char *opts = FF_FIELD_AT(char *, neg-
>>conversion_filters[conv_step].conversion_opts_offset, *graph);
>> +const char *opts = neg-
>>conversion_filters[conv_step].conversion_opts_offset == 0 ? NULL :
>> +   FF_FIELD_AT(char *, neg-
>>conversion_filters[conv_step].conversion_opts_offset, *graph);
>>  const char *name = neg->conversion_filters[conv_step].conversion_filter;
>>
>>  if (!(filter = avfilter_get_by_name(name))) {
>> diff --git a/libavfilter/formats.c b/libavfilter/formats.c
>> index c8e20e5b20..fee10fa0ee 100644
>> --- a/libavfilter/formats.c
>> +++ b/libavfilter/formats.c
>> @@ -331,6 +331,10 @@ static const AVFilterFormatsFilter filters_video[] = {
>>  .conversion_filter = "scale",
>>  .conversion_opts_offset = offsetof(AVFilterGraph, scale_sws_opts),
>>  },
>> +{
>> +.conversion_filter = "hwmap",
>> +.conversion_opts_offset = 0,
>> +}
>>  };
>>
>>  static const AVFilterFormatsFilter filters_audio[] = {
>> --
>> 2.39.1.windows.1
>>
>> ___
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v5 4/5] lavfi/format: wrap auto filters into structures

[Wu, Tong1]

Kindly ping


>>Possibility for infinite loops?
>
>>In what cases this helps, have graphs examples to test?
>
>Now we only have scale filter that can be auto inserted. This patch set
>basically gives the potential capability to add more auto filters. An auto
>hwmap filter is introduced for now. It can be beneficial to hwmap use cases.
>
>For example,
>ffmpeg -init_hw_device d3d11va=d3d11 -init_hw_device qsv=qsv@d3d11 -
>hwaccel d3d11va -hwaccel_output_format d3d11 -i input.mp4 -vf
>“hwmap=derive_device=qsv:extra_hw_frames=16,format=qsv” -c:v h264_qsv
>output.mp4
>
>Now we don’t have to explicitly specify hwmap. The auto filter mechanism will
>try to insert a hwmap filter automatically. The command line will be much
>simpler.
>
>ffmpeg -init_hw_device d3d11va=d3d11 -init_hw_device qsv=qsv@d3d11 -
>hwaccel d3d11va -hwaccel_output_format d3d11 -i input.mp4 -c:v h264_qsv
>output.mp4
>
>Could you explain a little bit why the loop could be infinite?
>___
>ffmpeg-devel mailing list
>ffmpeg-devel@ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 2/9] avcodec: add D3D12VA hardware accelerated H264 decoding

[Wu, Tong1]

>
>On Wed, May 31, 2023 at 11:07:15AM +0800, Tong Wu wrote:
>> From: Wu Jianhua 
>>
>> The implementation is based on:
>> https://learn.microsoft.com/en-us/windows/win32/medfound/direct3d-12-
>video-overview
>>
>> With the Direct3D 12 video decoding support, we can render or process
>> the decoded images by the pixel shaders or compute shaders directly
>> without the extra copy overhead, which is beneficial especially if you
>> are trying to render or post-process a 4K or 8K video.
>>
>> The command below is how to enable d3d12va:
>> ffmpeg -hwaccel d3d12va -i input.mp4 output.mp4
>>
>> Signed-off-by: Wu Jianhua 
>> Signed-off-by: Tong Wu 
>> ---
>>  configure   |   2 +
>>  libavcodec/Makefile |   3 +
>>  libavcodec/d3d11va.h|   3 -
>>  libavcodec/d3d12va.c| 552 
>>  libavcodec/d3d12va.h| 184 
>>  libavcodec/d3d12va_h264.c   | 210 ++
>>  libavcodec/dxva2.c  |  24 ++
>>  libavcodec/dxva2.h  |   3 -
>>  libavcodec/dxva2_h264.c |  12 +-
>>  libavcodec/dxva2_internal.h |  69 +++--
>>  libavcodec/h264_slice.c |   4 +
>>  libavcodec/h264dec.c|   3 +
>>  libavcodec/hwaccels.h   |   1 +
>>  libavcodec/hwconfig.h   |   2 +
>>  14 files changed, 1030 insertions(+), 42 deletions(-)
>>  create mode 100644 libavcodec/d3d12va.c
>>  create mode 100644 libavcodec/d3d12va.h
>>  create mode 100644 libavcodec/d3d12va_h264.c
>
>seems to break build on mingw64 here

Thanks, will fix in V3.


> make
>CC libavcodec/dxva2.o
>In file included from src/libavcodec/dxva2.c:33:0:
>src/libavcodec/dxva2.c: In function ‘ff_dxva2_get_surface_index’:
>src/libavcodec/dxva2_internal.h:48:43: error: pasting "." and "surface_count"
>does not give a valid preprocessing token
> #define D3D11VA_VAR(ctx, var) ctx->d3d11va.##var
>   ^
>src/libavcodec/dxva2_internal.h:123:114: note: in expansion of macro
>‘D3D11VA_VAR’
> #define DXVA2_CONTEXT_VAR(avctx, ctx, var) (avctx->pix_fmt ==
>AV_PIX_FMT_D3D12 ? 0 : (ff_dxva2_is_d3d11(avctx) ? D3D11VA_VAR(ctx,
>var) :  DXVA2_VAR(ctx, var)))
>   
>^~~
>src/libavcodec/dxva2_internal.h:127:49: note: in expansion of macro
>‘DXVA2_CONTEXT_VAR’
> #define DXVA_CONTEXT_COUNT(avctx, ctx)  DXVA2_CONTEXT_VAR(avctx,
>ctx, surface_count)
> ^
>src/libavcodec/dxva2.c:791:21: note: in expansion of macro
>‘DXVA_CONTEXT_COUNT’
> for (i = 0; i < DXVA_CONTEXT_COUNT(avctx, ctx); i++) {
> ^
>In file included from src/libavcodec/dxva2.c:33:0:
>src/libavcodec/dxva2_internal.h:40:39: error: pasting "." and "surface_count"
>does not give a valid preprocessing token
> #define DXVA2_VAR(ctx, var) ctx->dxva2.##var
>   ^
>src/libavcodec/dxva2_internal.h:123:139: note: in expansion of macro
>‘DXVA2_VAR’
> #define DXVA2_CONTEXT_VAR(avctx, ctx, var) (avctx->pix_fmt ==
>AV_PIX_FMT_D3D12 ? 0 : (ff_dxva2_is_d3d11(avctx) ? D3D11VA_VAR(ctx,
>var) :  DXVA2_VAR(ctx, var)))
>
>^
>src/libavcodec/dxva2_internal.h:127:49: note: in expansion of macro
>‘DXVA2_CONTEXT_VAR’
> #define DXVA_CONTEXT_COUNT(avctx, ctx)  DXVA2_CONTEXT_VAR(avctx,
>ctx, surface_count)
> ^
>src/libavcodec/dxva2.c:791:21: note: in expansion of macro
>‘DXVA_CONTEXT_COUNT’
> for (i = 0; i < DXVA_CONTEXT_COUNT(avctx, ctx); i++) {
> ^
>src/ffbuild/common.mak:81: recipe for target 'libavcodec/dxva2.o' failed
>make: *** [libavcodec/dxva2.o] Error 1
>
>[...]
>
>--
>Michael GnuPG fingerprint:
>9FF2128B147EF6730BADF133611EC787040B0FAB
>
>There will always be a question for which you do not know the correct answer.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 2/9] avcodec: add D3D12VA hardware accelerated H264 decoding

[Wu, Tong1]

>On Wednesday, May 31, 2023, Tong Wu 
>wrote:
>
>> From: Wu Jianhua 
>>
>> The implementation is based on:
>> https://learn.microsoft.com/en-us/windows/win32/medfound/
>> direct3d-12-video-overview
>>
>>
>
>> With the Direct3D 12 video decoding support, we can render or process
>> the decoded images by the pixel shaders or compute shaders directly
>
>do you have some demo regarding pixel shaders or compute shaders?

Currently we don't have a simple demo.
You can refer to the official samples. 
https://github.com/microsoft/DirectX-Graphics-Samples
>
>
>> without the extra copy overhead, which is beneficial especially if you
>
>are trying to render or post-process a 4K or 8K video.
>>
>> The command below is how to enable d3d12va:
>> ffmpeg -hwaccel d3d12va -i input.mp4 output.mp4
>>
>> Signed-off-by: Wu Jianhua 
>> Signed-off-by: Tong Wu 
>> ---
>>  configure   |   2 +
>>  libavcodec/Makefile |   3 +
>>  libavcodec/d3d11va.h|   3 -
>>  libavcodec/d3d12va.c| 552 
>>  libavcodec/d3d12va.h| 184 
>>  libavcodec/d3d12va_h264.c   | 210 ++
>>  libavcodec/dxva2.c  |  24 ++
>>  libavcodec/dxva2.h  |   3 -
>>  libavcodec/dxva2_h264.c |  12 +-
>>  libavcodec/dxva2_internal.h |  69 +++--
>>  libavcodec/h264_slice.c |   4 +
>>  libavcodec/h264dec.c|   3 +
>>  libavcodec/hwaccels.h   |   1 +
>>  libavcodec/hwconfig.h   |   2 +
>>  14 files changed, 1030 insertions(+), 42 deletions(-)
>>  create mode 100644 libavcodec/d3d12va.c
>>  create mode 100644 libavcodec/d3d12va.h
>>  create mode 100644 libavcodec/d3d12va_h264.c
>>
>> diff --git a/configure b/configure
>> index b86064e36f..f5dad4653f 100755
>> --- a/configure
>> +++ b/configure
>> @@ -3033,6 +3033,8 @@ h264_d3d11va_hwaccel_deps="d3d11va"
>>  h264_d3d11va_hwaccel_select="h264_decoder"
>>  h264_d3d11va2_hwaccel_deps="d3d11va"
>>  h264_d3d11va2_hwaccel_select="h264_decoder"
>> +h264_d3d12va_hwaccel_deps="d3d12va"
>> +h264_d3d12va_hwaccel_select="h264_decoder"
>>  h264_dxva2_hwaccel_deps="dxva2"
>>  h264_dxva2_hwaccel_select="h264_decoder"
>>  h264_nvdec_hwaccel_deps="nvdec"
>> diff --git a/libavcodec/Makefile b/libavcodec/Makefile
>> index 9aacc1d477..ae143d8821 100644
>> --- a/libavcodec/Makefile
>> +++ b/libavcodec/Makefile
>> @@ -977,6 +977,7 @@ OBJS-$(CONFIG_ADPCM_ZORK_DECODER) +=
>adpcm.o
>> adpcm_data.o
>>
>>  # hardware accelerators
>>  OBJS-$(CONFIG_D3D11VA)+= dxva2.o
>> +OBJS-$(CONFIG_D3D12VA)+= dxva2.o d3d12va.o
>>  OBJS-$(CONFIG_DXVA2)  += dxva2.o
>>  OBJS-$(CONFIG_NVDEC)  += nvdec.o
>>  OBJS-$(CONFIG_VAAPI)  += vaapi_decode.o
>> @@ -994,6 +995,7 @@ OBJS-$(CONFIG_H263_VAAPI_HWACCEL) +=
>> vaapi_mpeg4.o
>>  OBJS-$(CONFIG_H263_VIDEOTOOLBOX_HWACCEL)  += videotoolbox.o
>>  OBJS-$(CONFIG_H264_D3D11VA_HWACCEL)   += dxva2_h264.o
>>  OBJS-$(CONFIG_H264_DXVA2_HWACCEL) += dxva2_h264.o
>> +OBJS-$(CONFIG_H264_D3D12VA_HWACCEL)   += dxva2_h264.o
>d3d12va_h264.o
>>  OBJS-$(CONFIG_H264_NVDEC_HWACCEL) += nvdec_h264.o
>>  OBJS-$(CONFIG_H264_QSV_HWACCEL)   += qsvdec.o
>>  OBJS-$(CONFIG_H264_VAAPI_HWACCEL) += vaapi_h264.o
>> @@ -1277,6 +1279,7 @@ SKIPHEADERS+=
>> %_tablegen.h  \
>>
>>  SKIPHEADERS-$(CONFIG_AMF)  += amfenc.h
>>  SKIPHEADERS-$(CONFIG_D3D11VA)  += d3d11va.h dxva2_internal.h
>> +SKIPHEADERS-$(CONFIG_D3D12VA)  += d3d12va.h
>>  SKIPHEADERS-$(CONFIG_DXVA2)+= dxva2.h dxva2_internal.h
>>  SKIPHEADERS-$(CONFIG_JNI)  += ffjni.h
>>  SKIPHEADERS-$(CONFIG_LCMS2)+= fflcms2.h
>> diff --git a/libavcodec/d3d11va.h b/libavcodec/d3d11va.h
>> index 6816b6c1e6..27f40e5519 100644
>> --- a/libavcodec/d3d11va.h
>> +++ b/libavcodec/d3d11va.h
>> @@ -45,9 +45,6 @@
>>   * @{
>>   */
>>
>> -#define FF_DXVA2_WORKAROUND_SCALING_LIST_ZIGZAG 1 ///< Work
>around for
>> Direct3D11 and old UVD/UVD+ ATI video cards
>> -#define FF_DXVA2_WORKAROUND_INTEL_CLEARVIDEO2 ///< Work
>around for
>> Direct3D11 and old Intel GPUs with ClearVideo interface
>> -
>>  /**
>>   * This structure is used to provides the necessary configurations and
>> data
>>   * to the Direct3D11 FFmpeg HWAccel implementation.
>> diff --git a/libavcodec/d3d12va.c b/libavcodec/d3d12va.c
>> new file mode 100644
>> index 00..d80575441c
>> --- /dev/null
>> +++ b/libavcodec/d3d12va.c
>> @@ -0,0 +1,552 @@
>> +/*
>> + * Direct3D 12 HW acceleration video decoder
>> + *
>> + * copyright (c) 2022-2023 Wu Jianhua 
>> + *
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later 

Re: [FFmpeg-devel] [PATCH v5 4/5] lavfi/format: wrap auto filters into structures

[Wu, Tong1]

>Possibility for infinite loops?

>In what cases this helps, have graphs examples to test?

Now we only have scale filter that can be auto inserted. This patch set 
basically gives the potential capability to add more auto filters. An auto 
hwmap filter is introduced for now. It can be beneficial to hwmap use cases.

For example,
ffmpeg -init_hw_device d3d11va=d3d11 -init_hw_device qsv=qsv@d3d11 -hwaccel 
d3d11va -hwaccel_output_format d3d11 -i input.mp4 -vf 
“hwmap=derive_device=qsv:extra_hw_frames=16,format=qsv” -c:v h264_qsv output.mp4

Now we don’t have to explicitly specify hwmap. The auto filter mechanism will 
try to insert a hwmap filter automatically. The command line will be much 
simpler.

ffmpeg -init_hw_device d3d11va=d3d11 -init_hw_device qsv=qsv@d3d11 -hwaccel 
d3d11va -hwaccel_output_format d3d11 -i input.mp4 -c:v h264_qsv output.mp4

Could you explain a little bit why the loop could be infinite?
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] avcodec: add D3D12VA hardware accelerated H264, HEVC, VP9, and AV1 decoding

[Wu, Tong1]

> [PATCH v2] avcodec: add D3D12VA hardware accelerated H264, HEVC, VP9,
> and AV1 decoding
> 
> Patches attached.

>-HWACCEL_D3D11VA2(h264),
>+   HWACCEL_D3D11VA(h264),

This shouldn't be changed. It will block d3d11va h264dec.

>+#endif
>+#if CONFIG_H264_D3D12DEC_HWACCEL
>+   HWACCEL_D3D12DEC(h264),


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3 1/3] hwcontext_d3d11va: add mutiple supported DXGI formats

[Wu, Tong1]

> On 2/13/2023 11:45 PM, Tong Wu wrote:
> > Add support for VUYX, YUYV422, Y210, XV30, P012, Y212, XV36.
> >
> > The added formats work with qsv acceleration and will not have impact
> > on d3d11va acceleration(-hwaccel d3d11va) since so far these formats
> > are still not supported by using d3d11va acceleration.
> >
> > Hwupload and hwdownload can work with the added formats.
> >
> > Signed-off-by: Tong Wu 
> > ---
> >   libavutil/hwcontext_d3d11va.c | 7 +++
> >   1 file changed, 7 insertions(+)
> >
> > diff --git a/libavutil/hwcontext_d3d11va.c
> > b/libavutil/hwcontext_d3d11va.c index 363ec6a47d..aa50538d64 100644
> > --- a/libavutil/hwcontext_d3d11va.c
> > +++ b/libavutil/hwcontext_d3d11va.c
> > @@ -89,6 +89,13 @@ static const struct {
> >   { DXGI_FORMAT_B8G8R8A8_UNORM,AV_PIX_FMT_BGRA },
> >   { DXGI_FORMAT_R10G10B10A2_UNORM, AV_PIX_FMT_X2BGR10 },
> >   { DXGI_FORMAT_R16G16B16A16_FLOAT, AV_PIX_FMT_RGBAF16 },
> > +{ DXGI_FORMAT_AYUV, AV_PIX_FMT_VUYX },
> 
> Shouldn't this be VUYA?

Please refer to 
https://learn.microsoft.com/en-us/windows/win32/api/dxgiformat/ne-dxgiformat-dxgi_format

According to the doc, The mapping to the view channel is 
V->R8, 
U->G8, 
Y->B8, 
and A->A8.

It is equivalent to AV_PIX_FMT_VUYX in FFmpeg. Thanks.

> 
> > +{ DXGI_FORMAT_YUY2, AV_PIX_FMT_YUYV422 },
> > +{ DXGI_FORMAT_Y210, AV_PIX_FMT_Y210 },
> > +{ DXGI_FORMAT_Y410, AV_PIX_FMT_XV30 },
> > +{ DXGI_FORMAT_P016, AV_PIX_FMT_P012 },
> > +{ DXGI_FORMAT_Y216, AV_PIX_FMT_Y212 },
> > +{ DXGI_FORMAT_Y416, AV_PIX_FMT_XV36 },
> >   // Special opaque formats. The pix_fmt is merely a place holder, as 
> > the
> >   // opaque format cannot be accessed directly.
> >   { DXGI_FORMAT_420_OPAQUE,   AV_PIX_FMT_YUV420P },
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org
> with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v4 5/5] lavfi/format: add a hwmap auto conversion filter

[Wu, Tong1]

Ping for the patchset. Is there any comment on adding this auto hwmap filter?

Thanks,
Tong

> -Original Message-
> From: Wu, Tong1 
> Sent: Monday, July 4, 2022 4:10 PM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Wu, Tong1 
> Subject: [PATCH v4 5/5] lavfi/format: add a hwmap auto conversion filter
> 
> When two formats lists cannot be merged, a scale filter is auto-inserted.
> However, when it comes to hardware map, we have to manually add a
> hwmap filter to do the conversion. This patch introduces an auto hwmap
> filter to do the hwmap conversion automatically.
> 
> Signed-off-by: Tong Wu 
> ---
>  libavfilter/avfiltergraph.c | 3 ++-
>  libavfilter/formats.c   | 4 
>  2 files changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c index
> 102c1f7693..8e63007cac 100644
> --- a/libavfilter/avfiltergraph.c
> +++ b/libavfilter/avfiltergraph.c
> @@ -402,7 +402,8 @@ static int insert_auto_filter(AVFilterContext **convert,
> AVFilterGraph *graph,
>  AVFilterContext *ctx;
>  AVFilterLink *inlink, *outlink;
>  char inst_name[30];
> -const char *opts = FF_FIELD_AT(char *, neg-
> >conversion_filters[conv_step].conversion_opts_offset, *graph);
> +const char *opts = neg-
> >conversion_filters[conv_step].conversion_opts_offset == 0 ? NULL :
> +   FF_FIELD_AT(char *,
> + neg->conversion_filters[conv_step].conversion_opts_offset, *graph);
>  const char *name = neg->conversion_filters[conv_step].conversion_filter;
> 
>  if (!(filter = avfilter_get_by_name(name))) { diff --git 
> a/libavfilter/formats.c
> b/libavfilter/formats.c index c8e20e5b20..fee10fa0ee 100644
> --- a/libavfilter/formats.c
> +++ b/libavfilter/formats.c
> @@ -331,6 +331,10 @@ static const AVFilterFormatsFilter filters_video[] = {
>  .conversion_filter = "scale",
>  .conversion_opts_offset = offsetof(AVFilterGraph, scale_sws_opts),
>  },
> +{
> +.conversion_filter = "hwmap",
> +.conversion_opts_offset = 0,
> +}
>  };
> 
>  static const AVFilterFormatsFilter filters_audio[] = {
> --
> 2.35.1.windows.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3 3/3] avfilter/avfiltergraph: add an auto hwmap filter

[Wu, Tong1]

> -Original Message-
> From: ffmpeg-devel  On Behalf Of
> Nicolas George
> Sent: Sunday, July 3, 2022 9:28 PM
> To: FFmpeg development discussions and patches  de...@ffmpeg.org>
> Cc: Wu, Tong1 
> Subject: Re: [FFmpeg-devel] [PATCH v3 3/3] avfilter/avfiltergraph: add an
> auto hwmap filter
> 
> Tong Wu (12022-07-03):
> > When two formats lists cannot be merged, a scale filter is
> > auto-inserted. However, when it comes to hardware map, we have to
> > manually add a hwmap filter to do the conversion. This patch
> > introduces an auto hwmap filter to do the hwmap conversion
> > automatically and modifies the negotiation structures, making it
> > extensible for more auto filters.
> >
> > Signed-off-by: Tong Wu 
> > ---
> >  libavfilter/avfiltergraph.c | 183 
> >  libavfilter/formats.c   |  26 -
> >  libavfilter/formats.h   |   9 +-
> >  3 files changed, 150 insertions(+), 68 deletions(-)
> 
> Hi, thanks for the patch. It looks a bit hard to review due to the amount of
> moved code.
> 
> Can you split the patch into two: (1) moving and reindenting the code into an
> auxiliary function without functional changes and (2) adding the feature you
> wanted to add, please?
> 
> To do that without too much hassle, I would: first, start again from a new
> branch, make the "move to a function" commit; second, take the code with
> the extra feature from the other branch and let git turn it into a commit; 
> third,
> clean up that commit.
> 
> Thanks in advance.
> 
> --
>   Nicolas George

Sure, will do it ASAP.

Thanks,
Tong
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/3] avutil/hwcontext: add a function to get the AVHWDeviceType

[Wu, Tong1]

> Quoting Wu, Tong1 (2022-07-01 08:51:28)
> > Plus, do you think adding a AV_PIX_FMT_FLAG_HWACCEL check for the
> > input pixel format and change the function name to
> > av_hwdevice_get_type_by_hwaccel_pix_fmt will help?
> 
> I'd prefer not to, that name is way too long
> 
> > Let users know only hardware pixel formats are acceptable for this
> > function and like I said at this moment not any hardware format is
> > supported by more than one hardware devices.
> 
> Maybe just document in the doxy that it will return the preferred device type
> for a pixel format.

Sure, will do it in v3. Thanks for the comments.

> 
> Overall I don't insist on any changes here, mainly wondering if anyone can
> think of a potential future case where we'd want a single pixel format
> supported by multiple devices
> 
> --
> Anton Khirnov
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org
> with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/3] avutil/hwcontext: add a function to get the AVHWDeviceType

[Wu, Tong1]

> > > On Jun 30, 2022, at 7:56 PM, Andreas Rheinhardt
> >  wrote:
> > >
> > > "zhilizhao(赵志立)":
> > >>
> > >>
> > >>> On Jun 30, 2022, at 4:43 PM, Anton Khirnov 
> wrote:
> > >>>
> > >>> Quoting Tong Wu (2022-06-30 04:45:56)
> >  Add a function to get the corresponding AVHWDeviceType from a
> >  given hardware pixel format.
> > 
> >  Signed-off-by: Tong Wu 
> >  ---
> >  libavutil/hwcontext.c | 12  libavutil/hwcontext.h |
> >  9
> >  +
> >  2 files changed, 21 insertions(+)
> > 
> >  diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c index
> >  ab9ad3703e..3521ed34f4 100644
> >  --- a/libavutil/hwcontext.c
> >  +++ b/libavutil/hwcontext.c
> >  @@ -80,6 +80,18 @@ static const char *const hw_type_names[] = {
> > [AV_HWDEVICE_TYPE_VULKAN] = "vulkan", };
> > 
> >  +enum AVHWDeviceType av_hwdevice_get_type_by_pix_fmt(enum
> >  +AVPixelFormat fmt) {
> >  +int i, j;
> > >>>
> > >>> Nit: you can and should declare loop indices in the loop statement
> > >>> itself
> 
> Sure, will modify them in v3.
> 
> > >>>
> >  +for (i = 0; hw_table[i]; i++) {
> >  +for (j = 0; hw_table[i]->pix_fmts[j] != AV_PIX_FMT_NONE; j++) 
> >  {
> >  +if (hw_table[i]->pix_fmts[j] == fmt)
> >  +return hw_table[i]->type;
> >  +}
> >  +}
> >  +return AV_HWDEVICE_TYPE_NONE; }
> >  +
> >  enum AVHWDeviceType av_hwdevice_find_type_by_name(const char
> > *name)
> >  {
> > int type;
> >  diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h index
> >  c18b7e1e8b..97f94403e2 100644
> >  --- a/libavutil/hwcontext.h
> >  +++ b/libavutil/hwcontext.h
> >  @@ -229,6 +229,15 @@ typedef struct AVHWFramesContext {
> > int width, height;
> >  } AVHWFramesContext;
> > 
> >  +/**
> >  + * Get the device type by a given pixel format.
> >  + *
> >  + * @param fmt Pixel format from enum AVPixelFormat.
> >  + * @return The type from enum AVHWDeviceType, or
> > AV_HWDEVICE_TYPE_NONE if
> >  + * not found.
> >  + */
> >  +enum AVHWDeviceType av_hwdevice_get_type_by_pix_fmt(enum
> >  +AVPixelFormat fmt);
> > >>>
> > >>> I wonder if we should consider the possibility of a format being
> > >>> supported by more than one device type.
> > >>
> > >> For future proof, we can make it clear that there is no guarantee
> > >> that the device type is unique, e.g.,
> > >>
> > >> "Get any device type which support the given pixel format”
> > >>
> > >
> > > Then you'd need to return a list or modify the user accept an iterator.
> >
> > Iterator should be fine.
> >
> > However, the use case is unclear: since we only return an
> > AVHWDeviceType without description like av_pix_fmt_desc_get(), user
> > has little information to skip to the next one, unless user only wants
> > to get all of the device types which support a pixel format.
> 
> Since so far there's no hardware format being supported by more than one
> hardware device, can we just keep current implementation and clarify it in
> comments such as "Get any device type which support the given pixel
> format”?
> 
> Thanks,
> Tong
> 

Plus, do you think adding a AV_PIX_FMT_FLAG_HWACCEL check for the input pixel 
format and change the function name to av_hwdevice_get_type_by_hwaccel_pix_fmt 
will help? Let users know only hardware pixel formats are acceptable for this 
function and like I said at this moment not any hardware format is supported by 
more than one hardware devices.

Tong

> >
> > >
> > > - Andreas
> > > ___
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> >
> > ___
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org
> with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/3] avutil/hwcontext: add a function to get the AVHWDeviceType

[Wu, Tong1]

> > On Jun 30, 2022, at 7:56 PM, Andreas Rheinhardt
>  wrote:
> >
> > "zhilizhao(赵志立)":
> >>
> >>
> >>> On Jun 30, 2022, at 4:43 PM, Anton Khirnov  wrote:
> >>>
> >>> Quoting Tong Wu (2022-06-30 04:45:56)
>  Add a function to get the corresponding AVHWDeviceType from a given
>  hardware pixel format.
> 
>  Signed-off-by: Tong Wu 
>  ---
>  libavutil/hwcontext.c | 12  libavutil/hwcontext.h |  9
>  +
>  2 files changed, 21 insertions(+)
> 
>  diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c index
>  ab9ad3703e..3521ed34f4 100644
>  --- a/libavutil/hwcontext.c
>  +++ b/libavutil/hwcontext.c
>  @@ -80,6 +80,18 @@ static const char *const hw_type_names[] = {
> [AV_HWDEVICE_TYPE_VULKAN] = "vulkan", };
> 
>  +enum AVHWDeviceType av_hwdevice_get_type_by_pix_fmt(enum
>  +AVPixelFormat fmt) {
>  +int i, j;
> >>>
> >>> Nit: you can and should declare loop indices in the loop statement
> >>> itself

Sure, will modify them in v3.

> >>>
>  +for (i = 0; hw_table[i]; i++) {
>  +for (j = 0; hw_table[i]->pix_fmts[j] != AV_PIX_FMT_NONE; j++) {
>  +if (hw_table[i]->pix_fmts[j] == fmt)
>  +return hw_table[i]->type;
>  +}
>  +}
>  +return AV_HWDEVICE_TYPE_NONE;
>  +}
>  +
>  enum AVHWDeviceType av_hwdevice_find_type_by_name(const char
> *name)
>  {
> int type;
>  diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h index
>  c18b7e1e8b..97f94403e2 100644
>  --- a/libavutil/hwcontext.h
>  +++ b/libavutil/hwcontext.h
>  @@ -229,6 +229,15 @@ typedef struct AVHWFramesContext {
> int width, height;
>  } AVHWFramesContext;
> 
>  +/**
>  + * Get the device type by a given pixel format.
>  + *
>  + * @param fmt Pixel format from enum AVPixelFormat.
>  + * @return The type from enum AVHWDeviceType, or
> AV_HWDEVICE_TYPE_NONE if
>  + * not found.
>  + */
>  +enum AVHWDeviceType av_hwdevice_get_type_by_pix_fmt(enum
>  +AVPixelFormat fmt);
> >>>
> >>> I wonder if we should consider the possibility of a format being
> >>> supported by more than one device type.
> >>
> >> For future proof, we can make it clear that there is no guarantee
> >> that the device type is unique, e.g.,
> >>
> >> "Get any device type which support the given pixel format”
> >>
> >
> > Then you'd need to return a list or modify the user accept an iterator.
> 
> Iterator should be fine.
> 
> However, the use case is unclear: since we only return an AVHWDeviceType
> without description like av_pix_fmt_desc_get(), user has little information to
> skip to the next one, unless user only wants to get all of the device types
> which support a pixel format.

Since so far there's no hardware format being supported by more than one 
hardware device, can we just keep current implementation and clarify it in 
comments such as "Get any device type which support the given pixel format”? 

Thanks,
Tong

> 
> >
> > - Andreas
> > ___
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> 
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org
> with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 2/3] avfilter/vf_hwmap: get the AVHWDeviceType from outlink format

[Wu, Tong1]

On Thu, Jun 30, 2022 at 4:47 AM Tong Wu 
mailto:tong1.wu-at-intel@ffmpeg.org>> 
wrote:
When a derive_device_type is not specified, the hwmap filter should be
able to retrieve AVHWDeviceType from outlink->format and create
corresponding hwdevice context.

Signed-off-by: Tong Wu mailto:tong1...@intel.com>>
---
 libavfilter/vf_hwmap.c | 17 +
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/libavfilter/vf_hwmap.c b/libavfilter/vf_hwmap.c
index 2e03dfc1fe..136980c982 100644
--- a/libavfilter/vf_hwmap.c
+++ b/libavfilter/vf_hwmap.c
@@ -70,23 +70,32 @@ static int hwmap_config_output(AVFilterLink *outlink)
 device_is_derived = 0;

 if (inlink->hw_frames_ctx) {
+enum AVHWDeviceType type;
 hwfc = (AVHWFramesContext*)inlink->hw_frames_ctx->data;

 if (ctx->derive_device_type) {
-enum AVHWDeviceType type;
-
 type = av_hwdevice_find_type_by_name(ctx->derive_device_type);
 if (type == AV_HWDEVICE_TYPE_NONE) {
 av_log(avctx, AV_LOG_ERROR, "Invalid device type.\n");
 err = AVERROR(EINVAL);
 goto fail;
 }
+} else {
+type = av_hwdevice_get_type_by_pix_fmt(outlink->format);
+if (type == AV_HWDEVICE_TYPE_NONE) {
+av_log(avctx, AV_LOG_ERROR, "Could not get device type from "
+   "format %s.\n", av_get_pix_fmt_name(outlink->format));
+err = AVERROR(EINVAL);
+goto fail;
+}
+}

+if (!device || ctx->derive_device_type) {
 err = av_hwdevice_ctx_create_derived(, type,
- hwfc->device_ref, 0);
+hwfc->device_ref, 0);
 if (err < 0) {
 av_log(avctx, AV_LOG_ERROR, "Failed to created derived "
-   "device context: %d.\n", err);
+"device context: %d.\n", err);

Looks like mixed re-indentation and functional changes in same commit.

Yes this should not be re-indented, will update later.

Thanks,
Tong

 goto fail;
 }
 device_is_derived = 1;
--
2.35.1.windows.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with 
subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/3] avfilter/vf_hwmap: get the AVHWDeviceType from outlink format

[Wu, Tong1]

> > On Jun 29, 2022, at 2:32 PM, Tong Wu 
> wrote:
> >
> > When a derive_device_type is not specified, the hwmap filter should be
> > able to retrieve AVHWDeviceType from outlink->format and create
> > corresponding hwdevice context.
> >
> > Signed-off-by: Tong Wu 
> > ---
> > libavfilter/vf_hwmap.c | 26 +-
> > 1 file changed, 17 insertions(+), 9 deletions(-)
> >
> > diff --git a/libavfilter/vf_hwmap.c b/libavfilter/vf_hwmap.c index
> > 2e03dfc1fe..a0c2e134cf 100644
> > --- a/libavfilter/vf_hwmap.c
> > +++ b/libavfilter/vf_hwmap.c
> > @@ -72,26 +72,34 @@ static int hwmap_config_output(AVFilterLink
> *outlink)
> > if (inlink->hw_frames_ctx) {
> > hwfc = (AVHWFramesContext*)inlink->hw_frames_ctx->data;
> >
> > -if (ctx->derive_device_type) {
> > -enum AVHWDeviceType type;
> > +enum AVHWDeviceType type;
> 
> mixed declarations and code.
> 
> >
> > +if (ctx->derive_device_type) {
> > type = av_hwdevice_find_type_by_name(ctx->derive_device_type);
> > if (type == AV_HWDEVICE_TYPE_NONE) {
> > av_log(avctx, AV_LOG_ERROR, "Invalid device type.\n");
> > err = AVERROR(EINVAL);
> > goto fail;
> > }
> > -
> > -err = av_hwdevice_ctx_create_derived(, type,
> > - hwfc->device_ref, 0);
> > -if (err < 0) {
> > -av_log(avctx, AV_LOG_ERROR, "Failed to created derived "
> > -   "device context: %d.\n", err);
> > +} else {
> > +type = av_hwdevice_get_type_by_pix_fmt(outlink->format);
> > +if (type == AV_HWDEVICE_TYPE_NONE) {
> > +av_log(avctx, AV_LOG_ERROR, "Could not get device type 
> > from "
> > +   "format %s.\n", 
> > av_get_pix_fmt_name(outlink->format));
> > +err = AVERROR(EINVAL);
> > goto fail;
> > }
> > -device_is_derived = 1;
> > }
> >
> > +err = av_hwdevice_ctx_create_derived(, type,
> > + hwfc->device_ref, 0);
> 
> I think a new device should be created only if
> 
> if (device == NULL || ctx->derive_device_type != NULL)
> 
> Now a new device is created unconditionally.

Thanks a lot for the review. That makes sense. Will update them in v2.

> 
> > +if (err < 0) {
> > +av_log(avctx, AV_LOG_ERROR, "Failed to created derived "
> > +   "device context: %d.\n", err);
> > +goto fail;
> > +}
> > +device_is_derived = 1;
> > +
> > desc = av_pix_fmt_desc_get(outlink->format);
> > if (!desc) {
> > err = AVERROR(EINVAL);
> > --
> > 2.35.1.windows.2
> >
> > ___
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder encode VAAPI and D3D11 frames directly

[Wu, Tong1]

> Quoting Wu, Tong1 (2022-06-08 06:47:27)
> > >
> > > Quoting Tong Wu (2022-06-07 11:22:16)
> > > > QSV encoder is able to encode frames with VAAPI or D3D11 pixel
> > > > format directly. This patch adds support for qsv encoder to accept
> > > > VAAPI and
> > > > D3D11 pixel formats as input.
> > >
> > > This looks like an ad-hoc hack to me. Encoders should not do these
> > > kinds of tricks.
> > >
> > > --
> > > Anton Khirnov
> >
> > Thanks for the comments. The MFXSurface is based on VaSurface on Linux
> > and D3D texture on Windows. Since the QSV encoder can accept
> > AV_PIX_FMT_QSV as input, it seems kind of reasonable to accept VAAPI
> > and D3D as its input. And it just may not look like a 'real' trick,
> > let's say, for example, make QSV encoder accept VULKAN format
> > directly. By adding this patch, we just want QSV encoder have more
> > input format supports like what nvenc does.
> 
> The difference with nvenc is that the nvenc API actually supports d3d
> textures directly, our encoder wrapper merely passes them through.
> 
> Your patch, on the other hand, derives a new device inside the decoder.
> The intent behind the hwcontext interface is that such operations should be
> left to the library caller, and are actually quite easy to do. So I don't see 
> why
> is this patch really needed.
> 
> > Plus, this patch can really help the users who have hybrid transcode needs.
> 
> Could you elaborate? How would this patch be useful in this specific case.
> Why can't the callers dervice the device themselves?
> 
> --
> Anton Khirnov

It looks easier and more convenient for the users because they don't derive 
them manually.
But yes, I'm convinced that it may be not the work that an encoder should do. 
Thanks for the comments. 

Regards,
Tong
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder encode VAAPI and D3D11 frames directly

[Wu, Tong1]

> > -Original Message-
> > From: ffmpeg-devel  On Behalf Of
> > Xiang, Haihao
> > Sent: Thursday, June 9, 2022 8:48 AM
> > To: ffmpeg-devel@ffmpeg.org
> > Cc: Wu, Tong1 ; Chen, Wenbin
> > 
> > Subject: Re: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder
> > encode VAAPI and D3D11 frames directly
> >
> > On Wed, 2022-06-08 at 11:13 +, Soft Works wrote:
> > > > -Original Message-
> > > > From: ffmpeg-devel  On Behalf Of
> > Xiang,
> > > > Haihao
> > > > Sent: Wednesday, June 8, 2022 10:42 AM
> > > > To: ffmpeg-devel@ffmpeg.org
> > > > Cc: Wu, Tong1 ; Chen, Wenbin
> > 
> > > > Subject: Re: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV
> > encoder encode
> > > > VAAPI and D3D11 frames directly
> > > >
> > > > On Wed, 2022-06-08 at 05:08 +, Soft Works wrote:
> > > > > > -Original Message-
> > > > > > From: ffmpeg-devel  On
> > Behalf Of Tong
> > > > > > Wu
> > > > > > Sent: Tuesday, June 7, 2022 11:22 AM
> > > > > > To: ffmpeg-devel@ffmpeg.org
> > > > > > Cc: Tong Wu ; Wenbin Chen
> > 
> > > > > > Subject: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV
> > encoder encode
> > > > > > VAAPI
> > > > > > and D3D11 frames directly
> > >
> > > [..]
> > >
> > > > > > 2.35.1.windows.2
> > > > >
> > > > > Hi,
> > > > >
> > > > > thanks for submitting this patch. Though, I'm afraid, but this
> > > > >
> > > > > - fundamentally contradicts the logic of ffmpeg's handling of
> > hw
> > > >
> > > > acceleration,
> > > > >   hw device and hw frames contexts
> > > > > - adds code to an encoder, doing things an encoder is not
> > supposed to do-
> > > >
> > > > qsv
> > > > > encoders and decoders have their own context => QSV
> > > >
> > > > nvdec and nvenc have CUDA but nvenc can also support D3D11va, it
> > sounds make
> > > > sense for me to support D3D11va/vaapi in qsvenc too as
> > d3d11va/vaapi are
> > > > used
> > > > internally in MediaSDK.
> > >
> > > Can you please post a command line showing nvenc working with input
> > > from a D3D11VA decoder and without using any
> > hwmap/hwupload/hwdownload
> > > filters?
> > >
> >
> > According to the code below, nvenc may accept d3d11 frames directly,
> >
> > https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/nvenc.c#L46-
> > L72
> >
> > so the command below should work
> >
> > $> ffmpeg -y -hwaccel_output_format d3d11 -hwaccel d3d11va -i
> > input.mp4 -c:v
> > hevc_nvenc out.mp4
> 
> Right, it does work. Thanks for the command, I had tried like that before, but
> in a "wrong" branch.
> 
> 
> Now I took a bit of a deeper look into it and the ability of NVENC to encode
> from plain D3D11 frames. There are quite a few differences between NVENC
> and QSVENC.
> 
> 
> HW Frames Contexts
> --
> 
> QSVENV
> 
> MSDK cannot work with VAAPI frames, D3D9 frames or D3D11 frames
> directly.
> An application is always required to wrap such frames via mfxSurface and
> manage a collection of mfxSurface descriptions.
> It's an abstraction that allows coding against the MSDK API independent from
> the underlying technology.
> The technical representation of this in ffmpeg is the QSVFramesContext.
> When there's an input of plain VAAPI or D3D11 frames (hw frames context),
> then it is required to derive a new QSVFramesContext from the input hw
> frames context (e.g. via hwmap filter) where the procedure of deriving
> means to set up a new QSVFramesContext which does the required wrapping
> (or "mapping" as ffmpeg calls it).
> 
> I think that the way how this logic is reflected in ffmpeg is thought out very
> well and provides a high degree of flexibility.
> 
> 
> NVENC
> 
> The situation is very different here. Nvidia provides platform independency
> not by wrapping platform-specific GPU frame types, but instead uses its own
> custom type - CUDA memory/frames. This is what decoders are outputting,
> filters are using for input/output and encoders take as input.
> 
> What I do not know, is whether it would be possible to map D3D11 frames to
> CUDA frames and vice versa. In case, that 

Re: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder encode VAAPI and D3D11 frames directly

[Wu, Tong1]

> 
> Quoting Tong Wu (2022-06-07 11:22:16)
> > QSV encoder is able to encode frames with VAAPI or D3D11 pixel format
> > directly. This patch adds support for qsv encoder to accept VAAPI and
> > D3D11 pixel formats as input.
> 
> This looks like an ad-hoc hack to me. Encoders should not do these kinds of
> tricks.
> 
> --
> Anton Khirnov

Thanks for the comments. The MFXSurface is based on VaSurface on Linux and D3D 
texture on Windows. Since the QSV encoder can accept AV_PIX_FMT_QSV as input, 
it seems kind of reasonable to accept VAAPI and D3D as its input. And it just 
may not look like a 'real' trick, let's say, for example, make QSV encoder 
accept VULKAN format directly. By adding this patch, we just want QSV encoder 
have more input format supports like what nvenc does.

Plus, this patch can really help the users who have hybrid transcode needs.

Kindly Regards,
Tong 
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v4 1/3] avutil/hwcontext_qsv: derive QSV frames to D3D11VA frames

[Wu, Tong1]

> On Sat, 2022-05-07 at 02:42 +, Soft Works wrote:
> > > -Original Message-
> > > From: ffmpeg-devel  On Behalf Of
> > > Xiang, Haihao
> > > Sent: Saturday, May 7, 2022 4:36 AM
> > > To: ffmpeg-devel@ffmpeg.org
> > > Cc: Wu, Tong1 
> > > Subject: Re: [FFmpeg-devel] [PATCH v4 1/3] avutil/hwcontext_qsv:
> > > derive QSV frames to D3D11VA frames
> > >
> > > On Fri, 2022-05-06 at 05:57 +, Tong Wu wrote:
> > > > Fixes:
> > > > $ ffmpeg.exe -y -hwaccel qsv -init_hw_device d3d11va=d3d11 \
> > > > -init_hw_device qsv=qsv@d3d11 -c:v h264_qsv -i input.h264 \ -vf
> > > > "hwmap=derive_device=d3d11va,format=d3d11" -f null -
> > > >
> > > > Signed-off-by: Tong Wu 
> > > > ---
> > > >  libavutil/hwcontext_qsv.c | 16 +---
> > > >  1 file changed, 13 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c
> > > > index b28dcffe2a..bf150c8553 100644
> > > > --- a/libavutil/hwcontext_qsv.c
> > > > +++ b/libavutil/hwcontext_qsv.c
> > > > @@ -1281,12 +1281,22 @@ static int
> > >
> > > qsv_frames_derive_from(AVHWFramesContext
> > > > *dst_ctx,
> > > >  #if CONFIG_D3D11VA
> > > >  case AV_HWDEVICE_TYPE_D3D11VA:
> > > >  {
> > > > +D3D11_TEXTURE2D_DESC texDesc;
> > > > +dst_ctx->initial_pool_size = src_ctx-
> > > > initial_pool_size;
> > > >  AVD3D11VAFramesContext *dst_hwctx = dst_ctx->hwctx;
> > > > -mfxHDLPair *pair = (mfxHDLPair*)src_hwctx-
> > > > > surfaces[i].Data.MemId;
> > > >
> > > > -dst_hwctx->texture = (ID3D11Texture2D*)pair->first;
> > > > +dst_hwctx->texture_infos = av_calloc(src_hwctx-
> > > > nb_surfaces,
> > > > +
> > > > + sizeof(*dst_hwctx-
> > > > > texture_infos));
> > >
> > > Please check whether the pointer is NULL
> > >
> > > >  if (src_hwctx->frame_type &
> > >
> > > MFX_MEMTYPE_SHARED_RESOURCE)
> > > >  dst_hwctx->MiscFlags = D3D11_RESOURCE_MISC_SHARED;
> > > > -dst_hwctx->BindFlags =
> > >
> > > qsv_get_d3d11va_bind_flags(src_hwctx-
> > > > > frame_type);
> > > >
> > > > +for (i = 0; i < src_hwctx->nb_surfaces; i++) {
> > > > +mfxHDLPair *pair = (mfxHDLPair*)src_hwctx-
> > > > > surfaces[i].Data.MemId;
> > > >
> > > > +dst_hwctx->texture_infos[i].texture =
> > >
> > > (ID3D11Texture2D*)pair-
> > > > > first;
> > > >
> > > > +dst_hwctx->texture_infos[i].index = pair->second
> > > > + ==
> > > > (mfxMemId)MFX_INFINITE ? (intptr_t)0 : (intptr_t)pair->second;
> > > > +if (i == 0) {
> > > > +ID3D11Texture2D_GetDesc(dst_hwctx-
> > > > > texture_infos[i].texture, );
> > > >
> > > > +}
> > >
> > > Move this out of the for-loop ? You may call
> > > ID3D11Texture2D_GetDesc()
> > > below:
> > >
> > > ID3D11Texture2D_GetDesc(dst_hwctx->texture_infos[0].texture,
> > > );
> >
> > This could crash when src_hwctx->nb_surfaces is 0
> 
> src_hwctx->nb_surface should be greater than 0, see qsv_init_pool() and
> qsv_frames_derive_to().
> 
> On the other hand, dst_hwctx->texture_infos is NULL if src_hwctx-
> >nb_surfaces is 0. I commented that we should check whether the pointer is
> NULL. In addition, texDesc.BindFlags would be uninitialized if src_hwctx-
> >nb_surfaces is 0, so we couldn't use it in the following assignment.
> 
> dst_hwctx->BindFlags = texDesc.BindFlags;
> 
> Thanks
> Haihao
> 
Yes that's right. I am going to modify the code as suggested and resubmit them.

Regards,
Tong

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3 1/4] avutil/hwcontext_qsv: derive QSV frames to D3D11VA frames

[Wu, Tong1]

> -Original Message-
> From: Soft Works 
> Sent: Thursday, May 5, 2022 7:53 PM
> To: FFmpeg development discussions and patches  de...@ffmpeg.org>
> Cc: Wu, Tong1 
> Subject: RE: [FFmpeg-devel] [PATCH v3 1/4] avutil/hwcontext_qsv: derive
> QSV frames to D3D11VA frames
> 
> 
> 
> > -Original Message-
> > From: ffmpeg-devel  On Behalf Of
> Tong
> > Wu
> > Sent: Thursday, May 5, 2022 12:53 PM
> > To: ffmpeg-devel@ffmpeg.org
> > Cc: Tong Wu 
> > Subject: [FFmpeg-devel] [PATCH v3 1/4] avutil/hwcontext_qsv: derive
> > QSV frames to D3D11VA frames
> >
> > Fixes:
> > $ ffmpeg.exe -y -hwaccel qsv -init_hw_device d3d11va=d3d11 \
> > -init_hw_device qsv=qsv@d3d11 -c:v h264_qsv -i input.h264 \ -vf
> > "hwmap=derive_device=d3d11va,format=d3d11" -f null -
> >
> > Signed-off-by: Tong Wu 
> > ---
> >  libavutil/hwcontext_qsv.c | 15 +--
> >  1 file changed, 13 insertions(+), 2 deletions(-)
> >
> > diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c
> > index b28dcffe2a..65af7130b8 100644
> > --- a/libavutil/hwcontext_qsv.c
> > +++ b/libavutil/hwcontext_qsv.c
> > @@ -1281,12 +1281,23 @@ static int
> > qsv_frames_derive_from(AVHWFramesContext *dst_ctx,  #if
> CONFIG_D3D11VA
> >  case AV_HWDEVICE_TYPE_D3D11VA:
> >  {
> > +dst_ctx->initial_pool_size = src_ctx->initial_pool_size;
> >  AVD3D11VAFramesContext *dst_hwctx = dst_ctx->hwctx;
> > -mfxHDLPair *pair = (mfxHDLPair*)src_hwctx-
> > >surfaces[i].Data.MemId;
> > -dst_hwctx->texture = (ID3D11Texture2D*)pair->first;
> > +dst_hwctx->texture_infos = av_calloc(src_hwctx-
> > >nb_surfaces,
> > + sizeof(*dst_hwctx-
> > >texture_infos));
> >  if (src_hwctx->frame_type & MFX_MEMTYPE_SHARED_RESOURCE)
> >  dst_hwctx->MiscFlags = D3D11_RESOURCE_MISC_SHARED;
> >  dst_hwctx->BindFlags =
> > qsv_get_d3d11va_bind_flags(src_hwctx->frame_type);
> 
> This is not right. QSV frames are not only created from here
> (hwcontext_qsv) but also from qsvvpp.c AND also by MSDK internally as VPP
> output frames, which means that we cannot assume the rules we are using
> here being followed.
> 
> You need to query those flags from the (first) texture instead.
> 

Thanks, will resubmit the patchset.

> 
> > +for (i = 0; i < src_hwctx->nb_surfaces; i++) {
> > +mfxHDLPair* pair = (mfxHDLPair*)src_hwctx-
> > >surfaces[i].Data.MemId;
> > +dst_hwctx->texture_infos[i].texture =
> > (ID3D11Texture2D*)pair->first;
> > +if (dst_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET)
> > {
> > +dst_hwctx->texture_infos[i].index = 0;
> > +}
> > +else {
> > +dst_hwctx->texture_infos[i].index =
> > (intptr_t)pair->second;
> > +}
> > +}
> 
> Same as above here. With MSDK it is not guaranteed that VPP output frames
> will be array textures. This depends on the MSDK runtime version.
> They always have the flag D3D11_BIND_RENDER_TARGET, so that's not an
> appropriate condition.
> 
> I would check .second for MFX_INFINITE instead.
> 
> Kind regards,
> softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/5] avutil/hwcontext_d3d11va: fix the uninitialized texture bindflag

[Wu, Tong1]

> > -Original Message-
> > From: ffmpeg-devel  On Behalf Of
> > Hendrik Leppkes
> > Sent: Sunday, May 1, 2022 5:54 PM
> > To: FFmpeg development discussions and patches  > de...@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH v2 1/5] avutil/hwcontext_d3d11va:
> > fix the uninitialized texture bindflag
> >
> > On Sun, May 1, 2022 at 5:48 PM Hendrik Leppkes 
> > wrote:
> > >
> > > On Sun, May 1, 2022 at 7:09 AM Soft Works 
> > wrote:
> > > > I think that's what Hendrik wanted to point out as far as I
> > understood.
> > > >
> > >
> > > Basically, I want explicit behavior, not implicit defaults. Anytime
> > a
> > > hwcontext is created, something should tell it what its going to be
> > > used for, so we can determine which flags make sense (or ideally, it
> > > should just specify the flags).
> > >
> > > This patch creates an implicit default for one use-case, is this
> > going
> > > to work with all others? No, of course not, because it has to know
> > > what flags to set. Thats why explicitly setting those flags is
> > > important, instead of just fixing one implicit case.
> 
> I said I agree with you - basically, just that we need to differentiate 
> between
> the use case:
> 
> 1. Use via API
>=> No defaults should be applied, caller is responsible for specifying
>   the flags
> 
> 2. Use via ffmpeg cli
>=> Applying the render target flag would be safe here.
>   We could require this to set via parameter, but there won't ever
>   be a case to specify something different than the render target flag
> 
> Why? Let's look at the possible flags:
> 
> D3D11_BIND_DECODER
> In all decoding cases, this flag is set explicitly, so it overrides any 
> default we
> would set
> 
> D3D11_BIND_VIDEO_ENCODER
> Set explicitly when required, so it overrides any default we would set
> 
> D3D11_BIND_RENDER_TARGET
> All other cases require this flag (e.g. video processing)
> 
> No Flag
> Dead end, texture would be unusable for any kind of video processing
> 
> 
> > On that note, the example commandline it fixes just does hwupload and
> > nothing else - does that even require render target to be flagged?
> > From what I can tell it uses a simple
> > ID3D11DeviceContext::CopySubresourceRegion to copy from the staging
> > texture, which should not require any particular bind flags. Bind
> > Flags in turn would then depend on what you are actually trying to do
> > with the texture (shader input, etc), in this example... nothing?
> 
> We are still in the context of ffmpeg cli - you know that there are no shaders
> or 3d operations and no etc.
> 
> But at this point, you can derive to another context or you can hwmap.
> For all of those things, you need D3D11_BIND_RENDER_TARGET.
> 
> 
> Summary
> 
> As mentioned I see two possibilities:
> 
> 1. Use D3D11_BIND_RENDER_TARGET as a default when used in context of
>ffmpeg cli, otherwise no default flags
> 
> 2. Require a device initialization parameter in the command line
>(but it would always be about setting the render target flag
>and there's no case where you would NOT want to set it)
> 

Thanks for the possible solutions for this issue. The No.1 seems more 
reasonable because
No.2 just seems more unnecessary. But I will still need to find a better place 
to set the flag.
Before that I am going to resubmit the other 4 patches which have less comments 
and objections
for further review.

Thanks,
Tong 

> Let me know what you think.
> 
> Best regards
> softworkz
> 
> 
> 
> 
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org
> with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 3/5] avutil/hwcontext_d3d11va: add a format check for staging texture

[Wu, Tong1]

> > -Original Message-
> > From: ffmpeg-devel  On Behalf Of
> Tong
> > Wu
> > Sent: Friday, April 29, 2022 12:45 PM
> > To: ffmpeg-devel@ffmpeg.org
> > Cc: Tong Wu 
> > Subject: [FFmpeg-devel] [PATCH v2 3/5] avutil/hwcontext_d3d11va: add a
> > format check for staging texture
> >
> > The texDesc.Format needs to be filled in with a corresponding format.
> > I
> > add a format check to initialize the format in case sometimes the
> > ctx->internal->priv is not initialized, such as during the hwmap
> > process.
> 
> ctx->internal->priv is D3D11VAFramesContext. When it wouldn't be
> initialized, then hwmap couldn't work.
> 
> D3D11VAFramesContext.format is set during d3d11va_frames_init.
> You would need to find out whether init is not called or
> whether AVHWFramesContext.sw_format is not (yet) set during
> init.
> 

For the hwmap process, the new FramesContext is created in 
av_hwframe_ctx_create_derived and the init function is never called.

> 
> If that doesn't work out for some reason, I think the next best
> solution would be to add a 'format parameter' to
> d3d11va_create_staging_texture() and in d3d11va_transfer_data()
> (the only caller) do ID3D11Texture2D_GetDesc() on the frame
> texture ('texture' variable) and pass the returned format to
> d3d11va_create_staging_texture()
> 

I think this solution makes sense. I will resubmit the patch. Thanks for the 
review.

Regards,
Tong


> Kind regards,
> softworkz
> 
> 
> >
> > $ ffmpeg.exe -y -hwaccel qsv -init_hw_device d3d11va=d3d11 \
> > -init_hw_device qsv=qsv@d3d11 -c:v h264_qsv \
> > -i input.h264 -vf
> >
> "hwmap=derive_device=d3d11va,format=d3d11,hwdownload,format=nv12"
> \
> > -f null -
> >
> > Signed-off-by: Tong Wu 
> > ---
> >  libavutil/hwcontext_d3d11va.c | 16 
> >  1 file changed, 16 insertions(+)
> >
> > diff --git a/libavutil/hwcontext_d3d11va.c
> > b/libavutil/hwcontext_d3d11va.c
> > index db529acbb4..0ec0e07d3a 100644
> > --- a/libavutil/hwcontext_d3d11va.c
> > +++ b/libavutil/hwcontext_d3d11va.c
> > @@ -349,6 +349,8 @@ static int
> > d3d11va_create_staging_texture(AVHWFramesContext *ctx)
> >  AVD3D11VADeviceContext *device_hwctx = ctx->device_ctx->hwctx;
> >  D3D11VAFramesContext  *s = ctx->internal->priv;
> >  HRESULT hr;
> > +int i;
> > +
> >  D3D11_TEXTURE2D_DESC texDesc = {
> >  .Width  = ctx->width,
> >  .Height = ctx->height,
> > @@ -360,6 +362,20 @@ static int
> > d3d11va_create_staging_texture(AVHWFramesContext *ctx)
> >  .CPUAccessFlags = D3D11_CPU_ACCESS_READ |
> > D3D11_CPU_ACCESS_WRITE,
> >  };
> >
> > +if (!texDesc.Format) {
> > +for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) {
> > +if (ctx->sw_format == supported_formats[i].pix_fmt) {
> > +texDesc.Format = supported_formats[i].d3d_format;
> > +break;
> > +}
> > +}
> > +if (i == FF_ARRAY_ELEMS(supported_formats)) {
> > +av_log(ctx, AV_LOG_ERROR, "Unsupported pixel format:
> > %s\n",
> > +av_get_pix_fmt_name(ctx->sw_format));
> > +return AVERROR(EINVAL);
> > +}
> > +}
> > +
> >  hr = ID3D11Device_CreateTexture2D(device_hwctx->device, ,
> > NULL, >staging_texture);
> >  if (FAILED(hr)) {
> >  av_log(ctx, AV_LOG_ERROR, "Could not create the staging
> > texture (%lx)\n", (long)hr);
> 

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avutil/hwcontext_d3d11va: enable D3D11_RESOURCE_MISC_SHARED for texture

[Wu, Tong1]

> -Original Message-
> From: ffmpeg-devel  On Behalf Of
> Hendrik Leppkes
> Sent: Wednesday, April 20, 2022 2:35 PM
> To: FFmpeg development discussions and patches  de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] avutil/hwcontext_d3d11va: enable
> D3D11_RESOURCE_MISC_SHARED for texture
> 
> On Wed, Apr 20, 2022 at 6:11 AM Tong Wu
>  wrote:
> >
> > Add D3D11_RESOURCE_MISC_SHARED flag for texture to make it shareable.
> > This can fix the green frames issue when mapping from d3d11va to opencl.
> > Sample command line: ffmpeg.exe -hwaccel d3d11va
> > -hwaccel_output_format
> > d3d11 -i input.264 -vf
> > "hwmap=derive_device=opencl,format=opencl,hwdownload,format=nv12"
> -c:v
> > libx264 output.mp4
> >
> 
> The flags are configurable, we should not force them for specific cases.
> Specifically, there is also two ways to share a texture, this flag and the
> D3D11_RESOURCE_MISC_SHARED_NTHANDLE flag (which is preferable for
> new code), which would conflict with each other, making it impossible to set
> it.
> 

Thanks for the review. Since the flags should not be set for those specific 
cases
like that, is there any feasible way to enable the share resource between d3d11 
and opencl and make the sample command line work?

> - Hendrik
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org
> with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".