Re: [FFmpeg-devel] [PATCH 01/10, v3] avutil: add hwcontext_amf.

[Mark Thompson]

On 30/05/2024 14:08, Dmitrii Ovchinnikov wrote:
> Adds hwcontext_amf, which allows to use shared AMF
> context for the encoder, decoder and AMF-based filters,
> without copy to the host memory.
> It will also allow you to use some optimisations in
> the interaction of components (for example, SAV) and make a more
> manageable and optimal setup for using GPU devices with AMF
> in the case of a fully AMF pipeline.
> It will be a significant performance uplift when full AMF pipeline
> with filters is used.
> 
> We also plan to add Compression artefact removal filter in near feature.
> v2: cleanup header files
> v3: an unnecessary class has been removed.
> ---
>  libavutil/Makefile |   4 +
>  libavutil/hwcontext.c  |   4 +
>  libavutil/hwcontext.h  |   1 +
>  libavutil/hwcontext_amf.c  | 585 +
>  libavutil/hwcontext_amf.h  |  64 
>  libavutil/hwcontext_amf_internal.h |  44 +++
>  libavutil/hwcontext_internal.h |   1 +
>  libavutil/pixdesc.c|   4 +
>  libavutil/pixfmt.h |   5 +
>  9 files changed, 712 insertions(+)
>  create mode 100644 libavutil/hwcontext_amf.c
>  create mode 100644 libavutil/hwcontext_amf.h
>  create mode 100644 libavutil/hwcontext_amf_internal.h
> 
> ...
> +
> +static void amf_dummy_free(void *opaque, uint8_t *data)
> +{
> +
> +}
> +
> +static AVBufferRef *amf_pool_alloc(void *opaque, size_t size)
> +{
> +AVHWFramesContext *hwfc = (AVHWFramesContext *)opaque;
> +AVBufferRef *buf;
> +
> +buf = av_buffer_create(NULL, NULL, amf_dummy_free, hwfc, 
> AV_BUFFER_FLAG_READONLY);
> +if (!buf) {
> +av_log(hwfc, AV_LOG_ERROR, "Failed to create buffer for AMF 
> context.\n");
> +return NULL;
> +}
> +return buf;
> +}

You're still allocating nothing here?

I think what this means is that you don't actually want to implement frames 
context creation at all because it doesn't do anything.

If it is not possible to make an AMFSurface as anything other than an output 
from an AMF component then this would make sense, the decoder can allocate the 
internals.

Look at the DRM hwcontext for an example that works like this - the DRM objects 
can only be made as outputs from devices or by mapping, so there is no frame 
context implementation.

> +
> ...
> diff --git a/libavutil/hwcontext_amf.h b/libavutil/hwcontext_amf.h
> new file mode 100644
> index 00..ef2118dd4e
> --- /dev/null
> +++ b/libavutil/hwcontext_amf.h
> @@ -0,0 +1,64 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 
> USA
> + */
> +
> +
> +#ifndef AVUTIL_HWCONTEXT_AMF_H
> +#define AVUTIL_HWCONTEXT_AMF_H
> +
> +#include "pixfmt.h"
> +#include "hwcontext.h"
> +#include 
> +#include 
> +#include 
> +#include 
> +
> +/**
> + * This struct is allocated as AVHWDeviceContext.hwctx
> + */
> +typedef struct AVAMFDeviceContext {
> +HMODULElibrary;

What is this type?  (It looks like a Windows type, but I thought this was 
cross-platform.)

> +AMFFactory *factory;
> +AMFDebug   *debug;
> +AMFTrace   *trace;
> +void   *trace_writer;

Are all of these objects necessary to operation of the AMF device?  Please 
remove elements which are not necessary and add them to the private context if 
they are otherwise useful.

> +
> +int64_tversion; ///< version of AMF runtime

Why is the version necessary to expose in the public API?  Is it not possible 
to call the QueryVersion function after starting?

> +AMFContext *context;
> +intmem_type;
Is mem_type really necessary to expose in the piblic API?  Can the user not 
determine this by some API call?

> +} AVAMFDeviceContext;
> +
> +enum AMF_SURFACE_FORMAT av_amf_av_to_amf_format(enum AVPixelFormat fmt);
> +enum AVPixelFormat av_amf_to_av_format(enum AMF_SURFACE_FORMAT fmt);
> +

All of the following functions should not be public symbols.  You want to 
implement the hwcontext functions so that these all work without needing 
special implementation for AMF, they should not be individually callable 
because that is not useful.

> +int av_amf_context_create(AVAMFDeviceContext * context,
> + 

Re: [FFmpeg-devel] [PATCH 02/10, v3] avcodec: add amfdec.

[Mark Thompson]

On 30/05/2024 14:08, Dmitrii Ovchinnikov wrote:
> From: Evgeny Pavlov 
> 
> Added AMF based h264, hevc, av1 decoders.
> Co-authored-by: Dmitrii Ovchinnikov 
> v2: added encoder reinitialisation
> v3: use AMF_SURFACE_UNKNOWN to int decoder(ctx->output_format before)
> ---
>  libavcodec/Makefile|   7 +-
>  libavcodec/allcodecs.c |   3 +
>  libavcodec/amfdec.c| 696 +
>  libavcodec/amfdec.h|  63 
>  4 files changed, 767 insertions(+), 2 deletions(-)
>  create mode 100644 libavcodec/amfdec.c
>  create mode 100644 libavcodec/amfdec.h
> 
> ...
> +
> +const enum AVPixelFormat amf_dec_pix_fmts[] = {
> +AV_PIX_FMT_YUV420P,
> +AV_PIX_FMT_NV12,
> +AV_PIX_FMT_BGRA,
> +AV_PIX_FMT_ARGB,
> +AV_PIX_FMT_RGBA,
> +AV_PIX_FMT_GRAY8,
> +AV_PIX_FMT_BGR0,
> +AV_PIX_FMT_YUYV422,
> +AV_PIX_FMT_P010,
> +AV_PIX_FMT_P012,
> +AV_PIX_FMT_YUV420P10,
> +AV_PIX_FMT_YUV420P12,
> +AV_PIX_FMT_YUV420P16,
> +#if CONFIG_D3D11VA
> +AV_PIX_FMT_D3D11,
> +#endif
> +#if CONFIG_DXVA2
> +AV_PIX_FMT_DXVA2_VLD,
> +#endif
> +AV_PIX_FMT_AMF_SURFACE,
> +AV_PIX_FMT_NONE
> +};

What is this set of formats doing?  Most of them are ignored becase get_format 
below only ever offers two choices.

> +
> +static const AVCodecHWConfigInternal *const amf_hw_configs[] = {
> +&(const AVCodecHWConfigInternal) {
> +.public = {
> +.pix_fmt = AV_PIX_FMT_AMF_SURFACE,
> +.methods = AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX |

See below, I don't think it makes sense to have HW_FRAMES_CTX in this decoder.

> +   AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX,
> +.device_type = AV_HWDEVICE_TYPE_AMF,
> +},
> +.hwaccel = NULL,
> +},
> +NULL
> +};
> +
> ...
> +
> +static int amf_init_decoder(AVCodecContext *avctx)
> +{
> +AMFDecoderContext *ctx = avctx->priv_data;
> +AVAMFDeviceContext * internal = ctx->amf_device_ctx;
> +const wchar_t   *codec_id = NULL;
> +AMF_RESULT  res;
> +AMFBuffer   *buffer;
> +amf_int64   color_profile;
> +int pool_size = 36;
> +
> +ctx->drain = 0;
> +ctx->resolution_changed = 0;
> +
> +switch (avctx->codec->id) {
> +case AV_CODEC_ID_H264:
> +codec_id = AMFVideoDecoderUVD_H264_AVC;
> +break;
> +case AV_CODEC_ID_HEVC: {
> +if (avctx->profile == AV_PROFILE_HEVC_MAIN_10)

You won't know profile here?  It is an output field, the decoder has to set it 
once it determines it from the stream.

> +codec_id = AMFVideoDecoderHW_H265_MAIN10;
> +else
> +codec_id = AMFVideoDecoderHW_H265_HEVC;
> +} break;
> +case AV_CODEC_ID_AV1:
> +if (avctx->profile == AV_PROFILE_AV1_PROFESSIONAL)
> +codec_id = AMFVideoDecoderHW_AV1_12BIT;
> +else
> +codec_id = AMFVideoDecoderHW_AV1;
> +break;
> +default:
> +break;
> +}
> +AMF_RETURN_IF_FALSE(ctx, codec_id != NULL, AVERROR(EINVAL), "Codec %d is 
> not supported\n", avctx->codec->id);
> +
> +...> +
> +static int amf_decode_init(AVCodecContext *avctx)
> +{
> +AMFDecoderContext *ctx = avctx->priv_data;
> +int ret;
> +ctx->local_context = 0;
> +ctx->in_pkt = av_packet_alloc();
> +if (!ctx->in_pkt)
> +return AVERROR(ENOMEM);
> +
> +if (avctx->hw_frames_ctx){

This will never be set at init time because the user sets it in the get_format 
callback (see documentation for the field).

Even ignoring that, I don't see how this would make sense ayway?  The AMF 
frames context is a dummy shell containing nothing, so the 
AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX can't do anything useful.

(How are you testing this path?)

> +AVHWFramesContext *frames_ctx = 
> (AVHWFramesContext*)avctx->hw_frames_ctx->data;
> +if (frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_AMF) {
> +ctx->amf_device_ctx =  frames_ctx->device_ctx->hwctx;
> +}
> +}
> +else if  (avctx->hw_device_ctx && !avctx->hw_frames_ctx) {
> +AVHWDeviceContext   *hwdev_ctx;
> +AVHWFramesContext *hwframes_ctx;
> +hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data;
> +if (hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF)
> +{
> +ctx->amf_device_ctx =  hwdev_ctx->hwctx;
> +}
> +
> +avctx->hw_frames_ctx = av_hwframe_ctx_alloc(avctx->hw_device_ctx);
> +
> +if (!avctx->hw_frames_ctx) {
> +av_log(avctx, AV_LOG_ERROR, "av_hwframe_ctx_alloc failed\n");
> +return AVERROR(ENOMEM);
> +}
> +
> +hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data;
> +hwframes_ctx->width = FFALIGN(avctx->coded_width,  32);
> +hwframes_ctx->height= FFALIGN(avc

[FFmpeg-devel] [PATCH v3 1/3] lavc/h265_profile_level: Expand profile compatibility checking

[Mark Thompson]

Replace existing get_profile() with find_profile(), which finds the
lowest compatible profile rather than requiring an exact match.
---
Now with an enum tag.


 libavcodec/h265_profile_level.c | 80 +
 libavcodec/h265_profile_level.h | 70 -
 libavcodec/tests/h265_levels.c  |  2 +-
 libavcodec/vaapi_hevc.c |  2 +-
 libavcodec/vdpau_hevc.c |  2 +-
 5 files changed, 124 insertions(+), 32 deletions(-)

diff --git a/libavcodec/h265_profile_level.c b/libavcodec/h265_profile_level.c
index 7ff9681f65..1e96ec1c32 100644
--- a/libavcodec/h265_profile_level.c
+++ b/libavcodec/h265_profile_level.c
@@ -16,6 +16,8 @@
  * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
  */
 
+#include 
+
 #include "h265_profile_level.h"
 
 
@@ -40,6 +42,7 @@ static const H265LevelDescriptor h265_levels[] = {
 };
 
 static const H265ProfileDescriptor h265_profiles[] = {
+{ "Invalid" },
 // profile_idc   8bit   one-picture
 //   HT-profile  | 422chroma| lower-bit-rate
 //   |  14bit|  | 420chroma |  | CpbVclFactor MinCrScaleFactor
@@ -119,41 +122,62 @@ static const H265ProfileDescriptor h265_profiles[] = {
   5, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 2, 4000, 4400, 6.000, 0.5, 6 },
 };
 
+static_assert(H265_PROFILE_COUNT == FF_ARRAY_ELEMS(h265_profiles),
+  "Incorrect H.265 profiles table.");
+
 
-const H265ProfileDescriptor *ff_h265_get_profile(const H265RawProfileTierLevel 
*ptl)
+const int ff_h265_profile_compatible(const H265RawProfileTierLevel *ptl,
+ enum H265Profile profile)
 {
-int i;
+const H265ProfileDescriptor *desc;
+
+av_assert0(profile > H265_PROFILE_INVALID &&
+   profile < H265_PROFILE_COUNT);
 
 if (ptl->general_profile_space)
-return NULL;
+return 0;
 
-for (i = 0; i < FF_ARRAY_ELEMS(h265_profiles); i++) {
-const H265ProfileDescriptor *profile = &h265_profiles[i];
+desc = &h265_profiles[profile];
 
-if (ptl->general_profile_idc &&
-ptl->general_profile_idc != profile->profile_idc)
-continue;
-if (!ptl->general_profile_compatibility_flag[profile->profile_idc])
-continue;
+if (ptl->general_profile_idc &&
+ptl->general_profile_idc != desc->profile_idc)
+return 0;
+if (!ptl->general_profile_compatibility_flag[desc->profile_idc])
+return 0;
 
-#define check_flag(name) \
-if (profile->name < 2) { \
-if (profile->name != ptl->general_ ## name ## _constraint_flag) \
-continue; \
-}
-check_flag(max_14bit);
-check_flag(max_12bit);
-check_flag(max_10bit);
-check_flag(max_8bit);
-check_flag(max_422chroma);
-check_flag(max_420chroma);
-check_flag(max_monochrome);
-check_flag(intra);
-check_flag(one_picture_only);
-check_flag(lower_bit_rate);
+#define check_flag(flag) \
+if (desc->flag < 2 && \
+desc->flag > ptl->general_ ## flag ## _constraint_flag) \
+return 0;
+check_flag(max_14bit);
+check_flag(max_12bit);
+check_flag(max_10bit);
+check_flag(max_8bit);
+check_flag(max_422chroma);
+check_flag(max_420chroma);
+check_flag(max_monochrome);
+check_flag(intra);
+check_flag(one_picture_only);
+check_flag(lower_bit_rate);
 #undef check_flag
 
-return profile;
+return 1;
+}
+
+
+const H265ProfileDescriptor *ff_h265_get_profile(enum H265Profile profile)
+{
+av_assert0(profile > H265_PROFILE_INVALID &&
+   profile < H265_PROFILE_COUNT);
+
+return &h265_profiles[profile];
+}
+
+const H265ProfileDescriptor *ff_h265_find_profile(const 
H265RawProfileTierLevel *ptl)
+{
+for (int p = 1; p < H265_PROFILE_COUNT; p++) {
+if (ff_h265_profile_compatible(ptl, p))
+return &h265_profiles[p];
 }
 
 return NULL;
@@ -171,12 +195,12 @@ const H265LevelDescriptor *ff_h265_guess_level(const 
H265RawProfileTierLevel *pt
 int i;
 
 if (ptl)
-profile = ff_h265_get_profile(ptl);
+profile = ff_h265_find_profile(ptl);
 else
 profile = NULL;
 if (!profile) {
 // Default to using multiplication factors for Main profile.
-profile = &h265_profiles[4];
+profile = &h265_profiles[H265_PROFILE_MAIN];
 }
 
 pic_size = width * height;
diff --git a/libavcodec/h265_profile_level.h b/libavcodec/h265_profile_level.h
index cd30ac5c50..c019c738a7 100644
--- a/libavcodec/h265_profile_level.h
+++ b/libavcodec/h265_profile_level.h
@@ -24,6 +24,50 @@
 #include "cbs_h265.h"
 
 
+// Enumeration of all H.265 profiles.
+// The list is ordered to match table A.10; numeric values are an index
+// into the latest version of this table and have no codec meaning.
+enum H265Profile {
+H265_PROFILE_INVALID,
+H265_PROFILE_MONOCHROME,
+H265_PROFILE_MONOCHRO

[FFmpeg-devel] [PATCH v3 2/3] lavc: Add test for H.265 profile handling

[Mark Thompson]

---
 libavcodec/Makefile  |   2 +-
 libavcodec/tests/.gitignore  |   1 +
 libavcodec/tests/h265_profiles.c | 440 +++
 tests/fate/libavcodec.mak|   5 +
 4 files changed, 447 insertions(+), 1 deletion(-)
 create mode 100644 libavcodec/tests/h265_profiles.c

diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 8ab4398b6c..e7973fb2e6 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -1297,7 +1297,7 @@ TESTPROGS-$(CONFIG_MJPEG_ENCODER) += 
mjpegenc_huffman
 TESTPROGS-$(HAVE_MMX) += motion
 TESTPROGS-$(CONFIG_MPEGVIDEO) += mpeg12framerate
 TESTPROGS-$(CONFIG_H264_METADATA_BSF) += h264_levels
-TESTPROGS-$(CONFIG_HEVC_METADATA_BSF) += h265_levels
+TESTPROGS-$(CONFIG_HEVC_METADATA_BSF) += h265_levels h265_profiles
 TESTPROGS-$(CONFIG_RANGECODER)+= rangecoder
 TESTPROGS-$(CONFIG_SNOW_ENCODER)  += snowenc
 
diff --git a/libavcodec/tests/.gitignore b/libavcodec/tests/.gitignore
index 0df4ae10a0..bf29f03911 100644
--- a/libavcodec/tests/.gitignore
+++ b/libavcodec/tests/.gitignore
@@ -10,6 +10,7 @@
 /golomb
 /h264_levels
 /h265_levels
+/h265_profiles
 /htmlsubtitles
 /iirfilter
 /jpeg2000dwt
diff --git a/libavcodec/tests/h265_profiles.c b/libavcodec/tests/h265_profiles.c
new file mode 100644
index 00..37fc545558
--- /dev/null
+++ b/libavcodec/tests/h265_profiles.c
@@ -0,0 +1,440 @@
+/*
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/bprint.h"
+#include "libavutil/common.h"
+#include "libavcodec/h265_profile_level.h"
+
+
+enum {
+TYPE_MAIN,
+TYPE_SP,
+TYPE_REXT_INTER,
+TYPE_REXT_INTRA,
+TYPE_HT,
+TYPE_HT_INTRA,
+TYPE_SCC,
+TYPE_HT_SCC,
+TYPE_COUNT,
+};
+enum {
+DEPTH_8,
+DEPTH_10,
+DEPTH_12,
+DEPTH_14,
+DEPTH_16,
+DEPTH_COUNT,
+};
+enum {
+CHROMA_MONO,
+CHROMA_420,
+CHROMA_422,
+CHROMA_444,
+CHROMA_COUNT,
+};
+
+// Table of all profiles indexed by chroma subsampling, bit depth and
+// profile type.  This is currently only being used to verify that
+// profile properties are correct, but if there is some other need for
+// this lookup in lavc then the table should be moved to the common
+// profile-level code.  All profiles should appear exactly once in this
+// table (also verified by a test below).
+static enum H265Profile profile_table[CHROMA_COUNT][DEPTH_COUNT][TYPE_COUNT] = 
{
+[CHROMA_MONO] = {
+[DEPTH_8] = {
+[TYPE_REXT_INTER] = H265_PROFILE_MONOCHROME,
+},
+[DEPTH_10] = {
+[TYPE_REXT_INTER] = H265_PROFILE_MONOCHROME_10,
+},
+[DEPTH_12] = {
+[TYPE_REXT_INTER] = H265_PROFILE_MONOCHROME_12,
+},
+[DEPTH_16] = {
+[TYPE_REXT_INTER] = H265_PROFILE_MONOCHROME_16,
+},
+},
+[CHROMA_420] = {
+[DEPTH_8] = {
+[TYPE_MAIN]   = H265_PROFILE_MAIN,
+[TYPE_SP] = H265_PROFILE_MAIN_STILL_PICTURE,
+[TYPE_REXT_INTRA] = H265_PROFILE_MAIN_INTRA,
+[TYPE_SCC]= H265_PROFILE_SCREEN_EXTENDED_MAIN,
+},
+[DEPTH_10] = {
+[TYPE_MAIN]   = H265_PROFILE_MAIN_10,
+[TYPE_SP] = H265_PROFILE_MAIN_10_STILL_PICTURE,
+[TYPE_REXT_INTRA] = H265_PROFILE_MAIN_10_INTRA,
+[TYPE_SCC]= H265_PROFILE_SCREEN_EXTENDED_MAIN_10,
+},
+[DEPTH_12] = {
+[TYPE_REXT_INTER] = H265_PROFILE_MAIN_12,
+[TYPE_REXT_INTRA] = H265_PROFILE_MAIN_12_INTRA,
+},
+},
+[CHROMA_422] ={
+[DEPTH_10] = {
+[TYPE_REXT_INTER] = H265_PROFILE_MAIN_422_10,
+[TYPE_REXT_INTRA] = H265_PROFILE_MAIN_422_10_INTRA,
+},
+[DEPTH_12] = {
+[TYPE_REXT_INTER] = H265_PROFILE_MAIN_422_12,
+[TYPE_REXT_INTRA] = H265_PROFILE_MAIN_422_12_INTRA,
+},
+},
+[CHROMA_444] ={
+[DEPTH_8] = {
+[TYPE_SP] = H265_PROFILE_MAIN_444_STILL_PICTURE,
+[TYPE_REXT_INTER] = H265_PROFILE_MAIN_444,
+[TYPE_REXT_INTRA] = H265_PROFILE_MAIN_444_INTRA,
+[TYPE_HT] = H265_PROFILE_HIGH_THROUGHPUT

[FFmpeg-devel] [PATCH v3 3/3] lavc/vaapi_hevc: Don't require exact profiles

[Mark Thompson]

Rather than turning the constraint flags into a single profile and then
searching for that profile (and failing if it doesn't match any profile
exactly), instead search all supported profiles and use the first one
which supports the given set of constraint flags.
---
 libavcodec/vaapi_decode.c | 45 +++---
 libavcodec/vaapi_hevc.c   | 99 ++-
 libavcodec/vaapi_hevc.h   |  4 +-
 3 files changed, 87 insertions(+), 61 deletions(-)

diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
index 7c91d50f7b..c9d8add69f 100644
--- a/libavcodec/vaapi_decode.c
+++ b/libavcodec/vaapi_decode.c
@@ -393,7 +393,9 @@ static const struct {
 enum AVCodecID codec_id;
 int codec_profile;
 VAProfile va_profile;
-VAProfile (*profile_parser)(AVCodecContext *avctx);
+VAProfile (*match_profile)(AVCodecContext *avctx,
+   const VAProfile *profile_list,
+   int profile_count);
 } vaapi_profile_map[] = {
 #define MAP(c, p, v, ...) { AV_CODEC_ID_ ## c, AV_PROFILE_ ## p, VAProfile ## 
v, __VA_ARGS__ }
 MAP(MPEG2VIDEO,  MPEG2_SIMPLE,MPEG2Simple ),
@@ -420,9 +422,9 @@ static const struct {
 #endif
 #if VA_CHECK_VERSION(1, 2, 0) && CONFIG_HEVC_VAAPI_HWACCEL
 MAP(HEVC,HEVC_REXT,   None,
- ff_vaapi_parse_hevc_rext_scc_profile ),
+ ff_vaapi_hevc_match_rext_scc_profile ),
 MAP(HEVC,HEVC_SCC,None,
- ff_vaapi_parse_hevc_rext_scc_profile ),
+ ff_vaapi_hevc_match_rext_scc_profile ),
 #endif
 MAP(MJPEG,   MJPEG_HUFFMAN_BASELINE_DCT,
   JPEGBaseline),
@@ -505,22 +507,33 @@ static int vaapi_decode_make_config(AVCodecContext *avctx,
 vaapi_profile_map[i].codec_profile == AV_PROFILE_UNKNOWN)
 profile_match = 1;
 
-va_profile = vaapi_profile_map[i].profile_parser ?
- vaapi_profile_map[i].profile_parser(avctx) :
- vaapi_profile_map[i].va_profile;
 codec_profile = vaapi_profile_map[i].codec_profile;
-
-for (j = 0; j < profile_count; j++) {
-if (va_profile == profile_list[j]) {
-exact_match = profile_match;
+if (vaapi_profile_map[i].match_profile) {
+va_profile =
+vaapi_profile_map[i].match_profile(avctx, profile_list,
+   profile_count);
+if (va_profile != VAProfileNone) {
+matched_va_profile = va_profile;
+matched_ff_profile = codec_profile;
+exact_match = 1;
 break;
 }
-}
-if (j < profile_count) {
-matched_va_profile = va_profile;
-matched_ff_profile = codec_profile;
-if (exact_match)
-break;
+} else {
+va_profile = vaapi_profile_map[i].va_profile;
+
+for (j = 0; j < profile_count; j++) {
+if (va_profile == profile_list[j]) {
+exact_match = profile_match;
+break;
+}
+}
+
+if (j < profile_count) {
+matched_va_profile = va_profile;
+matched_ff_profile = codec_profile;
+if (exact_match)
+break;
+}
 }
 }
 av_freep(&profile_list);
diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c
index 6164a1d223..716a4a4b43 100644
--- a/libavcodec/vaapi_hevc.c
+++ b/libavcodec/vaapi_hevc.c
@@ -589,63 +589,74 @@ static int ptl_convert(const PTLCommon *general_ptl, 
H265RawProfileTierLevel *h2
 }
 
 /*
- * Find exact va_profile for HEVC Range Extension and Screen Content Coding 
Extension
+ * Find compatible va_profile for HEVC Range Extension and Screen
+ * Content Coding Extension profiles.
  */
-VAProfile ff_vaapi_parse_hevc_rext_scc_profile(AVCodecContext *avctx)
+VAProfile ff_vaapi_hevc_match_rext_scc_profile(AVCodecContext *avctx,
+   const VAProfile *profile_list,
+   int profile_count)
 {
 const HEVCContext *h = avctx->priv_data;
 const HEVCSPS *sps = h->ps.sps;
 const PTL *ptl = &sps->ptl;
 const PTLCommon *general_ptl = &ptl->general_ptl;
-const H265ProfileDescriptor *profile;
 H265RawProfileTierLevel h265_raw_ptl = {0};
 
+static const struct {
+enum H265Profile profile;
+VAProfile va_profile;
+} map[] = {
+#if VA_CHECK_VERSION(1, 2, 0)
+{ H265_PROFILE_MAIN_12,
+  VAProfileHEVCMain12 },
+{ H265_PROFILE_MAIN_422_10,
+  VAProfileHEVCMain422_10 },
+{ H265_PROFILE_MAIN_422_12,
+  VAProfileHEVCMain422_12 },
+{ H265_PROFILE_MAIN_444,
+  VAProfileHEVCMain444 },
+{ H265_PROFILE_MAIN_444_10,
+  VAProfile

[FFmpeg-devel] [PATCH v5 1/2] configure, lavu, lavc, lavfi: Remove libva 1.x support

[Mark Thompson]

libva 2.0 was released in 2017 and the 2.x versions are included in all
supported distributions nowadays.  Various features no longer need any
configure check after this change, including all codecs except AV1.
Note that the libva version is the API version plus one, so this is
removing support for VAAPI 0.x and requiring VAAPI 1.x.
---
Now squashed.


 configure  | 25 +++
 libavcodec/vaapi_decode.c  | 39 ++---
 libavcodec/vaapi_encode.c  | 78 ++
 libavcodec/vaapi_encode.h  |  9 
 libavcodec/vaapi_encode_h264.c | 18 
 libavcodec/vaapi_encode_h265.c |  2 -
 libavfilter/vaapi_vpp.c| 22 --
 libavutil/hwcontext_vaapi.c| 22 --
 8 files changed, 29 insertions(+), 186 deletions(-)

diff --git a/configure b/configure
index 6c5b8aab9a..06e6fa22f2 100755
--- a/configure
+++ b/configure
@@ -2630,7 +2630,6 @@ CONFIG_EXTRA="
 texturedsp
 texturedspenc
 tpeldsp
-vaapi_1
 vaapi_encode
 vc1dsp
 videodsp
@@ -3200,7 +3199,7 @@ hevc_dxva2_hwaccel_deps="dxva2 DXVA_PicParams_HEVC"
 hevc_dxva2_hwaccel_select="hevc_decoder"
 hevc_nvdec_hwaccel_deps="nvdec"
 hevc_nvdec_hwaccel_select="hevc_decoder"
-hevc_vaapi_hwaccel_deps="vaapi VAPictureParameterBufferHEVC"
+hevc_vaapi_hwaccel_deps="vaapi"
 hevc_vaapi_hwaccel_select="hevc_decoder"
 hevc_vdpau_hwaccel_deps="vdpau VdpPictureInfoHEVC"
 hevc_vdpau_hwaccel_select="hevc_decoder"
@@ -3272,7 +3271,7 @@ vp9_dxva2_hwaccel_deps="dxva2 DXVA_PicParams_VP9"
 vp9_dxva2_hwaccel_select="vp9_decoder"
 vp9_nvdec_hwaccel_deps="nvdec"
 vp9_nvdec_hwaccel_select="vp9_decoder"
-vp9_vaapi_hwaccel_deps="vaapi VADecPictureParameterBufferVP9_bit_depth"
+vp9_vaapi_hwaccel_deps="vaapi"
 vp9_vaapi_hwaccel_select="vp9_decoder"
 vp9_vdpau_hwaccel_deps="vdpau VdpPictureInfoVP9"
 vp9_vdpau_hwaccel_select="vp9_decoder"
@@ -3365,7 +3364,6 @@ hevc_qsv_decoder_select="hevc_mp4toannexb_bsf qsvdec"
 hevc_qsv_encoder_select="hevcparse qsvenc"
 hevc_rkmpp_decoder_deps="rkmpp"
 hevc_rkmpp_decoder_select="hevc_mp4toannexb_bsf"
-hevc_vaapi_encoder_deps="VAEncPictureParameterBufferHEVC"
 hevc_vaapi_encoder_select="atsc_a53 cbs_h265 vaapi_encode"
 hevc_v4l2m2m_decoder_deps="v4l2_m2m hevc_v4l2_m2m"
 hevc_v4l2m2m_decoder_select="hevc_mp4toannexb_bsf"
@@ -3374,7 +3372,6 @@ mjpeg_cuvid_decoder_deps="cuvid"
 mjpeg_qsv_decoder_select="qsvdec"
 mjpeg_qsv_encoder_deps="libmfx"
 mjpeg_qsv_encoder_select="qsvenc"
-mjpeg_vaapi_encoder_deps="VAEncPictureParameterBufferJPEG"
 mjpeg_vaapi_encoder_select="cbs_jpeg jpegtables vaapi_encode"
 mp3_mf_encoder_deps="mediafoundation"
 mpeg1_cuvid_decoder_deps="cuvid"
@@ -3403,7 +3400,6 @@ vp8_mediacodec_decoder_deps="mediacodec"
 vp8_mediacodec_encoder_deps="mediacodec"
 vp8_qsv_decoder_select="qsvdec"
 vp8_rkmpp_decoder_deps="rkmpp"
-vp8_vaapi_encoder_deps="VAEncPictureParameterBufferVP8"
 vp8_vaapi_encoder_select="vaapi_encode"
 vp8_v4l2m2m_decoder_deps="v4l2_m2m vp8_v4l2_m2m"
 vp8_v4l2m2m_encoder_deps="v4l2_m2m vp8_v4l2_m2m"
@@ -3412,7 +3408,6 @@ vp9_mediacodec_decoder_deps="mediacodec"
 vp9_mediacodec_encoder_deps="mediacodec"
 vp9_qsv_decoder_select="qsvdec"
 vp9_rkmpp_decoder_deps="rkmpp"
-vp9_vaapi_encoder_deps="VAEncPictureParameterBufferVP9"
 vp9_vaapi_encoder_select="vaapi_encode"
 vp9_qsv_encoder_deps="libmfx MFX_CODEC_VP9"
 vp9_qsv_encoder_select="qsvenc"
@@ -3966,9 +3961,9 @@ xfade_vulkan_filter_deps="vulkan spirv_compiler"
 yadif_cuda_filter_deps="ffnvcodec"
 yadif_cuda_filter_deps_any="cuda_nvcc cuda_llvm"
 yadif_videotoolbox_filter_deps="metal corevideo videotoolbox"
-hstack_vaapi_filter_deps="vaapi_1"
-vstack_vaapi_filter_deps="vaapi_1"
-xstack_vaapi_filter_deps="vaapi_1"
+hstack_vaapi_filter_deps="vaapi"
+vstack_vaapi_filter_deps="vaapi"
+xstack_vaapi_filter_deps="vaapi"
 hstack_qsv_filter_deps="libmfx"
 hstack_qsv_filter_select="qsvvpp"
 vstack_qsv_filter_deps="libmfx"
@@ -7271,7 +7266,7 @@ enabled libdrm &&
 check_pkg_config libdrm_getfb2 libdrm "xf86drmMode.h" drmModeGetFB2
 
 enabled vaapi &&
-check_pkg_config vaapi "libva >= 0.35.0" "va/va.h" vaInitialize
+check_pkg_config vaapi "libva >= 1.0.0" "va/va.h" vaInitialize
 
 if enabled vaapi; then
 case $target_os in
@@ -7287,18 +7282,10 @@ if enabled vaapi; then
 check_pkg_config vaapi_x11 "libva-x11" "va/va_x11.h" vaGetDisplay
 fi
 
-check_cpp_condition vaapi_1 "va/va.h" "VA_CHECK_VERSION(1, 0, 0)"
-
-check_type "va/va.h va/va_dec_hevc.h" "VAPictureParameterBufferHEVC"
-check_struct "va/va.h" "VADecPictureParameterBufferVP9" bit_depth
 check_struct "va/va.h" "VADecPictureParameterBufferAV1" bit_depth_idx
 check_type   "va/va.h va/va_vpp.h" 
"VAProcFilterParameterBufferHDRToneMapping"
 check_struct "va/va.h va/va_vpp.h" "VAProcPipelineCaps" rotation_flags
 check_struct "va/va.h va/va_vpp.h" "VAProcPipelineCaps" blend_flags
-check_type "va/va.h va/va_enc_hevc.h" "VAEncPictureParameterBufferHEVC"
-check_type "va/va.h

[FFmpeg-devel] [PATCH v5 2/2] hwcontext_vaapi: Deprecate quirks

[Mark Thompson]

These only apply to obsolete drivers which do not work with the
now-required libva 2.x.
---
Fixed checkheaders.


 libavutil/hwcontext_vaapi.c | 123 ++--
 libavutil/hwcontext_vaapi.h |  17 +
 2 files changed, 52 insertions(+), 88 deletions(-)

diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
index caff43d1ae..8591fb88ac 100644
--- a/libavutil/hwcontext_vaapi.c
+++ b/libavutil/hwcontext_vaapi.c
@@ -243,8 +243,7 @@ static int vaapi_frames_get_constraints(AVHWDeviceContext 
*hwdev,
 unsigned int fourcc;
 int err, i, j, attr_count, pix_fmt_count;
 
-if (config &&
-!(hwctx->driver_quirks & AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES)) {
+if (config) {
 attr_count = 0;
 vas = vaQuerySurfaceAttributes(hwctx->display, config->config_id,
0, &attr_count);
@@ -367,23 +366,6 @@ fail:
 return err;
 }
 
-static const struct {
-const char *friendly_name;
-const char *match_string;
-unsigned int quirks;
-} vaapi_driver_quirks_table[] = {
-{
-"Intel iHD",
-"ubit",
-AV_VAAPI_DRIVER_QUIRK_ATTRIB_MEMTYPE,
-},
-{
-"VDPAU wrapper",
-"Splitted-Desktop Systems VDPAU backend for VA-API",
-AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES,
-},
-};
-
 static int vaapi_device_init(AVHWDeviceContext *hwdev)
 {
 VAAPIDeviceContext *ctx = hwdev->hwctx;
@@ -436,36 +418,6 @@ static int vaapi_device_init(AVHWDeviceContext *hwdev)
 if (vendor_string)
 av_log(hwdev, AV_LOG_VERBOSE, "VAAPI driver: %s.\n", vendor_string);
 
-if (hwctx->driver_quirks & AV_VAAPI_DRIVER_QUIRK_USER_SET) {
-av_log(hwdev, AV_LOG_VERBOSE, "Using quirks set by user (%#x).\n",
-   hwctx->driver_quirks);
-} else {
-// Detect the driver in use and set quirk flags if necessary.
-hwctx->driver_quirks = 0;
-if (vendor_string) {
-for (i = 0; i < FF_ARRAY_ELEMS(vaapi_driver_quirks_table); i++) {
-if (strstr(vendor_string,
-   vaapi_driver_quirks_table[i].match_string)) {
-av_log(hwdev, AV_LOG_VERBOSE, "Matched driver string "
-   "as known nonstandard driver \"%s\", setting "
-   "quirks (%#x).\n",
-   vaapi_driver_quirks_table[i].friendly_name,
-   vaapi_driver_quirks_table[i].quirks);
-hwctx->driver_quirks |=
-vaapi_driver_quirks_table[i].quirks;
-break;
-}
-}
-if (!(i < FF_ARRAY_ELEMS(vaapi_driver_quirks_table))) {
-av_log(hwdev, AV_LOG_VERBOSE, "Driver not found in known "
-   "nonstandard list, using standard behaviour.\n");
-}
-} else {
-av_log(hwdev, AV_LOG_VERBOSE, "Driver has no vendor string, "
-   "assuming standard behaviour.\n");
-}
-}
-
 av_free(image_list);
 return 0;
 fail:
@@ -562,48 +514,43 @@ static int vaapi_frames_init(AVHWFramesContext *hwfc)
 }
 
 if (!hwfc->pool) {
-if (!(hwctx->driver_quirks & 
AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES)) {
-int need_memory_type = !(hwctx->driver_quirks & 
AV_VAAPI_DRIVER_QUIRK_ATTRIB_MEMTYPE);
-int need_pixel_format = 1;
-for (i = 0; i < avfc->nb_attributes; i++) {
-if (avfc->attributes[i].type == VASurfaceAttribMemoryType)
-need_memory_type  = 0;
-if (avfc->attributes[i].type == VASurfaceAttribPixelFormat)
-need_pixel_format = 0;
-}
-ctx->nb_attributes =
-avfc->nb_attributes + need_memory_type + need_pixel_format;
+int need_memory_type  = 1;
+int need_pixel_format = 1;
+for (i = 0; i < avfc->nb_attributes; i++) {
+if (avfc->attributes[i].type == VASurfaceAttribMemoryType)
+need_memory_type  = 0;
+if (avfc->attributes[i].type == VASurfaceAttribPixelFormat)
+need_pixel_format = 0;
+}
+ctx->nb_attributes =
+avfc->nb_attributes + need_memory_type + need_pixel_format;
 
-ctx->attributes = av_malloc(ctx->nb_attributes *
-sizeof(*ctx->attributes));
-if (!ctx->attributes) {
-err = AVERROR(ENOMEM);
-goto fail;
-}
+ctx->attributes = av_malloc(ctx->nb_attributes *
+sizeof(*ctx->attributes));
+if (!ctx->attributes) {
+err = AVERROR(ENOMEM);
+goto fail;
+}
 
-for (i = 0; i < avfc->nb_attributes; i++)
-ctx->attributes[i] = avfc->attributes[i];
-if (need_memory_type) {
-ctx->attributes

Re: [FFmpeg-devel] [PATCH 05/16] avutil: add common code for nvtegra

[Mark Thompson]

On 30/05/2024 20:43, averne wrote:
> This includes a new pixel format for nvtegra hardware frames, and several 
> objects for interaction with hardware blocks.
> In particular, this contains code for channels (handles to hardware engines), 
> maps (memory-mapped buffers shared with engines), and command buffers 
> (abstraction for building command lists sent to the engines).
> 
> Signed-off-by: averne 
> ---
>  configure  |2 +
>  libavutil/Makefile |4 +
>  libavutil/nvtegra.c| 1035 
>  libavutil/nvtegra.h|  258 +
>  libavutil/nvtegra_host1x.h |   94 
>  libavutil/pixdesc.c|4 +
>  libavutil/pixfmt.h |8 +
>  7 files changed, 1405 insertions(+)
>  create mode 100644 libavutil/nvtegra.c
>  create mode 100644 libavutil/nvtegra.h
>  create mode 100644 libavutil/nvtegra_host1x.h

I don't think it is reasonable for all of this to be public API surface of 
ffmpeg.

A separate library containing the headers and exposing some set of functions 
like this might make more sense.

If this has to be in ffmpeg then it really needs to all go in one library 
(libavcodec I guess) so that it's not exposing all this internal detail in the 
public API.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 06/16] avutil: add nvtegra hwcontext

[Mark Thompson]

On 30/05/2024 20:43, averne wrote:
> This includes hwdevice and hwframes objects.
> As the multimedia engines work with tiled surfaces (block linear in nvidia 
> jargon), two frame transfer methods are implemented.
> The first makes use of the VIC to perform the copy. Since some revisions of 
> the VIC (such as the one found in the tegra X1) did not support 10+ bit 
> formats, these go through two separate copy steps for the luma and chroma 
> planes.
> The second method copies on the CPU, and is used as a fallback if the VIC 
> constraints are not satisfied.
> 
> Signed-off-by: averne 
> ---
>  libavutil/Makefile |   7 +-
>  libavutil/hwcontext.c  |   4 +
>  libavutil/hwcontext.h  |   1 +
>  libavutil/hwcontext_internal.h |   1 +
>  libavutil/hwcontext_nvtegra.c  | 880 +
>  libavutil/hwcontext_nvtegra.h  |  85 
>  6 files changed, 976 insertions(+), 2 deletions(-)
>  create mode 100644 libavutil/hwcontext_nvtegra.c
>  create mode 100644 libavutil/hwcontext_nvtegra.h
> 
> ...> +
> +static int nvtegra_transfer_data(AVHWFramesContext *ctx, AVFrame *dst, const 
> AVFrame *src) {
> +const AVFrame *swframe;
> +bool from;
> +int num_planes, i;
> +
> +from= !dst->hw_frames_ctx;
> +swframe = from ? dst : src;
> +
> +if (swframe->hw_frames_ctx)
> +return AVERROR(ENOSYS);
> +
> +num_planes = av_pix_fmt_count_planes(swframe->format);
> +
> +for (i = 0; i < num_planes; ++i) {
> +if (((uintptr_t)swframe->data[i] & 0xff) || (swframe->linesize[i] & 
> 0xff)) {
> +av_log(ctx, AV_LOG_WARNING, "Frame address/pitch not aligned to 
> 256, "
> +"falling back to cpu transfer\n");
> +return nvtegra_cpu_transfer_data(ctx, dst, src, num_planes, 
> from);

Are you doing something somewhere to avoid this case?  It seems like it should 
be the normal one (given alignment is typically set signficantly lower than 
256), so this warning would be very spammy.

> +}
> +}
> +
> +return nvtegra_vic_transfer_data(ctx, dst, src, num_planes, from);
> +}
> +
> +const HWContextType ff_hwcontext_type_nvtegra = {
> +.type   = AV_HWDEVICE_TYPE_NVTEGRA,
> +.name   = "nvtegra",
> +
> +.device_hwctx_size  = sizeof(NVTegraDevicePriv),
> +.device_hwconfig_size   = 0,
> +.frames_hwctx_size  = 0,
> +
> +.device_create  = &nvtegra_device_create,
> +.device_init= &nvtegra_device_init,
> +.device_uninit  = &nvtegra_device_uninit,
> +
> +.frames_get_constraints = &nvtegra_frames_get_constraints,
> +.frames_init= &nvtegra_frames_init,
> +.frames_uninit  = &nvtegra_frames_uninit,
> +.frames_get_buffer  = &nvtegra_get_buffer,
> +
> +.transfer_get_formats   = &nvtegra_transfer_get_formats,
> +.transfer_data_to   = &nvtegra_transfer_data,
> +.transfer_data_from = &nvtegra_transfer_data,
> +
> +.pix_fmts = (const enum AVPixelFormat[]) {
> +AV_PIX_FMT_NVTEGRA,
> +AV_PIX_FMT_NONE,
> +},
> +};

What controls whether frames are linear or not?

It seems like the linear case could be exposed directly to the user rather than 
having to wrap it like this - the decoder could return read-only NV12 (or 
whatever) frames directly and they would work with other components.

> diff --git a/libavutil/hwcontext_nvtegra.h b/libavutil/hwcontext_nvtegra.h
> new file mode 100644
> index 00..8a2383d304
> --- /dev/null
> +++ b/libavutil/hwcontext_nvtegra.h
> @@ -0,0 +1,85 @@
> +/*
> + * Copyright (c) 2024 averne 
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License along
> + * with FFmpeg; if not, write to the Free Software Foundation, Inc.,
> + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
> + */
> +
> +#ifndef AVUTIL_HWCONTEXT_NVTEGRA_H
> +#define AVUTIL_HWCONTEXT_NVTEGRA_H
> +
> +#include 
> +
> +#include "hwcontext.h"
> +#include "buffer.h"
> +#include "frame.h"
> +#include "pixfmt.h"
> +
> +#include "nvtegra.h"
> +
> +/*
> + * Encode a hardware revision into a version number
> + */
> +#define AV_NVTEGRA_ENCODE_REV(maj, min) (((maj & 0xff) << 8) | (min & 0xff))
> +
> +/*
> + * Decode a version number
> + */
> +static inline void av_nvtegra_decode_rev(int rev, int *maj, int *min) {
> +*maj = (rev >> 8) & 0x

Re: [FFmpeg-devel] [PATCH 07/16] hwcontext_nvtegra: add dynamic frequency scaling routines

[Mark Thompson]

On 30/05/2024 20:43, averne wrote:
> To save on energy, the clock speed of multimedia engines should be adapted to 
> their workload.
> 
> Signed-off-by: averne 
> ---
>  libavutil/hwcontext_nvtegra.c | 165 ++
>  libavutil/hwcontext_nvtegra.h |   7 ++
>  2 files changed, 172 insertions(+)
> 
> ...
> diff --git a/libavutil/hwcontext_nvtegra.h b/libavutil/hwcontext_nvtegra.h
> index 8a2383d304..7c845951d9 100644
> --- a/libavutil/hwcontext_nvtegra.h
> +++ b/libavutil/hwcontext_nvtegra.h
> @@ -82,4 +82,11 @@ static inline AVNVTegraMap 
> *av_nvtegra_frame_get_fbuf_map(const AVFrame *frame)
>   */
>  int av_nvtegra_pixfmt_to_vic(enum AVPixelFormat fmt);
>  
> +/*
> + * Dynamic frequency scaling routines
> + */
> +int av_nvtegra_dfs_init(AVHWDeviceContext *ctx, AVNVTegraChannel *channel, 
> int width, int height, double framerate_hz);
> +int av_nvtegra_dfs_update(AVHWDeviceContext *ctx, AVNVTegraChannel *channel, 
> int bitstream_len, int decode_cycles);
> +int av_nvtegra_dfs_uninit(AVHWDeviceContext *ctx, AVNVTegraChannel *channel);
> +
>  #endif /* AVUTIL_HWCONTEXT_NVTEGRA_H */

This really isn't a sensible thing to have in the public API of ffmpeg.  Why on 
earth isn't this sort of detail dealt with by the kernel?  (Which can actually 
see all of the different processes using it, as well.)

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavc/vvc: Invalidate PPSs which refer to a changed SPS

[Mark Thompson]

On 15/06/2024 17:37, Frank Plowman wrote:
> n 15/06/2024 13:24, Nuo Mi wrote:
>> On Sat, Jun 15, 2024 at 2:35 PM Christophe Gisquet <
>> christophe.gisq...@gmail.com> wrote:
>>
>>> Le ven. 14 juin 2024, 11:39, Frank Plowman  a
>>> écrit :
>>>
 When the SPS associated with a particular SPS ID changes, invalidate all
 the PPSs which use that SPS ID.  Fixes crashes with illegal bitstreams.
 This is done in the CBS, rather than in libavcodec/vvc/ps.c like the SPS
 ID reuse validation, as parts of the CBS parsing process for PPSs
 depend on the SPS being referred to.

>>>
>>> I am uncertain about this. I have no definite knowledge nor proof, but I
>>> would have thought these are persistent, IE it's legal to update some of
>>> them, their validity depending on something else.
>>>
>>
>>> Wondering if the tested streams are thus conformant.
>>>
>>> But I don't know the actual rule. Maybe finding an EOB/EOS NUT? Related to
>>> some particular shape of a clean random access point, that would require
>>> retransmitting VPS/SPS/PPS/APS/... ?
>>>
>>> Asking Benjamin Bross might be a better option here.
>>>
>> Hi Chris,
>> spec said sps should not change in a CVS.  Frank has some patches to fix a
>> similar issue.
>> https://github.com/FFmpeg/FFmpeg/commit/2d79ae3f8a3306d24afe43ba505693a8dbefd21b
>>
>>
>> Hi Frank,
>> Did it crash before your error hand code in ps.c?
>> Could you send me the clip?
>>
>> Thank you
>>
> 
> Hi both,
> 
> Thank you for your reviews.
> 
> An example of a crashing bitstream which is fixed by this patch is ID
> 295 available here: https://github.com/ffvvc/tests/pull/43.  The
> relevant part of the bitstream is a sequence of NAL units
> 
> AU (decode_order=5)
> 18. SPS
> sps_seq_parameter_set_id = 0
> sps_ctb_log2_size_y = 5
> 19. PPS
> pps_pic_parameter_set_id = 0
> pps_seq_parameter_set_id = 0
> 20. IDR_N_LP
> ph_pic_order_cnt_lsb = 0
> NoOutputBeforeRecoveryFlag = 1
> ph_pic_parameter_set_id = 0
> 
> AU (decode_order=6)
> 21. AUD
> 22. VPS
> 23. SPS
> sps_seq_parameter_set_id = 0
> sps_ctb_log2_size_y = 7
> 24. PREFIX_APS
> 25. IDR_N_LP
> ph_pic_order_cnt_lsb = 0
> NoOutputBeforeRecoveryFlag = 1
> ph_pic_parameter_set_id = 0
> 
> The layout of SPSs alone is legal (not covered by the checks introduced
> in 2d79ae3f8a3306d24afe43ba505693a8dbefd21b) as the second AU is a CLVSS
> AU.  As a result, the bitstream crashes both before and after
> 2d79ae3f8a3306d24afe43ba505693a8dbefd21b.  What this patch does is
> produce an error when the VCL NAL unit in the second AU (25.) tries to
> use PPS ID 0, as the SPS NAL unit that PPS was defined with reference to
> (18.) is no longer available.
> 
> Christophe, is my interpretation of your point correct when I say you
> are suggesting that the above sequence may be legal, so long as the PPS
> still satisfies the new bounds etc. derived from the second SPS?  I did
> consider this, and I think it may be possible to implement by delaying
> CBS element validation and inference until libavcodec/vvc/ps.c.
> However, there are no bitstreams in the conformance suite which contain
> such a structure and this is different to how the native HEVC decoder
> behaves (see libavcodec/hevc/ps.c:72).

Is there some requirement in H.266 that in a single stream the PPS precedes the 
SPS?

Currently we effectively require that for a single stream because we use the 
SPS to enforce constraints on the PPS in both H.265 and H.266, but I'm not 
seeing a hard dependency and it looks like it will currently work on later 
stream starts as long as the parameters don't change too much.

In H.264 there is an explicit dependency because you need chroma_format_idc to 
parse scaling lists, but again this will usually work because it's unlikely to 
change inline.

If that is not required then this will discard the PPS of a stream where the 
SPS follows the PPS.  (Though I admit that before this it was only dubiously 
working because the bounds checking might be wrong.)

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v1] avcodec/cbs: Keep ff_cbs_trace_syntax_element

[Mark Thompson]

On 10/10/2023 04:30, Dai, Jianhui J wrote:

-Original Message-
From: Dai, Jianhui J 
Sent: Tuesday, October 10, 2023 10:57 AM
To: ffmpeg-devel@ffmpeg.org
Subject: [PATCH v1] avcodec/cbs: Keep ff_cbs_trace_syntax_element

Split ff_cbs_trace_syntax_element from ff_cbs_trace_read_log to decouple the
tracing from GetBitContext. This allows CBS implementations that do not have a
GetBitContext to directly use ff_cbs_trace_syntax_element to trace syntax
elements.

Signed-off-by: Jianhui Dai 
---
  libavcodec/cbs.c  | 41 +--
  libavcodec/cbs_internal.h |  4 
  2 files changed, 30 insertions(+), 15 deletions(-)

diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c index cdd7adebeb..2f2cfcfb31
100644
--- a/libavcodec/cbs.c
+++ b/libavcodec/cbs.c
@@ -498,26 +498,18 @@ void ff_cbs_trace_header(CodedBitstreamContext
*ctx,
  av_log(ctx->log_ctx, ctx->trace_level, "%s\n", name);  }

-void ff_cbs_trace_read_log(void *trace_context,
-   GetBitContext *gbc, int length,
-   const char *str, const int *subscripts,
-   int64_t value)
+void ff_cbs_trace_syntax_element(CodedBitstreamContext *ctx, int position,
+ const char *str, const int *subscripts,
+ const char *bits, int64_t value)
  {
-CodedBitstreamContext *ctx = trace_context;
  char name[256];
-char bits[256];
  size_t name_len, bits_len;
  int pad, subs, i, j, k, n;
-int position;
-
-av_assert0(value >= INT_MIN && value <= UINT32_MAX);

-position = get_bits_count(gbc);
+if (!ctx->trace_enable)
+return;

-av_assert0(length < 256);
-for (i = 0; i < length; i++)
-bits[i] = get_bits1(gbc) ? '1' : '0';
-bits[length] = 0;
+av_assert0(value >= INT_MIN && value <= UINT32_MAX);

  subs = subscripts ? subscripts[0] : 0;
  n = 0;
@@ -545,7 +537,7 @@ void ff_cbs_trace_read_log(void *trace_context,
  av_assert0(n == subs);

  name_len = strlen(name);
-bits_len = length;
+bits_len = strlen(bits);

  if (name_len + bits_len > 60)
  pad = bits_len + 2;
@@ -556,6 +548,25 @@ void ff_cbs_trace_read_log(void *trace_context,
 position, name, pad, bits, value);  }

+void ff_cbs_trace_read_log(void *trace_context,
+   GetBitContext *gbc, int length,
+   const char *str, const int *subscripts,
+   int64_t value) {
+CodedBitstreamContext *ctx = trace_context;
+char bits[256];
+int position;
+
+position = get_bits_count(gbc);
+
+av_assert0(length < 256);
+for (int i = 0; i < length; i++)
+bits[i] = get_bits1(gbc) ? '1' : '0';
+bits[length] = 0;
+
+ff_cbs_trace_syntax_element(ctx, position, str, subscripts, bits,
+value); }
+
  void ff_cbs_trace_write_log(void *trace_context,
  PutBitContext *pbc, int length,
  const char *str, const int *subscripts, diff --git
a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index
07220f1f3e..ff90ce467d 100644
--- a/libavcodec/cbs_internal.h
+++ b/libavcodec/cbs_internal.h
@@ -158,6 +158,10 @@ typedef struct CodedBitstreamType {  void
ff_cbs_trace_header(CodedBitstreamContext *ctx,
   const char *name);

+void ff_cbs_trace_syntax_element(CodedBitstreamContext *ctx, int position,
+ const char *name, const int *subscripts,
+ const char *bitstring, int64_t value);
+

  // Helper functions for read/write of common bitstream elements, including  //
generation of trace output. The simple functions are equivalent to


@Mark Thompson 
Could you please take a look if it's a valid change based on your last refactor?
It's intended to use for the reviewing cbs_vp8 patch.


The simple answer here is to forge a GetBitContext pointing at the right place, 
like the write tracing does.

However: for your VP8 case, I'm a bit unclear why it isn't using get_bits 
already?  The setup seems to be to stop normal parsing at the end of the 
uncompressed header and then read bytes through a pointer for the range coder 
rather than continuing with the bit reader.

If the range decoder used the GetBitContext to read the input instead of 
reading bytes from the array then your problem would be solved.  Doing that 
would also allow it to renormalise directly after each read rather than reading 
by bytes, so the actual bits consumed for each symbol would be visible in 
tracing.

(I admit I'm not very clear what your motivation for making a read-only CBS 
implementation for VP8 is, and that might guide the best answer.  If it's 
tracing then showing the consumed bits precisely seems useful, but if it

Re: [FFmpeg-devel] [PATCH] avcodec/amfenc: Fix for windows imprecise sleep

[Mark Thompson]

On 16/10/2023 10:13, Evgeny Pavlov wrote:

This commit reduces the sleep time on Windows to improve AMF encoding
performance on low resolution input videos.
This fix is for Windows only, because sleep() function isn't
very accurate on Windows OS.

Fix for issue #10622

Signed-off-by: Evgeny Pavlov 
---
  libavcodec/amfenc.c | 4 
  1 file changed, 4 insertions(+)

diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c
index 061859f85c..0c95465d6e 100644
--- a/libavcodec/amfenc.c
+++ b/libavcodec/amfenc.c
@@ -770,7 +770,11 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket 
*avpkt)
  if (query_output_data_flag == 0) {
  if (res_resubmit == AMF_INPUT_FULL || ctx->delayed_drain || (ctx->eof && 
res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max)) {
  block_and_wait = 1;
+#ifdef _WIN32
+av_usleep(0); //Sleep() is not precise on Windows OS.
+#else
  av_usleep(1000);
+#endif
  }
  }
  } while (block_and_wait);


Wasting lots of power by spinning on a CPU core does not seem like a good 
answer to this problem.  (I mean, presumably that is why Windows isn't 
honouring your request for a short sleep, because it wants timers to have 
larger gaps to avoid wasting power.)

Why is there a sleep here at all, anyway?  An API for hardware encoding should 
be providing a way for the caller to wait for an outstanding operation to 
complete.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH avcodec/amfenc: 10 bit support, v4, 1/3] avcodec/amfenc: Fixes the color information in the output.

[Mark Thompson]

On 09/10/2023 10:52, Evgeny Pavlov wrote:

From: Michael Fabian 'Xaymar' Dirks 

added 10 bit support for amf hevc.

before:

command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format 
d3d11 -i test_10bit_file.mkv -an -c:v h264_amf res.dx11_hw_h264.mkv
output -  Format of input frames context (p010le) is not supported by AMF.
command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format 
d3d11 -i test_10bit_file -an -c:v hevc_amf res.dx11_hw_hevc.mkv
output -  Format of input frames context (p010le) is not supported by AMF.

after:

command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format 
d3d11 -i test_10bit_file -an -c:v h264_amf res.dx11_hw_h264.mkv
output -  10-bit input video is not supported by AMF H264 encoder
command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format 
d3d11 -i test_10bit_file -an -c:v hevc_amf res.dx11_hw_hevc.mkv
output -  10bit file

v2 - lost line returned in ff_amf_pix_fmts
v3 - fixes after review
v4 - extract duplicated code, fix incorrect processing of 10-bit input for h264

Co-authored-by: Evgeny Pavlov 
---
  libavcodec/amfenc.c  | 37 +
  libavcodec/amfenc.h  |  3 +++
  libavcodec/amfenc_h264.c | 24 
  libavcodec/amfenc_hevc.c | 26 +-
  4 files changed, 85 insertions(+), 5 deletions(-)

diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c
index 061859f85c..0bd15dd812 100644
--- a/libavcodec/amfenc.c
+++ b/libavcodec/amfenc.c
@@ -60,6 +60,7 @@ const enum AVPixelFormat ff_amf_pix_fmts[] = {
  #if CONFIG_DXVA2
  AV_PIX_FMT_DXVA2_VLD,
  #endif
+AV_PIX_FMT_P010,
  AV_PIX_FMT_NONE
  };
  
@@ -72,6 +73,7 @@ static const FormatMap format_map[] =

  {
  { AV_PIX_FMT_NONE,   AMF_SURFACE_UNKNOWN },
  { AV_PIX_FMT_NV12,   AMF_SURFACE_NV12 },
+{ AV_PIX_FMT_P010,   AMF_SURFACE_P010 },
  { AV_PIX_FMT_BGR0,   AMF_SURFACE_BGRA },
  { AV_PIX_FMT_RGB0,   AMF_SURFACE_RGBA },
  { AV_PIX_FMT_GRAY8,  AMF_SURFACE_GRAY8 },
@@ -785,6 +787,41 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket 
*avpkt)
  return ret;
  }
  
+int ff_amf_get_color_profile(AVCodecContext *avctx)

+{
+amf_int64 color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN;


Can you explain what this CONVERTER_COLOR_PROFILE option is actually doing?

You've passed the primaries and transfer function to AMF options later on, but 
this seems to then only consider a subset of possible matrices rather than just 
passing that through as well to write into the VUI metadata.  Why?

(If this isn't supported by AMF then a more correct solution might be to insert 
a metadata BSF to edit the correct values into the output stream after it has 
been encoded.)


+if (avctx->color_range == AVCOL_RANGE_JPEG) {
+/// Color Space for Full (JPEG) Range
+switch (avctx->colorspace) {
+case AVCOL_SPC_SMPTE170M:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601;
+break;
+case AVCOL_SPC_BT709:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709;
+break;
+case AVCOL_SPC_BT2020_NCL:
+case AVCOL_SPC_BT2020_CL:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020;
+break;
+}
+} else {
+/// Color Space for Limited (MPEG) range
+switch (avctx->colorspace) {
+case AVCOL_SPC_SMPTE170M:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601;
+break;
+case AVCOL_SPC_BT709:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709;
+break;
+case AVCOL_SPC_BT2020_NCL:
+case AVCOL_SPC_BT2020_CL:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020;
+break;
+}
+}
+return color_profile;
+}
+
  const AVCodecHWConfigInternal *const ff_amfenc_hw_configs[] = {
  #if CONFIG_D3D11VA
  HW_CONFIG_ENCODER_FRAMES(D3D11, D3D11VA),
diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h
index 2dbd378ef8..62736ef579 100644
--- a/libavcodec/amfenc.h
+++ b/libavcodec/amfenc.h
@@ -21,6 +21,7 @@
  
  #include 
  
+#include 

  #include 
  #include 
  #include 
@@ -170,6 +171,8 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket 
*avpkt);
  */
  extern const enum AVPixelFormat ff_amf_pix_fmts[];
  
+int ff_amf_get_color_profile(AVCodecContext *avctx);

+
  /**
  * Error handling helper
  */
diff --git a/libavcodec/amfenc_h264.c b/libavcodec/amfenc_h264.c
index bd544d12df..f785e091c9 100644
--- a/libavcodec/amfenc_h264.c
+++ b/libavcodec/amfenc_h264.c
@@ -199,6 +199,8 @@ static av_cold int amf_encode_init_h264(AVCodecContext 
*avctx)
  AMFRate  framerate;
  AMFSize  framesize = AMFConstructSize(avctx->width, 
avctx->height);
  int  deblocking_filter = (a

[FFmpeg-devel] [PATCH] amfenc: Use a blocking call instead of sleeping and polling

[Mark Thompson]

---
On 17/10/2023 18:11, Evgeny Pavlov wrote:

The reason for using av_usleep() here is that AMF API doesn’t provide an
API for explicit wait. There are two modes to get output from encoder:

1. Polling with some sleep to avoid CPU thrashing – currently used in FFmpeg

2. Set timeout parameter on AMF encoder and QueryOutput call will block
till output is available or the timeout happens.

#2 is the preferable way but it is designed more to be used with a separate
polling thread. With a single-thread approach in FFmpeg, the use of timeout
can block input submission making things slower.  This is even more
pronounced when B-frames are enabled and several inputs are needed to produce
the first output.


This approach seems like it should work here?  Run non-blocking until the queue 
is full, then switch to blocking when you need to wait for some output.

I tried the patch enclosing (H.264 only, different proprties needed for other 
codecs), but it doesn't seem to work - the test assert always hits immediately 
and timing shows that QueryOutput didn't block even though the timeout should 
be set?  I'm probably doing something incorrect, maybe you would know how to 
fix it.


The condition of this sleep is in special events (primarily when amf input
queue is full), not the core loop part. During the experiments the cpu
increasing is about 2-4% or so, not a burst.


What cases are you experimenting with?

The most problematic case I can think of is multiple encodes running 
simultaneously sharing the same instance so that each one has to wait for 
others to complete and therefore all queues fill up.

The busy wait will end up being the only place where it can block (since 
everything else runs asynchronously), so you will peg one CPU at close to 100% 
per encode running.

Thanks,

- Mark

 libavcodec/amfenc.c | 22 +++---
 libavcodec/amfenc.h |  1 +
 2 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c
index 061859f85c..db7ddbb083 100644
--- a/libavcodec/amfenc.c
+++ b/libavcodec/amfenc.c
@@ -713,13 +713,22 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket 
*avpkt)
 }
 }

-
+block_and_wait = 0;
 do {
-block_and_wait = 0;
 // poll data
 if (!avpkt->data && !avpkt->buf) {
+int64_t timeout = block_and_wait ? 100 : 0;
+if (timeout != ctx->output_query_timeout) {
+av_log(avctx, AV_LOG_INFO, "Set output query timeout to 
%"PRId64"\n", timeout);
+AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder, 
AMF_VIDEO_ENCODER_QUERY_TIMEOUT, timeout);
+AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Failed to 
set output query timeout\n");
+ctx->output_query_timeout = timeout;
+}
+
 res_query = ctx->encoder->pVtbl->QueryOutput(ctx->encoder, &data);
 if (data) {
+av_log(avctx, AV_LOG_INFO, "QueryOutput returned with data\n");
+
 // copy data to packet
 AMFBuffer *buffer;
 AMFGuid guid = IID_AMFBuffer();
@@ -740,7 +749,13 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket 
*avpkt)
 data->pVtbl->Release(data);

 AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed 
with error %d\n", ret);
+} else {
+av_log(avctx, AV_LOG_INFO, "QueryOutput returned with nothing 
(%d)\n", res_query);
+// For testing, shouldn't hit this unless machine is otherwise 
very loaded.
+av_assert0(!block_and_wait);
 }
+
+block_and_wait = 0;
 }
 res_resubmit = AMF_OK;
 if (ctx->delayed_surface != NULL) { // try to resubmit frame
@@ -769,8 +784,9 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket 
*avpkt)

 if (query_output_data_flag == 0) {
 if (res_resubmit == AMF_INPUT_FULL || ctx->delayed_drain || (ctx->eof && 
res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max)) {
+av_log(avctx, AV_LOG_INFO, "Need to wait for output\n");
 block_and_wait = 1;
-av_usleep(1000);
+//av_usleep(1000);
 }
 }
 } while (block_and_wait);
diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h
index 2dbd378ef8..64c77115b6 100644
--- a/libavcodec/amfenc.h
+++ b/libavcodec/amfenc.h
@@ -72,6 +72,7 @@ typedef struct AmfContext {
 int delayed_drain;
 AMFSurface *delayed_surface;
 AVFrame*delayed_frame;
+int64_t output_query_timeout;

 // shift dts back by max_b_frames in timing
 AVFifo *timestamp_list;
--
2.39.2
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, o

Re: [FFmpeg-devel] [PATCH avcodec/amfenc: 10 bit support, v4, 1/3] avcodec/amfenc: Fixes the color information in the output.

[Mark Thompson]

On 17/10/2023 19:00, Evgeny Pavlov wrote:

On Mon, Oct 16, 2023 at 11:41 PM Mark Thompson  wrote:
...

@@ -785,6 +787,41 @@ int ff_amf_receive_packet(AVCodecContext *avctx,

AVPacket *avpkt)

   return ret;
   }

+int ff_amf_get_color_profile(AVCodecContext *avctx)
+{
+amf_int64 color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN;


Can you explain what this CONVERTER_COLOR_PROFILE option is actually doing?

You've passed the primaries and transfer function to AMF options later on,
but this seems to then only consider a subset of possible matrices rather
than just passing that through as well to write into the VUI metadata.  Why?

(If this isn't supported by AMF then a more correct solution might be to
insert a metadata BSF to edit the correct values into the output stream
after it has been encoded.)

When RGB surface is submitted, AMF encoder not only writes VUI header but

also does color conversion. In this case CONVERTER_COLOR_PROFILE defines
color conversion matrix. This conversion may happen with shaders or inside
VCN if it is capable to do so.

These are input parameters for conversion – if conversion is involved:
AMF_VIDEO_ENCODER_INPUT_COLOR_PROFILE – for color conversion: limited
number of color spaces is supported.
AMF_VIDEO_ENCODER_INPUT_TRANSFER_CHARACTERISTIC
AMF_VIDEO_ENCODER_INPUT_COLOR_PRIMARIES
AMF_VIDEO_ENCODER_INPUT_HDR_METADATA

These are output parameters for conversion and data for VUI:
AMF_VIDEO_ENCODER_OUTPUT_COLOR_PROFILE – for VUI only, used if color
conversion is done outside of AMF
AMF_VIDEO_ENCODER_OUTPUT_TRANSFER_CHARACTERISTIC
AMF_VIDEO_ENCODER_OUTPUT_COLOR_PRIMARIES
AMF_VIDEO_ENCODER_OUTPUT_HDR_METADATA

It would be possible to add unsupported color matrices via editing VUI in
the output stream but it is better to do it via a separate patch as
supported covers most common use cases.


How does the setup you have here support the most common case, where that the 
user has the input in the right format and just wants the colour properties 
that they have set (primaries/transfer/matrix/range/chroma location) to be 
written directly into the encoded stream?

(In most encoders this is the only case, since ad-hoc conversion of the input 
like this is generally considered out-of-scope and done separately.)


+if (avctx->color_range == AVCOL_RANGE_JPEG) {
+/// Color Space for Full (JPEG) Range
+switch (avctx->colorspace) {
+case AVCOL_SPC_SMPTE170M:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601;
+break;
+case AVCOL_SPC_BT709:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709;
+break;
+case AVCOL_SPC_BT2020_NCL:
+case AVCOL_SPC_BT2020_CL:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020;
+break;
+}
+} else {
+/// Color Space for Limited (MPEG) range
+switch (avctx->colorspace) {
+case AVCOL_SPC_SMPTE170M:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601;
+break;
+case AVCOL_SPC_BT709:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709;
+break;
+case AVCOL_SPC_BT2020_NCL:
+case AVCOL_SPC_BT2020_CL:
+color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020;
+break;
+}
+}
+return color_profile;
+}
+
   const AVCodecHWConfigInternal *const ff_amfenc_hw_configs[] = {
   #if CONFIG_D3D11VA
   HW_CONFIG_ENCODER_FRAMES(D3D11, D3D11VA),
...
+pix_fmt = avctx->hw_frames_ctx ?

((AVHWFramesContext*)avctx->hw_frames_ctx->data)->sw_format

+: avctx->pix_fmt;
+if (pix_fmt == AV_PIX_FMT_P010) {
+color_depth = AMF_COLOR_BIT_DEPTH_10;
+}
+AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder,

AMF_VIDEO_ENCODER_HEVC_COLOR_BIT_DEPTH, color_depth);

Is this the input bit depth or the codec bit depth?  (Can they be
different?)


According to AMF documentation, this property "sets the number of bits in
each pixel’s color component in the encoder’s compressed output bitstream".
We should set up correct bit depth for encoder if we have 10-bit input


Doesn't that mean it should be set based on the profile rather than the input 
bit depth?

(Or, if only the input bit depth is supported then you probably want to 
validate above that the profile and input bit depth actually match.)


+/// Color Transfer Characteristics (AMF matches ISO/IEC)
+AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder,

AMF_VIDEO_ENCODER_HEVC_OUTPUT_TRANSFER_CHARACTERISTIC,
(amf_int64)avctx->color_trc);

+/// Color Primaries (AMF matches ISO/IEC)
+AMF_ASSIGN_PROPERTY_INT64(res, ctx->encoder,

AMF_VIDEO_ENCODER_HEVC_OUTPUT_COLOR_PRIMARIES,
(amf_int64)avctx->color_primaries);

+


Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] amfenc: Use a blocking call instead of sleeping and polling

[Mark Thompson]

On 19/10/2023 17:13, Evgeny Pavlov wrote:

On Wed, Oct 18, 2023 at 10:36 PM Mark Thompson  wrote:


---
On 17/10/2023 18:11, Evgeny Pavlov wrote:

The reason for using av_usleep() here is that AMF API doesn’t provide an
API for explicit wait. There are two modes to get output from encoder:

1. Polling with some sleep to avoid CPU thrashing – currently used in

FFmpeg


2. Set timeout parameter on AMF encoder and QueryOutput call will block
till output is available or the timeout happens.

#2 is the preferable way but it is designed more to be used with a

separate

polling thread. With a single-thread approach in FFmpeg, the use of

timeout

can block input submission making things slower.  This is even more
pronounced when B-frames are enabled and several inputs are needed to

produce

the first output.


This approach seems like it should work here?  Run non-blocking until the
queue is full, then switch to blocking when you need to wait for some
output.

I tried the patch enclosing (H.264 only, different proprties needed for
other codecs), but it doesn't seem to work - the test assert always hits
immediately and timing shows that QueryOutput didn't block even though the
timeout should be set?  I'm probably doing something incorrect, maybe you
would know how to fix it.


The condition of this sleep is in special events (primarily when amf

input

queue is full), not the core loop part. During the experiments the cpu
increasing is about 2-4% or so, not a burst.


What cases are you experimenting with?

The most problematic case I can think of is multiple encodes running
simultaneously sharing the same instance so that each one has to wait for
others to complete and therefore all queues fill up.

The busy wait will end up being the only place where it can block (since
everything else runs asynchronously), so you will peg one CPU at close to
100% per encode running.

Thanks,

- Mark

   libavcodec/amfenc.c | 22 +++---
   libavcodec/amfenc.h |  1 +
   2 files changed, 20 insertions(+), 3 deletions(-)

...


Dynamic switching between non-blocking & blocking approaches isn’t
supported in AMF at this time.

We might request to implement this feature for AMF team, but it might took
some time to implement this.


That is unfortunate, but it sounds like something like this is required.


I would suggest using av_usleep(500) until this feature is implemented.


What cases are you experimenting with?


This issue is very easy to reproduce when:

1) low resolution transcoding

2) hardware accelerated decoding

The command line sample:  ffmpeg -hwaccel d3d11va -hwaccel_output_format
d3d11 -i input_480x360_h264.mp4 -c:v hevc_amf  output_480x360_hevc.mp4


To clarify, I meant: what cases are you experimenting with to verify that this 
doesn't cause problems elsewhere?

I agree (and can reproduce) that the specific case with one low-resolution 
stream slightly improves throughput at the cost of increased CPU use.

>> The most problematic case I can think of is multiple encodes running
>> simultaneously sharing the same instance so that each one has to wait for
>> others to complete and therefore all queues fill up.
>>
>> The busy wait will end up being the only place where it can block (since
>> everything else runs asynchronously), so you will peg one CPU at close to
>> 100% per encode running.

I tried this case with two 4K streams and indeed it is a huge regression.  CPU 
use goes from 1-2% of one core for both streams to spinning on two cores, 
around a 100x increase.

Total throughput also decreased by about 10% in my testing, though since I'm 
running on a low-power device that might be an artefact of the CPU spinning 
wasting so much power that other clocks are reduced.

(My test was two instances of

$ ./ffmpeg_g.exe -extra_hw_frames 100 -hwaccel d3d11va -hwaccel_output_format 
d3d11 -i input-4k.mp4 -an -vf loop=loop=20:size=100:start=0 -c:v h264_amf -f 
null -

running simulataneously, looking at the steady state in the loop after the 
first hundred frames with the decoder are complete.)

Please consider this patch rejected in its current form.  IMO this is a hole in 
the AMF API and it needs to be improved to be able to wait for operations to 
complete rather than polling in the user code.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/6] avcodec/cbs: Do not assert on traces beyond 255 bits

[Mark Thompson]

On 22/10/2023 01:35, Michael Niedermayer wrote:

Fixes: Assertion length < 256 failed at libavcodec/cbs.c:517
Fixes: 
62673/clusterfuzz-testcase-minimized-ffmpeg_BSF_TRACE_HEADERS_fuzzer-6490971837431808

Found-by: continuous fuzzing process 
https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer 
---
  libavcodec/cbs.c | 5 +
  1 file changed, 5 insertions(+)

diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c
index cdd7adebebd..2f5d0334a2a 100644
--- a/libavcodec/cbs.c
+++ b/libavcodec/cbs.c
@@ -514,6 +514,11 @@ void ff_cbs_trace_read_log(void *trace_context,
  
  position = get_bits_count(gbc);
  
+if (length >= 256) {

+av_log(ctx->log_ctx, ctx->trace_level, "trace of %d bits truncated at 
255\n", length);
+length = 255;
+}
+
  av_assert0(length < 256);
  for (i = 0; i < length; i++)
  bits[i] = get_bits1(gbc) ? '1' : '0';


IMO the assert is sensible (no syntax element is that large) and so this must 
be catching a bug somewhere else.  Please don't nullify the assert to hide the 
bug.

Can you share the input stream which hit this case?

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 1/2] hwcontext_d3d11: Add BGR0 support

[Mark Thompson]

The 8-bit four-component DXGI container is also used for three-component
RGB without alpha.
---
This list is only used for AV->DXGI mapping, so it doesn't matter that there 
are duplicate DXGI formats in the list.

 libavutil/hwcontext_d3d11va.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c
index cc8c97d2b6..1d249f2088 100644
--- a/libavutil/hwcontext_d3d11va.c
+++ b/libavutil/hwcontext_d3d11va.c
@@ -86,6 +86,7 @@ static const struct {
 } supported_formats[] = {
 { DXGI_FORMAT_NV12, AV_PIX_FMT_NV12 },
 { DXGI_FORMAT_P010, AV_PIX_FMT_P010 },
+{ DXGI_FORMAT_B8G8R8A8_UNORM,AV_PIX_FMT_BGR0 },
 { DXGI_FORMAT_B8G8R8A8_UNORM,AV_PIX_FMT_BGRA },
 { DXGI_FORMAT_R10G10B10A2_UNORM, AV_PIX_FMT_X2BGR10 },
 { DXGI_FORMAT_R16G16B16A16_FLOAT, AV_PIX_FMT_RGBAF16 },
--
2.39.2
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/2] lavfi/ddagrab: Fix 8-bit BGR output to not advertise an alpha component

[Mark Thompson]

There is never an alpha component here, so the actual format is BGR0
rather than BGRA.  This fixes cases which maintain the alpha component
and therefore generate unexpected results.
---
E.g. fixes download and encode with PNG to make an RGB PNG as expected, rather 
than an RGBA PNG with nothing in the alpha channel.  (Previously this was 
relying on the user to realise that the alpha channel contained nothing and 
ignore it.)

 doc/filters.texi   | 2 +-
 libavfilter/vsrc_ddagrab.c | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/filters.texi b/doc/filters.texi
index f5032ddf74..550f9a6ecc 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -28183,7 +28183,7 @@ It accepts the following values:
 @item auto
 Passes all supported output formats to DDA and returns what DDA decides to use.
 @item 8bit
-@item bgra
+@item bgrx
 8 Bit formats always work, and DDA will convert to them if neccesary.
 @item 10bit
 @item x2bgr10
diff --git a/libavfilter/vsrc_ddagrab.c b/libavfilter/vsrc_ddagrab.c
index 9c59faf53e..8ff3c97959 100644
--- a/libavfilter/vsrc_ddagrab.c
+++ b/libavfilter/vsrc_ddagrab.c
@@ -115,14 +115,14 @@ static const AVOption ddagrab_options[] = {
 { "output_fmt", "desired output format",   OFFSET(out_fmt),AV_OPT_TYPE_INT,  
  { .i64 = DXGI_FORMAT_B8G8R8A8_UNORM },0, INT_MAX, FLAGS, "output_fmt" },
 { "auto",   "let dda pick its preferred format", 0,
AV_OPT_TYPE_CONST,  { .i64 = 0 }, 0, INT_MAX, FLAGS, "output_fmt" },
 { "8bit",   "only output default 8 Bit format",  0,
AV_OPT_TYPE_CONST,  { .i64 = DXGI_FORMAT_B8G8R8A8_UNORM },0, INT_MAX, FLAGS, "output_fmt" },
-{ "bgra",   "only output 8 Bit BGRA",0,
AV_OPT_TYPE_CONST,  { .i64 = DXGI_FORMAT_B8G8R8A8_UNORM },0, INT_MAX, FLAGS, "output_fmt" },
+{ "bgrx",   "only output 8 Bit BGRX",0,
AV_OPT_TYPE_CONST,  { .i64 = DXGI_FORMAT_B8G8R8A8_UNORM },0, INT_MAX, FLAGS, "output_fmt" },
 { "10bit",  "only output default 10 Bit format", 0,
AV_OPT_TYPE_CONST,  { .i64 = DXGI_FORMAT_R10G10B10A2_UNORM }, 0, INT_MAX, FLAGS, "output_fmt" },
 { "x2bgr10","only output 10 Bit X2BGR10",0,
AV_OPT_TYPE_CONST,  { .i64 = DXGI_FORMAT_R10G10B10A2_UNORM }, 0, INT_MAX, FLAGS, "output_fmt" },
 { "16bit",  "only output default 16 Bit format", 0,
AV_OPT_TYPE_CONST,  { .i64 = DXGI_FORMAT_R16G16B16A16_FLOAT },0, INT_MAX, FLAGS, "output_fmt" },
 { "rgbaf16","only output 16 Bit RGBAF16",0,
AV_OPT_TYPE_CONST,  { .i64 = DXGI_FORMAT_R16G16B16A16_FLOAT },0, INT_MAX, FLAGS, "output_fmt" },
 { "allow_fallback", "don't error on fallback to default 8 Bit format",
OFFSET(allow_fallback), 
AV_OPT_TYPE_BOOL,   { .i64 = 0},   0,   1, FLAGS },
-{ "force_fmt",  "exclude BGRA from format list (experimental, discouraged by 
Microsoft)",
+{ "force_fmt",  "exclude BGRX from format list (experimental, discouraged by 
Microsoft)",
OFFSET(force_fmt),  
AV_OPT_TYPE_BOOL,   { .i64 = 0},   0,   1, FLAGS },
 { NULL }
 };
@@ -775,7 +775,7 @@ static av_cold int init_hwframes_ctx(AVFilterContext *avctx)
 switch (dda->raw_format) {
 case DXGI_FORMAT_B8G8R8A8_UNORM:
 av_log(avctx, AV_LOG_VERBOSE, "Probed 8 bit RGB frame format\n");
-dda->frames_ctx->sw_format = AV_PIX_FMT_BGRA;
+dda->frames_ctx->sw_format = AV_PIX_FMT_BGR0;
 break;
 case DXGI_FORMAT_R10G10B10A2_UNORM:
 av_log(avctx, AV_LOG_VERBOSE, "Probed 10 bit RGB frame format\n");
--
2.39.2
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] libavcodec/amfenc: Add more pixel formats support

[Mark Thompson]

On 20/10/2023 09:13, Evgeny Pavlov wrote:

On Tue, Jul 18, 2023 at 10:32 AM Evgeny Pavlov  wrote:


This commit adds BGRA, RGBA and ARGB pixel formats for AMF encoders

Signed-off-by: Evgeny Pavlov 
---
  libavcodec/amfenc.c | 3 +++
  1 file changed, 3 insertions(+)

diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c
index cb48f8c273..234cd012ef 100644
--- a/libavcodec/amfenc.c
+++ b/libavcodec/amfenc.c
@@ -74,6 +74,9 @@ static const FormatMap format_map[] =
  { AV_PIX_FMT_NV12,   AMF_SURFACE_NV12 },
  { AV_PIX_FMT_BGR0,   AMF_SURFACE_BGRA },
  { AV_PIX_FMT_RGB0,   AMF_SURFACE_RGBA },
+{ AV_PIX_FMT_BGRA,   AMF_SURFACE_BGRA },
+{ AV_PIX_FMT_RGBA,   AMF_SURFACE_RGBA },
+{ AV_PIX_FMT_ARGB,   AMF_SURFACE_ARGB },
  { AV_PIX_FMT_GRAY8,  AMF_SURFACE_GRAY8 },
  { AV_PIX_FMT_YUV420P,AMF_SURFACE_YUV420P },
  { AV_PIX_FMT_YUYV422,AMF_SURFACE_YUY2 },
--
2.37.3.windows.1

The purpose of this patch is to fix an issue with feeding ddagrab output

to AMF directly. The output of ddagrab might be BGRA, AMF supports this
input, but failed to encode due to missing mapping from AV_PIX_BGRA to
AMF_SURFACE_BGRA in amfenc.c
DXGI isn't firm to distinguish between RGBA & RGBX, so it is safer to
support both to avoid failures like with ddagrab.


See patch just sent to fix the bug in ddagrab that it incorrectly advertises an 
alpha channel.

Do you have ddagrab->amfenc working with just something to fix the alpha 
channel?  To make it work I also need to mess with the bind flags because amfenc 
wants to use the textures as shader resources.

This has been noted as a problem before, where a proper fix would require large changes 
in the format negotiation setup and so it hasn't been done.  I'm wondering whether adding 
"-bind_flags shader_resource" option to ddagrab would be a plausible hack for 
these cases, or whether that's a bit too obscure for an actual user?

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH] cbs_av1: Reject thirty-two zero bits in uvlc code

[Mark Thompson]

The spec allows at least thirty-two zero bits followed by a one to mean
2^32-1, with no constraint on the number of zeroes.  The libaom
reference decoder does not match this, instead reading thirty-two zeroes
but not the following one to mean 2^32-1.  These two interpretations are
incompatible and other implementations may follow one or the other.
Therefore reject thirty-two zeroes because the intended behaviour is not
clear.
---
libaom, dav1d and SVT-AV1 all have the same nonstandard behaviour of stopping 
at thirty-two zeroes and not reading the one.  gav1 just rejects thirty-two 
zeroes.

This is also a source of arbitrarily large single syntax elements to hit 
.

 libavcodec/cbs_av1.c | 18 +-
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c
index 1d9ac5ab44..13c749a25b 100644
--- a/libavcodec/cbs_av1.c
+++ b/libavcodec/cbs_av1.c
@@ -36,7 +36,7 @@ static int cbs_av1_read_uvlc(CodedBitstreamContext *ctx, 
GetBitContext *gbc,
 CBS_TRACE_READ_START();

 zeroes = 0;
-while (1) {
+while (zeroes < 32) {
 if (get_bits_left(gbc) < 1) {
 av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid uvlc code at "
"%s: bitstream ended.\n", name);
@@ -49,10 +49,18 @@ static int cbs_av1_read_uvlc(CodedBitstreamContext *ctx, 
GetBitContext *gbc,
 }

 if (zeroes >= 32) {
-// Note that the spec allows an arbitrarily large number of
-// zero bits followed by a one bit in this case, but the
-// libaom implementation does not support it.
-value = MAX_UINT_BITS(32);
+// The spec allows at least thirty-two zero bits followed by a
+// one to mean 2^32-1, with no constraint on the number of
+// zeroes.  The libaom reference decoder does not match this,
+// instead reading thirty-two zeroes but not the following one
+// to mean 2^32-1.  These two interpretations are incompatible
+// and other implementations may follow one or the other.
+// Therefore we reject thirty-two zeroes because the intended
+// behaviour is not clear.
+av_log(ctx->log_ctx, AV_LOG_ERROR, "Thirty-two zero bits in "
+   "%s uvlc code: considered invalid due to conflicting "
+   "standard and reference decoder behaviour.\n", name);
+return AVERROR_INVALIDDATA;
 } else {
 if (get_bits_left(gbc) < zeroes) {
 av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid uvlc code at "
--
2.39.2
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/6] avcodec/cbs: Do not assert on traces beyond 255 bits

[Mark Thompson]

On 23/10/2023 21:53, Michael Niedermayer wrote:

On Sun, Oct 22, 2023 at 03:34:20PM +0100, Mark Thompson wrote:

On 22/10/2023 01:35, Michael Niedermayer wrote:

Fixes: Assertion length < 256 failed at libavcodec/cbs.c:517
Fixes: 
62673/clusterfuzz-testcase-minimized-ffmpeg_BSF_TRACE_HEADERS_fuzzer-6490971837431808

Found-by: continuous fuzzing process 
https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer 
---
   libavcodec/cbs.c | 5 +
   1 file changed, 5 insertions(+)

diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c
index cdd7adebebd..2f5d0334a2a 100644
--- a/libavcodec/cbs.c
+++ b/libavcodec/cbs.c
@@ -514,6 +514,11 @@ void ff_cbs_trace_read_log(void *trace_context,
   position = get_bits_count(gbc);
+if (length >= 256) {
+av_log(ctx->log_ctx, ctx->trace_level, "trace of %d bits truncated at 
255\n", length);
+length = 255;
+}
+
   av_assert0(length < 256);
   for (i = 0; i < length; i++)
   bits[i] = get_bits1(gbc) ? '1' : '0';


IMO the assert is sensible (no syntax element is that large) and so this must 
be catching a bug somewhere else.  Please don't nullify the assert to hide the 
bug.

Can you share the input stream which hit this case?


will mail it to you

the backtrce is this:

 #7 0x505748 in ff_cbs_trace_read_log ffmpeg/libavcodec/cbs.c:517:5
 #8 0x5273f1 in cbs_av1_read_uvlc ffmpeg/libavcodec/cbs_av1.c:67:5
 #9 0x5273f1 in cbs_av1_read_timing_info 
ffmpeg/libavcodec/cbs_av1_syntax_template.c:168
 #10 0x5273f1 in cbs_av1_read_sequence_header_obu 
ffmpeg/libavcodec/cbs_av1_syntax_template.c:214
 #11 0x51278a in cbs_av1_read_unit ffmpeg/libavcodec/cbs_av1.c:856:19
 #12 0x4ff30a in cbs_read_fragment_content ffmpeg/libavcodec/cbs.c:209:15
 #13 0x4ff30a in cbs_read_data ffmpeg/libavcodec/cbs.c:276
 #14 0x4edc32 in trace_headers ffmpeg/libavcodec/trace_headers_bsf.c:113:11
 #15 0x4c9898 in LLVMFuzzerTestOneInput 
ffmpeg/tools/target_bsf_fuzzer.c:154:16
 #16 0x136900d in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, 
unsigned long) Fuzzer/build/../FuzzerLoop.cpp:495:13
 #17 0x135dbe2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned 
long) Fuzzer/build/../FuzzerDriver.cpp:273:6
 #18 0x1362de1 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char 
const*, unsigned long)) Fuzzer/build/../FuzzerDriver.cpp:690:9
 #19 0x135d8c0 in main Fuzzer/build/../FuzzerMain.cpp:20:10
 #20 0x7f456b8b8c86 in __libc_start_main 
/build/glibc-CVJwZb/glibc-2.27/csu/../csu/libc-start.c:310
 #21 0x41f179 in _start 
(ffmpeg/tools/target_bsf_trace_headers_fuzzer+0x41f179)


This is the case in 
<https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2023-October/315983.html>, and 
would be fixed by that patch.

Since the problem is a dubious feature of the standard which other 
implementations then do not follow I would appreciate thoughts from other 
people on what to do with it, though.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/2] avutil/hwcontext_vaapi: return ENOSYS for unsupported operation

[Mark Thompson]

On 27/10/2023 16:37, Zhao Zhili wrote:

From: Zhao Zhili 

av_hwframe_transfer_data try with src_ctx first. If the operation
failed with AVERROR(ENOSYS), it will try again with dst_ctx. Return
AVERROR(EINVAL) makes the second step being skipped.
---
  libavutil/hwcontext_vaapi.c | 10 +-
  1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
index 558fed94c6..bb28bcf588 100644
--- a/libavutil/hwcontext_vaapi.c
+++ b/libavutil/hwcontext_vaapi.c
@@ -217,7 +217,7 @@ static int vaapi_get_image_format(AVHWDeviceContext *hwdev,
  return 0;
  }
  }
-return AVERROR(EINVAL);
+return AVERROR(ENOSYS);
  }
  
  static int vaapi_frames_get_constraints(AVHWDeviceContext *hwdev,

@@ -805,19 +805,19 @@ static int vaapi_map_frame(AVHWFramesContext *hwfc,
  
  if (!ctx->derive_works && (flags & AV_HWFRAME_MAP_DIRECT)) {

  // Requested direct mapping but it is not possible.
-return AVERROR(EINVAL);
+return AVERROR(ENOSYS);
  }
  if (dst->format == AV_PIX_FMT_NONE)
  dst->format = hwfc->sw_format;
  if (dst->format != hwfc->sw_format && (flags & AV_HWFRAME_MAP_DIRECT)) {
  // Requested direct mapping but the formats do not match.
-return AVERROR(EINVAL);
+return AVERROR(ENOSYS);


This one seems wrong?  The user requested that (say) a YUV surface is directly 
mapped to an RGB frame.  That's an invalid request from the user, not an 
unsupported feature.

If you're changing the return values then this test and the previous one 
probably want to therefore be in the opposite order as well.


  }
  
  err = vaapi_get_image_format(hwfc->device_ctx, dst->format, &image_format);

  if (err < 0) {
  // Requested format is not a valid output format.
-return AVERROR(EINVAL);
+return err;
  }
  
  map = av_malloc(sizeof(*map));

@@ -992,7 +992,7 @@ static int vaapi_map_to_memory(AVHWFramesContext *hwfc, 
AVFrame *dst,
  if (dst->format != AV_PIX_FMT_NONE) {
  err = vaapi_get_image_format(hwfc->device_ctx, dst->format, NULL);
  if (err < 0)
-return AVERROR(ENOSYS);
+return err;
  }
  
  err = vaapi_map_frame(hwfc, dst, src, flags);


- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3] lavu/hwcontext_vaapi: Use vaMapBuffer2 for mapping image buffers

[Mark Thompson]

On 27/10/2023 11:00, David Rosca wrote:

This allows some optimizations in driver, such as not having to read
back the data if write-only mapping is requested.
---
v3: Fix another warning

  libavutil/hwcontext_vaapi.c | 11 +++
  1 file changed, 11 insertions(+)

diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
index 558fed94c6..86b0852c12 100644
--- a/libavutil/hwcontext_vaapi.c
+++ b/libavutil/hwcontext_vaapi.c
@@ -799,6 +799,9 @@ static int vaapi_map_frame(AVHWFramesContext *hwfc,
  VAStatus vas;
  void *address = NULL;
  int err, i;
+#if VA_CHECK_VERSION(1, 21, 0)
+uint32_t vaflags = 0;
+#endif
  
  surface_id = (VASurfaceID)(uintptr_t)src->data[3];

  av_log(hwfc, AV_LOG_DEBUG, "Map surface %#x.\n", surface_id);
@@ -882,7 +885,15 @@ static int vaapi_map_frame(AVHWFramesContext *hwfc,
  }
  }
  
+#if VA_CHECK_VERSION(1, 21, 0)

+if (flags & AV_HWFRAME_MAP_READ || !(flags & AV_HWFRAME_MAP_OVERWRITE))
+vaflags |= VA_MAPBUFFER_FLAG_READ;


I don't understand where the !overwrite has come from in this condition?

If the user requested write-only but not overwrite then they're expecting to 
write some pixels within the image (such as adding an overlay), but don't want 
to read anything.


+if (flags & AV_HWFRAME_MAP_WRITE)
+vaflags |= VA_MAPBUFFER_FLAG_WRITE;
+vas = vaMapBuffer2(hwctx->display, map->image.buf, &address, vaflags);
+#else
  vas = vaMapBuffer(hwctx->display, map->image.buf, &address);
+#endif
  if (vas != VA_STATUS_SUCCESS) {
  av_log(hwfc, AV_LOG_ERROR, "Failed to map image from surface "
 "%#x: %d (%s).\n", surface_id, vas, vaErrorStr(vas));


Please add a note that there is a compatibility layer in libva so that 
MapBuffer2 calls MapBuffer if the driver doesn't expose it directly, so this 
does work with older drivers.  (The patch looked wrong before I realised that.)

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3] lavu/hwcontext_vaapi: Use vaMapBuffer2 for mapping image buffers

[Mark Thompson]

On 27/10/2023 19:46, David Rosca wrote:

On Fri, Oct 27, 2023 at 7:14 PM Mark Thompson  wrote:

On 27/10/2023 11:00, David Rosca wrote:

This allows some optimizations in driver, such as not having to read
back the data if write-only mapping is requested.
---
v3: Fix another warning

   libavutil/hwcontext_vaapi.c | 11 +++
   1 file changed, 11 insertions(+)

diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
index 558fed94c6..86b0852c12 100644
--- a/libavutil/hwcontext_vaapi.c
+++ b/libavutil/hwcontext_vaapi.c
@@ -799,6 +799,9 @@ static int vaapi_map_frame(AVHWFramesContext *hwfc,
   VAStatus vas;
   void *address = NULL;
   int err, i;
+#if VA_CHECK_VERSION(1, 21, 0)
+uint32_t vaflags = 0;
+#endif

   surface_id = (VASurfaceID)(uintptr_t)src->data[3];
   av_log(hwfc, AV_LOG_DEBUG, "Map surface %#x.\n", surface_id);
@@ -882,7 +885,15 @@ static int vaapi_map_frame(AVHWFramesContext *hwfc,
   }
   }

+#if VA_CHECK_VERSION(1, 21, 0)
+if (flags & AV_HWFRAME_MAP_READ || !(flags & AV_HWFRAME_MAP_OVERWRITE))
+vaflags |= VA_MAPBUFFER_FLAG_READ;


I don't understand where the !overwrite has come from in this condition?


This logic is a couple lines ahead in the vaCreateImage path. If
AV_HWFRAME_MAP_OVERWRITE isn't set, it will call vaGetImage to read
the image data. And as vaDeriveImage + vaMapBuffer is read+write
mapping, I think the same logic needs to be applied to vaMapBuffer2
too.


The case is not the same as the one earlier, because ...



If the user requested write-only but not overwrite then they're expecting to 
write some pixels within the image (such as adding an overlay), but don't want 
to read anything.


Exactly for this case the read is needed. If the user writes only some
(not all) pixels of the image, then the rest of the image will be
invalid if a driver implements the mapping using staging texture
(which is what Mesa does).


... that is not what the documentation says the function has to do:

"""

/**
 * Map data store of the buffer into the client's address space
 * this interface could be used to convey the operation hint
 * backend driver could use these hint to optimize the implementations
 */

/** \brief VA_MAPBUFFER_FLAG_DEFAULT is used when there are no flag specified
 * same as VA_MAPBUFFER_FLAG_READ | VA_MAPBUFFER_FLAG_WRITE.
 */

/** \brief application will read the surface after map */
#define VA_MAPBUFFER_FLAG_READ1
/** \brief application will write the surface after map */
#define VA_MAPBUFFER_FLAG_WRITE   2

VAStatus vaMapBuffer2(
VADisplay dpy,
VABufferID buf_id,  /* in */
void **pbuf,/* out */
uint32_t flags  /* in */
);

"""

There is no suggestion here that setting WRITE & !READ implies that the user 
has to completely overwrite the surface.

A user reading this would reasonably set write-only in the case where they want 
to add an overlay, but your interpretation would make the driver discard the 
rest of the image and give an unexpected result.

If you believe the function is intended to work this way then can you submit a 
patch to libva to update the documentation to say what you expect?  (Which can 
also be seen by other driver implementers, so that they can comment on whether 
the additional meaning you assign to the flags is something they would support 
as well.)

Alternatively, it sounds like you would want to add an OVERWRITE flag to libva, 
for exactly the same reasons that it exists already in ffmpeg.


+if (flags & AV_HWFRAME_MAP_WRITE)
+vaflags |= VA_MAPBUFFER_FLAG_WRITE;
+vas = vaMapBuffer2(hwctx->display, map->image.buf, &address, vaflags);
+#else
   vas = vaMapBuffer(hwctx->display, map->image.buf, &address);
+#endif
   if (vas != VA_STATUS_SUCCESS) {
   av_log(hwfc, AV_LOG_ERROR, "Failed to map image from surface "
  "%#x: %d (%s).\n", surface_id, vas, vaErrorStr(vas));


Please add a note that there is a compatibility layer in libva so that 
MapBuffer2 calls MapBuffer if the driver doesn't expose it directly, so this 
does work with older drivers.  (The patch looked wrong before I realised that.)



Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] cbs_av1: Reject thirty-two zero bits in uvlc code

[Mark Thompson]

On 25/10/2023 21:55, Michael Niedermayer wrote:

On Sun, Oct 22, 2023 at 07:35:52PM +0100, Mark Thompson wrote:

The spec allows at least thirty-two zero bits followed by a one to mean
2^32-1, with no constraint on the number of zeroes.  The libaom
reference decoder does not match this, instead reading thirty-two zeroes
but not the following one to mean 2^32-1.  These two interpretations are
incompatible and other implementations may follow one or the other.
Therefore reject thirty-two zeroes because the intended behaviour is not
clear.
---
libaom, dav1d and SVT-AV1 all have the same nonstandard behaviour of stopping 
at thirty-two zeroes and not reading the one.  gav1 just rejects thirty-two 
zeroes.


I would suggest to contact the authors of the spec to bring this discrepancy
to their attention, unless this is already known and
unless this sequence is declared invalid by some other part of the spec
(which would make the discrepancy only occur in invalid sequences)

<https://github.com/AOMediaCodec/av1-spec/pull/343>

Since this only occurs in nonconforming streams, fixing the spec seems like the 
better option.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] avcodec/amfenc: add smart access video option

[Mark Thompson]

On 23/11/2023 09:41, Evgeny Pavlov wrote:

This commit adds option for enabling SmartAccess Video (SAV)
in AMF encoders. SAV is an AMD hardware-specific feature which
enables the parallelization of encode and decode streams across
multiple Video Codec Engine (VCN) hardware instances.

Signed-off-by: Evgeny Pavlov 
---
  libavcodec/amfenc.h  |  1 +
  libavcodec/amfenc_av1.c  | 18 ++
  libavcodec/amfenc_h264.c | 18 ++
  libavcodec/amfenc_hevc.c | 18 ++
  4 files changed, 55 insertions(+)


Can you explain a bit more about what this option actually does?  I can't find any 
details about it beyond nebulous "make things better", but presumably there is 
some tradeoff so you don't always enable it.

Some documentation explaining what it does and hinting when the user might want 
it on or off would be helpful (can be a separate patch).

Patch itself seems fine for a standalone option, though I would mildly prefer 
not to put meaningless marketing names in the code if it's possible to have a 
descriptive name instead.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] avcodec/amfenc: add smart access video option

[Mark Thompson]

On 26/11/2023 14:40, Dmitrii Ovchinnikov wrote:

The code looks significantly duplicated.


This is not moved to amfenc.c since the property has different names
  for different encoders, and many other properties (also common to
different encoders, but with different names) are separated in this way.


Seems like we could template this to avoid the duplication, something like:

#define PER_CODEC_OPTION(name) \
  (ctx->codec == AV1  ? AMF_VIDEO_ENCODER_AV1_  ## name : \
   ctx->codec == HEVC ? AMF_VIDEO_ENCODER_HEVC_ ## name : \
AMF_VIDEO_ENCODER_  ## name)

?

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v1] lavc/vaapi_encode_av1: Add qp option explicitly to set base q index

[Mark Thompson]

On 27/11/2023 00:58, fei.w.wang-at-intel@ffmpeg.org wrote:

From: Fei Wang 

Keep same way with librav1e/libsvtav1/qsv_av1.. to make it more
acceptable instead of using global option "-global_quality".

Fix #10615

Signed-off-by: Fei Wang 
---
  doc/encoders.texi | 1 +
  libavcodec/vaapi_encode_av1.c | 6 ++
  2 files changed, 7 insertions(+)

diff --git a/doc/encoders.texi b/doc/encoders.texi
index 27a9acf076..2cffc32daf 100644
--- a/doc/encoders.texi
+++ b/doc/encoders.texi
@@ -4079,6 +4079,7 @@ Each encoder also has its own specific options:
  @table @option
  
  @item av1_vaapi

+@option{qp} sets the value of @emph{base_q_index}.
  @option{profile} sets the value of @emph{seq_profile}.
  @option{tier} sets the value of @emph{seq_tier}.
  @option{level} sets the value of @emph{seq_level_idx}.
diff --git a/libavcodec/vaapi_encode_av1.c b/libavcodec/vaapi_encode_av1.c
index 5a9ff0f798..2e327fec5a 100644
--- a/libavcodec/vaapi_encode_av1.c
+++ b/libavcodec/vaapi_encode_av1.c
@@ -79,6 +79,7 @@ typedef struct VAAPIEncodeAV1Context {
  int cdef_param_size;
  
  /** user options */

+int qp;
  int profile;
  int level;
  int tier;
@@ -786,6 +787,9 @@ static av_cold int vaapi_encode_av1_init(AVCodecContext 
*avctx)
  return AVERROR(EINVAL);
  }
  
+if (priv->qp > 0)

+ctx->explicit_qp = priv->qp;
+
  ret = ff_vaapi_encode_init(avctx);
  if (ret < 0)
  return ret;
@@ -864,6 +868,8 @@ static av_cold int vaapi_encode_av1_close(AVCodecContext 
*avctx)
  static const AVOption vaapi_encode_av1_options[] = {
  VAAPI_ENCODE_COMMON_OPTIONS,
  VAAPI_ENCODE_RC_OPTIONS,
+{ "qp", "Base q index (for P-frames; scaled by qfactor/qoffset for I/B)",
+  OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 255, FLAGS },
  { "profile", "Set profile (seq_profile)",
OFFSET(profile), AV_OPT_TYPE_INT,
{ .i64 = AV_PROFILE_UNKNOWN }, AV_PROFILE_UNKNOWN, 0xff, FLAGS, 
"profile" },


Disagree; QP is not a concept in AV1.  Further, your examples from other 
encoders do not have a consistent view of what it should mean.

librav1e.c:

{ "qp", "use constant quantizer mode", OFFSET(quantizer), AV_OPT_TYPE_INT, 
{ .i64 = -1 }, -1, 255, VE },

0-255 is presumably the base_q_idx scale.

libsvtav1.c:

{ "qp", "Initial Quantizer level value", OFFSET(qp),  AV_OPT_TYPE_INT, 
{ .i64 = 0 }, 0, 63, VE },

0-63 is presumably the H.26x-qp-ish scale used by some VP9/AV1 encoders which 
maps nonlinearly to the internal scale.

qsv_av1 doesn't seem to have such an option.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] avcodec/amfenc: increase precision of Sleep() on Windows

[Mark Thompson]

On 13/11/2023 14:37, Evgeny Pavlov wrote:

This commit increase precision of Sleep() function on Windows.
This fix reduces the sleep time on Windows to improve AMF encoding
performance on low resolution input videos.

Fix for issue #10622

v2: use timeBeginPeriod/timeEndPeriod for increasing precision of Sleep()

Signed-off-by: Evgeny Pavlov 
---
  libavcodec/amfenc.c | 31 +++
  libavcodec/amfenc.h |  3 +++
  2 files changed, 34 insertions(+)

diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c
index 061859f85c..55e24856e8 100644
--- a/libavcodec/amfenc.c
+++ b/libavcodec/amfenc.c
@@ -42,7 +42,12 @@
  #endif
  
  #ifdef _WIN32

+#include 
  #include "compat/w32dlfcn.h"
+
+typedef MMRESULT (*timeapi_fun)(UINT uPeriod);
+#define WINMM_DLL "winmm.dll"
+
  #else
  #include 
  #endif
@@ -113,6 +118,9 @@ static int amf_load_library(AVCodecContext *avctx)
  AMFInit_Fn init_fun;
  AMFQueryVersion_Fn version_fun;
  AMF_RESULT res;
+#ifdef _WIN32
+timeapi_fun time_begin_fun;
+#endif
  
  ctx->delayed_frame = av_frame_alloc();

  if (!ctx->delayed_frame) {
@@ -145,6 +153,16 @@ static int amf_load_library(AVCodecContext *avctx)
  AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetTrace() failed 
with error %d\n", res);
  res = ctx->factory->pVtbl->GetDebug(ctx->factory, &ctx->debug);
  AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetDebug() failed 
with error %d\n", res);
+
+#ifdef _WIN32
+// Increase precision of Sleep() function on Windows platform
+ctx->winmm_lib = dlopen(WINMM_DLL, RTLD_NOW | RTLD_LOCAL);
+AMF_RETURN_IF_FALSE(ctx, ctx->winmm_lib != NULL, 0, "DLL %s failed to 
open\n", WINMM_DLL);
+time_begin_fun = (timeapi_fun)dlsym(ctx->winmm_lib, "timeBeginPeriod");
+AMF_RETURN_IF_FALSE(ctx, time_begin_fun != NULL, 0, "DLL %s failed to find function 
%s\n", WINMM_DLL, "timeBeginPeriod");
+time_begin_fun(1);
+#endif //_WIN32
+
  return 0;
  }
  
@@ -375,6 +393,9 @@ static int amf_init_encoder(AVCodecContext *avctx)

  int av_cold ff_amf_encode_close(AVCodecContext *avctx)
  {
  AmfContext *ctx = avctx->priv_data;
+#ifdef _WIN32
+timeapi_fun time_end_fun;
+#endif //_WIN32
  
  if (ctx->delayed_surface) {

  ctx->delayed_surface->pVtbl->Release(ctx->delayed_surface);
@@ -410,6 +431,16 @@ int av_cold ff_amf_encode_close(AVCodecContext *avctx)
  av_frame_free(&ctx->delayed_frame);
  av_fifo_freep2(&ctx->timestamp_list);
  
+#ifdef _WIN32

+if (ctx->winmm_lib) {
+time_end_fun = (timeapi_fun)dlsym(ctx->winmm_lib, "timeEndPeriod");
+AMF_RETURN_IF_FALSE(ctx, time_end_fun != NULL, 0, "DLL %s failed to find function 
%s\n", WINMM_DLL, "timeEndPeriod");
+time_end_fun(1);
+dlclose(ctx->winmm_lib);
+ctx->winmm_lib = NULL;
+}
+#endif //_WIN32
+
  return 0;
  }
  
diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h

index 2dbd378ef8..35bcf1dfe3 100644
--- a/libavcodec/amfenc.h
+++ b/libavcodec/amfenc.h
@@ -50,6 +50,9 @@ typedef struct AmfContext {
  AVClass*avclass;
  // access to AMF runtime
  amf_handle  library; ///< handle to DLL library
+#ifdef _WIN32
+amf_handle  winmm_lib; ///< handle to winmm DLL library
+#endif //_WIN32
  AMFFactory *factory; ///< pointer to AMF factory
  AMFDebug   *debug;   ///< pointer to AMF debug interface
  AMFTrace   *trace;   ///< pointer to AMF trace interface


Is it reasonable to set this global state from a library without the parent 
program knowing?  We'd really prefer not to affect the global state 
unexpectedly.

It's also unclear to me what the effect of this tradeoff on power is, given 
that the whole reason why this happens is that Windows is trying to keep the 
CPU asleep for as long as possible to save power.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] hwcontext_d3d11: Add BGR0 support

[Mark Thompson]

On 22/10/2023 15:54, Mark Thompson wrote:

The 8-bit four-component DXGI container is also used for three-component
RGB without alpha.
---
This list is only used for AV->DXGI mapping, so it doesn't matter that there 
are duplicate DXGI formats in the list.

  libavutil/hwcontext_d3d11va.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c
index cc8c97d2b6..1d249f2088 100644
--- a/libavutil/hwcontext_d3d11va.c
+++ b/libavutil/hwcontext_d3d11va.c
@@ -86,6 +86,7 @@ static const struct {
  } supported_formats[] = {
  { DXGI_FORMAT_NV12, AV_PIX_FMT_NV12 },
  { DXGI_FORMAT_P010, AV_PIX_FMT_P010 },
+    { DXGI_FORMAT_B8G8R8A8_UNORM,    AV_PIX_FMT_BGR0 },
  { DXGI_FORMAT_B8G8R8A8_UNORM,    AV_PIX_FMT_BGRA },
  { DXGI_FORMAT_R10G10B10A2_UNORM, AV_PIX_FMT_X2BGR10 },
  { DXGI_FORMAT_R16G16B16A16_FLOAT, AV_PIX_FMT_RGBAF16 },


Ping for set.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] lavc/cbs_h265: Add sps_multilayer_extension support

[Mark Thompson]

On 17/11/2023 13:03, Jun Zhao wrote:

Add sps_multilayer_extensio support.

Signed-off-by: Jun Zhao 
---
  libavcodec/cbs_h265.h |  3 +++
  libavcodec/cbs_h265_syntax_template.c | 12 +++-
  2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/libavcodec/cbs_h265.h b/libavcodec/cbs_h265.h
index 1b1195f198..15951269fd 100644
--- a/libavcodec/cbs_h265.h
+++ b/libavcodec/cbs_h265.h
@@ -330,6 +330,9 @@ typedef struct H265RawSPS {
  uint8_t persistent_rice_adaptation_enabled_flag;
  uint8_t cabac_bypass_alignment_enabled_flag;
  
+// Multilayer extension.

+uint8_t inter_view_mv_vert_constraint_flag;
+
  // Screen content coding extension.
  uint8_t sps_curr_pic_ref_enabled_flag;
  uint8_t palette_mode_enabled_flag;
diff --git a/libavcodec/cbs_h265_syntax_template.c 
b/libavcodec/cbs_h265_syntax_template.c
index 2d4b954718..1e3bc1acd8 100644
--- a/libavcodec/cbs_h265_syntax_template.c
+++ b/libavcodec/cbs_h265_syntax_template.c
@@ -717,6 +717,16 @@ static int FUNC(sps_range_extension)(CodedBitstreamContext 
*ctx, RWContext *rw,
  return 0;
  }
  
+static int FUNC(sps_multilayer_extension)(CodedBitstreamContext *ctx, RWContext *rw,

+  H265RawSPS *current)
+{
+int err;
+
+flag(inter_view_mv_vert_constraint_flag);
+
+return 0;
+}
+
  static int FUNC(sps_scc_extension)(CodedBitstreamContext *ctx, RWContext *rw,
 H265RawSPS *current)
  {
@@ -952,7 +962,7 @@ static int FUNC(sps)(CodedBitstreamContext *ctx, RWContext 
*rw,
  if (current->sps_range_extension_flag)
  CHECK(FUNC(sps_range_extension)(ctx, rw, current));
  if (current->sps_multilayer_extension_flag)
-return AVERROR_PATCHWELCOME;
+CHECK(FUNC(sps_multilayer_extension)(ctx, rw, current));
  if (current->sps_3d_extension_flag)
  return AVERROR_PATCHWELCOME;
  if (current->sps_scc_extension_flag)


This doesn't seem like it it sufficient - when in multilayer the SPS format 
isn't the same (§F.7.3.2.2.1).

Alternatively: maybe this works for the first layer (haven't verified this but 
seems plausible), and therefore to work there only it would be sufficient to 
check if MultiLayerExtSpsFlag is true in an SPS and stop if it is?

(A sample suitable for using with fate would help.)

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 10 bit support v5 1/3] avcodec/amfenc: Fixes the color information in the output.

[Mark Thompson]

On 31/10/2023 18:57, Evgeny Pavlov wrote:

From: Michael Fabian 'Xaymar' Dirks 

added 10 bit support for amf hevc.

before:

command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format 
d3d11 -i test_10bit_file.mkv -an -c:v h264_amf res.dx11_hw_h264.mkv
output -  Format of input frames context (p010le) is not supported by AMF.
command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format 
d3d11 -i test_10bit_file -an -c:v hevc_amf res.dx11_hw_hevc.mkv
output -  Format of input frames context (p010le) is not supported by AMF.

after:

command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format 
d3d11 -i test_10bit_file -an -c:v h264_amf res.dx11_hw_h264.mkv
output -  10-bit input video is not supported by AMF H264 encoder
command - ffmpeg.exe -hide_banner -y -hwaccel d3d11va -hwaccel_output_format 
d3d11 -i test_10bit_file -an -c:v hevc_amf res.dx11_hw_hevc.mkv
output -  10bit file

v2 - lost line returned in ff_amf_pix_fmts
v3 - fixes after review
v4 - extract duplicated code, fix incorrect processing of 10-bit input for h264
v5 - non-functional changes after review

Co-authored-by: Evgeny Pavlov 
---
  libavcodec/amfenc.c  | 37 +
  libavcodec/amfenc.h  |  3 +++
  libavcodec/amfenc_h264.c | 24 
  libavcodec/amfenc_hevc.c | 26 +-
  4 files changed, 85 insertions(+), 5 deletions(-)


There is something very wrong with how the header information is working here.

With this series applied, I ran:

ffmpeg_g.exe -report -y -i in.mp4 -an -c:v hevc_amf -bsf:v trace_headers 
-frames:v 1 out.mp4

My input file is:

  Stream #0:0[0x1](und), 60, 1/6: Video: hevc (Main 10) (hvc1 / 
0x31637668), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1 DAR 
16:9], 74462 kb/s, 59.94 fps, 59.94 tbr, 60k tbn (default)

[trace_headers @ 023184c753c0] Extradata
[trace_headers @ 023184c753c0] Sequence Parameter Set
...
[trace_headers @ 023184c753c0] 222 vui_parameters_present_flag  
   0 = 0

So no colour information at all in the headers, and the output file indeed says:

  Stream #0:0[0x1](und): Video: hevc (Main 10) (hev1 / 0x31766568), 
yuv420p10le(tv, progressive), 3840x2160, 977 kb/s, SAR 1:1 DAR 16:9, 59.94 fps, 
59.94 tbr, 60k tbn (default)

However!  Reading further:

[trace_headers @ 023184c753c0] Packet: 2039 bytes, key frame, pts 0, dts 0.
...
[trace_headers @ 023184c753c0] Sequence Parameter Set
...
[trace_headers @ 023184c753c0] 222 vui_parameters_present_flag  
   1 = 1
[trace_headers @ 023184c753c0] 223 aspect_ratio_info_present_flag   
   1 = 1
[trace_headers @ 023184c753c0] 224 aspect_ratio_idc 
 = 255
[trace_headers @ 023184c753c0] 232 sar_width
0001 = 1
[trace_headers @ 023184c753c0] 248 sar_height   
0001 = 1
[trace_headers @ 023184c753c0] 264 overscan_info_present_flag   
   0 = 0
[trace_headers @ 023184c753c0] 265 video_signal_type_present_flag   
   1 = 1
[trace_headers @ 023184c753c0] 266 video_format 
 101 = 5
[trace_headers @ 023184c753c0] 269 video_full_range_flag
   0 = 0
[trace_headers @ 023184c753c0] 270 colour_description_present_flag  
   1 = 1
[trace_headers @ 023184c753c0] 271 colour_primaries 
1001 = 9
[trace_headers @ 023184c753c0] 279 transfer_characteristics 
0001 = 16
[trace_headers @ 023184c753c0] 287 matrix_coefficients  
1001 = 9
[trace_headers @ 023184c753c0] 295 chroma_loc_info_present_flag 
   0 = 0
[trace_headers @ 023184c753c0] 296 neutral_chroma_indication_flag   
   0 = 0
[trace_headers @ 023184c753c0] 297 field_seq_flag   
   0 = 0
[trace_headers @ 023184c753c0] 298 frame_field_info_present_flag
   0 = 0
[trace_headers @ 023184c753c0] 299 default_display_window_flag  
   0 = 0
[trace_headers @ 023184c753c0] 300 vui_timing_info_present_flag 
   1 = 1
[trace_headers @ 023184c753c0] 301 vui_num_units_in_tick
00101001 = 1001
[trace_headers @ 023184c753c0] 333 vui_time_scale   
111010100110 = 6
[trace_headers @ 023184c753c0] 365 
vui_p

Re: [FFmpeg-devel] [PATCH 10 bit support v5 1/3] avcodec/amfenc: Fixes the color information in the output.

[Mark Thompson]

On 28/11/2023 14:39, Evgeny Pavlov wrote:

On Mon, Nov 27, 2023 at 8:47 PM Mark Thompson  wrote:


There is something very wrong with how the header information is working
here.

With this series applied, I ran:

ffmpeg_g.exe -report -y -i in.mp4 -an -c:v hevc_amf -bsf:v trace_headers
-frames:v 1 out.mp4

My input file is:

Stream #0:0[0x1](und), 60, 1/6: Video: hevc (Main 10) (hvc1 /
0x31637668), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1
DAR 16:9], 74462 kb/s, 59.94 fps, 59.94 tbr, 60k tbn (default)

[trace_headers @ 023184c753c0] Extradata
[trace_headers @ 023184c753c0] Sequence Parameter Set
...
[trace_headers @ 023184c753c0] 222
  vui_parameters_present_flag 0 = 0

So no colour information at all in the headers, and the output file indeed
says:

Stream #0:0[0x1](und): Video: hevc (Main 10) (hev1 / 0x31766568),
yuv420p10le(tv, progressive), 3840x2160, 977 kb/s, SAR 1:1 DAR 16:9, 59.94
fps, 59.94 tbr, 60k tbn (default)

However!  Reading further:

[trace_headers @ 023184c753c0] Packet: 2039 bytes, key frame, pts 0,
dts 0.
...
[trace_headers @ 023184c753c0] Sequence Parameter Set
...
[trace_headers @ 023184c753c0] 222
  vui_parameters_present_flag 1 = 1
[trace_headers @ 023184c753c0] 223
  aspect_ratio_info_present_flag  1 = 1
[trace_headers @ 023184c753c0] 224 aspect_ratio_idc
   = 255
[trace_headers @ 023184c753c0] 232 sar_width
   0001 = 1
[trace_headers @ 023184c753c0] 248 sar_height
  0001 = 1
[trace_headers @ 023184c753c0] 264 overscan_info_present_flag
 0 = 0
[trace_headers @ 023184c753c0] 265
  video_signal_type_present_flag  1 = 1
[trace_headers @ 023184c753c0] 266 video_format
   101 = 5
[trace_headers @ 023184c753c0] 269 video_full_range_flag
  0 = 0
[trace_headers @ 023184c753c0] 270
  colour_description_present_flag 1 = 1
[trace_headers @ 023184c753c0] 271 colour_primaries
  1001 = 9
[trace_headers @ 023184c753c0] 279 transfer_characteristics
  0001 = 16
[trace_headers @ 023184c753c0] 287 matrix_coefficients
   1001 = 9
[trace_headers @ 023184c753c0] 295
  chroma_loc_info_present_flag0 = 0
[trace_headers @ 023184c753c0] 296
  neutral_chroma_indication_flag  0 = 0
[trace_headers @ 023184c753c0] 297 field_seq_flag
 0 = 0
[trace_headers @ 023184c753c0] 298
  frame_field_info_present_flag   0 = 0
[trace_headers @ 023184c753c0] 299
  default_display_window_flag 0 = 0
[trace_headers @ 023184c753c0] 300
  vui_timing_info_present_flag1 = 1
[trace_headers @ 023184c753c0] 301 vui_num_units_in_tick
   00101001 = 1001
[trace_headers @ 023184c753c0] 333 vui_time_scale
  111010100110 = 6
[trace_headers @ 023184c753c0] 365
  vui_poc_proportional_to_timing_flag 1 = 1
[trace_headers @ 023184c753c0] 366
  vui_num_ticks_poc_diff_one_minus1   1 = 0

Comparing the to the original, the chroma sample location (collocated
top-left in the original, so the implied default is wrong) has been lost
but the colours are otherwise correct in the extraneous headers embedded in
the first packet.

So the colour information has kindof been passed through, except not in
the place in the headers which matters so it is mostly useless.  (I guess
it maybe works for raw streams with no headers?)

I think you need to fix whatever is making the headers not match the
actual stream content (which creates invalid files, mp4 and similar
containers with global headers need them to match).

Thanks,

- Mark



Could you test this issue with the latest AMD 23.11.1 driver? This issue
looks similar to issue #9195 in OBS Studio
https://github.com/obsproject/obs-studio/issues/9195. It was fixed in the
latest AMD driver.


I upgraded to 23.11.1 and see no change - the colour information is still 
missing in the header but not the stream, and the two different sequence 
parameter sets are identical to what they were before the change.

Can you share what your trace_headers output looks like for the out-of-band and 
in-band parameter sets?  Are they identical for you?  Mine below.

Thanks,

- Mark


[trace_headers @ 023184c753c0] Extradata

[trace_headers @ 023184c753c0] Sequence Parameter Set

[trace_headers

Re: [FFmpeg-devel] [PATCH v2] doc/bitstream_filters: add filter_units practical examples for removing closed captions

[Mark Thompson]

On 11/12/2023 16:33, Marth64 wrote:

Added v2 label and signed off as per proper procedure.

Signed-off-by: Marth64 
---
  doc/bitstream_filters.texi | 18 ++
  1 file changed, 18 insertions(+)

diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi
index dc4f85bac0..7c36c5346d 100644
--- a/doc/bitstream_filters.texi
+++ b/doc/bitstream_filters.texi
@@ -213,6 +213,24 @@ To remove all AUDs, SEI and filler from an H.265 stream:
  ffmpeg -i INPUT -c:v copy -bsf:v 'filter_units=remove_types=35|38-40' OUTPUT
  @end example
  
+SEI messages are commonly used to contain Closed Captions as well as other metadata such as dynamic HDR.

+This filter can be used to remove the SEI messages if desired.
+
+To remove SEI messages (including CC) from a MPEG-2 stream:
+@example
+ffmpeg -i INPUT -c:v copy -bsf:v 'filter_units=remove_types=178' OUTPUT
+@end example


MPEG-2 doesn't call these SEI, it is simply "user data".


+
+To remove SEI messages (including CC) from a H264 stream:
+@example
+ffmpeg -i INPUT -c:v copy -bsf:v 'filter_units=remove_types=6' OUTPUT
+@end example
+
+To remove SEI messages (including CC and dynamic HDR) from a HEVC stream:
+@example
+ffmpeg -i INPUT -c:v copy -bsf:v 'filter_units=remove_types=39' OUTPUT
+@end example


Are closed captions allowed in suffix SEI as well?  T.35 messages are certainly 
allowed there.

(I'm mildly inclined to think that a more specific bitstream filter able to 
mess with the SEI types would be more useful?  This approach feels like it has 
a lot of collateral damage.)

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 7/7] avcodec: add AV_CODEC_FLAG_CLEAR

[Mark Thompson]

On 07/12/2023 23:11, Marton Balint wrote:

On Thu, 7 Dec 2023, Anton Khirnov wrote:

Quoting Marton Balint (2023-12-06 09:22:20)

Signed-off-by: Marton Balint 
---
 doc/APIchanges |  3 +++
 doc/codecs.texi    | 14 ++
 libavcodec/avcodec.h   |  4 
 libavcodec/decode.c    |  6 ++
 libavcodec/options_table.h |  1 +
 libavcodec/version.h   |  2 +-
 6 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/doc/APIchanges b/doc/APIchanges
index 416e2bec5e..f839504a64 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -2,6 +2,9 @@ The last version increases of all libraries were on 2023-02-09

 API changes, most recent first:

+2023-12-xx - xxx - lavc 60.36.100 - avcodec.h
+  Add AV_CODEC_FLAG_CLEAR.


I'm not a huge fan of calling it just 'clear'. How about something more
descriptive like 'wipe_frames'.


Wipe reminds me of the wipe effect. How about 'predecode_clear'?


+
 2023-12-xx - xxx - lavu 58.33.100 - imgutils.h
   Add av_image_fill_color()

diff --git a/doc/codecs.texi b/doc/codecs.texi
index 5b950b4560..0504a535f2 100644
--- a/doc/codecs.texi
+++ b/doc/codecs.texi
@@ -76,6 +76,20 @@ Apply interlaced motion estimation.
 Use closed gop.
 @item output_corrupt
 Output even potentially corrupted frames.
+@item clear
+Clear the contents of the video buffer before decoding the next picture to it.
+
+Usually if only a part of a picture is affected by a decode error then the
+decoder (if it implements error concealment) tries to hide it by interpolating
+pixels from neighbouring areas or in some cases from the previous frame. Even
+without error concealment it is quite likely that the affected area will
+contain pixels from an earlier frame, due to frame pooling.
+
+For quality checking this might not be desirable, because it makes the errors
+less noticable. By using this flag, and combining it with disabled error
+concealment (@code{-ec 0}) it is possible to ensure that no leftover data from
+an earlier frame is presented in areas affected by decode errors.
+
 @end table

 @item time_base @var{rational number}
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 7fb44e28f4..97848e942f 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -312,6 +312,10 @@ typedef struct RcOverride{
  * loop filter.
  */
 #define AV_CODEC_FLAG_LOOP_FILTER (1 << 11)
+/**
+ * Clear frame buffer contents before decoding.


Clear the contents of each frame buffer after it is allocated with
AVCodecContext.get_buffer2() and before anything is decoded into it.

Note that this may have a significant performance cost.


ok.




+ */
+#define AV_CODEC_FLAG_CLEAR   (1 << 12)
 /**
  * Only decode/encode grayscale.
  */
diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index 2cfb3fcf97..f9b18a2c35 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -1675,6 +1675,12 @@ FF_ENABLE_DEPRECATION_WARNINGS

 validate_avframe_allocation(avctx, frame);

+    if (avctx->flags & AV_CODEC_FLAG_CLEAR && avctx->codec_type == 
AVMEDIA_TYPE_VIDEO) {
+    uint32_t color[4] = {0};
+    ptrdiff_t linesize[4] = {frame->linesize[0], frame->linesize[1], 
frame->linesize[2], frame->linesize[3]};
+    av_image_fill_color(frame->data, linesize, frame->format, color, 
frame->width, frame->height);


Should this check for errors?


Lack of error checking is intentional. av_image_fill_color might not support 
all pixel formats, definitely not support hwaccel formats. It might make sense 
to warn the user once, but I don't think propagating the error back is needed 
here.


I don't think I agree with this?  Even if you say it isn't, it still looks like 
a privacy and security feature and so people may treat it as such.

Consider a transcoding application with user-supplied ingest - without 
something like this, a malformed input from the user could leak random 
previously-deallocated memory, which could contain anything (frames from other 
users, private keys, etc.).

So, if the user has explicitly requested that frames are cleared then IMO it 
should either clear them or fail.

(I do think the feature is a good idea.  Supporting hardware formats there is 
probably straightforward in at least some cases if useful.)

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 10 bit support v5 1/3] avcodec/amfenc: Fixes the color information in the output.

[Mark Thompson]

On 29/11/2023 10:57, Evgeny Pavlov wrote:

On Tue, Nov 28, 2023 at 8:13 PM Mark Thompson  wrote:


I upgraded to 23.11.1 and see no change - the colour information is still
missing in the header but not the stream, and the two different sequence
parameter sets are identical to what they were before the change.

Can you share what your trace_headers output looks like for the
out-of-band and in-band parameter sets?  Are they identical for you?


Yes, it seems that they are identical for me and both have colour
information (please find my output below).
Is it possible to provide a video you tested? Probably I need to
test the patch on your video input.


Not the same, but here is a freely-available test video which has the same effect: 
<https://4kmedia.org/lg-new-york-hdr-uhd-4k-demo/>.  Output below.

It doesn't seem like this should be dependent on the underlying hardware since 
(I hope) it won't go to the hardware for header information.  Still, just in 
case: it's a 5850U APU running driver version 23.11.1 on Windows 10.

(Seems like it might be a good idea for the debug output to print the driver 
and hardware versions at least?)

The VPS out-of-order is also a weird effect - it makes it look like the two 
headers are generated by separate processes, which doesn't seem like a good 
idea when you want them to be identical.

Thanks,

- Mark


ffmpeg started on 2023-12-11 at 21:08:30

Report written to "ffmpeg-20231211-210830.log"

Log level: 48

Command line:

ffmpeg_g.exe -report -y -threads 1 -i in.ts -an -c:v hevc_amf -bsf:v 
trace_headers -frames:v 1 out.mp4

ffmpeg version N-112951-g8c718b936a Copyright (c) 2000-2023 the FFmpeg 
developers

  built with gcc 13.2.0 (Rev3, Built by MSYS2 project)

  configuration: --assert-level=2 --enable-debug --disable-optimizations 
--enable-gpl --enable-libx264 --enable-libx265 --enable-libaom --enable-opencl 
--enable-amf --enable-dxva2 --enable-d3d11va

  libavutil  58. 32.100 / 58. 32.100

  libavcodec 60. 35.100 / 60. 35.100

  libavformat60. 18.100 / 60. 18.100

  libavdevice60.  4.100 / 60.  4.100

  libavfilter 9. 14.100 /  9. 14.100

  libswscale  7.  6.100 /  7.  6.100

  libswresample   4. 13.100 /  4. 13.100

  libpostproc57.  4.100 / 57.  4.100

Splitting the commandline.

Reading option '-report' ... matched as option 'report' (generate a report) 
with argument '1'.

Reading option '-y' ... matched as option 'y' (overwrite output files) with 
argument '1'.

Reading option '-threads' ... matched as AVOption 'threads' with argument '1'.

Reading option '-i' ... matched as output url with argument 'in.ts'.

Reading option '-an' ... matched as option 'an' (disable audio) with argument 
'1'.

Reading option '-c:v' ... matched as option 'c' (codec name) with argument 
'hevc_amf'.

Reading option '-bsf:v' ... matched as option 'bsf' (A comma-separated list of 
bitstream filters) with argument 'trace_headers'.

Reading option '-frames:v' ... matched as option 'frames' (set the number of 
frames to output) with argument '1'.

Reading option 'out.mp4' ... matched as output url.

Finished splitting the commandline.

Parsing a group of options: global .

Applying option report (generate a report) with argument 1.

Applying option y (overwrite output files) with argument 1.

Successfully parsed a group of options.

Parsing a group of options: input url in.ts.

Successfully parsed a group of options.

Opening an input file: in.ts.

[AVFormatContext @ 018927c73700] Opening 'in.ts' for reading

[file @ 018927c2da40] Setting default whitelist 'file,crypto,data'

[mpegts @ 018927c73700] Format mpegts probed with size=2048 and score=50

[mpegts @ 018927c73700] stream=0 stream_type=24 pid=101 prog_reg_desc=

[mpegts @ 018927c73700] stream=1 stream_type=f pid=102 prog_reg_desc=

[mpegts @ 018927c73700] Before avformat_find_stream_info() pos: 0 bytes 
read:32768 seeks:0 nb_streams:2

[hevc @ 018927c76fc0] nal_unit_type: 35(AUD), nuh_layer_id: 0, temporal_id: 0

[hevc @ 018927c76fc0] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0

[hevc @ 018927c76fc0] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0

[hevc @ 018927c76fc0] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0

[hevc @ 018927c76fc0] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, 
temporal_id: 0

[hevc @ 018927c76fc0] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, 
temporal_id: 0

[hevc @ 018927c76fc0] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, 
temporal_id: 0

[hevc @ 018927c76fc0] nal_unit_type: 20(IDR_N_LP), nuh_layer_id: 0, 
temporal_id: 0

[hevc @ 018927c76fc0] nal_unit_type: 38(FD_NUT), nuh_layer

Re: [FFmpeg-devel] [PATCH] Improved the performance of 1 decode + N filter graphs and adaptive bitrate.

[Mark Thompson]

On 26/03/2019 22:07, Shaofei Wang wrote:
> It enabled MULTIPLE SIMPLE filter graph concurrency, which bring above about
> 4%~20% improvement in some 1:N scenarios by CPU or GPU acceleration
> 
> ...
> ---
> The patch will only effect on multiple SIMPLE filter graphs pipeline,
> Passed fate and refine the possible data race,
> AFL tested, without introducing extra crashs/hangs
> 
>  fftools/ffmpeg.c | 172 
> +--
>  fftools/ffmpeg.h |  13 +
>  2 files changed, 169 insertions(+), 16 deletions(-)

On further thought, I don't think this patch should continue in its present 
form.

The fundamental problem is that the bolted-on parallelism renders it pretty 
much unreviewable - the codebase here was not designed with any parallelism in 
mind, so it does not follow the usual rules expected of such code.  In 
particular, there are global variables in many places accessed without regard 
for data races, and that becomes fatal undefined behaviour once parallelism is 
added.  While in theory every one of those can be fixed (by adding more locks, 
or one big lock covering many instances together), it will be very hard to 
verify that all cases have been covered and the code will suffer significantly 
in readability/maintainability for the change.

To give an explicit example of the sort of problems you are hitting, the 
-benchmark_all option is entirely broken in the current proposed version.  It 
invokes undefined behaviour by writing to the current_time global in every 
thread.  Even if that were avoided by moving the value to a thread-local 
structure, it also gives wrong results - getrusage(RUSAGE_SELF) returns values 
for the whole process, so it won't say anything useful once there are multiple 
threads looking at the value in parallel.  Perhaps that is fixable by using 
some sort of per-thread storage and RUSAGE_THREAD (and some equivalent on 
Windows), but whatever that is certainly requires more investigation and 
probably a separate patch.

Three possible thoughts on where to go from here:

* Start by refactoring the code to make it easier to verify.  This would entail 
something like removing all global variable accesses from parallel paths, and 
then ensuring that each thread only ever writes to its own local working set 
(e.g. OutputStream structure) while marking all other structures as const.  
After such refactoring has happened, a new version of the present patch would 
then be possible to assess.

* Run exhaustively in tsan/valgrind/other race detectors and fix every problem 
it finds, then provide evidence from that to help with review.  Since these 
issues can be hidden behind specific option combinations (as the benchmark one 
is), this would need a lot of care to ensure full coverage, and some parts 
(like the benchmark one) would need rewriting anyway.  As noted above I'm not 
sure this would be very good for the code, but we could at least have some 
amount of confidence that the result was correct.

* Reconsider whether the patch is worth pursuing.  While I agree that a correct 
implementation of this would help in the specific use-cases you describe, I'm 
not sure that the set of users who gain from it is actually particularly large 
- most people wanting to maximise throughput in the way this change can help 
with already run multiple FFmpeg instances (or their own programs using the 
libraries) because the serial limitations of FFmpeg are well-known.


- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH] configure: Do not enable both OpenCL-VAAPI interop modes simultaneously

[Mark Thompson]

Beignet offers a far more flexiable and complete interface, so choose it
by default if available.
---
On 23/03/2019 12:27, Mark Thompson wrote:
> On 22/03/2019 01:40, Ruiling Song wrote:
>> ffmpeg | branch: master | Ruiling Song  | Fri Nov 23 
>> 13:39:12 2018 +0800| [61cb505d18b8a335bd118d88c05b9daf40eb5f9b] | committer: 
>> Ruiling Song
>>
>> lavu/opencl: replace va_ext.h with standard name
>>
>> Khronos OpenCL header (https://github.com/KhronosGroup/OpenCL-Headers)
>> uses cl_va_api_media_sharing_intel.h. And Intel's official OpenCL driver
>> for Intel GPU (https://github.com/intel/compute-runtime) was compiled
>> against Khronos OpenCL header. So it's better to align with Khronos.
>>
>> Signed-off-by: Ruiling Song 
>>
>>> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=61cb505d18b8a335bd118d88c05b9daf40eb5f9b
>> ---
>>
>>  configure| 2 +-
>>  libavutil/hwcontext_opencl.c | 2 +-
>>  2 files changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/configure b/configure
>> index a817479559..331393f8d5 100755
>> --- a/configure
>> +++ b/configure
>> @@ -6472,7 +6472,7 @@ fi
>>  
>>  if enabled_all opencl vaapi ; then
>>  enabled opencl_drm_beignet && enable opencl_vaapi_beignet
>> -check_type "CL/cl.h CL/va_ext.h" 
>> "clCreateFromVA_APIMediaSurfaceINTEL_fn" &&
>> +check_type "CL/cl.h CL/cl_va_api_media_sharing_intel.h" 
>> "clCreateFromVA_APIMediaSurfaceINTEL_fn" &&
>>  enable opencl_vaapi_intel_media
>>  fi
>>  
>> diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c
>> index d3df6221c4..b116c5b708 100644
>> --- a/libavutil/hwcontext_opencl.c
>> +++ b/libavutil/hwcontext_opencl.c
>> @@ -50,7 +50,7 @@
>>  #include 
>>  #endif
>>  #include 
>> -#include 
>> +#include 
>>  #include "hwcontext_vaapi.h"
>>  #endif
>>  
> 
> This broke the build when both are available.
> 
> $ make
> CC  libavutil/hwcontext_opencl.o
> src/libavutil/hwcontext_opencl.c: In function ‘opencl_device_derive’:
> src/libavutil/hwcontext_opencl.c:1236:5: error: duplicate case value
>  case AV_HWDEVICE_TYPE_VAAPI:
>  ^~~~
> src/libavutil/hwcontext_opencl.c:1205:5: note: previously used here
>  case AV_HWDEVICE_TYPE_VAAPI:
>  ^~~~
> src/libavutil/hwcontext_opencl.c: In function ‘opencl_map_to’:
> src/libavutil/hwcontext_opencl.c:2831:5: error: duplicate case value
>  case AV_PIX_FMT_VAAPI:
>  ^~~~
> src/libavutil/hwcontext_opencl.c:2825:5: note: previously used here
>  case AV_PIX_FMT_VAAPI:
>  ^~~~
> src/libavutil/hwcontext_opencl.c: In function ‘opencl_frames_derive_to’:
> src/libavutil/hwcontext_opencl.c:2873:5: error: duplicate case value
>  case AV_HWDEVICE_TYPE_VAAPI:
>  ^~~~
> src/libavutil/hwcontext_opencl.c:2866:5: note: previously used here
>  case AV_HWDEVICE_TYPE_VAAPI:
>  ^~~~
> make: *** [ffbuild/common.mak:60: libavutil/hwcontext_opencl.o] Error 1
> make: Target 'all' not remade because of errors.
> 
> $ cat config.h | grep HAVE_OPENCL
> #define HAVE_OPENCL_D3D11 0
> #define HAVE_OPENCL_DRM_ARM 0
> #define HAVE_OPENCL_DRM_BEIGNET 1
> #define HAVE_OPENCL_DXVA2 0
> #define HAVE_OPENCL_VAAPI_BEIGNET 1
> #define HAVE_OPENCL_VAAPI_INTEL_MEDIA 1
> 
> 
> I think in general the Beignet mapping is more useful if present since it has 
> far fewer constraints, so perhaps disable this one if Beignet is there?

 configure | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/configure b/configure
index 331393f8d5..c94f516224 100755
--- a/configure
+++ b/configure
@@ -6471,9 +6471,12 @@ if enabled_all opencl libdrm ; then
 fi
 
 if enabled_all opencl vaapi ; then
-enabled opencl_drm_beignet && enable opencl_vaapi_beignet
-check_type "CL/cl.h CL/cl_va_api_media_sharing_intel.h" 
"clCreateFromVA_APIMediaSurfaceINTEL_fn" &&
-enable opencl_vaapi_intel_media
+if enabled opencl_drm_beignet ; then
+enable opencl_vaapi_beignet
+else
+check_type "CL/cl.h CL/cl_va_api_media_sharing_intel.h" 
"clCreateFromVA_APIMediaSurfaceINTEL_fn" &&
+enable opencl_vaapi_intel_media
+fi
 fi
 
 if enabled_all opencl dxva2 ; then
-- 
2.19.2
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3 1/1] lavc/h264_levels: add MaxMBPS checking and update fate test.

[Mark Thompson]

On 20/03/2019 09:14, Decai Lin wrote:
> 1. add MaxMBPS checking for level idc setting to align with AVC spec
>AnnexA table A-1/A-6 level limits.
> 2. update h264 level fate test.
> 
> Signed-off-by: Decai Lin 
> ---
>  libavcodec/h264_levels.c   |  4 +++
>  libavcodec/h264_levels.h   |  1 +
>  libavcodec/h264_metadata_bsf.c |  9 ++-
>  libavcodec/tests/h264_levels.c | 58 
> +++---
>  libavcodec/vaapi_encode_h264.c |  7 +
>  5 files changed, 75 insertions(+), 4 deletions(-)
> 
> ...
> diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c
> index a17987a..0f6ce26 100644
> --- a/libavcodec/h264_metadata_bsf.c
> +++ b/libavcodec/h264_metadata_bsf.c
> @@ -223,6 +223,7 @@ static int h264_metadata_update_sps(AVBSFContext *bsf,
>  const H264LevelDescriptor *desc;
>  int64_t bit_rate;
>  int width, height, dpb_frames;
> +int framerate;
>  
>  if (sps->vui.nal_hrd_parameters_present_flag) {
>  bit_rate = 
> (sps->vui.nal_hrd_parameters.bit_rate_value_minus1[0] + 1) *
> @@ -244,7 +245,13 @@ static int h264_metadata_update_sps(AVBSFContext *bsf,
>  height = 16 * (sps->pic_height_in_map_units_minus1 + 1) *
>  (2 - sps->frame_mbs_only_flag);
>  
> -desc = ff_h264_guess_level(sps->profile_idc, bit_rate,
> +if (sps->vui.timing_info_present_flag) {
> +framerate = sps->vui.time_scale / sps->vui.num_units_in_tick;

This calculated number is the fieldrate, not the framerate.  I fixed that by 
adding the missing factor 2.

> +} else {
> +framerate = 0;
> +}
> +
> +desc = ff_h264_guess_level(sps->profile_idc, bit_rate, framerate,
> width, height, dpb_frames);
>  if (desc) {
>  level_idc = desc->level_idc;

LGTM with that, and applied.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v9 1/2] lavc/svt_hevc: add libsvt hevc encoder wrapper

[Mark Thompson]

On 27/03/2019 17:13, Vittorio Giovara wrote:
> On Tue, Mar 26, 2019 at 10:47 PM Jing Sun  wrote:
> 
>> Signed-off-by: Zhengxu Huang 
>> Signed-off-by: Hassene Tmar 
>> Signed-off-by: Jun Zhao 
>> Signed-off-by: Jing Sun 
>> ---
>>  configure|   4 +
>>  libavcodec/Makefile  |   1 +
>>  libavcodec/allcodecs.c   |   1 +
>>  libavcodec/libsvt_hevc.c | 502
>> +++
>>  4 files changed, 508 insertions(+)
>>  create mode 100644 libavcodec/libsvt_hevc.c
>>
>> ...
>> +if (svt_enc->hdr) {
>> +svt_enc->vui_info = 1;
>> +param->highDynamicRangeInput = svt_enc->hdr;
>> +}
>>
> 
> Where is the warning that notifies the lack of color properties support?

Looking at what the highDynamicRangeInput field actually does 
,
 I don't think it makes sense for this external "hdr" option at exist at all.

From the point of view of a user looking at the external options, they might 
expect that on setting this option some conversion takes place to actually 
create an HDR output.  In fact, that's not what it does - it just marks the 
output with some very specific colour properties, and any stream which doesn't 
actually have exactly those properties will then have incorrect metadata for 
display (possibly conflicting with container metadata, if the container has 
better support for colour properties than this encoder).

Perhaps to avoid confusion about what is actually happening the option could be 
removed and this check replaced with something like:

if (avctx->colorspace == AVCOL_SPC_BT2020_NCL &&
avctx->color_primaries == AVCOL_PRI_BT2020 &&
avctx->color_trc == AVCOL_TRC_SMPTE2084 &&
avctx->color_range == AVCOL_RANGE_MPEG &&
avctx->chroma_sample_location == AVCHROMA_LOC_TOPLEFT) {
param->highDynamicRangeInput = 1;
} else {
param->highDynamicRangeInput = 0;
// Maybe also a warning here in some cases?
}

That would then do the right thing for all streams which actually have the 
given properties, while not forcing incorrect display of anything else.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/4] lavc/qsvenc: fix hevc vps extradata issues

[Mark Thompson]

On 26/03/2019 19:46, Zhong Li wrote:
> cbs trace qsv vps header failed due to some reasons:
> 1. vps_temporal_id_nesting_flag is not set but spec required it must to
>be 1 when vps_max_sub_layers_minus1 is equal to 0.
> 2. vps_num_hrd_parameters is not set and written.
> 3. other issues in ff_hevc_encode_nal_vps() (have fixed in pervious commit).
> 
> Reproduce: ffmpeg -hwaccel qsv -v verbose -c:v h264_qsv -i 
> bbb_sunflower_1080p_30fps_normal.mp4 -vframes 1
> -c:v hevc_qsv  -bsf:v trace_headers -f null -
> 
> Signed-off-by: Zhong Li 
> ---
>  libavcodec/qsvenc_hevc.c | 16 +---
>  1 file changed, 9 insertions(+), 7 deletions(-)

I had a patch sitting around for a long time for this - see 
.  It 
just deletes all of the ad-hoc writing code and uses the tested CBS paths 
instead, which have the additional advantage of correctly preserving various 
things which the existing code doesn't cover at all (e.g. newer PTL flags which 
didn't exist when this was written).

The main problem with it was that I didn't have enough libmfx platforms at the 
time to be confident in testing of it, so it got stalled.

Would it be useful to resurrect that?

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH]lavc/vaap_hevc: Do not initialize fields twice

[Mark Thompson]

On 27/03/2019 13:58, Carl Eugen Hoyos wrote:
> Hi!
> 
> Attached patch fixes a warning when compiling vaapi with clang.
> 
> From 6463a3cf5730be9e9e6003f4aaf6c9fab7f68407 Mon Sep 17 00:00:00 2001
> From: Carl Eugen Hoyos 
> Date: Wed, 27 Mar 2019 14:52:35 +0100
> Subject: [PATCH] lavc/vaapi_hevc: Do not initialize fields twice.
> 
> Fixes the following compilation warnings:
> libavcodec/vaapi_hevc.c:155:21: warning: initializer overrides prior 
> initialization of this subobject [-Winitializer-overrides]
> .pic_fields.bits = {
>~^~~~
> libavcodec/vaapi_hevc.c:125:57: note: previous initialization is here
> .pic_fields.value = 0,
> ^
> libavcodec/vaapi_hevc.c:175:31: warning: initializer overrides prior 
> initialization of this subobject [-Winitializer-overrides]
> .slice_parsing_fields.bits = {
>  ~^~~~
> libavcodec/vaapi_hevc.c:126:57: note: previous initialization is here
> .slice_parsing_fields.value   = 0,
> ---
>  libavcodec/vaapi_hevc.c |2 --
>  1 file changed, 2 deletions(-)
> 
> diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c
> index 19aabcd..c69d63d 100644
> --- a/libavcodec/vaapi_hevc.c
> +++ b/libavcodec/vaapi_hevc.c
> @@ -122,8 +122,6 @@ static int vaapi_hevc_start_frame(AVCodecContext  
> *avctx,
>  pic->pic.output_surface = ff_vaapi_get_surface_id(h->ref->frame);
>  
>  pic->pic_param = (VAPictureParameterBufferHEVC) {
> -.pic_fields.value = 0,
> -.slice_parsing_fields.value   = 0,
>  .pic_width_in_luma_samples= sps->width,
>  .pic_height_in_luma_samples   = sps->height,
>  .log2_min_luma_coding_block_size_minus3   = 
> sps->log2_min_cb_size - 3,
> -- 
> 1.7.10.4

LGTM.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH]lavc/vaapi_h264: Do not set unused deprecated fields

[Mark Thompson]

On 27/03/2019 14:02, Carl Eugen Hoyos wrote:
> Hi!
> 
> I don't think setting vaapi fields that are deprecated makes sense, this
> also silences a warning when compiling with clang.
> 
> Please comment, Carl Eugen
> 
> From b1f4e64e577ffecd1eab7b95d8d6f90e261cdd74 Mon Sep 17 00:00:00 2001
> From: Carl Eugen Hoyos 
> Date: Wed, 27 Mar 2019 14:58:51 +0100
> Subject: [PATCH] lavc/vaapi_264: Do not set deprecated unused fields.
> 
> Fixes the following warnings:
> libavcodec/vaapi_h264.c:259:10: warning: 'num_slice_groups_minus1' is 
> deprecated [-Wdeprecated-declarations]
> .num_slice_groups_minus1= pps->slice_group_count 
> - 1,
>  ^
> libavcodec/vaapi_h264.c:260:10: warning: 'slice_group_map_type' is deprecated 
> [-Wdeprecated-declarations]
> .slice_group_map_type   = 
> pps->mb_slice_group_map_type,
>  ^
> libavcodec/vaapi_h264.c:261:10: warning: 'slice_group_change_rate_minus1' is 
> deprecated [-Wdeprecated-declarations]
> .slice_group_change_rate_minus1 = 0, /* FMO is not 
> implemented */
>  ^
> ---
>  libavcodec/vaapi_h264.c |3 ---
>  1 file changed, 3 deletions(-)
> 
> diff --git a/libavcodec/vaapi_h264.c b/libavcodec/vaapi_h264.c
> index 5854587..dd2a657 100644
> --- a/libavcodec/vaapi_h264.c
> +++ b/libavcodec/vaapi_h264.c
> @@ -256,9 +256,6 @@ static int vaapi_h264_start_frame(AVCodecContext  
> *avctx,
>  .log2_max_pic_order_cnt_lsb_minus4  = sps->log2_max_poc_lsb 
> - 4,
>  .delta_pic_order_always_zero_flag   = 
> sps->delta_pic_order_always_zero_flag,
>  },
> -.num_slice_groups_minus1= pps->slice_group_count 
> - 1,
> -.slice_group_map_type   = 
> pps->mb_slice_group_map_type,
> -.slice_group_change_rate_minus1 = 0, /* FMO is not 
> implemented */
>  .pic_init_qp_minus26= pps->init_qp - 26,
>  .pic_init_qs_minus26= pps->init_qs - 26,
>  .chroma_qp_index_offset = 
> pps->chroma_qp_index_offset[0],
> -- 
> 1.7.10.4

They aren't deprecated on all VAAPI versions we support (try building with 
libva < 2).

I think the change is ok anyway because zero is fine as a value for them?  The 
commit message should probably mention that if you want to go with it.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] qsv: fix the dangerous macro definitions

[Mark Thompson]

On 27/03/2019 10:24, Zhong Li wrote:
> Signed-off-by: Zhong Li 
> ---
>  libavcodec/qsv_internal.h | 8 
>  libavfilter/qsvvpp.h  | 8 
>  2 files changed, 8 insertions(+), 8 deletions(-)
> 
> diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h
> index 394c558883..86a5dbad98 100644
> --- a/libavcodec/qsv_internal.h
> +++ b/libavcodec/qsv_internal.h
> @@ -35,12 +35,12 @@
>  #define QSV_MAX_ENC_PAYLOAD 2   // # of mfxEncodeCtrl payloads supported
>  
>  #define QSV_VERSION_ATLEAST(MAJOR, MINOR)   \
> -(MFX_VERSION_MAJOR > (MAJOR) || \
> - MFX_VERSION_MAJOR == (MAJOR) && MFX_VERSION_MINOR >= (MINOR))
> +((MFX_VERSION_MAJOR > (MAJOR) || \
> + MFX_VERSION_MAJOR == (MAJOR) && MFX_VERSION_MINOR >= (MINOR)))

I don't understand why this makes a difference?

Removing the whitespace, I see:

-  (MFX_VERSION_MAJOR > (MAJOR) || MFX_VERSION_MAJOR == (MAJOR) && 
MFX_VERSION_MINOR >= (MINOR))
+ ((MFX_VERSION_MAJOR > (MAJOR) || MFX_VERSION_MAJOR == (MAJOR) && 
MFX_VERSION_MINOR >= (MINOR)))

which looks like you've just added redundant parentheses around the whole thing 
which already had them?

(Alternative question: what case is this trying to fix?  I could see a problem 
if the MFX_VERSION_* fields were macros expanding to something containing 
operators with lower precedence than relationals, but that doesn't change with 
what you've done here.)

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] configure: Do not enable both OpenCL-VAAPI interop modes simultaneously

[Mark Thompson]

On 27/03/2019 00:59, Song, Ruiling wrote:
>> -Original Message-
>> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf Of
>> Mark Thompson
>> Sent: Wednesday, March 27, 2019 7:39 AM
>> To: FFmpeg development discussions and patches 
>> Subject: [FFmpeg-devel] [PATCH] configure: Do not enable both OpenCL-VAAPI
>> interop modes simultaneously
>>
>> Beignet offers a far more flexiable and complete interface, so choose it
>> by default if available.
> Sorry I missed your last mail. Sure, I agree Beignet sharing is far more 
> flexible.
> The patch LGTM.

Applied.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] configure: include pkgconfig path as vaapi header search

[Mark Thompson]

On 20/03/2019 07:57, Zhong Li wrote:
> Currectly just standard header path and be found,
> check_type/struct will fail if vaapi is installed somewhere else.
> ---
>  configure | 18 ++
>  1 file changed, 10 insertions(+), 8 deletions(-)
> 
> diff --git a/configure b/configure
> index eaf543df96..0e3c2d24bf 100755
> --- a/configure
> +++ b/configure
> @@ -6024,14 +6024,6 @@ check_type "windows.h d3d11.h" "ID3D11VideoDecoder"
>  check_type "windows.h d3d11.h" "ID3D11VideoContext"
>  check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode 
> -D_WIN32_WINNT=0x0602
>  
> -check_type "va/va.h va/va_dec_hevc.h" "VAPictureParameterBufferHEVC"
> -check_struct "va/va.h" "VADecPictureParameterBufferVP9" bit_depth
> -check_struct "va/va.h va/va_vpp.h" "VAProcPipelineCaps" rotation_flags
> -check_type "va/va.h va/va_enc_hevc.h" "VAEncPictureParameterBufferHEVC"
> -check_type "va/va.h va/va_enc_jpeg.h" "VAEncPictureParameterBufferJPEG"
> -check_type "va/va.h va/va_enc_vp8.h"  "VAEncPictureParameterBufferVP8"
> -check_type "va/va.h va/va_enc_vp9.h"  "VAEncPictureParameterBufferVP9"
> -
>  check_type "vdpau/vdpau.h" "VdpPictureInfoHEVC"
>  
>  if enabled cuda_sdk; then
> @@ -6469,6 +6461,16 @@ if enabled vaapi; then
>  check_cpp_condition vaapi_1 "va/va.h" "VA_CHECK_VERSION(1, 0, 0)"
>  fi
>  
> +if enabled vaapi; then

Merge this into the previous block, which has the same condition.

> +check_type "va/va.h va/va_dec_hevc.h" "VAPictureParameterBufferHEVC"
> +check_struct "va/va.h" "VADecPictureParameterBufferVP9" bit_depth
> +check_struct "va/va.h va/va_vpp.h" "VAProcPipelineCaps" rotation_flags
> +check_type "va/va.h va/va_enc_hevc.h" "VAEncPictureParameterBufferHEVC"
> +check_type "va/va.h va/va_enc_jpeg.h" "VAEncPictureParameterBufferJPEG"
> +check_type "va/va.h va/va_enc_vp8.h"  "VAEncPictureParameterBufferVP8"
> +check_type "va/va.h va/va_enc_vp9.h"  "VAEncPictureParameterBufferVP9"
> +fi
> +
>  if enabled_all opencl libdrm ; then
>  check_type "CL/cl_intel.h" "clCreateImageFromFdINTEL_fn" &&
>  enable opencl_drm_beignet
> 

LGTM with that.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] lavc/qsvenc: get vps extradata from MSDK

[Mark Thompson]

On 27/03/2019 10:27, Zhong Li wrote:
> Signed-off-by: Zhong Li 
> ---
> V2: Fix the regression of qsv h264 encoding since no VPS for h264
> 
>  libavcodec/qsvenc.c  | 53 ++--
>  libavcodec/qsvenc.h  |  3 +++
>  libavcodec/qsvenc_hevc.c | 10 +---
>  3 files changed, 54 insertions(+), 12 deletions(-)
> 
> diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c
> index 576d88c259..2f128597db 100644
> --- a/libavcodec/qsvenc.c
> +++ b/libavcodec/qsvenc.c
> @@ -810,6 +810,16 @@ static int qsv_retrieve_enc_params(AVCodecContext 
> *avctx, QSVEncContext *q)
>  };
>  #endif
>  
> +#if QSV_HAVE_CO_VPS
> +uint8_t vps_buf[128];

Is this necessarily enough?  A VPS can be large when it defines sublayers.

> +mfxExtCodingOptionVPS extradata_vps = {
> +.Header.BufferId = MFX_EXTBUFF_CODING_OPTION_VPS,
> +.Header.BufferSz = sizeof(extradata_vps),
> +.VPSBuffer   = vps_buf,
> +.VPSBufSize  = sizeof(vps_buf),
> +};
> +#endif
> +
>  mfxExtBuffer *ext_buffers[] = {
>  (mfxExtBuffer*)&extradata,
>  (mfxExtBuffer*)&co,
> @@ -818,14 +828,24 @@ static int qsv_retrieve_enc_params(AVCodecContext 
> *avctx, QSVEncContext *q)
>  #endif
>  #if QSV_HAVE_CO3
>  (mfxExtBuffer*)&co3,
> +#endif
> +#if QSV_HAVE_CO_VPS
> +(mfxExtBuffer*)&extradata_vps,
>  #endif
>  };
>  
>  int need_pps = avctx->codec_id != AV_CODEC_ID_MPEG2VIDEO;
> -int ret;
> +int ret, extradata_offset = 0;
>  
>  q->param.ExtParam= ext_buffers;
> +
> +#if QSV_HAVE_CO_VPS
> +q->hevc_vps = ((avctx->codec_id == AV_CODEC_ID_HEVC) && 
> QSV_RUNTIME_VERSION_ATLEAST(q->ver, 1, 17));
> +q->param.NumExtParam = FF_ARRAY_ELEMS(ext_buffers) - 1 + q->hevc_vps;

The array definition and then this length feels a bit overly tricky - any 
change here will be quite confusing (consider adding a new ExtBuffer).

Making ext_buffers large enough with a running nb_ext_buffers total and then 
adding the extra one only if H.265 would feel safer to me?

> +#else
> +q->hevc_vps = 0;
>  q->param.NumExtParam = FF_ARRAY_ELEMS(ext_buffers);
> +#endif
>  
>  ret = MFXVideoENCODE_GetVideoParam(q->session, &q->param);
>  if (ret < 0)
> @@ -834,20 +854,37 @@ static int qsv_retrieve_enc_params(AVCodecContext 
> *avctx, QSVEncContext *q)
>  
>  q->packet_size = q->param.mfx.BufferSizeInKB * 
> q->param.mfx.BRCParamMultiplier * 1000;
>  
> -if (!extradata.SPSBufSize || (need_pps && !extradata.PPSBufSize)) {
> +if (!extradata.SPSBufSize || (need_pps && !extradata.PPSBufSize)
> +#if QSV_HAVE_CO_VPS
> +|| (q->hevc_vps && !extradata_vps.VPSBufSize)
> +#endif
> +) {
>  av_log(avctx, AV_LOG_ERROR, "No extradata returned from libmfx.\n");
>  return AVERROR_UNKNOWN;
>  }
>  
> -avctx->extradata = av_malloc(extradata.SPSBufSize + need_pps * 
> extradata.PPSBufSize +
> - AV_INPUT_BUFFER_PADDING_SIZE);
> +avctx->extradata_size = extradata.SPSBufSize + need_pps * 
> extradata.PPSBufSize;
> +#if QSV_HAVE_CO_VPS
> +avctx->extradata_size += q->hevc_vps * extradata_vps.VPSBufSize;
> +#endif
> +
> +avctx->extradata = av_malloc(avctx->extradata_size + 
> AV_INPUT_BUFFER_PADDING_SIZE);
>  if (!avctx->extradata)
>  return AVERROR(ENOMEM);
>  
> -memcpy(avctx->extradata,sps_buf, 
> extradata.SPSBufSize);
> -if (need_pps)
> -memcpy(avctx->extradata + extradata.SPSBufSize, pps_buf, 
> extradata.PPSBufSize);
> -avctx->extradata_size = extradata.SPSBufSize + need_pps * 
> extradata.PPSBufSize;
> +#if QSV_HAVE_CO_VPS
> +if (q->hevc_vps) {
> +memcpy(avctx->extradata, vps_buf, extradata_vps.VPSBufSize);
> +extradata_offset += extradata_vps.VPSBufSize;
> +}
> +#endif
> +
> +memcpy(avctx->extradata + extradata_offset, sps_buf, 
> extradata.SPSBufSize);
> +extradata_offset += extradata.SPSBufSize;
> +if (need_pps) {
> +memcpy(avctx->extradata + extradata_offset, pps_buf, 
> extradata.PPSBufSize);
> +extradata_offset += extradata.PPSBufSize;
> +}
>  memset(avctx->extradata + avctx->extradata_size, 0, 
> AV_INPUT_BUFFER_PADDING_SIZE);
>  
>  cpb_props = ff_add_cpb_side_data(avctx);
> diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h
> index 25105894f2..f01453e67f 100644
> --- a/libavcodec/qsvenc.h
> +++ b/libavcodec/qsvenc.h
> @@ -36,6 +36,7 @@
>  
>  #define QSV_HAVE_CO2 QSV_VERSION_ATLEAST(1, 6)
>  #define QSV_HAVE_CO3 QSV_VERSION_ATLEAST(1, 11)
> +#define QSV_HAVE_CO_VPS  QSV_VERSION_ATLEAST(1, 17)
>  
>  #define QSV_HAVE_TRELLIS QSV_VERSION_ATLEAST(1, 8)
>  #define QSV_HAVE_MAX_SLICE_SIZE QSV_VERSION_ATLEAST(1, 9)
> @@ -135,6 +136,8 @@ typedef struct QSVEncContext {
>  
>  mfxVersion  ver;
>  
> +int hevc_vps;
> +
>  // options set by the caller
>  int async_depth;
>  int idr_interval;
> diff --git a/libavcodec/q

Re: [FFmpeg-devel] [PATCH] lavc/vaapi_decode: add va_profile format map support for HEVC_REXT

[Mark Thompson]

On 28/03/2019 04:03, Linjie Fu wrote:
> HEVC_REXT will be map to {VAProfileHEVCMain422_10, VAProfileHEVCMain444,
> VAProfileHEVCMain444_10} in vaapi_profile_map[], since need to be 
> distinguished
> to select the exact va_profile.
> 
> Add va_profile -> AV_PIX_FMT map for FF_PROFILE_HEVC_REXT to match the
> exact va_profile.
> 
> Signed-off-by: Linjie Fu 
> ---
>  libavcodec/vaapi_decode.c | 29 +
>  1 file changed, 25 insertions(+), 4 deletions(-)
> 
> diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
> index 015154b879..1cb8683b7c 100644
> --- a/libavcodec/vaapi_decode.c
> +++ b/libavcodec/vaapi_decode.c
> @@ -414,6 +414,18 @@ static const struct {
>  #undef MAP
>  };
>  
> +static const struct {
> +VAProfile va_profile;
> +enum AVPixelFormat pix_fmt;
> +} rext_format_map[] = {
> +#define MAP(vp, av) { VAProfileHEVCMain ## vp, AV_PIX_FMT_ ## av }
> +MAP(422_10,  YUYV422),
> +MAP(422_10,  YUV422P10LE),
> +MAP(444, YUV444P),
> +MAP(444_10,  YUV444P10LE),

This doesn't work - you can't guess the rext profile from the chroma format and 
bit depth information, because the profiles are all overlapping (see table A.1).

You need to use the profile constraint flags to determine it - this lookup is 
implemented by ff_h265_get_profile(), but you'll need to extract all the flags 
to put into it.

> +#undef MAP
> +};
> +
>  /*
>   * Set *va_config and the frames_ref fields from the current codec parameters
>   * in avctx.
> @@ -426,7 +438,7 @@ static int vaapi_decode_make_config(AVCodecContext *avctx,
>  AVVAAPIHWConfig   *hwconfig= NULL;
>  AVHWFramesConstraints *constraints = NULL;
>  VAStatus vas;
> -int err, i, j;
> +int err, i, j, k;
>  const AVCodecDescriptor *codec_desc;
>  VAProfile *profile_list = NULL, matched_va_profile;
>  int profile_count, exact_match, matched_ff_profile;
> @@ -467,13 +479,22 @@ static int vaapi_decode_make_config(AVCodecContext 
> *avctx,
>  if (avctx->profile == vaapi_profile_map[i].codec_profile ||
>  vaapi_profile_map[i].codec_profile == FF_PROFILE_UNKNOWN)
>  profile_match = 1;
> -for (j = 0; j < profile_count; j++) {
> -if (vaapi_profile_map[i].va_profile == profile_list[j]) {
> +if (avctx->profile == FF_PROFILE_HEVC_REXT) {
> +/* find the exact va_profile for HEVC_REXT */
> +for (j = 0; j < FF_ARRAY_ELEMS(rext_format_map); j++) {
> +if (avctx->pix_fmt == rext_format_map[j].pix_fmt)
> +   break;
> +}
> +if (vaapi_profile_map[i].va_profile != 
> rext_format_map[j].va_profile)
> +continue;
> +}

Codec-specific stuff probably shouldn't be hidden in the middle of the generic 
code like this.

> +for (k = 0; k < profile_count; k++) {
> +if (vaapi_profile_map[i].va_profile == profile_list[k]) {
>  exact_match = profile_match;
>  break;
>  }
>  }
> -if (j < profile_count) {
> +if (k < profile_count) {
>  matched_va_profile = vaapi_profile_map[i].va_profile;
>  matched_ff_profile = vaapi_profile_map[i].codec_profile;
>  if (exact_match)
> 

When will it be possible to get hardware which supports these profiles?

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] configure: include pkgconfig path as vaapi header search

[Mark Thompson]

On 28/03/2019 05:17, Song, Ruiling wrote:

 Neo is the successor to Beignet, correct?
>>> Yes, that's the truth.
>>> Currently we simply checking against the specific header file of OpenCL,
>>> which is in-fact not accurate.
>>> I am not sure whether you would like to use Neo together with
>>> intel-media-driver, which is the most targeted opencl usage in FFmpeg.
>>> If that's the case, I think it may be hard to find a matching
>>> intel-media-driver to work with Neo release package.
>>> Because Neo release version depends on a very outdated libva revision.
>>> I just sent a patch to Neo to update libva revision dependency. Once they
>>> accept the patch and new Neo release package comes out,
>>> I think we can change to check against Neo package. People would not need
>>> to build Neo themselves then.
>>>
>>> Thanks!
>>> Ruiling

 Enabling similar functionality for Neo should allow for the same feature
 support for these not using Beignet.
>>>
>>>
>> Indeed, I'd want to use Neo + intel-media-driver.
>> Judging by the (relatively low) development activity on Beignet of late,
>> its' considered ready to deprecate in place of Neo, applicable on anything
>> newer than Kabylake.
> I think Mark don't have plan to deprecate Beignet now, and me too.
> FFmpeg-OpenCL currently use direct buffer sharing between OpenCL and vaapi 
> driver.
> One obvious limitation I didn't notice before is 10bit or 12bit buffer 
> sharing is not supported by Neo.
> I pinged the author of cl_intel_va_api_media_sharing, but got no response.
> Maybe I will take some effort to update the extension spec and implement them 
> in Neo myself.
> I am not sure any other Neo limitation that Mark wants to add?

For my point of view, the ideal thing to happen would be for Intel and AMD to 
work together to create a DRM object (kernel dma-buf) sharing extension for 
OpenCL which works on all of their platforms (especially the new ones, AMD ROCm 
and Intel NEO).

I don't think a VAAPI-specific extension is really the right way forward - as 
far as I can tell there is no good reason to tie it to that specific API (or, 
even worse, particular driver implementations thereof).  Compare Vulkan, which 
already does all of the interop on Linux via DRM object fds (and which 
therefore supports clean interop with Beignet, as well as with the Mesa and 
Intel VAAPI drivers themselves).

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavc/hevc_ps: parse constraint flags for HEVC REXT

[Mark Thompson]

On 01/04/2019 22:38, James Almer wrote:
> On 4/1/2019 6:43 AM, Linjie Fu wrote:
>> Parse all the constraint flags.
>>
>> It can be passed to hw decoders to detemine the exact profile for Range
>> Extension HEVC.
>>
>> Signed-off-by: Linjie Fu 
>> ---
>>  libavcodec/hevc_ps.c | 18 +++---
>>  libavcodec/hevc_ps.h | 10 ++
>>  2 files changed, 25 insertions(+), 3 deletions(-)
>>
>> diff --git a/libavcodec/hevc_ps.c b/libavcodec/hevc_ps.c
>> index 80df417e4f..1365a9d640 100644
>> --- a/libavcodec/hevc_ps.c
>> +++ b/libavcodec/hevc_ps.c
>> @@ -295,9 +295,21 @@ static int decode_profile_tier_level(GetBitContext *gb, 
>> AVCodecContext *avctx,
>>  ptl->non_packed_constraint_flag = get_bits1(gb);
>>  ptl->frame_only_constraint_flag = get_bits1(gb);
>>  
>> -skip_bits(gb, 16); // XXX_reserved_zero_44bits[0..15]
>> -skip_bits(gb, 16); // XXX_reserved_zero_44bits[16..31]
>> -skip_bits(gb, 12); // XXX_reserved_zero_44bits[32..43]
>> +ptl->max_12bit_constraint_flag= get_bits1(gb);
>> +ptl->max_10bit_constraint_flag= get_bits1(gb);
>> +ptl->max_8bit_constraint_flag = get_bits1(gb);
>> +ptl->max_422chroma_constraint_flag= get_bits1(gb);
>> +ptl->max_420chroma_constraint_flag= get_bits1(gb);
>> +ptl->max_monochrome_constraint_flag   = get_bits1(gb);
>> +ptl->intra_constraint_flag= get_bits1(gb);
>> +ptl->one_picture_only_constraint_flag = get_bits1(gb);
>> +ptl->lower_bit_rate_constraint_flag   = get_bits1(gb);
>> +
>> +skip_bits(gb, 16); // XXX_reserved_zero_34bits[0..15]
> 
> The first of these bits can be general_max_14bit_constraint_flag. I
> suppose no hardware really supports it?

I think it's worth getting these right anyway.

From the set of flags you've ended up with here I'm guessing you're working 
from a pre-2018 version of the standard?  Try the 201802 version - 
.

>> +skip_bits(gb, 16); // XXX_reserved_zero_34bits[16..31]
>> +skip_bits(gb, 2);  // XXX_reserved_zero_34bits[32..33]
>> +
>> +ptl->inbld_flag = get_bits1(gb);
> 
> Strictly speaking, you should be checking ptl->profile_idc and
> ptl->profile_compatibility_flag to parse these, as per section 7.3.3 in
> the spec.
> Despite the amount of bits being fixed and the default value being 0, it
> would be ideal to prevent misparsing damaged samples.

+1.  Future profiles which could assign completely different meanings to these 
bits are also a possibility, though I would hope the standard people would 
avoid doing that.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/cbs_av1: fix range of allowed values for obu_type

[Mark Thompson]

On 24/03/2019 22:28, James Almer wrote:
> 0 is a reserved value.
> 
> Signed-off-by: James Almer 
> ---
>  libavcodec/cbs_av1_syntax_template.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/cbs_av1_syntax_template.c 
> b/libavcodec/cbs_av1_syntax_template.c
> index 76eb90b279..35b030208b 100644
> --- a/libavcodec/cbs_av1_syntax_template.c
> +++ b/libavcodec/cbs_av1_syntax_template.c
> @@ -26,7 +26,7 @@ static int FUNC(obu_header)(CodedBitstreamContext *ctx, 
> RWContext *rw,
>  
>  fc(1, obu_forbidden_bit, 0, 0);
>  
> -fc(4, obu_type, 0, AV1_OBU_PADDING);
> +fc(4, obu_type, AV1_OBU_SEQUENCE_HEADER, AV1_OBU_PADDING);
>  flag(obu_extension_flag);
>  flag(obu_has_size_field);
>  
> 

Seems fine, but I'm not sure whether it is useful to distinguish 0 from the 
other reserved values (9-14) as invalid at this stage?  They will all pass 
through and return ENOSYS later when we come to read/write the content anyway.

(LGTM if you feel strongly.)

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/3] avcodec/cbs_av1: add support for Padding OBUs

[Mark Thompson]

On 25/03/2019 14:22, James Almer wrote:
> Based on itut_t35 Matadata OBU parsing code.
> 
> Signed-off-by: James Almer 
> ---
>  libavcodec/cbs_av1.c | 20 +
>  libavcodec/cbs_av1.h |  7 ++
>  libavcodec/cbs_av1_syntax_template.c | 32 
>  3 files changed, 59 insertions(+)
> 
> diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c
> index 02f168b58d..22330eabf3 100644
> --- a/libavcodec/cbs_av1.c
> +++ b/libavcodec/cbs_av1.c
> @@ -857,6 +857,11 @@ static void cbs_av1_free_tile_data(AV1RawTileData *td)
>  av_buffer_unref(&td->data_ref);
>  }
>  
> +static void cbs_av1_free_padding(AV1RawPadding *pd)
> +{
> +av_buffer_unref(&pd->payload_ref);
> +}
> +
>  static void cbs_av1_free_metadata(AV1RawMetadata *md)
>  {
>  switch (md->metadata_type) {
> @@ -883,6 +888,9 @@ static void cbs_av1_free_obu(void *unit, uint8_t *content)
>  case AV1_OBU_METADATA:
>  cbs_av1_free_metadata(&obu->obu.metadata);
>  break;
> +case AV1_OBU_PADDING:
> +cbs_av1_free_padding(&obu->obu.padding);
> +break;
>  }
>  
>  av_freep(&obu);
> @@ -1057,6 +1065,12 @@ static int cbs_av1_read_unit(CodedBitstreamContext 
> *ctx,
>  }
>  break;
>  case AV1_OBU_PADDING:
> +{
> +err = cbs_av1_read_padding(ctx, &gbc, &obu->obu.padding);
> +if (err < 0)
> +return err;
> +}
> +break;
>  default:
>  return AVERROR(ENOSYS);
>  }
> @@ -1182,6 +1196,12 @@ static int cbs_av1_write_obu(CodedBitstreamContext 
> *ctx,
>  }
>  break;
>  case AV1_OBU_PADDING:
> +{
> +err = cbs_av1_write_padding(ctx, pbc, &obu->obu.padding);
> +if (err < 0)
> +return err;
> +}
> +break;
>  default:
>  return AVERROR(ENOSYS);
>  }
> diff --git a/libavcodec/cbs_av1.h b/libavcodec/cbs_av1.h
> index 71ceff9427..e799964b72 100644
> --- a/libavcodec/cbs_av1.h
> +++ b/libavcodec/cbs_av1.h
> @@ -364,6 +364,12 @@ typedef struct AV1RawMetadata {
>  } metadata;
>  } AV1RawMetadata;
>  
> +typedef struct AV1RawPadding {
> +uint8_t *payload;
> +size_t   payload_size;
> +AVBufferRef *payload_ref;
> +} AV1RawPadding;
> +
>  
>  typedef struct AV1RawOBU {
>  AV1RawOBUHeader header;
> @@ -377,6 +383,7 @@ typedef struct AV1RawOBU {
>  AV1RawTileGroup  tile_group;
>  AV1RawTileList   tile_list;
>  AV1RawMetadata   metadata;
> +AV1RawPaddingpadding;
>  } obu;
>  } AV1RawOBU;
>  
> diff --git a/libavcodec/cbs_av1_syntax_template.c 
> b/libavcodec/cbs_av1_syntax_template.c
> index 76eb90b279..a6cafdd99f 100644
> --- a/libavcodec/cbs_av1_syntax_template.c
> +++ b/libavcodec/cbs_av1_syntax_template.c
> @@ -1763,3 +1763,35 @@ static int FUNC(metadata_obu)(CodedBitstreamContext 
> *ctx, RWContext *rw,
>  
>  return 0;
>  }
> +
> +static int FUNC(padding)(CodedBitstreamContext *ctx, RWContext *rw,
> + AV1RawPadding *current)
> +{
> +int i, err;
> +
> +HEADER("Padding");
> +
> +#ifdef READ
> +// The payload runs up to the start of the trailing bits, but there might
> +// be arbitrarily many trailing zeroes so we need to read through twice.
> +{
> +GetBitContext tmp = *rw;
> +current->payload_size = 0;
> +for (i = 0; get_bits_left(rw) >= 8; i++) {
> +if (get_bits(rw, 8))
> +current->payload_size = i;
> +}
> +*rw = tmp;
> +
> +current->payload_ref = av_buffer_alloc(current->payload_size);
> +if (!current->payload_ref)
> +return AVERROR(ENOMEM);
> +current->payload = current->payload_ref->data;
> +}
> +#endif

That looks factorisable.  Can we make a "measure length of payload and allocate 
buffer for it" function and use it in metadata_itu_t35 as well?

> +
> +for (i = 0; i < current->payload_size; i++)
> +xf(8, obu_padding_byte[i], current->payload[i], 0x00, 0xff, 1, i);
> +
> +return 0;
> +}
> 

Code looks sensible, but could you explain a bit more about why this is 
helpful.  Is there a use-case for preserving the actual padding bytes?  If 
that's some sort of CBR application, is it actually going to need to preserve 
the trailing zeroes as well?

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/3] avcodec/av1_metadata: add an option to remove Padding OBUs

[Mark Thompson]

On 25/03/2019 14:22, James Almer wrote:
> Signed-off-by: James Almer 
> ---
>  libavcodec/av1_metadata_bsf.c | 19 +++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/libavcodec/av1_metadata_bsf.c b/libavcodec/av1_metadata_bsf.c
> index 2b74b697e4..fe208feaf5 100644
> --- a/libavcodec/av1_metadata_bsf.c
> +++ b/libavcodec/av1_metadata_bsf.c
> @@ -46,6 +46,8 @@ typedef struct AV1MetadataContext {
>  
>  AVRational tick_rate;
>  int num_ticks_per_picture;
> +
> +int delete_padding;
>  } AV1MetadataContext;
>  
>  
> @@ -158,6 +160,19 @@ static int av1_metadata_filter(AVBSFContext *bsf, 
> AVPacket *out)
>  }
>  }
>  
> +if (ctx->delete_padding) {
> +for (i = 0; i < frag->nb_units; i++) {
> +if (frag->units[i].type == AV1_OBU_PADDING) {
> +err = ff_cbs_delete_unit(ctx->cbc, frag, i);
> +if (err < 0) {
> +av_log(bsf, AV_LOG_ERROR, "Failed to delete Padding 
> OBU.\n");
> +goto fail;
> +}
> +--i;
> +}
> +}
> +}
> +
>  err = ff_cbs_write_packet(ctx->cbc, out, frag);
>  if (err < 0) {
>  av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n");
> @@ -275,6 +290,10 @@ static const AVOption av1_metadata_options[] = {
>  OFFSET(num_ticks_per_picture), AV_OPT_TYPE_INT,
>  { .i64 = -1 }, -1, INT_MAX, FLAGS },
>  
> +{ "delete_padding", "Delete all Padding OBUs",
> +OFFSET(delete_padding), AV_OPT_TYPE_BOOL,
> +{ .i64 = 0 }, 0, 1, FLAGS},
> +
>  { NULL }
>  };
>  
> 

LGTM with matching doc update :)

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/3] avcodec/cbs_av1: fix parsing spatial_id

[Mark Thompson]

On 25/03/2019 14:22, James Almer wrote:
> Signed-off-by: James Almer 
> ---
>  libavcodec/cbs_av1.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c
> index 22330eabf3..77548084b6 100644
> --- a/libavcodec/cbs_av1.c
> +++ b/libavcodec/cbs_av1.c
> @@ -964,7 +964,7 @@ static int cbs_av1_read_unit(CodedBitstreamContext *ctx,
>  
>  if (obu->header.obu_extension_flag) {
>  priv->temporal_id = obu->header.temporal_id;
> -priv->spatial_id  = obu->header.temporal_id;
> +priv->spatial_id  = obu->header.spatial_id;

Oops :(

>  
>  if (obu->header.obu_type != AV1_OBU_SEQUENCE_HEADER &&
>  obu->header.obu_type != AV1_OBU_TEMPORAL_DELIMITER &&
> 
LGTM.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 2/9] cbs_vp9: Implement parser entrypoint

[Mark Thompson]

---
 libavcodec/cbs_vp9.c | 90 +++-
 1 file changed, 73 insertions(+), 17 deletions(-)

diff --git a/libavcodec/cbs_vp9.c b/libavcodec/cbs_vp9.c
index 0b5f137ed8..6ea4681d68 100644
--- a/libavcodec/cbs_vp9.c
+++ b/libavcodec/cbs_vp9.c
@@ -409,15 +409,23 @@ static int cbs_vp9_write_le(CodedBitstreamContext *ctx, 
PutBitContext *pbc,
 #undef byte_alignment
 
 
-static int cbs_vp9_split_fragment(CodedBitstreamContext *ctx,
-  CodedBitstreamFragment *frag,
-  int header)
+typedef int (*cbs_vp9_split_frame_callback)(CodedBitstreamContext *ctx,
+void *priv,
+const uint8_t *data,
+size_t data_size);
+
+static int cbs_vp9_split_frames(CodedBitstreamContext *ctx,
+void *priv, cbs_vp9_split_frame_callback cb,
+const uint8_t *data, size_t data_size)
 {
 uint8_t superframe_header;
 int err;
 
+if (data_size == 0)
+return 0;
+
 // Last byte in the packet.
-superframe_header = frag->data[frag->data_size - 1];
+superframe_header = data[data_size - 1];
 
 if ((superframe_header & 0xe0) == 0xc0) {
 VP9RawSuperframeIndex sfi;
@@ -427,8 +435,14 @@ static int cbs_vp9_split_fragment(CodedBitstreamContext 
*ctx,
 
 index_size = 2 + (((superframe_header & 0x18) >> 3) + 1) *
   ((superframe_header & 0x07) + 1);
+if (index_size > data_size) {
+av_log(ctx->log_ctx, AV_LOG_ERROR, "Superframe index (%"
+   SIZE_SPECIFIER" bytes) is larger than whole frame (%"
+   SIZE_SPECIFIER" bytes).\n", index_size, data_size);
+return AVERROR_INVALIDDATA;
+}
 
-err = init_get_bits(&gbc, frag->data + frag->data_size - index_size,
+err = init_get_bits(&gbc, data + data_size - index_size,
 8 * index_size);
 if (err < 0)
 return err;
@@ -439,34 +453,27 @@ static int cbs_vp9_split_fragment(CodedBitstreamContext 
*ctx,
 
 pos = 0;
 for (i = 0; i <= sfi.frames_in_superframe_minus_1; i++) {
-if (pos + sfi.frame_sizes[i] + index_size > frag->data_size) {
+if (pos + sfi.frame_sizes[i] + index_size > data_size) {
 av_log(ctx->log_ctx, AV_LOG_ERROR, "Frame %d too large "
"in superframe: %"PRIu32" bytes.\n",
i, sfi.frame_sizes[i]);
 return AVERROR_INVALIDDATA;
 }
 
-err = ff_cbs_insert_unit_data(ctx, frag, -1, 0,
-  frag->data + pos,
-  sfi.frame_sizes[i],
-  frag->data_ref);
+err = cb(ctx, priv, data + pos, sfi.frame_sizes[i]);
 if (err < 0)
 return err;
 
 pos += sfi.frame_sizes[i];
 }
-if (pos + index_size != frag->data_size) {
+if (pos + index_size != data_size) {
 av_log(ctx->log_ctx, AV_LOG_WARNING, "Extra padding at "
"end of superframe: %"SIZE_SPECIFIER" bytes.\n",
-   frag->data_size - (pos + index_size));
+   data_size - (pos + index_size));
 }
 
-return 0;
-
 } else {
-err = ff_cbs_insert_unit_data(ctx, frag, -1, 0,
-  frag->data, frag->data_size,
-  frag->data_ref);
+err = cb(ctx, priv, data, data_size);
 if (err < 0)
 return err;
 }
@@ -474,6 +481,23 @@ static int cbs_vp9_split_fragment(CodedBitstreamContext 
*ctx,
 return 0;
 }
 
+static int cbs_vp9_insert_unit(CodedBitstreamContext *ctx, void *priv,
+   const uint8_t *data, size_t data_size)
+{
+CodedBitstreamFragment *frag = priv;
+return ff_cbs_insert_unit_data(ctx, frag, -1, 0,
+   (uint8_t*)data, data_size,
+   frag->data_ref);
+}
+
+static int cbs_vp9_split_fragment(CodedBitstreamContext *ctx,
+  CodedBitstreamFragment *frag,
+  int header)
+{
+return cbs_vp9_split_frames(ctx, frag, &cbs_vp9_insert_unit,
+frag->data, frag->data_size);
+}
+
 static void cbs_vp9_free_frame(void *unit, uint8_t *content)
 {
 VP9RawFrame *frame = (VP9RawFrame*)content;
@@ -678,6 +702,36 @@ static void cbs_vp9_close(CodedBitstreamContext *ctx)
 av_freep(&priv->write_buffer);
 }
 
+static int cbs_vp9_parse_frame(CodedBitstreamContext *ctx, void *priv,
+   const uint8_t *data, size_t data_size)
+{
+VP9RawFrameHeader *frame = priv;
+GetBitConte

[FFmpeg-devel] [PATCH v2 9/9] av1_parser: Reindent

[Mark Thompson]

---
 libavcodec/av1_parser.c | 74 -
 1 file changed, 37 insertions(+), 37 deletions(-)

diff --git a/libavcodec/av1_parser.c b/libavcodec/av1_parser.c
index 4a743d92d4..b848b41050 100644
--- a/libavcodec/av1_parser.c
+++ b/libavcodec/av1_parser.c
@@ -88,43 +88,43 @@ static int av1_parser_parse(AVCodecParserContext *ctx,
 seq = av1->sequence_header;
 color = &seq->color_config;
 
-ctx->width  = av1->frame_width;
-ctx->height = av1->frame_height;
-
-ctx->key_frame = frame->frame_type == AV1_FRAME_KEY;
-
-avctx->profile = seq->seq_profile;
-avctx->level   = seq->seq_level_idx[0];
-
-switch (frame->frame_type) {
-case AV1_FRAME_KEY:
-case AV1_FRAME_INTRA_ONLY:
-ctx->pict_type = AV_PICTURE_TYPE_I;
-break;
-case AV1_FRAME_INTER:
-ctx->pict_type = AV_PICTURE_TYPE_P;
-break;
-case AV1_FRAME_SWITCH:
-ctx->pict_type = AV_PICTURE_TYPE_SP;
-break;
-}
-ctx->picture_structure = AV_PICTURE_STRUCTURE_FRAME;
-
-switch (av1->bit_depth) {
-case 8:
-ctx->format = color->mono_chrome ? AV_PIX_FMT_GRAY8
- : pix_fmts_8bit 
[color->subsampling_x][color->subsampling_y];
-break;
-case 10:
-ctx->format = color->mono_chrome ? AV_PIX_FMT_GRAY10
- : 
pix_fmts_10bit[color->subsampling_x][color->subsampling_y];
-break;
-case 12:
-ctx->format = color->mono_chrome ? AV_PIX_FMT_GRAY12
- : 
pix_fmts_12bit[color->subsampling_x][color->subsampling_y];
-break;
-}
-av_assert2(ctx->format != AV_PIX_FMT_NONE);
+ctx->width  = av1->frame_width;
+ctx->height = av1->frame_height;
+
+ctx->key_frame = frame->frame_type == AV1_FRAME_KEY;
+
+avctx->profile = seq->seq_profile;
+avctx->level   = seq->seq_level_idx[0];
+
+switch (frame->frame_type) {
+case AV1_FRAME_KEY:
+case AV1_FRAME_INTRA_ONLY:
+ctx->pict_type = AV_PICTURE_TYPE_I;
+break;
+case AV1_FRAME_INTER:
+ctx->pict_type = AV_PICTURE_TYPE_P;
+break;
+case AV1_FRAME_SWITCH:
+ctx->pict_type = AV_PICTURE_TYPE_SP;
+break;
+}
+ctx->picture_structure = AV_PICTURE_STRUCTURE_FRAME;
+
+switch (av1->bit_depth) {
+case 8:
+ctx->format = color->mono_chrome ? AV_PIX_FMT_GRAY8
+ : pix_fmts_8bit 
[color->subsampling_x][color->subsampling_y];
+break;
+case 10:
+ctx->format = color->mono_chrome ? AV_PIX_FMT_GRAY10
+ : 
pix_fmts_10bit[color->subsampling_x][color->subsampling_y];
+break;
+case 12:
+ctx->format = color->mono_chrome ? AV_PIX_FMT_GRAY12
+ : 
pix_fmts_12bit[color->subsampling_x][color->subsampling_y];
+break;
+}
+av_assert2(ctx->format != AV_PIX_FMT_NONE);
 
 end:
 s->cbc->log_ctx = NULL;
-- 
2.20.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 4/9] vp9_parser: Return stream properties

[Mark Thompson]

Rewrites the parser entirely, using CBS for header parsing.
---
 libavcodec/vp9_parser.c | 112 +---
 1 file changed, 82 insertions(+), 30 deletions(-)

diff --git a/libavcodec/vp9_parser.c b/libavcodec/vp9_parser.c
index c957a75667..6bf4f30e80 100644
--- a/libavcodec/vp9_parser.c
+++ b/libavcodec/vp9_parser.c
@@ -1,8 +1,5 @@
 /*
- * VP9 compatible video decoder
- *
- * Copyright (C) 2013 Ronald S. Bultje 
- * Copyright (C) 2013 Clément Bœsch 
+ * VP9 parser
  *
  * This file is part of FFmpeg.
  *
@@ -21,50 +18,105 @@
  * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
  */
 
-#include "libavutil/intreadwrite.h"
-#include "libavcodec/get_bits.h"
+#include "libavutil/avassert.h"
+#include "cbs.h"
+#include "cbs_vp9.h"
 #include "parser.h"
 
-static int parse(AVCodecParserContext *ctx,
- AVCodecContext *avctx,
- const uint8_t **out_data, int *out_size,
- const uint8_t *data, int size)
+typedef struct VP9ParserContext {
+CodedBitstreamContext *cbc;
+VP9RawFrameHeader frame_header;
+} VP9ParserContext;
+
+static const enum AVPixelFormat vp9_pix_fmts[3][2][2] = {
+{ // 8-bit.
+{ AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV440P },
+{ AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV420P },
+},
+{ // 10-bit.
+{ AV_PIX_FMT_YUV444P10, AV_PIX_FMT_YUV440P10 },
+{ AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV420P10 },
+},
+{ // 12-bit.
+{ AV_PIX_FMT_YUV444P12, AV_PIX_FMT_YUV440P12 },
+{ AV_PIX_FMT_YUV422P12, AV_PIX_FMT_YUV420P12 },
+},
+};
+
+static int vp9_parser_parse(AVCodecParserContext *ctx,
+AVCodecContext *avctx,
+const uint8_t **out_data, int *out_size,
+const uint8_t *data, int size)
 {
-GetBitContext gb;
-int res, profile, keyframe;
+VP9ParserContext *s = ctx->priv_data;
+const CodedBitstreamVP9Context *vp9 = s->cbc->priv_data;
+const VP9RawFrameHeader *fh;
+int err;
 
 *out_data = data;
 *out_size = size;
 
-if (!size || (res = init_get_bits8(&gb, data, size)) < 0)
-return size; // parsers can't return errors
-get_bits(&gb, 2); // frame marker
-profile  = get_bits1(&gb);
-profile |= get_bits1(&gb) << 1;
-if (profile == 3) profile += get_bits1(&gb);
-if (profile > 3)
-return size;
+ctx->key_frame = -1;
+ctx->pict_type = AV_PICTURE_TYPE_NONE;
+ctx->picture_structure = AV_PICTURE_STRUCTURE_UNKNOWN;
 
-avctx->profile = profile;
+if (!size)
+return 0;
 
-if (get_bits1(&gb)) {
-keyframe = 0;
-} else {
-keyframe  = !get_bits1(&gb);
+s->cbc->log_ctx = avctx;
+
+err = ff_cbs_parse_headers(s->cbc, &s->frame_header, data, size);
+if (err < 0) {
+av_log(avctx, AV_LOG_WARNING, "Failed to parse VP9 frame headers.\n");
+goto end;
 }
+fh = &s->frame_header;
 
-if (!keyframe) {
-ctx->pict_type = AV_PICTURE_TYPE_P;
-ctx->key_frame = 0;
-} else {
+avctx->profile = vp9->profile;
+avctx->level   = FF_LEVEL_UNKNOWN;
+
+ctx->width  = ctx->coded_width  = vp9->frame_width;
+ctx->height = ctx->coded_height = vp9->frame_height;
+
+if (fh->frame_type == VP9_KEY_FRAME) {
 ctx->pict_type = AV_PICTURE_TYPE_I;
 ctx->key_frame = 1;
+} else {
+ctx->pict_type = fh->intra_only ? AV_PICTURE_TYPE_I : 
AV_PICTURE_TYPE_P;
+ctx->key_frame = 0;
 }
 
+ctx->picture_structure = AV_PICTURE_STRUCTURE_FRAME;
+
+av_assert0(vp9->bit_depth == 8  ||
+   vp9->bit_depth == 10 ||
+   vp9->bit_depth == 12);
+
+ctx->format = vp9_pix_fmts[(vp9->bit_depth - 8) / 2]
+  [vp9->subsampling_x][vp9->subsampling_y];
+
+end:
+s->cbc->log_ctx = NULL;
+
 return size;
 }
 
+static av_cold int vp9_parser_init(AVCodecParserContext *ctx)
+{
+VP9ParserContext *s = ctx->priv_data;
+return ff_cbs_init(&s->cbc, AV_CODEC_ID_VP9, NULL);
+}
+
+static av_cold void vp9_parser_close(AVCodecParserContext *ctx)
+{
+VP9ParserContext *s = ctx->priv_data;
+ff_cbs_close(&s->cbc);
+}
+
 AVCodecParser ff_vp9_parser = {
 .codec_ids  = { AV_CODEC_ID_VP9 },
-.parser_parse   = parse,
+.priv_data_size = sizeof(VP9ParserContext),
+.parser_init= &vp9_parser_init,
+.parser_close   = &vp9_parser_close,
+.parser_parse   = &vp9_parser_parse,
 };
-- 
2.20.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 1/9] cbs: Add entrypoint for parser use

[Mark Thompson]

This can avoid copying due to lack of refcounting in parsers.
---
 libavcodec/cbs.c  |  9 +
 libavcodec/cbs.h  | 14 ++
 libavcodec/cbs_internal.h |  4 
 3 files changed, 27 insertions(+)

diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c
index c388be896b..ff98a1e8f4 100644
--- a/libavcodec/cbs.c
+++ b/libavcodec/cbs.c
@@ -280,6 +280,15 @@ int ff_cbs_read(CodedBitstreamContext *ctx,
 return cbs_read_fragment_content(ctx, frag);
 }
 
+int ff_cbs_parse_headers(CodedBitstreamContext *ctx, void *header,
+ const uint8_t *data, size_t size)
+{
+if (!ctx->codec->parse_headers)
+return AVERROR(ENOSYS);
+
+return ctx->codec->parse_headers(ctx, header, data, size);
+}
+
 
 int ff_cbs_write_fragment_data(CodedBitstreamContext *ctx,
CodedBitstreamFragment *frag)
diff --git a/libavcodec/cbs.h b/libavcodec/cbs.h
index 967dcd1468..be965ae258 100644
--- a/libavcodec/cbs.h
+++ b/libavcodec/cbs.h
@@ -278,6 +278,20 @@ int ff_cbs_read(CodedBitstreamContext *ctx,
 CodedBitstreamFragment *frag,
 const uint8_t *data, size_t size);
 
+/**
+ * Parse headers from a bitstream from a memory region, updating state
+ * but not storing intermediate results.
+ *
+ * If header is not NULL, the decomposition of the header of the last
+ * displayed unit in the stream is written there.  The type and size of
+ * this is codec-dependent.
+ *
+ * This is intended for use in parsers where only header parsing is
+ * required and the input is not refcounted.
+ */
+int ff_cbs_parse_headers(CodedBitstreamContext *ctx, void *header,
+ const uint8_t *data, size_t size);
+
 
 /**
  * Write the content of the fragment to its own internal buffer.
diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h
index 53f2e5d187..9783555292 100644
--- a/libavcodec/cbs_internal.h
+++ b/libavcodec/cbs_internal.h
@@ -55,6 +55,10 @@ typedef struct CodedBitstreamType {
 
 // Free the codec internal state.
 void (*close)(CodedBitstreamContext *ctx);
+
+// Parse headers only.
+int (*parse_headers)(CodedBitstreamContext *ctx, void *header,
+ const uint8_t *data, size_t size);
 } CodedBitstreamType;
 
 
-- 
2.20.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 7/9] cbs_av1: Fill context information for show-existing-frame

[Mark Thompson]

---
 libavcodec/cbs_av1_syntax_template.c | 16 
 1 file changed, 16 insertions(+)

diff --git a/libavcodec/cbs_av1_syntax_template.c 
b/libavcodec/cbs_av1_syntax_template.c
index 76eb90b279..a9bf78e4ad 100644
--- a/libavcodec/cbs_av1_syntax_template.c
+++ b/libavcodec/cbs_av1_syntax_template.c
@@ -1146,6 +1146,22 @@ static int 
FUNC(uncompressed_header)(CodedBitstreamContext *ctx, RWContext *rw,
 
 fb(3, frame_to_show_map_idx);
 frame = &priv->ref[current->frame_to_show_map_idx];
+if (!frame->valid) {
+av_log(ctx->log_ctx, AV_LOG_ERROR,
+   "Missing reference frame needed to show existing frame "
+   "(frame_to_show_map_idx = %d).\n",
+   current->frame_to_show_map_idx);
+return AVERROR_INVALIDDATA;
+}
+
+priv->bit_depth  = frame->bit_depth;
+priv->frame_width= frame->frame_width;
+priv->frame_height   = frame->frame_height;
+priv->upscaled_width = frame->upscaled_width;
+priv->render_width   = frame->render_width;
+priv->render_height  = frame->render_height;
+
+infer(frame_type, frame->frame_type);
 
 if (seq->decoder_model_info_present_flag &&
 !seq->timing_info.equal_picture_interval) {
-- 
2.20.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 5/9] lavc/qsvdec: Add VP9 decoder support

[Mark Thompson]

From: Zhong Li 

VP9 decoder is supported on Intel kabyLake+ platforms with MSDK Version 1.19+

Signed-off-by: Zhong Li 
---
On 20/03/2019 14:41, Li, Zhong wrote:
> Yes, QSV is a marketing name which is no equal to libmfx/MSDK.
> But would be better to keep consistent with others, such as "Intel 
> QSV-accelerated VP8 video decoding" in pervious changelog?

I don't think so?  VP9 decoding with the QSV hardware is already supported, 
this only adds the additional option of using libmfx to access the same thing 
as well.


 Changelog |  1 +
 configure |  6 ++
 libavcodec/allcodecs.c|  1 +
 libavcodec/qsv.c  |  5 +
 libavcodec/qsvdec_other.c | 31 ++-
 5 files changed, 43 insertions(+), 1 deletion(-)

diff --git a/Changelog b/Changelog
index cd4ed54f2c..d838873dd4 100644
--- a/Changelog
+++ b/Changelog
@@ -21,6 +21,7 @@ version :
 - Support decoding of HEVC 4:4:4 content in nvdec and cuviddec
 - removed libndi-newtek
 - agm decoder
+- Intel libmfx VP9 decoding
 
 
 version 4.1:
diff --git a/configure b/configure
index f6123f53e5..60cbbaf39e 100755
--- a/configure
+++ b/configure
@@ -3058,6 +3058,8 @@ vp8_v4l2m2m_decoder_deps="v4l2_m2m vp8_v4l2_m2m"
 vp8_v4l2m2m_encoder_deps="v4l2_m2m vp8_v4l2_m2m"
 vp9_cuvid_decoder_deps="cuvid"
 vp9_mediacodec_decoder_deps="mediacodec"
+vp9_qsv_decoder_deps="MFX_CODEC_VP9"
+vp9_qsv_decoder_select="qsvdec"
 vp9_rkmpp_decoder_deps="rkmpp"
 vp9_vaapi_encoder_deps="VAEncPictureParameterBufferVP9"
 vp9_vaapi_encoder_select="vaapi_encode"
@@ -6157,6 +6159,10 @@ enabled liblensfun&& require_pkg_config 
liblensfun lensfun lensfun.h lf_
 # can find the libraries and headers through other means.
 enabled libmfx&& { check_pkg_config libmfx libmfx "mfx/mfxvideo.h" 
MFXInit ||
{ require libmfx "mfx/mfxvideo.h" MFXInit 
"-llibmfx $advapi32_extralibs" && warn "using libmfx without pkg-config"; } }
+if enabled libmfx; then
+   check_cc MFX_CODEC_VP9 "mfx/mfxvp9.h mfx/mfxstructures.h" "MFX_CODEC_VP9"
+fi
+
 enabled libmodplug&& require_pkg_config libmodplug libmodplug 
libmodplug/modplug.h ModPlug_Load
 enabled libmp3lame&& require "libmp3lame >= 3.98.3" lame/lame.h 
lame_set_VBR_quality -lmp3lame $libm_extralibs
 enabled libmysofa && { check_pkg_config libmysofa libmysofa mysofa.h 
mysofa_load ||
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index 71fd74a07e..1f0233d971 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -776,6 +776,7 @@ extern AVCodec ff_vp8_v4l2m2m_encoder;
 extern AVCodec ff_vp8_vaapi_encoder;
 extern AVCodec ff_vp9_cuvid_decoder;
 extern AVCodec ff_vp9_mediacodec_decoder;
+extern AVCodec ff_vp9_qsv_decoder;
 extern AVCodec ff_vp9_vaapi_encoder;
 
 // The iterate API is not usable with ossfuzz due to the excessive size of 
binaries created
diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c
index bb0d79588c..389fdcff41 100644
--- a/libavcodec/qsv.c
+++ b/libavcodec/qsv.c
@@ -60,6 +60,11 @@ int ff_qsv_codec_id_to_mfx(enum AVCodecID codec_id)
 #endif
 case AV_CODEC_ID_MJPEG:
 return MFX_CODEC_JPEG;
+#if QSV_VERSION_ATLEAST(1, 19)
+case AV_CODEC_ID_VP9:
+return MFX_CODEC_VP9;
+#endif
+
 default:
 break;
 }
diff --git a/libavcodec/qsvdec_other.c b/libavcodec/qsvdec_other.c
index 03251d2c85..d7a6d79f63 100644
--- a/libavcodec/qsvdec_other.c
+++ b/libavcodec/qsvdec_other.c
@@ -1,5 +1,5 @@
 /*
- * Intel MediaSDK QSV based MPEG-2, VC-1 and VP8 decoders
+ * Intel MediaSDK QSV based MPEG-2, VC-1, VP8 and VP9 decoders
  *
  * copyright (c) 2015 Anton Khirnov
  *
@@ -255,3 +255,32 @@ AVCodec ff_vp8_qsv_decoder = {
 .wrapper_name   = "qsv",
 };
 #endif
+
+#if CONFIG_VP9_QSV_DECODER
+static const AVClass vp9_qsv_class = {
+.class_name = "vp9_qsv",
+.item_name  = av_default_item_name,
+.option = options,
+.version= LIBAVUTIL_VERSION_INT,
+};
+
+AVCodec ff_vp9_qsv_decoder = {
+.name   = "vp9_qsv",
+.long_name  = NULL_IF_CONFIG_SMALL("VP9 video (Intel Quick Sync Video 
acceleration)"),
+.priv_data_size = sizeof(QSVOtherContext),
+.type   = AVMEDIA_TYPE_VIDEO,
+.id = AV_CODEC_ID_VP9,
+.init   = qsv_decode_init,
+.decode = qsv_decode_frame,
+.flush  = qsv_decode_flush,
+.close  = qsv_decode_close,
+.capabilities   = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_DR1 | 
AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_HYBRID,
+.priv_class = &vp9_qsv_class,
+.pix_fmts   = (const enum AVPixelFormat[]){ AV_PIX_FMT_NV12,
+AV_PIX_FMT_P010,
+AV_PIX_FMT_QSV,
+AV_PIX_FMT_NONE },
+.hw_configs = ff_qsv_hw_configs,
+.wrapper_name   = "qsv",
+};
+#endif
-- 
2.20.1

[FFmpeg-devel] [PATCH v2 8/9] av1_parser: Use CBS parser interface

[Mark Thompson]

This simplifies the parser and improves performance by reducing the number
of allocations and eliminating redundant copies.
---
 libavcodec/av1_parser.c | 63 +
 1 file changed, 13 insertions(+), 50 deletions(-)

diff --git a/libavcodec/av1_parser.c b/libavcodec/av1_parser.c
index b916608d65..4a743d92d4 100644
--- a/libavcodec/av1_parser.c
+++ b/libavcodec/av1_parser.c
@@ -27,7 +27,7 @@
 
 typedef struct AV1ParseContext {
 CodedBitstreamContext *cbc;
-CodedBitstreamFragment temporal_unit;
+AV1RawFrameHeader frame_header;
 int parsed_extradata;
 } AV1ParseContext;
 
@@ -50,8 +50,10 @@ static int av1_parser_parse(AVCodecParserContext *ctx,
 const uint8_t *data, int size)
 {
 AV1ParseContext *s = ctx->priv_data;
-CodedBitstreamFragment *td = &s->temporal_unit;
 CodedBitstreamAV1Context *av1 = s->cbc->priv_data;
+AV1RawSequenceHeader *seq;
+AV1RawColorConfig *color;
+AV1RawFrameHeader *frame;
 int ret;
 
 *out_data = data;
@@ -66,67 +68,35 @@ static int av1_parser_parse(AVCodecParserContext *ctx,
 if (avctx->extradata_size && !s->parsed_extradata) {
 s->parsed_extradata = 1;
 
-ret = ff_cbs_read(s->cbc, td, avctx->extradata, avctx->extradata_size);
-if (ret < 0) {
+ret = ff_cbs_parse_headers(s->cbc, NULL,
+   avctx->extradata, avctx->extradata_size);
+if (ret < 0)
 av_log(avctx, AV_LOG_WARNING, "Failed to parse extradata.\n");
-}
-
-ff_cbs_fragment_reset(s->cbc, td);
 }
 
-ret = ff_cbs_read(s->cbc, td, data, size);
+ret = ff_cbs_parse_headers(s->cbc, &s->frame_header, data, size);
 if (ret < 0) {
 av_log(avctx, AV_LOG_ERROR, "Failed to parse temporal unit.\n");
 goto end;
 }
+frame = &s->frame_header;
 
 if (!av1->sequence_header) {
 av_log(avctx, AV_LOG_ERROR, "No sequence header available\n");
 goto end;
 }
+seq = av1->sequence_header;
+color = &seq->color_config;
 
-for (int i = 0; i < td->nb_units; i++) {
-CodedBitstreamUnit *unit = &td->units[i];
-AV1RawOBU *obu = unit->content;
-AV1RawSequenceHeader *seq = av1->sequence_header;
-AV1RawColorConfig *color = &seq->color_config;
-AV1RawFrameHeader *frame;
-int frame_type;
-
-if (unit->type == AV1_OBU_FRAME)
-frame = &obu->obu.frame.header;
-else if (unit->type == AV1_OBU_FRAME_HEADER)
-frame = &obu->obu.frame_header;
-else
-continue;
-
-if (frame->show_existing_frame) {
-AV1ReferenceFrameState *ref = 
&av1->ref[frame->frame_to_show_map_idx];
-
-if (!ref->valid) {
-av_log(avctx, AV_LOG_ERROR, "Invalid reference frame\n");
-goto end;
-}
-
-ctx->width  = ref->frame_width;
-ctx->height = ref->frame_height;
-frame_type  = ref->frame_type;
-
-ctx->key_frame = 0;
-} else if (!frame->show_frame) {
-continue;
-} else {
 ctx->width  = av1->frame_width;
 ctx->height = av1->frame_height;
-frame_type  = frame->frame_type;
 
-ctx->key_frame = frame_type == AV1_FRAME_KEY;
-}
+ctx->key_frame = frame->frame_type == AV1_FRAME_KEY;
 
 avctx->profile = seq->seq_profile;
 avctx->level   = seq->seq_level_idx[0];
 
-switch (frame_type) {
+switch (frame->frame_type) {
 case AV1_FRAME_KEY:
 case AV1_FRAME_INTRA_ONLY:
 ctx->pict_type = AV_PICTURE_TYPE_I;
@@ -155,11 +125,8 @@ static int av1_parser_parse(AVCodecParserContext *ctx,
 break;
 }
 av_assert2(ctx->format != AV_PIX_FMT_NONE);
-}
 
 end:
-ff_cbs_fragment_reset(s->cbc, td);
-
 s->cbc->log_ctx = NULL;
 
 return size;
@@ -182,9 +149,6 @@ static av_cold int av1_parser_init(AVCodecParserContext 
*ctx)
 if (ret < 0)
 return ret;
 
-s->cbc->decompose_unit_types= (CodedBitstreamUnitType 
*)decompose_unit_types;
-s->cbc->nb_decompose_unit_types = FF_ARRAY_ELEMS(decompose_unit_types);
-
 return 0;
 }
 
@@ -192,7 +156,6 @@ static void av1_parser_close(AVCodecParserContext *ctx)
 {
 AV1ParseContext *s = ctx->priv_data;
 
-ff_cbs_fragment_free(s->cbc, &s->temporal_unit);
 ff_cbs_close(&s->cbc);
 }
 
-- 
2.20.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 6/9] cbs_av1: Implement parser entrypoint

[Mark Thompson]

---
Unsure about this one - while the patch is short, it's rather invasive in a 
pretty ugly way with how it abuses the read call.  It will still do allocations 
for the decomposition because of that, even though we don't really want them.

Any ideas welcome.


 libavcodec/cbs_av1.c | 89 +++-
 1 file changed, 79 insertions(+), 10 deletions(-)

diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c
index 02f168b58d..0b54b5820c 100644
--- a/libavcodec/cbs_av1.c
+++ b/libavcodec/cbs_av1.c
@@ -776,13 +776,15 @@ static int cbs_av1_get_relative_dist(const 
AV1RawSequenceHeader *seq,
 #undef byte_alignment
 
 
-static int cbs_av1_split_fragment(CodedBitstreamContext *ctx,
-  CodedBitstreamFragment *frag,
-  int header)
+typedef int (*cbs_av1_split_obu_callback)(CodedBitstreamContext *ctx,
+  void *priv, int obu_type,
+  const uint8_t *data, size_t size);
+
+static int cbs_av1_split_obus(CodedBitstreamContext *ctx,
+  void *priv, cbs_av1_split_obu_callback cb,
+  const uint8_t *data, size_t size)
 {
 GetBitContext gbc;
-uint8_t *data;
-size_t size;
 uint64_t obu_length;
 int pos, err, trace;
 
@@ -790,9 +792,6 @@ static int cbs_av1_split_fragment(CodedBitstreamContext 
*ctx,
 trace = ctx->trace_enable;
 ctx->trace_enable = 0;
 
-data = frag->data;
-size = frag->data_size;
-
 if (INT_MAX / 8 < size) {
 av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid fragment: "
"too large (%"SIZE_SPECIFIER" bytes).\n", size);
@@ -837,8 +836,7 @@ static int cbs_av1_split_fragment(CodedBitstreamContext 
*ctx,
 goto fail;
 }
 
-err = ff_cbs_insert_unit_data(ctx, frag, -1, header.obu_type,
-  data, obu_length, frag->data_ref);
+err = cb(ctx, priv, header.obu_type, data, obu_length);
 if (err < 0)
 goto fail;
 
@@ -852,6 +850,23 @@ fail:
 return err;
 }
 
+static int cbs_av1_insert_obu(CodedBitstreamContext *ctx,
+  void *priv, int obu_type,
+  const uint8_t *data, size_t size)
+{
+CodedBitstreamFragment *frag = priv;
+return ff_cbs_insert_unit_data(ctx, frag, -1, obu_type,
+   (uint8_t*)data, size, frag->data_ref);
+}
+
+static int cbs_av1_split_fragment(CodedBitstreamContext *ctx,
+  CodedBitstreamFragment *frag,
+  int header)
+{
+return cbs_av1_split_obus(ctx, frag, &cbs_av1_insert_obu,
+  frag->data, frag->data_size);
+}
+
 static void cbs_av1_free_tile_data(AV1RawTileData *td)
 {
 av_buffer_unref(&td->data_ref);
@@ -895,6 +910,12 @@ static int cbs_av1_ref_tile_data(CodedBitstreamContext 
*ctx,
 {
 int pos;
 
+if (!unit->data_ref) {
+// Not refcounted - only parsing headers, so tile data will
+// not be needed.
+return 0;
+}
+
 pos = get_bits_count(gbc);
 if (pos >= 8 * unit->data_size) {
 av_log(ctx->log_ctx, AV_LOG_ERROR, "Bitstream ended before "
@@ -1324,6 +1345,52 @@ static void cbs_av1_close(CodedBitstreamContext *ctx)
 av_freep(&priv->write_buffer);
 }
 
+static int cbs_av1_parse_obu(CodedBitstreamContext *ctx,
+ void *priv, int obu_type,
+ const uint8_t *data, size_t data_size)
+{
+CodedBitstreamUnit unit;
+int err;
+
+// These OBU types will not affect parsing.
+if (obu_type == AV1_OBU_METADATA  ||
+obu_type == AV1_OBU_TILE_LIST ||
+obu_type == AV1_OBU_PADDING)
+return 0;
+
+unit = (CodedBitstreamUnit) {
+.type  = obu_type,
+.data  = (uint8_t*)data,
+.data_size = data_size,
+};
+
+err = cbs_av1_read_unit(ctx, &unit);
+if (err >= 0 && priv) {
+AV1RawOBU *obu = unit.content;
+switch (obu->header.obu_type) {
+case AV1_OBU_FRAME_HEADER:
+case AV1_OBU_REDUNDANT_FRAME_HEADER:
+memcpy(priv, &obu->obu.frame_header,
+   sizeof(obu->obu.frame_header));
+break;
+case AV1_OBU_FRAME:
+memcpy(priv, &obu->obu.frame.header,
+   sizeof(obu->obu.frame.header));
+break;
+}
+}
+
+av_buffer_unref(&unit.content_ref);
+
+return err;
+}
+
+static int cbs_av1_parse_headers(CodedBitstreamContext *ctx, void *header,
+ const uint8_t *data, size_t size)
+{
+return cbs_av1_split_obus(ctx, header, &cbs_av1_parse_obu, data, size);
+}
+
 const CodedBitstreamType ff_cbs_type_av1 = {
 .codec_id  = AV_CODEC_ID_AV1,
 
@@ -1335,4 +1402,6 @@ const CodedBitstreamType ff_cbs_type_av1 = {
 .assemb

[FFmpeg-devel] [PATCH v2 3/9] cbs_vp9: Fill context information for show-existing-frame

[Mark Thompson]

The frame being shown could have different properties to the last-decoded
one.
---
 libavcodec/cbs_vp9.h |  3 +++
 libavcodec/cbs_vp9_syntax_template.c | 23 +++
 2 files changed, 26 insertions(+)

diff --git a/libavcodec/cbs_vp9.h b/libavcodec/cbs_vp9.h
index 4c9b2f880d..c637c0d346 100644
--- a/libavcodec/cbs_vp9.h
+++ b/libavcodec/cbs_vp9.h
@@ -182,11 +182,14 @@ typedef struct VP9RawSuperframe {
 } VP9RawSuperframe;
 
 typedef struct VP9ReferenceFrameState {
+int valid;
 int frame_width;// RefFrameWidth
 int frame_height;   // RefFrameHeight
 int subsampling_x;  // RefSubsamplingX
 int subsampling_y;  // RefSubsamplingY
 int bit_depth;  // RefBitDepth
+int frame_type;
+int intra_only;
 } VP9ReferenceFrameState;
 
 typedef struct CodedBitstreamVP9Context {
diff --git a/libavcodec/cbs_vp9_syntax_template.c 
b/libavcodec/cbs_vp9_syntax_template.c
index 898cede329..811f5c12ce 100644
--- a/libavcodec/cbs_vp9_syntax_template.c
+++ b/libavcodec/cbs_vp9_syntax_template.c
@@ -290,10 +290,30 @@ static int 
FUNC(uncompressed_header)(CodedBitstreamContext *ctx, RWContext *rw,
 
 f(1, show_existing_frame);
 if (current->show_existing_frame) {
+VP9ReferenceFrameState *ref;
+
 f(3, frame_to_show_map_idx);
+ref = &vp9->ref[current->frame_to_show_map_idx];
+if (!ref->valid) {
+av_log(ctx->log_ctx, AV_LOG_ERROR,
+   "Missing reference frame needed to show existing frame "
+   "(frame_to_show_map_idx = %d).\n",
+   current->frame_to_show_map_idx);
+return AVERROR_INVALIDDATA;
+}
+
+vp9->frame_width   = ref->frame_width;
+vp9->frame_height  = ref->frame_height;
+vp9->subsampling_x = ref->subsampling_x;
+vp9->subsampling_y = ref->subsampling_y;
+vp9->bit_depth = ref->bit_depth;
+infer(frame_type, ref->frame_type);
+infer(intra_only, ref->intra_only);
+
 infer(header_size_in_bytes, 0);
 infer(refresh_frame_flags,  0x00);
 infer(loop_filter_level,0);
+
 return 0;
 }
 
@@ -374,11 +394,14 @@ static int 
FUNC(uncompressed_header)(CodedBitstreamContext *ctx, RWContext *rw,
 for (i = 0; i < VP9_NUM_REF_FRAMES; i++) {
 if (current->refresh_frame_flags & (1 << i)) {
 vp9->ref[i] = (VP9ReferenceFrameState) {
+.valid  = 1,
 .frame_width= vp9->frame_width,
 .frame_height   = vp9->frame_height,
 .subsampling_x  = vp9->subsampling_x,
 .subsampling_y  = vp9->subsampling_y,
 .bit_depth  = vp9->bit_depth,
+.frame_type = current->frame_type,
+.intra_only = current->intra_only,
 };
 }
 }
-- 
2.20.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] libavutil/hwcontext_opencl.c: fix bug in `opencl_get_plane_format`

[Mark Thompson]

On 06/04/2019 00:05, Jarek Samic wrote:
> The `opencl_get_plane_format` function was incorrectly determining the
> value used to set the image channel order. This resulted in all RGB
> pixel formats being set to the `CL_RGBA` pixel format, regardless of
> whether or not they actually *were* RGBA.
> 
> This patch fixes the issue by using the `offset` field on components
> rather than the loop index to determine the value of `order` for RGB
> pixel formats (and leaves the formula to determine `order` the same for
> other formats so as not to break those cases).
> 
> Signed-off-by: Jarek Samic 
> ---
> I'm including this in the email (but not the commit description) to
> make this patch easier to review. In order to make sure I was fixing
> the bug and not messing up channel order mapping for any of the other
> formats, I set up a few quick print statements within the function
> after the `order` value was set.
> 
> Here's the output without this patch:
> 
> ```
> [AVHWDeviceContext @ _] Maximum supported image size 16384x16384.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv420p, plane: 0
> [opencl_get_plane_format] order_num: 2, pixel_format: yuv420p, plane: 1
> [opencl_get_plane_format] order_num: 3, pixel_format: yuv420p, plane: 2
> [AVHWDeviceContext @ _] Format yuv420p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv422p, plane: 0
> [opencl_get_plane_format] order_num: 2, pixel_format: yuv422p, plane: 1
> [opencl_get_plane_format] order_num: 3, pixel_format: yuv422p, plane: 2
> [AVHWDeviceContext @ _] Format yuv422p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv444p, plane: 0
> [opencl_get_plane_format] order_num: 2, pixel_format: yuv444p, plane: 1
> [opencl_get_plane_format] order_num: 3, pixel_format: yuv444p, plane: 2
> [AVHWDeviceContext @ _] Format yuv444p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv410p, plane: 0
> [opencl_get_plane_format] order_num: 2, pixel_format: yuv410p, plane: 1
> [opencl_get_plane_format] order_num: 3, pixel_format: yuv410p, plane: 2
> [AVHWDeviceContext @ _] Format yuv410p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv411p, plane: 0
> [opencl_get_plane_format] order_num: 2, pixel_format: yuv411p, plane: 1
> [opencl_get_plane_format] order_num: 3, pixel_format: yuv411p, plane: 2
> [AVHWDeviceContext @ _] Format yuv411p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: gray, plane: 0
> [AVHWDeviceContext @ _] Format gray supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj420p, plane: 0
> [opencl_get_plane_format] order_num: 2, pixel_format: yuvj420p, plane: 1
> [opencl_get_plane_format] order_num: 3, pixel_format: yuvj420p, plane: 2
> [AVHWDeviceContext @ _] Format yuvj420p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj422p, plane: 0
> [opencl_get_plane_format] order_num: 2, pixel_format: yuvj422p, plane: 1
> [opencl_get_plane_format] order_num: 3, pixel_format: yuvj422p, plane: 2
> [AVHWDeviceContext @ _] Format yuvj422p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj444p, plane: 0
> [opencl_get_plane_format] order_num: 2, pixel_format: yuvj444p, plane: 1
> [opencl_get_plane_format] order_num: 3, pixel_format: yuvj444p, plane: 2
> [AVHWDeviceContext @ _] Format yuvj444p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: nv12, plane: 0
> [opencl_get_plane_format] order_num: 23, pixel_format: nv12, plane: 1
> [AVHWDeviceContext @ _] Format nv12 supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: nv21, plane: 0
> [opencl_get_plane_format] order_num: 23, pixel_format: nv21, plane: 1
> [AVHWDeviceContext @ _] Format nv21 supported.
> [opencl_get_plane_format] order_num: 1234, pixel_format: argb, plane: 0
> [AVHWDeviceContext @ _] Format argb supported.
> [opencl_get_plane_format] order_num: 1234, pixel_format: rgba, plane: 0
> [AVHWDeviceContext @ _] Format rgba supported.
> [opencl_get_plane_format] order_num: 1234, pixel_format: abgr, plane: 0
> [AVHWDeviceContext @ _] Format abgr supported.
> [opencl_get_plane_format] order_num: 1234, pixel_format: bgra, plane: 0
> [AVHWDeviceContext @ _] Format bgra supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: gray16le, plane: 0
> [AVHWDeviceContext @ _] Format gray16le supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv440p, plane: 0
> [opencl_get_plane_format] order_num: 2, pixel_format: yuv440p, plane: 1
> [opencl_get_plane_format] order_num: 3, pixel_format: yuv440p, plane: 2
> [AVHWDeviceContext @ _] Format yuv440p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj440p, plane: 0
> [opencl_get_plane_format] order_num: 2, pixel_format: yuvj440p, plane: 1
> [opencl_get_plane_format] order_num: 3, pixel_format: yuvj440p, plane: 2
> [AVHWDeviceContext @ _] Format yuvj440p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuva420p, plane: 0
> 

Re: [FFmpeg-devel] [PATCH] libavutil/hwcontext_opencl.c: fix bug in `opencl_get_plane_format`

[Mark Thompson]

On 07/04/2019 23:40, Cld fire wrote:
>>
>> Does anything go wrong if you unconditionally add comp->offset / (depth /
>> 8)?
> 
> 
> Yes. Occasionally there is a depth value less than 8:
> 
> [opencl_get_plane_format] depth is 2 < 8 for format: rgb8 on plane: 0
> [AVHWDeviceContext @ _] Format yuva420p supported.
> [opencl_get_plane_format] depth is 5 < 8 for format: rgb565be on plane: 0
> [opencl_get_plane_format] depth is 5 < 8 for format: rgb565le on plane: 0
> 
> Which causes an FPE (divide-by-zero). If I throw in an if statement to weed
> out those cases, that method of calculating the number results in the
> following output:
> 
> [AVHWDeviceContext @ _] Maximum supported image size 16384x16384.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv420p, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv420p, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv420p, plane: 2
> [AVHWDeviceContext @ _] Format yuv420p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv422p, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv422p, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv422p, plane: 2
> [AVHWDeviceContext @ _] Format yuv422p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv444p, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv444p, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv444p, plane: 2
> [AVHWDeviceContext @ _] Format yuv444p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv410p, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv410p, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv410p, plane: 2
> [AVHWDeviceContext @ _] Format yuv410p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv411p, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv411p, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv411p, plane: 2
> [AVHWDeviceContext @ _] Format yuv411p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: gray, plane: 0
> [AVHWDeviceContext @ _] Format gray supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj420p, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj420p, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj420p, plane: 2
> [AVHWDeviceContext @ _] Format yuvj420p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj422p, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj422p, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj422p, plane: 2
> [AVHWDeviceContext @ _] Format yuvj422p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj444p, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj444p, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj444p, plane: 2
> [AVHWDeviceContext @ _] Format yuvj444p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: nv12, plane: 0
> [opencl_get_plane_format] order_num: 12, pixel_format: nv12, plane: 1
> [AVHWDeviceContext @ _] Format nv12 supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: nv21, plane: 0
> [opencl_get_plane_format] order_num: 21, pixel_format: nv21, plane: 1
> [opencl_get_plane_format] order_num: 2341, pixel_format: argb, plane: 0
> [opencl_get_plane_format] order_num: 1234, pixel_format: rgba, plane: 0
> [AVHWDeviceContext @ _] Format rgba supported.
> [opencl_get_plane_format] order_num: 4321, pixel_format: abgr, plane: 0
> [opencl_get_plane_format] order_num: 3214, pixel_format: bgra, plane: 0
> [AVHWDeviceContext @ _] Format bgra supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: gray16le, plane: 0
> [AVHWDeviceContext @ _] Format gray16le supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv440p, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv440p, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv440p, plane: 2
> [AVHWDeviceContext @ _] Format yuv440p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj440p, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj440p, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel_format: yuvj440p, plane: 2
> [AVHWDeviceContext @ _] Format yuvj440p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuva420p, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuva420p, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel_format: yuva420p, plane: 2
> [opencl_get_plane_format] order_num: 1, pixel_format: yuva420p, plane: 3
> [AVHWDeviceContext @ _] Format yuva420p supported.
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv420p16le, plane: 0
> [opencl_get_plane_format] order_num: 1, pixel_format: yuv420p16le, plane: 1
> [opencl_get_plane_format] order_num: 1, pixel

Re: [FFmpeg-devel] [PATCH v2] libavutil/hwcontext_opencl.c: fix bug in `opencl_get_plane_format`

[Mark Thompson]

On 08/04/2019 03:01, Jarek Samic wrote:
> The `opencl_get_plane_format` function was incorrectly determining the
> value used to set the image channel order. This resulted in all RGB
> pixel formats being set to the `CL_RGBA` pixel format, regardless of
> whether or not they actually *were* RGBA.
> 
> This patch fixes the issue by using the `offset` and depth of components
> rather than the loop index to determine the value of `order`.
> 
> Signed-off-by: Jarek Samic 
> ---
> I have updated this patch in response to the comments on the first version.
> RGB is no longer special-cased, the 2, 3, and 4 mappings to `CL_R` have been
> removed, and the mapping for `CL_ARGB` has been changed to the correct value.
> 
>  libavutil/hwcontext_opencl.c | 8 +++-
>  1 file changed, 3 insertions(+), 5 deletions(-)
> 
> diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c
> index b116c5b708..593de1ca41 100644
> --- a/libavutil/hwcontext_opencl.c
> +++ b/libavutil/hwcontext_opencl.c
> @@ -1419,8 +1419,9 @@ static int opencl_get_plane_format(enum AVPixelFormat 
> pixfmt,
>  // from the same component.
>  if (step && comp->step != step)
>  return AVERROR(EINVAL);
> -order = order * 10 + c + 1;
> +
>  depth = comp->depth;
> +order = order * 10 + comp->offset / ((depth + 7) / 8) + 1;
>  step  = comp->step;
>  alpha = (desc->flags & AV_PIX_FMT_FLAG_ALPHA &&
>   c == desc->nb_components - 1);

This part LGTM, I can split it off and apply it on its own if you like.

> @@ -1456,14 +1457,11 @@ static int opencl_get_plane_format(enum AVPixelFormat 
> pixfmt,
>  case order: image_format->image_channel_order = type; break;
>  switch (order) {
>  CHANNEL_ORDER(1,CL_R);
> -CHANNEL_ORDER(2,CL_R);
> -CHANNEL_ORDER(3,CL_R);
> -CHANNEL_ORDER(4,CL_R);
>  CHANNEL_ORDER(12,   CL_RG);
>  CHANNEL_ORDER(23,   CL_RG);

23 should be gone too, I think?

>  CHANNEL_ORDER(1234, CL_RGBA);
> +CHANNEL_ORDER(2341, CL_ARGB);
>  CHANNEL_ORDER(3214, CL_BGRA);
> -CHANNEL_ORDER(4123, CL_ARGB);

I'm not sure I believe this part:

1 = R
2 = G
3 = B
4 = A

gives

RGBA -> 1234
BGRA -> 3214
ARGB -> 4123
ABGR -> 4321

The others match, so why would ARGB be different?  2341 should be GBAR.

(Can you try this with multiple ARGB sources or OpenCL ICDs?  Maybe there is a 
bug somewhere else...)

>  #ifdef CL_ABGR
>  CHANNEL_ORDER(4321, CL_ABGR);
>  #endif
> 

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi: add nlmeans_opencl filter

[Mark Thompson]

On 01/04/2019 08:52, Ruiling Song wrote:
> Signed-off-by: Ruiling Song 
> ---
> This filter runs about 2x faster on integrated GPU than nlmeans on my Skylake 
> CPU.
> Anybody like to give some comments?

Nice!

>  configure   |   1 +
>  doc/filters.texi|   4 +
>  libavfilter/Makefile|   1 +
>  libavfilter/allfilters.c|   1 +
>  libavfilter/opencl/nlmeans.cl   | 108 +
>  libavfilter/opencl_source.h |   1 +
>  libavfilter/vf_nlmeans_opencl.c | 390 
>  7 files changed, 506 insertions(+)
>  create mode 100644 libavfilter/opencl/nlmeans.cl
>  create mode 100644 libavfilter/vf_nlmeans_opencl.c
> 
> diff --git a/configure b/configure
> index f6123f53e5..a233512491 100755
> --- a/configure
> +++ b/configure
> @@ -3460,6 +3460,7 @@ mpdecimate_filter_select="pixelutils"
>  minterpolate_filter_select="scene_sad"
>  mptestsrc_filter_deps="gpl"
>  negate_filter_deps="lut_filter"
> +nlmeans_opencl_filter_deps="opencl"
>  nnedi_filter_deps="gpl"
>  ocr_filter_deps="libtesseract"
>  ocv_filter_deps="libopencv"
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 867607d870..21c2c1a4b5 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -19030,6 +19030,10 @@ Apply erosion filter with threshold0 set to 30, 
> threshold1 set 40, threshold2 se
>  @end example
>  @end itemize
>  
> +@section nlmeans_opencl
> +
> +Non-local Means denoise filter through OpenCL, this filter accepts same 
> options as @ref{nlmeans}.
> +
>  @section overlay_opencl
>  
>  Overlay one video on top of another.
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index fef6ec5c55..92039bfdcf 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -291,6 +291,7 @@ OBJS-$(CONFIG_MIX_FILTER)+= vf_mix.o
>  OBJS-$(CONFIG_MPDECIMATE_FILTER) += vf_mpdecimate.o
>  OBJS-$(CONFIG_NEGATE_FILTER) += vf_lut.o
>  OBJS-$(CONFIG_NLMEANS_FILTER)+= vf_nlmeans.o
> +OBJS-$(CONFIG_NLMEANS_OPENCL_FILTER) += vf_nlmeans_opencl.o opencl.o 
> opencl/nlmeans.o
>  OBJS-$(CONFIG_NNEDI_FILTER)  += vf_nnedi.o
>  OBJS-$(CONFIG_NOFORMAT_FILTER)   += vf_format.o
>  OBJS-$(CONFIG_NOISE_FILTER)  += vf_noise.o
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index c51ae0f3c7..2a6390c92d 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -277,6 +277,7 @@ extern AVFilter ff_vf_mix;
>  extern AVFilter ff_vf_mpdecimate;
>  extern AVFilter ff_vf_negate;
>  extern AVFilter ff_vf_nlmeans;
> +extern AVFilter ff_vf_nlmeans_opencl;
>  extern AVFilter ff_vf_nnedi;
>  extern AVFilter ff_vf_noformat;
>  extern AVFilter ff_vf_noise;
> diff --git a/libavfilter/opencl/nlmeans.cl b/libavfilter/opencl/nlmeans.cl
> new file mode 100644
> index 00..dcb04834ca
> --- /dev/null
> +++ b/libavfilter/opencl/nlmeans.cl
> @@ -0,0 +1,108 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 
> USA
> + */
> +
> +const sampler_t sampler = (CLK_NORMALIZED_COORDS_FALSE |
> +   CLK_ADDRESS_CLAMP_TO_EDGE   |
> +   CLK_FILTER_NEAREST);
> +
> +kernel void horiz_sum(__global uint4 *ii,
> +  __read_only image2d_t src,
> +  int width,
> +  int height,
> +  int4 dx,
> +  int4 dy)
> +{
> +
> +int y = get_global_id(0);
> +int work_size = get_global_size(0);
> +
> +uint4 sum = (uint4)(0);
> +float4 s2;
> +for (int i = 0; i < width; i++) {
> +float s1 = read_imagef(src, sampler, (int2)(i, y)).x;
> +s2.x = read_imagef(src, sampler, (int2)(i+dx.x, y+dy.x)).x;
> +s2.y = read_imagef(src, sampler, (int2)(i+dx.y, y+dy.y)).x;
> +s2.z = read_imagef(src, sampler, (int2)(i+dx.z, y+dy.z)).x;
> +s2.w = read_imagef(src, sampler, (int2)(i+dx.w, y+dy.w)).x;
> +sum += convert_uint4((s1-s2)*(s1-s2) * 255*255);
> +ii[y * width + i] = sum;
> +}
> +}
> +
> +kernel void vert_sum(__global uint4 *ii,
> + int width,
> + int height)
> +{
> +int x = 

Re: [FFmpeg-devel] [RFC 0/6] Add V4L2 request API hwaccels

[Mark Thompson]

On 08/04/2019 21:09, Jonas Karlman wrote:
> Hello,
> 
> This is a request for comments on a new hwaccel using the V4L2 request API
> that was created in collaboration with Jernej Skrabec.
> 
> The V4L2 ctrls needed for statless decoding is not yet stable and reside in
> private kernel headers. This patchset adds a copy of the kernel private 
> headers
> needed for the hwaccel to compile.

When is the interface likely to become stable?

I don't think including kernel headers here is a good idea.  Please just check 
for appropriate headers - if the user is capable of building this then they can 
also install the headers for their kernel.


If I wanted to buy a single Allwinner SBC to test this on, what should I get?

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] libavutil/hwcontext_opencl.c: fix bug in `opencl_get_plane_format`

[Mark Thompson]

On 09/04/2019 02:08, Song, Ruiling wrote:
>> -Original Message-
>> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf Of
>> Mark Thompson
>> Sent: Tuesday, April 9, 2019 3:49 AM
>> To: ffmpeg-devel@ffmpeg.org
>> Subject: Re: [FFmpeg-devel] [PATCH v2] libavutil/hwcontext_opencl.c: fix bug 
>> in
>> `opencl_get_plane_format`
>>
>> On 08/04/2019 03:01, Jarek Samic wrote:
>>> The `opencl_get_plane_format` function was incorrectly determining the
>>> value used to set the image channel order. This resulted in all RGB
>>> pixel formats being set to the `CL_RGBA` pixel format, regardless of
>>> whether or not they actually *were* RGBA.
>>>
>>> This patch fixes the issue by using the `offset` and depth of components
>>> rather than the loop index to determine the value of `order`.
>>>
>>> Signed-off-by: Jarek Samic 
>>> ---
>>> I have updated this patch in response to the comments on the first version.
>>> RGB is no longer special-cased, the 2, 3, and 4 mappings to `CL_R` have been
>>> removed, and the mapping for `CL_ARGB` has been changed to the correct
>> value.
>>>
>>>  libavutil/hwcontext_opencl.c | 8 +++-
>>>  1 file changed, 3 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c
>>> index b116c5b708..593de1ca41 100644
>>> --- a/libavutil/hwcontext_opencl.c
>>> +++ b/libavutil/hwcontext_opencl.c
>>> @@ -1419,8 +1419,9 @@ static int opencl_get_plane_format(enum
>> AVPixelFormat pixfmt,
>>>  // from the same component.
>>>  if (step && comp->step != step)
>>>  return AVERROR(EINVAL);
>>> -order = order * 10 + c + 1;
>>> +
>>>  depth = comp->depth;
>>> +order = order * 10 + comp->offset / ((depth + 7) / 8) + 1;
>>>  step  = comp->step;
>>>  alpha = (desc->flags & AV_PIX_FMT_FLAG_ALPHA &&
>>>   c == desc->nb_components - 1);
>>
>> This part LGTM, I can split it off and apply it on its own if you like.
>>
>>> @@ -1456,14 +1457,11 @@ static int opencl_get_plane_format(enum
>> AVPixelFormat pixfmt,
>>>  case order: image_format->image_channel_order = type; break;
>>>  switch (order) {
>>>  CHANNEL_ORDER(1,CL_R);
>>> -CHANNEL_ORDER(2,CL_R);
>>> -CHANNEL_ORDER(3,CL_R);
>>> -CHANNEL_ORDER(4,CL_R);
>>>  CHANNEL_ORDER(12,   CL_RG);
>>>  CHANNEL_ORDER(23,   CL_RG);
>>
>> 23 should be gone too, I think?
> Agree.
>>
>>>  CHANNEL_ORDER(1234, CL_RGBA);
>>> +CHANNEL_ORDER(2341, CL_ARGB);
>>>  CHANNEL_ORDER(3214, CL_BGRA);
>>> -CHANNEL_ORDER(4123, CL_ARGB);
>>
>> I'm not sure I believe this part:
>>
>> 1 = R
>> 2 = G
>> 3 = B
>> 4 = A
> The above assumption is not true.
> The new logic changes to use combination of offset-index of RGBA.
> So for CL_ARGB, the R offset at 2, G is offset at 3, B is offset at 4, A is 
> offset at 1.
> So, it is 2341 that maps to ARGB.
> It's interesting that these two ways of representing the swizzle sometime 
> match, sometime not.

Urgh, yes.  I was totally missing that the change above also changes the 
interpretation of the values.

With that I agree the values are correct, so LGTM.

Applied with a slightly more direct commit title.

Thanks!

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 3/7] lavfi/vaapi: Factorise out common code for parameter buffer setup

[Mark Thompson]

Also enables cropping on all VAAPI filters, inherited from the existing
support in scale_vaapi.
---
 libavfilter/vaapi_vpp.c| 55 --
 libavfilter/vaapi_vpp.h|  8 -
 libavfilter/vf_deinterlace_vaapi.c | 33 --
 libavfilter/vf_misc_vaapi.c| 34 +++---
 libavfilter/vf_procamp_vaapi.c | 34 +++---
 libavfilter/vf_scale_vaapi.c   | 38 -
 libavfilter/vf_transpose_vaapi.c   | 44 
 7 files changed, 87 insertions(+), 159 deletions(-)

diff --git a/libavfilter/vaapi_vpp.c b/libavfilter/vaapi_vpp.c
index c5bbc3b85b..647ddc0811 100644
--- a/libavfilter/vaapi_vpp.c
+++ b/libavfilter/vaapi_vpp.c
@@ -248,6 +248,52 @@ int ff_vaapi_vpp_colour_standard(enum AVColorSpace av_cs)
 }
 }
 
+int ff_vaapi_vpp_init_params(AVFilterContext *avctx,
+ VAProcPipelineParameterBuffer *params,
+ const AVFrame *input_frame,
+ AVFrame *output_frame)
+{
+VAAPIVPPContext *ctx = avctx->priv;
+VASurfaceID input_surface;
+
+ctx->input_region = (VARectangle) {
+.x  = input_frame->crop_left,
+.y  = input_frame->crop_top,
+.width  = input_frame->width -
+ (input_frame->crop_left + input_frame->crop_right),
+.height = input_frame->height -
+ (input_frame->crop_top + input_frame->crop_bottom),
+};
+output_frame->crop_top= 0;
+output_frame->crop_bottom = 0;
+output_frame->crop_left   = 0;
+output_frame->crop_right  = 0;
+
+input_surface = (VASurfaceID)(uintptr_t)input_frame->data[3],
+
+*params = (VAProcPipelineParameterBuffer) {
+.surface = input_surface,
+.surface_region  = &ctx->input_region,
+.surface_color_standard  =
+ff_vaapi_vpp_colour_standard(input_frame->colorspace),
+.output_region   = NULL,
+.output_background_color = VAAPI_VPP_BACKGROUND_BLACK,
+.output_color_standard   =
+ff_vaapi_vpp_colour_standard(input_frame->colorspace),
+.pipeline_flags  = 0,
+.filter_flags= VA_FRAME_PICTURE,
+
+// Filter and reference data filled by the filter itself.
+
+#if VA_CHECK_VERSION(1, 1, 0)
+.rotation_state = VA_ROTATION_NONE,
+.mirror_state   = VA_MIRROR_NONE,
+#endif
+};
+
+return 0;
+}
+
 int ff_vaapi_vpp_make_param_buffers(AVFilterContext *avctx,
 int type,
 const void *data,
@@ -279,12 +325,15 @@ int ff_vaapi_vpp_make_param_buffers(AVFilterContext 
*avctx,
 
 int ff_vaapi_vpp_render_picture(AVFilterContext *avctx,
 VAProcPipelineParameterBuffer *params,
-VASurfaceID output_surface)
+AVFrame *output_frame)
 {
+VAAPIVPPContext *ctx = avctx->priv;
+VASurfaceID output_surface;
 VABufferID params_id;
 VAStatus vas;
-int err = 0;
-VAAPIVPPContext *ctx   = avctx->priv;
+int err;
+
+output_surface = (VASurfaceID)(uintptr_t)output_frame->data[3];
 
 vas = vaBeginPicture(ctx->hwctx->display,
  ctx->va_context, output_surface);
diff --git a/libavfilter/vaapi_vpp.h b/libavfilter/vaapi_vpp.h
index 96f720f07d..1e2b4a1066 100644
--- a/libavfilter/vaapi_vpp.h
+++ b/libavfilter/vaapi_vpp.h
@@ -42,6 +42,7 @@ typedef struct VAAPIVPPContext {
 
 AVBufferRef   *input_frames_ref;
 AVHWFramesContext *input_frames;
+VARectangleinput_region;
 
 enum AVPixelFormat output_format;
 int output_width;   // computed width
@@ -69,6 +70,11 @@ int ff_vaapi_vpp_config_output(AVFilterLink *outlink);
 
 int ff_vaapi_vpp_colour_standard(enum AVColorSpace av_cs);
 
+int ff_vaapi_vpp_init_params(AVFilterContext *avctx,
+ VAProcPipelineParameterBuffer *params,
+ const AVFrame *input_frame,
+ AVFrame *output_frame);
+
 int ff_vaapi_vpp_make_param_buffers(AVFilterContext *avctx,
 int type,
 const void *data,
@@ -77,6 +83,6 @@ int ff_vaapi_vpp_make_param_buffers(AVFilterContext *avctx,
 
 int ff_vaapi_vpp_render_picture(AVFilterContext *avctx,
 VAProcPipelineParameterBuffer *params,
-VASurfaceID output_surface);
+AVFrame *output_frame);
 
 #endif /* AVFILTER_VAAPI_VPP_H */
diff --git a/libavfilter/vf_deinterlace_vaapi.c 
b/libavfilter/vf_deinterlace_vaapi.c
index f67a1c8e79..2c147310c9 100644
--- a/libavfilter/vf_deinterlace_vaapi.c
+++ b/libavfilter/vf_deinterlace_vaapi.c
@@ -181,12 +181,11 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, 
AVFrame *input_frame

[FFmpeg-devel] [PATCH v4 2/7] doc/indevs: Add example using cropping to capture part of a plane

[Mark Thompson]

---
 doc/indevs.texi | 8 
 1 file changed, 8 insertions(+)

diff --git a/doc/indevs.texi b/doc/indevs.texi
index 1d5ed65773..a4f0f608d7 100644
--- a/doc/indevs.texi
+++ b/doc/indevs.texi
@@ -910,6 +910,14 @@ Capture from CRTC ID 42 at 60fps, map the result to VAAPI, 
convert to NV12 and e
 ffmpeg -crtc_id 42 -framerate 60 -f kmsgrab -i - -vf 
'hwmap=derive_device=vaapi,scale_vaapi=w=1920:h=1080:format=nv12' -c:v 
h264_vaapi output.mp4
 @end example
 
+@item
+To capture only part of a plane the output can be cropped - this can be used 
to capture
+a single window, as long as it has a known absolute position.  For example, to 
capture
+and encode the middle quarter of a 1920x1080 plane:
+@example
+ffmpeg -f kmsgrab -i - -vf 
'hwmap=derive_device=vaapi,crop=960:540:480:270,scale_vaapi=960:540:nv12' -c:v 
h264_vaapi output.mp4
+@end example
+
 @end itemize
 
 @section lavfi
-- 
2.20.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 1/7] vf_crop: Add support for cropping hardware frames

[Mark Thompson]

Set the cropping fields in the AVFrame.
---
On 26/03/2019 10:59, Song, Ruiling wrote:> 
> I think we need to make scale_vaapi evaluate input dimensions considering 
> crop information. What do you think?

I agree.  But the cropping information is currently carried on the frame, not 
at any higher level (from the codec context or on the filter link), so we don't 
have any idea how big the output frames need to be at setup time when we have 
to set the size on the output link.  Making it work requires carrying more 
complete information through filter setup, similar to the problem with colour 
range and other properties which aren't reflected in the existing format.  
(This is on my to-do list.)


 libavfilter/vf_crop.c | 74 +--
 1 file changed, 51 insertions(+), 23 deletions(-)

diff --git a/libavfilter/vf_crop.c b/libavfilter/vf_crop.c
index 84be4c7d0d..7f6b0f03d3 100644
--- a/libavfilter/vf_crop.c
+++ b/libavfilter/vf_crop.c
@@ -98,9 +98,17 @@ static int query_formats(AVFilterContext *ctx)
 
 for (fmt = 0; av_pix_fmt_desc_get(fmt); fmt++) {
 const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(fmt);
-if (!(desc->flags & (AV_PIX_FMT_FLAG_HWACCEL | 
AV_PIX_FMT_FLAG_BITSTREAM)) &&
-!((desc->log2_chroma_w || desc->log2_chroma_h) && !(desc->flags & 
AV_PIX_FMT_FLAG_PLANAR)) &&
-(ret = ff_add_format(&formats, fmt)) < 0)
+if (desc->flags & AV_PIX_FMT_FLAG_BITSTREAM)
+continue;
+if (!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) {
+// Not usable if there is any subsampling but the format is
+// not planar (e.g. YUYV422).
+if ((desc->log2_chroma_w || desc->log2_chroma_h) &&
+!(desc->flags & AV_PIX_FMT_FLAG_PLANAR))
+continue;
+}
+ret = ff_add_format(&formats, fmt);
+if (ret < 0)
 return ret;
 }
 
@@ -157,8 +165,14 @@ static int config_input(AVFilterLink *link)
 s->var_values[VAR_POS]   = NAN;
 
 av_image_fill_max_pixsteps(s->max_step, NULL, pix_desc);
-s->hsub = pix_desc->log2_chroma_w;
-s->vsub = pix_desc->log2_chroma_h;
+
+if (pix_desc->flags & AV_PIX_FMT_FLAG_HWACCEL) {
+s->hsub = 1;
+s->vsub = 1;
+} else {
+s->hsub = pix_desc->log2_chroma_w;
+s->vsub = pix_desc->log2_chroma_h;
+}
 
 if ((ret = av_expr_parse_and_eval(&res, (expr = s->w_expr),
   var_names, s->var_values,
@@ -237,9 +251,15 @@ fail_expr:
 static int config_output(AVFilterLink *link)
 {
 CropContext *s = link->src->priv;
+const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(link->format);
 
-link->w = s->w;
-link->h = s->h;
+if (desc->flags & AV_PIX_FMT_FLAG_HWACCEL) {
+// Hardware frames adjust the cropping regions rather than
+// changing the frame size.
+} else {
+link->w = s->w;
+link->h = s->h;
+}
 link->sample_aspect_ratio = s->out_sar;
 
 return 0;
@@ -252,9 +272,6 @@ static int filter_frame(AVFilterLink *link, AVFrame *frame)
 const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(link->format);
 int i;
 
-frame->width  = s->w;
-frame->height = s->h;
-
 s->var_values[VAR_N] = link->frame_count_out;
 s->var_values[VAR_T] = frame->pts == AV_NOPTS_VALUE ?
 NAN : frame->pts * av_q2d(link->time_base);
@@ -285,22 +302,33 @@ static int filter_frame(AVFilterLink *link, AVFrame 
*frame)
 (int)s->var_values[VAR_N], s->var_values[VAR_T], 
s->var_values[VAR_POS],
 s->x, s->y, s->x+s->w, s->y+s->h);
 
-frame->data[0] += s->y * frame->linesize[0];
-frame->data[0] += s->x * s->max_step[0];
-
-if (!(desc->flags & AV_PIX_FMT_FLAG_PAL || desc->flags & FF_PSEUDOPAL)) {
-for (i = 1; i < 3; i ++) {
-if (frame->data[i]) {
-frame->data[i] += (s->y >> s->vsub) * frame->linesize[i];
-frame->data[i] += (s->x * s->max_step[i]) >> s->hsub;
+if (desc->flags & AV_PIX_FMT_FLAG_HWACCEL) {
+frame->crop_top   += s->y;
+frame->crop_left  += s->x;
+frame->crop_bottom = frame->height - frame->crop_top - 
frame->crop_bottom - s->h;
+frame->crop_right  = frame->width  - frame->crop_left - 
frame->crop_right - s->w;
+
+} else {
+frame->width  = s->w;
+frame->height = s->h;
+
+frame->data[0] += s->y * frame->linesize[0];
+frame->data[0] += s->x * s->max_step[0];
+
+if (!(desc->flags & AV_PIX_FMT_FLAG_PAL || desc->flags & 
FF_PSEUDOPAL)) {
+for (i = 1; i < 3; i ++) {
+if (frame->data[i]) {
+frame->data[i] += (s->y >> s->vsub) * frame->linesize[i];
+frame->data[i] += (s->x * s->max_step[i]) >> s->hsub;
+}
 }
 }
-}
 
-/* alpha plane */
-if (frame->data[3]) {
-frame->data[3] += s->y * frame->

[FFmpeg-devel] [PATCH v4 4/7] vf_misc_vaapi: Add missing return value checks

[Mark Thompson]

Parameter buffer creation can fail.
---
 libavfilter/vf_misc_vaapi.c | 15 ++-
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/libavfilter/vf_misc_vaapi.c b/libavfilter/vf_misc_vaapi.c
index 6f31a04293..6fbd453741 100644
--- a/libavfilter/vf_misc_vaapi.c
+++ b/libavfilter/vf_misc_vaapi.c
@@ -84,10 +84,9 @@ static int denoise_vaapi_build_filter_params(AVFilterContext 
*avctx)
 denoise.value =  map(ctx->denoise, DENOISE_MIN, DENOISE_MAX,
  caps.range.min_value,
  caps.range.max_value);
-ff_vaapi_vpp_make_param_buffers(avctx, VAProcFilterParameterBufferType,
-&denoise, sizeof(denoise), 1);
-
-return 0;
+return ff_vaapi_vpp_make_param_buffers(avctx,
+   VAProcFilterParameterBufferType,
+   &denoise, sizeof(denoise), 1);
 }
 
 static int sharpness_vaapi_build_filter_params(AVFilterContext *avctx)
@@ -116,11 +115,9 @@ static int 
sharpness_vaapi_build_filter_params(AVFilterContext *avctx)
   SHARPNESS_MIN, SHARPNESS_MAX,
   caps.range.min_value,
   caps.range.max_value);
-ff_vaapi_vpp_make_param_buffers(avctx,
-VAProcFilterParameterBufferType,
-&sharpness, sizeof(sharpness), 1);
-
-return 0;
+return ff_vaapi_vpp_make_param_buffers(avctx,
+   VAProcFilterParameterBufferType,
+   &sharpness, sizeof(sharpness), 1);
 }
 
 static int misc_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame)
-- 
2.20.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 6/7] vf_scale_vaapi: Add options to configure output colour properties

[Mark Thompson]

The "out_color_matrix" and "out_range" properties match the same options
in vf_scale; the others attempt to follow the same pattern.
---
 libavfilter/vf_scale_vaapi.c | 70 
 1 file changed, 70 insertions(+)

diff --git a/libavfilter/vf_scale_vaapi.c b/libavfilter/vf_scale_vaapi.c
index ae2471b821..2f8e07df5c 100644
--- a/libavfilter/vf_scale_vaapi.c
+++ b/libavfilter/vf_scale_vaapi.c
@@ -39,6 +39,17 @@ typedef struct ScaleVAAPIContext {
 
 char *w_expr;  // width expression string
 char *h_expr;  // height expression string
+
+char *colour_primaries_string;
+char *colour_transfer_string;
+char *colour_matrix_string;
+int colour_range;
+char *chroma_location_string;
+
+enum AVColorPrimaries colour_primaries;
+enum AVColorTransferCharacteristic colour_transfer;
+enum AVColorSpace colour_matrix;
+enum AVChromaLocation chroma_location;
 } ScaleVAAPIContext;
 
 static const char *scale_vaapi_mode_name(int mode)
@@ -110,6 +121,17 @@ static int scale_vaapi_filter_frame(AVFilterLink *inlink, 
AVFrame *input_frame)
 if (err < 0)
 return err;
 
+if (ctx->colour_primaries != AVCOL_PRI_UNSPECIFIED)
+output_frame->color_primaries = ctx->colour_primaries;
+if (ctx->colour_transfer != AVCOL_TRC_UNSPECIFIED)
+output_frame->color_trc = ctx->colour_transfer;
+if (ctx->colour_matrix != AVCOL_SPC_UNSPECIFIED)
+output_frame->colorspace = ctx->colour_matrix;
+if (ctx->colour_range != AVCOL_RANGE_UNSPECIFIED)
+output_frame->color_range = ctx->colour_range;
+if (ctx->chroma_location != AVCHROMA_LOC_UNSPECIFIED)
+output_frame->chroma_location = ctx->chroma_location;
+
 err = ff_vaapi_vpp_init_params(avctx, ¶ms,
input_frame, output_frame);
 if (err < 0)
@@ -155,6 +177,24 @@ static av_cold int scale_vaapi_init(AVFilterContext *avctx)
 vpp_ctx->output_format = AV_PIX_FMT_NONE;
 }
 
+#define STRING_OPTION(var_name, func_name, default_value) do { \
+if (ctx->var_name ## _string) { \
+int var = av_ ## func_name ## _from_name(ctx->var_name ## 
_string); \
+if (var < 0) { \
+av_log(avctx, AV_LOG_ERROR, "Invalid %s.\n", #var_name); \
+return AVERROR(EINVAL); \
+} \
+ctx->var_name = var; \
+} else { \
+ctx->var_name = default_value; \
+} \
+} while (0)
+
+STRING_OPTION(colour_primaries, color_primaries, AVCOL_PRI_UNSPECIFIED);
+STRING_OPTION(colour_transfer,  color_transfer,  AVCOL_TRC_UNSPECIFIED);
+STRING_OPTION(colour_matrix,color_space, AVCOL_SPC_UNSPECIFIED);
+STRING_OPTION(chroma_location,  chroma_location, AVCHROMA_LOC_UNSPECIFIED);
+
 return 0;
 }
 
@@ -178,6 +218,36 @@ static const AVOption scale_vaapi_options[] = {
   0, AV_OPT_TYPE_CONST, { .i64 = VA_FILTER_SCALING_HQ }, 0, 0, FLAGS,  
"mode" },
 { "nl_anamorphic", "Use nolinear anamorphic scaling algorithm",
   0, AV_OPT_TYPE_CONST, { .i64 = VA_FILTER_SCALING_NL_ANAMORPHIC }, 0, 
0, FLAGS,  "mode" },
+
+// These colour properties match the ones of the same name in vf_scale.
+{ "out_color_matrix", "Output colour matrix coefficient set",
+  OFFSET(colour_matrix_string), AV_OPT_TYPE_STRING, { .str = NULL }, 
.flags = FLAGS },
+{ "out_range", "Output colour range",
+  OFFSET(colour_range), AV_OPT_TYPE_INT, { .i64 = AVCOL_RANGE_UNSPECIFIED 
},
+  AVCOL_RANGE_UNSPECIFIED, AVCOL_RANGE_JPEG, FLAGS, "range" },
+{ "full","Full range",
+  0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_JPEG }, 0, 0, FLAGS, "range" 
},
+{ "limited", "Limited range",
+  0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_MPEG }, 0, 0, FLAGS, "range" 
},
+{ "jpeg","Full range",
+  0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_JPEG }, 0, 0, FLAGS, "range" 
},
+{ "mpeg","Limited range",
+  0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_MPEG }, 0, 0, FLAGS, "range" 
},
+{ "tv",  "Limited range",
+  0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_MPEG }, 0, 0, FLAGS, "range" 
},
+{ "pc",  "Full range",
+  0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_JPEG }, 0, 0, FLAGS, "range" 
},
+// These colour properties are new here.
+{ "out_color_primaries", "Output colour primaries",
+  OFFSET(colour_primaries_string), AV_OPT_TYPE_STRING,
+  { .str = NULL }, .flags = FLAGS },
+{ "out_color_transfer", "Output colour transfer characteristics",
+  OFFSET(colour_transfer_string),  AV_OPT_TYPE_STRING,
+  { .str = NULL }, .flags = FLAGS },
+{ "out_chroma_location", "Output chroma sample location",
+  OFFSET(chroma_location_string),  AV_OPT_TYPE_STRING,
+  { .str = NULL }, .flags = FLAGS },
+
 { NULL },
 };
 
-- 
2.20.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailma

[FFmpeg-devel] [PATCH v4 5/7] lavfi/vaapi: Improve support for colour properties

[Mark Thompson]

Attempts to pick the set of supported colour properties best matching the
input.  Output is then set with the same values, except for the colour
matrix which may change when converting between RGB and YUV.
---
On 04/03/2019 11:36, Song, Ruiling wrote:
> Why using these magic number instead of meaningful enum value?

See .  These 
values are standardised in H.273 (or ISO/IEC 23001-8:2016), which are then used 
directly in both FFmpeg and various codecs like MPEG-2, H.264, H.265 and AV1.  
On balance it seemed clearer to keep the numeric values rather than to turn 
them into symbolic names which would be much harder to read.

> And two entries added for VAProcColorStandardBT601, seems that only the first 
> will be returned?

It was meant to check both cases, now fixed.

>> +// Give scores to the possible options and choose the lowest one.
>> +// An exact match will score zero and therefore always be chosen, as
>> +// will a partial match where all unmatched elements are explicitly
>> +// unspecified.  (If all elements are unspecified this will use the
>> +// first available value.)  If no options match at all then just
>> +// pass "none" to the driver and let it make its own choice.
> Here (a*4+b*2+c)  is chosen as the score function, I am not sure whether (a + 
> b + c) is just ok?

I made the weights different to try to avoid confusing draws where it might 
match to a different colourspace on one run and then a different transfer 
characteristic on another.

The choice of matrix > transfer > primaries is mostly arbitrary coming from 
vague intuition about how extreme the bad cases are (wrong matrix can be wildly 
incorrect (YCoCg interpreted as YUV, say), wrong transfer can mess up the 
intensity in a confusing way but should mostly resemble the intended image, 
wrong primaries will shift colours and look weird but stay be mostly coherent). 
 I'm happy to change that if you have a different opinion!

>> +best_score = -1;
>> +worst_score = 4 * (props->colorspace != AVCOL_SPC_UNSPECIFIED) +
>> +  2 * (props->color_trc != AVCOL_TRC_UNSPECIFIED) +
>> +  (props->color_primaries != AVCOL_PRI_UNSPECIFIED);
> Seems that the outer loop here is just used to re-iterate through nb_vacs to 
> find the best match again?
> Can we remove the outer-loop-over-k like below?
> 
> best_va_standard = VAProcColorStandardNone;
> for (i = 0; i < nb_vacs; i++) {
>   ...
>   ...
>// Only include things which matched something.
> if ((worst_score == 0 || score < worst_score) &&
> (best_score == -1 || score < best_score)) {
> best_score = score;
>   best_va_standard = t->va_color_standard;
>   }
> }
> props->va_color_standard = best_va_standard;

Yes, that approach would be nicer.  Done.

Thanks,

- Mark


 libavfilter/vaapi_vpp.c| 285 +++--
 libavfilter/vaapi_vpp.h|   2 -
 libavfilter/vf_deinterlace_vaapi.c |   8 +-
 libavfilter/vf_misc_vaapi.c|   7 +-
 libavfilter/vf_procamp_vaapi.c |   7 +-
 libavfilter/vf_scale_vaapi.c   |   8 +-
 libavfilter/vf_transpose_vaapi.c   |   7 +-
 7 files changed, 292 insertions(+), 32 deletions(-)

diff --git a/libavfilter/vaapi_vpp.c b/libavfilter/vaapi_vpp.c
index 647ddc0811..5091d2508a 100644
--- a/libavfilter/vaapi_vpp.c
+++ b/libavfilter/vaapi_vpp.c
@@ -234,18 +234,275 @@ fail:
 return err;
 }
 
-int ff_vaapi_vpp_colour_standard(enum AVColorSpace av_cs)
+typedef struct VAAPIColourProperties {
+VAProcColorStandardType va_color_standard;
+
+enum AVColorPrimaries color_primaries;
+enum AVColorTransferCharacteristic color_trc;
+enum AVColorSpace colorspace;
+
+uint8_t va_chroma_sample_location;
+uint8_t va_color_range;
+
+enum AVColorRange color_range;
+enum AVChromaLocation chroma_sample_location;
+} VAAPIColourProperties;
+
+static const VAAPIColourProperties vaapi_colour_standard_map[] = {
+{ VAProcColorStandardBT601,   5,  6,  5 },
+{ VAProcColorStandardBT601,   6,  6,  6 },
+{ VAProcColorStandardBT709,   1,  1,  1 },
+{ VAProcColorStandardBT470M,  4,  4,  4 },
+{ VAProcColorStandardBT470BG, 5,  5,  5 },
+{ VAProcColorStandardSMPTE170M,   6,  6,  6 },
+{ VAProcColorStandardSMPTE240M,   7,  7,  7 },
+{ VAProcColorStandardGenericFilm, 8,  1,  1 },
+#if VA_CHECK_VERSION(1, 1, 0)
+{ VAProcColorStandardSRGB,1, 13,  0 },
+{ VAProcColorStandardXVYCC601,1, 11,  5 },
+{ VAProcColorStandardXVYCC709,1, 11,  1 },
+{ VAProcColorStandardBT2020,  9, 14,  9 },
+#endif
+};
+
+static void vaapi_vpp_fill_colour_standard(VAAPIColourProperties *props,
+   VAProcColorStandardType *vacs,
+   int nb_vacs)
+{
+const VAAPIColourPrope

[FFmpeg-devel] [PATCH v4 7/7] vaapi_encode: Warn if input has cropping information

[Mark Thompson]

Cropping is not supported by VAAPI encode.
---
On 26/03/2019 10:59, Song, Ruiling wrote:> 
> And do we need to add warning message against crop information in encoder if 
> user failed to add some vaapi filter after crop?
> Seems that vaapi encoder does not encode correctly with crop?

That's a good idea :)


 libavcodec/vaapi_encode.c | 19 +++
 libavcodec/vaapi_encode.h |  4 
 2 files changed, 23 insertions(+)

diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index 2dda451882..c3d8944c3c 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -913,6 +913,21 @@ static int vaapi_encode_clear_old(AVCodecContext *avctx)
 return 0;
 }
 
+static int vaapi_encode_check_frame(AVCodecContext *avctx,
+const AVFrame *frame)
+{
+VAAPIEncodeContext *ctx = avctx->priv_data;
+
+if ((frame->crop_top  || frame->crop_bottom ||
+ frame->crop_left || frame->crop_right) && !ctx->crop_warned) {
+av_log(avctx, AV_LOG_WARNING, "Cropping information on input "
+   "frames ignored due to lack of API support.\n");
+ctx->crop_warned = 1;
+}
+
+return 0;
+}
+
 int ff_vaapi_encode_send_frame(AVCodecContext *avctx, const AVFrame *frame)
 {
 VAAPIEncodeContext *ctx = avctx->priv_data;
@@ -923,6 +938,10 @@ int ff_vaapi_encode_send_frame(AVCodecContext *avctx, 
const AVFrame *frame)
 av_log(avctx, AV_LOG_DEBUG, "Input frame: %ux%u (%"PRId64").\n",
frame->width, frame->height, frame->pts);
 
+err = vaapi_encode_check_frame(avctx, frame);
+if (err < 0)
+return err;
+
 pic = vaapi_encode_alloc(avctx);
 if (!pic)
 return AVERROR(ENOMEM);
diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h
index 44a8db566e..12efee2d08 100644
--- a/libavcodec/vaapi_encode.h
+++ b/libavcodec/vaapi_encode.h
@@ -314,6 +314,10 @@ typedef struct VAAPIEncodeContext {
 int idr_counter;
 int gop_counter;
 int end_of_stream;
+
+// The encoder does not support cropping information, so warn about
+// it the first time we encounter any nonzero crop fields.
+int crop_warned;
 } VAAPIEncodeContext;
 
 enum {
-- 
2.20.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH RFC v2 2/3] libavcodec: Add thumbnail output to vaapi_h264 decoder

[Mark Thompson]

On 08/04/2019 09:53, Zachary Zhou wrote:
> This is sample code for reference
> 
> HW support for decode+scaling in a single HW command (VDBOX+SFC).
> The primary target usage is video analytics, but can be used playback,
> transcoding, etc.
> 
> For VAAPI -
> https://github.com/intel/libva
> basically, it allows multiple outputs (in different resolutions) using the 
> decode context in a single call (you can search for “additional_outputs” in 
> va.h).
> 
> VAAPI sample code -
> https://github.com/intel/libva-utils/commit/957a269f02b00760b7e807643c821ee26abc529b
> ---
>  libavcodec/avcodec.h   |   8 +++
>  libavcodec/decode.c|  16 +
>  libavcodec/options_table.h |   4 ++
>  libavcodec/vaapi_decode.c  | 122 ++---
>  libavcodec/vaapi_decode.h  |  30 +
>  libavcodec/vaapi_h264.c|  13 
>  6 files changed, 185 insertions(+), 8 deletions(-)

This doesn't fit into the libavcodec API at all.  If it is desirable to have 
some sort of generic multiple-output mechanism in libavcodec, please open a 
separate discussion considering that API specifically.

For the particular hardware instance you are talking about here, can you 
explain what this mechanism actually helps with?  My understanding was that the 
VDBOX / SFC chaining existed to save a small amount of power in restricted 
playback cases going directly to the display (that is, for maximising playback 
time on battery devices).  Since you are writing everything back to memory 
immediately here anyway, I don't see how it helps compared to just doing the 
scale as a filter operation.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/h264_parse: change prefix to avpriv for usage in avformat mxf muxer

[Mark Thompson]

On 09/04/2019 23:14, Baptiste Coudurier wrote:
> ---
>  libavcodec/h264_parse.c  | 2 +-
>  libavcodec/h264_parser.c | 2 +-
>  libavcodec/h264_ps.c | 4 ++--
>  libavcodec/h264_ps.h | 4 ++--
>  libavcodec/h264dec.c | 6 +++---
>  5 files changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/libavcodec/h264_ps.c b/libavcodec/h264_ps.c
> index 17bfa780ce..980b1e189d 100644
> --- a/libavcodec/h264_ps.c
> +++ b/libavcodec/h264_ps.c
> @@ -330,8 +330,8 @@ void ff_h264_ps_uninit(H264ParamSets *ps)
>  ps->sps = NULL;
>  }
>  
> -int ff_h264_decode_seq_parameter_set(GetBitContext *gb, AVCodecContext 
> *avctx,
> - H264ParamSets *ps, int 
> ignore_truncation)
> +int avpriv_h264_decode_seq_parameter_set(GetBitContext *gb, AVCodecContext 
> *avctx,
> + H264ParamSets *ps, int 
> ignore_truncation)
>  {
>  AVBufferRef *sps_buf;
>  int profile_idc, level_idc, constraint_set_flags = 0;
> diff --git a/libavcodec/h264_ps.h b/libavcodec/h264_ps.h
> index e967b9cbcf..d422ce122e 100644
> --- a/libavcodec/h264_ps.h
> +++ b/libavcodec/h264_ps.h
> @@ -149,8 +149,8 @@ typedef struct H264ParamSets {
>  /**
>   * Decode SPS
>   */
> -int ff_h264_decode_seq_parameter_set(GetBitContext *gb, AVCodecContext 
> *avctx,
> - H264ParamSets *ps, int 
> ignore_truncation);
> +int avpriv_h264_decode_seq_parameter_set(GetBitContext *gb, AVCodecContext 
> *avctx,
> + H264ParamSets *ps, int 
> ignore_truncation);

Making the H.264 decoder's internal SPS and PPS structures fixed API really 
doesn't feel like a good idea to me, but I admit I don't have a better answer 
to the problem you're facing.  Copying everything is also pretty terrible.  
Adding a new API to parse the headers and return the information you want might 
be nicer considering just this change, but extensibility for the inevitable 
subsequent patch which wants some extra piece of information in future would be 
a big pain.

If you're going to go with this, please add namespace prefixes to the 
structures it affects ("SPS" and "PPS", since you're now including them in code 
which isn't explicitly H.264), and also add big warnings to the header 
indicating that the structures are now fixed and can't be modified at all 
except on major bumps.  (Though maybe wait for other comments before actually 
making any changes.)

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH, v4] lavc/vaapi_decode: find exact va_profile for HEVC_REXT

[Mark Thompson]

On 09/04/2019 03:33, Linjie Fu wrote:
> Use the profile constraint flags to determine the exact va_profile for
> HEVC_REXT.
> 
> Directly cast PTLCommon to H265RawProfileTierLevel,

Please don't.  The two structures aren't really connected, they shouldn't be 
assumed to be compatible.

> and use 
> ff_h265_get_profile
> to get the exact profile.
> 
> Add h265_profile_level.o to build objects for vaapi_decode to fix the compile
> dependency issue.
> 
> Signed-off-by: Linjie Fu 
> ---
> [v2]: use constraint flags to determine the exact profile, expose the
> codec-specific stuff at the beginning.
> [v3]: move the VA version check to fix the compile issue.
> [v4]: fix the build issue.
> 
>  libavcodec/Makefile   |  2 +-
>  libavcodec/vaapi_decode.c | 73 +++
>  2 files changed, 59 insertions(+), 16 deletions(-)
> 
> diff --git a/libavcodec/Makefile b/libavcodec/Makefile
> index 15c43a8a6a..c0df0ad90a 100644
> --- a/libavcodec/Makefile
> +++ b/libavcodec/Makefile
> @@ -857,7 +857,7 @@ OBJS-$(CONFIG_ADPCM_YAMAHA_ENCODER)   += adpcmenc.o 
> adpcm_data.o
>  OBJS-$(CONFIG_D3D11VA)+= dxva2.o
>  OBJS-$(CONFIG_DXVA2)  += dxva2.o
>  OBJS-$(CONFIG_NVDEC)  += nvdec.o
> -OBJS-$(CONFIG_VAAPI)  += vaapi_decode.o
> +OBJS-$(CONFIG_VAAPI)  += vaapi_decode.o 
> h265_profile_level.o
>  OBJS-$(CONFIG_VIDEOTOOLBOX)   += videotoolbox.o
>  OBJS-$(CONFIG_VDPAU)  += vdpau.o
>  
> diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
> index 69512e1d45..47db6c874a 100644
> --- a/libavcodec/vaapi_decode.c
> +++ b/libavcodec/vaapi_decode.c
> @@ -24,6 +24,8 @@
>  #include "decode.h"
>  #include "internal.h"
>  #include "vaapi_decode.h"
> +#include "hevcdec.h"
> +#include "h265_profile_level.h"
>  
>  
>  int ff_vaapi_decode_make_param_buffer(AVCodecContext *avctx,
> @@ -401,6 +403,33 @@ static const struct {
>  #undef MAP
>  };
>  
> +/*
> + * Find exact va_profile for HEVC Range Extension
> + */
> +static VAProfile vaapi_decode_find_exact_profile(AVCodecContext *avctx)
> +{
> +const HEVCContext *h = avctx->priv_data;
> +const HEVCSPS *sps = h->ps.sps;
> +const PTL *ptl = &(sps->ptl);
> +const PTLCommon *general_ptl = &(ptl->general_ptl);
> +const H265ProfileDescriptor *profile;
> +
> +/* PTLCommon should match the member sequence in 
> H265RawProfileTierLevel*/
> +profile = ff_h265_get_profile((H265RawProfileTierLevel *)general_ptl);

This can return NULL.

> +
> +#if VA_CHECK_VERSION(1, 2, 0)
> +if (!strcmp(profile->name, "Main 4:2:2 10"))
> +return VAProfileHEVCMain422_10;
> +else if (!strcmp(profile->name, "Main 4:4:4"))
> +return VAProfileHEVCMain444;
> +else if (!strcmp(profile->name, "Main 4:4:4 10"))
> +return VAProfileHEVCMain444_10;
> +#else
> +av_log(avctx, AV_LOG_WARNING, "HEVC profile %s is "
> +"not supported with this VA version.\n", 
> profile->name);
> +#endif
> +return VAProfileNone;
> +}
>  /*
>   * Set *va_config and the frames_ref fields from the current codec parameters
>   * in avctx.
> @@ -447,24 +476,38 @@ static int vaapi_decode_make_config(AVCodecContext 
> *avctx,
>  matched_va_profile = VAProfileNone;
>  exact_match = 0;
>  
> -for (i = 0; i < FF_ARRAY_ELEMS(vaapi_profile_map); i++) {
> -int profile_match = 0;
> -if (avctx->codec_id != vaapi_profile_map[i].codec_id)
> -continue;
> -if (avctx->profile == vaapi_profile_map[i].codec_profile ||
> -vaapi_profile_map[i].codec_profile == FF_PROFILE_UNKNOWN)
> -profile_match = 1;
> -for (j = 0; j < profile_count; j++) {
> -if (vaapi_profile_map[i].va_profile == profile_list[j]) {
> -exact_match = profile_match;
> +if (avctx->profile == FF_PROFILE_HEVC_REXT) {
> +/* find the exact va_profile for HEVC_REXT */
> +VAProfile va_profile = vaapi_decode_find_exact_profile(avctx);
> +for (i = 0; i < profile_count; i++) {
> +if (va_profile == profile_list[i]) {
> +exact_match = 1;
> +matched_va_profile = va_profile;
> +matched_ff_profile = FF_PROFILE_HEVC_REXT;
>  break;
>  }
>  }
> -if (j < profile_count) {
> -matched_va_profile = vaapi_profile_map[i].va_profile;
> -matched_ff_profile = vaapi_profile_map[i].codec_profile;
> -if (exact_match)
> -break;
> +} else {
> +/* find the exact va_profile according to codec_id and profile */
> +for (i = 0; i < FF_ARRAY_ELEMS(vaapi_profile_map); i++) {
> +int profile_match = 0;
> +if (avctx->codec_id != vaapi_profile_map[i].codec_id)
> +continue;
> +if

Re: [FFmpeg-devel] [PATCH] lavfi: add colorkey_opencl filter

[Mark Thompson]

On 10/04/2019 04:37, Jarek Samic wrote:
> This is a direct port of the CPU filter.
> 
> Signed-off-by: Jarek Samic 
> ---
> This is my submission for the GSoC OpenCL video filters project qualification 
> task.
> 
> Command you can use to try it out:
> 
> ./ffmpeg -i some_video -i some_img -init_hw_device opencl=gpu 
> -filter_hw_device gpu -filter_complex "[0:v]format=rgba, hwupload, 
> colorkey_opencl=yellow:0.4:0.2, hwdownload, 
> format=rgba[over];[1:v][over]overlay" output
> 
> Based on simple observation of that command running vs. an equivalent one 
> with the CPU
> colorkey filter, it would appear that the OpenCL version is ~10-20% faster 
> including
> the overhead to upload / download from the GPU (at least with my test input 
> and
> hardware).
> 
> You will notice that I am using overlay rather than overlay_opencl above. I 
> am not
> sure what's going on, but the overlay_opencl filter is not working for me: 
> every time
> I try to use it it either stops after the first couple of frames or spits out
> duplicate frames forever. Even just running this command that is basically a 
> copy of
> the example command to overlay an image logo on the top-left corner of an 
> input video:
> 
> ./ffmpeg -i ../video.mp4 -i ../img.png -init_hw_device opencl=gpu 
> -filter_hw_device gpu -filter_complex "[0:v]hwupload[a], [1:v]format=yuv420p, 
> hwupload[b], [a][b]overlay_opencl, hwdownload, format=yuv420p" ../vid_test.mp4
> 
> Results in an infinite number of duplicate frames. (The format of the input 
> video is
> yuv420p, to be clear.)
> 
> Before I take the time to dig in and investigate what's going on, is anyone 
> else
> aware of what could be causing this (or any existing known issues)?

The lack of checks in overlay_opencl generally leaves a lot to be desired.  
It's is intended for (and only really tested with) hardware video in YUV 
formats (primarily overlaying YUV or YUVA on YUV, for things like subtitles or 
pips), but is rather variable in how well it works at that.

A tiny bit of hacking can get me to something which works with RGBA here.  
Changing:

diff --git a/libavfilter/vf_overlay_opencl.c b/libavfilter/vf_overlay_opencl.c
index e9c853203b..09070e106a 100644
--- a/libavfilter/vf_overlay_opencl.c
+++ b/libavfilter/vf_overlay_opencl.c
@@ -81,7 +81,7 @@ static int overlay_opencl_load(AVFilterContext *avctx,
 }
 
 if (main_planes == overlay_planes) {
-if (main_desc->nb_components == overlay_desc->nb_components)
+if (main_desc->nb_components == overlay_desc->nb_components && 0)
 kernel = "overlay_no_alpha";
 else
 kernel = "overlay_internal_alpha";

to force the upper layer alpha to take effect directly I can make it work to 
blend one RGBA video on top of another with:

./ffmpeg_g -y -init_hw_device vaapi=va:/dev/dri/renderD128 -init_hw_device 
opencl=ocl@va -hwaccel vaapi -hwaccel_output_format vaapi -i in1.mp4 -hwaccel 
vaapi -hwaccel_output_format vaapi -i in2.mp4 -an -filter_hw_device ocl 
-filter_complex 
'[0:v]scale_vaapi=format=rgba,hwmap,colorkey_opencl=white:0.4:0.0[over];[1:v]scale_vaapi=format=rgba,hwmap[under];[under][over]overlay_opencl,hwmap=derive_device=vaapi:reverse=1,scale_vaapi=format=nv12[out]'
 -map '[out]' -c:v h264_vaapi out.mp4

(Intel GPU running Beignet for interop, the whole thing manages close to 100fps 
at 1080p.)

Patches welcome to make overlay_opencl not be so flaky, I guess :)

>  configure|   1 +
>  doc/filters.texi |  33 +
>  libavfilter/Makefile |   2 +
>  libavfilter/allfilters.c |   1 +
>  libavfilter/opencl/colorkey.cl   |  45 ++
>  libavfilter/opencl_source.h  |   1 +
>  libavfilter/vf_colorkey_opencl.c | 234 +++
>  7 files changed, 317 insertions(+)
>  create mode 100644 libavfilter/opencl/colorkey.cl
>  create mode 100644 libavfilter/vf_colorkey_opencl.c

git says:

$ cat \[FFmpeg-devel\]\ \[PATCH\]\ lavfi\:\ add\ colorkey_opencl\ filter.eml | 
git am
Applying: lavfi: add colorkey_opencl filter
.git/rebase-apply/patch:234: trailing whitespace.
static int colorkey_opencl_init(AVFilterContext* avctx) 
.git/rebase-apply/patch:169: new blank line at EOF.
+
.git/rebase-apply/patch:421: new blank line at EOF.
+
warning: 3 lines add whitespace errors.

> diff --git a/configure b/configure
> index f6123f53e5..a4dd9ee167 100755
> --- a/configure
> +++ b/configure
> @@ -3410,6 +3410,7 @@ boxblur_filter_deps="gpl"
>  boxblur_opencl_filter_deps="opencl gpl"
>  bs2b_filter_deps="libbs2b"
>  colormatrix_filter_deps="gpl"
> +colorkey_opencl_filter_deps="opencl"
>  convolution_opencl_filter_deps="opencl"
>  convolve_filter_deps="avcodec"
>  convolve_filter_select="fft"
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 867607d870..390c8b97cf 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -19030,6 +19030,39 @@ Apply erosion filter with threshold0 set to 30, 
> threshold1 set 40, threshold2 s

Re: [FFmpeg-devel] [PATCH] lavfi: add colorkey_opencl filter

[Mark Thompson]

On 11/04/2019 03:42, Cld fire wrote:
> On Wed, Apr 10, 2019 at 6:10 PM Mark Thompson  wrote:
>>> +// Make sure the input is a format we support
>>> +if (fmt != AV_PIX_FMT_ARGB &&
>>> +fmt != AV_PIX_FMT_RGBA &&
>>> +fmt != AV_PIX_FMT_ABGR &&
>>> +fmt != AV_PIX_FMT_BGRA &&
>>> +fmt != AV_PIX_FMT_NONE
>>
>> Why NONE here?
>>
> 
> It was in the list of formats that the CPU filter said it supported and,
> after briefly looking around, it appeared
> that it was used as a generic special case of some kind (?). There's no
> documentation on the enum variant,
> though, so wasn't able to figure out its exact meaning. Judging by your
> question, perhaps it is irrelevant for
> HW frames?

You mean the list at 
<http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavfilter/vf_colorkey.c;h=3d65e59d42c6fa480e00f4b7ab079677bcac876a;hb=HEAD#l113>?
  AV_PIX_FMT_NONE (-1) is the invalid enum AVPixelFormat value used as the 
sentinel for the end of the list passed to ff_make_format_list().

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi: add nlmeans_opencl filter

[Mark Thompson]

On 12/04/2019 08:38, Song, Ruiling wrote:
 +#define RELEASE_KERNEL(k)\
 +do { \
 +if (k) { \
 +cle = clReleaseKernel(k);\
 +if (cle != CL_SUCCESS)   \
 +av_log(avctx, AV_LOG_ERROR, "Failed to release " \
 +   "kernel: %d.\n", cle);\
 +}\
 +} while(0)
>>>
>>> This appears multiple times here and also in other filters.  Maybe it 
>>> should be a
>>> macro in opencl.h like CL_SET_KERNEL_ARG?
> Hi Mark,
> 
> I am rethinking about this problem, can we just simply call clReleaseKernel() 
> and not checking the input and the error_code.
> OpenCL spec has require implementation to check the input argument. So I 
> think we can just ignore the if-null check.

I'm not sure that's true?  The spec allows a CL_INVALID_KERNEL error, but 
doesn't offer any clear indication of when it should be returned (NULL is 
distinguished in other cases, but not here).  Random pointers certainly do 
crash implementations, so they aren't interpreting it as a requirement to 
validate the pointer generally (against some list in the context, say).

The standard ICD loader does have a null check returning CL_INVALID_KERNEL, but 
there is no requirement that it is used rather than linking to a particular ICD 
directly.

> As we are destroying the objects, is it still useful to care the error code 
> returned?

Probably not, I agree.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/cbs_av1: add a function to get a payload size without trailing zero bytes

[Mark Thompson]

On 13/04/2019 20:21, James Almer wrote:
> Factor it out from cbs_av1_read_metadata_itut_t35()
> 
> Signed-off-by: James Almer 
> ---
>  libavcodec/cbs_av1.c | 11 +++
>  libavcodec/cbs_av1_syntax_template.c | 10 +-
>  2 files changed, 12 insertions(+), 9 deletions(-)
> 
> diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c
> index f02cca2dad..0c03ca28af 100644
> --- a/libavcodec/cbs_av1.c
> +++ b/libavcodec/cbs_av1.c
> @@ -574,6 +574,17 @@ static int cbs_av1_get_relative_dist(const 
> AV1RawSequenceHeader *seq,
>  return diff;
>  }
>  
> +static int cbs_av1_get_payload_bytes_left(GetBitContext *gbc)
> +{
> +GetBitContext tmp = *gbc;
> +int size = 0;
> +for (int i = 0; get_bits_left(&tmp) >= 8; i++) {
> +if (get_bits(&tmp, 8))
> +size = i;
> +}
> +return size;
> +}

I don't think it matters because of get_bits, but maybe this should be size_t?

> +
>  
>  #define HEADER(name) do { \
>  ff_cbs_trace_header(ctx, name); \
> diff --git a/libavcodec/cbs_av1_syntax_template.c 
> b/libavcodec/cbs_av1_syntax_template.c
> index 76eb90b279..56009145e8 100644
> --- a/libavcodec/cbs_av1_syntax_template.c
> +++ b/libavcodec/cbs_av1_syntax_template.c
> @@ -1674,15 +1674,7 @@ static int 
> FUNC(metadata_itut_t35)(CodedBitstreamContext *ctx, RWContext *rw,
>  #ifdef READ
>  // The payload runs up to the start of the trailing bits, but there might
>  // be arbitrarily many trailing zeroes so we need to read through twice.
> -{
> -GetBitContext tmp = *rw;
> -current->payload_size = 0;
> -for (i = 0; get_bits_left(rw) >= 8; i++) {
> -if (get_bits(rw, 8))
> -current->payload_size = i;
> -}
> -*rw = tmp;
> -}
> +current->payload_size = cbs_av1_get_payload_bytes_left(rw);
>  
>  current->payload_ref = av_buffer_alloc(current->payload_size);
>  if (!current->payload_ref)
> 

LGTM in any case.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/2 v2] avcodec/cbs_av1: add support for Padding OBUs

[Mark Thompson]

On 13/04/2019 20:25, James Almer wrote:
> Based on itut_t35 Matadata OBU parsing code.
> 
> Signed-off-by: James Almer 
> ---
>  libavcodec/cbs_av1.c | 20 
>  libavcodec/cbs_av1.h |  7 +++
>  libavcodec/cbs_av1_syntax_template.c | 24 
>  3 files changed, 51 insertions(+)
> 
> ...
> diff --git a/libavcodec/cbs_av1_syntax_template.c 
> b/libavcodec/cbs_av1_syntax_template.c
> index 56009145e8..675bfe5bb4 100644
> --- a/libavcodec/cbs_av1_syntax_template.c
> +++ b/libavcodec/cbs_av1_syntax_template.c
> @@ -1755,3 +1755,27 @@ static int FUNC(metadata_obu)(CodedBitstreamContext 
> *ctx, RWContext *rw,
>  
>  return 0;
>  }
> +
> +static int FUNC(padding)(CodedBitstreamContext *ctx, RWContext *rw,

The standard calls this function "padding_obu".

> + AV1RawPadding *current)
> +{
> +int i, err;
> +
> +HEADER("Padding");
> +
> +#ifdef READ
> +// The payload runs up to the start of the trailing bits, but there might
> +// be arbitrarily many trailing zeroes so we need to read through twice.
> +current->payload_size = cbs_av1_get_payload_bytes_left(rw);
> +
> +current->payload_ref = av_buffer_alloc(current->payload_size);
> +if (!current->payload_ref)
> +return AVERROR(ENOMEM);
> +current->payload = current->payload_ref->data;
> +#endif
> +
> +for (i = 0; i < current->payload_size; i++)
> +xf(8, obu_padding_byte[i], current->payload[i], 0x00, 0xff, 1, i);
> +
> +return 0;
> +}
> 

LGTM with that.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] lavfi: add colorkey_opencl filter

[Mark Thompson]

On 14/04/2019 05:27, Jarek Samic wrote:
> This is a direct port of the CPU filter.
> 
> Signed-off-by: Jarek Samic 
> ---
> I've made the changes requested from the first patch. I also investigated 
> splitting the kernel into two kernels in order to remove the blending if 
> branch; I noticed negligible performance improvement (if any at all) with my 
> test case and hardware, but I've left it split up as it's possible that it 
> makes a difference with different hardware (and it's very little change in 
> the code).

Fair enough, that makes sense :)

>  configure|   1 +
>  doc/filters.texi |  33 +
>  libavfilter/Makefile |   2 +
>  libavfilter/allfilters.c |   1 +
>  libavfilter/opencl/colorkey.cl   |  53 +++
>  libavfilter/opencl_source.h  |   1 +
>  libavfilter/vf_colorkey_opencl.c | 243 +++
>  7 files changed, 334 insertions(+)
>  create mode 100644 libavfilter/opencl/colorkey.cl
>  create mode 100644 libavfilter/vf_colorkey_opencl.c
> 
> ...
> diff --git a/libavfilter/opencl/colorkey.cl b/libavfilter/opencl/colorkey.cl
> new file mode 100644
> index 00..82ab5c8832
> --- /dev/null
> +++ b/libavfilter/opencl/colorkey.cl
> @@ -0,0 +1,53 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 
> USA
> + */
> +
> +float4 get_pixel(image2d_t src, int2 loc) {
> +const sampler_t sampler = CLK_NORMALIZED_COORDS_FALSE |
> +CLK_FILTER_NEAREST;

The Mali driver doesn't like this:

"""
[Parsed_colorkey_opencl_2 @ 0x83a040c0] Failed to build program: -11.
[Parsed_colorkey_opencl_2 @ 0x83a040c0] Build log:
:21:21: error: declaring sampler variable in this context is not allowed
const sampler_t sampler = CLK_NORMALIZED_COORDS_FALSE |
^

error: Compiler frontend failed (error code 59)
"""

From the standard:

"""
The sampler type (sampler_t) can only be used as the type of a function 
argument or a
variable declared in the program scope or the outermost scope of a kernel 
function. The
behavior of a sampler variable declared in a non-outermost scope of a kernel 
function is
implementation-defined. A sampler argument or variable cannot be modified.
"""

I think move it into the program scope (and then inline the get_pixel function, 
since it no longer does very much).

> +
> +return read_imagef(src, sampler, loc);
> +}
> +
> +__kernel void colorkey_blend(
> +__read_only  image2d_t src,
> +__write_only image2d_t dst,
> +float4 colorkey_rgba,
> +float similarity,
> +float blend
> +) {
> +int2 loc = (int2)(get_global_id(0), get_global_id(1));
> +float4 pixel = get_pixel(src, loc);
> +float diff = distance(pixel.xyz, colorkey_rgba.xyz);
> +
> +pixel.s3 = clamp((diff - similarity) / blend, 0.0f, 1.0f);
> +write_imagef(dst, loc, pixel);
> +}
> +
> +__kernel void colorkey(
> +__read_only  image2d_t src,
> +__write_only image2d_t dst,
> +float4 colorkey_rgba,
> +float similarity
> +) {
> +int2 loc = (int2)(get_global_id(0), get_global_id(1));
> +float4 pixel = get_pixel(src, loc);
> +float diff = distance(pixel.xyz, colorkey_rgba.xyz);
> +
> +pixel.s3 = (diff > similarity) ? 1.0 : 0.0;

1.0f, 0.0f.  (The compiler probably optimises this away, but it's sensible to 
get into the habit of always avoiding doubles.)

> +write_imagef(dst, loc, pixel);
> +}
> diff --git a/libavfilter/opencl_source.h b/libavfilter/opencl_source.h
> index 4118138c30..51f7178cf2 100644
> --- a/libavfilter/opencl_source.h
> +++ b/libavfilter/opencl_source.h
> @@ -20,6 +20,7 @@
>  #define AVFILTER_OPENCL_SOURCE_H
>  
>  extern const char *ff_opencl_source_avgblur;
> +extern const char *ff_opencl_source_colorkey;
>  extern const char *ff_opencl_source_colorspace_common;
>  extern const char *ff_opencl_source_convolution;
>  extern const char *ff_opencl_source_neighbor;
> diff --git a/libavfilter/vf_colorkey_opencl.c 
> b/libavfilter/vf_colorkey_opencl.c
> new file mode 100644
> index 00..2790a01cae
> --- /dev/null
> +++ b/libavfilter/vf_colorkey_opencl.c
> @@ -0,0 +1,243 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribu

Re: [FFmpeg-devel] [PATCH V2 2/2] lavfi/opencl: add nlmeans_opencl filter

[Mark Thompson]

On 12/04/2019 16:09, Ruiling Song wrote:
> Signed-off-by: Ruiling Song 

I can't work out where the problem is, but there is something really weirdly 
nondeterministic going on here.

E.g.

$ ./ffmpeg_g -y -init_hw_device opencl:0.0 -i 
~/video/test/jellyfish-120-mbps-4k-uhd-hevc-10bit.mkv -an -filter_hw_device 
opencl0 -vf format=yuv420p,hwupload,nlmeans_opencl,hwdownload,format=yuv420p 
-frames:v 10 -f framemd5 -
...
0,  0,  0,1, 12441600, 8b8805818076b23ae6f80ec2b5a349d4
0,  1,  1,1, 12441600, 7a7fdaa083dc337cfb6af31b643f30a3
0,  2,  2,1, 12441600, b10ef2a1e5125cc67e262e086f8040b5
0,  3,  3,1, 12441600, c06b53ad90e0357e537df41b63d5b1dc
0,  4,  4,1, 12441600, 5aa2da07703859a3dee080847dd17d46
0,  5,  5,1, 12441600, 733364c6be6af825057e905a6092937d
0,  6,  6,1, 12441600, 47edae2dec956a582b04babb745d26b0
0,  7,  7,1, 12441600, 4e45fe8268df4298d06a17ab8e46c3e9
0,  8,  8,1, 12441600, 960d722a3f8787c9191299a114c04174
0,  9,  9,1, 12441600, e759c07ee4834a9cf94bfcb4128e7612
$ ./ffmpeg_g -y -init_hw_device opencl:0.0 -i 
~/video/test/jellyfish-120-mbps-4k-uhd-hevc-10bit.mkv -an -filter_hw_device 
opencl0 -vf format=yuv420p,hwupload,nlmeans_opencl,hwdownload,format=yuv420p 
-frames:v 10 -f framemd5 -
0,  0,  0,1, 12441600, 8b8805818076b23ae6f80ec2b5a349d4
[Parsed_nlmeans_opencl_2 @ 0x5557ae580d00] integral image overflow 2157538
0,  1,  1,1, 12441600, bce72e10a9f1118940c5a8392ad78ec3
0,  2,  2,1, 12441600, b10ef2a1e5125cc67e262e086f8040b5
0,  3,  3,1, 12441600, c06b53ad90e0357e537df41b63d5b1dc
0,  4,  4,1, 12441600, 5aa2da07703859a3dee080847dd17d46
0,  5,  5,1, 12441600, 733364c6be6af825057e905a6092937d
0,  6,  6,1, 12441600, 47edae2dec956a582b04babb745d26b0
0,  7,  7,1, 12441600, 4e45fe8268df4298d06a17ab8e46c3e9
0,  8,  8,1, 12441600, 960d722a3f8787c9191299a114c04174
0,  9,  9,1, 12441600, e759c07ee4834a9cf94bfcb4128e7612
$ ./ffmpeg_g -y -init_hw_device opencl:0.0 -i 
~/video/test/jellyfish-120-mbps-4k-uhd-hevc-10bit.mkv -an -filter_hw_device 
opencl0 -vf format=yuv420p,hwupload,nlmeans_opencl,hwdownload,format=yuv420p 
-frames:v 10 -f framemd5 -
0,  0,  0,1, 12441600, 8b8805818076b23ae6f80ec2b5a349d4
0,  1,  1,1, 12441600, 7a7fdaa083dc337cfb6af31b643f30a3
[Parsed_nlmeans_opencl_2 @ 0x557c51fbfe80] integral image overflow 2098545
0,  2,  2,1, 12441600, 68b390535adc5cfa0f8a7942c42a47ca
0,  3,  3,1, 12441600, c06b53ad90e0357e537df41b63d5b1dc
0,  4,  4,1, 12441600, 5aa2da07703859a3dee080847dd17d46
0,  5,  5,1, 12441600, 733364c6be6af825057e905a6092937d
0,  6,  6,1, 12441600, 47edae2dec956a582b04babb745d26b0
0,  7,  7,1, 12441600, 4e45fe8268df4298d06a17ab8e46c3e9
0,  8,  8,1, 12441600, 960d722a3f8787c9191299a114c04174
0,  9,  9,1, 12441600, e759c07ee4834a9cf94bfcb4128e7612

Frame 1 gave an overflow on the second run, and gets a different answer, then 
frame 2 in the same way on the third run?  I can't characterise when this 
happens, it seems to be pretty random with low probability.

(Input here is a 4K file from , but I don't think it 
matters - I saw it with others sometimes as well.)

>  configure   |   1 +
>  doc/filters.texi|   4 +
>  libavfilter/Makefile|   1 +
>  libavfilter/allfilters.c|   1 +
>  libavfilter/opencl/nlmeans.cl   | 115 +
>  libavfilter/opencl_source.h |   1 +
>  libavfilter/vf_nlmeans_opencl.c | 442 
>  7 files changed, 565 insertions(+)
>  create mode 100644 libavfilter/opencl/nlmeans.cl
>  create mode 100644 libavfilter/vf_nlmeans_opencl.c

Code all looks fine, as far as I can tell.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH V2 1/2] lavfi/opencl: add more opencl helper macro

[Mark Thompson]

On 12/04/2019 16:09, Ruiling Song wrote:
> Signed-off-by: Ruiling Song 
> ---
>  libavfilter/opencl.h | 38 ++
>  1 file changed, 38 insertions(+)
> 
> diff --git a/libavfilter/opencl.h b/libavfilter/opencl.h
> index 0b06232ade..0fa5b49d3f 100644
> --- a/libavfilter/opencl.h
> +++ b/libavfilter/opencl.h
> @@ -73,6 +73,44 @@ typedef struct OpenCLFilterContext {
>  goto fail; \
>  }  \
>  } while(0)
> +/**
> +  * release an OpenCL Kernel
> +  */
> +#define CL_RELEASE_KERNEL(k)  \
> +do {  \
> +if (k) {  \
> +cle = clReleaseKernel(k); \
> +if (cle != CL_SUCCESS)\
> +av_log(avctx, AV_LOG_ERROR, "Failed to release "  \
> +   "OpenCL kernel: %d.\n", cle);  \
> +} \
> +} while(0)
> +
> +/**
> +  * release an OpenCL Memory Object
> +  */
> +#define CL_RELEASE_MEMORY(m)  \
> +do {  \
> +if (m) {  \
> +cle = clReleaseMemObject(m);  \
> +if (cle != CL_SUCCESS)\
> +av_log(avctx, AV_LOG_ERROR, "Failed to release "  \
> +   "OpenCL memory: %d.\n", cle);  \
> +} \
> +} while(0)
> +
> +/**
> +  * release an OpenCL Command Queue
> +  */
> +#define CL_RELEASE_QUEUE(q)   \
> +do {  \
> +if (q) {  \
> +cle = clReleaseCommandQueue(q);   \
> +if (cle != CL_SUCCESS)\
> +av_log(avctx, AV_LOG_ERROR, "Failed to release "  \
> +   "cl command queue: %d.\n", cle);   \
> +} \
> +} while(0)
>  
>  /**
>   * Return that all inputs and outputs support only AV_PIX_FMT_OPENCL.
> 

LGTM.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/h264_ps: fix storage size for offset_for_ref_frame

[Mark Thompson]

On 11/04/2019 04:10, James Almer wrote:
> On 4/10/2019 3:30 PM, James Almer wrote:
>> The spec defines the valid range of values to be INT32_MIN + 1 to INT32_MAX, 
>> inclusive.
>>
>> Signed-off-by: James Almer 
>> ---
>> A good example of why making offsets and sizes of structs like this tied to 
>> the
>> ABI is not a good idea.
>>
>>  libavcodec/h264_ps.h | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/libavcodec/h264_ps.h b/libavcodec/h264_ps.h
>> index e967b9cbcf..9014326dfb 100644
>> --- a/libavcodec/h264_ps.h
>> +++ b/libavcodec/h264_ps.h
>> @@ -81,7 +81,7 @@ typedef struct SPS {
>>  uint32_t num_units_in_tick;
>>  uint32_t time_scale;
>>  int fixed_frame_rate_flag;
>> -short offset_for_ref_frame[256]; // FIXME dyn aloc?
>> +int32_t offset_for_ref_frame[256];
> 
> The doxy for get_se_golomb() doesn't mention the range of values it can
> handle, but seeing there's also a get_se_golomb_long(), I guess the
> relevant line in h264_ps.c should now use the latter instead?

I think it's correct to do that.  Seems highly unlikely anyone would ever hit 
it outside a conformance-checking context, though - using anything other than 
pic_order_cnt_type 0 for nontrivial reference structures is madness.

>>  int bitstream_restriction_flag;
>>  int num_reorder_frames;
>>  int scaling_matrix_present;

There are some other fields with int32_t range which are using get_se_golomb() 
- e.g. offset_for_non_ref_pic.  I guess they should use get_se_golomb_long() as 
above.  They're also plain ints - do they want to be explicitly int32_t?

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/cbs_av1: add support for Scalability Metadata

[Mark Thompson]

On 14/04/2019 23:04, James Almer wrote:
> Signed-off-by: James Almer 
> ---
> This will make the AV1RawObu struct weigh about ~3kb instead of ~1kb.
> 
>  libavcodec/av1.h | 33 
>  libavcodec/cbs_av1.h | 15 +-
>  libavcodec/cbs_av1_syntax_template.c | 45 ++--
>  3 files changed, 90 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/av1.h b/libavcodec/av1.h
> index f2ec39c86b..6c0e32485b 100644
> --- a/libavcodec/av1.h
> +++ b/libavcodec/av1.h
> @@ -127,4 +127,37 @@ enum {
>  AV1_CSP_COLOCATED = 2, // -> AVCHROMA_LOC_TOPLEFT.
>  };
>  
> +// Scalability modes (section 6.7.5)
> +enum {
> +AV1_SCALABILITY_L1T2 = 0,
> +AV1_SCALABILITY_L1T3 = 1,
> +AV1_SCALABILITY_L2T1 = 2,
> +AV1_SCALABILITY_L2T2 = 3,
> +AV1_SCALABILITY_L2T3 = 4,
> +AV1_SCALABILITY_S2T1 = 5,
> +AV1_SCALABILITY_S2T2 = 6,
> +AV1_SCALABILITY_S2T3 = 7,
> +AV1_SCALABILITY_L2T1h = 8,
> +AV1_SCALABILITY_L2T2h = 9,
> +AV1_SCALABILITY_L2T3h = 10,
> +AV1_SCALABILITY_S2T1h = 11,
> +AV1_SCALABILITY_S2T2h = 12,
> +AV1_SCALABILITY_S2T3h = 13,
> +AV1_SCALABILITY_SS = 14,
> +AV1_SCALABILITY_L3T1 = 15,
> +AV1_SCALABILITY_L3T2 = 16,
> +AV1_SCALABILITY_L3T3 = 17,
> +AV1_SCALABILITY_S3T1 = 18,
> +AV1_SCALABILITY_S3T2 = 19,
> +AV1_SCALABILITY_S3T3 = 20,
> +AV1_SCALABILITY_L3T2_KEY = 21,
> +AV1_SCALABILITY_L3T3_KEY = 22,
> +AV1_SCALABILITY_L4T5_KEY = 23,
> +AV1_SCALABILITY_L4T7_KEY = 24,
> +AV1_SCALABILITY_L3T2_KEY_SHIFT = 25,
> +AV1_SCALABILITY_L3T3_KEY_SHIFT = 26,
> +AV1_SCALABILITY_L4T5_KEY_SHIFT = 27,
> +AV1_SCALABILITY_L4T7_KEY_SHIFT = 28,
> +};
> +
>  #endif /* AVCODEC_AV1_H */
> diff --git a/libavcodec/cbs_av1.h b/libavcodec/cbs_av1.h
> index e799964b72..1fb668ada4 100644
> --- a/libavcodec/cbs_av1.h
> +++ b/libavcodec/cbs_av1.h
> @@ -325,7 +325,20 @@ typedef struct AV1RawMetadataHDRMDCV {
>  
>  typedef struct AV1RawMetadataScalability {
>  uint8_t scalability_mode_idc;
> -// TODO: more stuff.
> +uint8_t spatial_layers_cnt_minus_1;
> +uint8_t spatial_layer_dimensions_present_flag;
> +uint8_t spatial_layer_description_present_flag;
> +uint8_t temporal_group_description_present_flag;
> +uint8_t scalability_structure_reserved_3bits;
> +uint16_t spatial_layer_max_width[4];
> +uint16_t spatial_layer_max_height[4];
> +uint8_t spatial_layer_ref_id[4];
> +uint8_t temporal_group_size;
> +uint8_t temporal_group_temporal_id[255];
> +uint8_t temporal_group_temporal_switching_up_point_flag[255];
> +uint8_t temporal_group_spatial_switching_up_point_flag[255];
> +uint8_t temporal_group_ref_cnt[255];
> +uint8_t temporal_group_ref_pic_diff[255][7];
>  } AV1RawMetadataScalability;
>  
>  typedef struct AV1RawMetadataITUTT35 {
> diff --git a/libavcodec/cbs_av1_syntax_template.c 
> b/libavcodec/cbs_av1_syntax_template.c
> index 0e019aa113..ef48173470 100644
> --- a/libavcodec/cbs_av1_syntax_template.c
> +++ b/libavcodec/cbs_av1_syntax_template.c
> @@ -1653,12 +1653,53 @@ static int 
> FUNC(metadata_hdr_mdcv)(CodedBitstreamContext *ctx, RWContext *rw,
>  return 0;
>  }
>  
> +static int FUNC(scalability_structure)(CodedBitstreamContext *ctx, RWContext 
> *rw,
> +   AV1RawMetadataScalability *current)
> +{
> +int err, i, j;
> +
> +fb(2, spatial_layers_cnt_minus_1);
> +flag(spatial_layer_dimensions_present_flag);
> +flag(spatial_layer_description_present_flag);
> +flag(temporal_group_description_present_flag);
> +fc(3, scalability_structure_reserved_3bits, 0, 0);
> +if (current->spatial_layer_dimensions_present_flag) {
> +for (i = 0; i <= current->spatial_layers_cnt_minus_1; i++) {
> +fbs(16, spatial_layer_max_width[i], 1, i);
> +fbs(16, spatial_layer_max_height[i], 1, i);

Can we verify against the "must not be larger than max_frame_*_minus_1 + 1" 
constraint here?  (I think a sequence header should be available.)

> +}
> +}
> +if (current->spatial_layer_description_present_flag) {
> +for (i = 0; i <= current->spatial_layers_cnt_minus_1; i++)
> +fbs(8, spatial_layer_ref_id[i], 1, i);
> +}
> +if (current->temporal_group_description_present_flag) {
> +fb(8, temporal_group_size);
> +for (i = 0; i < current->temporal_group_size; i++) {
> +fbs(3, temporal_group_temporal_id[i], 1, i);
> +flags(temporal_group_temporal_switching_up_point_flag[i], 1, i);
> +flags(temporal_group_spatial_switching_up_point_flag[i], 1, i);
> +fbs(3, temporal_group_ref_cnt[i], 1, i);
> +for (j = 0; j < current->temporal_group_ref_cnt[i]; j++) {
> +fbs(8, temporal_group_ref_pic_diff[i][j], 2, i, j);
> +}
> +}
> +}
> +
> +return 0;
> +}
> +
>  static int FUNC(metadata_scalabili

Re: [FFmpeg-devel] [PATCH 1/5] avcodec/cbs: add helper functions and macros to read and write signed values

[Mark Thompson]

On 15/04/2019 22:17, James Almer wrote:
> Signed-off-by: James Almer 
> ---
>  libavcodec/cbs.c  | 79 +++
>  libavcodec/cbs_internal.h | 20 +-
>  2 files changed, 98 insertions(+), 1 deletion(-)

Looks like a sensible addition, some comments below.

> diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c
> index c388be896b..726bd582f5 100644
> --- a/libavcodec/cbs.c
> +++ b/libavcodec/cbs.c
> @@ -504,6 +504,85 @@ int ff_cbs_write_unsigned(CodedBitstreamContext *ctx, 
> PutBitContext *pbc,
>  return 0;
>  }
>  
> +int ff_cbs_read_signed(CodedBitstreamContext *ctx, GetBitContext *gbc,
> +   int width, const char *name,
> +   const int *subscripts, int32_t *write_to,
> +   int32_t range_min, int32_t range_max)
> +{
> +int32_t value;
> +int position;
> +
> +av_assert0(width > 0 && width <= 32);
> +
> +if (get_bits_left(gbc) < width) {
> +av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid value at "
> +   "%s: bitstream ended.\n", name);
> +return AVERROR_INVALIDDATA;
> +}
> +
> +if (ctx->trace_enable)
> +position = get_bits_count(gbc);
> +
> +value = get_sbits_long(gbc, width);
> +
> +if (ctx->trace_enable) {
> +char bits[33];
> +int i;
> +for (i = 0; i < width; i++)
> +bits[i] = value & (1 << (width - i - 1)) ? '1' : '0';

1 << 31 is undefined behaviour for 32-bit int.

The unsigned versions are careful to right-shift unsigned values to avoid 
overflow problems; a possible fix might be to strip the sign bit and then do 
the same thing.

> +bits[i] = 0;
> +
> +ff_cbs_trace_syntax_element(ctx, position, name, subscripts,
> +bits, value);
> +}
> +
> +if (value < range_min || value > range_max) {
> +av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: "
> +   "%"PRId32", but must be in [%"PRId32",%"PRId32"].\n",
> +   name, value, range_min, range_max);
> +return AVERROR_INVALIDDATA;
> +}
> +
> +*write_to = value;
> +return 0;
> +}
> +
> +int ff_cbs_write_signed(CodedBitstreamContext *ctx, PutBitContext *pbc,
> +int width, const char *name,
> +const int *subscripts, int32_t value,
> +int32_t range_min, int32_t range_max)
> +{
> +av_assert0(width > 0 && width <= 32);
> +
> +if (value < range_min || value > range_max) {
> +av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: "
> +   "%"PRId32", but must be in [%"PRId32",%"PRId32"].\n",
> +   name, value, range_min, range_max);
> +return AVERROR_INVALIDDATA;
> +}
> +
> +if (put_bits_left(pbc) < width)
> +return AVERROR(ENOSPC);
> +
> +if (ctx->trace_enable) {
> +char bits[33];
> +int i;
> +for (i = 0; i < width; i++)
> +bits[i] = value & (1 << (width - i - 1)) ? '1' : '0';

As above.

> +bits[i] = 0;
> +
> +ff_cbs_trace_syntax_element(ctx, put_bits_count(pbc),
> +name, subscripts, bits, value);
> +}
> +
> +if (width < 32)
> +put_sbits(pbc, width, value);
> +else
> +put_bits32(pbc, value);
> +
> +return 0;
> +}
> +
>  
>  int ff_cbs_alloc_unit_content(CodedBitstreamContext *ctx,
>CodedBitstreamUnit *unit,
> diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h
> index 53f2e5d187..6ab85679dd 100644
> --- a/libavcodec/cbs_internal.h
> +++ b/libavcodec/cbs_internal.h
> @@ -81,10 +81,28 @@ int ff_cbs_write_unsigned(CodedBitstreamContext *ctx, 
> PutBitContext *pbc,
>const int *subscripts, uint32_t value,
>uint32_t range_min, uint32_t range_max);
>  
> -// The largest value representable in N bits, suitable for use as
> +int ff_cbs_read_signed(CodedBitstreamContext *ctx, GetBitContext *gbc,
> +   int width, const char *name,
> +   const int *subscripts, int32_t *write_to,
> +   int32_t range_min, int32_t range_max);
> +
> +int ff_cbs_write_signed(CodedBitstreamContext *ctx, PutBitContext *pbc,
> +int width, const char *name,
> +const int *subscripts, int32_t value,
> +int32_t range_min, int32_t range_max);
> +
> +// The largest unsigned value representable in N bits, suitable for use as
>  // range_max in the above functions.
>  #define MAX_UINT_BITS(length) ((UINT64_C(1) << (length)) - 1)
>  
> +// The largest signed value representable in N bits, suitable for use as
> +// range_max in the above functions.
> +#define MAX_INT_BITS(length) ((INT64_C(1) << (length - 1)) - 1)

Not so good for, say, MAX_INT_BITS(a ? b : c).

> +
> +// The smallest signed value representable in N

Re: [FFmpeg-devel] [PATCH 2/5] avcodec/cbs_h2645: add helper macros for signed values

[Mark Thompson]

On 15/04/2019 22:17, James Almer wrote:
> Signed-off-by: James Almer 
> ---
>  libavcodec/cbs_h2645.c | 20 
>  1 file changed, 20 insertions(+)
> 
> diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c
> index e74f8dce81..a205293b3c 100644
> --- a/libavcodec/cbs_h2645.c
> +++ b/libavcodec/cbs_h2645.c
> @@ -255,6 +255,8 @@ static int cbs_write_se_golomb(CodedBitstreamContext 
> *ctx, PutBitContext *pbc,
>  #define flag(name) u(1, name, 0, 1)
>  #define ue(name, range_min, range_max) \
>  xue(name, current->name, range_min, range_max, 0)
> +#define i(width, name, range_min, range_max) \
> +xi(width, name, current->name, range_min, range_max, 0)

I know it's right, but defining "i" as a macro feels like a pretty terrible 
idea in C :P

>  #define se(name, range_min, range_max) \
>  xse(name, current->name, range_min, range_max, 0)
>  
> @@ -264,6 +266,8 @@ static int cbs_write_se_golomb(CodedBitstreamContext 
> *ctx, PutBitContext *pbc,
>  xu(1, name, current->name, 0, 1, subs, __VA_ARGS__)
>  #define ues(name, range_min, range_max, subs, ...) \
>  xue(name, current->name, range_min, range_max, subs, __VA_ARGS__)
> +#define is(width, name, range_min, range_max, subs, ...) \
> +xi(width, name, current->name, range_min, range_max, subs, 
> __VA_ARGS__)
>  #define ses(name, range_min, range_max, subs, ...) \
>  xse(name, current->name, range_min, range_max, subs, __VA_ARGS__)
>  
> @@ -291,6 +295,13 @@ static int cbs_write_se_golomb(CodedBitstreamContext 
> *ctx, PutBitContext *pbc,
>   &value, range_min, range_max)); \
>  var = value; \
>  } while (0)
> +#define xi(width, name, var, range_min, range_max, subs, ...) do { \
> +int32_t value = range_min; \
> +CHECK(ff_cbs_read_signed(ctx, rw, width, #name, \
> + SUBSCRIPTS(subs, __VA_ARGS__), \
> + &value, range_min, range_max)); \
> +var = value; \
> +} while (0)
>  #define xse(name, var, range_min, range_max, subs, ...) do { \
>  int32_t value = range_min; \
>  CHECK(cbs_read_se_golomb(ctx, rw, #name, \
> @@ -338,6 +349,7 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext 
> *gbc)
>  #undef READWRITE
>  #undef RWContext
>  #undef xu
> +#undef xi
>  #undef xue
>  #undef xse
>  #undef infer
> @@ -362,6 +374,12 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext 
> *gbc)
>SUBSCRIPTS(subs, __VA_ARGS__), \
>value, range_min, range_max)); \
>  } while (0)
> +#define xi(width, name, var, range_min, range_max, subs, ...) do { \
> +int32_t value = var; \
> +CHECK(ff_cbs_write_signed(ctx, rw, width, #name, \
> +  SUBSCRIPTS(subs, __VA_ARGS__), \
> +  value, range_min, range_max)); \
> +} while (0)
>  #define xse(name, var, range_min, range_max, subs, ...) do { \
>  int32_t value = var; \
>  CHECK(cbs_write_se_golomb(ctx, rw, #name, \
> @@ -402,9 +420,11 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext 
> *gbc)
>  #undef READWRITE
>  #undef RWContext
>  #undef xu
> +#undef xi
>  #undef xue
>  #undef xse
>  #undef u
> +#undef i
>  #undef flag
>  #undef ue
>  #undef se
> 

LGTM.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/5] avcodec/cbs_h264: fix storage type for time_offset in Pic Timing SEI

[Mark Thompson]

On 15/04/2019 22:17, James Almer wrote:
> The spec defines it as a signed value.
> 
> Signed-off-by: James Almer 
> ---
> The only sample i could find with time_offset values it's in the fate suite,
> and in all cases it's 0.
> 
>  libavcodec/cbs_h264.h | 2 +-
>  libavcodec/cbs_h264_syntax_template.c | 5 +++--
>  2 files changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/cbs_h264.h b/libavcodec/cbs_h264.h
> index 92277e4750..b5eee7c370 100644
> --- a/libavcodec/cbs_h264.h
> +++ b/libavcodec/cbs_h264.h
> @@ -253,7 +253,7 @@ typedef struct H264RawSEIPicTimestamp {
>  uint8_t minutes_value;
>  uint8_t hours_flag;
>  uint8_t hours_value;
> -uint32_t time_offset;
> +int32_t time_offset;
>  } H264RawSEIPicTimestamp;
>  
>  typedef struct H264RawSEIPicTiming {
> diff --git a/libavcodec/cbs_h264_syntax_template.c 
> b/libavcodec/cbs_h264_syntax_template.c
> index 4da4c5da67..07b4cddb5e 100644
> --- a/libavcodec/cbs_h264_syntax_template.c
> +++ b/libavcodec/cbs_h264_syntax_template.c
> @@ -592,8 +592,9 @@ static int FUNC(sei_pic_timestamp)(CodedBitstreamContext 
> *ctx, RWContext *rw,
>  time_offset_length = 24;
>  
>  if (time_offset_length > 0)
> -u(time_offset_length, time_offset,
> -  0, MAX_UINT_BITS(time_offset_length));
> +i(time_offset_length, time_offset,
> +  MIN_INT_BITS(time_offset_length),
> +  MAX_INT_BITS(time_offset_length));
>  else
>  infer(time_offset, 0);
>  
> 

LGTM.

I'm glad the standard gets plenty of use out of that i(v) syntax element type 
definition.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 4/5] avcodec/cbs_h265: fix storage type for time_offset_value in Time Code SEI

[Mark Thompson]

On 15/04/2019 22:17, James Almer wrote:
> The spec defines it as an array of signed values, inferred to 0 when not
> present.
> 
> Signed-off-by: James Almer 
> ---
> Couldn't find any sample using it.
> 
>  libavcodec/cbs_h265.h | 2 +-
>  libavcodec/cbs_h265_syntax_template.c | 7 +--
>  2 files changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/cbs_h265.h b/libavcodec/cbs_h265.h
> index d216caca2b..0c0e4f84b0 100644
> --- a/libavcodec/cbs_h265.h
> +++ b/libavcodec/cbs_h265.h
> @@ -659,7 +659,7 @@ typedef struct H265RawSEITimeCode {
>  uint8_t  minutes_flag[3];
>  uint8_t  hours_flag[3];
>  uint8_t  time_offset_length[3];
> -uint32_t time_offset_value[3];
> +int32_t  time_offset_value[3];
>  } H265RawSEITimeCode;
>  
>  typedef struct H265RawSEIMasteringDisplayColourVolume {
> diff --git a/libavcodec/cbs_h265_syntax_template.c 
> b/libavcodec/cbs_h265_syntax_template.c
> index f1e1bb0e7e..bbd363a389 100644
> --- a/libavcodec/cbs_h265_syntax_template.c
> +++ b/libavcodec/cbs_h265_syntax_template.c
> @@ -1986,8 +1986,11 @@ static int FUNC(sei_time_code)(CodedBitstreamContext 
> *ctx, RWContext *rw,
>  
>  us(5, time_offset_length[i], 0, 31, 1, i);
>  if (current->time_offset_length[i] > 0)
> -us(current->time_offset_length[i], time_offset_value[i],
> -   0, MAX_UINT_BITS(current->time_offset_length[i]), 1, i);
> +is(current->time_offset_length[i], time_offset_value[i],
> +   MIN_INT_BITS(current->time_offset_length[i]),
> +   MAX_INT_BITS(current->time_offset_length[i]), 1, i);
> +else
> +infer(time_offset_value[i], 0);
>  }
>  }
>  
> 

LGTM.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 5/5] avcodec/cbs_av1: use the new signed value read/write functions

[Mark Thompson]

On 15/04/2019 22:17, James Almer wrote:
> Signed-off-by: James Almer 
> ---
> The sample https://0x0.st/sljR.webm appears to be parsed the exact same way
> after this patch.
> 
>  libavcodec/cbs_av1.c | 68 ++--
>  1 file changed, 8 insertions(+), 60 deletions(-)
> 
> diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c
> index 41dd4c97ac..eb2d03ef43 100644
> --- a/libavcodec/cbs_av1.c
> +++ b/libavcodec/cbs_av1.c
> @@ -207,62 +207,6 @@ static int cbs_av1_write_leb128(CodedBitstreamContext 
> *ctx, PutBitContext *pbc,
>  return 0;
>  }
>  
> -static int cbs_av1_read_su(CodedBitstreamContext *ctx, GetBitContext *gbc,
> -   int width, const char *name,
> -   const int *subscripts, int32_t *write_to)
> -{
> -int position;
> -int32_t value;
> -
> -if (ctx->trace_enable)
> -position = get_bits_count(gbc);
> -
> -if (get_bits_left(gbc) < width) {
> -av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid signed value at "
> -   "%s: bitstream ended.\n", name);
> -return AVERROR_INVALIDDATA;
> -}
> -
> -value = get_sbits(gbc, width);
> -
> -if (ctx->trace_enable) {
> -char bits[33];
> -int i;
> -for (i = 0; i < width; i++)
> -bits[i] = value & (1 << (width - i - 1)) ? '1' : '0';

Ohright, it was bad here too.  (But only ever used on small values in AV1, so 
it couldn't actually be hit.)

> -bits[i] = 0;
> -
> -ff_cbs_trace_syntax_element(ctx, position,
> -name, subscripts, bits, value);
> -}
> -
> -*write_to = value;
> -return 0;
> -}
> -
> -static int cbs_av1_write_su(CodedBitstreamContext *ctx, PutBitContext *pbc,
> -int width, const char *name,
> -const int *subscripts, int32_t value)
> -{
> -if (put_bits_left(pbc) < width)
> -return AVERROR(ENOSPC);
> -
> -if (ctx->trace_enable) {
> -char bits[33];
> -int i;
> -for (i = 0; i < width; i++)
> -bits[i] = value & (1 << (width - i - 1)) ? '1' : '0';
> -bits[i] = 0;
> -
> -ff_cbs_trace_syntax_element(ctx, put_bits_count(pbc),
> -name, subscripts, bits, value);
> -}
> -
> -put_sbits(pbc, width, value);
> -
> -return 0;
> -}
> -
>  static int cbs_av1_read_ns(CodedBitstreamContext *ctx, GetBitContext *gbc,
> uint32_t n, const char *name,
> const int *subscripts, uint32_t *write_to)
> @@ -639,8 +583,10 @@ static size_t 
> cbs_av1_get_payload_bytes_left(GetBitContext *gbc)
>  
>  #define xsu(width, name, var, subs, ...) do { \
>  int32_t value = 0; \
> -CHECK(cbs_av1_read_su(ctx, rw, width, #name, \
> -  SUBSCRIPTS(subs, __VA_ARGS__), &value)); \
> +CHECK(ff_cbs_read_signed(ctx, rw, width, #name, \
> + SUBSCRIPTS(subs, __VA_ARGS__), &value, \
> + MIN_INT_BITS(width), \
> + MAX_INT_BITS(width))); \
>  var = value; \
>  } while (0)
>  
> @@ -723,8 +669,10 @@ static size_t 
> cbs_av1_get_payload_bytes_left(GetBitContext *gbc)
>  } while (0)
>  
>  #define xsu(width, name, var, subs, ...) do { \
> -CHECK(cbs_av1_write_su(ctx, rw, width, #name, \
> -   SUBSCRIPTS(subs, __VA_ARGS__), var)); \
> +CHECK(ff_cbs_write_signed(ctx, rw, width, #name, \
> +  SUBSCRIPTS(subs, __VA_ARGS__), var, \
> +  MIN_INT_BITS(width), \
> +  MAX_INT_BITS(width))); \
>  } while (0)
>  
>  #define uvlc(name, range_min, range_max) do { \
> 

LGTM.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/5] avcodec/cbs_h2645: add helper macros for signed values

[Mark Thompson]

On 17/04/2019 00:01, James Almer wrote:
> On 4/16/2019 7:57 PM, Mark Thompson wrote:
>> On 15/04/2019 22:17, James Almer wrote:
>>> Signed-off-by: James Almer 
>>> ---
>>>  libavcodec/cbs_h2645.c | 20 
>>>  1 file changed, 20 insertions(+)
>>>
>>> diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c
>>> index e74f8dce81..a205293b3c 100644
>>> --- a/libavcodec/cbs_h2645.c
>>> +++ b/libavcodec/cbs_h2645.c
>>> @@ -255,6 +255,8 @@ static int cbs_write_se_golomb(CodedBitstreamContext 
>>> *ctx, PutBitContext *pbc,
>>>  #define flag(name) u(1, name, 0, 1)
>>>  #define ue(name, range_min, range_max) \
>>>  xue(name, current->name, range_min, range_max, 0)
>>> +#define i(width, name, range_min, range_max) \
>>> +xi(width, name, current->name, range_min, range_max, 0)
>>
>> I know it's right, but defining "i" as a macro feels like a pretty terrible 
>> idea in C :P
> 
> Eh, i guess :p. It at least didn't blow up here.
> 
> What would be best? s? su like in cbs_av1? Something else?

To clarify, I do think it's correct to call it i() here to match the H.26[45] 
standards.

Any other option would probably be more confusing.  And, as you note, it 
actually doesn't blow up because none of the many uses of i are followed by (.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/5] avcodec/cbs: add helper functions and macros to read and write signed values

[Mark Thompson]

On 16/04/2019 23:54, James Almer wrote:
> On 4/16/2019 7:45 PM, Mark Thompson wrote:
>> On 15/04/2019 22:17, James Almer wrote:
>>> Signed-off-by: James Almer 
>>> ---
>>>  libavcodec/cbs.c  | 79 +++
>>>  libavcodec/cbs_internal.h | 20 +-
>>>  2 files changed, 98 insertions(+), 1 deletion(-)
>>
>> Looks like a sensible addition, some comments below.
>>
>>> diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c
>>> index c388be896b..726bd582f5 100644
>>> --- a/libavcodec/cbs.c
>>> +++ b/libavcodec/cbs.c
>>> @@ -504,6 +504,85 @@ int ff_cbs_write_unsigned(CodedBitstreamContext *ctx, 
>>> PutBitContext *pbc,
>>>  return 0;
>>>  }
>>>  
>>> +int ff_cbs_read_signed(CodedBitstreamContext *ctx, GetBitContext *gbc,
>>> +   int width, const char *name,
>>> +   const int *subscripts, int32_t *write_to,
>>> +   int32_t range_min, int32_t range_max)
>>> +{
>>> +int32_t value;
>>> +int position;
>>> +
>>> +av_assert0(width > 0 && width <= 32);
>>> +
>>> +if (get_bits_left(gbc) < width) {
>>> +av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid value at "
>>> +   "%s: bitstream ended.\n", name);
>>> +return AVERROR_INVALIDDATA;
>>> +}
>>> +
>>> +if (ctx->trace_enable)
>>> +position = get_bits_count(gbc);
>>> +
>>> +value = get_sbits_long(gbc, width);
>>> +
>>> +if (ctx->trace_enable) {
>>> +char bits[33];
>>> +int i;
>>> +for (i = 0; i < width; i++)
>>> +bits[i] = value & (1 << (width - i - 1)) ? '1' : '0';
>>
>> 1 << 31 is undefined behaviour for 32-bit int.
>>
>> The unsigned versions are careful to right-shift unsigned values to avoid 
>> overflow problems; a possible fix might be to strip the sign bit and then do 
>> the same thing.
> 
> Would "1U << (width - i - 1)" be enough?

Probably?  A negative value would be promoted to unsigned as positive , 
which I think does the right thing.

>>> +bits[i] = 0;
>>> +
>>> +ff_cbs_trace_syntax_element(ctx, position, name, subscripts,
>>> +bits, value);
>>> +}
>>> +
>>> +if (value < range_min || value > range_max) {
>>> +av_log(ctx->log_ctx, AV_LOG_ERROR, "%s out of range: "
>>> +   "%"PRId32", but must be in [%"PRId32",%"PRId32"].\n",
>>> +   name, value, range_min, range_max);
>>> +return AVERROR_INVALIDDATA;
>>> +}
>>> +
>>> +*write_to = value;
>>> +return 0;
>>> +}

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH V2 2/2] lavfi/opencl: add nlmeans_opencl filter

[Mark Thompson]

On 17/04/2019 03:43, Song, Ruiling wrote:
>> -Original Message-
>> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf Of
>> Mark Thompson
>> Sent: Wednesday, April 17, 2019 5:28 AM
>> To: ffmpeg-devel@ffmpeg.org
>> Subject: Re: [FFmpeg-devel] [PATCH V2 2/2] lavfi/opencl: add nlmeans_opencl
>> filter
>>
>> On 12/04/2019 16:09, Ruiling Song wrote:
>>> Signed-off-by: Ruiling Song 
>>
>> I can't work out where the problem is, but there is something really weirdly
>> nondeterministic going on here.
>>
>> E.g.
>>
>> $ ./ffmpeg_g -y -init_hw_device opencl:0.0 -i 
>> ~/video/test/jellyfish-120-mbps-
>> 4k-uhd-hevc-10bit.mkv -an -filter_hw_device opencl0 -vf
>> format=yuv420p,hwupload,nlmeans_opencl,hwdownload,format=yuv420p -
>> frames:v 10 -f framemd5 -
>> ...
>> 0,  0,  0,1, 12441600, 
>> 8b8805818076b23ae6f80ec2b5a349d4
>> 0,  1,  1,1, 12441600, 
>> 7a7fdaa083dc337cfb6af31b643f30a3
>> 0,  2,  2,1, 12441600, 
>> b10ef2a1e5125cc67e262e086f8040b5
>> 0,  3,  3,1, 12441600, 
>> c06b53ad90e0357e537df41b63d5b1dc
>> 0,  4,  4,1, 12441600, 
>> 5aa2da07703859a3dee080847dd17d46
>> 0,  5,  5,1, 12441600, 
>> 733364c6be6af825057e905a6092937d
>> 0,  6,  6,1, 12441600, 
>> 47edae2dec956a582b04babb745d26b0
>> 0,  7,  7,1, 12441600, 
>> 4e45fe8268df4298d06a17ab8e46c3e9
>> 0,  8,  8,1, 12441600, 
>> 960d722a3f8787c9191299a114c04174
>> 0,  9,  9,1, 12441600, 
>> e759c07ee4834a9cf94bfcb4128e7612
>> $ ./ffmpeg_g -y -init_hw_device opencl:0.0 -i 
>> ~/video/test/jellyfish-120-mbps-
>> 4k-uhd-hevc-10bit.mkv -an -filter_hw_device opencl0 -vf
>> format=yuv420p,hwupload,nlmeans_opencl,hwdownload,format=yuv420p -
>> frames:v 10 -f framemd5 -
>> 0,  0,  0,1, 12441600, 
>> 8b8805818076b23ae6f80ec2b5a349d4
>> [Parsed_nlmeans_opencl_2 @ 0x5557ae580d00] integral image overflow
>> 2157538
>> 0,  1,  1,1, 12441600, 
>> bce72e10a9f1118940c5a8392ad78ec3
>> 0,  2,  2,1, 12441600, 
>> b10ef2a1e5125cc67e262e086f8040b5
>> 0,  3,  3,1, 12441600, 
>> c06b53ad90e0357e537df41b63d5b1dc
>> 0,  4,  4,1, 12441600, 
>> 5aa2da07703859a3dee080847dd17d46
>> 0,  5,  5,1, 12441600, 
>> 733364c6be6af825057e905a6092937d
>> 0,  6,  6,1, 12441600, 
>> 47edae2dec956a582b04babb745d26b0
>> 0,  7,  7,1, 12441600, 
>> 4e45fe8268df4298d06a17ab8e46c3e9
>> 0,  8,  8,1, 12441600, 
>> 960d722a3f8787c9191299a114c04174
>> 0,  9,  9,1, 12441600, 
>> e759c07ee4834a9cf94bfcb4128e7612
>> $ ./ffmpeg_g -y -init_hw_device opencl:0.0 -i 
>> ~/video/test/jellyfish-120-mbps-
>> 4k-uhd-hevc-10bit.mkv -an -filter_hw_device opencl0 -vf
>> format=yuv420p,hwupload,nlmeans_opencl,hwdownload,format=yuv420p -
>> frames:v 10 -f framemd5 -
>> 0,  0,  0,1, 12441600, 
>> 8b8805818076b23ae6f80ec2b5a349d4
>> 0,  1,  1,1, 12441600, 
>> 7a7fdaa083dc337cfb6af31b643f30a3
>> [Parsed_nlmeans_opencl_2 @ 0x557c51fbfe80] integral image overflow
>> 2098545
>> 0,  2,  2,1, 12441600, 
>> 68b390535adc5cfa0f8a7942c42a47ca
>> 0,  3,  3,1, 12441600, 
>> c06b53ad90e0357e537df41b63d5b1dc
>> 0,  4,  4,1, 12441600, 
>> 5aa2da07703859a3dee080847dd17d46
>> 0,  5,  5,1, 12441600, 
>> 733364c6be6af825057e905a6092937d
>> 0,  6,  6,1, 12441600, 
>> 47edae2dec956a582b04babb745d26b0
>> 0,  7,  7,1, 12441600, 
>> 4e45fe8268df4298d06a17ab8e46c3e9
>> 0,  8,  8,1, 12441600, 
>> 960d722a3f8787c9191299a114c04174
>> 0,  9,  9,1, 12441600, 
>> e759c07ee4834a9cf94bfcb4128e7612
>>
>> Frame 1 gave an overflow on the second run, and gets a different answer, then
>> frame 2 in the same way on the third run?  I can't characterise when this
>> happens, it seems to be pretty random with low probability.
> 
> I tried to reproduce on my SKL and KBL, with Beignet and Neo. And didn't 
> reproduc

Re: [FFmpeg-devel] [PATCH v3] lavfi: add colorkey_opencl filter

[Mark Thompson]

On 17/04/2019 03:08, Jarek Samic wrote:
> This is a direct port of the CPU filter.
> 
> Signed-off-by: Jarek Samic 
> ---
> More fixes based on the comments from the second version of the patch (moving 
> sampler declaration into the program scope, `f`-suffixing constants, 
> attaching the `*` sigil to the variable name rather than the data type).
> 
>  configure|   1 +
>  doc/filters.texi |  33 +
>  libavfilter/Makefile |   2 +
>  libavfilter/allfilters.c |   1 +
>  libavfilter/opencl/colorkey.cl   |  49 +++
>  libavfilter/opencl_source.h  |   1 +
>  libavfilter/vf_colorkey_opencl.c | 244 +++
>  7 files changed, 331 insertions(+)
>  create mode 100644 libavfilter/opencl/colorkey.cl
>  create mode 100644 libavfilter/vf_colorkey_opencl.c

LGTM, and applied.

Thanks!

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".