PR #24980 opened by Marcos Ashton (MarcosAsh) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24980 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24980.patch
Fixes AV1 hardware decoding when a new sequence header raises the maximum frame size. get_pixel_format() keeps the negotiated hwaccel if the software pixel format didn't change. It never checked the size. In av1-1-b8-03-sizeup.ivf the second sequence header goes from 352x288 to 1280x720. The frames context stayed at 352x288, so NVDEC failed with CUDA_ERROR_INVALID_VALUE and lost 7 frames, and Vulkan returned 1280x720 frames on 352x288 images. hwdownload then read past the end of them. The decoder now remembers the size it negotiated the hwaccel for and only keeps it if the new maximum fits. A shown keyframe refreshes every reference, so nothing is lost by renegotiating there. With this, sizeup is bit-exact on both CUDA and Vulkan, and every other AV1 conformance stream I tested gives the same result as before. One thing I didn't cover is a damaged stream whose frame size is bigger than its sequence maximum. That can still produce a frame bigger than the frames context, and I think it's better as a separate check in the frame header path. I tested on an RTX 5070 Ti with NVIDIA 595.91.07. >From a800738d7b3314c39216117ceef7c0e93ec79b15 Mon Sep 17 00:00:00 2001 From: Marcos Ashton Iglesias <[email protected]> Date: Fri, 9 Oct 2026 18:52:06 +0100 Subject: [PATCH] avcodec/av1dec: renegotiate the hwaccel when the max frame size grows A new sequence header kept the negotiated hwaccel whenever the software pixel format was unchanged, even if its new maximum frame size no longer fit the surfaces. The second sequence header in av1-1-b8-03-sizeup.ivf raises the maximum from 352x288 to 1280x720, and with -hwaccel cuda -hwaccel_output_format cuda NVDEC failed with CUDA_ERROR_INVALID_VALUE, while with Vulkan the 1280x720 frames came out on 352x288 images. Keep the hwaccel only if the new maximum fits the size it was negotiated for. Follow-up to 51538457af0. --- libavcodec/av1dec.c | 9 +++++++-- libavcodec/av1dec.h | 2 ++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index d7ccebfbd5..4e4cca81ce 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -694,8 +694,11 @@ static int get_pixel_format(AVCodecContext *avctx) *fmtp = AV_PIX_FMT_NONE; /* the negotiated hwaccel can only be kept if the underlying pixel - * format did not change, otherwise its surfaces have the wrong format */ - if (pix_fmt == avctx->sw_pix_fmt) + * format did not change and the new maximum frame size still fits its + * surfaces */ + if (pix_fmt == avctx->sw_pix_fmt && + avctx->coded_width <= s->hwaccel_width && + avctx->coded_height <= s->hwaccel_height) for (int i = 0; pix_fmts[i] != pix_fmt; i++) if (pix_fmts[i] == avctx->pix_fmt) { s->pix_fmt = pix_fmt; @@ -717,6 +720,8 @@ static int get_pixel_format(AVCodecContext *avctx) } s->pix_fmt = pix_fmt; + s->hwaccel_width = avctx->coded_width; + s->hwaccel_height = avctx->coded_height; avctx->pix_fmt = ret; av_log(avctx, AV_LOG_DEBUG, "AV1 decode get format: %s.\n", diff --git a/libavcodec/av1dec.h b/libavcodec/av1dec.h index 10c807f73f..5743b7bb41 100644 --- a/libavcodec/av1dec.h +++ b/libavcodec/av1dec.h @@ -87,6 +87,8 @@ typedef struct AV1DecContext { AVCodecContext *avctx; enum AVPixelFormat pix_fmt; + /* coded size the hwaccel was negotiated for */ + int hwaccel_width, hwaccel_height; CodedBitstreamContext *cbc; CodedBitstreamFragment current_obu; AVPacket *pkt; -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
