PR #24807 opened by Zhao Zhili (quink) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24807 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24807.patch
The VideoToolbox encoder reorders H.264 output with a two level b-pyramid, as it does for HEVC, but our code reports a depth of one, so the encoder outputs frames with dts greater than pts. The VideoToolbox API does not expose the reorder depth, and parsing the bitstream is no help: VUI field num_reorder_frames is optional and VideoToolbox writes no VUI. The value comes from measuring encoder output, which cannot cover devices that were not tested, so it is published as an upper bound: overstating the depth only shifts dts earlier, while understating is what breaks the pts >= dts ordering. # Summary of changes Briefly describe what this PR does and why. <!-- If this PR requires new FATE test samples, attach them to the PR and list their target paths below (relative to the fate-suite root). Attached filenames must match the sample's filename: ```fate-samples # e.g. vorbis/new-sample.ogg ``` --> >From 033b4ce3912546ffe43495b778afac9356cf64ee Mon Sep 17 00:00:00 2001 From: Zhao Zhili <[email protected]> Date: Wed, 30 Sep 2026 13:04:51 +0800 Subject: [PATCH] avcodec/videotoolboxenc: report a reorder depth of two for H.264 The VideoToolbox encoder reorders H.264 output with a two level b-pyramid, as it does for HEVC, but our code reports a depth of one, so the encoder outputs frames with dts greater than pts. The VideoToolbox API does not expose the reorder depth, and parsing the bitstream is no help: VUI field num_reorder_frames is optional and VideoToolbox writes no VUI. The value comes from measuring encoder output, which cannot cover devices that were not tested, so it is published as an upper bound: overstating the depth only shifts dts earlier, while understating is what breaks the pts >= dts ordering. --- libavcodec/videotoolboxenc.c | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index aa5e4313b9..e3ec0ea809 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -64,6 +64,23 @@ enum { kVTQPModulationLevel_Disable = 0 }; # define TARGET_CPU_ARM64 0 #endif +/* + * Upper bound of the reorder depth of VideoToolbox H.264 and HEVC + * output, in frames. The encoder shifts the decode timestamps + * VideoToolbox reports by the pts distance of this many frames, and + * the value is published as AVCodecContext.has_b_frames; understating + * it outputs frames with dts greater than pts. + * + * The VideoToolbox API does not report the depth, so the value was + * measured from encoder output on macOS 15 (Apple silicon). It bounds + * the real depth instead of matching it everywhere: the software + * H.264 encoder (require_sw) reorders by one frame, and untested + * devices may reorder less. Keeping one value for every encoder is + * safe because overstating only shifts dts earlier, and it is simpler + * than branching per encoder. + */ +#define REORDER_DELAY 2 + typedef OSStatus (*getParameterSetAtIndex)(CMFormatDescriptionRef videoDesc, size_t parameterSetIndex, const uint8_t **parameterSetPointerOut, @@ -1658,7 +1675,7 @@ static int vtenc_configure_encoder(AVCodecContext *avctx) if (vtctx->codec_id == AV_CODEC_ID_H264) { vtctx->get_param_set_func = CMVideoFormatDescriptionGetH264ParameterSetAtIndex; - vtctx->has_b_frames = avctx->max_b_frames > 0; + vtctx->has_b_frames = avctx->max_b_frames > 0 ? REORDER_DELAY : 0; if(vtctx->has_b_frames && (0xFF & vtctx->profile) == AV_PROFILE_H264_BASELINE){ av_log(avctx, AV_LOG_WARNING, "Cannot use B-frames with baseline profile. Output will not contain B-frames.\n"); vtctx->has_b_frames = 0; @@ -1674,8 +1691,7 @@ static int vtenc_configure_encoder(AVCodecContext *avctx) vtctx->get_param_set_func = compat_keys.CMVideoFormatDescriptionGetHEVCParameterSetAtIndex; if (!vtctx->get_param_set_func) return AVERROR(EINVAL); if (!get_vt_hevc_profile_level(avctx, &profile_level)) return AVERROR(EINVAL); - // HEVC has b-byramid - vtctx->has_b_frames = avctx->max_b_frames > 0 ? 2 : 0; + vtctx->has_b_frames = avctx->max_b_frames > 0 ? REORDER_DELAY : 0; } else if (vtctx->codec_id == AV_CODEC_ID_PRORES) { avctx->codec_tag = av_bswap32(codec_type); } @@ -1789,9 +1805,8 @@ static av_cold int vtenc_init(AVCodecContext *avctx) if (!status && has_b_frames_cfbool) { //Some devices don't output B-frames for main profile, even if requested. - // HEVC has b-pyramid if (CFBooleanGetValue(has_b_frames_cfbool)) - vtctx->has_b_frames = avctx->codec_id == AV_CODEC_ID_HEVC ? 2 : 1; + vtctx->has_b_frames = REORDER_DELAY; else vtctx->has_b_frames = 0; CFRelease(has_b_frames_cfbool); -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
