PR #24628 opened by Daniel Campos Ramos (danielcamposramos) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24628 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24628.patch
# Summary of changes Honor the lifetime fields of frame-packing-arrangement SEI messages when exporting `AV_FRAME_DATA_STEREO3D` from the H.264, HEVC and VVC decoders. H.264 previously cleared `frame_packing.present` at every access unit even when `frame_packing_arrangement_repetition_period` was non-zero. HEVC/VVC discarded `frame_packing_arrangement_persistence_flag`, which made a non-persistent HEVC message sticky and left VVC without the scalar state copy needed between frame contexts. This change: - reads the HEVC/VVC persistence flag; - exports the current frame first, then retains or clears the arrangement from the codec-specific repetition/persistence field; - copies scalar frame-packing state in H.264 thread-context and VVC frame-context updates (HEVC already did this explicitly); - clears retained H.264 state on a decoder flush; and - adds H.264 and HEVC FATE samples covering persistence, cancellation and a current-frame-only arrangement. The H.264 test forces four frame threads. ## Standards basis - [ITU-T H.264 (01/2012)](https://www.itu.int/rec/T-REC-H.264-201201-S/en), clauses D.1.25 (syntax) and D.2.25 (semantics), `frame_packing_arrangement_repetition_period` and `frame_packing_arrangement_cancel_flag`. - [ITU-T H.265 (04/2013)](https://www.itu.int/rec/T-REC-H.265-201304-S/en), clauses D.2.16 (syntax) and D.3.16 (semantics), `frame_packing_arrangement_persistence_flag` and `frame_packing_arrangement_cancel_flag`. - [ITU-T H.274 (08/2020)](https://www.itu.int/rec/T-REC-H.274-202008-S/en), clauses 8.6.1 (syntax) and 8.6.2 (semantics), used with VVC. The x264 files that led to this report physically repeat the message at IDRs. That is measured encoder/file behaviour, not the normative lifetime boundary; the decoder lifetime comes from the fields above. ## Verification - `fate-h264-frame-packing-persistence`: pass - `fate-hevc-frame-packing-persistence`: pass - H.264 persistent fixture at 1/2/4/8 threads: 64/64 frames each - HEVC persistent fixture at 1/2/4/8 threads: 64/64 frames each - H.264 period 0: 1/64; cancel at frame 16: frames 0–15 only - HEVC persistence 0: 1/64; cancel at frame 16: frames 0–15 only - VVC state propagation was code-audited and compiled in the VVC-enabled build; no local VVC fixture was available, so VVC runtime coverage is not claimed. - `git diff --check`: pass - `tools/patcheck`: only Makefile/single-line-if false positives and the expected minor-change changelog advisory The root cause and film-grain precedent were identified by mpv maintainer kasper93 during review of [mpv PR #18490](https://github.com/mpv-player/mpv/pull/18490#discussion_r4066349407). ## Collaboration provenance Daniel Campos Ramos directed the work, verified the claims and measurements, and is responsible for the submission. Kimi K3, running inside Claude CLI, performed the initial FFmpeg code-path investigation, first implementation and fixture exploration. OpenAI Codex independently reviewed the decoder lifetime, corrected the fixture framing/syntax, found the H.264 frame-thread propagation gap and VVC scalar-state copy, and built the FATE regression coverage. ```fate-samples h264/frame-packing-persistence.h264 hevc/frame-packing-persistence.hevc ``` Sample integrity (sha256): h264 `69c9fc28d9a653e10359f57136bc45609236485b71e1613646b9cca77c4baeb2`, hevc `7dee15628845e2956716ca7927e10545e07c3e9808d3794a27cf659ccde08536`. From baea3d9ee840019c6cf46919d42b70f7ff1309ee Mon Sep 17 00:00:00 2001 From: Daniel Campos Ramos <[email protected]> Date: Mon, 21 Sep 2026 22:23:28 -0300 Subject: [PATCH] avcodec/h2645_sei: honor frame-packing persistence H.264 frame-packing state is currently cleared for every access unit even when arrangement_repetition_period is non-zero. HEVC and VVC parse the same message but discard arrangement_persistence_flag, leaving HEVC state sticky and preventing VVC state from propagating between frame contexts. Read and apply the codec-specific persistence field after exporting side data for the current frame. Copy the scalar state through the H.264 and VVC context-update paths, and clear retained H.264 state on a decoder flush. Add H.264 and HEVC FATE samples covering persistent, cancelled and current-frame-only arrangements. The H.264 test uses four frame threads to exercise context propagation. Suggested-by: Kacper Michajlow (kasper93) Link: https://github.com/mpv-player/mpv/pull/18490#discussion_r4066349407 Signed-off-by: Daniel Campos Ramos <[email protected]> --- libavcodec/h2645_sei.c | 13 ++++++++++++- libavcodec/h2645_sei.h | 1 + libavcodec/h264_sei.c | 3 ++- libavcodec/h264_slice.c | 1 + libavcodec/h264dec.c | 1 + libavcodec/vvc/sei.c | 1 + tests/fate/h264.mak | 3 +++ tests/fate/hevc.mak | 3 +++ tests/ref/fate/h264-frame-packing-persistence | 8 ++++++++ tests/ref/fate/hevc-frame-packing-persistence | 8 ++++++++ 10 files changed, 40 insertions(+), 2 deletions(-) create mode 100644 tests/ref/fate/h264-frame-packing-persistence create mode 100644 tests/ref/fate/hevc-frame-packing-persistence diff --git a/libavcodec/h2645_sei.c b/libavcodec/h2645_sei.c index 735199733c..cc65275b3c 100644 --- a/libavcodec/h2645_sei.c +++ b/libavcodec/h2645_sei.c @@ -145,7 +145,7 @@ static int decode_frame_packing_arrangement(H2645SEIFramePacking *h, if (IS_H264(codec_id)) h->arrangement_repetition_period = get_ue_golomb_long(gb); else - skip_bits1(gb); // frame_packing_arrangement_persistence_flag + h->arrangement_persistence_flag = get_bits1(gb); } // H.264: frame_packing_arrangement_extension_flag, // HEVC: upsampled_aspect_ratio_flag @@ -576,6 +576,17 @@ int ff_h2645_sei_to_frame(AVFrame *frame, H2645SEI *sei, } } + /* The arrangement outlives the message that carried it: H.264 gives a + * repetition period, HEVC/VVC a persistence flag, and a later message + * (or the cancel flag) supersedes either. Mirror the film grain + * persistence handling below. */ + if (fp->present) { + if (IS_H264(codec_id)) + fp->present = !!fp->arrangement_repetition_period; + else + fp->present = fp->arrangement_persistence_flag; + } + if (sei->display_orientation.present && (sei->display_orientation.anticlockwise_rotation || sei->display_orientation.hflip || diff --git a/libavcodec/h2645_sei.h b/libavcodec/h2645_sei.h index 54288e2e26..9929c0ed1a 100644 --- a/libavcodec/h2645_sei.h +++ b/libavcodec/h2645_sei.h @@ -66,6 +66,7 @@ typedef struct H2645SEIFramePacking { int arrangement_cancel_flag; ///< is previous arrangement canceled, -1 if never received (currently H.264 only) SEIFpaType arrangement_type; int arrangement_repetition_period; + int arrangement_persistence_flag; ///< HEVC/VVC only int content_interpretation_type; int quincunx_sampling_flag; int current_frame_is_frame0_flag; diff --git a/libavcodec/h264_sei.c b/libavcodec/h264_sei.c index 6819fb75c4..0f215cdfb6 100644 --- a/libavcodec/h264_sei.c +++ b/libavcodec/h264_sei.c @@ -54,7 +54,8 @@ void ff_h264_sei_uninit(H264SEIContext *h) h->picture_timing.present = 0; h->buffering_period.present = 0; - h->common.frame_packing.present = 0; + /* frame_packing is retained across access units so that + * ff_h2645_sei_to_frame() can apply its repetition period. */ h->common.display_orientation.present = 0; ff_h2645_sei_reset(&h->common); diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index 2bcb64378d..864eb2a82a 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -450,6 +450,7 @@ int ff_h264_update_thread_context(AVCodecContext *dst, if (ret < 0) return ret; + h->sei.common.frame_packing = h1->sei.common.frame_packing; h->sei.common.unregistered.x264_build = h1->sei.common.unregistered.x264_build; if (!h->cur_pic_ptr) diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index c38f95f48c..7962e9acfe 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -483,6 +483,7 @@ static av_cold void h264_decode_flush(AVCodecContext *avctx) ff_h264_flush_change(h); ff_h264_sei_uninit(&h->sei); + h->sei.common.frame_packing.present = 0; for (i = 0; i < H264_MAX_PICTURE_COUNT; i++) ff_h264_unref_picture(&h->DPB[i]); diff --git a/libavcodec/vvc/sei.c b/libavcodec/vvc/sei.c index 241943f3c1..806be477c0 100644 --- a/libavcodec/vvc/sei.c +++ b/libavcodec/vvc/sei.c @@ -273,6 +273,7 @@ int ff_vvc_sei_replace(VVCSEI *dst, const VVCSEI *src) { dst->picture_hash.present = 0; // drop hash dst->frame_field_info.present = 0; // drop field info + dst->common.frame_packing = src->common.frame_packing; return ff_h2645_sei_ctx_replace(&dst->common, &src->common); } diff --git a/tests/fate/h264.mak b/tests/fate/h264.mak index 2e3f826e57..675aee506a 100644 --- a/tests/fate/h264.mak +++ b/tests/fate/h264.mak @@ -253,6 +253,9 @@ FATE_H264-$(call FRAMECRC, MPEGTS, H264, H264_PARSER MP3_DECODER SCALE_FILTER AR FATE_H264_FFPROBE-$(call DEMDEC, MATROSKA, H264) += fate-h264-dts_5frames FATE_H264_FFPROBE-$(call DEMDEC, MATROSKA, H264) += fate-h264-skip-pred-pts FATE_H264_FFPROBE-$(call PARSERDEMDEC, H264, H264, H264) += fate-h264-afd + +fate-h264-frame-packing-persistence: CMD = threads=4 probeframes -show_entries frame=side_data_list:frame_tags= -of compact $(TARGET_SAMPLES)/h264/frame-packing-persistence.h264 +FATE_H264_FFPROBE-$(call PARSERDEMDEC, H264, H264, H264) += fate-h264-frame-packing-persistence FATE_H264_FFPROBE-$(call PARSERDEMDEC, H264, H264, H264) += fate-h264-skip-pred \ fate-h264-skip-pred-fields diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak index 10b6cd59ae..d4709045a8 100644 --- a/tests/fate/hevc.mak +++ b/tests/fate/hevc.mak @@ -282,6 +282,9 @@ FATE_HEVC_FFPROBE-$(call PARSERDEMDEC, HEVC, HEVC, HEVC) += fate-hevc-monochrome fate-hevc-afd-tc-sei: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -bitexact -flags output_corrupt -show_entries frame_side_data_list -select_streams v $(TARGET_SAMPLES)/mpegts/loewe.ts FATE_HEVC_FFPROBE-$(call PARSERDEMDEC, HEVC, HEVC, HEVC, EXTRACT_EXTRADATA_BSF) += fate-hevc-afd-tc-sei +fate-hevc-frame-packing-persistence: CMD = threads=4 probeframes -show_entries frame=side_data_list:frame_tags= -of compact $(TARGET_SAMPLES)/hevc/frame-packing-persistence.hevc +FATE_HEVC_FFPROBE-$(call PARSERDEMDEC, HEVC, HEVC, HEVC) += fate-hevc-frame-packing-persistence + fate-hevc-hdr10-plus-metadata: CMD = probeframes -show_entries frame=side_data_list $(TARGET_SAMPLES)/hevc/hdr10_plus_h265_sample.hevc FATE_HEVC_FFPROBE-$(call DEMDEC, HEVC, HEVC) += fate-hevc-hdr10-plus-metadata diff --git a/tests/ref/fate/h264-frame-packing-persistence b/tests/ref/fate/h264-frame-packing-persistence new file mode 100644 index 0000000000..4a41412eb9 --- /dev/null +++ b/tests/ref/fate/h264-frame-packing-persistence @@ -0,0 +1,8 @@ +frame|side_datum/stereo_3d:side_data_type=Stereo 3D +frame|side_datum/stereo_3d:side_data_type=Stereo 3D +frame| +frame| +frame|side_datum/stereo_3d:side_data_type=Stereo 3D +frame| +frame| +frame| diff --git a/tests/ref/fate/hevc-frame-packing-persistence b/tests/ref/fate/hevc-frame-packing-persistence new file mode 100644 index 0000000000..4a41412eb9 --- /dev/null +++ b/tests/ref/fate/hevc-frame-packing-persistence @@ -0,0 +1,8 @@ +frame|side_datum/stereo_3d:side_data_type=Stereo 3D +frame|side_datum/stereo_3d:side_data_type=Stereo 3D +frame| +frame| +frame|side_datum/stereo_3d:side_data_type=Stereo 3D +frame| +frame| +frame| -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
