PR #24341 opened by dizhurnikita URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24341 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24341.patch
mov_estimate_video_delay() estimates the H.264 output reorder depth by measuring how far out of order the PTS sequence runs, but iterates every index entry, including entries flagged AVINDEX_DISCARD_FRAME. Those entries are decoded as references only and are never output, so they cannot contribute to the depth of the output reorder buffer. After edit list processing, mov_fix_index() emits each edit segment's entries through to the second keyframe past its end and flags everything outside the segment as discard. The seam between the discarded pre-roll of one segment and the kept content of the previous one reads as a large backward PTS jump, so the estimate saturates at MAX_REORDER_DELAY on any file with two or more non-empty edit segments and a GOP of 8 or more. QuickTime screen recordings are one common source of such files. codecpar->video_delay reaches the decoder as has_b_frames, and h264_select_output_frame() withholds a picture until pics exceeds it, so a value of 16 keeps 17 pictures in delayed_pic[]. This is inconsistent with the bitstream's own signalling: with the nvdec hwaccel the decode surface pool is sized from the SPS, as ref_frame_count plus num_reorder_frames, which for such files is far smaller. Every retained picture holds a surface, the pool is exhausted, and decoding fails with "No decoder surfaces left". Skip discard flagged entries when accumulating the estimate. The tts cursor advance stays outside the new condition and still runs for every entry: the tts table is indexed by sample rather than by output frame, so it must advance for discarded samples as well or it desynchronises from the index. On an affected file has_b_frames drops from 16 to 2, matching the same content with the edit list removed. Decoded output is unchanged. --- The second patch adds a FATE test covering this. It needs a sample in the fate-suite as `mov/mov_editlist_reorder_delay.mp4`; a request has gone to [email protected], and the file is already on streams.videolan.org at `incoming/mov_editlist_reorder_delay.mp4`. The sample is synthetic — 320x240 testsrc2, 4 s at 30 fps, H.264 with GOP 12, three B-frames and b-pyramid, and an edit list holding two non-empty segments. No FFmpeg muxer writes a multi-segment elst, so that atom was written directly, which is why the media has to be in fate-suite rather than generated by the test. To see the bug without building the test, on any file with two non-empty edit segments: ffprobe -show_entries stream=has_b_frames -select_streams v <sample> 16 before the patch, 2 after. From f79d0c14dcf6595dda2dca853c31b41a40c66342 Mon Sep 17 00:00:00 2001 From: Nikita Dizhur <[email protected]> Date: Mon, 31 Aug 2026 19:28:20 -0500 Subject: [PATCH 1/2] avformat/mov: exclude discarded entries from video delay estimation mov_estimate_video_delay() estimates the H.264 output reorder depth by measuring how far out of order the PTS sequence runs, but iterates every index entry, including entries flagged AVINDEX_DISCARD_FRAME. Those entries are decoded as references only and are never output, so they cannot contribute to the depth of the output reorder buffer. After edit list processing, mov_fix_index() emits each edit segment's entries through to the second keyframe past its end and flags everything outside the segment as discard. The seam between the discarded pre-roll of one segment and the kept content of the previous one reads as a large backward PTS jump, so the estimate saturates at MAX_REORDER_DELAY on any file with two or more non-empty edit segments and a GOP of 8 or more. QuickTime screen recordings are one common source of such files. codecpar->video_delay reaches the decoder as has_b_frames, and h264_select_output_frame() withholds a picture until pics exceeds it, so a value of 16 keeps 17 pictures in delayed_pic[]. This is inconsistent with the bitstream's own signalling: with the nvdec hwaccel the decode surface pool is sized from the SPS, as ref_frame_count plus num_reorder_frames, which for such files is far smaller. Every retained picture holds a surface, the pool is exhausted, and decoding fails with "No decoder surfaces left". Skip discard flagged entries when accumulating the estimate. The tts cursor advance stays outside the new condition and still runs for every entry: the tts table is indexed by sample rather than by output frame, so it must advance for discarded samples as well or it desynchronises from the index. On an affected file has_b_frames drops from 16 to 2, matching the same content with the edit list removed. Decoded output is unchanged. Signed-off-by: Nikita Dizhur <[email protected]> --- libavformat/mov.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/libavformat/mov.c b/libavformat/mov.c index f53ce693f8..8e5429ec88 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -4415,6 +4415,9 @@ static void mov_estimate_video_delay(MOVContext *c, AVStream* st) st->codecpar->codec_id == AV_CODEC_ID_H264) { st->codecpar->video_delay = 0; for (int ind = 0; ind < sti->nb_index_entries && ctts_ind < msc->tts_count; ++ind) { + // Discarded frames are decoded as references only and never output, + // so they do not contribute to the output reorder depth. + if (!(sti->index_entries[ind].flags & AVINDEX_DISCARD_FRAME)) { // Point j to the last elem of the buffer and insert the current pts there. j = buf_start; buf_start = (buf_start + 1); @@ -4442,6 +4445,7 @@ static void mov_estimate_video_delay(MOVContext *c, AVStream* st) j = r; } st->codecpar->video_delay = FFMAX(st->codecpar->video_delay, num_swaps); + } ctts_sample++; if (ctts_sample == msc->tts_data[ctts_ind].count) { -- 2.52.0 From 3bea0edeb59e43206a3330daa727871ee4364ce9 Mon Sep 17 00:00:00 2001 From: Nikita Dizhur <[email protected]> Date: Mon, 31 Aug 2026 19:28:20 -0500 Subject: [PATCH 2/2] fate/mov: add edit list reorder depth test Covers mov_estimate_video_delay() on a file with two non-empty edit segments, where discard flagged index entries can saturate the reorder depth estimate at MAX_REORDER_DELAY. The sample is synthetic: 320x240 testsrc2, 4 seconds at 30 fps, H.264 with GOP 12, three B-frames and b-pyramid so that a ctts table is present. Its elst holds two non-empty segments, the second starting mid-GOP. No muxer in FFmpeg writes a multi-segment edit list, so the elst was built by rewriting the atom directly, which is why the media has to be added to fate-suite rather than generated by the test. Without the discard check in mov_estimate_video_delay() this file reports has_b_frames=16; with it, 2. Signed-off-by: Nikita Dizhur <[email protected]> --- tests/fate/mov.mak | 2 ++ tests/ref/fate/mov-guess-delay-editlist | 3 +++ 2 files changed, 5 insertions(+) create mode 100644 tests/ref/fate/mov-guess-delay-editlist diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak index 1a81962626..f481b9047a 100644 --- a/tests/fate/mov.mak +++ b/tests/fate/mov.mak @@ -36,6 +36,7 @@ FATE_MOV_FFPROBE-$(call FRAMEMD5, MOV, H264, H264_PARSER) += fate-mov-neg-firstp fate-mov-guess-delay-1 \ fate-mov-guess-delay-2 \ fate-mov-guess-delay-3 \ + fate-mov-guess-delay-editlist \ fate-mov-mp4-with-mov-in24-ver \ fate-mov-mime-codecstring \ fate-mov-t35-cdsc-track \ @@ -160,6 +161,7 @@ fate-mov-gpmf-remux: REF = e919915c5cd22c849e2aba281ddaf0c8 fate-mov-guess-delay-1: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -show_entries stream=has_b_frames -select_streams v $(TARGET_SAMPLES)/h264/h264_3bf_nopyramid_nobsrestriction.mp4 fate-mov-guess-delay-2: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -show_entries stream=has_b_frames -select_streams v $(TARGET_SAMPLES)/h264/h264_3bf_pyramid_nobsrestriction.mp4 fate-mov-guess-delay-3: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -show_entries stream=has_b_frames -select_streams v $(TARGET_SAMPLES)/h264/h264_4bf_pyramid_nobsrestriction.mp4 +fate-mov-guess-delay-editlist: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -show_entries stream=has_b_frames -select_streams v $(TARGET_SAMPLES)/mov/mov_editlist_reorder_delay.mp4 fate-mov-faststart-4gb-overflow: CMD = run tools/qt-faststart$(EXESUF) $(TARGET_SAMPLES)/mov/faststart-4gb-overflow.mov $(TARGET_PATH)/faststart-4gb-overflow-output.mov > /dev/null ; do_md5sum faststart-4gb-overflow-output.mov | cut -d " " -f1 ; rm faststart-4gb-overflow-output.mov fate-mov-faststart-4gb-overflow: CMP = oneline diff --git a/tests/ref/fate/mov-guess-delay-editlist b/tests/ref/fate/mov-guess-delay-editlist new file mode 100644 index 0000000000..248de1c3ea --- /dev/null +++ b/tests/ref/fate/mov-guess-delay-editlist @@ -0,0 +1,3 @@ +[STREAM] +has_b_frames=2 +[/STREAM] -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
