PR #25124 opened by hyena URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25124 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25124.patch
Base: 447b72d90f2c0738c973e33532cfd325e2f3c9a5 ## Problem Decoding a damaged H.264 stream with frame threads can produce output that changes from run to run and differs from single-threaded decoding. The issue was found with `0070461469_2.h264` from the test set of the BSCV bitstream-corruption benchmark (Liu et al., NeurIPS 2023 Datasets and Benchmarks; https://github.com/LIUTIGHE/BSCV-Dataset). It is a damaged progressive 1280x720 stream of 32 MB with 91 frames; the local copy has SHA-256 b31683a21442f40105b16b6e069da98e72bb7ae1f8f627a6dcfc2975d9a33b2a. On unpatched 5cb9fcd211 (see Validation), 24 eight-thread runs produced 24 different outputs, and every run differed from single-threaded output in 21 frames. Single-threaded output was the same in every run. To check: ```sh s=0070461469_2.h264 ffmpeg -nostdin -v quiet -threads 1 -i $s -f framemd5 - | md5sum for i in $(seq 24); do ffmpeg -nostdin -v quiet -threads 8 -thread_type frame -i $s -f framemd5 - | md5sum done | sort | uniq -c ``` ## Causes Four defects in error concealment's H.264 handling contribute: 1. **Reference waits use the wrong unit, or are missing.** H.264 reports ThreadFrame progress in pixel rows. `guess_mv()` waits for a macroblock-row index instead, and the intra/inter SAD heuristic in `is_intra_more_likely()` reads two reference macroblock rows without waiting at all. Concealment can therefore read reference rows that another frame thread has not finished decoding. 2. **Predictors are read from unavailable motion.** For macroblocks without usable list-0 motion, the motion and reference buffers of a pooled picture can still hold values from a previous user of the buffer. `guess_mv()` copies them into temporal and spatial predictors without checking. 3. **Concealed motion is not stored in the picture.** `h264_er_decode_mb()` fills the slice motion cache used for reconstruction, but not the picture's motion field. Later prediction and filtering then read motion that does not describe the reconstructed macroblock. When an invalid reference prevents reconstruction, the reference entries are left as if valid. 4. **The filter skip test uses invalid motion.** The concealment filters skip a boundary when the motion on both sides is similar. They do so even when an H.264 block has no valid list-0 motion, so the decision depends on stale buffer contents. ## Changes 1. avcodec/error_resilience: wait for the last pixel row each operation needs. For a field-coded reference, wait on both fields. 2. avcodec/error_resilience: check macroblock type, list-0 use and reference index before using a predictor. Mark unavailable predictors with ref -1 so that the existing selection skips them. 3. avcodec/h264dec: write the concealment vector to all sixteen 4x4 blocks of the picture's motion field. Invalidate all four list-0 reference entries when reconstruction is skipped. This updates the strict CIP reference. 4. avcodec/error_resilience: require valid forward motion on both sides before skipping a filter boundary. Use the actual 8x8 quadrant for luma and the macroblock for chroma. This reuses the helper from patch 2. 5. fate/h264: add three synthetic damaged streams with shared references at explicit 1, 2 and 8 frame threads, plus 2- and 8-thread targets for the existing strict CIP test. All code changes are limited to H.264 paths, selected by codec ID or by the H.264 ThreadFrame. Other codecs keep their current behaviour. ## Effect **BSCV stream above, on 5cb9fcd211, 8 frame threads, 72 runs** (24 runs each with 1, 6 and 12 decoders running concurrently): - 62 runs differed from single-threaded output in 8 frames; - 4 runs differed in 7 frames; - 6 runs were identical to single-threaded output; - each group of 24 runs gave 2 or 3 distinct outputs. In these measurements the series substantially reduces variation, but it does **not** make this stream deterministic, and it is not a complete fix. Without patch 1, the same stream again gave 23 distinct outputs in 24 runs. These frequencies describe the checked runs, not a guaranteed distribution. We believe the remainder comes from concealment rewriting rows whose progress has already been reported to other frame threads; this has not been traced step by step. A separate change keeps concealment out of such rows, combined with clearing coefficients left by failed macroblock decoding. On the earlier afead44165 base, all 72 runs of that combination matched single-threaded output. That change alters concealment output for damaged input, so it will be proposed separately as an RFC. It is not part of this PR. **Strict data-partitioning CIP.** The existing test is pinned to one thread because concealment was not stable with threads. On unpatched 5cb9fcd211, at 2, 3, 4, 8 and 16 frame threads, none of 24 runs matched single-threaded output, and at 2, 3 and 8 threads the output also varied between runs. With this series, 24 of 24 runs matched at every one of those thread counts. The motion writeback in patch 3 is needed for this result: without it, 2 and 8 threads again match in 0 of 24 runs. The new `fate-h264-er-cip-strict-2/-8` targets use the same reference as the existing test. **Real-footage excerpts.** The first GOP of two other BSCV streams (`26d8d48248_2`, 15 frames; `b19d6e149a_2`, 16 frames), cut without re-encoding, also reproduces individual fixes. Removing patch 2 changes two frames of the first. On the second, removing patch 3 changes nine or ten frames, and removing patch 4 changes ten. With the series, both excerpts matched single-threaded output in 24/24 runs at 1, 2 and 8 frame threads. They are not FATE samples, since we have no permission to redistribute them. `real-video-repro/` in the sample download has the byte ranges, hashes and an extraction script. ## Single-thread image quality Patches 2 and 3 change single-threaded concealment output, so we measured luma PSNR against each clean decode (MSE pooled over all frames). **Synthetic test streams.** er-narrow 26.28 to 28.87 dB, er-mv-predictor40 44.87 to 44.56 dB, er-filter-bframes 40.934 to 40.928 dB. Removing patch 2 restores 44.87 dB on er-mv-predictor40. **BSCV test set.** All 661 damaged/clean pairs in the local copy of the BSCV test set, single-threaded (84,286 frames, frame counts and picture-header sequences checked against the clean stream): | Change in luma PSNR | Streams | |---|---:| | better by more than 0.1 dB | 104 | | within +/-0.1 dB | 466 | | worse by more than 0.1 dB | 91 | The mean per-stream change was +0.013 dB (median -0.001 dB), ranging from -5.37 dB (cba3e31e88_2, 33.10 to 27.73 dB) to +1.41 dB. **The large per-stream differences mostly reflect leftover buffer contents.** The unpatched decoder's result depends on what reused motion and reference buffers happen to contain. We checked this on the five largest losses and the five largest gains with an experiment-only build. It overwrites the pooled motion and reference tables in `alloc_picture()`. - Reset to "unavailable" (motion 0, reference -1): the unpatched decoder came within about 0.004 dB of the series on all ten streams. - Filled with a different constant: the unpatched result moved by up to 5.9 dB (34.66 to 28.75 dB on one stream). - The series' output was bit-identical in all three cases. On cba3e31e88_2, patch 2 alone or patch 3 alone reproduces the whole loss. Each one stops concealment from using the leftover values. These controls support dependence on pre-existing motion/ref contents as the source of the large difference in these tested cases. Removing that dependence can lower PSNR on an individual stream; it does not establish that the series improves concealment quality in general. The other 651 streams were not run with the experiment build, and PSNR is not a measure of subjective quality. <details> <summary>Experiment-only change used for this check (not part of this PR)</summary> In `alloc_picture()` in libavcodec/h264_slice.c, after `pic->motion_val[i] = pic->motion_val_base[i] + 4;`: ```c if (getenv("ER_POISON")) { const int v = atoi(getenv("ER_POISON")); const size_t n = 2 * ((size_t)(h->mb_width * 4 + 1) * h->mb_height * 4 + 4); int16_t *mvb = (int16_t *)pic->motion_val_base[i]; for (size_t k = 0; k < n; k++) mvb[k] = (k & 1) ? -v : v; memset(pic->ref_index[i], v ? 0 : 0xff, 4 * h->mb_stride * h->mb_height); } ``` `ER_POISON=0` gives the "unavailable" case; `ER_POISON=40` gives the constant case. </details> ## Tests Each new stream was checked by removing one fix at a time, on 5cb9fcd211, 12 runs each at 1 and 8 threads: | Test stream | Removed fix | Frames changed | |---|---|---| | er-narrow (32x256, 36 frames, 9,955 bytes) | patch 3 | 22 | | er-mv-predictor40 (640x352, 50 frames, 274,395 bytes) | patch 2 | 24 at 1 thread, 9 at 8 threads | | er-filter-bframes (1280x720, 24 frames, 384,096 bytes) | patch 4 | 3 | | er-filter-bframes | patch 3 | 2 at 1 thread, 1 at 8 threads | | strict CIP, 2 and 8 threads | patch 3 | 9 | No stream detects patch 1 reliably. Removing it changed strict CIP at 8 threads in 1 of 24 runs, and the other streams not at all. er-filter-bframes is the first GOP of a 50-frame stream, cut before the next SPS/IDR. The streams are libx264 encodes of FFmpeg's testsrc2 pattern, damaged by deterministic byte flips (predictor, filter) or NAL truncation (narrow). The download below contains the clean encodes, the damage script, the seeds, the x264 SEI options and a script that regenerates the three files byte for byte. ## Validation On 447b72d90f with this series (x86-64 Linux, GCC 11.4): - `make fate`, THREADS=1: all 5898 configured software tests passed. - `make fate-h264`, THREADS=1 and THREADS=8: 242/242 each. - Stress: the four affected tests (narrow, predictor, filter, strict CIP) at 1, 2 and 8 frame threads, 24 runs each, with 12 decoder processes running concurrently. Every run matched the reference. - AddressSanitizer build (`--toolchain=gcc-asan`): `make fate-h264` at THREADS=1 and 8 passed 242/242. Decoding the four BSCV streams named here at 1 and 8 threads produced no sanitizer report. - Valgrind memcheck: the three new streams and the strict CIP stream at 1 and 8 threads reported 0 errors. - Speed: decoding 0070461469_2 and its clean source, 7 runs each, gave medians within 2.5% of unpatched at 1 and 8 threads. The clean stream, where the changed code is not reached, showed the same spread, so we consider this noise. Between 5cb9fcd211, where the measurements in the earlier sections were made, and 447b72d90f, upstream changed only scale_cuda files. Rebasing changed the series only by adding function comments. On the older base afead44165, builds configured with `--disable-x86asm` produced the same frames as the default build for the checked streams. ## Limitations - These are repeated-run checks on specific inputs. They are not a determinism guarantee for damaged streams in general. - Not covered: slice threading, PAFF (the field-reference wait has no dedicated reproducer), MBAFF boundaries, and other CPUs. - No ThreadSanitizer results are included. - Concealed-motion writeback also touches already-published rows. Its role in the remaining race has not been traced; the ablation results do not prove it is uninvolved. - Other issues seen while testing are not addressed here: - failed macroblocks stay marked as available in slice_table; - damaged slices can decode past the start of the next slice; - the vertical-motion term in the ER filter skip test adds the two vectors rather than subtracting them. ## Samples The new tests need three files added to fate-suite: - h264/er-narrow.h264 SHA-256 5a512f0fab9b7df6b1694c9110fca3c4baeb7d7168d810c1db79c7def4e706aa - h264/er-mv-predictor40.h264 SHA-256 84db98030f680e711f33a5d1447939d25dd0d1ab976a0b677253e030ef31ad6a - h264/er-filter-bframes.h264 SHA-256 c6f4829de28c0f98abda85c385a219009b965f733cf1c737f2b61729cd8b6a57 Download (samples and generation records): https://github.com/KingOfHyena/ffmpeg-test-samples/releases/download/h264-er-v3/series-a-fate-samples-with-repro-notes.zip Patch 5 cannot pass FATE until these files are in fate-suite. The download contains no BSCV media. The BSCV hashes given here are of our local copy and have not been compared with a fresh public download. >From 0cffe10fa9b00c46bc1234583026d11b1520104a Mon Sep 17 00:00:00 2001 From: hyena <[email protected]> Date: Fri, 16 Oct 2026 06:04:02 -0700 Subject: [PATCH 1/5] avcodec/error_resilience: await H.264 reference pixel rows H.264 reports ThreadFrame progress in pixel rows, but guess_mv() waits using a macroblock-row index. The concealment intra/inter heuristic also reads two reference macroblock rows without waiting for H.264 progress. Both can read a reference picture before the decoding thread has finished the required rows. Wait for the last pixel row needed by each operation. For a field-coded reference, wait for the corresponding progress in both fields. A null progress buffer needs no wait; avoid waiting on the current picture's own progress buffer. Leave the other codecs' progress path unchanged. These waits reduce run-to-run variation on the reported damaged progressive stream, but do not make its output deterministic. With the series, repeated eight-thread runs still produce multiple outputs; removing this wait fix produced 23 distinct outputs in 24 checked runs. The small regression samples only detect the missing wait intermittently. There is no dedicated PAFF reproducer. Signed-off-by: hyena <[email protected]> --- libavcodec/error_resilience.c | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/libavcodec/error_resilience.c b/libavcodec/error_resilience.c index 8cf5bc6a3c..14d9a31965 100644 --- a/libavcodec/error_resilience.c +++ b/libavcodec/error_resilience.c @@ -407,6 +407,28 @@ static av_always_inline void add_blocklist(int (*blocklist)[2], int *blocklist_l blocklist[(*blocklist_length)++][1] = mb_y; } +/** + * Wait until pixel row row of the H.264 reference picture has been decoded. + * H.264 reports frame-thread progress in pixel rows of each field. + */ +static void await_h264_reference(const ERContext *s, int row) +{ + const ThreadFrame *tf = s->last_pic.tf; + + /* A null progress buffer needs no synchronization. The identity check + * defensively avoids waiting on the picture being concealed itself. + */ + if (!tf->progress || tf->progress == s->cur_pic.tf->progress) + return; + + if (s->last_pic.field_picture) { + ff_thread_await_progress(tf, row / 2, 0); + ff_thread_await_progress(tf, (row - 1) / 2, 1); + } else { + ff_thread_await_progress(tf, row, 0); + } +} + static void guess_mv(ERContext *s) { int (*blocklist)[2], (*next_blocklist)[2]; @@ -432,9 +454,9 @@ static void guess_mv(ERContext *s) num_avail = 0; if (s->last_pic.motion_val[0]) { - if (s->last_pic.tf) - ff_thread_await_progress(s->last_pic.tf, mb_height-1, 0); - else + if (s->last_pic.tf) { + await_h264_reference(s, 16 * mb_height - 1); + } else ff_thread_progress_await(s->last_pic.progress, mb_height - 1); } for (i = 0; i < mb_width * mb_height; i++) { @@ -787,7 +809,8 @@ static int is_intra_more_likely(ERContext *s) mb_x * 16 + mb_y * 16 * linesize[0]; if (s->avctx->codec_id == AV_CODEC_ID_H264) { - // FIXME + /* Both SADs below read the reference, including its next MB row. */ + await_h264_reference(s, 16 * (mb_y + 2) - 1); } else { ff_thread_progress_await(s->last_pic.progress, mb_y); } -- 2.54.0 >From 7cb0b4be9dd148be392117644b52dea51a49aab5 Mon Sep 17 00:00:00 2001 From: hyena <[email protected]> Date: Fri, 16 Oct 2026 06:04:04 -0700 Subject: [PATCH 2/5] avcodec/error_resilience: reject unavailable H.264 motion predictors An H.264 picture's motion buffers may contain values from a previous use of the buffer when a macroblock does not have usable list-0 motion. Error concealment currently copies these values into temporal and spatial predictors without checking whether the motion is available. Check the macroblock type, list-0 use and reference index before reading the predictor. Set an unavailable predictor's reference to -1 so that the existing candidate selection skips it. Retain the existing behavior for other codecs. The damaged 640x352 regression stream exercises this path: without this change its single-thread output differs in 24 frames, and frame-threaded decoding does not match single-thread decoding. Signed-off-by: hyena <[email protected]> --- libavcodec/error_resilience.c | 82 +++++++++++++++++++++++------------ 1 file changed, 55 insertions(+), 27 deletions(-) diff --git a/libavcodec/error_resilience.c b/libavcodec/error_resilience.c index 14d9a31965..33c14bc34f 100644 --- a/libavcodec/error_resilience.c +++ b/libavcodec/error_resilience.c @@ -253,6 +253,25 @@ fail: av_freep(&dist); } +/** + * Check whether a block of pic has usable list-0 motion. + * + * For H.264, the motion and reference tables of a block without such motion + * can retain data from a previous user of the pooled buffer. Other codecs + * always report the motion as usable. + * + * @param mb_xy macroblock index + * @param block 8x8 block index within the macroblock, 0-3 + */ +static int has_forward_mv(const ERContext *s, const ERPicture *pic, + int mb_xy, int block) +{ + return s->avctx->codec_id != AV_CODEC_ID_H264 || + (!IS_INTRA(pic->mb_type[mb_xy]) && + (pic->mb_type[mb_xy] & MB_TYPE_L0) && + pic->ref_index[0][4 * mb_xy + block] >= 0); +} + /** * simple horizontal deblocking filter used for error resilience * @param w width in 8 pixel blocks @@ -429,6 +448,24 @@ static void await_h264_reference(const ERContext *s, int row) } } +/** + * Read the list-0 motion vector and reference of a predictor macroblock. + * An unavailable predictor yields a zero vector and *ref = -1, which the + * candidate selection in guess_mv() skips. + */ +static void get_mv_predictor(const ERContext *s, const ERPicture *pic, + int mb_xy, int mot_index, int mv[2], int *ref) +{ + if (has_forward_mv(s, pic, mb_xy, 0)) { + mv[0] = pic->motion_val[0][mot_index][0]; + mv[1] = pic->motion_val[0][mot_index][1]; + *ref = pic->ref_index[0][4 * mb_xy]; + } else { + mv[0] = mv[1] = 0; + *ref = -1; + } +} + static void guess_mv(ERContext *s) { int (*blocklist)[2], (*next_blocklist)[2]; @@ -476,9 +513,12 @@ static void guess_mv(ERContext *s) const int mb_y= mb_xy / s->mb_stride; const int mb_x= mb_xy % s->mb_stride; const int mot_index= (mb_x + mb_y*mot_stride) * mot_step; - s->cur_pic.motion_val[0][mot_index][0]= s->last_pic.motion_val[0][mot_index][0]; - s->cur_pic.motion_val[0][mot_index][1]= s->last_pic.motion_val[0][mot_index][1]; - s->cur_pic.ref_index[0][4*mb_xy] = s->last_pic.ref_index[0][4*mb_xy]; + int mv[2], ref; + + get_mv_predictor(s, &s->last_pic, mb_xy, mot_index, mv, &ref); + s->cur_pic.motion_val[0][mot_index][0] = mv[0]; + s->cur_pic.motion_val[0][mot_index][1] = mv[1]; + s->cur_pic.ref_index[0][4 * mb_xy] = ref; } } @@ -565,39 +605,27 @@ static void guess_mv(ERContext *s) mot_index = (mb_x + mb_y * mot_stride) * mot_step; if (mb_x > 0 && fixed[mb_xy - 1] > 1) { - mv_predictor[pred_count][0] = - s->cur_pic.motion_val[0][mot_index - mot_step][0]; - mv_predictor[pred_count][1] = - s->cur_pic.motion_val[0][mot_index - mot_step][1]; - ref[pred_count] = - s->cur_pic.ref_index[0][4 * (mb_xy - 1)]; + get_mv_predictor(s, &s->cur_pic, mb_xy - 1, + mot_index - mot_step, + mv_predictor[pred_count], &ref[pred_count]); pred_count++; } if (mb_x + 1 < mb_width && fixed[mb_xy + 1] > 1) { - mv_predictor[pred_count][0] = - s->cur_pic.motion_val[0][mot_index + mot_step][0]; - mv_predictor[pred_count][1] = - s->cur_pic.motion_val[0][mot_index + mot_step][1]; - ref[pred_count] = - s->cur_pic.ref_index[0][4 * (mb_xy + 1)]; + get_mv_predictor(s, &s->cur_pic, mb_xy + 1, + mot_index + mot_step, + mv_predictor[pred_count], &ref[pred_count]); pred_count++; } if (mb_y > 0 && fixed[mb_xy - mb_stride] > 1) { - mv_predictor[pred_count][0] = - s->cur_pic.motion_val[0][mot_index - mot_stride * mot_step][0]; - mv_predictor[pred_count][1] = - s->cur_pic.motion_val[0][mot_index - mot_stride * mot_step][1]; - ref[pred_count] = - s->cur_pic.ref_index[0][4 * (mb_xy - s->mb_stride)]; + get_mv_predictor(s, &s->cur_pic, mb_xy - mb_stride, + mot_index - mot_stride * mot_step, + mv_predictor[pred_count], &ref[pred_count]); pred_count++; } if (mb_y + 1<mb_height && fixed[mb_xy + mb_stride] > 1) { - mv_predictor[pred_count][0] = - s->cur_pic.motion_val[0][mot_index + mot_stride * mot_step][0]; - mv_predictor[pred_count][1] = - s->cur_pic.motion_val[0][mot_index + mot_stride * mot_step][1]; - ref[pred_count] = - s->cur_pic.ref_index[0][4 * (mb_xy + s->mb_stride)]; + get_mv_predictor(s, &s->cur_pic, mb_xy + mb_stride, + mot_index + mot_stride * mot_step, + mv_predictor[pred_count], &ref[pred_count]); pred_count++; } if (pred_count == 0) -- 2.54.0 >From bd84dcaf64634625941a0367b914d9d8ee654a22 Mon Sep 17 00:00:00 2001 From: hyena <[email protected]> Date: Fri, 16 Oct 2026 06:04:06 -0700 Subject: [PATCH 3/5] avcodec/h264dec: write concealed motion back to the picture h264_er_decode_mb() fills the slice motion cache for reconstruction but does not store that motion in the picture's motion field. Subsequent concealment prediction and filtering can therefore read motion that does not describe the reconstructed macroblock. Write the selected vector to all sixteen 4x4 blocks of the concealed macroblock. When an invalid reference prevents reconstruction, invalidate all four list-0 reference entries instead of leaving them available. Update the strict data-partitioning CIP reference for the resulting filter decisions. Stale picture vectors could cause filtering between blocks reconstructed with the same motion. With this series, repeated strict CIP checks at 2, 3, 4, 8 and 16 frame threads match single-thread output. Removing this motion writeback restores the mismatch at 2 and 8 threads. This change corrects motion metadata; it does not clear residual coefficients left by failed macroblock decoding. Signed-off-by: hyena <[email protected]> --- libavcodec/h264dec.c | 6 ++++++ .../ref/fate/h264-data-partitioning-cip-strict | 18 +++++++++--------- 2 files changed, 15 insertions(+), 9 deletions(-) diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index 597bdae548..91bcd2b9d3 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -90,6 +90,8 @@ static void h264_er_decode_mb(void *opaque, int ref, int mv_dir, int mv_type, } if ((sl->ref_list[0][ref].reference&3) != 3) { av_log(h->avctx, AV_LOG_DEBUG, "Reference invalid\n"); + fill_rectangle(&h->cur_pic.ref_index[0][4 * sl->mb_xy], + 2, 2, 2, -1, 1); return; } fill_rectangle(&h->cur_pic.ref_index[0][4 * sl->mb_xy], @@ -97,6 +99,10 @@ static void h264_er_decode_mb(void *opaque, int ref, int mv_dir, int mv_type, fill_rectangle(&sl->ref_cache[0][scan8[0]], 4, 4, 8, ref, 1); fill_rectangle(sl->mv_cache[0][scan8[0]], 4, 4, 8, pack16to32((*mv)[0][0][0], (*mv)[0][0][1]), 4); + /* Keep the picture's motion field consistent with the reconstructed MB. */ + fill_rectangle(&h->cur_pic.motion_val[0][4 * mb_x + 4 * mb_y * h->b_stride], + 4, 4, h->b_stride, + pack16to32((*mv)[0][0][0], (*mv)[0][0][1]), 4); sl->mb_mbaff = sl->mb_field_decoding_flag = 0; ff_h264_hl_decode_mb(h, &h->slice_ctx[0]); diff --git a/tests/ref/fate/h264-data-partitioning-cip-strict b/tests/ref/fate/h264-data-partitioning-cip-strict index 73e882dedc..09b49469bd 100644 --- a/tests/ref/fate/h264-data-partitioning-cip-strict +++ b/tests/ref/fate/h264-data-partitioning-cip-strict @@ -4,12 +4,12 @@ #dimensions 0: 176x144 #sar 0: 0/1 0, 0, 0, 1, 38016, 0x1b6c95f8 -0, 1, 1, 1, 38016, 0xd3d595d5 -0, 2, 2, 1, 38016, 0xea00960e -0, 3, 3, 1, 38016, 0x9420960e -0, 4, 4, 1, 38016, 0x79df960c -0, 5, 5, 1, 38016, 0x76dc960b -0, 6, 6, 1, 38016, 0x304b960b -0, 7, 7, 1, 38016, 0x08ee960b -0, 8, 8, 1, 38016, 0xf5a295cc -0, 9, 9, 1, 38016, 0xf7b595cc +0, 1, 1, 1, 38016, 0x1b6c95f8 +0, 2, 2, 1, 38016, 0xd2579638 +0, 3, 3, 1, 38016, 0xd2579638 +0, 4, 4, 1, 38016, 0xd2579638 +0, 5, 5, 1, 38016, 0xd2579638 +0, 6, 6, 1, 38016, 0xd2579638 +0, 7, 7, 1, 38016, 0xd2579638 +0, 8, 8, 1, 38016, 0xeb0395f8 +0, 9, 9, 1, 38016, 0xeb0395f8 -- 2.54.0 >From 2b74fd55cb3d372b56c9f2e8c02b26d35e140098 Mon Sep 17 00:00:00 2001 From: hyena <[email protected]> Date: Fri, 16 Oct 2026 06:04:08 -0700 Subject: [PATCH 4/5] avcodec/error_resilience: validate motion used by concealment filtering The concealment filters use motion similarity to skip a boundary, even when an H.264 block has no valid list-0 motion. Such blocks can retain unrelated motion-buffer values, making the filtering decision depend on buffer reuse. Require usable forward motion on both sides before applying this skip condition. For luma, select the reference entry for the actual 8x8 quadrant; for chroma, use the containing macroblock. Reuse the motion availability helper introduced earlier in this series. Other codecs keep their existing behavior. The 24-frame damaged B-frame regression stream changes in three frames when this check is removed. Signed-off-by: hyena <[email protected]> --- libavcodec/error_resilience.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/libavcodec/error_resilience.c b/libavcodec/error_resilience.c index 33c14bc34f..6a65db4c19 100644 --- a/libavcodec/error_resilience.c +++ b/libavcodec/error_resilience.c @@ -272,6 +272,19 @@ static int has_forward_mv(const ERContext *s, const ERPicture *pic, pic->ref_index[0][4 * mb_xy + block] >= 0); } +/** + * Check whether the current picture has usable list-0 motion for a filter + * block. x and y are in the 8x8 units used by h_block_filter() and + * v_block_filter(): 8x8 luma blocks, or whole macroblocks for chroma. + */ +static int has_block_mv(const ERContext *s, int x, int y, int is_luma) +{ + const int mb_xy = (x >> is_luma) + (y >> is_luma) * s->mb_stride; + const int block = is_luma ? (x & 1) + 2 * (y & 1) : 0; + + return has_forward_mv(s, &s->cur_pic, mb_xy, block); +} + /** * simple horizontal deblocking filter used for error resilience * @param w width in 8 pixel blocks @@ -302,6 +315,8 @@ static void h_block_filter(ERContext *s, uint8_t *dst, int w, if (!(left_damage || right_damage)) continue; // both undamaged if ((!left_intra) && (!right_intra) && + has_block_mv(s, b_x, b_y, is_luma) && + has_block_mv(s, b_x + 1, b_y, is_luma) && FFABS(left_mv[0] - right_mv[0]) + FFABS(left_mv[1] + right_mv[1]) < 2) continue; @@ -374,6 +389,8 @@ static void v_block_filter(ERContext *s, uint8_t *dst, int w, int h, continue; // both undamaged if ((!top_intra) && (!bottom_intra) && + has_block_mv(s, b_x, b_y, is_luma) && + has_block_mv(s, b_x, b_y + 1, is_luma) && FFABS(top_mv[0] - bottom_mv[0]) + FFABS(top_mv[1] + bottom_mv[1]) < 2) continue; -- 2.54.0 >From df49b01da09efb4e58d34fe7a571c452d496ee83 Mon Sep 17 00:00:00 2001 From: hyena <[email protected]> Date: Fri, 16 Oct 2026 06:04:10 -0700 Subject: [PATCH 5/5] fate/h264: cover concealment motion state across frame-thread counts Add three damaged progressive H.264 streams with shared framemd5 references at explicit one-, two- and eight-frame-thread settings: * er-narrow: concealed-motion writeback; * er-mv-predictor40: rejection of unavailable motion predictors; * er-filter-bframes: validation of motion used by concealment filtering. The B-frame stream contains one 24-frame GOP and ends before the SPS/IDR of the next GOP. Removing the respective fixes changes the checked output. Add explicit two- and eight-thread checks of the existing strict data-partitioning CIP stream. Its output now matches the single-thread reference in the checked runs; reverting concealed-motion writeback restores the mismatch. Retain the original one-thread target and update its comment to identify the explicit frame-thread variants. These tests exercise specific regressions. They do not guarantee determinism for all damaged inputs or reliably force the reference-wait race on every run. Signed-off-by: hyena <[email protected]> --- tests/fate/h264.mak | 30 +++++++++++++- tests/ref/fate/h264-er-filter-bframes | 33 +++++++++++++++ tests/ref/fate/h264-er-mv-predictor40 | 59 +++++++++++++++++++++++++++ tests/ref/fate/h264-er-narrow | 45 ++++++++++++++++++++ 4 files changed, 166 insertions(+), 1 deletion(-) create mode 100644 tests/ref/fate/h264-er-filter-bframes create mode 100644 tests/ref/fate/h264-er-mv-predictor40 create mode 100644 tests/ref/fate/h264-er-narrow diff --git a/tests/fate/h264.mak b/tests/fate/h264.mak index 1ee70f216d..78a1d41b6a 100644 --- a/tests/fate/h264.mak +++ b/tests/fate/h264.mak @@ -258,6 +258,33 @@ FATE_H264_FFPROBE-$(call PARSERDEMDEC, H264, H264, H264) += fate-h264-timecode-p FATE_H264_FFPROBE-$(call PARSERDEMDEC, H264, H264, H264) += fate-h264-skip-pred \ fate-h264-skip-pred-fields +# Error concealment motion state at explicit frame-thread counts. +FATE_H264-$(call FRAMEMD5, H264, H264, H264_PARSER) += fate-h264-er-narrow-1 fate-h264-er-narrow-2 fate-h264-er-narrow-8 +fate-h264-er-narrow-1 fate-h264-er-narrow-2 fate-h264-er-narrow-8: REF = $(SRC_PATH)/tests/ref/fate/h264-er-narrow +fate-h264-er-narrow-1: CMD = threads=1 thread_type=frame framemd5 -i $(TARGET_SAMPLES)/h264/er-narrow.h264 -threads:v 1 +fate-h264-er-narrow-2: CMD = threads=2 thread_type=frame framemd5 -i $(TARGET_SAMPLES)/h264/er-narrow.h264 -threads:v 1 +fate-h264-er-narrow-8: CMD = threads=8 thread_type=frame framemd5 -i $(TARGET_SAMPLES)/h264/er-narrow.h264 -threads:v 1 + +FATE_H264-$(call FRAMEMD5, H264, H264, H264_PARSER) += fate-h264-er-mv-predictor40-1 fate-h264-er-mv-predictor40-2 fate-h264-er-mv-predictor40-8 +fate-h264-er-mv-predictor40-1 fate-h264-er-mv-predictor40-2 fate-h264-er-mv-predictor40-8: REF = $(SRC_PATH)/tests/ref/fate/h264-er-mv-predictor40 +fate-h264-er-mv-predictor40-1: CMD = threads=1 thread_type=frame framemd5 -i $(TARGET_SAMPLES)/h264/er-mv-predictor40.h264 -threads:v 1 +fate-h264-er-mv-predictor40-2: CMD = threads=2 thread_type=frame framemd5 -i $(TARGET_SAMPLES)/h264/er-mv-predictor40.h264 -threads:v 1 +fate-h264-er-mv-predictor40-8: CMD = threads=8 thread_type=frame framemd5 -i $(TARGET_SAMPLES)/h264/er-mv-predictor40.h264 -threads:v 1 + +FATE_H264-$(call FRAMEMD5, H264, H264, H264_PARSER) += fate-h264-er-filter-bframes-1 fate-h264-er-filter-bframes-2 fate-h264-er-filter-bframes-8 +fate-h264-er-filter-bframes-1 fate-h264-er-filter-bframes-2 fate-h264-er-filter-bframes-8: REF = $(SRC_PATH)/tests/ref/fate/h264-er-filter-bframes +fate-h264-er-filter-bframes-1: CMD = threads=1 thread_type=frame framemd5 -i $(TARGET_SAMPLES)/h264/er-filter-bframes.h264 -threads:v 1 +fate-h264-er-filter-bframes-2: CMD = threads=2 thread_type=frame framemd5 -i $(TARGET_SAMPLES)/h264/er-filter-bframes.h264 -threads:v 1 +fate-h264-er-filter-bframes-8: CMD = threads=8 thread_type=frame framemd5 -i $(TARGET_SAMPLES)/h264/er-filter-bframes.h264 -threads:v 1 + +FATE_H264-$(call FRAMECRC, H264, H264, H264_PARSER) += fate-h264-er-cip-strict-2 +fate-h264-er-cip-strict-2: REF = $(SRC_PATH)/tests/ref/fate/h264-data-partitioning-cip-strict +fate-h264-er-cip-strict-2: CMD = threads=2 thread_type=frame framecrc -bug 0 -i $(TARGET_SAMPLES)/h264/data_partitioning_cip.h264 -threads:v 1 + +FATE_H264-$(call FRAMECRC, H264, H264, H264_PARSER) += fate-h264-er-cip-strict-8 +fate-h264-er-cip-strict-8: REF = $(SRC_PATH)/tests/ref/fate/h264-data-partitioning-cip-strict +fate-h264-er-cip-strict-8: CMD = threads=8 thread_type=frame framecrc -bug 0 -i $(TARGET_SAMPLES)/h264/data_partitioning_cip.h264 -threads:v 1 + FATE_SAMPLES_AVCONV += $(FATE_H264-yes) FATE_SAMPLES_FFPROBE += $(FATE_H264_FFPROBE-yes) fate-h264: $(FATE_H264-yes) $(FATE_H264_FFPROBE-yes) @@ -480,7 +507,8 @@ fate-h264-data-partitioning-ab: CMD = framecrc -i $(TARGET_SAM # constrained_intra_pred_flag=1, which changes the nC derivation (9.2.1) fate-h264-data-partitioning-cip: CMD = framecrc -bug h264_dp_nnz -i $(TARGET_SAMPLES)/h264/data_partitioning_cip.h264 # same stream read the way 9.2.1 is written instead of the way JM does it; that -# desynchronises the residual, so pin the threads to keep concealment stable +# desynchronises the residual. Keep a single-thread reference; the explicit +# frame-thread variants above use the same reference. fate-h264-data-partitioning-cip-strict: CMD = threads=1 framecrc -bug 0 -i $(TARGET_SAMPLES)/h264/data_partitioning_cip.h264 fate-h264-intra-refresh-recovery: CMD = framecrc -i $(TARGET_SAMPLES)/h264/intra_refresh.h264 -frames:v 10 fate-h264-invalid-ref-mod: CMD = framecrc -i $(TARGET_SAMPLES)/h264/h264refframeregression.mp4 -an -frames 10 -pix_fmt yuv420p10le -vf scale diff --git a/tests/ref/fate/h264-er-filter-bframes b/tests/ref/fate/h264-er-filter-bframes new file mode 100644 index 0000000000..be8b4efa79 --- /dev/null +++ b/tests/ref/fate/h264-er-filter-bframes @@ -0,0 +1,33 @@ +#format: frame checksums +#version: 2 +#hash: MD5 +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 1280x720 +#sar 0: 1/1 +#stream#, dts, pts, duration, size, hash +0, 0, 0, 1, 1382400, 6dc052610531bc88534053eb4cbb7423 +0, 1, 1, 1, 1382400, c2dbb3ed83d2605a16fd75a91fa0343c +0, 2, 2, 1, 1382400, 3f5dd910054ed4eceab689794f1cd004 +0, 3, 3, 1, 1382400, 03f8958fc7cdc7d0d1b2d5ea2b2c1ce0 +0, 4, 4, 1, 1382400, 64be57728277d8e535d573d74e3302b4 +0, 5, 5, 1, 1382400, db8c2f9f414477e530768d338cea902c +0, 6, 6, 1, 1382400, ad199fcd98f34ff5b448a870c34deb54 +0, 7, 7, 1, 1382400, 507c8f74ec472c88b3596c03c6502bae +0, 8, 8, 1, 1382400, 28c340533485b4dd70017f622ffad577 +0, 9, 9, 1, 1382400, eb981d45b52862a4a8079d728397ad2d +0, 10, 10, 1, 1382400, 0230f77b56b0e57eb6b133b428f43e77 +0, 11, 11, 1, 1382400, 45cc387201e37cb8ab50604c59b774e1 +0, 12, 12, 1, 1382400, 9d9dfd7ad64be77625701f9093b1aa13 +0, 13, 13, 1, 1382400, 56b5021d5be5e9a42906780120aa5313 +0, 14, 14, 1, 1382400, ff4684319f603d44db7b66a1accddf53 +0, 15, 15, 1, 1382400, d118356bd7d7b919ce58f39292d1b98d +0, 16, 16, 1, 1382400, ee457870087a48211ee45475e17619c1 +0, 17, 17, 1, 1382400, 171be295410c9db4644d269c638a03d8 +0, 18, 18, 1, 1382400, df253ddb078853777c6d479644405f3f +0, 19, 19, 1, 1382400, 43d2f4bfd5dd8386b195230879718a8a +0, 20, 20, 1, 1382400, cad4c883a5afea79de786a40e76ea483 +0, 21, 21, 1, 1382400, 53363bbe53b63f25bbf6253b804aa2b8 +0, 22, 22, 1, 1382400, c92781dd0dca9b663474a0aec5c795c1 +0, 23, 23, 1, 1382400, 2a50df2c0b87539a4a243a96b29a98af diff --git a/tests/ref/fate/h264-er-mv-predictor40 b/tests/ref/fate/h264-er-mv-predictor40 new file mode 100644 index 0000000000..d594144dfc --- /dev/null +++ b/tests/ref/fate/h264-er-mv-predictor40 @@ -0,0 +1,59 @@ +#format: frame checksums +#version: 2 +#hash: MD5 +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 640x352 +#sar 0: 1/1 +#stream#, dts, pts, duration, size, hash +0, 0, 0, 1, 337920, cda610c0fac144ab8b75dbeea41a0042 +0, 1, 1, 1, 337920, e2668e9647d474aeca45abd703002543 +0, 2, 2, 1, 337920, 57161daf1da036920ef219656fdfad27 +0, 3, 3, 1, 337920, 029310dd2593660d1b5a49a84b5f5a52 +0, 4, 4, 1, 337920, 3b9cbfa56f12c716c92f282ad95321db +0, 5, 5, 1, 337920, d27db9b8267f664ffcfbeb0e4795b22e +0, 6, 6, 1, 337920, 4b151a267c55b121f7b8309ca01de47f +0, 7, 7, 1, 337920, 943ce6acdcc90357b1c9f09ad17cd8fe +0, 8, 8, 1, 337920, 216e002a15b7d682763c9d593e506cf0 +0, 9, 9, 1, 337920, 52b22b4e2f69ed199d336cfab383d234 +0, 10, 10, 1, 337920, cbc6a3e6134f387cec63563f1202a133 +0, 11, 11, 1, 337920, a2fea94e82fc822cad336327cb3826f6 +0, 12, 12, 1, 337920, 348ba2034c4d6653b2ff8124ea747a47 +0, 13, 13, 1, 337920, 03cc1f67a35628fda43dc03b06577a6c +0, 14, 14, 1, 337920, c3fea414a175b9f47bc5cd0430fc8de1 +0, 15, 15, 1, 337920, c5f371844f9f38727e2f29168382398c +0, 16, 16, 1, 337920, a453f24504c3b70e9757b64c39bd1a05 +0, 17, 17, 1, 337920, fcd65f44c6e21688ab790e4efa8ff912 +0, 18, 18, 1, 337920, c928cb8a26ba227b263da9a941d90674 +0, 19, 19, 1, 337920, fc2e5f25801de8386a98c0b34e4d7112 +0, 20, 20, 1, 337920, 2fc7f337bd2cd8c4859aca7dcb763210 +0, 21, 21, 1, 337920, 2b1de2f64682f1f64624d140620dd205 +0, 22, 22, 1, 337920, 6ff40881601632f3f2a562ade3a25877 +0, 23, 23, 1, 337920, 52a6d77d4b2e4cc1f404469c0b0564c3 +0, 24, 24, 1, 337920, 337d1ac1d5023896f2b3a11f58a1651c +0, 25, 25, 1, 337920, 570a605c1bf15db9877261d2990e736b +0, 26, 26, 1, 337920, c29ccbd1b02f70df1700a0099934278a +0, 27, 27, 1, 337920, 5bd1414c3e73e569f0addb5376adf14c +0, 28, 28, 1, 337920, b7a3e7e04a4d6caf06742d2ed6ee37f8 +0, 29, 29, 1, 337920, 6da09e450c72cb4421c1f4f33f82b6db +0, 30, 30, 1, 337920, 00dd2f28bd496a08590669c6fecf26b9 +0, 31, 31, 1, 337920, 3cd4b9cf057a77b51057f89745874481 +0, 32, 32, 1, 337920, d5894d4d126aa47c8a9688c5054a0045 +0, 33, 33, 1, 337920, 4fea9e5ea032d43eb4dffe990d61b53a +0, 34, 34, 1, 337920, 43f817e00afe4ac7e5eafbcc80b80874 +0, 35, 35, 1, 337920, 838e1de46e47b385827590de2d359443 +0, 36, 36, 1, 337920, 7066247d856e979cef01bddaa9fe61f2 +0, 37, 37, 1, 337920, 8ba25b0439154f255275d1ceee2b0087 +0, 38, 38, 1, 337920, fccbee0c8e910ff82e5a9438d6d01b67 +0, 39, 39, 1, 337920, 6b02c772ce9adeedf2eecdccdd39c15a +0, 40, 40, 1, 337920, b827c7b45390e411955894e32bc49cfd +0, 41, 41, 1, 337920, c9b5ab4a3fc830edc92c90b1107ed6e9 +0, 42, 42, 1, 337920, 8d8b652dabbc41defbd96c28b02b9641 +0, 43, 43, 1, 337920, bd78402deb01d2a108efe742f75d87ba +0, 44, 44, 1, 337920, 821252691da823a6e93f03a4a29a0368 +0, 45, 45, 1, 337920, c7983e624e09de9560d72504129769ad +0, 46, 46, 1, 337920, 9366984fdc4e0983d9f306218e93961c +0, 47, 47, 1, 337920, 667f3f8aede8193d12916fc41ef039b6 +0, 48, 48, 1, 337920, 4d77b3309b325a577d1696ce49f31a8d +0, 49, 49, 1, 337920, 2b2886bb265013781b3e28a9ece99966 diff --git a/tests/ref/fate/h264-er-narrow b/tests/ref/fate/h264-er-narrow new file mode 100644 index 0000000000..51e75ed6c0 --- /dev/null +++ b/tests/ref/fate/h264-er-narrow @@ -0,0 +1,45 @@ +#format: frame checksums +#version: 2 +#hash: MD5 +#tb 0: 1/24 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 32x256 +#sar 0: 1/1 +#stream#, dts, pts, duration, size, hash +0, 0, 0, 2, 12288, aedb678ee0d36e9dba39beaa345d2241 +0, 2, 2, 2, 12288, aedb678ee0d36e9dba39beaa345d2241 +0, 4, 4, 2, 12288, aedb678ee0d36e9dba39beaa345d2241 +0, 6, 6, 2, 12288, a2d26f77bb30ebf659ef5e84e391daa7 +0, 8, 8, 2, 12288, 7d4834ef494601a84dd351d3d2f6d8ba +0, 10, 10, 2, 12288, a4ebb7318428e62a58e1a3dba0b968cd +0, 12, 12, 2, 12288, 92242a16edeb013200387011e7bfa1dd +0, 14, 14, 2, 12288, 78b55b04fc92a30d9f60e3b5e75b6bf0 +0, 16, 16, 2, 12288, 6246741882726dfa2fc01e92276d03da +0, 18, 18, 2, 12288, 5833705890d9dbef71ad309b817fd850 +0, 20, 20, 2, 12288, 5b65694accce1c6469081a3f2adf3a8c +0, 22, 22, 2, 12288, 7f7ff10f42633953de5b4017e85cc669 +0, 24, 24, 2, 12288, 6cf0bbfa3c0566666b466c0454f0c15f +0, 26, 26, 2, 12288, c3b800b8ebea676ec0463806835c438c +0, 28, 28, 2, 12288, deb2d0c56c93a107f9552c369f0536ef +0, 30, 30, 2, 12288, 1c2e1c15ded5f4281a00bfc3528efdb9 +0, 32, 32, 2, 12288, 7c4dd7bc5bf0ac8854f12d0953b821d8 +0, 34, 34, 2, 12288, dc4f3e0012c91028d42f8bc57ac60ff6 +0, 36, 36, 2, 12288, 797b7475e9023c737084179c07f4f2a5 +0, 38, 38, 2, 12288, ab5e51a71f50988020ac2a70553fc119 +0, 40, 40, 2, 12288, 90c6ab67a154c57ae51495465400e411 +0, 42, 42, 2, 12288, 5035453a473f7764221fda4d8834acf6 +0, 44, 44, 2, 12288, 1d30e962eed037549327f4d77915a647 +0, 46, 46, 2, 12288, 33a5e974d79a08867338de25bd96cf14 +0, 48, 48, 2, 12288, 5035453a473f7764221fda4d8834acf6 +0, 50, 50, 2, 12288, 7e70c8f21f01a57c69260077edcd0eaf +0, 52, 52, 2, 12288, 9a4af6ce0503cc35e5e029715f30bfcd +0, 54, 54, 2, 12288, 0352eac1c1440cb280e2b74cb01d0c6f +0, 56, 56, 2, 12288, b29a864062754ac0a1c2ea801dc1ef41 +0, 58, 58, 2, 12288, 0dd5ac794455906a33f8758e67e8eb40 +0, 60, 60, 2, 12288, f56f8d8730a52103a8068114b50570a4 +0, 62, 62, 2, 12288, f705b8f4f9f42030328858e1d3d6c02b +0, 64, 64, 2, 12288, 0b5422cf1dcc232392cbec213ea1fa7c +0, 66, 66, 2, 12288, ff58dba581f52e02b1f8e826068d20ba +0, 68, 68, 2, 12288, cd86c80f66a81106c28013aa5a2752ba +0, 70, 70, 2, 12288, f44c590b84dec91195fdd9ea3cf271fd -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
