PR #25684 opened by kierank URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25684 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25684.patch
With constrained intra prediction the reference arrays are pre-filled before substitution with memset(..., 128, ...), a byte fill: above 8 bits every sample becomes 0x8080, outside the legal range, and top[-1] gets 128 rather than mid-grey. Fill them with 1 << (BIT_DEPTH - 1), the value the same function already uses when no samples are available. EXTEND() replaces memset because that value is not a byte pattern above 8 bits; 2 * MAX_TB_SIZE is a multiple of its 4-sample step, so it covers the same range as before. The placeholders are always overwritten before use, so the decoded output does not change: 21 HM-encoded streams (8 and 10 bit, CTB 16/32/64, constrained intra prediction with slices starting mid CTB row, including the case where a slice starts at the up-right CTB) decode identically before and after this change and to the HM reference, and also with the placeholder set to an arbitrary value. Found and written with AI >From 4a18e1711ca5854f12e8f42ca0742b72b200d730 Mon Sep 17 00:00:00 2001 From: Kieran Kunhya <[email protected]> Date: Mon, 2 Nov 2026 16:45:01 +0000 Subject: [PATCH] avcodec/hevc/pred_template: use the neutral value for CIP placeholders With constrained intra prediction the reference arrays are pre-filled before substitution with memset(..., 128, ...), a byte fill: above 8 bits every sample becomes 0x8080, outside the legal range, and top[-1] gets 128 rather than mid-grey. Fill them with 1 << (BIT_DEPTH - 1), the value the same function already uses when no samples are available. EXTEND() replaces memset because that value is not a byte pattern above 8 bits; 2 * MAX_TB_SIZE is a multiple of its 4-sample step, so it covers the same range as before. The placeholders are always overwritten before use, so the decoded output does not change: 21 HM-encoded streams (8 and 10 bit, CTB 16/32/64, constrained intra prediction with slices starting mid CTB row, including the case where a slice starts at the up-right CTB) decode identically before and after this change and to the HM reference, and also with the placeholder set to an arbitrary value. --- libavcodec/hevc/pred_template.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libavcodec/hevc/pred_template.c b/libavcodec/hevc/pred_template.c index 63e075e2827..1b47a53d800 100644 --- a/libavcodec/hevc/pred_template.c +++ b/libavcodec/hevc/pred_template.c @@ -206,9 +206,9 @@ do { \ for (i = 0; i < max; i += 2) cand_up_right |= (MVF(x_right_pu + i, y_top_pu).pred_flag == PF_INTRA); } - memset(left, 128, 2 * MAX_TB_SIZE*sizeof(pixel)); - memset(top , 128, 2 * MAX_TB_SIZE*sizeof(pixel)); - top[-1] = 128; + EXTEND(left, 1 << (BIT_DEPTH - 1), 2 * MAX_TB_SIZE); + EXTEND(top , 1 << (BIT_DEPTH - 1), 2 * MAX_TB_SIZE); + top[-1] = 1 << (BIT_DEPTH - 1); } if (cand_up_left) { left[-1] = POS(-1, -1); -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
