PR #25467 opened by ohah URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25467 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25467.patch
# Summary of changes Fix incorrect inverse quantization for valid 12-bit HEVC QP indices 74 and 75. `rem6` and `div6` allocate 76 entries but initialize only 74. The implicit zero entries select a scale of 40 instead of `51 << 12` and `57 << 12`. The fix appends the missing remainder/quotient values without changing the other entries or adding arithmetic to the decoding path. The second commit adds two single-frame 128x64 12-bit 4:4:4 FATE tests. Both commits include Signed-off-by trailers and were prepared with `git format-patch`, then applied through the Forgejo patch editor. ## Independent reproduction The attached synthetic inputs were generated with x265 4.2. Expected pixels were copied from the encoder's reconstructed output (`x265_picture` from `encoder_encode`), independently of FFmpeg. Temporary instrumentation confirmed that the inputs exercise dequantization indices 74 and 75 respectively; the instrumentation is not included in these commits. | Input | Samples | Different before | Different after | | --- | ---: | ---: | ---: | | QP 74 | 24,576 | 16,377 | 0 | | QP 75 | 24,576 | 16,379 | 0 | The FATE MD5 references also match the raw encoder reconstruction directly. [Generator, reconstructed YUV, hashes and reproduction instructions](https://github.com/ohah/FFmpeg/tree/df6c311a005073ab714089c5a7b727098e30bb1d/repro/hevc-qp74-75). ## Validation Tested on macOS arm64 against master `46f23be8b1976ed4c8ec85584daabcf0e57a7b9e`: - Both new FATE tests fail with checksum differences on the original decoder and pass after the fix, including `THREADS=4`. - `make -j8 fate` without external samples passes 316 available tests in the minimal configuration below. This is not a run of the entire external FATE sample suite or all enabled codecs. - `git diff --check` passes. `tools/patcheck` was run with GNU grep/findutils; its symbol-prefix warnings refer to Makefile commands, and its minor-change changelog warning does not require a Changelog entry for this fix. ```sh ./configure --disable-everything --disable-autodetect --disable-doc \ --disable-network --disable-programs --enable-ffmpeg \ --enable-decoder=hevc --enable-parser=hevc \ --enable-encoder=rawvideo,pcm_s16le --enable-demuxer=hevc \ --enable-muxer=rawvideo,framecrc,framemd5 \ --enable-protocol=file,pipe --enable-filter=null,format,scale \ --enable-swscale make -j8 make fate-hevc-12bit-qp74 fate-hevc-12bit-qp75 SAMPLES=/path/to/samples make fate-hevc-12bit-qp74 fate-hevc-12bit-qp75 SAMPLES=/path/to/samples THREADS=4 make -j8 fate ``` ## New FATE samples The attached files are 7,217 and 6,781 bytes. They are synthetic and contain no photographs or personal data. Please add them to the external FATE suite at the following paths; attached filenames match the basenames: ```fate-samples hevc/12bit-444-qp74.hevc hevc/12bit-444-qp75.hevc ``` >From 97ff87d7d88968523157640e41c9c03e63691e55 Mon Sep 17 00:00:00 2001 From: ohah <[email protected]> Date: Mon, 26 Oct 2026 10:05:43 +0000 Subject: [PATCH 1/2] avcodec/hevc: initialize inverse quantization tables through QP 75 rem6 and div6 each have room for 76 entries but only initialize the first 74. The remaining entries are implicitly zero, so QP 74 and 75 use an inverse quantization scale of 40 instead of 51 << 12 and 57 << 12, respectively. These indices are valid for 12-bit HEVC, where the bit-depth offset is 24. Append the missing remainder and quotient entries so high-QP 12-bit residuals use the scaling specified in H.265 section 8.6.3. Synthetic 12-bit 4:4:4 inputs exercising each index differ from x265's reconstruction in 16377 and 16379 samples before this change. Both outputs match the encoder reconstruction exactly after the fix. Signed-off-by: ohah <[email protected]> --- libavcodec/hevc/cabac.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/hevc/cabac.c b/libavcodec/hevc/cabac.c index 57f43be06f3..74dbcc720c6 100644 --- a/libavcodec/hevc/cabac.c +++ b/libavcodec/hevc/cabac.c @@ -1110,14 +1110,14 @@ void ff_hevc_hls_residual_coding(HEVCLocalContext *lc, const HEVCPPS *pps, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, - 4, 5, 0, 1, 2, 3, 4, 5, 0, 1 + 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3 }; static const uint8_t div6[51 + 4 * 6 + 1] = { 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, - 10, 10, 11, 11, 11, 11, 11, 11, 12, 12 + 10, 10, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12 }; int qp_y = lc->qp_y; -- 2.54.0 >From 50972f5e67a50ec0198dcf3bf198e98196aa1e6f Mon Sep 17 00:00:00 2001 From: ohah <[email protected]> Date: Mon, 26 Oct 2026 10:06:49 +0000 Subject: [PATCH 2/2] fate/hevc: cover 12-bit inverse quantization at QP 74 and 75 Add two synthetic 128x64 12-bit 4:4:4 single-frame inputs that exercise the last two dequantization table entries. The references match the reconstructed pictures returned by the x265 4.2 encoder API. Both tests fail on the unmodified decoder and pass with the missing table entries initialized. The sample sizes are 7217 and 6781 bytes. Samples for hevc/12bit-444-qp74.hevc and hevc/12bit-444-qp75.hevc: https://raw.githubusercontent.com/ohah/FFmpeg/df6c311a005073ab714089c5a7b727098e30bb1d/repro/hevc-qp74-75/qp74.hevc https://raw.githubusercontent.com/ohah/FFmpeg/df6c311a005073ab714089c5a7b727098e30bb1d/repro/hevc-qp74-75/qp75.hevc The same directory contains the generator, independent reconstructed YUV outputs, reproduction instructions and SHA-256 hashes. Signed-off-by: ohah <[email protected]> --- tests/fate/hevc.mak | 5 +++++ tests/ref/fate/hevc-12bit-qp74 | 10 ++++++++++ tests/ref/fate/hevc-12bit-qp75 | 10 ++++++++++ 3 files changed, 25 insertions(+) create mode 100644 tests/ref/fate/hevc-12bit-qp74 create mode 100644 tests/ref/fate/hevc-12bit-qp75 diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak index b39358167ee..09143bbaa35 100644 --- a/tests/fate/hevc.mak +++ b/tests/fate/hevc.mak @@ -351,6 +351,11 @@ FATE_HEVC-$(call FRAMECRC, MOV, HEVC) += fate-hevc-alpha fate-hevc-color-reserved: CMD = framecrc -bsf:v hevc_metadata=colour_primaries=0:transfer_characteristics=0:matrix_coefficients=3 -i $(TARGET_SAMPLES)/hevc-conformance/AMP_A_Samsung_4.bit -vf scale,format=nv12 -frames:v 1 FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_METADATA_BSF SCALE_FILTER) += fate-hevc-color-reserved +# Exercise the last two inverse-quantization table entries with 12-bit 4:4:4. +fate-hevc-12bit-qp74: CMD = framemd5 -i $(TARGET_SAMPLES)/hevc/12bit-444-qp74.hevc -pix_fmt yuv444p12le +fate-hevc-12bit-qp75: CMD = framemd5 -i $(TARGET_SAMPLES)/hevc/12bit-444-qp75.hevc -pix_fmt yuv444p12le +FATE_HEVC-$(call FRAMEMD5, HEVC, HEVC, HEVC_PARSER) += fate-hevc-12bit-qp74 fate-hevc-12bit-qp75 + FATE_SAMPLES_AVCONV += $(FATE_HEVC-yes) FATE_SAMPLES_FFPROBE += $(FATE_HEVC_FFPROBE-yes) diff --git a/tests/ref/fate/hevc-12bit-qp74 b/tests/ref/fate/hevc-12bit-qp74 new file mode 100644 index 00000000000..086adebfe86 --- /dev/null +++ b/tests/ref/fate/hevc-12bit-qp74 @@ -0,0 +1,10 @@ +#format: frame checksums +#version: 2 +#hash: MD5 +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 128x64 +#sar 0: 0/1 +#stream#, dts, pts, duration, size, hash +0, 0, 0, 1, 49152, dd156676d75cc4801291a3a4d52ab375 diff --git a/tests/ref/fate/hevc-12bit-qp75 b/tests/ref/fate/hevc-12bit-qp75 new file mode 100644 index 00000000000..5e49351e404 --- /dev/null +++ b/tests/ref/fate/hevc-12bit-qp75 @@ -0,0 +1,10 @@ +#format: frame checksums +#version: 2 +#hash: MD5 +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 128x64 +#sar 0: 0/1 +#stream#, dts, pts, duration, size, hash +0, 0, 0, 1, 49152, 28f5c39656c53f3dc6c147200e7223f2 -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
