PR #25902 opened by Marti2203
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25902
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25902.patch
For formats with more than 8 bits, the two histograms and the contrast change
table have `1 << depth` entries, but `compute_histogram16()` and
`midequalizer16()` index them with the raw 16-bit sample. A sample with bits
set above the depth, e.g. from a `gray9le` rawvideo input with arbitrary words,
reads and writes far past the 512-entry histogram:
ffmpeg -f rawvideo -pix_fmt gray9le -s 64x64 -i in.gray9 \
-f rawvideo -pix_fmt gray9le -s 64x64 -i in.gray9 \
-filter_complex "[0:v][1:v]midequalizer" -f null -
with `in.gray9` holding words up to 0xffff gives an ASan SEGV in
`compute_histogram16()` (`vf_midequalizer.c:160`).
The patch clips the sample to the last entry before using it as an index, in
both places. In-depth input is not affected: a gray9le file with in-range
words, the yuv420p10 `geq` graph from #25104 and a gray16 `testsrc` pair give
the same framemd5 before and after. The out-of-depth file now runs without an
ASan report.
This touches different functions from #25104 (`compute_contrast_change()`), and
the two patches apply together.
This issue was found by an LLM-Enhanced Static Analysis Engine and verified
manually.
>From 44f55aecb2b97119942c7cc9f385112dbe9d6fab Mon Sep 17 00:00:00 2001
From: Martin Mirchev <[email protected]>
Date: Tue, 10 Nov 2026 12:08:24 +0800
Subject: [PATCH] avfilter/vf_midequalizer: clip out-of-depth samples to the
histogram
For formats with more than 8 bits, the histograms and the contrast
change table have 1 << depth entries, but compute_histogram16() and
midequalizer16() index them with the raw 16-bit sample. A sample with
bits set above the depth, e.g. from a gray9le rawvideo input with
arbitrary words, reads and writes far past the 512-entry histogram.
Clip the sample to the last entry before using it as an index.
Signed-off-by: Martin Mirchev <[email protected]>
---
libavfilter/vf_midequalizer.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/libavfilter/vf_midequalizer.c b/libavfilter/vf_midequalizer.c
index 9720cefdf20..5aa90024dc7 100644
--- a/libavfilter/vf_midequalizer.c
+++ b/libavfilter/vf_midequalizer.c
@@ -157,7 +157,7 @@ static void compute_histogram16(const uint16_t *src,
ptrdiff_t linesize,
for (y = 0; y < h; y++) {
for (x = 0; x < w; x++) {
- histogram[src[x]] += 1;
+ histogram[FFMIN(src[x], hsize - 1)] += 1;
}
src += linesize;
}
@@ -230,7 +230,7 @@ static void midequalizer16(const uint8_t *in0, const
uint8_t *in1,
for (y = 0; y < h0; y++) {
for (x = 0; x < w0; x++) {
- d[x] = cchange[i[x]];
+ d[x] = cchange[FFMIN(i[x], hsize - 1)];
}
d += dlinesize / 2;
i += linesize1 / 2;
--
2.54.0
_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]