PR #25294 opened by Marti2203 URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25294 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25294.patch
In `rgbashift` with `edge=wrap`, the alpha plane is read from the wrong row: `rgbawrap_slice8/16` computes the source row as `(x - av) % h` instead of `(y - av) % h`, so every output alpha row is put together from a diagonal of the input. The R/G/B planes and `edge=smear` are fine. Because of this the alpha of gbrap, gbrap10, gbrap12 and gbrap16 gets scrambled even when all shifts are 0: ``` SRC='testsrc2=s=64x64:d=0.04,format=gbrap,geq=r=r(X\,Y):g=g(X\,Y):b=b(X\,Y):a=X*3+Y*5' ffmpeg -f lavfi -i "$SRC" -f md5 - # 86928129687935e642cb9dec93917a49 ffmpeg -f lavfi -i "$SRC" -vf rgbashift=edge=smear -f md5 - # 86928129687935e642cb9dec93917a49 ffmpeg -f lavfi -i "$SRC" -vf rgbashift=edge=wrap -f md5 - # before: 37cd02dc14275d56d67feedadb97fb78 ``` Before the patch 4032 of the 4096 alpha bytes of that 64x64 frame change, and the gbrap10le and gbrap16le outputs differ from the input too. After the patch `edge=wrap` with zero shifts gives back the input for all three formats (0 bytes changed). With non-zero shifts (`ah=-2:av=3`), each output alpha pixel now matches input pixel ((x+2) mod w, (y-3) mod h). The second commit adds `fate-filter-rgbashift-wrap` and `fate-filter-rgbashift-wrap-16`, which run the 8-bit and 16-bit wrap paths on a geq-generated gbrap/gbrap16le frame with non-zero shifts on every plane. Both fail without the fix and pass with it (THREADS=1 and 4). The existing `fate-filter-chromashift-smear` and `fate-filter-chromashift-wrap` still pass. This issue was found by an LLM-Enhanced Static Analysis Engine and verified manually. 🤖 Generated with [Claude Code](https://claude.com/claude-code) From 8af5d7cef8efbff87f5fa67351524526239b4560 Mon Sep 17 00:00:00 2001 From: Martin Mirchev <[email protected]> Date: Wed, 21 Oct 2026 18:16:35 +0800 Subject: [PATCH 1/2] avfilter/vf_chromashift: use the row index for the wrapped alpha row In rgbashift with edge=wrap, the source row of the alpha plane was computed from the column x instead of the row y, so each output alpha row was assembled from a diagonal of the input. The alpha plane of gbrap, gbrap10/12/16 was scrambled even with all shifts set to 0, while edge=smear and the R/G/B planes were correct. Signed-off-by: Martin Mirchev <[email protected]> --- libavfilter/vf_chromashift.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/vf_chromashift.c b/libavfilter/vf_chromashift.c index 29691afbdb1..66963ef5002 100644 --- a/libavfilter/vf_chromashift.c +++ b/libavfilter/vf_chromashift.c @@ -283,7 +283,7 @@ static int rgbawrap_slice ## depth(AVFilterContext *ctx, void *arg, int jobnr, i continue; \ for (int x = 0; x < w; x++) { \ int ax = (x - ah) % w; \ - int ay = (x - av) % h; \ + int ay = (y - av) % h; \ \ if (ax < 0) \ ax += w; \ -- 2.54.0 From 123817fd58f0692a85df406fd535662d8ffa840d Mon Sep 17 00:00:00 2001 From: Martin Mirchev <[email protected]> Date: Wed, 21 Oct 2026 18:17:29 +0800 Subject: [PATCH 2/2] fate/filter-video: add rgbashift edge=wrap tests with alpha Cover the 8-bit and 16-bit wrap paths of rgbashift on formats with an alpha plane, which previously had no test. Signed-off-by: Martin Mirchev <[email protected]> --- tests/fate/filter-video.mak | 4 ++++ tests/ref/fate/filter-rgbashift-wrap | 10 ++++++++++ tests/ref/fate/filter-rgbashift-wrap-16 | 10 ++++++++++ 3 files changed, 24 insertions(+) create mode 100644 tests/ref/fate/filter-rgbashift-wrap create mode 100644 tests/ref/fate/filter-rgbashift-wrap-16 diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak index 4a065ec0320..0188eaa0742 100644 --- a/tests/fate/filter-video.mak +++ b/tests/fate/filter-video.mak @@ -477,6 +477,10 @@ FATE_FILTER-$(call FILTERFRAMECRC, TESTSRC2 CHROMASHIFT) += fate-filter-chromash fate-filter-chromashift-smear: CMD = framecrc -lavfi testsrc2=r=5:d=1,chromashift=cbh=-1:cbv=1:crh=2:crv=-2:edge=smear -pix_fmt yuv420p fate-filter-chromashift-wrap: CMD = framecrc -lavfi testsrc2=r=5:d=1,chromashift=cbh=-1:cbv=1:crh=2:crv=-2:edge=wrap -pix_fmt yuv420p +FATE_FILTER-$(call FILTERFRAMECRC, TESTSRC2 FORMAT GEQ RGBASHIFT) += fate-filter-rgbashift-wrap fate-filter-rgbashift-wrap-16 +fate-filter-rgbashift-wrap: CMD = framecrc -lavfi 'testsrc2=s=64x64:r=5:d=1,format=gbrap,geq=r=X*4:g=Y*4:b=255-X*2:a=X+2*Y+N*8,rgbashift=rh=-1:gv=1:bh=2:ah=-3:av=2:edge=wrap' -pix_fmt gbrap +fate-filter-rgbashift-wrap-16: CMD = framecrc -lavfi 'testsrc2=s=64x64:r=5:d=1,format=gbrap16le,geq=r=X*4:g=Y*4:b=255-X*2:a=X+2*Y+N*8,rgbashift=rh=-1:gv=1:bh=2:ah=-3:av=2:edge=wrap' -pix_fmt gbrap16le + FATE_FILTER-$(call FILTERFRAMECRC, TESTSRC2 FPS DECIMATE) += fate-filter-decimate fate-filter-decimate: CMD = framecrc -lavfi testsrc2=r=24:d=10,fps=60,decimate=5,decimate=4,decimate=3 -pix_fmt yuv420p diff --git a/tests/ref/fate/filter-rgbashift-wrap b/tests/ref/fate/filter-rgbashift-wrap new file mode 100644 index 00000000000..ac5a248c517 --- /dev/null +++ b/tests/ref/fate/filter-rgbashift-wrap @@ -0,0 +1,10 @@ +#tb 0: 1/5 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 64x64 +#sar 0: 1/1 +0, 0, 0, 1, 16384, 0xdadea9ef +0, 1, 1, 1, 16384, 0x56ed29fe +0, 2, 2, 1, 16384, 0xd2eda9fe +0, 3, 3, 1, 16384, 0x4efc2a0d +0, 4, 4, 1, 16384, 0xcafcaa0d diff --git a/tests/ref/fate/filter-rgbashift-wrap-16 b/tests/ref/fate/filter-rgbashift-wrap-16 new file mode 100644 index 00000000000..94c08039ca1 --- /dev/null +++ b/tests/ref/fate/filter-rgbashift-wrap-16 @@ -0,0 +1,10 @@ +#tb 0: 1/5 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 64x64 +#sar 0: 1/1 +0, 0, 0, 1, 32768, 0xb5cba9ef +0, 1, 1, 1, 32768, 0xadda29fe +0, 2, 2, 1, 32768, 0xa5e9a9fe +0, 3, 3, 1, 32768, 0x9df82a0d +0, 4, 4, 1, 32768, 0x9607aa0d -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
