PR #23314 opened by yuyong05
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23314
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23314.patch
In yuv420_gbrp_ssse3, the boundary safeguard check "h_size * 3 >
FFABS(dstStride[0])" was erroneously set based on probably packed RGB24
formats (where each pixel spans 3 bytes per row).
For GBRP (planar GBR), each plane contains only 1 component per pixel
per row, meaning dstStride[0] corresponds exactly to width.
Multiplying h_size by 3 mistakenly triggers the condition for normal
widths, decreasing h_size by 8. This leaves the rightmost 8 pixels
of every row completely uninitialized (black).
Fix this by checking "h_size > FFABS(dstStride[0])" instead.
How to Reproduce the error:
1. Generate buggy and fixed outputs as PNGs using the 600x600 pipeline:
buggy output without the fix
$ ffmpeg -f lavfi -i color=c=red:size=600x600:rate=1 \
-vf format=yuv420p,format=gbrp \
-frames:v 1 -y buggy_red_600.png
fixed output with the fix
$ ffmpeg -f lavfi -i color=c=red:size=600x600:rate=1 \
-vf format=yuv420p,format=gbrp \
-frames:v 1 -y fixed_red_600.png
2. Verify buggy_red_600.png in an image viewer:
A strict, 8-pixel wide vertical black stripe (columns 592 to 599) is
clearly visible running top-to-bottom down the rightmost edge of the
image.
3. Verify fixed_red_600.png in an image viewer as well:
The output renders a perfect, uniform, fully solid red square across
the entire 600x600 canvas, indicating the boundary bug is successfully
resolved.
# Summary of changes
Briefly describe what this PR does and why.
<!--
If this PR requires new FATE test samples, attach them to the PR and
list their target paths below (relative to the fate-suite root).
Attached filenames must match the sample's filename:
```fate-samples
# e.g. vorbis/new-sample.ogg
```
-->
>From 56e25c1c5c17259a5379b401ab40702520bb6ddb Mon Sep 17 00:00:00 2001
From: Yong Yu <[email protected]>
Date: Tue, 2 Jun 2026 09:57:13 -0700
Subject: [PATCH] swscale/x86/yuv2rgb: fix planar GBRP boundary check in
ssse3 assembly
In yuv420_gbrp_ssse3, the boundary safeguard check "h_size * 3 >
FFABS(dstStride[0])" was erroneously set based on probably packed RGB24
formats (where each pixel spans 3 bytes per row).
For GBRP (planar GBR), each plane contains only 1 component per pixel
per row, meaning dstStride[0] corresponds exactly to width.
Multiplying h_size by 3 mistakenly triggers the condition for normal
widths, decreasing h_size by 8. This leaves the rightmost 8 pixels
of every row completely uninitialized (black).
Fix this by checking "h_size > FFABS(dstStride[0])" instead.
How to Reproduce the error:
1. Generate buggy and fixed outputs as PNGs using the 600x600 pipeline:
buggy output without the fix
$ ffmpeg -f lavfi -i color=c=red:size=600x600:rate=1 \
-vf format=yuv420p,format=gbrp \
-frames:v 1 -y buggy_red_600.png
fixed output with the fix
$ ffmpeg -f lavfi -i color=c=red:size=600x600:rate=1 \
-vf format=yuv420p,format=gbrp \
-frames:v 1 -y fixed_red_600.png
2. Verify buggy_red_600.png in an image viewer:
A strict, 8-pixel wide vertical black stripe (columns 592 to 599) is
clearly visible running top-to-bottom down the rightmost edge of the
image.
3. Verify fixed_red_600.png in an image viewer as well:
The output renders a perfect, uniform, fully solid red square across
the entire 600x600 canvas, indicating the boundary bug is successfully
resolved.
---
libswscale/x86/yuv2rgb.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libswscale/x86/yuv2rgb.c b/libswscale/x86/yuv2rgb.c
index a1463867a2..2291c84e53 100644
--- a/libswscale/x86/yuv2rgb.c
+++ b/libswscale/x86/yuv2rgb.c
@@ -216,7 +216,7 @@ static inline int yuv420_gbrp_ssse3(SwsInternal *c, const
uint8_t *const src[],
int y, h_size, vshift;
h_size = (c->opts.dst_w + 7) & ~7;
- if (h_size * 3 > FFABS(dstStride[0]))
+ if (h_size > FFABS(dstStride[0]))
h_size -= 8;
vshift = c->opts.src_format != AV_PIX_FMT_YUV422P;
--
2.52.0
_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
