PR #25081 opened by Forgejo_Fairy
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25081
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25081.patch
RGB-to-NV12/YUV420P conversion fails with ENOTSUP when either dimension is
unchanged because scale_vulkan selects libswscale’s Vulkan backend, which
currently rejects subsampled formats. Keep these conversions on the existing
shader path. This fixes the routing regression introduced by 940e407c64a5.
Pixel validation also exposed swapped red and blue for BGR0/BGRA on the shader
path, including the resizing workaround in #24419. A separate commit accounts
for their storage in RGBA Vulkan images by swapping the corresponding
conversion-matrix coefficients.
Fixes #24419.
Reproducer:
ffmpeg -hide_banner -loglevel verbose -init_hw_device vulkan=vk:0
-filter_hw_device vk -f lavfi -i testsrc=size=128x96:rate=1 -vf
"format=bgr0,hwupload,setparams=colorspace=bt709:range=full,scale_vulkan=w=iw:h=ih:format=nv12:out_range=full,hwdownload,format=nv12"
-frames:v 1 -f null -
Validation on Linux x86-64 with an NVIDIA RTX 5090, driver 595.84, based on
master 0eeb79fcc5b2:
- Verified the routing regression by building and testing 940e407c64a5 and its
immediate parent e88d4ef71866.
- Passed 64 conversion cases: BGR0/RGBA × NV12/YUV420P × full/limited range ×
bilinear/nearest × unchanged dimensions, width-only change, height-only change,
and both dimensions changed. Input was 128×96; changed dimensions were reduced
by two.
- Equivalent BGR0 and RGBA inputs produce byte-identical output in every case.
- Existing RGBA resize results and BGR0-to-YUV444P/YUV444P-to-RGBA libswscale
controls remain byte-identical to unmodified master.
- Passed 18 five-frame solid-color comparisons covering BGR0/BGRA/RGBA,
NV12/YUV420P/YUV444P and both ranges; output agrees with software BT.709
conversion within one code value.
- Confirmed the original failure on release/9.0 and successful execution on
release/8.1.
- Build and git diff --check pass. Validation used direct GPU integration
tests; no full FATE run was performed.
From db9b883ea0fe49754a877f13c42d2e6bb7d88d2a Mon Sep 17 00:00:00 2001
From: Michael Niedermayer <[email protected]>
Date: Wed, 14 Oct 2026 18:20:55 +0000
Subject: [PATCH 1/2] avfilter/scale_vulkan: keep RGB to subsampled YUV on the
shader path
The Vulkan swscale backend rejects subsampled formats. Routing RGB to NV12 or
YUV420P through it when either dimension is unchanged makes conversions that
the filter already supports fail with ENOTSUP.
Keep these conversions on the existing shader path, including when neither
dimension changes. Leave other swscale conversions unchanged.
Fixes: 940e407c64a5161a3cc58430da7a2db0b420b013
Fixes: #24419
Assisted-by: Fairy
---
libavfilter/vf_scale_vulkan.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/libavfilter/vf_scale_vulkan.c b/libavfilter/vf_scale_vulkan.c
index c60f26acf5..06609b53cf 100644
--- a/libavfilter/vf_scale_vulkan.c
+++ b/libavfilter/vf_scale_vulkan.c
@@ -372,7 +372,11 @@ static int scale_vulkan_config_output(AVFilterLink
*outlink)
return AVERROR_PATCHWELCOME;
}
} else if ((inlink->w == outlink->w || inlink->h == outlink->h) &&
- (s->vkctx.input_format != s->vkctx.output_format)) {
+ (s->vkctx.input_format != s->vkctx.output_format) &&
+ /* swscale's Vulkan backend does not support subsampling yet. */
+ !(ff_vk_mt_is_np_rgb(s->vkctx.input_format) &&
+ (s->vkctx.output_format == AV_PIX_FMT_NV12 ||
+ s->vkctx.output_format == AV_PIX_FMT_YUV420P))) {
s->sws = sws_alloc_context();
if (!s->sws)
return AVERROR(ENOMEM);
--
2.52.0
From a3d6f4ddaaac9beb880cc0b99a9a3f886073b9f7 Mon Sep 17 00:00:00 2001
From: Michael Niedermayer <[email protected]>
Date: Wed, 14 Oct 2026 18:23:08 +0000
Subject: [PATCH 2/2] avfilter/scale_vulkan: account for BGR channel order in
RGB conversion
BGR0 and BGRA frames use RGBA Vulkan images with identity component mapping.
The conversion shader therefore reads B, G, R where its matrix expects R, G, B,
swapping red and blue in YUV output.
Swap the corresponding matrix coefficients for these input formats. This also
corrects the resizing workaround for #24419.
Assisted-by: Fairy
---
libavfilter/vf_scale_vulkan.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/libavfilter/vf_scale_vulkan.c b/libavfilter/vf_scale_vulkan.c
index 06609b53cf..37668d25bd 100644
--- a/libavfilter/vf_scale_vulkan.c
+++ b/libavfilter/vf_scale_vulkan.c
@@ -175,6 +175,12 @@ static av_cold int init_filter(AVFilterContext *ctx,
AVFrame *in)
for (int y = 0; y < 3; y++)
for (int x = 0; x < 3; x++)
s->opts.yuv_matrix[x][y] = tmp_mat[x][y];
+ /* These formats use RGBA images with B and R stored in reverse order.
*/
+ if (s->vkctx.input_format == AV_PIX_FMT_BGR0 ||
+ s->vkctx.input_format == AV_PIX_FMT_BGRA) {
+ for (int x = 0; x < 3; x++)
+ FFSWAP(float, s->opts.yuv_matrix[x][0],
s->opts.yuv_matrix[x][2]);
+ }
s->opts.yuv_matrix[3][3] = 1.0;
}
--
2.52.0
_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
