PR #24823 opened by Marcos Ashton (MarcosAsh) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24823 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24823.patch
Fixes wrong output in four Vulkan filters. I found them with a nightly job on my GPU that compares FFmpeg's hardware paths with the software ones. hflip_vulkan, vflip_vulkan and flip_vulkan mirrored to size - pos instead of size - 1 - pos, so the picture moved by one pixel and the first column or row was never written. transpose_vulkan mixed input and output coordinates. The default cclock_flip left part of the output unwritten when the width was larger than the height rounded up to 32, or the height larger than the width. clock fell through into clock_flip and came out vertically mirrored, and the other two were off by one. It now reads the source pixel for each output pixel, the same way the OpenCL filter does. dir values 4 to 7 were accepted but behaved as cclock_flip, so they now map to passthrough=landscape like in vf_transpose. avgblur_vulkan used the radius minus one, only normalised with the horizontal radius, and read pixels outside the frame as black. It now does what avgblur does: the documented radius, limited to half the chroma plane, with the edge pixels repeated. This changes existing output, since every radius now blurs one pixel more, which is what the documentation says it should do. gblur_vulkan had the same black border and gets the same edge clamp. I tested against the software filters on testsrc2 in yuv420p, yuv444p and their 10 bit versions, at 64x36, 67x41, 320x180 and 180x320. flip and all four transpose directions, plus dir=5 and dir=6, are bit-identical to hflip, vflip and transpose in all 126 cases, where before only 4 were. avgblur went from 7 to 32 dB PSNR against avgblur to 51 to 68 dB, and the rest is float vs integer rounding. gblur went from 24 to 32 dB against gblur to 34 to 43 dB, it's a different algorithm so they won't match exactly. I've only tested it on an RTX 5070 Ti with NVIDIA 595.91.07. >From ea460ece81477606077b2aeba98add36912d0e66 Mon Sep 17 00:00:00 2001 From: Marcos Ashton Iglesias <[email protected]> Date: Wed, 30 Sep 2026 19:37:41 +0100 Subject: [PATCH 1/5] avfilter/vulkan/flip: fix off-by-one in the mirrored position The mirror of pos is size - 1 - pos, not size - pos. The picture was shifted by one pixel and column (or row) 0 was never written. --- libavfilter/vulkan/flip.comp.glsl | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libavfilter/vulkan/flip.comp.glsl b/libavfilter/vulkan/flip.comp.glsl index a0d556ba8c..71cb8ea007 100644 --- a/libavfilter/vulkan/flip.comp.glsl +++ b/libavfilter/vulkan/flip.comp.glsl @@ -47,9 +47,9 @@ void main() ivec2 dst; switch (flip) { - case FLIP_HORIZONTAL: dst = ivec2(size.x - pos.x, pos.y); break; - case FLIP_VERTICAL: dst = ivec2(pos.x, size.y - pos.y); break; - case FLIP_BOTH: dst = ivec2(size.xy - pos.xy); break; + case FLIP_HORIZONTAL: dst = ivec2(size.x - 1 - pos.x, pos.y); break; + case FLIP_VERTICAL: dst = ivec2(pos.x, size.y - 1 - pos.y); break; + case FLIP_BOTH: dst = ivec2(size.xy - 1 - pos.xy); break; default: dst = pos; break; } -- 2.52.0 >From 1516f95cb20668d440aeaec46b3614f2c8c2b490 Mon Sep 17 00:00:00 2001 From: Marcos Ashton Iglesias <[email protected]> Date: Wed, 30 Sep 2026 19:37:41 +0100 Subject: [PATCH 2/5] avfilter/vulkan/transpose: fix all four directions The shader runs per output pixel but mixed input and output coordinates: cclock_flip left part of the output unwritten, clock came out mirrored and cclock/clock_flip were off by one. Read the source pixel for each output pixel instead, like the OpenCL filter. --- libavfilter/vulkan/transpose.comp.glsl | 27 ++++++-------------------- 1 file changed, 6 insertions(+), 21 deletions(-) diff --git a/libavfilter/vulkan/transpose.comp.glsl b/libavfilter/vulkan/transpose.comp.glsl index 6387b21cc8..40d9b18250 100644 --- a/libavfilter/vulkan/transpose.comp.glsl +++ b/libavfilter/vulkan/transpose.comp.glsl @@ -30,14 +30,6 @@ layout (local_size_x_id = 253, local_size_y_id = 254, local_size_z_id = 255) in; layout (set = 0, binding = 0) uniform readonly image2D input_img[]; layout (set = 0, binding = 1) uniform writeonly image2D output_img[]; -#define TRANSPOSE_CCLOCK_FLIP 0 -#define TRANSPOSE_CLOCK 1 -#define TRANSPOSE_CCLOCK 2 -#define TRANSPOSE_CLOCK_FLIP 3 -#define TRANSPOSE_REVERSAL 4 -#define TRANSPOSE_HFLIP 5 -#define TRANSPOSE_VFLIP 6 - layout (push_constant, scalar) uniform pushConstants { int dir; }; @@ -45,22 +37,15 @@ layout (push_constant, scalar) uniform pushConstants { void main() { ivec2 pos = ivec2(gl_GlobalInvocationID.xy); - if (dir == TRANSPOSE_CCLOCK || dir == TRANSPOSE_CLOCK || dir == TRANSPOSE_CLOCK_FLIP) - pos = pos.yx; - - ivec2 size = imageSize(input_img[nonuniformEXT(gl_LocalInvocationID.z)]); + ivec2 size = imageSize(output_img[nonuniformEXT(gl_LocalInvocationID.z)]); if (any(greaterThanEqual(pos, size))) return; - ivec2 dst; - switch (dir) { - case TRANSPOSE_CCLOCK: dst = ivec2(size.y - pos.y, pos.x); break; - case TRANSPOSE_CLOCK: pos = ivec2(pos.x, size.y - pos.y); /* fall */ - case TRANSPOSE_CLOCK_FLIP: dst = ivec2(size.yx - pos.yx); break; - default: dst = pos.yx; break; - } + /* size is the output size, so the input is size.yx */ + ivec2 src = ivec2((dir & 2) != 0 ? size.y - 1 - pos.y : pos.y, + (dir & 1) != 0 ? size.x - 1 - pos.x : pos.x); - vec4 res = imageLoad(input_img[nonuniformEXT(gl_LocalInvocationID.z)], pos); + vec4 res = imageLoad(input_img[nonuniformEXT(gl_LocalInvocationID.z)], src); - imageStore(output_img[nonuniformEXT(gl_LocalInvocationID.z)], dst, res); + imageStore(output_img[nonuniformEXT(gl_LocalInvocationID.z)], pos, res); } -- 2.52.0 >From 3abafc024b70015679c0256b476001db5c27013f Mon Sep 17 00:00:00 2001 From: Marcos Ashton Iglesias <[email protected]> Date: Wed, 30 Sep 2026 19:37:41 +0100 Subject: [PATCH 3/5] avfilter/vf_avgblur_vulkan: use the documented radius and edge handling sizeX/sizeY were used as radius - 1, the normalisation ignored the vertical radius and pixels outside the frame were read as black. Match avgblur: use the radii, limited to half the chroma plane, and repeat the edge pixels. Existing uses now blur one pixel more, as documented. --- libavfilter/vf_avgblur_vulkan.c | 12 ++++++++---- libavfilter/vulkan/avgblur.comp.glsl | 2 +- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/libavfilter/vf_avgblur_vulkan.c b/libavfilter/vf_avgblur_vulkan.c index be11a76070..c198720cb6 100644 --- a/libavfilter/vf_avgblur_vulkan.c +++ b/libavfilter/vf_avgblur_vulkan.c @@ -20,6 +20,7 @@ #include "libavutil/random_seed.h" #include "libavutil/opt.h" +#include "libavutil/pixdesc.h" #include "vulkan_filter.h" #include "filters.h" @@ -53,6 +54,7 @@ static av_cold int init_filter(AVFilterContext *ctx, AVFrame *in) AvgBlurVulkanContext *s = ctx->priv; FFVulkanContext *vkctx = &s->vkctx; const int planes = av_pix_fmt_count_planes(s->vkctx.output_format); + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(s->vkctx.output_format); s->qf = ff_vk_qf_find(vkctx, VK_QUEUE_COMPUTE_BIT, 0); if (!s->qf) { @@ -90,11 +92,13 @@ static av_cold int init_filter(AVFilterContext *ctx, AVFrame *in) RET(ff_vk_shader_register_exec(vkctx, &s->e, &s->shd)); s->initialized = 1; - s->opts.filter_len[0] = s->size_x - 1; - s->opts.filter_len[1] = s->size_y - 1; + /* like avgblur, at most half the chroma plane */ + s->opts.filter_len[0] = FFMIN(s->size_x, + AV_CEIL_RSHIFT(vkctx->output_width, desc->log2_chroma_w) / 2); + s->opts.filter_len[1] = FFMIN(s->size_y, + AV_CEIL_RSHIFT(vkctx->output_height, desc->log2_chroma_h) / 2); - s->opts.filter_norm[0] = s->opts.filter_len[0]*2 + 1; - s->opts.filter_norm[0] = 1.0/(s->opts.filter_norm[0]*s->opts.filter_norm[0]); + s->opts.filter_norm[0] = 1.0/((s->opts.filter_len[0]*2 + 1)*(s->opts.filter_len[1]*2 + 1)); s->opts.filter_norm[1] = s->opts.filter_norm[0]; s->opts.filter_norm[2] = s->opts.filter_norm[0]; s->opts.filter_norm[3] = s->opts.filter_norm[0]; diff --git a/libavfilter/vulkan/avgblur.comp.glsl b/libavfilter/vulkan/avgblur.comp.glsl index 4cfd4f433d..1a6f26ad07 100644 --- a/libavfilter/vulkan/avgblur.comp.glsl +++ b/libavfilter/vulkan/avgblur.comp.glsl @@ -54,7 +54,7 @@ void main() for (int y = -filter_len.y; y <= filter_len.y; y++) for (int x = -filter_len.x; x <= filter_len.x; x++) sum += imageLoad(input_img[nonuniformEXT(gl_LocalInvocationID.z)], - pos + ivec2(x, y)); + clamp(pos + ivec2(x, y), ivec2(0), size - 1)); imageStore(output_img[nonuniformEXT(gl_LocalInvocationID.z)], pos, sum * filter_norm); -- 2.52.0 >From cb2b5b6ea48aa347c8e827b730574e7c47040732 Mon Sep 17 00:00:00 2001 From: Marcos Ashton Iglesias <[email protected]> Date: Wed, 30 Sep 2026 19:37:41 +0100 Subject: [PATCH 4/5] avfilter/vulkan/gblur: repeat the edge pixels instead of reading black Taps outside the frame were read as 0, which darkened the borders. --- libavfilter/vulkan/gblur.comp.glsl | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavfilter/vulkan/gblur.comp.glsl b/libavfilter/vulkan/gblur.comp.glsl index 77c337020e..5eb95ae86b 100644 --- a/libavfilter/vulkan/gblur.comp.glsl +++ b/libavfilter/vulkan/gblur.comp.glsl @@ -54,8 +54,8 @@ void main() for(int i = 1; i < kernel.length(); i++) { ivec2 offs = gl_WorkGroupSize.x > gl_WorkGroupSize.y ? ivec2(i, 0) : ivec2(0, i); - sum += imageLoad(input_img[P_IDX], pos + offs) * kernel[i]; - sum += imageLoad(input_img[P_IDX], pos - offs) * kernel[i]; + sum += imageLoad(input_img[P_IDX], min(pos + offs, size - 1)) * kernel[i]; + sum += imageLoad(input_img[P_IDX], max(pos - offs, ivec2(0))) * kernel[i]; } imageStore(output_img[P_IDX], pos, sum); -- 2.52.0 >From 86bde5d3b14caed987858b8deb3c5d10e92d62ca Mon Sep 17 00:00:00 2001 From: Marcos Ashton Iglesias <[email protected]> Date: Wed, 30 Sep 2026 19:37:41 +0100 Subject: [PATCH 5/5] avfilter/vf_transpose_vulkan: handle the deprecated dir values 4-7 These silently behaved as cclock_flip. Map them like vf_transpose does. --- libavfilter/vf_transpose_vulkan.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/libavfilter/vf_transpose_vulkan.c b/libavfilter/vf_transpose_vulkan.c index 3e6b39d464..50c8b0f3f8 100644 --- a/libavfilter/vf_transpose_vulkan.c +++ b/libavfilter/vf_transpose_vulkan.c @@ -153,6 +153,13 @@ static int config_props_output(AVFilterLink *outlink) AVFilterLink *inlink = avctx->inputs[0]; FilterLink *inl = ff_filter_link(inlink); + if (s->dir & 4) { + av_log(avctx, AV_LOG_WARNING, + "dir values greater than 3 are deprecated, use the passthrough option instead\n"); + s->dir &= 3; + s->passthrough = TRANSPOSE_PT_TYPE_LANDSCAPE; + } + if ((inlink->w >= inlink->h && s->passthrough == TRANSPOSE_PT_TYPE_LANDSCAPE) || (inlink->w <= inlink->h && s->passthrough == TRANSPOSE_PT_TYPE_PORTRAIT)) { av_log(avctx, AV_LOG_VERBOSE, -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
