PR #25708 opened by luoye-cpu
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25708
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25708.patch
Two correctness fixes for the CUDA hardware-frame path of the libvmaf
filters.
**Dependency.** The series is based on #24973 ("fix error paths") — its
fourth commit adds the `cuCtxPopCurrent()` call in the loop this series
edits, so the commits above are stacked on that branch and the two series
touch adjacent lines of the same loop. The branch is self-contained: the
four dependency commits are included, so CI can build it before #24973
merges; once #24973 merges this series will be rebased so the PR diff shows
only the two commits below.
## Commit 1 — accept more planar pixel formats
`supported_formats[]` of the `libvmaf_cuda` filter only listed
`AV_PIX_FMT_YUV420P` and `AV_PIX_FMT_YUV444P16`, so 10/12-bit and 4:2:2 CUDA
frames were rejected with `Unsupported input format: yuv420p10le` before the
filter could look at them. The missing 8/10/12-bit planar variants match
what the software path already maps in `pix_fmt_map()`.
The 16-bit 4:2:0/4:2:2 combinations are intentionally not added: they have
not been exercised on the CUDA path yet — this is a verification gap, not a
known backend limitation. Note that `AV_PIX_FMT_YUV444P16` (the pre-existing
entry) is a macro alias of `AV_PIX_FMT_YUV444P16LE` on little-endian
platforms (`AV_PIX_FMT_NE(YUV444P16BE, YUV444P16LE)` in `libavutil/pixfmt.h`),
so it is the same enum value as the LE variant style used by the new
entries. Semi-planar formats (nv12/p010, as produced by NVDEC) remain
unsupported and require `scale_cuda=format=<planar>` first — a format
requirement, not a copy cost.
## Commit 2 — copy all planes of CUDA hardware frames
`copy_picture_data_cuda()` loops over `pix_desc->nb_components` but breaks
after the first `cuMemcpy2D()`, so only the luma plane is copied into the
`VmafPicture`; chroma planes are left as zeroed pool memory and chroma-based
features read zeros. With `feature=name=speed_chroma` the CUDA path reported
a pooled `speed_chroma_uv` of 0 where the CPU path reports about 19.65,
measured against upstream libvmaf.
The accepted formats are fully planar, with one component per plane.
Therefore, iterating over `nb_components` copies all planes for the formats
supported by this filter. The error path of the same loop (context pop on
`cuMemcpy2D()` failure) is #24973's fourth commit.
## Verification
- Both commits compile with `CONFIG_LIBVMAF_CUDA_FILTER=yes` against
upstream libvmaf 3.2.0 headers, on current master (rebased on top of
#24973; the drift from the original base to today's master — 700 commits —
touches none of this file's areas).
- **10-bit 4:2:0 CUDA run** (`hwupload_cuda` → `libvmaf_cuda`, 1080p, 30
frames) succeeds after commit 1, and **the CPU and CUDA paths produced
identical pooled results for the tested 10-bit input, including non-zero
chroma feature values** (`speed_chroma_uv` 36.609951 / 33.162926 on the
first two frames, bit-identical on both paths). This exercises commit 1's
whitelist, commit 2's chroma copy, and the two-bytes-per-sample width
arithmetic together.
- The same whitelist/copy combination is additionally covered by a 13-case
matrix against an libvmaf-3.2.0-based CUDA build recorded earlier
(hwupload 420/422/444 × 8/10-bit incl. 4K/480p/non-aligned 322×182, NVDEC
H.264/HEVC; CPU/GPU `maxdiff=0`; NVDEC-vs-software-decode SHA-256
identity) — the 10-bit run above re-verifies the whitelist entry of that
matrix on current master.
- 12-bit is whitelisted but not exercised at the FFmpeg level (`scale_cuda`
has no planar 12-bit output); covered at the CLI/API level.
- The error path of the same loop (context pop on `cuMemcpy2D()` failure) is
handled by #24973; a residual quirk — `do_vmaf_cuda()` maps the copy error
to `AVERROR(ENOMEM)` — predates both series and is probably better folded
into #24973's error-path scope.
From b1e0e7a72fa667b9978946ad14c0de51a4ac555e Mon Sep 17 00:00:00 2001
From: lusoris <[email protected]>
Date: Fri, 9 Oct 2026 18:15:48 +0200
Subject: [PATCH 1/6] avfilter/vf_libvmaf: free the distorted frame on error
The distorted frame returned by ff_framesync_dualinput_get() is owned
by the caller. do_vmaf() and do_vmaf_cuda() returned without freeing it
when copying a picture or vmaf_read_pictures() failed, leaking the
frame and keeping its buffer pool alive.
Signed-off-by: lusoris <[email protected]>
---
libavfilter/vf_libvmaf.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/libavfilter/vf_libvmaf.c b/libavfilter/vf_libvmaf.c
index 851b0982f02..2564736354a 100644
--- a/libavfilter/vf_libvmaf.c
+++ b/libavfilter/vf_libvmaf.c
@@ -237,6 +237,7 @@ static int do_vmaf(FFFrameSync *fs)
err = get_picture(ref, &pic_ref, s->bpc);
if (err) {
av_log(ctx, AV_LOG_ERROR, "problem during vmaf_picture_alloc.\n");
+ av_frame_free(&dist);
return AVERROR(ENOMEM);
}
@@ -244,12 +245,14 @@ static int do_vmaf(FFFrameSync *fs)
if (err) {
av_log(ctx, AV_LOG_ERROR, "problem during vmaf_picture_alloc.\n");
vmaf_picture_unref(&pic_ref);
+ av_frame_free(&dist);
return AVERROR(ENOMEM);
}
err = vmaf_read_pictures(s->vmaf, &pic_ref, &pic_dist, s->frame_cnt++);
if (err) {
av_log(ctx, AV_LOG_ERROR, "problem during vmaf_read_pictures.\n");
+ av_frame_free(&dist);
return AVERROR(EINVAL);
}
@@ -865,6 +868,7 @@ static int do_vmaf_cuda(FFFrameSync* fs)
frames_ctx->sw_format);
if (err) {
av_log(ctx, AV_LOG_ERROR, "problem during copy_picture_data_cuda.\n");
+ av_frame_free(&dist);
return AVERROR(ENOMEM);
}
@@ -872,12 +876,14 @@ static int do_vmaf_cuda(FFFrameSync* fs)
frames_ctx->sw_format);
if (err) {
av_log(ctx, AV_LOG_ERROR, "problem during copy_picture_data_cuda.\n");
+ av_frame_free(&dist);
return AVERROR(ENOMEM);
}
err = vmaf_read_pictures(s->vmaf, &pic_ref, &pic_dist, s->frame_cnt++);
if (err) {
av_log(ctx, AV_LOG_ERROR, "problem during vmaf_read_pictures.\n");
+ av_frame_free(&dist);
return AVERROR(EINVAL);
}
--
2.54.0
From d9661a5c4910e98c89191cedce705b97e7924104 Mon Sep 17 00:00:00 2001
From: lusoris <[email protected]>
Date: Fri, 9 Oct 2026 18:15:57 +0200
Subject: [PATCH 2/6] avfilter/vf_libvmaf: count a frame only after libvmaf
accepted it
frame_cnt was incremented inside the vmaf_read_pictures() call, so a
frame libvmaf rejected was still counted. uninit() then asked for the
pooled score over [0, frame_cnt - 1], which includes the index that has
no features, and pooling failed for every model even though all
earlier frames were scored.
Increment the counter only when vmaf_read_pictures() succeeds.
Signed-off-by: lusoris <[email protected]>
---
libavfilter/vf_libvmaf.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/libavfilter/vf_libvmaf.c b/libavfilter/vf_libvmaf.c
index 2564736354a..ae9b690377a 100644
--- a/libavfilter/vf_libvmaf.c
+++ b/libavfilter/vf_libvmaf.c
@@ -249,12 +249,13 @@ static int do_vmaf(FFFrameSync *fs)
return AVERROR(ENOMEM);
}
- err = vmaf_read_pictures(s->vmaf, &pic_ref, &pic_dist, s->frame_cnt++);
+ err = vmaf_read_pictures(s->vmaf, &pic_ref, &pic_dist, s->frame_cnt);
if (err) {
av_log(ctx, AV_LOG_ERROR, "problem during vmaf_read_pictures.\n");
av_frame_free(&dist);
return AVERROR(EINVAL);
}
+ s->frame_cnt++;
return ff_filter_frame(ctx->outputs[0], dist);
}
@@ -880,12 +881,13 @@ static int do_vmaf_cuda(FFFrameSync* fs)
return AVERROR(ENOMEM);
}
- err = vmaf_read_pictures(s->vmaf, &pic_ref, &pic_dist, s->frame_cnt++);
+ err = vmaf_read_pictures(s->vmaf, &pic_ref, &pic_dist, s->frame_cnt);
if (err) {
av_log(ctx, AV_LOG_ERROR, "problem during vmaf_read_pictures.\n");
av_frame_free(&dist);
return AVERROR(EINVAL);
}
+ s->frame_cnt++;
return ff_filter_frame(ctx->outputs[0], dist);
}
--
2.54.0
From 107e0d4f66cf0888ebb9ef136556a93ebeaeed1b Mon Sep 17 00:00:00 2001
From: lusoris <[email protected]>
Date: Fri, 9 Oct 2026 18:16:06 +0200
Subject: [PATCH 3/6] avfilter/vf_libvmaf: do not print a score when pooling
fails
When vmaf_score_pooled() failed, uninit() logged the error and then
printed "VMAF score:" with vmaf_score, which the failed call never
wrote. The line reports an uninitialized stack value as a result.
Skip the score line for a model whose pooled score could not be
computed.
Signed-off-by: lusoris <[email protected]>
---
libavfilter/vf_libvmaf.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/libavfilter/vf_libvmaf.c b/libavfilter/vf_libvmaf.c
index ae9b690377a..d69772b6c32 100644
--- a/libavfilter/vf_libvmaf.c
+++ b/libavfilter/vf_libvmaf.c
@@ -661,6 +661,7 @@ static av_cold void uninit(AVFilterContext *ctx)
if (err) {
av_log(ctx, AV_LOG_ERROR,
"problem getting pooled vmaf score.\n");
+ continue;
}
av_log(ctx, AV_LOG_INFO, "VMAF score: %f\n", vmaf_score);
--
2.54.0
From d3810053e652d60e433d5df230c3c04ae7c7ea47 Mon Sep 17 00:00:00 2001
From: lusoris <[email protected]>
Date: Fri, 9 Oct 2026 18:16:15 +0200
Subject: [PATCH 4/6] avfilter/vf_libvmaf: pop the CUDA context when cuMemcpy2D
fails
copy_picture_data_cuda() pushes the device context before the copy and
returned without popping it when cuMemcpy2D() failed, leaving the
context current on the filter thread.
Signed-off-by: lusoris <[email protected]>
---
libavfilter/vf_libvmaf.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/libavfilter/vf_libvmaf.c b/libavfilter/vf_libvmaf.c
index d69772b6c32..ca4c511509e 100644
--- a/libavfilter/vf_libvmaf.c
+++ b/libavfilter/vf_libvmaf.c
@@ -835,8 +835,10 @@ static int copy_picture_data_cuda(VmafContext* vmaf,
m.Height = dst->h[i];
err = cu->cuMemcpy2D(&m);
- if (err)
+ if (err) {
+ cu->cuCtxPopCurrent(NULL);
return AVERROR_EXTERNAL;
+ }
break;
}
--
2.54.0
From e8785e8fceb5ac06c183b31da3addc36c4265c8f Mon Sep 17 00:00:00 2001
From: luoye-cpu <[email protected]>
Date: Mon, 19 Oct 2026 18:55:54 +0800
Subject: [PATCH 5/6] avfilter/vf_libvmaf: accept more planar pixel formats for
CUDA frames
supported_formats[] decides which sw_format of CUDA hardware frames the
libvmaf_cuda filter accepts. Only AV_PIX_FMT_YUV420P and AV_PIX_FMT_YUV444P16
were listed, so uploading 10/12-bit or 4:2:2 frames with hwupload_cuda failed
with "Unsupported input format: yuv420p10le".
List the missing 8/10/12-bit planar variants, matching the pixel formats the
software path already maps in pix_fmt_map(). The 16-bit 4:2:0/4:2:2
combinations are intentionally not added: the mapping exists, but they have
not been exercised on the CUDA path. Semi-planar formats (nv12/p010, as
produced by NVDEC) remain unsupported and require scale_cuda=format=<planar>
before the filter.
Signed-off-by: luoye-cpu <[email protected]>
---
libavfilter/vf_libvmaf.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/libavfilter/vf_libvmaf.c b/libavfilter/vf_libvmaf.c
index ca4c511509e..10ca7b0a502 100644
--- a/libavfilter/vf_libvmaf.c
+++ b/libavfilter/vf_libvmaf.c
@@ -722,6 +722,14 @@ const FFFilter ff_vf_libvmaf = {
#if CONFIG_LIBVMAF_CUDA_FILTER
static const enum AVPixelFormat supported_formats[] = {
AV_PIX_FMT_YUV420P,
+ AV_PIX_FMT_YUV422P,
+ AV_PIX_FMT_YUV444P,
+ AV_PIX_FMT_YUV420P10LE,
+ AV_PIX_FMT_YUV422P10LE,
+ AV_PIX_FMT_YUV444P10LE,
+ AV_PIX_FMT_YUV420P12LE,
+ AV_PIX_FMT_YUV422P12LE,
+ AV_PIX_FMT_YUV444P12LE,
AV_PIX_FMT_YUV444P16,
};
--
2.54.0
From cfa0bbc4bb86bb483b42a1e173516c20e2008c3e Mon Sep 17 00:00:00 2001
From: luoye-cpu <[email protected]>
Date: Tue, 3 Nov 2026 18:30:05 +0800
Subject: [PATCH 6/6] avfilter/vf_libvmaf: copy all planes of CUDA hardware
frames
copy_picture_data_cuda() loops over the planes of the source pixel format
but breaks out after the first cuMemcpy2D(), so only the luma plane is
copied into the VmafPicture. Chroma-based features then read zeroed
memory: with feature=name=speed_chroma the pooled speed_chroma_uv score
is 0 on the CUDA path while the CPU path reports about 19.65 for the same
input, measured against upstream libvmaf.
The accepted formats are fully planar, with one component per plane.
Therefore, iterating over nb_components copies all planes for the formats
supported by this filter.
Depends on the error-path series ("avfilter/vf_libvmaf: fix error paths",
#24973), whose fourth commit adds the cuCtxPopCurrent() call that this
hunk context builds on; the two series touch adjacent lines of the same
loop.
Signed-off-by: luoye-cpu <[email protected]>
---
libavfilter/vf_libvmaf.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/libavfilter/vf_libvmaf.c b/libavfilter/vf_libvmaf.c
index 10ca7b0a502..fbd48e71caf 100644
--- a/libavfilter/vf_libvmaf.c
+++ b/libavfilter/vf_libvmaf.c
@@ -847,7 +847,6 @@ static int copy_picture_data_cuda(VmafContext* vmaf,
cu->cuCtxPopCurrent(NULL);
return AVERROR_EXTERNAL;
}
- break;
}
err = cu->cuCtxPopCurrent(NULL);
--
2.54.0
_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
