PR #25109 opened by Kilian (lusoris) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25109 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25109.patch
Applications importing P010 or P012 DRM frames into VAAPI fail with "DRM format not supported by VAAPI" when the chroma layer uses DRM_FORMAT_GR1616. Vulkan exports VK_FORMAT_R16G16_UNORM this way, but the VAAPI import table only accepts RG1616 for these formats. vulkan_fmt_to_drm() takes the first matching entry, which is GR1616 before RG1616. GR1616 puts R in the low 16 bits, matching the memory order of the Vulkan plane. The VAAPI table already accepts both GR88 and RG88 for NV12. What the patch does: - accepts R16 plus GR1616 layers for P010 and P012 in `libavutil/hwcontext_vaapi.c`, alongside the existing RG1616 entries Testing, Intel Arc A380 (iHD 26.3.5) and AMD Radeon 610M (radeonsi), on n9.0.2: - in a build that also waited on the CPU before the map, Vulkan P010 frames map to VAAPI, read back bit-exact and encode with hevc_vaapi Main10; without this change the map is refused on both GPUs - with this change alone, a small libavutil test program uploads VAAPI frames, maps them to DRM PRIME and back to VAAPI (both drivers export R16 plus GR1616 layers), downloads them and compares each row: P010 and P012 match bit-exact for all 10 frames on both GPUs - the distribution's n9.0.2 control build rejects the same imports with EINVAL (-22); its distribution patch state was not verified - on master af25036ac0 (GCC 16.2.1, `--enable-gpl --enable-vulkan --enable-vaapi --enable-libdrm --enable-nvenc`): the commit applies cleanly, `libavutil/hwcontext_vaapi.c` and `libavutil/hwcontext_vulkan.c` build without warnings, and FATE (3106 targets) fails the same 17 tests as unpatched master, all deflate-output tests (zlib, flashsv, mpng, png, apng, pdv) whose decoded output matches and only the compressed bytes differ, from the host zlib-ng; the GPU checks were run on n9.0.2, not rerun on master For review: accepting GR1616 alongside RG1616 for both P010 and P012, as already done for the two chroma orderings of NV12. >From 1904d5a6173cc7aefe9e3a6f03f2894a6b783b2f Mon Sep 17 00:00:00 2001 From: lusoris <[email protected]> Date: Sat, 10 Oct 2026 08:30:55 +0200 Subject: [PATCH] avutil/hwcontext_vaapi: accept GR1616 for P010/P012 DRM import hwcontext_vulkan exports the chroma plane of P010 and P012 frames, VK_FORMAT_R16G16_UNORM, as DRM_FORMAT_GR1616: vulkan_fmt_to_drm() returns the first match in vulkan_drm_format_map, which lists GR1616 before RG1616. The VAAPI DRM import table only knows RG1616 for these formats, so mapping a 10- or 12-bit Vulkan frame to VAAPI fails with "DRM format not supported by VAAPI". GR1616 carries R in the low 16 bits, which is the memory order of a Vulkan R16G16 plane. Accept it next to RG1616, as the table already does with GR88 and RG88 for NV12, which Vulkan exports as GR88. Tested on an Intel Arc A380 (iHD 26.3.5) and an AMD Radeon 610M (radeonsi), in a build that also waited on the CPU before the map instead of attaching a sync file: Vulkan P010 frames map to VAAPI, read back bit-exact and encode with hevc_vaapi Main10. Without this change the map is refused on both. With this change alone, P010 and P012 frames also survive a VAAPI to DRM to VAAPI round trip bit-exact on both GPUs (10 frames each, both drivers export R16 plus GR1616 layers), which stock n9.0.2 refuses with EINVAL. Signed-off-by: lusoris <[email protected]> --- libavutil/hwcontext_vaapi.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index b04bf243ca..2592351148 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -1065,9 +1065,11 @@ static const struct { DRM_MAP(NV12, 1, DRM_FORMAT_NV12), #if defined(VA_FOURCC_P010) && defined(DRM_FORMAT_R16) DRM_MAP(P010, 2, DRM_FORMAT_R16, DRM_FORMAT_RG1616), + DRM_MAP(P010, 2, DRM_FORMAT_R16, DRM_FORMAT_GR1616), #endif #if defined(VA_FOURCC_P012) && defined(DRM_FORMAT_R16) DRM_MAP(P012, 2, DRM_FORMAT_R16, DRM_FORMAT_RG1616), + DRM_MAP(P012, 2, DRM_FORMAT_R16, DRM_FORMAT_GR1616), #endif DRM_MAP(BGRA, 1, DRM_FORMAT_ARGB8888), DRM_MAP(BGRX, 1, DRM_FORMAT_XRGB8888), -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
