PR #25707 opened by Kilian (lusoris) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25707 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25707.patch
This makes the Vulkan video decode and encode barriers keep a concrete queue family owner, as `ff_vk_frame_barrier()` does after #24970. The automated review of #24970 found the gap and called it pre-existing. Depends on #24970. The first commit here is that PR's commit; the change for review is the second commit, 72601c55d7. **Problem.** The decode output and reference barriers (`libavcodec/vulkan_decode.c`) and the encode source barrier (`libavcodec/vulkan_encode.c`) copy the frame's tracked queue family into `srcQueueFamilyIndex` and pass `VK_QUEUE_FAMILY_IGNORED` as the destination. When the image queues collapse to a single family, `hwcontext_vulkan` creates the frames with `VK_SHARING_MODE_EXCLUSIVE` and tracks that family as the owner. These barriers then request a transfer to `VK_QUEUE_FAMILY_IGNORED`, which an exclusive image does not allow (VUID-VkImageMemoryBarrier2-image-09118), and `ff_vk_exec_update_frame()` records `VK_QUEUE_FAMILY_IGNORED` as the owner. **Change.** - The rule from #24970 moves into `ff_vk_barrier_dst_qf()` in `libavutil/vulkan.h`, and `ff_vk_frame_barrier()` calls it with unchanged behaviour. - The three codec barriers use it for `dstQueueFamilyIndex`; their other image fields are unchanged. - These are the only barriers in libavcodec, libavfilter and libavutil whose source family is not `VK_QUEUE_FAMILY_IGNORED`. Frames tracked as `VK_QUEUE_FAMILY_IGNORED` (every device with several image queue families) and frames owned by the external or foreign family get the same barriers as before. **Testing, on 72601c55d7 against its parent 5fba16eedb, x86-64, GCC 16.2.1:** - The Vulkan objects and a minimal `ffmpeg` build without new warnings. - A host-side check of the helper over six source and six requested families gives the expected result for all cases. With the external-family guard removed, the check fails. - H.264 Vulkan decode of 8 frames with the Khronos validation layer, `-f framemd5`, on an NVIDIA RTX 4090 (proprietary driver) and an Intel Arc A380 (ANV): identical frame hashes before and after, and the same validation messages on both sides, none of them about queue family ownership. Both of those devices keep decode and encode in their own queue families, so the single-family configuration was not reached here; on these devices the change leaves every barrier as it was. >From 5fba16eedb5d0cbaf4f69cd51bcef02db938ead8 Mon Sep 17 00:00:00 2001 From: lusoris <[email protected]> Date: Fri, 9 Oct 2026 02:54:28 +0200 Subject: [PATCH 1/2] avutil/vulkan: don't release frames to VK_QUEUE_FAMILY_IGNORED On a device with a single queue family, hwcontext_vulkan creates frames with VK_SHARING_MODE_EXCLUSIVE and tracks them as owned by that family. ff_vk_frame_barrier() copies the tracked family into srcQueueFamilyIndex, while every filter and codec caller passes VK_QUEUE_FAMILY_IGNORED as the new family. The first barrier on such a frame therefore has unequal indices, which the specification treats as a queue family ownership transfer, and its destination is VK_QUEUE_FAMILY_IGNORED, which an exclusive image does not allow (VUID-VkImageMemoryBarrier2-image-09118). The Khronos validation layer accepts the barrier but records it as a release that is never acquired. It does not apply the layout transition and reports every later use of the image against the old layout: VUID-vkCmdDispatch-imageLayout-00344 and VUID-vkCmdDraw-None-09600 for shaders that access the image, and VUID-VkCopyMemoryToImageInfo-dstImageLayout-09059 and VUID-VkCopyImageToMemoryInfo-srcImageLayout-09064 for the host image copies in hwupload and hwdownload. When the caller passes VK_QUEUE_FAMILY_IGNORED and the frame is owned by a concrete queue family, keep that family as the destination. Equal indices perform no transfer, and the tracked owner stays the family that hwcontext_vulkan uses for its own transitions on such devices. Frames on devices with several queue families are tracked as VK_QUEUE_FAMILY_IGNORED and are unaffected, as are frames owned by the external or foreign queue family. Tested with Mesa lavapipe, which has one queue family, and validation layer 1.4.363: gblur_vulkan and scdet_vulkan between hwupload and hwdownload no longer report the four layout errors. Intel ANV, RADV and the NVIDIA proprietary driver report the same messages as before. The validation layer gap is reported as https://github.com/KhronosGroup/Vulkan-ValidationLayers/issues/13405. Signed-off-by: lusoris <[email protected]> --- libavutil/vulkan.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/libavutil/vulkan.c b/libavutil/vulkan.c index 6d5038a5848..361b17ef65c 100644 --- a/libavutil/vulkan.c +++ b/libavutil/vulkan.c @@ -2239,6 +2239,7 @@ void ff_vk_frame_barrier(FFVulkanContext *s, FFVkExecContext *e, uint32_t new_qf) { int found = -1; + uint32_t src_qf, dst_qf = new_qf; AVVkFrame *vkf = (AVVkFrame *)pic->data[0]; const int nb_images = ff_vk_count_images(vkf); for (int i = 0; i < e->nb_frame_deps; i++) @@ -2248,6 +2249,21 @@ void ff_vk_frame_barrier(FFVulkanContext *s, FFVkExecContext *e, break; } + src_qf = found >= 0 ? e->queue_family_dst[found] : vkf->queue_family[0]; + + /* Unequal queue family indices define a queue family ownership transfer, + * and an exclusive image can only be transferred to a queue family that + * exists (or to the external or foreign one), never to + * VK_QUEUE_FAMILY_IGNORED (VUID-VkImageMemoryBarrier2-image-09118). + * Frames on a device with a single queue family are tracked as owned by + * that family, so a caller that asks for no particular family keeps the + * current owner: equal indices mean no transfer. */ + if (new_qf == VK_QUEUE_FAMILY_IGNORED && + src_qf != VK_QUEUE_FAMILY_IGNORED && + src_qf != VK_QUEUE_FAMILY_EXTERNAL && + src_qf != VK_QUEUE_FAMILY_FOREIGN_EXT) + dst_qf = src_qf; + /* Earlier submissions are ordered by the frame's semaphore, which makes * their writes available and visible, so only accesses recorded in this * command buffer belong in the source scope. */ @@ -2261,8 +2277,8 @@ void ff_vk_frame_barrier(FFVulkanContext *s, FFVkExecContext *e, .dstAccessMask = new_access, .oldLayout = found >= 0 ? e->layout_dst[found] : vkf->layout[0], .newLayout = new_layout, - .srcQueueFamilyIndex = found >= 0 ? e->queue_family_dst[found] : vkf->queue_family[0], - .dstQueueFamilyIndex = new_qf, + .srcQueueFamilyIndex = src_qf, + .dstQueueFamilyIndex = dst_qf, .image = vkf->img[i], .subresourceRange = (VkImageSubresourceRange) { .aspectMask = VK_IMAGE_ASPECT_COLOR_BIT, -- 2.54.0 >From 72601c55d760f5334796d145083c0268daec0478 Mon Sep 17 00:00:00 2001 From: lusoris <[email protected]> Date: Tue, 3 Nov 2026 10:48:16 +0100 Subject: [PATCH 2/2] avcodec/vulkan: keep the queue family owner in video barriers The video decode output and reference barriers and the video encode source barrier copy the frame's tracked queue family into srcQueueFamilyIndex and pass VK_QUEUE_FAMILY_IGNORED as the destination. When the image queues collapse to a single family, hwcontext_vulkan creates the frames with VK_SHARING_MODE_EXCLUSIVE and tracks that family as their owner, so these barriers request a transfer to VK_QUEUE_FAMILY_IGNORED, which an exclusive image does not allow (VUID-VkImageMemoryBarrier2-image-09118), and ff_vk_exec_update_frame() then records VK_QUEUE_FAMILY_IGNORED as the owner. Move the rule that ff_vk_frame_barrier() already applies into ff_vk_barrier_dst_qf() and use it for these three barriers as well: a request for no particular family keeps a concrete owner. Frames tracked as VK_QUEUE_FAMILY_IGNORED, which covers every device with several image queue families, and frames owned by the external or foreign queue family get the same barriers as before. The other image fields of these barriers are unchanged. Signed-off-by: lusoris <[email protected]> --- libavcodec/vulkan_decode.c | 6 ++++-- libavcodec/vulkan_encode.c | 3 ++- libavutil/vulkan.c | 16 ++-------------- libavutil/vulkan.h | 23 +++++++++++++++++++++++ 4 files changed, 31 insertions(+), 17 deletions(-) diff --git a/libavcodec/vulkan_decode.c b/libavcodec/vulkan_decode.c index 5cf338b0d35..c5680229d41 100644 --- a/libavcodec/vulkan_decode.c +++ b/libavcodec/vulkan_decode.c @@ -536,7 +536,8 @@ int ff_vk_decode_frame(AVCodecContext *avctx, VK_IMAGE_LAYOUT_VIDEO_DECODE_DST_KHR : VK_IMAGE_LAYOUT_VIDEO_DECODE_DPB_KHR, /* Spec, 07252 utter madness */ .srcQueueFamilyIndex = vkf->queue_family[0], - .dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED, + .dstQueueFamilyIndex = ff_vk_barrier_dst_qf(vkf->queue_family[0], + VK_QUEUE_FAMILY_IGNORED), .image = vkf->img[0], .subresourceRange = (VkImageSubresourceRange) { .aspectMask = vp->view.aspect, @@ -609,7 +610,8 @@ int ff_vk_decode_frame(AVCodecContext *avctx, .oldLayout = rvkf->layout[0], .newLayout = VK_IMAGE_LAYOUT_VIDEO_DECODE_DPB_KHR, .srcQueueFamilyIndex = rvkf->queue_family[0], - .dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED, + .dstQueueFamilyIndex = ff_vk_barrier_dst_qf(rvkf->queue_family[0], + VK_QUEUE_FAMILY_IGNORED), .image = rvkf->img[0], .subresourceRange = (VkImageSubresourceRange) { .aspectMask = rvp->view.aspect_ref, diff --git a/libavcodec/vulkan_encode.c b/libavcodec/vulkan_encode.c index 9d3c9b04c57..8a96c77c766 100644 --- a/libavcodec/vulkan_encode.c +++ b/libavcodec/vulkan_encode.c @@ -377,7 +377,8 @@ static int vulkan_encode_issue(AVCodecContext *avctx, .oldLayout = vkf->layout[0], .newLayout = VK_IMAGE_LAYOUT_VIDEO_ENCODE_SRC_KHR, .srcQueueFamilyIndex = vkf->queue_family[0], - .dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED, + .dstQueueFamilyIndex = ff_vk_barrier_dst_qf(vkf->queue_family[0], + VK_QUEUE_FAMILY_IGNORED), .image = vkf->img[0], .subresourceRange = (VkImageSubresourceRange) { .aspectMask = vp->in.aspect, diff --git a/libavutil/vulkan.c b/libavutil/vulkan.c index 361b17ef65c..deec3c96f43 100644 --- a/libavutil/vulkan.c +++ b/libavutil/vulkan.c @@ -2239,7 +2239,7 @@ void ff_vk_frame_barrier(FFVulkanContext *s, FFVkExecContext *e, uint32_t new_qf) { int found = -1; - uint32_t src_qf, dst_qf = new_qf; + uint32_t src_qf, dst_qf; AVVkFrame *vkf = (AVVkFrame *)pic->data[0]; const int nb_images = ff_vk_count_images(vkf); for (int i = 0; i < e->nb_frame_deps; i++) @@ -2250,19 +2250,7 @@ void ff_vk_frame_barrier(FFVulkanContext *s, FFVkExecContext *e, } src_qf = found >= 0 ? e->queue_family_dst[found] : vkf->queue_family[0]; - - /* Unequal queue family indices define a queue family ownership transfer, - * and an exclusive image can only be transferred to a queue family that - * exists (or to the external or foreign one), never to - * VK_QUEUE_FAMILY_IGNORED (VUID-VkImageMemoryBarrier2-image-09118). - * Frames on a device with a single queue family are tracked as owned by - * that family, so a caller that asks for no particular family keeps the - * current owner: equal indices mean no transfer. */ - if (new_qf == VK_QUEUE_FAMILY_IGNORED && - src_qf != VK_QUEUE_FAMILY_IGNORED && - src_qf != VK_QUEUE_FAMILY_EXTERNAL && - src_qf != VK_QUEUE_FAMILY_FOREIGN_EXT) - dst_qf = src_qf; + dst_qf = ff_vk_barrier_dst_qf(src_qf, new_qf); /* Earlier submissions are ordered by the frame's semaphore, which makes * their writes available and visible, so only accesses recorded in this diff --git a/libavutil/vulkan.h b/libavutil/vulkan.h index 1b92e6a4b2e..f1c908c1445 100644 --- a/libavutil/vulkan.h +++ b/libavutil/vulkan.h @@ -363,6 +363,29 @@ static inline int ff_vk_count_images(AVVkFrame *f) return cnt; } +/** + * Return the destination queue family for a barrier on an image currently + * owned by src_qf, when the caller asked for new_qf. + * + * Unequal queue family indices define a queue family ownership transfer, + * and an exclusive image can only be transferred to a queue family that + * exists (or to the external or foreign one), never to + * VK_QUEUE_FAMILY_IGNORED (VUID-VkImageMemoryBarrier2-image-09118). + * Frames on a device with a single queue family are tracked as owned by + * that family, so a caller that asks for no particular family keeps the + * current owner: equal indices mean no transfer. + */ +static inline uint32_t ff_vk_barrier_dst_qf(uint32_t src_qf, uint32_t new_qf) +{ + if (new_qf == VK_QUEUE_FAMILY_IGNORED && + src_qf != VK_QUEUE_FAMILY_IGNORED && + src_qf != VK_QUEUE_FAMILY_EXTERNAL && + src_qf != VK_QUEUE_FAMILY_FOREIGN_EXT) + return src_qf; + + return new_qf; +} + static inline const void *ff_vk_find_struct(const void *chain, VkStructureType stype) { const VkBaseInStructure *in = chain; -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
