PR #24743 opened by Forgejo_Fairy URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24743 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24743.patch
Firefox Vulkan direct export is reported to corrupt video with FFmpeg 8.1.2, while cherry-picking 25e187f84949 fixes playback on the reporter's system. The relevant changes are still absent from release/8.1 and n8.1.3. This series proposes the requested backport for a subsequent 8.1 maintenance release. Addresses #23932. The series contains three unchanged upstream patches: - 25e187f84949 (#22566): correct DRM plane descriptions, synchronization, and modifier-aware frame/decode setup. - f7ca6f748163 (#22785): correct the qualifier warning and misleading modifier logging introduced by that change. - ab7b6ef0a2c5 (#22795): correct DRM-map failure cleanup and restore an unused-variable annotation. All three cherry-pick without conflicts onto release/8.1 at 45e8e0a3ff8c. Each backport's stable patch ID matches upstream. Original authorship is preserved; no public headers are changed. Validation: - Built on Linux AArch64 and x86-64 with GCC 13.3 and Vulkan headers 1.4.350, with libdrm and H.264/HEVC/AV1 Vulkan decoding enabled. - `make -j8 fate-libavutil`: all 41 enabled tests passed on each architecture. - `git diff --check` passed. - RTX 5090, NVIDIA driver 595.84: all 1,205 frames of the [Mozilla sample](https://bugzilla.mozilla.org/attachment.cgi?id=9598741) produced identical frame hashes through Vulkan decode/download and software decoding, both outputting NV12. Sample SHA-256: `db04b5ef9804e8a8e93a5b7b7b8fb8503ac59a6e6c17f38a2c2161ec79ceefa1`. The x86-64 build configuration was: ./configure --disable-everything --disable-doc --disable-autodetect --enable-ffmpeg --enable-ffprobe --enable-vulkan --enable-libdrm --extra-cflags=-I/path/to/Vulkan-Headers-1.4.350/include --enable-decoder=h264,hevc,av1 --enable-hwaccel=h264_vulkan,hevc_vulkan,av1_vulkan --enable-parser=h264,hevc,av1 --enable-demuxer=mov,matroska --enable-protocol=file --enable-muxer=null,framemd5 --enable-encoder=wrapped_avframe,rawvideo --enable-filter=hwdownload,hwupload,hwmap,format,scale,testsrc2 --enable-indev=lavfi The sample comparison was: ./ffmpeg -v warning -threads 1 -hwaccel vulkan -hwaccel_output_format vulkan -i bug.mp4 -an -vf hwdownload,format=nv12 -f framemd5 -y vulkan.md5 ./ffmpeg -v warning -i bug.mp4 -an -vf format=nv12 -f framemd5 -y software.md5 cmp software.md5 vulkan.md5 These are build, utility-test, and decode/download checks. They do not validate Firefox's direct-export path or the QCOM-specific cleanup case. The container has no Firefox/display session, and the GPU/driver differs from the reported configurations. A Firefox retest of the complete candidate has been requested in #23932; the original commit's successful browser test is documented there and in [Mozilla bug 2048591, comment 6](https://bugzilla.mozilla.org/show_bug.cgi?id=2048591#c6). >From cee26cb74ab124ddbd07df8c34856fbb33f9d538 Mon Sep 17 00:00:00 2001 From: Tymur Boiko <[email protected]> Date: Tue, 27 Jan 2026 18:16:33 +0100 Subject: [PATCH 1/3] vulkan: fix DRM map, decode barriers, and video frame setup for modifier output When mapping Vulkan Video frames to DMA-BUF, synchronize using an exportable binary semaphore and sync_fd where supported. Submit a lightweight exec that waits on each plane's timeline semaphore at the current value, signals a SYNC_FD-exportable binary semaphore, then export with vkGetSemaphoreFdKHR. Store that binary semaphore in AVVkFrameInternal and reuse it across maps instead of creating and destroying each time: for VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_SYNC_FD_BIT, copy transference means a successful vkGetSemaphoreFdKHR unsignals the semaphore like a wait, so it can be signaled again on the next map submit. If export is unavailable, fall back to vkWaitSemaphores. Moved drm_sync_sem destroy to vulkan_free_internal Export dma-buf fds with GetMemoryFdKHR for each populated f->mem[i], iterating up to the sw_format plane count instead of stopping at the image count, so multi-memory bindings are not skipped. Describe DRM layers using max(sw planes, image count) and query subresource layout with the correct aspect and image index when one VkImage backs multiple planes. Reference the source hw_frames_ctx on the mapped frame and close dma-buf fds on failure paths. For DMA-BUF-capable pools, honor VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT from format export queries when binding memory. With DRM modifiers and a video profile in create_pnext, preserve caller usage and image flags instead of overwriting them from generic supported_usage probing; use the modifier list create info when probing export flags for modifier tiling. Include VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR from the output frames context's usage together with DST (fixes VUID-VkVideoBeginCodingInfoKHR-slotIndex-07245) instead of adding DPB usage only when !is_current. In ff_vk_decode_add_slice, pass VkVideoProfileListInfoKHR (from the output frames context's create_pnext) as the pNext argument to ff_vk_get_pooled_buffer instead of the full create_pnext chain. In ff_vk_frame_params, set tiling to OPTIMAL only when it is not already DRM_FORMAT_MODIFIER_EXT. In ff_vk_decode_init, when the output pool's create_pnext includes VkImageDrmFormatModifierListCreateInfoEXT, initialize the DPB pool with that modifier-list pNext and DRM_FORMAT_MODIFIER_EXT tiling; otherwise use VkVideoProfileListInfoKHR and OPTIMAL as before. When VK_VIDEO_DECODE_CAPABILITY_DPB_AND_OUTPUT_DISTINCT_BIT_KHR is unset, the output and DPB pools cannot use different layouts or tiling, so the DPB pool must match the output pool. Also fix av_hwframe_map ioctl sync_fd export, multi-planar semaphore handling, and related failure-path cleanup. Signed-off-by: Tymur Boiko <[email protected]> (cherry picked from commit 25e187f8494966377a4b9d077260ce7b501a911c) Assisted-by: Fairy --- libavcodec/vulkan_decode.c | 36 +++++-- libavutil/hwcontext_vulkan.c | 194 ++++++++++++++++++++++++++++++----- 2 files changed, 201 insertions(+), 29 deletions(-) diff --git a/libavcodec/vulkan_decode.c b/libavcodec/vulkan_decode.c index 9ab8d45aa9..afca62b304 100644 --- a/libavcodec/vulkan_decode.c +++ b/libavcodec/vulkan_decode.c @@ -233,7 +233,8 @@ int ff_vk_decode_prepare_frame(FFVulkanDecodeContext *dec, AVFrame *pic, (AVVkFrame *)pic->data[0], hwfc->format[0], VK_IMAGE_USAGE_VIDEO_DECODE_DST_BIT_KHR | - (!is_current ? VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR : 0)); + (hwfc->usage & VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR)); + // the above fixes VUID-VkVideoBeginCodingInfoKHR-slotIndex-07245 if (err < 0) return err; @@ -335,12 +336,21 @@ int ff_vk_decode_add_slice(AVCodecContext *avctx, FFVulkanDecodePicture *vp, * easier, and gives us ample headroom. */ buf_size = 2 << av_log2(buf_size); + /* When the frames context uses DRM modifier tiling, + * hwctx->create_pnext contains VkImageDrmFormatModifierListCreateInfoEXT, + * which per spec does not extend VkBufferCreateInfo, so we need to find + * the VkVideoProfileListInfoKHR structure within it. */ + void *buf_pnext = ctx->s.hwfc->create_pnext; + if (ctx->s.hwfc->tiling == VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT) + buf_pnext = (void *)ff_vk_find_struct(ctx->s.hwfc->create_pnext, + VK_STRUCTURE_TYPE_VIDEO_PROFILE_LIST_INFO_KHR); + err = ff_vk_get_pooled_buffer(&ctx->s, &ctx->buf_pool, &new_ref, DECODER_IS_SDR(avctx->codec_id) ? (VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT) : VK_BUFFER_USAGE_VIDEO_DECODE_SRC_BIT_KHR, - ctx->s.hwfc->create_pnext, buf_size, + buf_pnext, buf_size, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | (DECODER_IS_SDR(avctx->codec_id) ? VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT : 0x0)); @@ -1217,7 +1227,9 @@ int ff_vk_frame_params(AVCodecContext *avctx, AVBufferRef *hw_frames_ctx) } } - hwfc->tiling = VK_IMAGE_TILING_OPTIMAL; + if (hwfc->tiling != VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT) + hwfc->tiling = VK_IMAGE_TILING_OPTIMAL; + hwfc->usage = VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_STORAGE_BIT | VK_IMAGE_USAGE_SAMPLED_BIT; @@ -1382,10 +1394,22 @@ int ff_vk_decode_init(AVCodecContext *avctx) dpb_frames->height = s->frames->height; dpb_hwfc = dpb_frames->hwctx; - dpb_hwfc->create_pnext = (void *)ff_vk_find_struct(ctx->s.hwfc->create_pnext, - VK_STRUCTURE_TYPE_VIDEO_PROFILE_LIST_INFO_KHR); + void *profile_list = (void *)ff_vk_find_struct(ctx->s.hwfc->create_pnext, + VK_STRUCTURE_TYPE_VIDEO_PROFILE_LIST_INFO_KHR); + /* Reference (DPB) images use the same tiling and pNext chain as output. + * If VK_VIDEO_DECODE_CAPABILITY_DPB_AND_OUTPUT_DISTINCT_BIT_KHR is 0, the + * driver does not support separate output and DPB with different layouts/tiling. */ + void *drm_create_pnext = ff_vk_find_struct(ctx->s.hwfc->create_pnext, + VK_STRUCTURE_TYPE_IMAGE_DRM_FORMAT_MODIFIER_LIST_CREATE_INFO_EXT); + if (drm_create_pnext) { + dpb_hwfc->create_pnext = drm_create_pnext; + dpb_hwfc->tiling = VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT; + av_assert2(ff_vk_find_struct(drm_create_pnext, VK_STRUCTURE_TYPE_VIDEO_PROFILE_LIST_INFO_KHR)); + } else { + dpb_hwfc->create_pnext = profile_list; + dpb_hwfc->tiling = VK_IMAGE_TILING_OPTIMAL; + } dpb_hwfc->format[0] = s->hwfc->format[0]; - dpb_hwfc->tiling = VK_IMAGE_TILING_OPTIMAL; dpb_hwfc->usage = VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR; if (ctx->common.layered_dpb) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 65e2256e2d..ec651f65db 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -190,11 +190,18 @@ typedef struct VulkanFramesPriv { /* Properties for DRM modifier for each plane in the image */ VkDrmFormatModifierPropertiesEXT drm_format_modifier_properties[5]; + + /* Set when physical device reports DEDICATED_ONLY for DMA-BUF export (try_export_flags) */ + int export_requires_dedicated; } VulkanFramesPriv; typedef struct AVVkFrameInternal { pthread_mutex_t update_mutex; + /* Binary semaphore for SYNC_FD export at DRM map time. Created once lazily, + * re-signaled each time via a submit in vulkan_map_to_drm. */ + VkSemaphore drm_sync_sem; + #if CONFIG_CUDA /* Importing external memory into cuda is really expensive so we keep the * memory imported all the time */ @@ -2370,7 +2377,7 @@ static int alloc_mem(AVHWDeviceContext *ctx, VkMemoryRequirements *req, return 0; } -static void vulkan_free_internal(AVVkFrame *f) +static void vulkan_free_internal(VulkanDevicePriv *p, AVVkFrame *f) { av_unused AVVkFrameInternal *internal = f->internal; @@ -2402,6 +2409,10 @@ static void vulkan_free_internal(AVVkFrame *f) } #endif + if (internal->drm_sync_sem != VK_NULL_HANDLE) + p->vkctx.vkfn.DestroySemaphore(p->p.act_dev, internal->drm_sync_sem, + p->p.alloc); + pthread_mutex_destroy(&internal->update_mutex); av_freep(&f->internal); } @@ -2429,7 +2440,7 @@ static void vulkan_frame_free(AVHWFramesContext *hwfc, AVVkFrame *f) vk->WaitSemaphores(hwctx->act_dev, &sem_wait, UINT64_MAX); } - vulkan_free_internal(f); + vulkan_free_internal(p, f); for (int i = 0; i < nb_images; i++) { vk->DestroyImage(hwctx->act_dev, f->img[i], hwctx->alloc); @@ -2476,6 +2487,10 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, vk->GetImageMemoryRequirements2(hwctx->act_dev, &req_desc, &req); + av_log(hwfc, AV_LOG_TRACE, + "plane %d: driver reports prefersDedicatedAllocation=%i requiresDedicatedAllocation=%i\n", + img_cnt, ded_req.prefersDedicatedAllocation, ded_req.requiresDedicatedAllocation); + if (f->tiling == VK_IMAGE_TILING_LINEAR) req.memoryRequirements.size = FFALIGN(req.memoryRequirements.size, p->props.properties.limits.minMemoryMapAlignment); @@ -2483,6 +2498,8 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, /* In case the implementation prefers/requires dedicated allocation */ use_ded_mem = ded_req.prefersDedicatedAllocation | ded_req.requiresDedicatedAllocation; + if (((VulkanFramesPriv *)hwfc->hwctx)->export_requires_dedicated) + use_ded_mem = 1; if (use_ded_mem) ded_alloc.image = f->img[img_cnt]; @@ -2854,7 +2871,8 @@ static void try_export_flags(AVHWFramesContext *hwfc, .type = VK_IMAGE_TYPE_2D, .tiling = hwctx->tiling, .usage = hwctx->usage, - .flags = VK_IMAGE_CREATE_ALIAS_BIT, + .flags = (hwctx->tiling == VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT && has_mods) ? + (hwctx->img_flags) : (VkImageCreateFlags)(VK_IMAGE_CREATE_ALIAS_BIT), }; nb_mods = has_mods ? drm_mod_info->drmFormatModifierCount : 1; @@ -2865,9 +2883,17 @@ static void try_export_flags(AVHWFramesContext *hwfc, ret = vk->GetPhysicalDeviceImageFormatProperties2(dev_hwctx->phys_dev, &pinfo, &props); + av_log(hwfc, AV_LOG_VERBOSE, "GetPhysicalDeviceImageFormatProperties2: mod[%d]=0x%llx -> %s\n", + i, has_mods ? (unsigned long long)phy_dev_mod_info.drmFormatModifier : 0ULL, + ret == VK_SUCCESS ? "OK" : "FAIL"); if (ret == VK_SUCCESS) { *iexp |= exp; *comp_handle_types |= eprops.externalMemoryProperties.compatibleHandleTypes; + if (exp == VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT) { + VulkanFramesPriv *fp = hwfc->hwctx; + fp->export_requires_dedicated = !!(eprops.externalMemoryProperties.externalMemoryFeatures & + VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT); + } } } } @@ -2895,7 +2921,8 @@ static AVBufferRef *vulkan_pool_alloc(void *opaque, size_t size) ? VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT : VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_KMT_BIT); #else - if (p->vkctx.extensions & FF_VK_EXT_EXTERNAL_FD_MEMORY) + if ((p->vkctx.extensions & FF_VK_EXT_EXTERNAL_FD_MEMORY) && + (hwctx->tiling != VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT)) try_export_flags(hwfc, &eiinfo.handleTypes, &e, VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT); @@ -2914,8 +2941,10 @@ static AVBufferRef *vulkan_pool_alloc(void *opaque, size_t size) err = create_frame(hwfc, &f, hwctx->tiling, hwctx->usage, hwctx->img_flags, hwctx->nb_layers, eiinfo.handleTypes ? &eiinfo : hwctx->create_pnext); - if (err) + if (err) { + av_log(hwfc, AV_LOG_ERROR, "vulkan_pool_alloc failed: create_frame failed: %d\n", err); return NULL; + } err = alloc_bind_mem(hwfc, f, eminfo, sizeof(*eminfo)); if (err) @@ -2943,6 +2972,7 @@ static AVBufferRef *vulkan_pool_alloc(void *opaque, size_t size) return avbuf; fail: + av_log(hwfc, AV_LOG_ERROR, "vulkan_pool_alloc failed with error %d\n", err); vulkan_frame_free(hwfc, f); return NULL; } @@ -3044,7 +3074,14 @@ static int vulkan_frames_init(AVHWFramesContext *hwfc) } /* Lone DPB images do not need additional flags. */ - if (!is_lone_dpb) { + /* With DRM modifier + video profile the caller has already chosen a valid + * usage/img_flags/chain; do not add usage or img_flags (supported_usage does + * not consider the actual modifier or video profile). */ + int drm_mod_with_video = (hwctx->tiling == VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT && + ff_vk_find_struct(hwctx->create_pnext, + VK_STRUCTURE_TYPE_VIDEO_PROFILE_LIST_INFO_KHR)); + + if (!is_lone_dpb && !drm_mod_with_video) { /* Image usage flags */ hwctx->usage |= supported_usage & (VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT | @@ -3969,7 +4006,7 @@ static int vulkan_export_to_cuda(AVHWFramesContext *hwfc, return 0; fail: - vulkan_free_internal(dst_f); + vulkan_free_internal(p, dst_f); return err; } @@ -3980,6 +4017,7 @@ static int vulkan_transfer_data_from_cuda(AVHWFramesContext *hwfc, CUcontext dummy; AVVkFrame *dst_f; AVVkFrameInternal *dst_int; + VulkanDevicePriv *p = hwfc->device_ctx->hwctx; VulkanFramesPriv *fp = hwfc->hwctx; const int planes = av_pix_fmt_count_planes(hwfc->sw_format); const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(hwfc->sw_format); @@ -4058,7 +4096,7 @@ static int vulkan_transfer_data_from_cuda(AVHWFramesContext *hwfc, fail: CHECK_CU(cu->cuCtxPopCurrent(&dummy)); - vulkan_free_internal(dst_f); + vulkan_free_internal(p, dst_f); av_buffer_unref(&dst->buf[0]); return err; } @@ -4128,6 +4166,72 @@ static VkImageAspectFlags plane_index_to_aspect(int plane) { return VK_IMAGE_ASPECT_MEMORY_PLANE_0_BIT_EXT; } +#ifdef DMA_BUF_IOCTL_EXPORT_SYNC_FILE +static int vulkan_drm_export_sync_fd(AVHWFramesContext *hwfc, AVVkFrame *f, + VulkanFramesPriv *fp, int nb_sems) +{ + int sync_fd = -1; + VkResult ret; + VulkanDevicePriv *p = hwfc->device_ctx->hwctx; + AVVulkanDeviceContext *hwctx = &p->p; + FFVulkanFunctions *vk = &p->vkctx.vkfn; + + if (f->internal->drm_sync_sem == VK_NULL_HANDLE) { + VkExportSemaphoreCreateInfo exp_info = { + .sType = VK_STRUCTURE_TYPE_EXPORT_SEMAPHORE_CREATE_INFO, + .handleTypes = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_SYNC_FD_BIT, + }; + VkSemaphoreTypeCreateInfo type_info = { + .sType = VK_STRUCTURE_TYPE_SEMAPHORE_TYPE_CREATE_INFO, + .pNext = &exp_info, + .semaphoreType = VK_SEMAPHORE_TYPE_BINARY, + }; + VkSemaphoreCreateInfo sem_create = { + .sType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO, + .pNext = &type_info, + }; + ret = vk->CreateSemaphore(hwctx->act_dev, &sem_create, hwctx->alloc, + &f->internal->drm_sync_sem); + if (ret != VK_SUCCESS) { + av_log(hwctx, AV_LOG_ERROR, "Failed to create DRM export semaphore: %s\n", + ff_vk_ret2str(ret)); + return AVERROR_EXTERNAL; + } + } + + /* Submit a lightweight exec that waits on the timeline semaphore + * (true last operation on the frame) and signals the binary semaphore, + * so any Vulkan frame can get a SYNC_FD regardless of origin. */ + FFVkExecContext *exec = ff_vk_exec_get(&p->vkctx, &fp->compute_exec); + if (ff_vk_exec_start(&p->vkctx, exec) >= 0) { + for (int i = 0; i < nb_sems; i++) + ff_vk_exec_add_dep_wait_sem(&p->vkctx, exec, f->sem[i], + f->sem_value[i], + VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT); + ff_vk_exec_add_dep_bool_sem(&p->vkctx, exec, &f->internal->drm_sync_sem, 1, + VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT, 0); + if (ff_vk_exec_submit(&p->vkctx, exec) >= 0) { + VkSemaphoreGetFdInfoKHR get_fd_info = { + .sType = VK_STRUCTURE_TYPE_SEMAPHORE_GET_FD_INFO_KHR, + .semaphore = f->internal->drm_sync_sem, + .handleType = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_SYNC_FD_BIT, + }; + ret = vk->GetSemaphoreFdKHR(hwctx->act_dev, &get_fd_info, &sync_fd); + if (ret != VK_SUCCESS) { + av_log(hwctx, AV_LOG_WARNING, + "Failed to get sync fd from DRM map export semaphore: %s\n", + ff_vk_ret2str(ret)); + sync_fd = -1; + } + } else { + ff_vk_exec_discard_deps(&p->vkctx, exec); + } + } + + return sync_fd; +} +#endif + static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, const AVFrame *src, int flags) { @@ -4138,15 +4242,14 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, AVVulkanDeviceContext *hwctx = &p->p; FFVulkanFunctions *vk = &p->vkctx.vkfn; VulkanFramesPriv *fp = hwfc->hwctx; + const int planes = av_pix_fmt_count_planes(hwfc->sw_format); const int nb_images = ff_vk_count_images(f); VkImageDrmFormatModifierPropertiesEXT drm_mod = { .sType = VK_STRUCTURE_TYPE_IMAGE_DRM_FORMAT_MODIFIER_PROPERTIES_EXT, }; - VkSemaphoreWaitInfo wait_info = { - .sType = VK_STRUCTURE_TYPE_SEMAPHORE_WAIT_INFO, - .flags = 0x0, - .semaphoreCount = nb_images, - }; + const int nb_sems = nb_images; + + int sync_fd = -1; AVDRMFrameDescriptor *drm_desc = av_mallocz(sizeof(*drm_desc)); if (!drm_desc) @@ -4156,11 +4259,28 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, if (err < 0) goto end; - /* Wait for the operation to finish so we can cleanly export it. */ - wait_info.pSemaphores = f->sem; - wait_info.pValues = f->sem_value; +#ifdef DMA_BUF_IOCTL_EXPORT_SYNC_FILE + if ((p->vkctx.extensions & FF_VK_EXT_EXTERNAL_FD_SEM) && + f->tiling == VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT && + vk->GetSemaphoreFdKHR && vk->CreateSemaphore) { + err = vulkan_drm_export_sync_fd(hwfc, f, fp, nb_sems); + if (err < 0) + goto end; + sync_fd = err; + err = 0; + } +#endif - vk->WaitSemaphores(hwctx->act_dev, &wait_info, UINT64_MAX); + if (sync_fd < 0) { + VkSemaphoreWaitInfo wait_info = { + .sType = VK_STRUCTURE_TYPE_SEMAPHORE_WAIT_INFO, + .flags = 0x0, + .semaphoreCount = nb_sems, + .pSemaphores = f->sem, + .pValues = f->sem_value, + }; + vk->WaitSemaphores(hwctx->act_dev, &wait_info, UINT64_MAX); + } err = ff_hwframe_map_create(src->hw_frames_ctx, dst, src, &vulkan_unmap_to_drm, drm_desc); if (err < 0) @@ -4174,7 +4294,7 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, goto end; } - for (int i = 0; (i < nb_images) && (f->mem[i]); i++) { + for (int i = 0; (i < planes) && (f->mem[i]); i++) { VkMemoryGetFdInfoKHR export_info = { .sType = VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR, .memory = f->mem[i], @@ -4189,12 +4309,30 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, goto end; } +#if HAVE_LINUX_DMA_BUF_H && defined(DMA_BUF_IOCTL_IMPORT_SYNC_FILE) + if (sync_fd >= 0) { + int dup_fd = dup(sync_fd); + if (dup_fd >= 0) { + struct dma_buf_import_sync_file import_info = { + .flags = DMA_BUF_SYNC_WRITE, + .fd = dup_fd, + }; + if (ioctl(drm_desc->objects[i].fd, DMA_BUF_IOCTL_IMPORT_SYNC_FILE, &import_info) < 0) + av_log(hwfc, AV_LOG_WARNING, "DMA_BUF_IOCTL_IMPORT_SYNC_FILE failed: %s\n", av_err2str(AVERROR(errno))); + close(dup_fd); + } else { + av_log(hwfc, AV_LOG_WARNING, "dup(sync_fd) failed: %s\n", av_err2str(AVERROR(errno))); + } + } +#endif + drm_desc->nb_objects++; drm_desc->objects[i].size = f->size[i]; drm_desc->objects[i].format_modifier = drm_mod.drmFormatModifier; } - drm_desc->nb_layers = nb_images; + /* NV12 has 2 planes but 1 image/semaphore */ + drm_desc->nb_layers = FFMAX(planes, nb_images); for (int i = 0; i < drm_desc->nb_layers; i++) { VkFormat plane_vkfmt = av_vkfmt_from_pixfmt(hwfc->sw_format)[i]; @@ -4209,13 +4347,14 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, for (int j = 0; j < drm_desc->layers[i].nb_planes; j++) { VkSubresourceLayout layout; + int aspect_plane = (nb_images == 1) ? i : j; VkImageSubresource sub = { - .aspectMask = plane_index_to_aspect(j), + .aspectMask = plane_index_to_aspect(aspect_plane), }; drm_desc->layers[i].planes[j].object_index = FFMIN(i, drm_desc->nb_objects - 1); - vk->GetImageSubresourceLayout(hwctx->act_dev, f->img[i], &sub, &layout); + vk->GetImageSubresourceLayout(hwctx->act_dev, f->img[FFMIN(i, nb_images - 1)], &sub, &layout); drm_desc->layers[i].planes[j].offset = layout.offset; drm_desc->layers[i].planes[j].pitch = layout.rowPitch; } @@ -4235,13 +4374,21 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, dst->width = src->width; dst->height = src->height; dst->data[0] = (uint8_t *)drm_desc; + dst->hw_frames_ctx = av_buffer_ref(src->hw_frames_ctx); + + if (sync_fd >= 0) + close(sync_fd); av_log(hwfc, AV_LOG_VERBOSE, "Mapped AVVkFrame to a DRM object!\n"); return 0; end: + for (int i = 0; i < drm_desc->nb_objects; i++) + close(drm_desc->objects[i].fd); av_free(drm_desc); + if (sync_fd >= 0) + close(sync_fd); return err; } @@ -4721,7 +4868,7 @@ end: static int vulkan_transfer_data_to(AVHWFramesContext *hwfc, AVFrame *dst, const AVFrame *src) { - av_unused VulkanDevicePriv *p = hwfc->device_ctx->hwctx; + VulkanDevicePriv *p = hwfc->device_ctx->hwctx; switch (src->format) { #if CONFIG_CUDA @@ -4751,6 +4898,7 @@ static int vulkan_transfer_data_to_cuda(AVHWFramesContext *hwfc, AVFrame *dst, CUcontext dummy; AVVkFrame *dst_f; AVVkFrameInternal *dst_int; + VulkanDevicePriv *p = hwfc->device_ctx->hwctx; VulkanFramesPriv *fp = hwfc->hwctx; const int planes = av_pix_fmt_count_planes(hwfc->sw_format); const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(hwfc->sw_format); @@ -4831,7 +4979,7 @@ static int vulkan_transfer_data_to_cuda(AVHWFramesContext *hwfc, AVFrame *dst, fail: CHECK_CU(cu->cuCtxPopCurrent(&dummy)); - vulkan_free_internal(dst_f); + vulkan_free_internal(p, dst_f); av_buffer_unref(&dst->buf[0]); return err; } -- 2.52.0 >From e4240e1087cebe3058654011ea1ff2591951f6a7 Mon Sep 17 00:00:00 2001 From: Tymur Boiko <[email protected]> Date: Sat, 11 Apr 2026 01:36:06 +0200 Subject: [PATCH 2/3] vulkan: fix -Wdiscarded-qualifiers warning and misleading DRM modifier log ff_vk_find_struct returns const void *, so storing it in const void *drm_create_pnext fixes the initialization warning but then dpb_hwfc->create_pnext = drm_create_pnext assigns const void * to void *, triggering the same warning at that line. The right fix is a (void *) cast at the call site, same as done for buf_pnext. Also restrict the GetPhysicalDeviceImageFormatProperties2 verbose log in try_export_flags to the DRM modifier path only: when has_mods is false the log always printed mod[0]=0x0, which is misleading since no DRM modifier is involved. Signed-off-by: Tymur Boiko <[email protected]> (cherry picked from commit f7ca6f74816311efa07e0cf6db6c7522b4bb847c) Assisted-by: Fairy --- libavcodec/vulkan_decode.c | 4 ++-- libavutil/hwcontext_vulkan.c | 7 ++++--- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/libavcodec/vulkan_decode.c b/libavcodec/vulkan_decode.c index afca62b304..b4df8b971e 100644 --- a/libavcodec/vulkan_decode.c +++ b/libavcodec/vulkan_decode.c @@ -1399,8 +1399,8 @@ int ff_vk_decode_init(AVCodecContext *avctx) /* Reference (DPB) images use the same tiling and pNext chain as output. * If VK_VIDEO_DECODE_CAPABILITY_DPB_AND_OUTPUT_DISTINCT_BIT_KHR is 0, the * driver does not support separate output and DPB with different layouts/tiling. */ - void *drm_create_pnext = ff_vk_find_struct(ctx->s.hwfc->create_pnext, - VK_STRUCTURE_TYPE_IMAGE_DRM_FORMAT_MODIFIER_LIST_CREATE_INFO_EXT); + void *drm_create_pnext = (void *)ff_vk_find_struct(ctx->s.hwfc->create_pnext, + VK_STRUCTURE_TYPE_IMAGE_DRM_FORMAT_MODIFIER_LIST_CREATE_INFO_EXT); if (drm_create_pnext) { dpb_hwfc->create_pnext = drm_create_pnext; dpb_hwfc->tiling = VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT; diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index ec651f65db..942ad726de 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -2883,9 +2883,10 @@ static void try_export_flags(AVHWFramesContext *hwfc, ret = vk->GetPhysicalDeviceImageFormatProperties2(dev_hwctx->phys_dev, &pinfo, &props); - av_log(hwfc, AV_LOG_VERBOSE, "GetPhysicalDeviceImageFormatProperties2: mod[%d]=0x%llx -> %s\n", - i, has_mods ? (unsigned long long)phy_dev_mod_info.drmFormatModifier : 0ULL, - ret == VK_SUCCESS ? "OK" : "FAIL"); + if (has_mods) + av_log(hwfc, AV_LOG_VERBOSE, "GetPhysicalDeviceImageFormatProperties2: mod[%d]=0x%llx -> %s\n", + i, (unsigned long long)phy_dev_mod_info.drmFormatModifier, + ret == VK_SUCCESS ? "OK" : "FAIL"); if (ret == VK_SUCCESS) { *iexp |= exp; *comp_handle_types |= eprops.externalMemoryProperties.compatibleHandleTypes; -- 2.52.0 >From 02a4641be4b7c9e7458ba572d080d3222832338e Mon Sep 17 00:00:00 2001 From: nyanmisaka <[email protected]> Date: Sun, 12 Apr 2026 20:50:38 +0800 Subject: [PATCH 3/3] hwcontext_vulkan: fix double free when vulkan_map_to_drm fails The multiplanar image with storage_bit enabled fails to be exported to DMA-BUF on the QCOM turnip driver, thus triggering this double-free issue. ``` [Parsed_hwmap_2 @ 0xffff5c002a70] Configure hwmap vulkan -> drm_prime. [hwmap @ 0xffff5c001180] Filter input: vulkan, 1920x1080 (0). [AVHWFramesContext @ 0xffff5c004e00] Unable to export the image as a FD! free(): double free detected in tcache 2 Aborted ``` Additionally, add back an av_unused attribute. Otherwise, the compiler will complain about unused variables when CUDA is not enabled. Signed-off-by: nyanmisaka <[email protected]> (cherry picked from commit ab7b6ef0a2c54c363abcb87b0df7b6f71d256f08) Assisted-by: Fairy --- libavutil/hwcontext_vulkan.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 942ad726de..9e53c2347c 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -4249,7 +4249,7 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, .sType = VK_STRUCTURE_TYPE_IMAGE_DRM_FORMAT_MODIFIER_PROPERTIES_EXT, }; const int nb_sems = nb_images; - + int free_drm_desc_on_err = 1; int sync_fd = -1; AVDRMFrameDescriptor *drm_desc = av_mallocz(sizeof(*drm_desc)); @@ -4287,6 +4287,9 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, if (err < 0) goto end; + /* It will be freed in ff_hwframe_map_create callback */ + free_drm_desc_on_err = 0; + ret = vk->GetImageDrmFormatModifierPropertiesEXT(hwctx->act_dev, f->img[0], &drm_mod); if (ret != VK_SUCCESS) { @@ -4387,7 +4390,8 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, end: for (int i = 0; i < drm_desc->nb_objects; i++) close(drm_desc->objects[i].fd); - av_free(drm_desc); + if (free_drm_desc_on_err) + av_free(drm_desc); if (sync_fd >= 0) close(sync_fd); return err; @@ -4869,7 +4873,7 @@ end: static int vulkan_transfer_data_to(AVHWFramesContext *hwfc, AVFrame *dst, const AVFrame *src) { - VulkanDevicePriv *p = hwfc->device_ctx->hwctx; + av_unused VulkanDevicePriv *p = hwfc->device_ctx->hwctx; switch (src->format) { #if CONFIG_CUDA -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
