#11437: Vulkan encoder race conditions and conversion failures
-------------------------------------+-------------------------------------
Reporter: Capitrium | Type: defect
Status: new | Priority: normal
Component: | Version: git-
undetermined | master
Keywords: h264_vulkan | Blocked By:
h264_nvenc |
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
We're comparing the new `h264_vulkan` encoder against the `h264_nvenc`
encoder we currently use with our NVIDIA T4 GPUs, and are running into
inconsistent behaviour and what looks like race conditions in the vulkan
encoder.
ffmpeg version 7.1.git (from https://ffmpeg.org/releases/ffmpeg-snapshot-
git.tar.bz2 on 2025-01-28)
ffmpeg commands:
cuda (works):
{{{
ffmpeg -v 9 -loglevel 99 -init_hw_device "cuda=cu:0,debug=1" -hwaccel
cuvid -hwaccel_output_format cuda -i /tmp/bbb.mp4 -filter_hw_device cu
-c:v h264_nvenc -an -y /tmp/output-cuda.mp4
}}}
vulkan (fails):
{{{
ffmpeg -v 9 -loglevel 99 -init_hw_device "vulkan=vk:0,debug=1" -hwaccel
vulkan -hwaccel_output_format vulkan -i /tmp/bbb.mp4 -filter_hw_device vk
-c:v h264_vulkan -an -y /tmp/output-vulkan.mp4
}}}
input file:
Get the first 30s of Big Buck Bunny with this command:
{{{
ffmpeg -ss 00:00:00 -t 30 -i
http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_60fps_normal.mp4
-c copy /tmp/bbb.mp4
}}}
The full debug logs are attached, but here's an excerpt from the vulkan
logs:
{{{
...
[h264_vulkan @ 0xaaaaba9bacc0] Pick nothing to encode next - need more
input for reference pictures.
[h264_vulkan @ 0xaaaaba9bacc0] Input frame: 1920x1080 (5).
[h264_vulkan @ 0xaaaaba9bacc0] Pick normal P-picture to encode next.
[h264_vulkan @ 0xaaaabcb904f0] Creating a buffer of 6332416 bytes, usage:
0x8000, flags: 0xa
[h264_vulkan @ 0xaaaaba9bacc0] Default RefPicList0 for fn=1/poc=3:
fn=0/poc=0
[AVHWDeviceContext @ 0xaaaab9309dd0] Validation Error: [ SYNC-HAZARD-
WRITE-RACING-READ ] Object 0: handle = 0xaaaabc4311e0, type =
VK_OBJECT_TYPE_QUEUE; | MessageID = 0xccb77929 | vkQueueSubmit2(): Hazard
WRITE_RACING_READ for entry 0, VkCommandBuffer 0xffff440099b0[], Submitted
access info (submitted_usage: SYNC_IMAGE_LAYOUT_TRANSITION, command:
vkCmdPipelineBarrier2, seq_no: 1, VkImage 0x470000000047[], reset_no: 1).
Access info (prior_usage: SYNC_VIDEO_DECODE_VIDEO_DECODE_READ,
read_barriers:
VK_PIPELINE_STAGE_2_BOTTOM_OF_PIPE_BIT|VK_PIPELINE_STAGE_2_VIDEO_DECODE_BIT_KHR,
queue: VkQueue 0xaaaabc27dd60[], submit: 7, batch: 0, batch_tag: 65,
command: vkCmdDecodeVideoKHR, command_buffer: VkCommandBuffer
0xffff3c0cc0d0[], seq_no: 2, reset_no: 5).
[h264_vulkan @ 0xaaaaba9bacc0] Frame 0/0 encoded
[h264_vulkan @ 0xaaaaba9bacc0] Output packet: pts 2, dts 1, size 834
bytes.
[h264_vulkan @ 0xaaaaba9bacc0] Input frame: 1920x1080 (6).
[h264 @ 0xaaaabcb268b0] Decoding frame, 141 bytes, 1 slices
[AVHWDeviceContext @ 0xaaaab9309dd0] Validation Error: [ SYNC-HAZARD-READ-
RACING-WRITE ] Object 0: handle = 0xaaaabc27dd60, type =
VK_OBJECT_TYPE_QUEUE; | MessageID = 0x29910a35 | vkQueueSubmit2(): Hazard
READ_RACING_WRITE for entry 0, VkCommandBuffer 0xffff3c0c39d0[], Submitted
access info (submitted_usage: SYNC_VIDEO_DECODE_VIDEO_DECODE_READ,
command: vkCmdDecodeVideoKHR, seq_no: 2, reset_no: 7). Access info
(prior_usage: SYNC_VIDEO_DECODE_VIDEO_DECODE_WRITE, write_barriers:
SYNC_HOST_HOST_READ|SYNC_HOST_HOST_WRITE|SYNC_VIDEO_ENCODE_VIDEO_ENCODE_READ|SYNC_VIDEO_ENCODE_VIDEO_ENCODE_WRITE|SYNC_ACCELERATION_STRUCTURE_COPY_ACCELERATION_STRUCTURE_READ|SYNC_ACCELERATION_STRUCTURE_COPY_ACCELERATION_STRUCTURE_WRITE|SYNC_ACCELERATION_STRUCTURE_COPY_TRANSFER_READ|SYNC_ACCELERATION_STRUCTURE_COPY_TRANSFER_WRITE|SYNC_COPY_TRANSFER_READ|SYNC_COPY_TRANSFER_WRITE|SYNC_RESOLVE_TRANSFER_READ|SYNC_RESOLVE_TRANSFER_WRITE|SYNC_BLIT_TRANSFER_READ|SYNC_BLIT_TRANSFER_WRITE|SYNC_CLEAR_TRANSFER_WRITE,
queue: VkQueue 0xaaaabc3577a[h264 @ 0xaaaaba9af5b0] nal_unit_type: 1(Coded
slice of a non-IDR picture), nal_ref_idc: 0
[vf#0:0 @ 0xaaaaba9ae540] Clipping frame in rate conversion by 0.019997
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xaaaaba9ac3c0] stream 0, sample 25, dts 416333
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xaaaaba9ac3c0] stream 1, sample 13, dts 416000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xaaaaba9ac3c0] stream 0, sample 25, dts 416333
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xaaaaba9ac3c0] stream 1, sample 14, dts 448000
[h264 @ 0xaaaaba9af5b0] Decoding frame, 364 bytes, 1 slices
[h264 @ 0xaaaabc6a9970] nal_unit_type: 1(Coded slice of a non-IDR
picture), nal_ref_idc: 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xaaaaba9ac3c0] stream 0, sample 26, dts 433000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xaaaaba9ac3c0] stream 1, sample 14, dts 448000
[vf#0:0 @ 0xaaaaba9ae540] Clipping frame in rate conversion by 0.019997
[AVHWDeviceContext @ 0xaaaab9309dd0] Validation Error: [ SYNC-HAZARD-READ-
RACING-WRITE ] Object 0: handle = 0xaaaabc3577a0, type =
VK_OBJECT_TYPE_QUEUE; | MessageID = 0x29910a35 | vkQueueSubmit2(): Hazard
READ_RACING_WRITE for entry 0, VkCommandBuffer 0xffff3c0c7940[], Submitted
access info (submitted_usage: SYNC_VIDEO_DECODE_VIDEO_DECODE_READ,
command: vkCmdDecodeVideoKHR, seq_no: 2, reset_no: 7). Access info
(prior_usage: SYNC_VIDEO_DECODE_VIDEO_DECODE_WRITE, write_barriers:
SYNC_HOST_HOST_READ|SYNC_HOST_HOST_WRITE|SYNC_VIDEO_ENCODE_VIDEO_ENCODE_READ|SYNC_VIDEO_ENCODE_VIDEO_ENCODE_WRITE|SYNC_ACCELERATION_STRUCTURE_COPY_ACCELERATION_STRUCTURE_READ|SYNC_ACCELERATION_STRUCTURE_COPY_ACCELERATION_STRUCTURE_WRITE|SYNC_ACCELERATION_STRUCTURE_COPY_TRANSFER_READ|SYNC_ACCELERATION_STRUCTURE_COPY_TRANSFER_WRITE|SYNC_COPY_TRANSFER_READ|SYNC_COPY_TRANSFER_WRITE|SYNC_RESOLVE_TRANSFER_READ|SYNC_RESOLVE_TRANSFER_WRITE|SYNC_BLIT_TRANSFER_READ|SYNC_BLIT_TRANSFER_WRITE|SYNC_CLEAR_TRANSFER_WRITE,
queue: VkQueue 0xaaaabc27dd6[h264_vulkan @ 0xaaaaba9bacc0] Pick B-picture
at depth 1 to encode next.
[h264_vulkan @ 0xaaaaba9bacc0] Default RefPicList0 for fn=2/poc=1:
fn=0/poc=0 fn=1/poc=3
[h264_vulkan @ 0xaaaaba9bacc0] Default RefPicList1 for fn=2/poc=1:
fn=1/poc=3 fn=0/poc=0
[h264_vulkan @ 0xaaaaba9bacc0] Frame 3/1 encoded
[h264_vulkan @ 0xaaaaba9bacc0] Output packet: pts 5, dts 2, size 16272
bytes.
[h264_vulkan @ 0xaaaaba9bacc0] Input frame: 1920x1080 (7).
...
[h264_vulkan @ 0xaaaaba9bacc0] Frame 39/37 encoded
[h264_vulkan @ 0xaaaaba9bacc0] Output packet: pts 41, dts 38, size 3549
bytes.
[h264_vulkan @ 0xaaaaba9bacc0] Input frame: 1920x1080 (43).
[h264 @ 0xaaaabc6a9970] Decoding frame, 327 bytes, 1 slices
[vf#0:0 @ 0xaaaaba9ae540] Clipping frame in rate conversion by 0.019997
[h264_vulkan @ 0xaaaaba9bacc0] Pick B-picture at depth 1 to encode next.
[h264_vulkan @ 0xaaaaba9bacc0] Default RefPicList0 for fn=14/poc=38:
fn=12/poc=36 fn=13/poc=39
[h264_vulkan @ 0xaaaaba9bacc0] Default RefPicList1 for fn=14/poc=38:
fn=13/poc=39 fn=12/poc=36
[h264_vulkan @ 0xaaaaba9bacc0] Encode failed: -1163371440.
[vost#0:0/h264_vulkan @ 0xaaaaba9bbff0] [enc:h264_vulkan @ 0xaaaaba9aeaf0]
Error submitting video frame to the encoder
[vost#0:0/h264_vulkan @ 0xaaaaba9bbff0] [enc:h264_vulkan @ 0xaaaaba9aeaf0]
Error encoding a frame: Error number -1163371440 occurred
[vost#0:0/h264_vulkan @ 0xaaaaba9bbff0] Task finished with error code:
-1163371440 (Error number -1163371440 occurred)
[vost#0:0/h264_vulkan @ 0xaaaaba9bbff0] Terminating thread with return
code -1163371440 (Error number -1163371440 occurred)
[vf#0:0 @ 0xaaaaba9b0280] All consumers returned EOF
[out#0/mp4 @ 0xaaaaba81cfa0] All streams finished
[out#0/mp4 @ 0xaaaaba81cfa0] Terminating thread with return code 0
(success)
[vf#0:0 @ 0xaaaaba9b0280] Terminating thread with return code 0 (success)
[vist#0:0/h264 @ 0xaaaaba9bba20] [dec:h264 @ 0xaaaaba9ab4b0] Decoder
returned EOF, finishing
[vist#0:0/h264 @ 0xaaaaba9bba20] [dec:h264 @ 0xaaaaba9ab4b0] Terminating
thread with return code 0 (success)
[vist#0:0/h264 @ 0xaaaaba9bba20] All consumers of this stream are done
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0xaaaabc611400] All consumers are done
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0xaaaabc611400] Terminating thread with
return code 0 (success)
[h264 @ 0xaaaabc9935d0] nal_unit_type: 1(Coded slice of a non-IDR
picture), nal_ref_idc: 0
[h264 @ 0xaaaabc9935d0] Decoding frame, 285 bytes, 1 slices
[AVIOContext @ 0xaaaabcb78450] Statistics: 389789 bytes written, 2 seeks,
5 writeouts
[out#0/mp4 @ 0xaaaaba81cfa0] Output file #0 (/tmp/output-vulkan.mp4):
[out#0/mp4 @ 0xaaaaba81cfa0] Output stream #0:0 (video): 42 frames
encoded; 38 packets muxed (388233 bytes);
[out#0/mp4 @ 0xaaaaba81cfa0] Total: 38 packets (388233 bytes) muxed
[out#0/mp4 @ 0xaaaaba81cfa0] video:379KiB audio:0KiB subtitle:0KiB other
streams:0KiB global headers:0KiB muxing overhead: 0.399760%
frame= 38 fps=0.0 q=-0.0 Lsize= 381KiB time=00:00:00.65
bitrate=4797.4kbits/s speed=3.38x
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0xaaaabc611400] Input file #0
(/tmp/bbb.mp4):
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0xaaaabc611400] Input stream #0:0
(video): 72 packets read (77578 bytes); 57 frames decoded; 0 decode
errors;
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0xaaaabc611400] Total: 72 packets (77578
bytes) demuxed
[AVIOContext @ 0xaaaab977fd70] Statistics: 192903 bytes read, 2 seeks
Conversion failed!
[AVHWDeviceContext @ 0xaaaab9309dd0] Validation Error: [ VUID-
vkDestroyDevice-device-05137 ] Object 0: handle = 0xd700000000d7, type =
VK_OBJECT_TYPE_BUFFER; | MessageID = 0x4872eaa0 | vkCreateDevice(): OBJ
ERROR : For VkDevice 0xaaaaba9cef00[], VkBuffer 0xd700000000d7[] has not
been destroyed. The Vulkan spec states: All child objects created on
device must have been destroyed prior to destroying device
(https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html
#VUID-vkDestroyDevice-device-05137)
...
}}}
To me, the `SYNC-HAZARD-READ-RACING-WRITE` and `SYNC-HAZARD-WRITE-RACING-
READ` lines point to some kind of race condition. It's worth noting that
I've seen the vulkan encode succeed sporadically with a shorter/smaller
input video (7sec./720p/23.98fps), and I'm pretty sure I've also seen
`SYNC-HAZARD-WRITE-RACING-WRITE` in other tests as well, but that one
doesn't appear in these logs.
Interestingly, ffmpeg doesn't report any vulkan decoders in `-codecs`,
despite being built with `--enable-vulkan`:
{{{
# ffmpeg -codecs | grep vulkan
ffmpeg version 7.1.git Copyright (c) 2000-2025 the FFmpeg developers
built with gcc 13 (Ubuntu 13.2.0-23ubuntu4)
configuration: --disable-doc --disable-ffplay --disable-stripping
--disable-decoder=libopenjpeg --enable-cuda --enable-cuda-nvcc --enable-
nvenc --enable-cuvid --enable-libnpp --enable-vulkan --enable-libshaderc
--enable-shared --enable-nonfree --enable-frei0r --enable-gpl --enable-
libfdk_aac --enable-libass --enable-libfreetype --enable-libfontconfig
--enable-libfribidi --enable-libvidstab --enable-libmp3lame --enable-
libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-
libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-
openssl --enable-postproc --enable-small --enable-version3 --extra-
cflags='-I/opt/ffmpeg/include -I/opt/ffmpeg/vulkan/include
-I/opt/ffmpeg/include/ffnvcodec -I/usr/local/cuda/include/' --extra-
ldflags='-L/opt/ffmpeg/lib -L/usr/local/cuda/lib64/' --extra-libs=-ldl
--nvccflags='-gencode arch=compute_75,code=sm_75 -O2' --prefix=/opt/ffmpeg
libavutil 59. 56.100 / 59. 56.100
libavcodec 61. 31.101 / 61. 31.101
libavformat 61. 9.106 / 61. 9.106
libavdevice 61. 4.100 / 61. 4.100
libavfilter 10. 9.100 / 10. 9.100
libswscale 8. 13.100 / 8. 13.100
libswresample 5. 4.100 / 5. 4.100
libpostproc 58. 4.100 / 58. 4.100
DEV..S ffv1 (encoders: ffv1 ffv1_vulkan)
DEV.LS h264 (decoders: h264 h264_v4l2m2m h264_cuvid)
(encoders: libx264 libx264rgb h264_nvenc h264_v4l2m2m h264_vulkan)
DEV.L. hevc (decoders: hevc hevc_v4l2m2m hevc_cuvid)
(encoders: libx265 hevc_nvenc hevc_v4l2m2m hevc_vulkan)
}}}
There are also some other open issues regarding vulkan decoding/encoding:
https://trac.ffmpeg.org/ticket/10970
https://trac.ffmpeg.org/ticket/11219
https://trac.ffmpeg.org/ticket/11296
So, two questions:
1. Are the Vulkan video APIs in ffmpeg considered stable as of 7.1? Or are
they still experimental?
2. Should ffmpeg be reporting any Vulkan video decoders when built with
`--enable-vulkan`? Or are extra configure flags required to enable Vulkan
video decoding?
Thanks!
--
Ticket URL: <https://trac.ffmpeg.org/ticket/11437>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
_______________________________________________
FFmpeg-trac mailing list
FFmpeg-trac@avcodec.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-trac
To unsubscribe, visit link above, or email
ffmpeg-trac-requ...@ffmpeg.org with subject "unsubscribe".
