#10915: using hardware encoding and decoding, change the resolution of videos,
and
merge them using concat
-------------------------------------+-------------------------------------
Reporter: elias | Owner: (none)
tsolis |
Type: defect | Status: new
Priority: normal | Component:
| undetermined
Version: unspecified | Resolution:
Keywords: | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Comment (by Player701):
I think I'm having a similar issue with overlay, also using hardware
decoding and encoding.
The following command line will work if both streams have the same
resolution, otherwise it fails:
{{{
ffmpeg -loglevel verbose \
-hwaccel vaapi -hwaccel_output_format vaapi -i rtmp://localhost/stream1 \
-hwaccel vaapi -hwaccel_output_format vaapi -i rtmp://localhost/stream2 \
-filter_complex "[0:v][1:v]overlay_vaapi=w=500" \
-c:v h264_vaapi -global_quality 25 -g 50 -r 25 -an -f null -
}}}
{{{
ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 14 (SUSE Linux)
configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64
--incdir=/usr/include/ffmpeg --extra-cflags='-O2 -Wall -U_FORTIFY_SOURCE
-D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables
-fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type
-flto=auto -ffat-lto-objects -g' --optflags='-O2 -Wall -U_FORTIFY_SOURCE
-D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables
-fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type
-flto=auto -ffat-lto-objects -g' --disable-htmlpages --enable-pic
--disable-stripping --enable-shared --disable-static --enable-gpl
--enable-version3 --enable-libsmbclient --disable-openssl --enable-gnutls
--enable-ladspa --enable-libshaderc --enable-vulkan --enable-libplacebo
--enable-libaom --enable-libass --enable-libbluray --enable-libbs2b
--enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libdc1394
--enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-
libfribidi --enable-libgsm --enable-libjack --enable-libjxl --enable-
librist --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg
--enable-libopenmpt --enable-libopenh264-dlopen --enable-libopus --enable-
libpulse --enable-librav1e --enable-librubberband --enable-libsvtav1
--enable-libsoxr --enable-libspeex --enable-libssh --enable-libsrt
--enable-libtheora --enable-libtwolame --enable-libvidstab --enable-
libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-
libxml2 --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lto
--enable-lv2 --enable-libvpl --enable-vaapi --enable-vdpau --enable-
version3 --enable-libfdk-aac-dlopen --enable-nonfree --enable-libopencore-
amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-libx264
--enable-libx265 --enable-libxvid
libavutil 59. 39.100 / 59. 39.100
libavcodec 61. 19.100 / 61. 19.100
libavformat 61. 7.100 / 61. 7.100
libavdevice 61. 3.100 / 61. 3.100
libavfilter 10. 4.100 / 10. 4.100
libswscale 8. 3.100 / 8. 3.100
libswresample 5. 3.100 / 5. 3.100
libpostproc 58. 3.100 / 58. 3.100
[tcp @ 0x555f7fb057c0] Starting connection attempt to ::1 port 1935
[tcp @ 0x555f7fb057c0] Connection attempt to ::1 port 1935 failed:
Connection refused
[tcp @ 0x555f7fb057c0] Starting connection attempt to 127.0.0.1 port 1935
[tcp @ 0x555f7fb057c0] Successfully connected to 127.0.0.1 port 1935
[h264 @ 0x555f7fb50480] Reinit context to 2688x1520, pix_fmt: yuvj420p
Selecting decoder 'h264' because of requested hwaccel method vaapi
Input #0, flv, from 'rtmp://localhost/stream1':
Metadata:
|RtmpSampleAccess: true
Server : NGINX RTMP (github.com/arut/nginx-rtmp-module)
displayWidth : 2688
displayHeight : 1520
fps : 26
profile :
level :
Duration: 00:00:00.00, start: 17194.931000, bitrate: N/A
Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
Stream #0:1: Video: h264 (High), 1 reference frame, yuvj420p(pc,
progressive, left), 2688x1520 [SAR 1:1 DAR 168:95], 26 fps, 26 tbr, 1k tbn
[tcp @ 0x555f7fb13740] Starting connection attempt to ::1 port 1935
[tcp @ 0x555f7fb13740] Connection attempt to ::1 port 1935 failed:
Connection refused
[tcp @ 0x555f7fb13740] Starting connection attempt to 127.0.0.1 port 1935
[tcp @ 0x555f7fb13740] Successfully connected to 127.0.0.1 port 1935
[h264 @ 0x555f7fba4fc0] Reinit context to 2560x1440, pix_fmt: yuvj420p
Selecting decoder 'h264' because of requested hwaccel method vaapi
Input #1, flv, from 'rtmp://localhost/stream2':
Metadata:
|RtmpSampleAccess: true
Server : NGINX RTMP (github.com/arut/nginx-rtmp-module)
displayWidth : 2560
displayHeight : 1440
fps : 25
profile :
level :
Duration: 00:00:00.00, start: 17200.176000, bitrate: N/A
Stream #1:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
Stream #1:1: Video: h264 (Main), 1 reference frame, yuvj420p(pc, bt709,
progressive, left), 2560x1440, 25 fps, 25 tbr, 1k tbn
[out#0/null @ 0x555f7fb08100] Creating output stream from unlabeled output
of complex filtergraph 0. This overrides automatic video mapping.
[vost#0:0/h264_vaapi @ 0x555f7fb29740] Created video stream from complex
filtergraph 0:[overlay_vaapi:default]
[vost#0:0/h264_vaapi @ 0x555f7fb29740]
[out#0/null @ 0x555f7fb08100] No explicit maps, mapping streams
automatically...
[fc#0 @ 0x555f7fb02f40] Binding input with label '0:v' to input stream 0:1
[AVHWDeviceContext @ 0x555f811e3e00] Trying to use DRM render node for
device 0.
[AVHWDeviceContext @ 0x555f811e3e00] libva: VA-API version 1.22.0
[AVHWDeviceContext @ 0x555f811e3e00] libva: Trying to open
/usr/lib64/dri/iHD_drv_video.so
[AVHWDeviceContext @ 0x555f811e3e00] libva: Found init function
__vaDriverInit_1_22
[AVHWDeviceContext @ 0x555f811e3e00] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x555f811e3e00] Initialised VAAPI connection: version
1.22
[AVHWDeviceContext @ 0x555f811e3e00] VAAPI driver: Intel iHD driver for
Intel(R) Gen Graphics - 25.1.1 ().
[AVHWDeviceContext @ 0x555f811e3e00] Driver not found in known nonstandard
list, using standard behaviour.
[fc#0 @ 0x555f7fb02f40] Binding input with label '1:v' to input stream 1:1
Stream mapping:
Stream #0:1 (h264) -> overlay_vaapi
Stream #1:1 (h264) -> overlay_vaapi
overlay_vaapi:default -> Stream #0:0 (h264_vaapi)
[vost#0:0/h264_vaapi @ 0x555f7fb29740] Starting thread...
[fc#0 @ 0x555f7fb02f40] Starting thread...
[vist#0:1/h264 @ 0x555f7fba3c40] [dec:h264 @ 0x555f7fb417c0] Starting
thread...
[vist#1:1/h264 @ 0x555f7fb12ac0] [dec:h264 @ 0x555f7fb40180] Starting
thread...
[in#0/flv @ 0x555f7fb04a40] Starting thread...
[in#1/flv @ 0x555f7fb16140] Starting thread...
Press [q] to stop, [?] for help
[h264 @ 0x555f812ca640] Reinit context to 2688x1520, pix_fmt: vaapi
[h264 @ 0x555f810132c0] Reinit context to 2560x1440, pix_fmt: vaapi
[graph 0 input from stream 0:1 @ 0x7f04f4005a80] w:2688 h:1520
pixfmt:vaapi tb:1/1000 fr:26/1 sar:1/1 csp:unknown range:pc
[graph 0 input from stream 1:1 @ 0x7f04f4006540] w:2560 h:1440
pixfmt:vaapi tb:1/1000 fr:25/1 sar:0/1 csp:bt709 range:pc
[auto_scale_0 @ 0x7f04f4008200] w:iw h:ih flags:'' interl:0
[Parsed_overlay_vaapi_0 @ 0x7f04f4005440] auto-inserting filter
'auto_scale_0' between the filter 'graph 0 input from stream 1:1' and the
filter 'Parsed_overlay_vaapi_0'
Impossible to convert between the formats supported by the filter 'graph 0
input from stream 1:1' and the filter 'auto_scale_0'
[fc#0 @ 0x555f7fb02f40] Error reinitializing filters!
[fc#0 @ 0x555f7fb02f40] Task finished with error code: -38 (Function not
implemented)
[fc#0 @ 0x555f7fb02f40] Terminating thread with return code -38 (Function
not implemented)
[vost#0:0/h264_vaapi @ 0x555f7fb29740] Encoder thread received EOF
[vost#0:0/h264_vaapi @ 0x555f7fb29740] Could not open encoder before EOF
[vost#0:0/h264_vaapi @ 0x555f7fb29740] Task finished with error code: -22
(Invalid argument)
[vost#0:0/h264_vaapi @ 0x555f7fb29740] Terminating thread with return code
-22 (Invalid argument)
[vist#0:1/h264 @ 0x555f7fba3c40] [dec:h264 @ 0x555f7fb417c0] Decoder
returned EOF, finishing
[vist#0:1/h264 @ 0x555f7fba3c40] [dec:h264 @ 0x555f7fb417c0] Terminating
thread with return code 0 (success)
[vist#0:1/h264 @ 0x555f7fba3c40] All consumers of this stream are done
[in#0/flv @ 0x555f7fb04a40] All consumers are done
[in#0/flv @ 0x555f7fb04a40] Terminating thread with return code 0
(success)
[vist#1:1/h264 @ 0x555f7fb12ac0] [dec:h264 @ 0x555f7fb40180] Decoder
returned EOF, finishing
[vist#1:1/h264 @ 0x555f7fb12ac0] [dec:h264 @ 0x555f7fb40180] Terminating
thread with return code 0 (success)
[vist#1:1/h264 @ 0x555f7fb12ac0] All consumers of this stream are done
[in#1/flv @ 0x555f7fb16140] All consumers are done
[in#1/flv @ 0x555f7fb16140] Terminating thread with return code 0
(success)
[out#0/null @ 0x555f7fb08100] Nothing was written into output file,
because at least one of its streams received no packets.
frame= 0 fps=0.0 q=0.0 Lsize= 0KiB time=N/A bitrate=N/A speed=N/A
[in#0/flv @ 0x555f7fb04a40] Input file #0 (rtmp://localhost/stream1):
[in#0/flv @ 0x555f7fb04a40] Input stream #0:1 (video): 36 packets read
(680287 bytes); 12 frames decoded; 0 decode errors;
[in#0/flv @ 0x555f7fb04a40] Total: 36 packets (680287 bytes) demuxed
[AVIOContext @ 0x555f7fb185c0] Statistics: 804302 bytes read, 0 seeks
[in#1/flv @ 0x555f7fb16140] Input file #1 (rtmp://localhost/stream2):
[in#1/flv @ 0x555f7fb16140] Input stream #1:1 (video): 26 packets read
(1154248 bytes); 2 frames decoded; 0 decode errors;
[in#1/flv @ 0x555f7fb16140] Total: 26 packets (1154248 bytes) demuxed
[AVIOContext @ 0x555f7fba2a80] Statistics: 1785858 bytes read, 0 seeks
Conversion failed!
}}}
It looks like FFmpeg tries to insert a software scaling filter, which
obviously fails because there are only hardware frames to work with.
As of now, I have to resort to software decoding because of this issue. I
can still use hardware encoding with hwupload:
{{{
ffmpeg -init_hw_device vaapi=hw -filter_hw_device hw \
-i rtmp://localhost/stream1 \
-i rtmp://localhost/stream2 \
-filter_complex
"[0:v]format=nv12,hwupload=extra_hw_frames=120[i0];[1:v]format=nv12,hwupload=extra_hw_frames=120[i1];[i0][i1]overlay_vaapi=w=500"
\
-c:v h264_vaapi -global_quality 25 -g 50 -r 25 -an -f null -
}}}
--
Ticket URL: <https://trac.ffmpeg.org/ticket/10915#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
_______________________________________________
FFmpeg-trac mailing list
FFmpeg-trac@avcodec.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-trac
To unsubscribe, visit link above, or email
ffmpeg-trac-requ...@ffmpeg.org with subject "unsubscribe".
