I'm trying to get hardware accelerated screen recording to work for recording screencasts. I've gotten kmsgrab working to record my screen and encode the result as an h265 MKV file, however the end result has slightly different colors than actually appear on my screen, and I'm unable to figure out the discrepancy.
I recently found this documentation: https://trac.ffmpeg.org/wiki/Hardware/VAAPI#ScreenCapture which mentioned the possibility of using kmsgrab and VAAPI to do the recording and encoding all in the GPU while using very little CPU power. Prior to that, I was using x11grab similar to the first two commands from that link, and I had similar (but possibly slightly different) color discrepency issues. Here is my ffmpeg command: ffmpeg \ -loglevel verbose \ -framerate 60 \ -f kmsgrab -i - \ -vf hwmap=derive_device=vaapi \ -codec:v hevc_vaapi \ -qp:v 10 \ out.mkv And here is the output: ffmpeg version 3.4.7 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 9.2.0 (GCC) configuration: --disable-static --prefix=/nix/store/z3hr6nlr5v78n2q6dh9jxj93lqwfvx4h-ffmpeg-3.4.7 --arch=x86_64 --target_os=linux --enable-gpl --enable-version3 --enable-shared --enable-pic --enable-runtime-cpudetect --enable-hardcoded-tables --enable-pthreads --disable-w32threads --disable-os2threads --enable-network --enable-pixelutils --enable-ffmpeg --disable-ffplay --enable-ffprobe --disable-ffserver --enable-avcodec --enable-avdevice --enable-avfilter --enable-avformat --enable-avresample --enable-avutil --enable-postproc --enable-swresample --enable-swscale --disable-doc --enable-bzlib --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libmp3lame --enable-iconv --enable-libtheora --enable-libssh --enable-vaapi --enable-libdrm --enable-vdpau --enable-libvorbis --enable-libvpx --enable-lzma --disable-opengl --enable-libpulse --enable-sdl2 --enable-libsoxr --enable-libx264 --enable-libxvid --enable-zlib --enable-libopus --enable-libspeex --enable-libx265 --disable-debug --enable-optimizations --disable-extra-warnings --disable-stripping libavutil 55. 78.100 / 55. 78.100 libavcodec 57.107.100 / 57.107.100 libavformat 57. 83.100 / 57. 83.100 libavdevice 57. 10.100 / 57. 10.100 libavfilter 6.107.100 / 6.107.100 libavresample 3. 7. 0 / 3. 7. 0 libswscale 4. 8.100 / 4. 8.100 libswresample 2. 9.100 / 2. 9.100 libpostproc 54. 7.100 / 54. 7.100 [AVHWDeviceContext @ 0xebe3c0] Opened DRM device /dev/dri/card0: driver i915 version 1.6.0. [kmsgrab @ 0xebd6a0] Using plane 31 to locate framebuffers. [kmsgrab @ 0xebd6a0] Template framebuffer is 114: 2560x1440 32bpp 24b depth. Input #0, kmsgrab, from 'pipe:': Duration: N/A, start: 1588034012.025087, bitrate: N/A Stream #0:0: Video: wrapped_avframe, 1 reference frame, drm_prime, 2560x1440, 59.94 tbr, 1000k tbn, 1000k tbc Stream mapping: Stream #0:0 -> #0:0 (wrapped_avframe (native) -> hevc (hevc_vaapi)) [graph 0 input from stream 0:0 @ 0xed1b00] w:2560 h:1440 pixfmt:drm_prime tb:1/1000000 fr:60000/1001 sar:0/1 sws_param:flags=2 [AVHWDeviceContext @ 0xed45c0] libva: VA-API version 1.5.0 [AVHWDeviceContext @ 0xed45c0] libva: va_getDriverName() returns 0 [AVHWDeviceContext @ 0xed45c0] libva: Trying to open /run/opengl-driver/lib/dri/i965_drv_video.so [AVHWDeviceContext @ 0xed45c0] libva: Found init function __vaDriverInit_1_5 [AVHWDeviceContext @ 0xed45c0] libva: va_openDriver() returns 0 [AVHWDeviceContext @ 0xed45c0] Initialised VAAPI connection: version 1.5 [AVHWDeviceContext @ 0xed45c0] Matched "Intel i965 driver for Intel(R) Kaby Lake - 2.4.0" as known driver "Intel i965 (Quick Sync)". [hevc_vaapi @ 0xec3bc0] Input 2560x1440 -> Surface 2560x1440 -> CTU 80x45. Output #0, matroska, to 'out.mkv': Metadata: encoder : Lavf57.83.100 Stream #0:0: Video: hevc (hevc_vaapi) (Main), 1 reference frame, vaapi_vld, 2560x1440, q=2-31, 59.94 fps, 1k tbn, 59.94 tbc Metadata: encoder : Lavc57.107.100 hevc_vaapi frame= 154 fps= 46 q=-0.0 Lsize= 1715kB time=00:00:03.60 bitrate=3898.2kbits/s speed=1.08x video:1714kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.113014% Input file #0 (pipe:): Input stream #0:0 (video): 154 packets read (81312 bytes); 154 frames decoded; Total: 154 packets (81312 bytes) demuxed Output file #0 (out.mkv): Output stream #0:0 (video): 154 frames encoded; 154 packets muxed (1754650 bytes); Total: 154 packets (1754650 bytes) muxed Exiting normally, received signal 2. The resulting video has these incorrect colors: https://i.imgur.com/Jgzpjyt.png (attached as well) The bottom is what was originally on my screen, and the top is what was recorded. These are what the colors were recorded as: #F1FFE7 -> #FFFFF9 #59CD90 -> #4CCA92 #FAA613 -> #FFAE00 #3FA7D6 -> #2DA7E6 #A10702 -> #B40700 My understanding is that this is likely something to do with the pixel format somewhere along the way, however I can't figure out where it needs to be corrected. I've attempted adding a `-format bgr0` (the default) before the `-f kmsgrab` line and changing `bgr0` to a variety of things with very little effect. Using `bgra` resulted in no change, and using `rgb0`/`rgba` resulted in completely different colors. I tried almost every other format from `ffmpeg -pix_fmts`, and they all failed with some sort of error. Using the same command with the addition of `-format bgr24` (as a random example): Routing option format to both codec and muxer layer [AVHWDeviceContext @ 0x20725c0] Opened DRM device /dev/dri/card0: driver i915 version 1.6.0. [kmsgrab @ 0x2071820] Using plane 31 to locate framebuffers. [kmsgrab @ 0x2071820] Template framebuffer is 114: 2560x1440 32bpp 24b depth. Input #0, kmsgrab, from 'pipe:': Duration: N/A, start: 1588034631.592462, bitrate: N/A Stream #0:0: Video: wrapped_avframe, 1 reference frame, drm_prime, 2560x1440, 59.94 tbr, 1000k tbn, 1000k tbc Stream mapping: Stream #0:0 -> #0:0 (wrapped_avframe (native) -> hevc (hevc_vaapi)) [graph 0 input from stream 0:0 @ 0x2085880] w:2560 h:1440 pixfmt:drm_prime tb:1/1000000 fr:60000/1001 sar:0/1 sws_param:flags=2 [AVHWDeviceContext @ 0x2088680] libva: VA-API version 1.5.0 [AVHWDeviceContext @ 0x2088680] libva: va_getDriverName() returns 0 [AVHWDeviceContext @ 0x2088680] libva: Trying to open /run/opengl-driver/lib/dri/i965_drv_video.so [AVHWDeviceContext @ 0x2088680] libva: Found init function __vaDriverInit_1_5 [AVHWDeviceContext @ 0x2088680] libva: va_openDriver() returns 0 [AVHWDeviceContext @ 0x2088680] Initialised VAAPI connection: version 1.5 [AVHWDeviceContext @ 0x2088680] Matched "Intel i965 driver for Intel(R) Kaby Lake - 2.4.0" as known driver "Intel i965 (Quick Sync)". [AVHWFramesContext @ 0x2095a80] DRM format not supported by VAAPI. [Parsed_hwmap_0 @ 0x20855a0] Failed to map frame: -22. Error while filtering: Invalid argument Failed to inject frame into filter network: Invalid argument Error while processing the decoded data for stream #0:0 Conversion failed! Is this `kmsgrab` `-format` option likely to be what I'm looking for? Is my framebuffer using some strange pixel format other than `bgr0` for some reason? The other thing I tried messing with was `scale_vaapi=format=$format` in the filter chain. The main two being used were `nv12` and `p010`, however I don't really know what they mean. Originally I was seeing the format in the filter chain first and then using `hwupload` until I found this which is faster (since everything stays in the GPU from recording to encoding). I also messed with options for `-color_range`, however all values for it appeared to make things worse. (And I don't know exactly what it's doing. :/) Here's my `vainfo` output, if that's helpful: libva info: VA-API version 1.5.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /run/opengl-driver/lib/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_1_5 libva info: va_openDriver() returns 0 vainfo: VA-API version: 1.5 (libva 2.4.0) vainfo: Driver version: Intel i965 driver for Intel(R) Kaby Lake - 2.4.0 vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264Main : VAEntrypointEncSliceLP VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice VAProfileH264High : VAEntrypointEncSliceLP VAProfileH264MultiviewHigh : VAEntrypointVLD VAProfileH264MultiviewHigh : VAEntrypointEncSlice VAProfileH264StereoHigh : VAEntrypointVLD VAProfileH264StereoHigh : VAEntrypointEncSlice VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileNone : VAEntrypointVideoProc VAProfileJPEGBaseline : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointEncPicture VAProfileVP8Version0_3 : VAEntrypointVLD VAProfileVP8Version0_3 : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointEncSlice VAProfileHEVCMain10 : VAEntrypointVLD VAProfileHEVCMain10 : VAEntrypointEncSlice VAProfileVP9Profile0 : VAEntrypointVLD VAProfileVP9Profile0 : VAEntrypointEncSlice VAProfileVP9Profile2 : VAEntrypointVLD For what it's worth, I'm running on NixOS 20.03 with these settings for OpenGL: hardware.opengl = { enable = true; driSupport32Bit = true; extraPackages = with pkgs; [ vaapiIntel # (2.4.0) vaapiVdpau # (0.7.4) libvdpau-va-gl # (0.4.2) intel-media-driver # (19.3.0) ]; }; I'm not sure if these can be different, but here are the outputs of `-h encoder=` commands for `hevc_vaapi` and `h264_vaapi`: `ffmpeg -hide_banner -h encoder=hevc_vaapi`: Encoder hevc_vaapi [H.265/HEVC (VAAPI)]: General capabilities: delay Threading capabilities: none Supported pixel formats: vaapi_vld h265_vaapi AVOptions: -qp <int> E..V.... Constant QP (for P-frames; scaled by qfactor/qoffset for I/B) (from 0 to 52) (default 25) `ffmpeg -hide_banner -h encoder=h264_vaapi`: Encoder h264_vaapi [H.264/AVC (VAAPI)]: General capabilities: delay Threading capabilities: none Supported pixel formats: vaapi_vld h264_vaapi AVOptions: -qp <int> E..V.... Constant QP (for P-frames; scaled by qfactor/qoffset for I/B) (from 0 to 52) (default 20) -quality <int> E..V.... Set encode quality (trades off against speed, higher is faster) (from 0 to 8) (default 0) -low_power <int> E..V.... Use low-power encoding mode (experimental: only supported on some platforms, does not support all features) (from 0 to 1) (default 0) -coder <int> E..V.... Entropy coder type (from 0 to 1) (default cabac) cavlc E..V.... cabac E..V.... vlc E..V.... ac E..V.... I briefly tried with QSV instead of VAAPI, however I was having similar issues. Is there any benefit to using QSV over VAAPI? _______________________________________________ ffmpeg-user mailing list [email protected] https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email [email protected] with subject "unsubscribe".
