Re: [FFmpeg-devel] [WIP] XComposite window capture demuxer (Linux)
> If that was the only concern against it, then I'd say it is acceptable. > But there are some other concerns, also it seems to me xcbgrab can be > improved to reach the same smoothness as this attempt, and that was the > main distinctive feature of it. With that advantage lost, I'd say fixing > xcbgrab is the better bet. I would think yes - that would be a good outcome - but please note xcbgrab doesn't seem to work when a window is borderless and/or full screen - which sorts of defeat the purpose of capturing 3d apps running full screen? Do you have a patch for xcbgrab? I will try and capture similar video. If the patch is about the av_sleep(half_expected_time), the code of xcomposite uses the very same logic (heavily inspired by it), but it works smoother - it would be interesting to see CPU usage of xcbgrab after the patch. On 20/05/2020 07:28, Marton Balint wrote: On Tue, 19 May 2020, Nicolas George wrote: Marton Balint (12020-05-19): As Nicolas mentioned, kmsgrab practically requires root. Also, I tried it on Intel half a year ago on Ubuntu 18.04, and it simply does not work correctly. There were crashes, there were random failures with cryptic error messages, usually at the beginning of the capture, somtimes in the middle of it. And the captured frame was missing some of the drawn opengl primitives, it looked like kmsgrab grabbed the frame from the back buffer and not the front. So kmsgab on Intel is a no-no for me. Thanks for the confirmation. Could you weigh in on the issue of filters/devices changing the OpenGL global state, please? If that was the only concern against it, then I'd say it is acceptable. But there are some other concerns, also it seems to me xcbgrab can be improved to reach the same smoothness as this attempt, and that was the main distinctive feature of it. With that advantage lost, I'd say fixing xcbgrab is the better bet. Regards, Marton ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [WIP] XComposite window capture demuxer (Linux)
Hi Lynne, Thanks for the feedback. Some more discussion points below. > There already is a zero-overhead capture on linux - kmsgrab. It works on AMD and Intel. Does it work on Nvidia too? Does it have smooth capture? Does it work for 3d applications? >> Wouldn't a hwaccel frame imply no choice for encoder (as in now we get a true uncompressed RGBA buffer which can be encoded as we see fit - if we were to get hwacell wouldn't we be forced to use the encoded data as is)? >> > > No. My point was around if you use say NVenc to compress frames, you wouldn't want to decompress-recompress to avoid artefacts? > I wouldn't be surprised if the xlib code has some PTS issues and schedules downloading a frame late. I'm far from being an expert, but I would be surprised if that had issues considering it's the main Linux screen capture we have? The xcb calls to grab a frame are quite simple/straightforward (both shm and non shm versions). > Maybe its worth fixing that instead of adding a yet another x11 capture? This is a xcomposite capture, I would say it's different than a x11 raw capture. > I'm against any OpenGL code being ran anywhere in our libraries since it will disturb OpenGL's global > state, breaking any OpenGL users of our libraries (there are quite a lot). Yes, there are also people who > want to screencapture and use OpenGL. I'm one. > This is also why we don't have OpenGL filtering in libavfilter. > So that's a big NAK for me. Fair enough - Understood why it would be a NAK from you. Would there be a way to mark this as experimental? Out of curiosity if I was proposing/using the native NVidia proprietary library for screen capture (linked at runtime dynamically), would that be more acceptable in terms of conflicts (I wouldn't like it because it wouldn't support AMD/Intel based hardware)? > As-is now, I don't see much value in this patch. I wrote this because I wanted to capture 3d apps at 60 FPS with high quality without having to drop frames. If there is a better way with ffmpeg/libav*, please feel free to suggest - otherwise currently folks would be forced to use OBS and not use ffmpeg/libav* based solutions. The video games segment under Linux is expanding, would ffmpeg/libav* have the right infra/codecs to cater for it? Can we capture true 60 FPS of a 3d app today? Do we have native Vulkan/OpenGL capturers? If yes then I would agree with you (no point in having this patch), otherwise there is value in it. On 19/05/2020 09:33, Lynne wrote: May 19, 2020, 09:23 by e...@fastwebnet.it: Hi Marton, Thanks very much for the feedback; below answers to your points - let me know further feedback if any. And sorry, I cannot say how useful this would be, maybe now is the time for people to speak up if somebody is particularly against adding this for any reason. I haven't been able to capture video-games/3d apps running in full screen with x11grab, and when running in windowed mode the capture was sub optimal in terms of quality (lost frames/choppy/etc etc). Unless we have better solutions with ffmpeg/libav* (which I'm not aware of :) this xcompgrab would target such audience (smooth capture alas more CPU usage). But again, if there's already a better capture, then this has been an academic exercise :) There already is a zero-overhead capture on linux - kmsgrab. It works on AMD and Intel. - Is there a way to keep the captured textures as an OpenGL/VAAPI/NVenc/Vulkan object, so the frames can work as some kind of hwaccel frames? Can this improve performance? I think there would be. I would need to find more in terms of documentation for both hwaccel frames structures/management. Please feel free to point me to guides/code/samples etc etc. Uploading to hardware frames is something uses can do themselves already. Unless there is a native interop, I don't see how doing the upload in the avdevice helps. Wouldn't a hwaccel frame imply no choice for encoder (as in now we get a true uncompressed RGBA buffer which can be encoded as we see fit - if we were to get hwacell wouldn't we be forced to use the encoded data as is)? No. - What can be the reason between the quality/smoothness difference of x11grab and the Compositor capturer? My suspicion is that OpenGL has access to the buffered data on the videocard itself, hence able to get smooth frames. Related to both these questions, I've asked the same on Stackoverflow: https://stackoverflow.com/questions/61613441/is-there-a-better-more-efficient-way-to-capture-composite-x-windows-in-linux Unfortunately no one has been able to give me an answer (I did set a bounty for it) so I posted my own understanding. I wouldn't be surprised if the xlib code has some PTS issues and schedules downloading a frame late. Maybe its worth fixing that instead of adding a yet another x11 capture? - Maybe some openGL glue can be shared with libavdevice/opengl_enc.c? Take a look, I am not sure,
Re: [FFmpeg-devel] [WIP] XComposite window capture demuxer (Linux)
Hi Marton, Thanks very much for the feedback; below answers to your points - let me know further feedback if any. > And sorry, I cannot say how useful this would be, maybe now is the time > for people to speak up if somebody is particularly against adding this > for any reason. I haven't been able to capture video-games/3d apps running in full screen with x11grab, and when running in windowed mode the capture was sub optimal in terms of quality (lost frames/choppy/etc etc). Unless we have better solutions with ffmpeg/libav* (which I'm not aware of :) this xcompgrab would target such audience (smooth capture alas more CPU usage). But again, if there's already a better capture, then this has been an academic exercise :) > - Is there a way to keep the captured textures as an > OpenGL/VAAPI/NVenc/Vulkan object, so the frames can work as some kind of > hwaccel frames? Can this improve performance? I think there would be. I would need to find more in terms of documentation for both hwaccel frames structures/management. Please feel free to point me to guides/code/samples etc etc. Wouldn't a hwaccel frame imply no choice for encoder (as in now we get a true uncompressed RGBA buffer which can be encoded as we see fit - if we were to get hwacell wouldn't we be forced to use the encoded data as is)? > - What can be the reason between the quality/smoothness difference of > x11grab and the Compositor capturer? My suspicion is that OpenGL has access to the buffered data on the videocard itself, hence able to get smooth frames. Related to both these questions, I've asked the same on Stackoverflow: https://stackoverflow.com/questions/61613441/is-there-a-better-more-efficient-way-to-capture-composite-x-windows-in-linux Unfortunately no one has been able to give me an answer (I did set a bounty for it) so I posted my own understanding. > As for the code, I only took a quick look, here are some comments > > - probably GCC atomics should be replaced by atomic > functions. Done. > - Unless there is a reason, I think av_malloc/av_free is preferred for > memory alloc functions. Done. > - Maybe some openGL glue can be shared with libavdevice/opengl_enc.c? > Take a look, I am not sure, because that was implemented with SDL and > cross platform capabilities in mind, but since your capture device is > only for linux (or not?), then maybe it makes more sense to keep it > separete? This is indeed a very specific implementation for Linux, I would agree that perhaps not sure this makes sense to use shared OpenGL functions? > - Can the Xlib functions replaced with xcb? I kind of think that would > be preferable if it is a sensible thing to do. I understand where you're coming from and it makes sense to give it a shot - my only fear is that we may need to use specific Xlib functions anyway. But let me give it a try and see where we go. On 18/05/2020 20:11, Marton Balint wrote: On Mon, 18 May 2020, Emanuele Oriani wrote: Hi Marton/ffmpeg devs/all, Haven't seen any response to the proposed XComposite windows capture. As per below the reasoning for adding this code to libav*/ffmpeg is that seems to be better in quality than x11grab, although it uses more CPU in its current implementation (uses OpenGL and PBO - optional - to fetch XComposite windows). Please let me know if it's still of interest and/or you have further technical feedback. I've posted some performance tests in the thread below. Here links of two captures (one with x11grab, the other with xcompgrab) and the sources: https://send.firefox.com/download/df859689f4cc6429/#EN6w9upyIMHqO7n_1hMTaQ https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c Thanks in advance for your time and feedback, Thanks. I am not familiar with the details of opengl/compositing capturing, but here are a few questions coming into my mind. Not only for you, but for other developers as well: - Is there a way to keep the captured textures as an OpenGL/VAAPI/NVenc/Vulkan object, so the frames can work as some kind of hwaccel frames? Can this improve performance? - What can be the reason between the quality/smoothness difference of x11grab and the Compositor capturer? As for the code, I only took a quick look, here are some comments - probably GCC atomics should be replaced by atomic functions. - Unless there is a reason, I think av_malloc/av_free is preferred for memory alloc functions. - Maybe some openGL glue can be shared with libavdevice/opengl_enc.c? Take a look, I am not sure, because that was implemented with SDL and cross platform capabilities in mind, but since your capture device is only for linux (or not?), then maybe it makes more sense to keep it separete? - Can the Xlib functions replaced with xcb? I kind of think that would be preferable if it is a sensible thing to do. And sorry, I cannot say how useful this woul
Re: [FFmpeg-devel] [WIP] XComposite window capture demuxer (Linux)
Hi Marton/ffmpeg devs/all, Haven't seen any response to the proposed XComposite windows capture. As per below the reasoning for adding this code to libav*/ffmpeg is that seems to be better in quality than x11grab, although it uses more CPU in its current implementation (uses OpenGL and PBO - optional - to fetch XComposite windows). Please let me know if it's still of interest and/or you have further technical feedback. I've posted some performance tests in the thread below. Here links of two captures (one with x11grab, the other with xcompgrab) and the sources: https://send.firefox.com/download/df859689f4cc6429/#EN6w9upyIMHqO7n_1hMTaQ https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c Thanks in advance for your time and feedback, Emanuele Ps. Apologies if I've missed reposes on this list On 10/05/2020 19:01, Emanuele Oriani wrote: Hi Marton/all, I've re-uploaded the firefox send link (expired) with some sample capture at 60 FPS: https://send.firefox.com/download/51b45decae720c08/#r5o4J2SgCJZndRdMOxxRBg Please note I have integrated usage of OpenGL's Pixel Buffer Objects - now the memory management of the captured framed can be managed by the drivers and hopefully should be DMA and less memory hops. This new code works on both Nvidia and AMD. CPU usage is still higher than x11grab (similar levels as below). Let me know your thoughts. Thanks, Emanuele Ps. quick link to demuxer: https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c On 08/05/2020 11:43, Emanuele Oriani wrote: Hi Marton, Capturing at 60 FPS 1920x1080, the difference is even more noticeable: x11grab is choppy, xcompgrab is smooth (this time I used H264 ultrafast to encode). https://send.firefox.com/download/17001ef60837a5ec/#FaQTa-dP4MB28YfPNMXxuw CPU performance: real user sys - xcompgrab: 10.32s, 18.2s, 0.38s - x11grab: 10.20s, 11.6s, 0.22s Definitely much more CPU usage, but the output quality can't be compared. Please note no frames were lost/skipped, everything was processed on time for both tests. Thanks, Emanuele On 08/05/2020 11:05, Emanuele Oriani wrote: Hi Marton, TL;DR xcompgrab uses more CPU but produces much better streams than x11grab. I have the following CPU usage performance report (on my i7-8700k, Nvidia 2080 Ti RTX, governor set to 'performance', on Ubuntu 18.04 using Gnome shell). The capture has been a surface area of 1720x1376 for 10 seconds at 30 FPS, only getting the frames, not trying to encode them into a file: real user sys - xcompgrab: 10.13s, 2.06s, 0.085s - x11grab: 10.05s, 0.20s, 0.035s from a first outlook x11grab is much more CPU efficient, almost 90% less CPU usage, but upon starting to record the video, it feels like it's 'choppier' and loses frames. Below videos have been captured at 1920x1080, the encoder is exactly the same with same settings and the total CPU time is greater for xcompgrab (as expected). But if you look at both, the xcompgrab doesn't lose frames/is not choppier, instead x11grab seems to be much worse. I've uploaded both 10 secs segments at: https://send.firefox.com/download/0cbf01f9126519c7/#DPKzIfjemAFm263ZSdMOBw The code used to grab and encode is https://github.com/Emanem/replayer again very simple, one thread gets the packets/frames, the other picks the frames and encodes them. Performance of this run has been: real user sys - xcompgrab: 10.42s, 12.5s, 0.22s - x11grab: 10.21s, 8.90s, 0.31s As expected x11grab yields better CPU usage but the quality of output is somehow lacking. Let me know your thoughts. Thanks, Emanuele On 07/05/2020 19:52, Marton Balint wrote: On Thu, 7 May 2020, Emanuele Oriani wrote: Hi FFMPEG devel, I have been writing a simple XComposite window capture demuxer, heavily inspired from x11grab sources and OBS Window capture logic/code. Have you compared performance to x11grab for various resolutions and frame rates? Do you have some numbers? Thanks, Marton I would like to give back to the community and share the sources; before I would formally submit for a review/patch, I would really appreciate if someone could have a high level pass at the code (it's relatively simple actually) and point out any major issues/problems/... The code is available as part of a simple test application on github, at: https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c Being the author of the code I would be happy to re-license it to be used in ffmpeg/libav* mainline once I get the OK from you. In order to compile/execute this code, one needs the following dev packages: libav*-dev, libx11-dev, libxcomposite-dev, OpenGL headers/libs. Libraries to be linked are: -lavcodec -lavformat -lavdevice -lavutil -lswscale -lX11 -lXcomposite -lGL I've tested it and it's working fine on both AMD and Nvidia hardware. Let me know if this is of interest, otherwise no worries and thanks again
Re: [FFmpeg-devel] [WIP] XComposite window capture demuxer (Linux)
Hi Marton/all, I've re-uploaded the firefox send link (expired) with some sample capture at 60 FPS: https://send.firefox.com/download/51b45decae720c08/#r5o4J2SgCJZndRdMOxxRBg Please note I have integrated usage of OpenGL's Pixel Buffer Objects - now the memory management of the captured framed can be managed by the drivers and hopefully should be DMA and less memory hops. This new code works on both Nvidia and AMD. CPU usage is still higher than x11grab (similar levels as below). Let me know your thoughts. Thanks, Emanuele Ps. quick link to demuxer: https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c On 08/05/2020 11:43, Emanuele Oriani wrote: Hi Marton, Capturing at 60 FPS 1920x1080, the difference is even more noticeable: x11grab is choppy, xcompgrab is smooth (this time I used H264 ultrafast to encode). https://send.firefox.com/download/17001ef60837a5ec/#FaQTa-dP4MB28YfPNMXxuw CPU performance: real user sys - xcompgrab: 10.32s, 18.2s, 0.38s - x11grab: 10.20s, 11.6s, 0.22s Definitely much more CPU usage, but the output quality can't be compared. Please note no frames were lost/skipped, everything was processed on time for both tests. Thanks, Emanuele On 08/05/2020 11:05, Emanuele Oriani wrote: Hi Marton, TL;DR xcompgrab uses more CPU but produces much better streams than x11grab. I have the following CPU usage performance report (on my i7-8700k, Nvidia 2080 Ti RTX, governor set to 'performance', on Ubuntu 18.04 using Gnome shell). The capture has been a surface area of 1720x1376 for 10 seconds at 30 FPS, only getting the frames, not trying to encode them into a file: real user sys - xcompgrab: 10.13s, 2.06s, 0.085s - x11grab: 10.05s, 0.20s, 0.035s from a first outlook x11grab is much more CPU efficient, almost 90% less CPU usage, but upon starting to record the video, it feels like it's 'choppier' and loses frames. Below videos have been captured at 1920x1080, the encoder is exactly the same with same settings and the total CPU time is greater for xcompgrab (as expected). But if you look at both, the xcompgrab doesn't lose frames/is not choppier, instead x11grab seems to be much worse. I've uploaded both 10 secs segments at: https://send.firefox.com/download/0cbf01f9126519c7/#DPKzIfjemAFm263ZSdMOBw The code used to grab and encode is https://github.com/Emanem/replayer again very simple, one thread gets the packets/frames, the other picks the frames and encodes them. Performance of this run has been: real user sys - xcompgrab: 10.42s, 12.5s, 0.22s - x11grab: 10.21s, 8.90s, 0.31s As expected x11grab yields better CPU usage but the quality of output is somehow lacking. Let me know your thoughts. Thanks, Emanuele On 07/05/2020 19:52, Marton Balint wrote: On Thu, 7 May 2020, Emanuele Oriani wrote: Hi FFMPEG devel, I have been writing a simple XComposite window capture demuxer, heavily inspired from x11grab sources and OBS Window capture logic/code. Have you compared performance to x11grab for various resolutions and frame rates? Do you have some numbers? Thanks, Marton I would like to give back to the community and share the sources; before I would formally submit for a review/patch, I would really appreciate if someone could have a high level pass at the code (it's relatively simple actually) and point out any major issues/problems/... The code is available as part of a simple test application on github, at: https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c Being the author of the code I would be happy to re-license it to be used in ffmpeg/libav* mainline once I get the OK from you. In order to compile/execute this code, one needs the following dev packages: libav*-dev, libx11-dev, libxcomposite-dev, OpenGL headers/libs. Libraries to be linked are: -lavcodec -lavformat -lavdevice -lavutil -lswscale -lX11 -lXcomposite -lGL I've tested it and it's working fine on both AMD and Nvidia hardware. Let me know if this is of interest, otherwise no worries and thanks again for the fantastic library/software. Emanuele ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". __
Re: [FFmpeg-devel] [WIP] XComposite window capture demuxer (Linux)
Hi Marton, Capturing at 60 FPS 1920x1080, the difference is even more noticeable: x11grab is choppy, xcompgrab is smooth (this time I used H264 ultrafast to encode). https://send.firefox.com/download/17001ef60837a5ec/#FaQTa-dP4MB28YfPNMXxuw CPU performance: realuser sys - xcompgrab: 10.32s, 18.2s, 0.38s - x11grab: 10.20s, 11.6s, 0.22s Definitely much more CPU usage, but the output quality can't be compared. Please note no frames were lost/skipped, everything was processed on time for both tests. Thanks, Emanuele On 08/05/2020 11:05, Emanuele Oriani wrote: Hi Marton, TL;DR xcompgrab uses more CPU but produces much better streams than x11grab. I have the following CPU usage performance report (on my i7-8700k, Nvidia 2080 Ti RTX, governor set to 'performance', on Ubuntu 18.04 using Gnome shell). The capture has been a surface area of 1720x1376 for 10 seconds at 30 FPS, only getting the frames, not trying to encode them into a file: real user sys - xcompgrab: 10.13s, 2.06s, 0.085s - x11grab: 10.05s, 0.20s, 0.035s from a first outlook x11grab is much more CPU efficient, almost 90% less CPU usage, but upon starting to record the video, it feels like it's 'choppier' and loses frames. Below videos have been captured at 1920x1080, the encoder is exactly the same with same settings and the total CPU time is greater for xcompgrab (as expected). But if you look at both, the xcompgrab doesn't lose frames/is not choppier, instead x11grab seems to be much worse. I've uploaded both 10 secs segments at: https://send.firefox.com/download/0cbf01f9126519c7/#DPKzIfjemAFm263ZSdMOBw The code used to grab and encode is https://github.com/Emanem/replayer again very simple, one thread gets the packets/frames, the other picks the frames and encodes them. Performance of this run has been: real user sys - xcompgrab: 10.42s, 12.5s, 0.22s - x11grab: 10.21s, 8.90s, 0.31s As expected x11grab yields better CPU usage but the quality of output is somehow lacking. Let me know your thoughts. Thanks, Emanuele On 07/05/2020 19:52, Marton Balint wrote: On Thu, 7 May 2020, Emanuele Oriani wrote: Hi FFMPEG devel, I have been writing a simple XComposite window capture demuxer, heavily inspired from x11grab sources and OBS Window capture logic/code. Have you compared performance to x11grab for various resolutions and frame rates? Do you have some numbers? Thanks, Marton I would like to give back to the community and share the sources; before I would formally submit for a review/patch, I would really appreciate if someone could have a high level pass at the code (it's relatively simple actually) and point out any major issues/problems/... The code is available as part of a simple test application on github, at: https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c Being the author of the code I would be happy to re-license it to be used in ffmpeg/libav* mainline once I get the OK from you. In order to compile/execute this code, one needs the following dev packages: libav*-dev, libx11-dev, libxcomposite-dev, OpenGL headers/libs. Libraries to be linked are: -lavcodec -lavformat -lavdevice -lavutil -lswscale -lX11 -lXcomposite -lGL I've tested it and it's working fine on both AMD and Nvidia hardware. Let me know if this is of interest, otherwise no worries and thanks again for the fantastic library/software. Emanuele ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [WIP] XComposite window capture demuxer (Linux)
Hi Marton, TL;DR xcompgrab uses more CPU but produces much better streams than x11grab. I have the following CPU usage performance report (on my i7-8700k, Nvidia 2080 Ti RTX, governor set to 'performance', on Ubuntu 18.04 using Gnome shell). The capture has been a surface area of 1720x1376 for 10 seconds at 30 FPS, only getting the frames, not trying to encode them into a file: realuser sys - xcompgrab: 10.13s, 2.06s, 0.085s - x11grab: 10.05s, 0.20s, 0.035s from a first outlook x11grab is much more CPU efficient, almost 90% less CPU usage, but upon starting to record the video, it feels like it's 'choppier' and loses frames. Below videos have been captured at 1920x1080, the encoder is exactly the same with same settings and the total CPU time is greater for xcompgrab (as expected). But if you look at both, the xcompgrab doesn't lose frames/is not choppier, instead x11grab seems to be much worse. I've uploaded both 10 secs segments at: https://send.firefox.com/download/0cbf01f9126519c7/#DPKzIfjemAFm263ZSdMOBw The code used to grab and encode is https://github.com/Emanem/replayer again very simple, one thread gets the packets/frames, the other picks the frames and encodes them. Performance of this run has been: realuser sys - xcompgrab: 10.42s, 12.5s, 0.22s - x11grab: 10.21s, 8.90s, 0.31s As expected x11grab yields better CPU usage but the quality of output is somehow lacking. Let me know your thoughts. Thanks, Emanuele On 07/05/2020 19:52, Marton Balint wrote: On Thu, 7 May 2020, Emanuele Oriani wrote: Hi FFMPEG devel, I have been writing a simple XComposite window capture demuxer, heavily inspired from x11grab sources and OBS Window capture logic/code. Have you compared performance to x11grab for various resolutions and frame rates? Do you have some numbers? Thanks, Marton I would like to give back to the community and share the sources; before I would formally submit for a review/patch, I would really appreciate if someone could have a high level pass at the code (it's relatively simple actually) and point out any major issues/problems/... The code is available as part of a simple test application on github, at: https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c Being the author of the code I would be happy to re-license it to be used in ffmpeg/libav* mainline once I get the OK from you. In order to compile/execute this code, one needs the following dev packages: libav*-dev, libx11-dev, libxcomposite-dev, OpenGL headers/libs. Libraries to be linked are: -lavcodec -lavformat -lavdevice -lavutil -lswscale -lX11 -lXcomposite -lGL I've tested it and it's working fine on both AMD and Nvidia hardware. Let me know if this is of interest, otherwise no worries and thanks again for the fantastic library/software. Emanuele ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [WIP] XComposite window capture demuxer (Linux)
Hi FFMPEG devel, I have been writing a simple XComposite window capture demuxer, heavily inspired from x11grab sources and OBS Window capture logic/code. I would like to give back to the community and share the sources; before I would formally submit for a review/patch, I would really appreciate if someone could have a high level pass at the code (it's relatively simple actually) and point out any major issues/problems/... The code is available as part of a simple test application on github, at: https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c Being the author of the code I would be happy to re-license it to be used in ffmpeg/libav* mainline once I get the OK from you. In order to compile/execute this code, one needs the following dev packages: libav*-dev, libx11-dev, libxcomposite-dev, OpenGL headers/libs. Libraries to be linked are: -lavcodec -lavformat -lavdevice -lavutil -lswscale -lX11 -lXcomposite -lGL I've tested it and it's working fine on both AMD and Nvidia hardware. Let me know if this is of interest, otherwise no worries and thanks again for the fantastic library/software. Emanuele ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
