Re: [FFmpeg-devel] [PATCHv12] libavcodec: v4l2: add support for v4l2 mem2mem codecs
On 09/20/2017 08:31 PM, wm4 wrote: On Wed, 20 Sep 2017 19:01:52 -0700 Jorge Ramirez-Ortiz wrote: Btw. I noticed that this apparently never sets chroma_location? I didnt find anything in v4l2 to retrieve the chroma_location from the kernel driver (the API doesnt handle this information). does ffmpeg need this field and if so, can it be retrieved from somewhere else (bitstream perhaps)? Yes, chroma location matters. Some users notice if the location is incorrect. It can be retrieved from the bitstream, but that would require you to parse it. Which probably would require additional code. (You can't rely on the pre-filled parameters by libavformat, because an API user doesn't have to use libavformat.) ok. I suggest you make the kernel devs add this parameter. There might be more such missing parameters, I noticed this one only by chance. I see. I'll find out what is the process to modify the API and why the current drivers don't use it. Notice that -even if the API change is accepted in the coming months - all the drivers currently available in the field wont be using it in the near future though (they will be returning a default, null I guess). I just dont have any control on the drivers or the API itself: do you think this will be blocking the upstream process? ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCHv12] libavcodec: v4l2: add support for v4l2 mem2mem codecs
On Wed, 20 Sep 2017 19:01:52 -0700 Jorge Ramirez-Ortiz wrote: > > Btw. I noticed that this apparently never sets chroma_location? > > I didnt find anything in v4l2 to retrieve the chroma_location from the kernel > driver (the API doesnt handle this information). > does ffmpeg need this field and if so, can it be retrieved from somewhere > else > (bitstream perhaps)? Yes, chroma location matters. Some users notice if the location is incorrect. It can be retrieved from the bitstream, but that would require you to parse it. Which probably would require additional code. (You can't rely on the pre-filled parameters by libavformat, because an API user doesn't have to use libavformat.) I suggest you make the kernel devs add this parameter. There might be more such missing parameters, I noticed this one only by chance. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCHv12] libavcodec: v4l2: add support for v4l2 mem2mem codecs
On 09/19/2017 12:35 PM, wm4 wrote: On Mon, 11 Sep 2017 16:26:33 +0200 Jorge Ramirez-Ortiz wrote: This patchset enhances Alexis Ballier's original patch and validates it using Qualcomm's Venus hardware (driver recently landed upstream [1]). This has been tested on Qualcomm's DragonBoard 410c and 820c Configure/make scripts have been validated on Ubuntu 10.04 and 16.04. Tested decoders: - h264 - h263 - mpeg4 - vp8 - vp9 - hevc Tested encoders: - h264 - h263 - mpeg4 Tested transcoding (concurrent encoding/decoding) Some of the changes introduced: - v4l2: code cleanup and abstractions added - v4l2: follow the new encode/decode api. - v4l2: fix display size for NV12 output pool. - v4l2: handle EOS. - v4l2: vp8 and mpeg4 decoding and encoding. - v4l2: hevc and vp9 support. - v4l2: generate EOF on dequeue errors. - v4l2: h264_mp4toannexb filtering. - v4l2: fixed make install and fate issues. - v4l2: codecs enabled/disabled depending on pixfmt defined - v4l2: pass timebase/framerate to the context - v4l2: runtime decoder reconfiguration. - v4l2: add more frame information - v4l2: free hardware resources on last reference being released - v4l2: encoding: disable b-frames for upstreaming (patch required) [1] https://lwn.net/Articles/697956/ Reviewed-by: Jorge Ramirez Reviewed-by: Alexis Ballier Tested-by: Jorge Ramirez --- I assume this version fixes the lifetime issues I've pointed out. I also assume the data flow issues got solved. With that, it would look good to me, I guess. If nobody else has comments it could be applied. Btw. I noticed that this apparently never sets chroma_location? I didnt find anything in v4l2 to retrieve the chroma_location from the kernel driver (the API doesnt handle this information). does ffmpeg need this field and if so, can it be retrieved from somewhere else (bitstream perhaps)? just posted v13. I think the overall quality and feature set should be acceptable to be merged upstream (but of course, looking forward for comments if not and will continue to be as responsive as possible) ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCHv12] libavcodec: v4l2: add support for v4l2 mem2mem codecs
On 09/19/2017 12:54 PM, wm4 wrote: On Tue, 19 Sep 2017 12:48:06 -0700 Jorge Ramirez-Ortiz wrote: I also assume the data flow issues got solved. data flow? Wasn't there some confusion about how send/receive works, and how to connect it to how v4l2 data flow works? ah yes, you are right. yes that was fixed as well (removing the timeouts when waiting for buffers). (Also could we replace all instances of v4l2 with v4l? The original v4l is dead AFAIK, and the 2 is redundant and confusing.) do you mind if we keep the v4l2 (after all the API is called that way); just that v4l doesn't mean much anymore and it might create confusion to v4l2 developers also notice that the kernel interface file is called videodev2.h. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCHv12] libavcodec: v4l2: add support for v4l2 mem2mem codecs
On 09/19/2017 12:35 PM, wm4 wrote: On Mon, 11 Sep 2017 16:26:33 +0200 Jorge Ramirez-Ortiz wrote: This patchset enhances Alexis Ballier's original patch and validates it using Qualcomm's Venus hardware (driver recently landed upstream [1]). This has been tested on Qualcomm's DragonBoard 410c and 820c Configure/make scripts have been validated on Ubuntu 10.04 and 16.04. Tested decoders: - h264 - h263 - mpeg4 - vp8 - vp9 - hevc Tested encoders: - h264 - h263 - mpeg4 Tested transcoding (concurrent encoding/decoding) Some of the changes introduced: - v4l2: code cleanup and abstractions added - v4l2: follow the new encode/decode api. - v4l2: fix display size for NV12 output pool. - v4l2: handle EOS. - v4l2: vp8 and mpeg4 decoding and encoding. - v4l2: hevc and vp9 support. - v4l2: generate EOF on dequeue errors. - v4l2: h264_mp4toannexb filtering. - v4l2: fixed make install and fate issues. - v4l2: codecs enabled/disabled depending on pixfmt defined - v4l2: pass timebase/framerate to the context - v4l2: runtime decoder reconfiguration. - v4l2: add more frame information - v4l2: free hardware resources on last reference being released - v4l2: encoding: disable b-frames for upstreaming (patch required) [1] https://lwn.net/Articles/697956/ Reviewed-by: Jorge Ramirez Reviewed-by: Alexis Ballier Tested-by: Jorge Ramirez --- I assume this version fixes the lifetime issues I've pointed out. yes, the buffer issues are resolved and the user can now close the encoder/decoder without losing the buffer references. I also assume the data flow issues got solved. data flow? With that, it would look good to me, I guess. If nobody else has comments it could be applied. please let me post v13 - today/tomorrow- which cleans the code a bit more making it easier to understand and maintain (maybe it will trigger more questions?) Btw. I noticed that this apparently never sets chroma_location? I'll look into that for v13 as well. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCHv12] libavcodec: v4l2: add support for v4l2 mem2mem codecs
On Tue, 19 Sep 2017 12:48:06 -0700 Jorge Ramirez-Ortiz wrote: > > I > > also assume the data flow issues got solved. > > data flow? Wasn't there some confusion about how send/receive works, and how to connect it to how v4l2 data flow works? (Also could we replace all instances of v4l2 with v4l? The original v4l is dead AFAIK, and the 2 is redundant and confusing.) ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCHv12] libavcodec: v4l2: add support for v4l2 mem2mem codecs
On Mon, 11 Sep 2017 16:26:33 +0200 Jorge Ramirez-Ortiz wrote: > This patchset enhances Alexis Ballier's original patch and validates > it using Qualcomm's Venus hardware (driver recently landed upstream > [1]). > > This has been tested on Qualcomm's DragonBoard 410c and 820c > Configure/make scripts have been validated on Ubuntu 10.04 and > 16.04. > > Tested decoders: >- h264 >- h263 >- mpeg4 >- vp8 >- vp9 >- hevc > > Tested encoders: >- h264 >- h263 >- mpeg4 > > Tested transcoding (concurrent encoding/decoding) > > Some of the changes introduced: > - v4l2: code cleanup and abstractions added > - v4l2: follow the new encode/decode api. > - v4l2: fix display size for NV12 output pool. > - v4l2: handle EOS. > - v4l2: vp8 and mpeg4 decoding and encoding. > - v4l2: hevc and vp9 support. > - v4l2: generate EOF on dequeue errors. > - v4l2: h264_mp4toannexb filtering. > - v4l2: fixed make install and fate issues. > - v4l2: codecs enabled/disabled depending on pixfmt defined > - v4l2: pass timebase/framerate to the context > - v4l2: runtime decoder reconfiguration. > - v4l2: add more frame information > - v4l2: free hardware resources on last reference being released > - v4l2: encoding: disable b-frames for upstreaming (patch required) > > [1] https://lwn.net/Articles/697956/ > > Reviewed-by: Jorge Ramirez > Reviewed-by: Alexis Ballier > Tested-by: Jorge Ramirez > --- I assume this version fixes the lifetime issues I've pointed out. I also assume the data flow issues got solved. With that, it would look good to me, I guess. If nobody else has comments it could be applied. Btw. I noticed that this apparently never sets chroma_location? ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCHv12] libavcodec: v4l2: add support for v4l2 mem2mem codecs
This patchset enhances Alexis Ballier's original patch and validates it using Qualcomm's Venus hardware (driver recently landed upstream [1]). This has been tested on Qualcomm's DragonBoard 410c and 820c Configure/make scripts have been validated on Ubuntu 10.04 and 16.04. Tested decoders: - h264 - h263 - mpeg4 - vp8 - vp9 - hevc Tested encoders: - h264 - h263 - mpeg4 Tested transcoding (concurrent encoding/decoding) Some of the changes introduced: - v4l2: code cleanup and abstractions added - v4l2: follow the new encode/decode api. - v4l2: fix display size for NV12 output pool. - v4l2: handle EOS. - v4l2: vp8 and mpeg4 decoding and encoding. - v4l2: hevc and vp9 support. - v4l2: generate EOF on dequeue errors. - v4l2: h264_mp4toannexb filtering. - v4l2: fixed make install and fate issues. - v4l2: codecs enabled/disabled depending on pixfmt defined - v4l2: pass timebase/framerate to the context - v4l2: runtime decoder reconfiguration. - v4l2: add more frame information - v4l2: free hardware resources on last reference being released - v4l2: encoding: disable b-frames for upstreaming (patch required) [1] https://lwn.net/Articles/697956/ Reviewed-by: Jorge Ramirez Reviewed-by: Alexis Ballier Tested-by: Jorge Ramirez --- Changelog | 1 + configure | 28 ++ libavcodec/Makefile | 15 ++ libavcodec/allcodecs.c| 9 + libavcodec/v4l2_buffers.c | 448 libavcodec/v4l2_buffers.h | 121 + libavcodec/v4l2_context.c | 638 ++ libavcodec/v4l2_context.h | 208 +++ libavcodec/v4l2_fmt.c | 182 + libavcodec/v4l2_fmt.h | 34 +++ libavcodec/v4l2_m2m.c | 332 libavcodec/v4l2_m2m.h | 94 +++ libavcodec/v4l2_m2m_dec.c | 226 libavcodec/v4l2_m2m_enc.c | 350 + 14 files changed, 2686 insertions(+) create mode 100644 libavcodec/v4l2_buffers.c create mode 100644 libavcodec/v4l2_buffers.h create mode 100644 libavcodec/v4l2_context.c create mode 100644 libavcodec/v4l2_context.h create mode 100644 libavcodec/v4l2_fmt.c create mode 100644 libavcodec/v4l2_fmt.h create mode 100644 libavcodec/v4l2_m2m.c create mode 100644 libavcodec/v4l2_m2m.h create mode 100644 libavcodec/v4l2_m2m_dec.c create mode 100644 libavcodec/v4l2_m2m_enc.c diff --git a/Changelog b/Changelog index ca0758a..03c242b 100644 --- a/Changelog +++ b/Changelog @@ -47,6 +47,7 @@ version : - SUP/PGS subtitle muxer - convolve video filter - VP9 tile threading support +- V4L2 mem2mem HW assisted codecs version 3.3: - CrystalHD decoder moved to new decode API diff --git a/configure b/configure index 4dd11ef..74a396c 100755 --- a/configure +++ b/configure @@ -185,6 +185,7 @@ Individual component options: --enable-filter=NAME enable filter NAME --disable-filter=NAMEdisable filter NAME --disable-filtersdisable all filters + --disable-v4l2_m2m disable V4L2 mem2mem code [autodetect] External library support: @@ -1624,6 +1625,7 @@ HWACCEL_AUTODETECT_LIBRARY_LIST=" vda vdpau videotoolbox +v4l2_m2m xvmc " @@ -2751,6 +2753,7 @@ omx_rpi_select="omx" qsvdec_select="qsv" qsvenc_select="qsv" vaapi_encode_deps="vaapi" +v4l2_m2m_deps_any="linux_videodev2_h" hwupload_cuda_filter_deps="cuda" scale_npp_filter_deps="cuda libnpp" @@ -2760,6 +2763,8 @@ nvenc_deps="cuda" nvenc_deps_any="dlopen LoadLibrary" nvenc_encoder_deps="nvenc" +h263_v4l2m2m_decoder_deps="v4l2_m2m h263_v4l2_m2m" +h263_v4l2m2m_encoder_deps="v4l2_m2m h263_v4l2_m2m" h264_crystalhd_decoder_select="crystalhd h264_mp4toannexb_bsf h264_parser" h264_cuvid_decoder_deps="cuda cuvid" h264_cuvid_decoder_select="h264_mp4toannexb_bsf" @@ -2778,6 +2783,8 @@ h264_vda_decoder_deps="vda" h264_vda_decoder_select="h264_decoder" h264_vdpau_decoder_deps="vdpau" h264_vdpau_decoder_select="h264_decoder" +h264_v4l2m2m_decoder_deps="v4l2_m2m h264_v4l2_m2m" +h264_v4l2m2m_encoder_deps="v4l2_m2m h264_v4l2_m2m" hevc_cuvid_decoder_deps="cuda cuvid" hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf" hevc_mediacodec_decoder_deps="mediacodec" @@ -2789,12 +2796,15 @@ hevc_qsv_encoder_deps="libmfx" hevc_qsv_encoder_select="hevcparse qsvenc" hevc_vaapi_encoder_deps="VAEncPictureParameterBufferHEVC" hevc_vaapi_encoder_select="vaapi_encode golomb" +hevc_v4l2m2m_decoder_deps="v4l2_m2m hevc_v4l2_m2m" +hevc_v4l2m2m_encoder_deps="v4l2_m2m hevc_v4l2_m2m" mjpeg_cuvid_decoder_deps="cuda cuvid" mjpeg_vaapi_encoder_deps="VAEncPictureParameterBufferJPEG" mjpeg_vaapi_encoder_select="vaapi_encode jpegtables" mpeg1_cuvid_decoder_deps="cuda cuvid" mpeg1
