Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
On Sun, 2020-09-06 at 15:25 +0100, Mark Thompson wrote:
> On 04/09/2020 19:24, Rogozhkin, Dmitry V wrote:
> > On Thu, 2020-09-03 at 01:02 +, Rogozhkin, Dmitry V wrote:
> > > >
> > > > (If you do implement it then you can delete all of the ad-hoc
> > > > treatment in ffmpeg, like has been done for the other hardware
> > > > codecs.)
> > >
> > > I like deleting code:). Ok, this sounds good. Let's try to
> > > understand
> > > what might be missing in the current implementation since I
> > > honestly
> > > don't see any gaps - it just works.
> >
> > @Mark. We did internal review and believe that DEVICE_CTX path is
> > actually ready to be used and just needs to be activated. Can you,
> > please, let me know how you would like to proceed:
> > 1. We can either consider review and apply the fix first (this
> > patch)
> > then deal with ad_hoc in non-related patch series
> > 2. Or we can go with the bigger patch series right away and address
> > both device_ctx + ad_hoc
> >
> > 2nd variant might require longer time to verify and review which
> > would
> > hold the fix. What are your thoughts?
>
> I'm not sure what you have tested, because it definitely doesn't
> work.
I was testing transcode and indeed I overlooked that it falls into
using system memory instead of HW frames. My fault.
>
> If you return the hardware surface format from get_format() with
> METHOD_HW_DEVICE_CTX then it just ignores you and gives you software
> frames anyway, because it only supports that case with
> METHOD_HW_FRAMES_CTX.
Can you, please, guide me towards what should be implemented/fixed in
qsv path? I afraid that implementing this fully on my own is beyond of
my current knowledge of qsv path in ffmpeg. Some guidance would be
appreciated. Maybe there is implementation for some other accel type I
can refer to?
>
> For example, with below patch to test it in the hw_decode example:
>
> $ doc/examples/hw_decode qsv test.264 /dev/null
> Assertion frame->format == AV_PIX_FMT_QSV failed at
> src/doc/examples/hw_decode.c:108
> Aborted
>
> (It incorrectly returned an NV12 frame.)
>
> - Mark
>
>
> diff --git a/doc/examples/hw_decode.c b/doc/examples/hw_decode.c
> index 71be6e6709..45f401a179 100644
> --- a/doc/examples/hw_decode.c
> +++ b/doc/examples/hw_decode.c
> @@ -105,6 +105,8 @@ static int decode_write(AVCodecContext *avctx,
> AVPacket *packet)
> goto fail;
> }
>
> +av_assert0(frame->format == AV_PIX_FMT_QSV);
> +
> if (frame->format == hw_pix_fmt) {
> /* retrieve data from GPU to CPU */
> if ((ret = av_hwframe_transfer_data(sw_frame, frame,
> 0)) < 0) {
> @@ -191,6 +193,8 @@ int main(int argc, char *argv[])
> }
> video_stream = ret;
>
> +decoder = avcodec_find_decoder_by_name("h264_qsv");
> +
> for (i = 0;; i++) {
> const AVCodecHWConfig *config =
> avcodec_get_hw_config(decoder, i);
> if (!config) {
> diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c
> index fc25dc73e5..f2fac17354 100644
> --- a/libavcodec/qsvdec.c
> +++ b/libavcodec/qsvdec.c
> @@ -47,7 +47,8 @@ const AVCodecHWConfigInternal *ff_qsv_hw_configs[]
> = {
> &(const AVCodecHWConfigInternal) {
> .public = {
> .pix_fmt = AV_PIX_FMT_QSV,
> -.methods = AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX |
> +.methods = AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX |
> + AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX |
> AV_CODEC_HW_CONFIG_METHOD_AD_HOC,
> .device_type = AV_HWDEVICE_TYPE_QSV,
> },
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
On 04/09/2020 19:24, Rogozhkin, Dmitry V wrote:
On Thu, 2020-09-03 at 01:02 +, Rogozhkin, Dmitry V wrote:
(If you do implement it then you can delete all of the ad-hoc
treatment in ffmpeg, like has been done for the other hardware
codecs.)
I like deleting code:). Ok, this sounds good. Let's try to understand
what might be missing in the current implementation since I honestly
don't see any gaps - it just works.
@Mark. We did internal review and believe that DEVICE_CTX path is
actually ready to be used and just needs to be activated. Can you,
please, let me know how you would like to proceed:
1. We can either consider review and apply the fix first (this patch)
then deal with ad_hoc in non-related patch series
2. Or we can go with the bigger patch series right away and address
both device_ctx + ad_hoc
2nd variant might require longer time to verify and review which would
hold the fix. What are your thoughts?
I'm not sure what you have tested, because it definitely doesn't work.
If you return the hardware surface format from get_format() with
METHOD_HW_DEVICE_CTX then it just ignores you and gives you software frames
anyway, because it only supports that case with METHOD_HW_FRAMES_CTX.
For example, with below patch to test it in the hw_decode example:
$ doc/examples/hw_decode qsv test.264 /dev/null
Assertion frame->format == AV_PIX_FMT_QSV failed at
src/doc/examples/hw_decode.c:108
Aborted
(It incorrectly returned an NV12 frame.)
- Mark
diff --git a/doc/examples/hw_decode.c b/doc/examples/hw_decode.c
index 71be6e6709..45f401a179 100644
--- a/doc/examples/hw_decode.c
+++ b/doc/examples/hw_decode.c
@@ -105,6 +105,8 @@ static int decode_write(AVCodecContext *avctx, AVPacket
*packet)
goto fail;
}
+av_assert0(frame->format == AV_PIX_FMT_QSV);
+
if (frame->format == hw_pix_fmt) {
/* retrieve data from GPU to CPU */
if ((ret = av_hwframe_transfer_data(sw_frame, frame, 0)) < 0) {
@@ -191,6 +193,8 @@ int main(int argc, char *argv[])
}
video_stream = ret;
+decoder = avcodec_find_decoder_by_name("h264_qsv");
+
for (i = 0;; i++) {
const AVCodecHWConfig *config = avcodec_get_hw_config(decoder, i);
if (!config) {
diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c
index fc25dc73e5..f2fac17354 100644
--- a/libavcodec/qsvdec.c
+++ b/libavcodec/qsvdec.c
@@ -47,7 +47,8 @@ const AVCodecHWConfigInternal *ff_qsv_hw_configs[] = {
&(const AVCodecHWConfigInternal) {
.public = {
.pix_fmt = AV_PIX_FMT_QSV,
-.methods = AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX |
+.methods = AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX |
+ AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX |
AV_CODEC_HW_CONFIG_METHOD_AD_HOC,
.device_type = AV_HWDEVICE_TYPE_QSV,
},
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
On Thu, 2020-09-03 at 01:02 +, Rogozhkin, Dmitry V wrote: > > > > (If you do implement it then you can delete all of the ad-hoc > > treatment in ffmpeg, like has been done for the other hardware > > codecs.) > > I like deleting code:). Ok, this sounds good. Let's try to understand > what might be missing in the current implementation since I honestly > don't see any gaps - it just works. @Mark. We did internal review and believe that DEVICE_CTX path is actually ready to be used and just needs to be activated. Can you, please, let me know how you would like to proceed: 1. We can either consider review and apply the fix first (this patch) then deal with ad_hoc in non-related patch series 2. Or we can go with the bigger patch series right away and address both device_ctx + ad_hoc 2nd variant might require longer time to verify and review which would hold the fix. What are your thoughts? ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
On Wed, 2020-09-02 at 20:37 +0100, Mark Thompson wrote: > > > I still don't see the full picture. What I am looking for at the > > > first > > > place is how maintainers and architects envision hwaccel to work > > > in > > > general. Basically, there are few ways to specify the device (- > > > hwaccel, > > > -init_hw_device, -filter_hw_device), questions are: > > > 1. What's scope of applicability of each option w/ explanation > > > why > > > each > > > of the option is actually needed (as one of the examples: why - > > > filter_hw_device is needed and why -init_hw_device can't be used > > > instead? > > -init_hw_device is a global option which defines a device and adds it > to the list in ffmpeg; you can have as many of these as you like. > > When setting up a codec which might be able to use a device it tries > to match against that list, unless you have an explicit > -hwaccel_device option for that input. If you do, then it looks for > the device with that name. > > To maintain compatibility with old command-lines where > -hwaccel_device took a string interpreted by special code for each > API in ffmpeg (now removed), if it doesn't find a device with the > given name then it will try to use it to create a new one. > > (So "-hwaccel foo -hwaccel_device bar" when "bar" hasn't been > previously defined ends up with the effect of "-init_hw_device > foo=quux:bar -hwaccel foo -hwaccel_device quux".) > > This doesn't apply to the libmfx decoders, because they have their > own ad-hoc code. Getting rid of that so they work in the same way as > everything else would be a good thing. > > -filter_hw_device selects a device from the list to use while > filtering (e.g. by hwupload). > > If there is exactly one device in the list when making a filter graph > then it will use that, but if there are multiple then it has no way > to guess which one you meant so the option is needed. Hm. I am pretty sure I saw the case when for one of qsv pipelines when I was forced to add -filter_hw_device to make things workable thought there was only one device specified. That's why I was specially asking about the filter option. This was long ago however and I don't remember what exactly I ran. Besides, this might have messed up with the fact that qsv decoder don't work with -init_hw_device... I think I need to pay attention on this long past use case after we will deal with the decoder story... If I will note some strange behavior, I'll post update. > > > 2. Since there are few methods how component can get a device: > > > what's > > > the priority order? (for example, if device can be deducted from > > > incoming frames, is there a way to override it w/ some command > > > line > > > options?) > > Devices from incoming frames have to win, because they are stored on > that device - it doesn't make sense to select a different device at > that point (you would need a separate download/upload or mapping step > to do that). > > - Mark Thank you, Mark. Your description greatly helps and aligns with expectations I have. > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
On Wed, 2020-09-02 at 20:37 +0100, Mark Thompson wrote: > > In general, per my taste, I would try to use the following device > > specification working with QSV on Linux across all command lines: > > > > -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device > > qsv=hw@va > > -filter_hw_device hw > > > > My primary goal is to make it workable for all pipelines. Hence > > current > > patch. > > Implementing AV_HW_CONFIG_METHOD_HW_DEVICE_CTX would make sense to > allow the legacy decoders to use normal device selection, but your > patch seems to just set the flag indicating support without actually > implementing the feature? Because feature just worked as soon as I added flag indicating the support. For what I see in the code, implementation is already there, it just was not properly enabled. Here it is: https://github.com/FFmpeg/FFmpeg/blob/7b1ed4b53a4b32865dd9990f555929e803f64b64/libavcodec/qsvdec.c#L130. It seems you Mark was original author of this piece of code, but later it was patched by Zhong. Can you folks try to evaluate whether current implementation is enough? We've give it a try via our CI regression tests which all pass (Linux only). This gives some assurity, but still. > > (If you do implement it then you can delete all of the ad-hoc > treatment in ffmpeg, like has been done for the other hardware > codecs.) I like deleting code:). Ok, this sounds good. Let's try to understand what might be missing in the current implementation since I honestly don't see any gaps - it just works. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
On Wed, 2020-09-02 at 20:37 +0100, Mark Thompson wrote: > On 02/09/2020 15:36, Rogozhkin, Dmitry V wrote: > > On Wed, 2020-09-02 at 14:21 +, Rogozhkin, Dmitry V wrote: > > > On Wed, 2020-09-02 at 08:41 +, Soft Works wrote: > > > > ... > > > > Small suggestion: let's move discussion around -qsv_device and > > -hwaccel_device options entirely to the "ffmpeg_qsv: use > > -hwaccel_device to specify a device for VAAPI backend" thread and a > > return focus on the original patch which is not about -qsv_device, but > > about this command line: > > > > ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device > > qsv=hw@va -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y output.h264 > > I wouldn't recommend doing this; using the appropriate acceleration API is > preferred to messing with the vendor-specific full-offload decoders: > > ffmpeg -init_hw_device vaapi:/dev/dri/renderD129 -hwaccel vaapi -i input.h264 > -vf hwmap=derive_device=qsv -c:v hevc_qsv -y output.h264 > > (Or ... -init_hw_device dxva2:1 -hwaccel dxva2 ...) > > > For what I see it uses valid device specifications and still in > > original ffmpeg implementation it results in: > > - decoder is being run on /dev/dri/renderD128 > > - encoder is being run on /dev/dri/renderD129 > > The libmfx decoders need you to set -qsv_device, which is a hack option for > ffmpeg to allow selecting a device on them because they doesn't support the > normal device selection. > How about to extend -hwaccel_device to accept drm path for QSV on Linux in another thread? According to https://trac.ffmpeg.org/ticket/7649, -qsv_device was added to workaround device select issue on Linux. We may deprecate -qsv_device once we allow -hwaccel_device to accept drm path. > The libmfx encoders do support the normal setup in libavcodec, so ffmpeg > matched hevc_qsv to the libmfx device you created. > > > In general, per my taste, I would try to use the following device > > specification working with QSV on Linux across all command lines: > > > > -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device qsv=hw@va > > -filter_hw_device hw > > > > My primary goal is to make it workable for all pipelines. Hence current > > patch. > > Implementing AV_HW_CONFIG_METHOD_HW_DEVICE_CTX would make sense to allow the > legacy decoders to use normal device selection, but your patch seems to just > set the flag indicating support without actually implementing the feature? > > (If you do implement it then you can delete all of the ad-hoc treatment in > ffmpeg, like has been done for the other hardware codecs.) AVCodecContext.hw_device_ctx is set in ffmpeg_hw.c as soon as AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX is set in AVCodecHWConfig. And AVCodecContext.hw_device_ctx has been taken into account in the qsv path. Thanks Haihao > > > > > ... > > > > > > I still don't see the full picture. What I am looking for at the > > > first > > > place is how maintainers and architects envision hwaccel to work in > > > general. Basically, there are few ways to specify the device (- > > > hwaccel, > > > -init_hw_device, -filter_hw_device), questions are: > > > 1. What's scope of applicability of each option w/ explanation why > > > each > > > of the option is actually needed (as one of the examples: why - > > > filter_hw_device is needed and why -init_hw_device can't be used > > > instead? > > -init_hw_device is a global option which defines a device and adds it to the > list in ffmpeg; you can have as many of these as you like. > > When setting up a codec which might be able to use a device it tries to match > against that list, unless you have an explicit -hwaccel_device option for that > input. If you do, then it looks for the device with that name. > > To maintain compatibility with old command-lines where -hwaccel_device took a > string interpreted by special code for each API in ffmpeg (now removed), if it > doesn't find a device with the given name then it will try to use it to create > a new one. > > (So "-hwaccel foo -hwaccel_device bar" when "bar" hasn't been previously > defined ends up with the effect of "-init_hw_device foo=quux:bar -hwaccel foo > -hwaccel_device quux".) > > This doesn't apply to the libmfx decoders, because they have their own ad-hoc > code. Getting rid of that so they work in the same way as everything else > would be a good thing. > > -filter_hw_device selects a device from the list to use while filtering (e.g. > by hwupload). > > If there is exactly one device in the list when making a filter graph then it > will use that, but if there are multiple then it has no way to guess which one > you meant so the option is needed. > > > > 2. Since there are few methods how component can get a device: what's > > > the priority order? (for example, if device can be deducted from > > > incoming frames, is there a way to override it w/ some command line > > > options?) > > Devices from incoming frames have to win, because they are stored on
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
> -Original Message- > From: ffmpeg-devel On Behalf Of > Mark Thompson > Sent: Wednesday, September 2, 2020 9:38 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use > initialized device > > On 02/09/2020 15:36, Rogozhkin, Dmitry V wrote: > > On Wed, 2020-09-02 at 14:21 +, Rogozhkin, Dmitry V wrote: > >> On Wed, 2020-09-02 at 08:41 +, Soft Works wrote: > >>> ... > > > > Small suggestion: let's move discussion around -qsv_device and > > -hwaccel_device options entirely to the "ffmpeg_qsv: use > > -hwaccel_device to specify a device for VAAPI backend" thread and a > > return focus on the original patch which is not about -qsv_device, but > > about this command line: > > > > ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device > > qsv=hw@va -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y output.h264 > > I wouldn't recommend doing this; using the appropriate acceleration API is > preferred to messing with the vendor-specific full-offload decoders: > > ffmpeg -init_hw_device vaapi:/dev/dri/renderD129 -hwaccel vaapi -i > input.h264 -vf hwmap=derive_device=qsv -c:v hevc_qsv -y output.h264 > > (Or ... -init_hw_device dxva2:1 -hwaccel dxva2 ...) Hi Mark, In the latter case I have to disagree. We've gone through so many Tests and the QSV decoders have almost always resulted in better overall performance compared to their DXVA2 and D3D11VA counterparts, when using a full hw pipeline qsv_dec > qsv_vpp > qsv_enc. Also, capabilities may differ (supported color formats, resolutions, profiles..) Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
On 02/09/2020 15:36, Rogozhkin, Dmitry V wrote: On Wed, 2020-09-02 at 14:21 +, Rogozhkin, Dmitry V wrote: On Wed, 2020-09-02 at 08:41 +, Soft Works wrote: ... Small suggestion: let's move discussion around -qsv_device and -hwaccel_device options entirely to the "ffmpeg_qsv: use -hwaccel_device to specify a device for VAAPI backend" thread and a return focus on the original patch which is not about -qsv_device, but about this command line: ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device qsv=hw@va -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y output.h264 I wouldn't recommend doing this; using the appropriate acceleration API is preferred to messing with the vendor-specific full-offload decoders: ffmpeg -init_hw_device vaapi:/dev/dri/renderD129 -hwaccel vaapi -i input.h264 -vf hwmap=derive_device=qsv -c:v hevc_qsv -y output.h264 (Or ... -init_hw_device dxva2:1 -hwaccel dxva2 ...) For what I see it uses valid device specifications and still in original ffmpeg implementation it results in: - decoder is being run on /dev/dri/renderD128 - encoder is being run on /dev/dri/renderD129 The libmfx decoders need you to set -qsv_device, which is a hack option for ffmpeg to allow selecting a device on them because they doesn't support the normal device selection. The libmfx encoders do support the normal setup in libavcodec, so ffmpeg matched hevc_qsv to the libmfx device you created. In general, per my taste, I would try to use the following device specification working with QSV on Linux across all command lines: -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device qsv=hw@va -filter_hw_device hw My primary goal is to make it workable for all pipelines. Hence current patch. Implementing AV_HW_CONFIG_METHOD_HW_DEVICE_CTX would make sense to allow the legacy decoders to use normal device selection, but your patch seems to just set the flag indicating support without actually implementing the feature? (If you do implement it then you can delete all of the ad-hoc treatment in ffmpeg, like has been done for the other hardware codecs.) ... I still don't see the full picture. What I am looking for at the first place is how maintainers and architects envision hwaccel to work in general. Basically, there are few ways to specify the device (- hwaccel, -init_hw_device, -filter_hw_device), questions are: 1. What's scope of applicability of each option w/ explanation why each of the option is actually needed (as one of the examples: why - filter_hw_device is needed and why -init_hw_device can't be used instead? -init_hw_device is a global option which defines a device and adds it to the list in ffmpeg; you can have as many of these as you like. When setting up a codec which might be able to use a device it tries to match against that list, unless you have an explicit -hwaccel_device option for that input. If you do, then it looks for the device with that name. To maintain compatibility with old command-lines where -hwaccel_device took a string interpreted by special code for each API in ffmpeg (now removed), if it doesn't find a device with the given name then it will try to use it to create a new one. (So "-hwaccel foo -hwaccel_device bar" when "bar" hasn't been previously defined ends up with the effect of "-init_hw_device foo=quux:bar -hwaccel foo -hwaccel_device quux".) This doesn't apply to the libmfx decoders, because they have their own ad-hoc code. Getting rid of that so they work in the same way as everything else would be a good thing. -filter_hw_device selects a device from the list to use while filtering (e.g. by hwupload). If there is exactly one device in the list when making a filter graph then it will use that, but if there are multiple then it has no way to guess which one you meant so the option is needed. 2. Since there are few methods how component can get a device: what's the priority order? (for example, if device can be deducted from incoming frames, is there a way to override it w/ some command line options?) Devices from incoming frames have to win, because they are stored on that device - it doesn't make sense to select a different device at that point (you would need a separate download/upload or mapping step to do that). - Mark ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
On Wed, 2020-09-02 at 14:21 +, Rogozhkin, Dmitry V wrote: > On Wed, 2020-09-02 at 08:41 +, Soft Works wrote: > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Rogozhkin, Dmitry V > > > Sent: Wednesday, September 2, 2020 8:45 AM > > > To: ffmpeg-devel@ffmpeg.org > > > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv > > > decoders > > > to use > > > initialized device > > > > > > On Wed, 2020-09-02 at 04:32 +, Soft Works wrote: > > > > > -Original Message- > > > > > From: ffmpeg-devel On > > > > > Behalf > > > > > Of > > > > > Soft Works > > > > > Sent: Wednesday, September 2, 2020 6:13 AM > > > > > To: FFmpeg development discussions and patches > > > > de...@ffmpeg.org> > > > > > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv > > > > > decoders > > > > > to use initialized device > > > > > > > > > > > > > > > > > > -Original Message- > > > > > > > From: ffmpeg-devel On > > > > > > Behalf > > > > > > > Of Dmitry Rogozhkin > > > > > > > Sent: Wednesday, September 2, 2020 4:44 AM > > > > > > > To: ffmpeg-devel@ffmpeg.org > > > > > > > Cc: Dmitry Rogozhkin > > > > > > > Subject: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv > > > > > > > decoders > > > > > > > to use initialized device > > > > > > > > > > > > > > qsv decoders did not allow to use devices explicitly > > > > > > > initialized > > > > > > > on the command line and actually were using default > > > > > > > device. > > > > > > > This > > > > > > > starts to cause confusion with intel discrete GPUs since > > > > > > > in > > > > > > > this > > > > > > > case decoder might run on default integrated GPU device > > > > > > > (/dev/dri/renderD128) and encoder on the device specified > > > > > > > on the > > > > > > > command line > > > > > > > > > > > > (/dev/dri/renderD129). > > > > > > > > > > > > > > Example: > > > > > > > ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 > > > > > > > -init_hw_device qsv=hw@va \ > > > > > > > -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y > > > > > > > output.h264 > > > > > > > > > > I apologize, I picked the wrong thing. The qsv_device > > > > > parameter > > > > > is > > > > > what allows setting the device for a QSV decoder: > > > > > > > > > > fmpeg -qsv_device /dev/dri/renderD128 -c:v:0 h264_qsv > > > > > -hwaccel:v:0 > > > > > qsv -i INPUT > > > > > > > > > > Kind regards, > > > > > softworkz > > > > > > > > Here's the commit where the parameter had been added: > > > > > > > > > > https://github.com/FFmpeg/FFmpeg/commit/1a79b8f8d2b5d26c60c237d6e5 > > > 8587 > > > > 3238e46914 > > > > > > I am aware of this option. > > > > > > > -qsv_device /dev/dri/renderD129 > > > > > > By itself this don’t work. Both decoder and encoder will run on > > > /dev/dri/renderD128 instead. > > > > > > > -hwaccel qsv -qsv_device /dev/dri/renderD129 > > > > > > Adding -hwaccel helps. This works. However, to me this is non- > > > intuitive: why qsv_device should be used instead of > > > hwaccel_device > > > while > > > ffmpeg help gives a different hint: > > > -hwaccel hwaccel name use HW accelerated decoding > > > -hwaccel_device devicename select a device for HW > > > acceleration > > > From this > > > perspective Haihao’s patch which is currently on the mailing list > > > makes sense > > > to me it just simplifies things. > > > > In case of QSV, the meaning of hwaccel_device is already defined: > > > > Adding new values does not break functionality. I believe that > definition for
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
On Wed, 2020-09-02 at 08:41 +, Soft Works wrote: > > -Original Message- > > From: ffmpeg-devel On Behalf Of > > Rogozhkin, Dmitry V > > Sent: Wednesday, September 2, 2020 8:45 AM > > To: ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders > > to use > > initialized device > > > > On Wed, 2020-09-02 at 04:32 +, Soft Works wrote: > > > > -Original Message- > > > > From: ffmpeg-devel On Behalf > > > > Of > > > > Soft Works > > > > Sent: Wednesday, September 2, 2020 6:13 AM > > > > To: FFmpeg development discussions and patches > > > de...@ffmpeg.org> > > > > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv > > > > decoders > > > > to use initialized device > > > > > > > > > > > > > > > -Original Message- > > > > > > From: ffmpeg-devel On > > > > Behalf > > > > > > Of Dmitry Rogozhkin > > > > > > Sent: Wednesday, September 2, 2020 4:44 AM > > > > > > To: ffmpeg-devel@ffmpeg.org > > > > > > Cc: Dmitry Rogozhkin > > > > > > Subject: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv > > > > > > decoders > > > > > > to use initialized device > > > > > > > > > > > > qsv decoders did not allow to use devices explicitly > > > > > > initialized > > > > > > on the command line and actually were using default device. > > > > > > This > > > > > > starts to cause confusion with intel discrete GPUs since in > > > > > > this > > > > > > case decoder might run on default integrated GPU device > > > > > > (/dev/dri/renderD128) and encoder on the device specified > > > > > > on the > > > > > > command line > > > > > > > > > > (/dev/dri/renderD129). > > > > > > > > > > > > Example: > > > > > > ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 > > > > > > -init_hw_device qsv=hw@va \ > > > > > > -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y output.h264 > > > > > > > > I apologize, I picked the wrong thing. The qsv_device parameter > > > > is > > > > what allows setting the device for a QSV decoder: > > > > > > > > fmpeg -qsv_device /dev/dri/renderD128 -c:v:0 h264_qsv > > > > -hwaccel:v:0 > > > > qsv -i INPUT > > > > > > > > Kind regards, > > > > softworkz > > > > > > Here's the commit where the parameter had been added: > > > > > > > https://github.com/FFmpeg/FFmpeg/commit/1a79b8f8d2b5d26c60c237d6e5 > > 8587 > > > 3238e46914 > > > > I am aware of this option. > > > > > -qsv_device /dev/dri/renderD129 > > > > By itself this don’t work. Both decoder and encoder will run on > > /dev/dri/renderD128 instead. > > > > > -hwaccel qsv -qsv_device /dev/dri/renderD129 > > > > Adding -hwaccel helps. This works. However, to me this is non- > > intuitive: why qsv_device should be used instead of hwaccel_device > > while > > ffmpeg help gives a different hint: > > -hwaccel hwaccel name use HW accelerated decoding > > -hwaccel_device devicename select a device for HW acceleration > > From this > > perspective Haihao’s patch which is currently on the mailing list > > makes sense > > to me it just simplifies things. > > In case of QSV, the meaning of hwaccel_device is already defined: > Adding new values does not break functionality. I believe that definition for hwaccel_device on Linux for qsv can be extended w/ drm device specifiers for the convenience of overyone. As I noted in other thread, msdk on Linux does not distinguish hw,hw2,hw3 - these are all treated the same and library requires external device setting via SetHandle. > device selects a value in ‘MFX_IMPL_*’. Allowed values are: > auto sw hw auto_any hw_any hw2 hw3 hw4 > > From my understanding, that's the reason why the qsv_device parameter > was introduced. > > > > > Unfortunately ffmpeg device selection options are quite confusing. > > That's true without doubt. Sadly, there's not much consistency among > all hw accelerations in general. > > > This will work. But returning back to transcoding cas
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
> -Original Message- > From: ffmpeg-devel On Behalf Of > Rogozhkin, Dmitry V > Sent: Wednesday, September 2, 2020 8:45 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use > initialized device > > On Wed, 2020-09-02 at 04:32 +, Soft Works wrote: > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Soft Works > > > Sent: Wednesday, September 2, 2020 6:13 AM > > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org> > > > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders > > > to use initialized device > > > > > > > > > > > > -Original Message- > > > > > From: ffmpeg-devel On > Behalf > > > > > Of Dmitry Rogozhkin > > > > > Sent: Wednesday, September 2, 2020 4:44 AM > > > > > To: ffmpeg-devel@ffmpeg.org > > > > > Cc: Dmitry Rogozhkin > > > > > Subject: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders > > > > > to use initialized device > > > > > > > > > > qsv decoders did not allow to use devices explicitly initialized > > > > > on the command line and actually were using default device. This > > > > > starts to cause confusion with intel discrete GPUs since in this > > > > > case decoder might run on default integrated GPU device > > > > > (/dev/dri/renderD128) and encoder on the device specified on the > > > > > command line > > > > > > > > (/dev/dri/renderD129). > > > > > > > > > > Example: > > > > > ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 > > > > > -init_hw_device qsv=hw@va \ > > > > > -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y output.h264 > > > I apologize, I picked the wrong thing. The qsv_device parameter is > > > what allows setting the device for a QSV decoder: > > > > > > fmpeg -qsv_device /dev/dri/renderD128 -c:v:0 h264_qsv -hwaccel:v:0 > > > qsv -i INPUT > > > > > > Kind regards, > > > softworkz > > > > Here's the commit where the parameter had been added: > > > https://github.com/FFmpeg/FFmpeg/commit/1a79b8f8d2b5d26c60c237d6e5 > 8587 > > 3238e46914 > > I am aware of this option. > > > -qsv_device /dev/dri/renderD129 > By itself this don’t work. Both decoder and encoder will run on > /dev/dri/renderD128 instead. > > > -hwaccel qsv -qsv_device /dev/dri/renderD129 > Adding -hwaccel helps. This works. However, to me this is non- > intuitive: why qsv_device should be used instead of hwaccel_device while > ffmpeg help gives a different hint: > -hwaccel hwaccel name use HW accelerated decoding > -hwaccel_device devicename select a device for HW acceleration From this > perspective Haihao’s patch which is currently on the mailing list makes sense > to me it just simplifies things. In case of QSV, the meaning of hwaccel_device is already defined: device selects a value in ‘MFX_IMPL_*’. Allowed values are: auto sw hw auto_any hw_any hw2 hw3 hw4 From my understanding, that's the reason why the qsv_device parameter was introduced. > > Unfortunately ffmpeg device selection options are quite confusing. That's true without doubt. Sadly, there's not much consistency among all hw accelerations in general. > This will work. But returning back to transcoding case: there are 2 > significantly > different command lines which should be used for transcoding and encoding > to make things run on /dev/dri/renderD129. > This is inconvenient to handle… And additionally there is also - > filter_hw_device which also contributes to the complication. Also there are > special marks in documentation for the qsv “Unlike most other values, this > option does not enable accelerated decoding (that is used automatically > whenever a qsv decoder is selected), but accelerated transcoding, without > copying the frames into the system memory. For it to work, both the > decoder and the encoder must support QSV acceleration and no filters must > be used.” > One missing thing seems to be documentation on the scope of - > init_hw_device option applicability. This seems to be a global option, but in > the example from the commit message encoder actually takes device from it, > but decoder just ignores it and goes with default device. Why? This does not > seem to be right. > > Can someone, please, shine the light on how all these device selection > options were supposed to work? From my und
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
On Wed, 2020-09-02 at 04:32 +, Soft Works wrote: > > -Original Message- > > From: ffmpeg-devel On Behalf Of > > Soft Works > > Sent: Wednesday, September 2, 2020 6:13 AM > > To: FFmpeg development discussions and patches > de...@ffmpeg.org> > > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders > > to use > > initialized device > > > > > > > > > -Original Message- > > > > From: ffmpeg-devel On Behalf > > > > Of > > > > Dmitry Rogozhkin > > > > Sent: Wednesday, September 2, 2020 4:44 AM > > > > To: ffmpeg-devel@ffmpeg.org > > > > Cc: Dmitry Rogozhkin > > > > Subject: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders > > > > to > > > > use initialized device > > > > > > > > qsv decoders did not allow to use devices explicitly > > > > initialized on > > > > the command line and actually were using default device. This > > > > starts > > > > to cause confusion with intel discrete GPUs since in this case > > > > decoder might run on default integrated GPU device > > > > (/dev/dri/renderD128) and encoder on the device specified on > > > > the > > > > command line > > > > > > (/dev/dri/renderD129). > > > > > > > > Example: > > > > ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 > > > > -init_hw_device > > > > qsv=hw@va \ > > > > -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y output.h264 > > I apologize, I picked the wrong thing. The qsv_device parameter is > > what > > allows setting the device for a QSV decoder: > > > > fmpeg -qsv_device /dev/dri/renderD128 -c:v:0 h264_qsv -hwaccel:v:0 > > qsv -i > > INPUT > > > > Kind regards, > > softworkz > > Here's the commit where the parameter had been added: > https://github.com/FFmpeg/FFmpeg/commit/1a79b8f8d2b5d26c60c237d6e585873238e46914 I am aware of this option. > -qsv_device /dev/dri/renderD129 By itself this don’t work. Both decoder and encoder will run on /dev/dri/renderD128 instead. > -hwaccel qsv -qsv_device /dev/dri/renderD129 Adding -hwaccel helps. This works. However, to me this is non- intuitive: why qsv_device should be used instead of hwaccel_device while ffmpeg help gives a different hint: -hwaccel hwaccel name use HW accelerated decoding -hwaccel_device devicename select a device for HW acceleration From this perspective Haihao’s patch which is currently on the mailing list makes sense to me it just simplifies things. Unfortunately ffmpeg device selection options are quite confusing. One of the problems with -qsv_device is that the following pure encoding won’t work: ffmpeg -hwaccel qsv -qsv_device /dev/dri/renderD129 -f rawvideo -pix_fmt yuv420p -s:v 1920x1080 -r 30 -i a.yuv -c:v h264_qsv out.h264 Execution will happen on /dev/dri/renderD128 instead. To make it work on 129 you will need to run something like the following: ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device qsv=hw@va -f rawvideo -pix_fmt yuv420p -s:v 1920x1080 -r 30 -i a.yuv -c:v h264_qsv out.h264 This will work. But returning back to transcoding case: there are 2 significantly different command lines which should be used for transcoding and encoding to make things run on /dev/dri/renderD129. This is inconvenient to handle… And additionally there is also -filter_hw_device which also contributes to the complication. Also there are special marks in documentation for the qsv “Unlike most other values, this option does not enable accelerated decoding (that is used automatically whenever a qsv decoder is selected), but accelerated transcoding, without copying the frames into the system memory. For it to work, both the decoder and the encoder must support QSV acceleration and no filters must be used.” One missing thing seems to be documentation on the scope of -init_hw_device option applicability. This seems to be a global option, but in the example from the commit message encoder actually takes device from it, but decoder just ignores it and goes with default device. Why? This does not seem to be right. Can someone, please, shine the light on how all these device selection options were supposed to work? /Eventually my patch tries to fix behavior in accordance with my understanding of the supposed behavior./ > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Wednesday, September 2, 2020 6:13 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use > initialized device > > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Dmitry Rogozhkin > > > Sent: Wednesday, September 2, 2020 4:44 AM > > > To: ffmpeg-devel@ffmpeg.org > > > Cc: Dmitry Rogozhkin > > > Subject: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to > > > use initialized device > > > > > > qsv decoders did not allow to use devices explicitly initialized on > > > the command line and actually were using default device. This starts > > > to cause confusion with intel discrete GPUs since in this case > > > decoder might run on default integrated GPU device > > > (/dev/dri/renderD128) and encoder on the device specified on the > > > command line > > (/dev/dri/renderD129). > > > > > > Example: > > > ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device > > > qsv=hw@va \ > > > -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y output.h264 > I apologize, I picked the wrong thing. The qsv_device parameter is what > allows setting the device for a QSV decoder: > > fmpeg -qsv_device /dev/dri/renderD128 -c:v:0 h264_qsv -hwaccel:v:0 qsv -i > INPUT > > Kind regards, > softworkz Here's the commit where the parameter had been added: https://github.com/FFmpeg/FFmpeg/commit/1a79b8f8d2b5d26c60c237d6e585873238e46914 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Wednesday, September 2, 2020 5:33 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use > initialized device > > > > > -Original Message- > > From: ffmpeg-devel On Behalf Of > > Dmitry Rogozhkin > > Sent: Wednesday, September 2, 2020 4:44 AM > > To: ffmpeg-devel@ffmpeg.org > > Cc: Dmitry Rogozhkin > > Subject: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use > > initialized device > > > > qsv decoders did not allow to use devices explicitly initialized on > > the command line and actually were using default device. This starts > > to cause confusion with intel discrete GPUs since in this case decoder > > might run on default integrated GPU device (/dev/dri/renderD128) and > > encoder on the device specified on the command line > (/dev/dri/renderD129). > > > > Example: > > ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device > > qsv=hw@va \ > > -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y output.h264 > > > > Hi Dmitry, > > How about this: > > -init_hw_device qsv=vad:hw_any,child_device=/dev/dri/renderD128 > I apologize, I picked the wrong thing. The qsv_device parameter is what allows setting the device for a QSV decoder: fmpeg -qsv_device /dev/dri/renderD128 -c:v:0 h264_qsv -hwaccel:v:0 qsv -i INPUT Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device
> -Original Message- > From: ffmpeg-devel On Behalf Of > Dmitry Rogozhkin > Sent: Wednesday, September 2, 2020 4:44 AM > To: ffmpeg-devel@ffmpeg.org > Cc: Dmitry Rogozhkin > Subject: [FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use > initialized device > > qsv decoders did not allow to use devices explicitly initialized on the > command line and actually were using default device. This starts to cause > confusion with intel discrete GPUs since in this case decoder might run on > default integrated GPU device (/dev/dri/renderD128) and encoder on the > device specified on the command line (/dev/dri/renderD129). > > Example: > ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device > qsv=hw@va \ > -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y output.h264 > Hi Dmitry, How about this: -init_hw_device qsv=vad:hw_any,child_device=/dev/dri/renderD128 Unfortunately I do not have one of the new Intel discrete GPUs so I wasn't able to test, but the above parameter is working at least in a way that it fails when the specified child_device is wrong (e.g. Nvidia GPU)). Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
